2024-04-24T22:54:25.645 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:54:25.646 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176 --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/orig.config.yaml 2024-04-24T22:54:25.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:54:25.787 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672176' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNtCvtSRI2vvieS7a0f2wLuYoSUaOdUhJSds0f7VI6tcSVlFIUV/sCuusRdLyBvTp3nmEODXqwhch75mvpkSYKU= smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVH1MecDR1pQVW0zS6YEJCXUZACVbMMrx5ISS3t5qYEgVeWxyyq3P41cqIOULW08LSI271UgkUD2mUCLv2qfNU= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN3kCpOqLD6WJWTL1M35+Fg2BnfMV7wriz8r1+bABwMWq+/ba5srLDpwEBGzBjzlYmnDvX0MTAta7sGy2wZHpSQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T22:54:25.787 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:54:25.788 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:54:25.788 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:54:25.790 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:54:25.813 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:54:25.813 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:54:25.814 INFO:teuthology.packaging:ref: None 2024-04-24T22:54:25.814 INFO:teuthology.packaging:tag: None 2024-04-24T22:54:25.814 INFO:teuthology.packaging:branch: squid 2024-04-24T22:54:25.814 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:54:25.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:54:26.009 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:54:26.010 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:54:26.018 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:54:26.018 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:54:26.050 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:54:26.069 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:54:26.088 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:54:26.114 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:49:32.284470', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNtCvtSRI2vvieS7a0f2wLuYoSUaOdUhJSds0f7VI6tcSVlFIUV/sCuusRdLyBvTp3nmEODXqwhch75mvpkSYKU='} 2024-04-24T22:54:26.143 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:49:32.281743', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVH1MecDR1pQVW0zS6YEJCXUZACVbMMrx5ISS3t5qYEgVeWxyyq3P41cqIOULW08LSI271UgkUD2mUCLv2qfNU='} 2024-04-24T22:54:26.170 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:49:32.283219', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN3kCpOqLD6WJWTL1M35+Fg2BnfMV7wriz8r1+bABwMWq+/ba5srLDpwEBGzBjzlYmnDvX0MTAta7sGy2wZHpSQ='} 2024-04-24T22:54:26.170 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:54:26.189 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T22:54:26.189 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T22:54:26.189 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T22:54:26.189 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:54:26.321 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f478e36c160>, signals=[15]) 2024-04-24T22:54:26.322 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:54:26.350 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:54:26.351 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-04-24T22:54:26.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:54:26.444 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-04-24T22:54:26.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:54:26.532 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-04-24T22:54:26.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:54:26.648 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:54:26.668 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-24T22:54:26.688 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-24T22:54:26.688 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-24T22:54:26.752 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:54:26.753 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:54:26.754 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:54:26.754 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-24T22:54:26.782 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-04-24T22:54:26.801 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-04-24T22:54:26.802 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-04-24T22:54:26.863 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-04-24T22:54:26.863 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-04-24T22:54:26.863 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-04-24T22:54:26.863 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:54:26.863 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-04-24T22:54:26.863 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:54:26.864 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:54:26.864 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-04-24T22:54:26.882 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-04-24T22:54:26.901 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-04-24T22:54:26.901 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-04-24T22:54:26.959 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-04-24T22:54:26.960 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:54:26.961 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:54:26.961 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:54:26.961 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:54:26.961 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:54:26.961 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:54:26.961 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-04-24T22:54:26.991 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:54:27.030 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:54:27.041 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:54:27.041 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:54:27.046 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:54:27.052 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:54:27.069 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:54:27.088 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:54:27.089 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:54:27.108 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:54:27.114 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:54:27.144 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:54:27.247 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:54:27.268 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:54:27.269 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-24T22:54:27.269 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-24T22:54:27.269 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:54:27.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:27.287 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-24T22:54:27.346 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:54:27.346 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-437.el9.x86_64 2024-04-24T22:54:27.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-04-24T22:54:29.858 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.0 MB 00:00 2024-04-24T22:54:33.784 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-04-24T22:54:38.800 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-24T22:54:40.334 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-24T22:54:43.794 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-04-24T22:54:49.111 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T22:54:51.017 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:54:51.017 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:54:51.114 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-24T22:54:51.115 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-24T22:54:51.115 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-24T22:54:51.209 DEBUG:teuthology.orchestra.run.smithi033:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:54:51.729 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 10:54:49 PM UTC. 2024-04-24T22:54:51.854 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalling: 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:54:51.871 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 255 k 2024-04-24T22:54:51.872 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2024-04-24T22:54:51.872 INFO:teuthology.orchestra.run.smithi033.stderr:Operation aborted. 2024-04-24T22:54:51.925 INFO:teuthology.orchestra.run.smithi033.stdout:Is this ok [y/N]: 2024-04-24T22:54:51.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum reinstall -y kernel || true 2024-04-24T22:54:52.443 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:54:49 PM UTC. 2024-04-24T22:54:52.564 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalling: 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-24T22:54:52.565 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:54:52.566 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 255 k 2024-04-24T22:54:52.566 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2024-04-24T22:54:52.566 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-24T22:54:52.770 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-24T22:54:52.771 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:54:52.771 INFO:teuthology.orchestra.run.smithi033.stdout:Total 1.2 MB/s | 255 kB 00:00 2024-04-24T22:54:52.792 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-24T22:54:52.983 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-24T22:54:52.983 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-24T22:54:52.988 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-24T22:54:52.989 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-24T22:54:53.126 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-24T22:54:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:54:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:54:53.857 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:54:53.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:54:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:54:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:54:54.551 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalled: 2024-04-24T22:54:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:54:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:54:54.551 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-24T22:54:54.720 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:54:54.769 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:54:54.769 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:54:54.769 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:54:54.770 DEBUG:teuthology.parallel:result is None 2024-04-24T22:54:54.770 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:54:54.777 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:54:54.777 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:54:54.780 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:54:54.783 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:54:54.800 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:54:54.807 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:54:54.817 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:54:54.817 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:54:54.835 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:54:54.841 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:54:54.954 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:54:54.961 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:54:54.961 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:54:54.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:54.979 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:54:54.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:54.996 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:54:54.999 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:54:55.022 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:54:55.049 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:54:55.057 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:54:55.072 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:54:55.077 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:54:55.083 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:54:55.109 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:54:55.111 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:54:55.118 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:54:55.118 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:54:55.120 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:54:55.128 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:54:55.179 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:54:55.188 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:54:55.189 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:54:55.191 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:54:55.196 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:54:55.235 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:54:55.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:54:55.364 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:54:55.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:54:55.445 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T22:54:55.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:54:55.509 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:54:55.554 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:54:55.607 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:54:55.690 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:54:55.736 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T22:54:55.736 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:54:55.804 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:54:55.848 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:54:55.893 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:54:55.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:54:56.018 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T22:54:56.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:54:56.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-04-24T22:54:56.086 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-04-24T22:54:56.089 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-04-24T22:54:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:54:56.125 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:54:56.166 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:54:56.494 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:54:56.502 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:54:56.502 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:54:56.514 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:54:56.525 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:54:56.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-04-24T22:54:56.575 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-04-24T22:54:56.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-04-24T22:54:56.623 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-04-24T22:54:56.625 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-04-24T22:54:56.672 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-04-24T22:54:56.673 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:54:56.674 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-04-24T22:54:56.697 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-04-24T22:54:56.697 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-04-24T22:54:56.723 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-04-24T22:54:56.723 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-04-24T22:54:56.744 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-04-24T22:54:56.744 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:54:56.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:54:56.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:56.773 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:54:56.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:56.818 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:54:56.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:56.847 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:54:56.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:54:56.882 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:54:56.921 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:54:56.956 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:54:56.970 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T22:54:56.976 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T22:54:57.122 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:54:57.140 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:54:57.141 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi038.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T22:59:19.152 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-04-24T22:59:19.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-24T22:59:19.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:59:19.235 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-24T22:59:19.315 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-24T22:59:19.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-04-24T22:59:19.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:59:19.410 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-04-24T22:59:19.495 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-04-24T22:59:19.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-24T22:59:19.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:59:19.571 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-04-24T22:59:19.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-24T22:59:19.654 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T22:59:19.665 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T22:59:19.665 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:59:19.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:59:19.668 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:59:19.668 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:59:19.670 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:59:19.670 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:59:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:59:19.702 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:59:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:59:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:59:19.723 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-04-24T22:59:19.743 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-04-24T22:59:19.753 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-04-24T22:59:19.754 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-04-24T22:59:19.759 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-04-24T22:59:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:59:19.769 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-04-24T22:59:19.770 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:59:19.781 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:59:19.782 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:59:19.783 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:59:19.795 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:59:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:59:19.845 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:59:19.849 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:59:19.849 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-24T22:59:19.849 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.849 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.849 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.850 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.851 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:59:19.852 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-04-24T22:59:19.852 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.852 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.852 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.852 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.856 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:59:19.859 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:59:19.859 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-04-24T22:59:19.860 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.860 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.860 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.860 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:19.861 INFO:teuthology.run_tasks:Running task install... 2024-04-24T22:59:19.871 DEBUG:teuthology.task.install:project ceph 2024-04-24T22:59:19.871 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T22:59:19.871 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:59:19.871 INFO:teuthology.task.install:Using flavor: default 2024-04-24T22:59:19.880 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T22:59:19.880 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T22:59:19.881 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:59:19.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:59:19.883 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:59:19.883 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:59:19.884 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:59:19.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:59:20.038 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:59:20.038 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:59:20.044 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:59:20.044 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:59:20.106 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:59:20.106 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:59:20.147 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:59:20.147 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T22:59:20.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:59:20.156 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:59:20.156 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T22:59:20.156 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:59:20.175 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:59:20.176 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:59:20.181 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:59:20.182 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:59:20.214 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:59:20.214 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T22:59:20.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:59:20.242 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:59:20.242 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:59:20.243 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:59:20.245 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:59:20.310 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:59:20.318 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:59:20.321 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:59:20.381 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-04-24T22:59:20.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-04-24T22:59:20.386 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:59:20.386 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-04-24T22:59:20.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-04-24T22:59:20.453 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-04-24T22:59:20.454 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-04-24T22:59:20.608 INFO:teuthology.orchestra.run.smithi117.stdout:48 files removed 2024-04-24T22:59:20.624 INFO:teuthology.orchestra.run.smithi033.stdout:48 files removed 2024-04-24T22:59:20.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:59:20.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:59:20.691 INFO:teuthology.orchestra.run.smithi038.stdout:48 files removed 2024-04-24T22:59:20.718 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:59:21.311 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-24T22:59:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-04-24T22:59:21.404 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-04-24T22:59:21.670 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-24T22:59:21.770 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-24T22:59:21.788 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-04-24T22:59:22.104 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-24T22:59:22.130 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-24T22:59:22.205 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-24T22:59:25.214 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.0 MB 00:02 2024-04-24T22:59:25.249 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-24T22:59:27.114 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T22:59:27.278 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:05 2024-04-24T22:59:27.907 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-24T22:59:28.996 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-24T22:59:31.922 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-24T22:59:33.679 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-04-24T22:59:34.448 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-24T22:59:34.533 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-24T22:59:35.230 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-24T22:59:35.658 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-24T22:59:35.950 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-24T22:59:36.915 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T22:59:37.268 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T22:59:40.841 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-24T22:59:42.006 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-24T22:59:42.485 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-04-24T22:59:43.043 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:43.044 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:59:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:59:43.056 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:59:43.057 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:59:43.059 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:59:43.061 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:59:43.062 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:59:43.127 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-24T22:59:43.137 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-24T22:59:43.137 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-24T22:59:43.137 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-24T22:59:43.137 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-24T22:59:43.137 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:59:43.137 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:59:43.139 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:59:43.140 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:59:43.141 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:59:43.142 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:59:43.144 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:59:43.145 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:59:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:59:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:59:43.148 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 4 Packages 2024-04-24T22:59:43.149 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T22:59:43.150 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 220 M 2024-04-24T22:59:43.150 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-24T22:59:44.107 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:44.108 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:59:44.119 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:59:44.120 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:59:44.125 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:59:44.126 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:59:44.185 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:59:44.195 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:59:44.196 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:59:44.197 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:59:44.198 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:59:44.199 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:59:44.200 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:59:44.200 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:59:44.201 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:59:44.201 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:59:44.201 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:59:44.201 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:59:44.201 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:59:44.202 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:59:44.203 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:59:44.204 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:59:44.205 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:59:44.206 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:59:44.207 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-24T22:59:44.208 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-04-24T22:59:44.209 INFO:teuthology.orchestra.run.smithi117.stdout:Install 148 Packages 2024-04-24T22:59:44.209 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 4 Packages 2024-04-24T22:59:44.209 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T22:59:44.210 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 220 M 2024-04-24T22:59:44.210 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-24T22:59:44.434 INFO:teuthology.orchestra.run.smithi033.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-24T22:59:44.688 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:44.688 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:44.701 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:59:44.701 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:59:44.702 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:59:44.703 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:59:44.705 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:59:44.707 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:59:44.708 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:59:44.773 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-24T22:59:44.786 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:59:44.787 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:59:44.788 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:59:44.789 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:59:44.790 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:59:44.791 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:59:44.792 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:59:44.793 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:59:44.794 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:59:44.795 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:59:44.796 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:59:44.797 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-04-24T22:59:44.798 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 4 Packages 2024-04-24T22:59:44.799 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T22:59:44.799 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 220 M 2024-04-24T22:59:44.799 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-04-24T22:59:45.251 INFO:teuthology.orchestra.run.smithi033.stdout:(2/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.0 MB/s | 5.4 MB 00:01 2024-04-24T22:59:45.301 INFO:teuthology.orchestra.run.smithi033.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.8 MB/s | 145 kB 00:00 2024-04-24T22:59:45.443 INFO:teuthology.orchestra.run.smithi033.stdout:(4/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 882 kB/s | 890 kB 00:01 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi033.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 2.4 MB 00:00 2024-04-24T22:59:45.582 INFO:teuthology.orchestra.run.smithi117.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-24T22:59:45.852 INFO:teuthology.orchestra.run.smithi033.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-24T22:59:45.878 INFO:teuthology.orchestra.run.smithi038.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-24T22:59:46.474 INFO:teuthology.orchestra.run.smithi117.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 998 kB/s | 890 kB 00:00 2024-04-24T22:59:46.477 INFO:teuthology.orchestra.run.smithi033.stdout:(7/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 1.4 MB/s | 1.5 MB 00:01 2024-04-24T22:59:46.657 INFO:teuthology.orchestra.run.smithi117.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 790 kB/s | 145 kB 00:00 2024-04-24T22:59:46.820 INFO:teuthology.orchestra.run.smithi038.stdout:(2/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-24T22:59:46.912 INFO:teuthology.orchestra.run.smithi038.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.5 MB/s | 145 kB 00:00 2024-04-24T22:59:46.946 INFO:teuthology.orchestra.run.smithi038.stdout:(4/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 834 kB/s | 890 kB 00:01 2024-04-24T22:59:47.379 INFO:teuthology.orchestra.run.smithi038.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 5.1 MB/s | 2.4 MB 00:00 2024-04-24T22:59:47.802 INFO:teuthology.orchestra.run.smithi033.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 8.6 MB/s | 17 MB 00:01 2024-04-24T22:59:47.877 INFO:teuthology.orchestra.run.smithi033.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 337 kB/s | 25 kB 00:00 2024-04-24T22:59:47.879 INFO:teuthology.orchestra.run.smithi038.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-04-24T22:59:48.172 INFO:teuthology.orchestra.run.smithi038.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.9 MB/s | 4.7 MB 00:00 2024-04-24T22:59:48.749 INFO:teuthology.orchestra.run.smithi117.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-24T22:59:48.790 INFO:teuthology.orchestra.run.smithi117.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-24T22:59:49.566 INFO:teuthology.orchestra.run.smithi117.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-24T22:59:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 7.1 MB/s | 10 MB 00:01 2024-04-24T22:59:49.697 INFO:teuthology.orchestra.run.smithi038.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 302 kB/s | 25 kB 00:00 2024-04-24T22:59:51.232 INFO:teuthology.orchestra.run.smithi117.stdout:(7/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 3.4 MB/s | 21 MB 00:06 2024-04-24T22:59:51.516 INFO:teuthology.orchestra.run.smithi038.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.6 MB/s | 17 MB 00:03 2024-04-24T22:59:51.625 INFO:teuthology.orchestra.run.smithi038.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.0 MB/s | 222 kB 00:00 2024-04-24T22:59:51.725 INFO:teuthology.orchestra.run.smithi038.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 271 kB/s | 27 kB 00:00 2024-04-24T22:59:51.757 INFO:teuthology.orchestra.run.smithi117.stdout:(8/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 1.6 MB/s | 4.7 MB 00:02 2024-04-24T22:59:51.817 INFO:teuthology.orchestra.run.smithi038.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 345 kB/s | 32 kB 00:00 2024-04-24T22:59:51.849 INFO:teuthology.orchestra.run.smithi117.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 275 kB/s | 25 kB 00:00 2024-04-24T22:59:52.009 INFO:teuthology.orchestra.run.smithi038.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.8 MB/s | 745 kB 00:00 2024-04-24T22:59:52.109 INFO:teuthology.orchestra.run.smithi038.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi038.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.1 MB/s | 127 kB 00:00 2024-04-24T22:59:52.394 INFO:teuthology.orchestra.run.smithi038.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.8 MB/s | 501 kB 00:00 2024-04-24T22:59:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 1.6 MB/s | 10 MB 00:06 2024-04-24T22:59:53.019 INFO:teuthology.orchestra.run.smithi033.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.1 MB/s | 222 kB 00:00 2024-04-24T22:59:53.127 INFO:teuthology.orchestra.run.smithi033.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 250 kB/s | 27 kB 00:00 2024-04-24T22:59:53.236 INFO:teuthology.orchestra.run.smithi033.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 293 kB/s | 32 kB 00:00 2024-04-24T22:59:53.384 INFO:teuthology.orchestra.run.smithi038.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.4 MB/s | 5.3 MB 00:00 2024-04-24T22:59:53.485 INFO:teuthology.orchestra.run.smithi038.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 448 kB/s | 45 kB 00:00 2024-04-24T22:59:53.586 INFO:teuthology.orchestra.run.smithi038.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-24T22:59:53.686 INFO:teuthology.orchestra.run.smithi033.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 1.6 MB/s | 745 kB 00:00 2024-04-24T22:59:53.687 INFO:teuthology.orchestra.run.smithi038.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.6 MB/s | 162 kB 00:00 2024-04-24T22:59:53.803 INFO:teuthology.orchestra.run.smithi038.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 321 kB 00:00 2024-04-24T22:59:53.862 INFO:teuthology.orchestra.run.smithi033.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 932 kB/s | 163 kB 00:00 2024-04-24T22:59:53.929 INFO:teuthology.orchestra.run.smithi038.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.3 MB/s | 299 kB 00:00 2024-04-24T22:59:53.987 INFO:teuthology.orchestra.run.smithi033.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.0 MB/s | 127 kB 00:00 2024-04-24T22:59:54.037 INFO:teuthology.orchestra.run.smithi038.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 920 kB/s | 99 kB 00:00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 787 kB/s | 85 kB 00:00 2024-04-24T22:59:54.437 INFO:teuthology.orchestra.run.smithi033.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 1.1 MB/s | 501 kB 00:00 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi117.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 3.1 MB/s | 10 MB 00:03 2024-04-24T22:59:54.529 INFO:teuthology.orchestra.run.smithi038.stdout:(26/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.3 MB/s | 21 MB 00:08 2024-04-24T22:59:54.657 INFO:teuthology.orchestra.run.smithi117.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 222 kB 00:00 2024-04-24T22:59:54.688 INFO:teuthology.orchestra.run.smithi038.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.1 MB/s | 171 kB 00:00 2024-04-24T22:59:54.749 INFO:teuthology.orchestra.run.smithi117.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 295 kB/s | 27 kB 00:00 2024-04-24T22:59:54.763 INFO:teuthology.orchestra.run.smithi038.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 424 kB/s | 32 kB 00:00 2024-04-24T22:59:54.825 INFO:teuthology.orchestra.run.smithi117.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 423 kB/s | 32 kB 00:00 2024-04-24T22:59:54.888 INFO:teuthology.orchestra.run.smithi038.stdout:(29/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.2 MB/s | 3.1 MB 00:00 2024-04-24T22:59:54.914 INFO:teuthology.orchestra.run.smithi038.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 941 kB/s | 141 kB 00:00 2024-04-24T22:59:55.175 INFO:teuthology.orchestra.run.smithi117.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.1 MB/s | 745 kB 00:00 2024-04-24T22:59:55.277 INFO:teuthology.orchestra.run.smithi117.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-24T22:59:55.375 INFO:teuthology.orchestra.run.smithi117.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.2 MB/s | 127 kB 00:00 2024-04-24T22:59:55.636 INFO:teuthology.orchestra.run.smithi117.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 1.9 MB/s | 501 kB 00:00 2024-04-24T22:59:55.906 INFO:teuthology.orchestra.run.smithi038.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.5 MB/s | 3.5 MB 00:01 2024-04-24T22:59:56.023 INFO:teuthology.orchestra.run.smithi038.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.1 MB/s | 249 kB 00:00 2024-04-24T22:59:56.029 INFO:teuthology.orchestra.run.smithi033.stdout:(18/152): ceph-common-19.0.0-2445.gd2c45d97.el9 1.8 MB/s | 21 MB 00:11 2024-04-24T22:59:56.121 INFO:teuthology.orchestra.run.smithi033.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 493 kB/s | 45 kB 00:00 2024-04-24T22:59:56.139 INFO:teuthology.orchestra.run.smithi038.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 424 kB/s | 49 kB 00:00 2024-04-24T22:59:56.248 INFO:teuthology.orchestra.run.smithi038.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 141 kB/s | 15 kB 00:00 2024-04-24T22:59:56.288 INFO:teuthology.orchestra.run.smithi033.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 799 kB/s | 133 kB 00:00 2024-04-24T22:59:56.374 INFO:teuthology.orchestra.run.smithi038.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.1 MB/s | 264 kB 00:00 2024-04-24T22:59:56.455 INFO:teuthology.orchestra.run.smithi033.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 972 kB/s | 162 kB 00:00 2024-04-24T22:59:56.691 INFO:teuthology.orchestra.run.smithi038.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 2.4 MB/s | 762 kB 00:00 2024-04-24T22:59:56.763 INFO:teuthology.orchestra.run.smithi033.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.0 MB/s | 321 kB 00:00 2024-04-24T22:59:56.791 INFO:teuthology.orchestra.run.smithi038.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 182 kB/s | 18 kB 00:00 2024-04-24T22:59:56.942 INFO:teuthology.orchestra.run.smithi038.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-04-24T22:59:56.955 INFO:teuthology.orchestra.run.smithi033.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 299 kB 00:00 2024-04-24T22:59:57.039 INFO:teuthology.orchestra.run.smithi033.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T22:59:57.122 INFO:teuthology.orchestra.run.smithi033.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-24T22:59:57.275 INFO:teuthology.orchestra.run.smithi038.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-24T22:59:57.645 INFO:teuthology.orchestra.run.smithi117.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 2.6 MB/s | 5.3 MB 00:02 2024-04-24T22:59:57.720 INFO:teuthology.orchestra.run.smithi117.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 602 kB/s | 45 kB 00:00 2024-04-24T22:59:57.784 INFO:teuthology.orchestra.run.smithi038.stdout:(40/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 6.0 MB/s | 48 MB 00:08 2024-04-24T22:59:57.795 INFO:teuthology.orchestra.run.smithi117.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-24T22:59:57.809 INFO:teuthology.orchestra.run.smithi038.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 366 kB/s | 195 kB 00:00 2024-04-24T22:59:57.943 INFO:teuthology.orchestra.run.smithi038.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-24T22:59:57.945 INFO:teuthology.orchestra.run.smithi117.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.1 MB/s | 162 kB 00:00 2024-04-24T22:59:57.993 INFO:teuthology.orchestra.run.smithi038.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T22:59:58.043 INFO:teuthology.orchestra.run.smithi038.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T22:59:58.093 INFO:teuthology.orchestra.run.smithi038.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T22:59:58.121 INFO:teuthology.orchestra.run.smithi117.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.8 MB/s | 321 kB 00:00 2024-04-24T22:59:58.152 INFO:teuthology.orchestra.run.smithi038.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-04-24T22:59:58.177 INFO:teuthology.orchestra.run.smithi038.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-24T22:59:58.263 INFO:teuthology.orchestra.run.smithi117.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.1 MB/s | 299 kB 00:00 2024-04-24T22:59:58.311 INFO:teuthology.orchestra.run.smithi038.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-04-24T22:59:58.336 INFO:teuthology.orchestra.run.smithi038.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-24T22:59:58.345 INFO:teuthology.orchestra.run.smithi033.stdout:(26/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 4.6 MB/s | 48 MB 00:10 2024-04-24T22:59:58.347 INFO:teuthology.orchestra.run.smithi117.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T22:59:58.370 INFO:teuthology.orchestra.run.smithi038.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-24T22:59:58.387 INFO:teuthology.orchestra.run.smithi038.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-04-24T22:59:58.414 INFO:teuthology.orchestra.run.smithi033.stdout:(27/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 2.4 MB/s | 3.1 MB 00:01 2024-04-24T22:59:58.422 INFO:teuthology.orchestra.run.smithi117.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T22:59:58.437 INFO:teuthology.orchestra.run.smithi038.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-24T22:59:58.454 INFO:teuthology.orchestra.run.smithi038.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-04-24T22:59:58.482 INFO:teuthology.orchestra.run.smithi033.stdout:(28/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 5.3 MB 00:04 2024-04-24T22:59:58.507 INFO:teuthology.orchestra.run.smithi033.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 346 kB/s | 32 kB 00:00 2024-04-24T22:59:58.532 INFO:teuthology.orchestra.run.smithi033.stdout:(30/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 918 kB/s | 171 kB 00:00 2024-04-24T22:59:58.538 INFO:teuthology.orchestra.run.smithi038.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-04-24T22:59:58.622 INFO:teuthology.orchestra.run.smithi038.stdout:(55/152): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-04-24T22:59:58.639 INFO:teuthology.orchestra.run.smithi038.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-24T22:59:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:(31/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 879 kB/s | 141 kB 00:00 2024-04-24T22:59:58.689 INFO:teuthology.orchestra.run.smithi038.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-24T22:59:58.789 INFO:teuthology.orchestra.run.smithi117.stdout:(26/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 1.8 MB/s | 17 MB 00:09 2024-04-24T22:59:58.881 INFO:teuthology.orchestra.run.smithi038.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 187 kB/s | 45 kB 00:00 2024-04-24T22:59:58.935 INFO:teuthology.orchestra.run.smithi033.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 853 kB/s | 249 kB 00:00 2024-04-24T22:59:58.956 INFO:teuthology.orchestra.run.smithi038.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-04-24T22:59:58.982 INFO:teuthology.orchestra.run.smithi038.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-04-24T22:59:59.031 INFO:teuthology.orchestra.run.smithi117.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 709 kB/s | 171 kB 00:00 2024-04-24T22:59:59.048 INFO:teuthology.orchestra.run.smithi038.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-24T22:59:59.060 INFO:teuthology.orchestra.run.smithi033.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 395 kB/s | 49 kB 00:00 2024-04-24T22:59:59.074 INFO:teuthology.orchestra.run.smithi038.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-04-24T22:59:59.116 INFO:teuthology.orchestra.run.smithi038.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-24T22:59:59.141 INFO:teuthology.orchestra.run.smithi038.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-04-24T22:59:59.147 INFO:teuthology.orchestra.run.smithi117.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 273 kB/s | 32 kB 00:00 2024-04-24T22:59:59.191 INFO:teuthology.orchestra.run.smithi038.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-04-24T22:59:59.193 INFO:teuthology.orchestra.run.smithi033.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 115 kB/s | 15 kB 00:00 2024-04-24T22:59:59.281 INFO:teuthology.orchestra.run.smithi117.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.0 MB/s | 141 kB 00:00 2024-04-24T22:59:59.283 INFO:teuthology.orchestra.run.smithi038.stdout:(66/152): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-04-24T22:59:59.423 INFO:teuthology.orchestra.run.smithi117.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 3.1 MB/s | 3.1 MB 00:01 2024-04-24T22:59:59.510 INFO:teuthology.orchestra.run.smithi033.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 833 kB/s | 264 kB 00:00 2024-04-24T22:59:59.592 INFO:teuthology.orchestra.run.smithi038.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-24T22:59:59.602 INFO:teuthology.orchestra.run.smithi033.stdout:(36/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.2 MB/s | 3.5 MB 00:01 2024-04-24T22:59:59.677 INFO:teuthology.orchestra.run.smithi033.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 244 kB/s | 18 kB 00:00 2024-04-24T22:59:59.684 INFO:teuthology.orchestra.run.smithi038.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-04-24T22:59:59.734 INFO:teuthology.orchestra.run.smithi038.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-24T22:59:59.793 INFO:teuthology.orchestra.run.smithi038.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.3 MB/s | 4.7 MB 00:00 2024-04-24T22:59:59.819 INFO:teuthology.orchestra.run.smithi033.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-24T22:59:59.835 INFO:teuthology.orchestra.run.smithi038.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-24T22:59:59.885 INFO:teuthology.orchestra.run.smithi038.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-24T22:59:59.919 INFO:teuthology.orchestra.run.smithi038.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.0 MB/s | 1.0 MB 00:00 2024-04-24T22:59:59.961 INFO:teuthology.orchestra.run.smithi033.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2024-04-24T22:59:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:00:00.011 INFO:teuthology.orchestra.run.smithi033.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-24T23:00:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-24T23:00:00.062 INFO:teuthology.orchestra.run.smithi033.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-24T23:00:00.111 INFO:teuthology.orchestra.run.smithi038.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-24T23:00:00.136 INFO:teuthology.orchestra.run.smithi038.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-04-24T23:00:00.162 INFO:teuthology.orchestra.run.smithi038.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-24T23:00:00.187 INFO:teuthology.orchestra.run.smithi038.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-24T23:00:00.212 INFO:teuthology.orchestra.run.smithi033.stdout:(42/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 1.1 MB/s | 762 kB 00:00 2024-04-24T23:00:00.212 INFO:teuthology.orchestra.run.smithi038.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-04-24T23:00:00.246 INFO:teuthology.orchestra.run.smithi033.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-04-24T23:00:00.263 INFO:teuthology.orchestra.run.smithi038.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-24T23:00:00.287 INFO:teuthology.orchestra.run.smithi033.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-24T23:00:00.313 INFO:teuthology.orchestra.run.smithi038.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T23:00:00.329 INFO:teuthology.orchestra.run.smithi033.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-24T23:00:00.363 INFO:teuthology.orchestra.run.smithi033.stdout:(46/152): python3-ply-3.11-14.el9.noarch.rpm 706 kB/s | 106 kB 00:00 2024-04-24T23:00:00.363 INFO:teuthology.orchestra.run.smithi038.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T23:00:00.391 INFO:teuthology.orchestra.run.smithi033.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-24T23:00:00.414 INFO:teuthology.orchestra.run.smithi038.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-04-24T23:00:00.472 INFO:teuthology.orchestra.run.smithi038.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T23:00:00.480 INFO:teuthology.orchestra.run.smithi033.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-04-24T23:00:00.522 INFO:teuthology.orchestra.run.smithi033.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-04-24T23:00:00.523 INFO:teuthology.orchestra.run.smithi038.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T23:00:00.547 INFO:teuthology.orchestra.run.smithi033.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-24T23:00:00.572 INFO:teuthology.orchestra.run.smithi033.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-24T23:00:00.598 INFO:teuthology.orchestra.run.smithi038.stdout:(87/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.4 MB/s | 6.0 MB 00:00 2024-04-24T23:00:00.598 INFO:teuthology.orchestra.run.smithi033.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-24T23:00:00.623 INFO:teuthology.orchestra.run.smithi033.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-24T23:00:00.624 INFO:teuthology.orchestra.run.smithi038.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-24T23:00:00.640 INFO:teuthology.orchestra.run.smithi038.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-24T23:00:00.674 INFO:teuthology.orchestra.run.smithi033.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-24T23:00:00.707 INFO:teuthology.orchestra.run.smithi038.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.8 MB/s | 5.1 MB 00:00 2024-04-24T23:00:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-24T23:00:00.724 INFO:teuthology.orchestra.run.smithi038.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 532 kB/s | 54 kB 00:00 2024-04-24T23:00:00.758 INFO:teuthology.orchestra.run.smithi038.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-24T23:00:00.775 INFO:teuthology.orchestra.run.smithi038.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-04-24T23:00:00.782 INFO:teuthology.orchestra.run.smithi033.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-24T23:00:00.807 INFO:teuthology.orchestra.run.smithi033.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-24T23:00:00.817 INFO:teuthology.orchestra.run.smithi038.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-24T23:00:00.841 INFO:teuthology.orchestra.run.smithi033.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-24T23:00:00.842 INFO:teuthology.orchestra.run.smithi038.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-04-24T23:00:00.876 INFO:teuthology.orchestra.run.smithi038.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-24T23:00:00.900 INFO:teuthology.orchestra.run.smithi033.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi038.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-24T23:00:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-24T23:00:00.935 INFO:teuthology.orchestra.run.smithi038.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-04-24T23:00:00.959 INFO:teuthology.orchestra.run.smithi033.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-24T23:00:00.968 INFO:teuthology.orchestra.run.smithi038.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T23:00:00.992 INFO:teuthology.orchestra.run.smithi033.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-04-24T23:00:00.993 INFO:teuthology.orchestra.run.smithi038.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T23:00:01.018 INFO:teuthology.orchestra.run.smithi033.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-24T23:00:01.035 INFO:teuthology.orchestra.run.smithi038.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-04-24T23:00:01.051 INFO:teuthology.orchestra.run.smithi033.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T23:00:01.068 INFO:teuthology.orchestra.run.smithi033.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-24T23:00:01.160 INFO:teuthology.orchestra.run.smithi033.stdout:(66/152): ceph-mgr-diskprediction-local-19.0.0- 2.8 MB/s | 7.4 MB 00:02 2024-04-24T23:00:01.161 INFO:teuthology.orchestra.run.smithi038.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-04-24T23:00:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-24T23:00:01.386 INFO:teuthology.orchestra.run.smithi033.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-24T23:00:01.469 INFO:teuthology.orchestra.run.smithi033.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-24T23:00:01.494 INFO:teuthology.orchestra.run.smithi033.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-24T23:00:01.565 INFO:teuthology.orchestra.run.smithi117.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 1.5 MB/s | 3.5 MB 00:02 2024-04-24T23:00:01.634 INFO:teuthology.orchestra.run.smithi038.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-24T23:00:01.670 INFO:teuthology.orchestra.run.smithi033.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-24T23:00:01.720 INFO:teuthology.orchestra.run.smithi033.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-24T23:00:01.762 INFO:teuthology.orchestra.run.smithi038.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-04-24T23:00:01.766 INFO:teuthology.orchestra.run.smithi117.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.2 MB/s | 249 kB 00:00 2024-04-24T23:00:01.820 INFO:teuthology.orchestra.run.smithi033.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-24T23:00:01.854 INFO:teuthology.orchestra.run.smithi033.stdout:(74/152): postgresql-13.14-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-24T23:00:01.862 INFO:teuthology.orchestra.run.smithi038.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 783 kB/s | 548 kB 00:00 2024-04-24T23:00:01.879 INFO:teuthology.orchestra.run.smithi033.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:00:01.881 INFO:teuthology.orchestra.run.smithi117.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 423 kB/s | 49 kB 00:00 2024-04-24T23:00:01.904 INFO:teuthology.orchestra.run.smithi038.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-24T23:00:01.921 INFO:teuthology.orchestra.run.smithi033.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-04-24T23:00:01.929 INFO:teuthology.orchestra.run.smithi038.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 116 kB/s | 19 kB 00:00 2024-04-24T23:00:01.947 INFO:teuthology.orchestra.run.smithi033.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-24T23:00:01.972 INFO:teuthology.orchestra.run.smithi033.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-24T23:00:01.997 INFO:teuthology.orchestra.run.smithi033.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-24T23:00:02.022 INFO:teuthology.orchestra.run.smithi033.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-04-24T23:00:02.072 INFO:teuthology.orchestra.run.smithi038.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-24T23:00:02.096 INFO:teuthology.orchestra.run.smithi038.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 135 kB/s | 26 kB 00:00 2024-04-24T23:00:02.097 INFO:teuthology.orchestra.run.smithi033.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-24T23:00:02.121 INFO:teuthology.orchestra.run.smithi038.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 255 kB/s | 49 kB 00:00 2024-04-24T23:00:02.123 INFO:teuthology.orchestra.run.smithi033.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-24T23:00:02.147 INFO:teuthology.orchestra.run.smithi038.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-24T23:00:02.148 INFO:teuthology.orchestra.run.smithi033.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T23:00:02.173 INFO:teuthology.orchestra.run.smithi033.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T23:00:02.189 INFO:teuthology.orchestra.run.smithi038.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-24T23:00:02.199 INFO:teuthology.orchestra.run.smithi033.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-24T23:00:02.222 INFO:teuthology.orchestra.run.smithi038.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-04-24T23:00:02.248 INFO:teuthology.orchestra.run.smithi038.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-24T23:00:02.249 INFO:teuthology.orchestra.run.smithi033.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-24T23:00:02.273 INFO:teuthology.orchestra.run.smithi038.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 537 kB/s | 43 kB 00:00 2024-04-24T23:00:02.274 INFO:teuthology.orchestra.run.smithi033.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-24T23:00:02.298 INFO:teuthology.orchestra.run.smithi038.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-24T23:00:02.299 INFO:teuthology.orchestra.run.smithi117.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 37 kB/s | 15 kB 00:00 2024-04-24T23:00:02.299 INFO:teuthology.orchestra.run.smithi033.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T23:00:02.324 INFO:teuthology.orchestra.run.smithi038.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-04-24T23:00:02.324 INFO:teuthology.orchestra.run.smithi033.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi038.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-24T23:00:02.350 INFO:teuthology.orchestra.run.smithi033.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:00:02.366 INFO:teuthology.orchestra.run.smithi038.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-24T23:00:02.400 INFO:teuthology.orchestra.run.smithi033.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-04-24T23:00:02.425 INFO:teuthology.orchestra.run.smithi038.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-24T23:00:02.442 INFO:teuthology.orchestra.run.smithi033.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-24T23:00:02.450 INFO:teuthology.orchestra.run.smithi038.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-24T23:00:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-24T23:00:02.494 INFO:teuthology.orchestra.run.smithi033.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T23:00:02.524 INFO:teuthology.orchestra.run.smithi117.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 264 kB 00:00 2024-04-24T23:00:02.526 INFO:teuthology.orchestra.run.smithi033.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-24T23:00:02.560 INFO:teuthology.orchestra.run.smithi033.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-24T23:00:02.610 INFO:teuthology.orchestra.run.smithi033.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-24T23:00:02.617 INFO:teuthology.orchestra.run.smithi038.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-24T23:00:02.634 INFO:teuthology.orchestra.run.smithi038.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-04-24T23:00:02.643 INFO:teuthology.orchestra.run.smithi033.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-04-24T23:00:02.659 INFO:teuthology.orchestra.run.smithi038.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 786 kB/s | 144 kB 00:00 2024-04-24T23:00:02.686 INFO:teuthology.orchestra.run.smithi033.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-24T23:00:02.709 INFO:teuthology.orchestra.run.smithi038.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-24T23:00:02.735 INFO:teuthology.orchestra.run.smithi038.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-24T23:00:02.752 INFO:teuthology.orchestra.run.smithi038.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-24T23:00:02.752 INFO:teuthology.orchestra.run.smithi033.stdout:(99/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.8 MB/s | 5.1 MB 00:00 2024-04-24T23:00:02.778 INFO:teuthology.orchestra.run.smithi033.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 695 kB/s | 64 kB 00:00 2024-04-24T23:00:02.802 INFO:teuthology.orchestra.run.smithi038.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-04-24T23:00:02.803 INFO:teuthology.orchestra.run.smithi033.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-24T23:00:02.944 INFO:teuthology.orchestra.run.smithi038.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-24T23:00:02.961 INFO:teuthology.orchestra.run.smithi038.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-04-24T23:00:02.970 INFO:teuthology.orchestra.run.smithi033.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-24T23:00:02.982 INFO:teuthology.orchestra.run.smithi117.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-04-24T23:00:02.986 INFO:teuthology.orchestra.run.smithi038.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 432 kB/s | 79 kB 00:00 2024-04-24T23:00:03.053 INFO:teuthology.orchestra.run.smithi038.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 868 kB/s | 58 kB 00:00 2024-04-24T23:00:03.070 INFO:teuthology.orchestra.run.smithi038.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-24T23:00:03.071 INFO:teuthology.orchestra.run.smithi033.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-04-24T23:00:03.095 INFO:teuthology.orchestra.run.smithi038.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-24T23:00:03.096 INFO:teuthology.orchestra.run.smithi033.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-04-24T23:00:03.099 INFO:teuthology.orchestra.run.smithi117.stdout:(37/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-24T23:00:03.125 INFO:teuthology.orchestra.run.smithi117.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 129 kB/s | 18 kB 00:00 2024-04-24T23:00:03.129 INFO:teuthology.orchestra.run.smithi033.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 584 kB/s | 19 kB 00:00 2024-04-24T23:00:03.146 INFO:teuthology.orchestra.run.smithi038.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-04-24T23:00:03.171 INFO:teuthology.orchestra.run.smithi038.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-24T23:00:03.188 INFO:teuthology.orchestra.run.smithi033.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-04-24T23:00:03.196 INFO:teuthology.orchestra.run.smithi038.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-24T23:00:03.221 INFO:teuthology.orchestra.run.smithi038.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-24T23:00:03.263 INFO:teuthology.orchestra.run.smithi033.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-04-24T23:00:03.271 INFO:teuthology.orchestra.run.smithi038.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 792 kB/s | 59 kB 00:00 2024-04-24T23:00:03.288 INFO:teuthology.orchestra.run.smithi033.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-24T23:00:03.297 INFO:teuthology.orchestra.run.smithi038.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-24T23:00:03.308 INFO:teuthology.orchestra.run.smithi117.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 386 kB/s | 80 kB 00:00 2024-04-24T23:00:03.322 INFO:teuthology.orchestra.run.smithi033.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-04-24T23:00:03.322 INFO:teuthology.orchestra.run.smithi038.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-24T23:00:03.347 INFO:teuthology.orchestra.run.smithi033.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-24T23:00:03.347 INFO:teuthology.orchestra.run.smithi038.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-24T23:00:03.372 INFO:teuthology.orchestra.run.smithi033.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-24T23:00:03.389 INFO:teuthology.orchestra.run.smithi038.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-24T23:00:03.414 INFO:teuthology.orchestra.run.smithi033.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-04-24T23:00:03.415 INFO:teuthology.orchestra.run.smithi038.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-24T23:00:03.440 INFO:teuthology.orchestra.run.smithi038.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-24T23:00:03.448 INFO:teuthology.orchestra.run.smithi033.stdout:(113/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-24T23:00:03.457 INFO:teuthology.orchestra.run.smithi038.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-24T23:00:03.473 INFO:teuthology.orchestra.run.smithi033.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-04-24T23:00:03.499 INFO:teuthology.orchestra.run.smithi033.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 838 kB 00:00 2024-04-24T23:00:03.524 INFO:teuthology.orchestra.run.smithi033.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-24T23:00:03.532 INFO:teuthology.orchestra.run.smithi038.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-24T23:00:03.550 INFO:teuthology.orchestra.run.smithi033.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-04-24T23:00:03.567 INFO:teuthology.orchestra.run.smithi117.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 756 kB/s | 195 kB 00:00 2024-04-24T23:00:03.607 INFO:teuthology.orchestra.run.smithi033.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-24T23:00:03.609 INFO:teuthology.orchestra.run.smithi117.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-04-24T23:00:03.633 INFO:teuthology.orchestra.run.smithi033.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-24T23:00:03.658 INFO:teuthology.orchestra.run.smithi033.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-04-24T23:00:03.683 INFO:teuthology.orchestra.run.smithi033.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-24T23:00:03.708 INFO:teuthology.orchestra.run.smithi033.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-24T23:00:03.733 INFO:teuthology.orchestra.run.smithi038.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 1.3 MB 00:00 2024-04-24T23:00:03.734 INFO:teuthology.orchestra.run.smithi033.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-24T23:00:03.759 INFO:teuthology.orchestra.run.smithi033.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 211 kB/s | 11 kB 00:00 2024-04-24T23:00:03.785 INFO:teuthology.orchestra.run.smithi033.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-24T23:00:03.810 INFO:teuthology.orchestra.run.smithi033.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-04-24T23:00:03.835 INFO:teuthology.orchestra.run.smithi033.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-24T23:00:03.861 INFO:teuthology.orchestra.run.smithi033.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-24T23:00:03.874 INFO:teuthology.orchestra.run.smithi038.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-04-24T23:00:03.894 INFO:teuthology.orchestra.run.smithi033.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-04-24T23:00:03.901 INFO:teuthology.orchestra.run.smithi117.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 760 kB/s | 253 kB 00:00 2024-04-24T23:00:03.919 INFO:teuthology.orchestra.run.smithi033.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 792 kB/s | 46 kB 00:00 2024-04-24T23:00:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-04-24T23:00:03.986 INFO:teuthology.orchestra.run.smithi033.stdout:(132/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-04-24T23:00:04.012 INFO:teuthology.orchestra.run.smithi033.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-24T23:00:04.045 INFO:teuthology.orchestra.run.smithi033.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-24T23:00:04.070 INFO:teuthology.orchestra.run.smithi033.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-24T23:00:04.096 INFO:teuthology.orchestra.run.smithi033.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-04-24T23:00:04.121 INFO:teuthology.orchestra.run.smithi033.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-04-24T23:00:04.142 INFO:teuthology.orchestra.run.smithi117.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 441 kB/s | 106 kB 00:00 2024-04-24T23:00:04.146 INFO:teuthology.orchestra.run.smithi033.stdout:(138/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-24T23:00:04.163 INFO:teuthology.orchestra.run.smithi033.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-24T23:00:04.189 INFO:teuthology.orchestra.run.smithi033.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-24T23:00:04.205 INFO:teuthology.orchestra.run.smithi033.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 609 kB/s | 36 kB 00:00 2024-04-24T23:00:04.208 INFO:teuthology.orchestra.run.smithi038.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.2 MB/s | 3.1 MB 00:00 2024-04-24T23:00:04.222 INFO:teuthology.orchestra.run.smithi033.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-04-24T23:00:04.226 INFO:teuthology.orchestra.run.smithi117.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-04-24T23:00:04.247 INFO:teuthology.orchestra.run.smithi033.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-04-24T23:00:04.273 INFO:teuthology.orchestra.run.smithi033.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-24T23:00:04.298 INFO:teuthology.orchestra.run.smithi033.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-24T23:00:04.315 INFO:teuthology.orchestra.run.smithi033.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-24T23:00:04.340 INFO:teuthology.orchestra.run.smithi033.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-04-24T23:00:04.382 INFO:teuthology.orchestra.run.smithi033.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-24T23:00:04.500 INFO:teuthology.orchestra.run.smithi038.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.2 MB/s | 3.4 MB 00:01 2024-04-24T23:00:04.504 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:00:04.504 INFO:teuthology.orchestra.run.smithi038.stdout:Total 11 MB/s | 220 MB 00:19 2024-04-24T23:00:04.566 INFO:teuthology.orchestra.run.smithi033.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-04-24T23:00:04.652 INFO:teuthology.orchestra.run.smithi117.stdout:(45/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 3.8 MB/s | 48 MB 00:12 2024-04-24T23:00:04.699 INFO:teuthology.orchestra.run.smithi033.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-24T23:00:04.752 INFO:teuthology.orchestra.run.smithi117.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 222 kB/s | 135 kB 00:00 2024-04-24T23:00:04.844 INFO:teuthology.orchestra.run.smithi117.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 202 kB/s | 125 kB 00:00 2024-04-24T23:00:04.908 INFO:teuthology.orchestra.run.smithi033.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-04-24T23:00:04.911 INFO:teuthology.orchestra.run.smithi117.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 830 kB/s | 215 kB 00:00 2024-04-24T23:00:04.936 INFO:teuthology.orchestra.run.smithi117.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-24T23:00:04.969 INFO:teuthology.orchestra.run.smithi117.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 647 kB/s | 21 kB 00:00 2024-04-24T23:00:05.011 INFO:teuthology.orchestra.run.smithi117.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-24T23:00:05.037 INFO:teuthology.orchestra.run.smithi117.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 851 kB/s | 107 kB 00:00 2024-04-24T23:00:05.054 INFO:teuthology.orchestra.run.smithi117.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 781 kB/s | 33 kB 00:00 2024-04-24T23:00:05.087 INFO:teuthology.orchestra.run.smithi117.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-24T23:00:05.104 INFO:teuthology.orchestra.run.smithi117.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-24T23:00:05.130 INFO:teuthology.orchestra.run.smithi117.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-04-24T23:00:05.155 INFO:teuthology.orchestra.run.smithi117.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T23:00:05.180 INFO:teuthology.orchestra.run.smithi117.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-24T23:00:05.214 INFO:teuthology.orchestra.run.smithi117.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T23:00:05.256 INFO:teuthology.orchestra.run.smithi117.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-24T23:00:05.272 INFO:teuthology.orchestra.run.smithi117.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-04-24T23:00:05.298 INFO:teuthology.orchestra.run.smithi117.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-24T23:00:05.323 INFO:teuthology.orchestra.run.smithi117.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-24T23:00:05.357 INFO:teuthology.orchestra.run.smithi117.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T23:00:05.374 INFO:teuthology.orchestra.run.smithi117.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-24T23:00:05.441 INFO:teuthology.orchestra.run.smithi117.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-04-24T23:00:05.524 INFO:teuthology.orchestra.run.smithi117.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-24T23:00:05.574 INFO:teuthology.orchestra.run.smithi033.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 2.7 MB/s | 3.4 MB 00:01 2024-04-24T23:00:05.577 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:00:05.577 INFO:teuthology.orchestra.run.smithi033.stdout:Total 9.8 MB/s | 220 MB 00:22 2024-04-24T23:00:05.591 INFO:teuthology.orchestra.run.smithi117.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-24T23:00:05.610 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-04-24T23:00:05.633 INFO:teuthology.orchestra.run.smithi117.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-24T23:00:05.650 INFO:teuthology.orchestra.run.smithi117.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T23:00:05.692 INFO:teuthology.orchestra.run.smithi117.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-24T23:00:05.726 INFO:teuthology.orchestra.run.smithi117.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-24T23:00:05.751 INFO:teuthology.orchestra.run.smithi117.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T23:00:05.769 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-04-24T23:00:05.769 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-04-24T23:00:05.793 INFO:teuthology.orchestra.run.smithi117.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-24T23:00:05.868 INFO:teuthology.orchestra.run.smithi117.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-24T23:00:05.893 INFO:teuthology.orchestra.run.smithi117.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-24T23:00:05.919 INFO:teuthology.orchestra.run.smithi117.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-04-24T23:00:05.944 INFO:teuthology.orchestra.run.smithi117.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-24T23:00:05.970 INFO:teuthology.orchestra.run.smithi117.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-24T23:00:05.994 INFO:teuthology.orchestra.run.smithi117.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-24T23:00:06.086 INFO:teuthology.orchestra.run.smithi117.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-24T23:00:06.112 INFO:teuthology.orchestra.run.smithi117.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-24T23:00:06.137 INFO:teuthology.orchestra.run.smithi117.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T23:00:06.162 INFO:teuthology.orchestra.run.smithi117.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T23:00:06.187 INFO:teuthology.orchestra.run.smithi117.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-24T23:00:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-24T23:00:06.229 INFO:teuthology.orchestra.run.smithi117.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-24T23:00:06.255 INFO:teuthology.orchestra.run.smithi117.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-04-24T23:00:06.272 INFO:teuthology.orchestra.run.smithi117.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-24T23:00:06.289 INFO:teuthology.orchestra.run.smithi117.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-24T23:00:06.364 INFO:teuthology.orchestra.run.smithi117.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-24T23:00:06.381 INFO:teuthology.orchestra.run.smithi117.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-04-24T23:00:06.398 INFO:teuthology.orchestra.run.smithi117.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-24T23:00:06.423 INFO:teuthology.orchestra.run.smithi117.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-04-24T23:00:06.440 INFO:teuthology.orchestra.run.smithi117.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 855 kB/s | 36 kB 00:00 2024-04-24T23:00:06.457 INFO:teuthology.orchestra.run.smithi117.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-24T23:00:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-24T23:00:06.516 INFO:teuthology.orchestra.run.smithi117.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-24T23:00:06.541 INFO:teuthology.orchestra.run.smithi117.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:00:06.575 INFO:teuthology.orchestra.run.smithi117.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T23:00:06.600 INFO:teuthology.orchestra.run.smithi117.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-04-24T23:00:06.685 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-24T23:00:06.825 INFO:teuthology.orchestra.run.smithi117.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-24T23:00:06.847 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-24T23:00:06.848 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-24T23:00:06.934 INFO:teuthology.orchestra.run.smithi117.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 5.0 MB/s | 548 kB 00:00 2024-04-24T23:00:06.985 INFO:teuthology.orchestra.run.smithi117.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 6.0 MB/s | 309 kB 00:00 2024-04-24T23:00:07.010 INFO:teuthology.orchestra.run.smithi117.stdout:(105/152): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2024-04-24T23:00:07.035 INFO:teuthology.orchestra.run.smithi117.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-04-24T23:00:07.068 INFO:teuthology.orchestra.run.smithi117.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 781 kB/s | 26 kB 00:00 2024-04-24T23:00:07.102 INFO:teuthology.orchestra.run.smithi117.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-04-24T23:00:07.136 INFO:teuthology.orchestra.run.smithi117.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi117.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-04-24T23:00:07.245 INFO:teuthology.orchestra.run.smithi117.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-24T23:00:07.286 INFO:teuthology.orchestra.run.smithi117.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-24T23:00:07.312 INFO:teuthology.orchestra.run.smithi117.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-24T23:00:07.337 INFO:teuthology.orchestra.run.smithi117.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-04-24T23:00:07.368 INFO:teuthology.orchestra.run.smithi117.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-24T23:00:07.388 INFO:teuthology.orchestra.run.smithi117.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-24T23:00:07.413 INFO:teuthology.orchestra.run.smithi117.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-24T23:00:07.446 INFO:teuthology.orchestra.run.smithi117.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-24T23:00:07.473 INFO:teuthology.orchestra.run.smithi117.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-24T23:00:07.497 INFO:teuthology.orchestra.run.smithi117.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-04-24T23:00:07.522 INFO:teuthology.orchestra.run.smithi117.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-24T23:00:07.581 INFO:teuthology.orchestra.run.smithi117.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-04-24T23:00:07.606 INFO:teuthology.orchestra.run.smithi117.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-24T23:00:07.623 INFO:teuthology.orchestra.run.smithi117.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-04-24T23:00:07.640 INFO:teuthology.orchestra.run.smithi117.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-04-24T23:00:07.642 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-04-24T23:00:07.643 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-04-24T23:00:07.657 INFO:teuthology.orchestra.run.smithi117.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 690 kB/s | 23 kB 00:00 2024-04-24T23:00:07.674 INFO:teuthology.orchestra.run.smithi117.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2024-04-24T23:00:07.691 INFO:teuthology.orchestra.run.smithi117.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 585 kB/s | 20 kB 00:00 2024-04-24T23:00:07.716 INFO:teuthology.orchestra.run.smithi117.stdout:(129/152): protobuf-compiler-3.14.0-13.el9.x86_ 774 kB/s | 863 kB 00:01 2024-04-24T23:00:07.741 INFO:teuthology.orchestra.run.smithi117.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 922 kB/s | 46 kB 00:00 2024-04-24T23:00:07.784 INFO:teuthology.orchestra.run.smithi117.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-24T23:00:07.800 INFO:teuthology.orchestra.run.smithi117.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-04-24T23:00:07.825 INFO:teuthology.orchestra.run.smithi117.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-24T23:00:07.842 INFO:teuthology.orchestra.run.smithi117.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 393 kB/s | 16 kB 00:00 2024-04-24T23:00:07.869 INFO:teuthology.orchestra.run.smithi117.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-24T23:00:07.885 INFO:teuthology.orchestra.run.smithi117.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 668 kB/s | 28 kB 00:00 2024-04-24T23:00:07.910 INFO:teuthology.orchestra.run.smithi117.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-24T23:00:07.927 INFO:teuthology.orchestra.run.smithi117.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-04-24T23:00:07.961 INFO:teuthology.orchestra.run.smithi117.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-24T23:00:07.986 INFO:teuthology.orchestra.run.smithi117.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 614 kB/s | 36 kB 00:00 2024-04-24T23:00:08.011 INFO:teuthology.orchestra.run.smithi117.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-24T23:00:08.036 INFO:teuthology.orchestra.run.smithi117.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-24T23:00:08.061 INFO:teuthology.orchestra.run.smithi117.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-24T23:00:08.087 INFO:teuthology.orchestra.run.smithi117.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T23:00:08.112 INFO:teuthology.orchestra.run.smithi117.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-24T23:00:08.146 INFO:teuthology.orchestra.run.smithi117.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-24T23:00:08.171 INFO:teuthology.orchestra.run.smithi117.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-24T23:00:08.330 INFO:teuthology.orchestra.run.smithi117.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-24T23:00:08.663 INFO:teuthology.orchestra.run.smithi117.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.6 MB/s | 3.4 MB 00:00 2024-04-24T23:00:08.743 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-24T23:00:08.744 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-24T23:00:08.963 INFO:teuthology.orchestra.run.smithi117.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 1.3 MB 00:00 2024-04-24T23:00:09.423 INFO:teuthology.orchestra.run.smithi117.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-24T23:00:09.514 INFO:teuthology.orchestra.run.smithi117.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-04-24T23:00:09.517 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:00:09.517 INFO:teuthology.orchestra.run.smithi117.stdout:Total 8.7 MB/s | 220 MB 00:25 2024-04-24T23:00:10.629 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-24T23:00:10.772 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-24T23:00:10.772 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-24T23:00:12.531 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-24T23:00:12.532 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-24T23:00:13.752 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-04-24T23:00:13.854 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:00:13.935 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:00:14.181 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:00:14.222 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:00:14.248 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:00:14.291 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:00:14.675 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:00:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-24T23:00:14.790 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:14.887 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:00:14.906 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:14.934 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:15.017 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:15.046 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:15.102 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:00:15.165 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:00:15.201 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:00:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:00:15.230 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:00:15.243 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:00:15.271 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:00:15.376 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:15.422 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:15.476 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:15.498 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:15.521 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:15.621 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:00:15.722 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:15.741 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:15.773 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:15.838 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:15.886 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:00:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:15.978 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:00:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:16.014 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:00:16.049 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:00:16.124 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:00:16.142 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:00:16.165 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:00:16.192 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:00:16.209 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:00:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:16.241 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:00:16.269 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:00:16.325 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:16.340 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:00:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:16.401 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:00:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:16.430 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:00:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:16.465 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:00:16.543 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:00:16.584 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:00:16.628 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:00:16.639 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:16.656 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:00:16.671 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:16.728 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:00:16.771 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:00:16.782 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:00:16.848 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:00:16.879 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:00:16.912 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:00:16.947 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:00:16.993 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:00:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:00:17.050 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:00:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:00:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:00:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:00:17.158 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:00:17.230 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:00:17.290 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:00:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:00:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:00:17.441 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:00:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:00:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:00:17.601 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:00:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:00:17.696 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:00:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:00:17.723 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:00:17.767 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:00:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:00:17.804 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:00:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:00:17.933 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:00:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:00:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:00:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:00:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:00:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:00:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:00:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:00:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:00:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:00:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:00:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:00:18.545 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:00:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:00:18.607 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-24T23:00:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:00:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:00:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:00:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:00:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:00:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:00:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:00:18.730 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:00:18.751 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:00:18.784 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:00:18.810 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:00:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:00:18.826 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:00:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:00:18.851 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:00:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:00:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:00:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:00:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:00:18.980 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:00:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:00:19.038 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:00:19.094 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:00:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:00:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:00:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:00:19.206 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:00:19.226 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:00:19.241 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:00:19.260 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:00:19.275 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:00:19.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:00:19.299 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:00:19.320 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:19.331 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:00:19.332 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:00:19.360 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:00:19.376 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:00:19.395 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:00:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:00:19.428 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:00:19.440 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:19.462 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:19.470 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:00:19.505 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:00:19.553 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:19.564 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:00:19.583 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:19.594 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:00:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:00:19.636 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:00:19.652 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:00:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:00:19.691 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:00:19.739 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:00:19.777 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:00:19.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:00:19.815 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:19.819 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:00:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:00:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:00:19.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:00:19.902 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:00:19.909 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:00:19.919 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:19.989 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:00:20.004 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:20.026 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:20.044 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:20.127 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:00:20.158 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:00:20.187 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:00:20.221 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:00:20.241 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:00:20.275 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:20.308 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:00:20.338 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:00:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:00:20.383 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:00:20.405 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:00:20.448 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:00:20.478 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:00:20.482 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:00:20.508 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:00:20.542 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:00:20.573 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:00:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:00:20.621 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:00:20.641 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:00:20.658 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:00:20.660 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:00:20.685 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:00:20.707 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:00:20.708 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:00:20.717 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:00:20.741 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:00:20.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:00:20.813 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:00:20.819 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:00:20.843 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:00:20.867 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:00:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:00:20.891 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:00:20.906 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:00:20.942 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:00:20.980 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:00:21.017 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:00:21.018 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:00:21.053 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:00:21.096 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:00:21.125 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:00:21.193 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:00:21.195 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:00:21.232 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:00:21.239 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:00:21.312 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:00:21.313 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:00:21.446 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:00:21.500 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:00:21.646 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:00:21.692 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:00:21.726 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:00:21.828 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:00:21.884 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:00:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:00:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:00:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:00:22.021 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:00:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:00:22.103 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:00:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:00:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:00:22.188 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:00:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:00:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:00:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:00:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:00:22.422 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:00:22.483 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:00:22.538 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:00:22.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:00:22.651 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:00:22.687 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:00:22.734 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:00:22.770 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:00:22.813 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:00:22.854 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:00:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:00:22.991 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:00:23.035 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:00:23.065 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:00:23.103 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:00:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:00:23.201 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:00:23.249 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:00:23.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:00:23.315 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:00:23.339 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:00:23.393 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:00:23.414 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:00:23.479 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:00:23.507 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:00:23.541 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:00:23.541 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:00:23.572 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:00:23.599 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:00:23.634 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:00:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:00:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:00:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:00:23.769 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:00:23.802 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:00:23.843 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:00:23.868 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:00:23.878 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:00:23.912 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:00:24.003 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:24.008 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:00:24.033 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:00:24.048 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:00:24.070 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:00:24.095 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:00:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:00:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:00:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:00:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:00:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:00:24.578 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:00:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:00:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:24.678 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:24.708 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:00:24.908 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:00:24.950 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:00:24.960 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:24.996 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:00:25.030 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:00:25.035 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:25.057 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:00:25.093 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:00:25.128 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:00:25.196 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:00:25.283 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:00:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:00:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:00:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:00:25.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:00:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:00:25.621 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:25.693 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:25.723 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:00:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:00:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:00:27.095 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:27.131 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:27.592 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:00:27.675 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:00:27.700 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:00:27.736 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:00:27.742 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:27.818 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:00:27.818 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:00:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:28.048 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:00:28.083 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:28.172 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:28.204 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:00:28.240 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:00:28.710 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:00:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:28.744 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:28.815 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:28.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:00:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:28.838 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:00:28.838 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:00:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:31.157 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:31.194 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:31.730 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:31.780 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:31.880 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:31.880 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:00:31.880 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:00:31.881 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:31.915 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:39.746 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-04-24T23:00:39.747 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:39.926 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:39.942 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:39.943 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:39.943 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:00:39.943 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:40.671 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-04-24T23:00:40.671 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-04-24T23:00:40.672 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-04-24T23:00:40.672 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-04-24T23:00:40.672 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-04-24T23:00:40.672 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-04-24T23:00:40.672 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-04-24T23:00:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:40.850 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:40.851 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:00:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:41.480 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:41.507 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:41.507 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:41.507 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:00:41.507 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:41.507 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:41.507 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:41.739 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:41.762 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:41.762 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:41.762 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:00:41.763 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:41.763 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:41.763 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:42.223 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:42.246 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:42.354 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:00:42.380 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:42.406 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:42.407 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:42.407 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:00:42.407 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:42.407 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:42.407 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:42.411 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:00:42.475 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:00:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:42.662 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:42.662 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:00:42.662 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:42.663 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:42.671 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:00:42.717 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:00:42.797 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:00:42.885 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:43.027 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:43.066 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:00:43.107 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:00:43.122 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:43.134 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:00:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:43.147 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:43.147 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:00:43.147 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:43.147 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:43.147 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:43.253 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:00:43.312 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-04-24T23:00:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:00:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:43.565 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:43.565 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:00:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:43.570 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:00:43.624 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:00:43.704 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:00:43.722 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:43.750 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:43.783 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:00:43.794 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:43.848 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:00:43.882 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:43.935 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:43.990 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:00:44.022 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:00:44.049 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:00:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:44.682 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:44.714 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:00:44.780 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:00:44.813 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:44.972 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:45.009 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:45.036 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:45.061 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:45.061 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:45.061 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:00:45.061 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:45.061 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:45.062 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:45.309 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:45.309 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:00:45.309 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:45.309 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:45.745 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:45.866 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:00:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:45.915 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:00:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:45.970 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:00:46.025 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:46.051 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:46.153 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:00:46.157 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:46.204 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:00:46.282 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:00:46.330 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:00:46.361 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:46.370 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:46.386 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:46.386 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:46.387 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:00:46.387 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:46.387 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:46.387 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:46.437 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:46.448 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:46.512 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:46.550 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:00:46.554 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:00:46.586 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:00:46.604 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:00:46.613 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:00:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:47.045 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:47.070 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:47.110 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:00:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:00:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:00:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:00:47.549 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:00:47.581 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:47.606 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:47.606 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:47.606 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:00:47.606 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:47.606 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:47.606 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:47.649 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:47.672 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:47.672 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:47.672 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:00:47.672 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:47.725 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:47.748 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:47.748 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:47.749 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:00:47.749 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:47.749 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:47.749 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:48.084 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:48.108 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:48.108 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:48.108 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:00:48.108 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:48.108 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:48.108 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:00:48.240 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:48.278 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:48.470 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:00:48.503 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:00:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:48.528 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:48.570 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:48.593 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:48.593 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:48.593 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:00:48.593 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:48.637 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:48.661 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:48.661 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:48.661 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:00:48.661 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:48.661 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:48.661 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:48.980 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:49.004 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:49.004 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:49.004 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:00:49.004 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:49.004 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:49.005 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:00:49.247 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:49.279 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:49.380 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:49.543 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:00:49.573 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:49.596 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:49.596 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:49.596 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:00:49.596 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:49.596 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:49.596 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:49.640 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:49.651 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:49.757 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:00:49.813 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:00:50.708 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:00:50.734 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:50.757 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:50.757 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:50.757 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:00:50.757 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:50.757 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:50.757 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:50.801 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:50.823 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:00:50.823 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:50.868 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:50.891 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:50.891 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:50.891 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:00:50.891 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:50.892 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:50.892 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:51.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:51.221 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:51.221 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:51.221 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:00:51.222 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:51.222 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:51.222 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:00:52.872 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:00:52.922 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:00:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:00:53.003 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:00:53.223 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:00:53.759 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:00:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:00:53.841 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:00:53.873 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:00:53.900 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:00:54.027 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:54.063 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:00:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:00:54.176 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:00:54.814 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:54.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:54.869 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:54.981 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:00:55.069 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:00:55.111 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:00:55.195 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:00:55.224 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:00:55.224 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:55.299 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:55.332 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:55.333 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:55.409 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:55.486 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:55.739 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:00:55.789 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:00:55.798 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:00:55.830 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:00:55.862 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:00:55.889 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:00:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:00:55.937 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:00:56.020 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:00:56.070 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:00:56.073 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:00:56.073 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:56.119 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:00:56.148 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:00:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:56.286 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:56.359 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:56.749 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:56.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:56.807 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:57.643 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:00:57.733 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:00:57.774 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:00:57.856 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:00:57.888 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:00:57.888 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:57.974 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:58.110 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:58.198 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:01.364 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:01:01.365 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:01:01.366 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:01:01.366 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:01:01.366 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:01:01.366 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:01:01.367 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:01:01.368 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:01:01.369 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:01:01.370 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:01:01.371 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:01:01.372 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:01:01.373 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:01:01.374 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:01:01.375 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:01:01.376 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:01:01.377 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:01:02.257 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:01:02.258 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:01:02.258 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:01:02.258 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:01:02.258 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:01:02.258 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:01:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:01:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:01:02.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:01:02.262 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:01:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:01:02.265 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:01:02.266 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:01:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:01:02.268 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:01:03.516 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:03.516 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:01:03.516 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-04-24T23:01:03.516 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.517 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.518 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:01:03.519 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.520 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:01:03.521 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:01:03.522 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:01:03.523 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:01:03.524 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:01:03.525 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.526 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-24T23:01:03.527 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-24T23:01:03.920 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:03.997 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:03.997 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:01:03.997 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:01:03.997 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:01:03.997 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:01:03.997 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:01:03.998 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:01:03.999 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:01:04.000 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:01:04.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:01:04.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:01:04.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:01:04.014 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:01:04.015 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:01:04.016 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:01:04.017 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:01:04.018 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:01:04.018 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:01:04.018 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:01:04.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:01:04.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:01:04.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:01:04.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:01:04.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:01:04.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:01:04.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:01:04.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:01:04.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:01:04.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:01:04.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:01:04.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:01:04.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:01:04.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:01:04.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:01:04.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:01:04.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:01:04.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:01:04.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:01:04.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:01:04.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:01:04.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-24T23:01:04.398 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.399 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:04.400 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.401 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:01:04.402 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:01:04.404 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:01:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:01:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.407 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:01:04.408 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:01:04.409 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:01:04.410 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:01:04.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:01:04.410 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-24T23:01:04.751 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:06.085 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:06.086 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-24T23:01:06.086 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:01:06.086 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:01:06.086 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:01:06.086 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:01:06.086 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.087 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.087 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.087 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.087 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.087 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.087 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.088 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.088 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.088 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.088 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.088 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.090 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.090 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.090 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.090 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.090 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:06.090 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:01:06.091 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:01:06.092 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:01:06.093 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:01:06.094 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.095 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:01:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:01:06.097 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:01:06.098 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:01:06.099 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:01:06.100 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:01:06.101 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:01:06.102 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:01:06.102 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:01:06.102 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-24T23:01:06.462 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:06.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:06.609 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:01:06.632 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:06.633 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:06.633 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:01:06.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:06.774 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:01:06.801 INFO:teuthology.orchestra.run.smithi038.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:06.801 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:06.801 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:01:06.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:06.945 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:01:06.966 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:06.966 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:06.966 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:01:06.968 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:01:06.968 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:01:06.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:01:06.995 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:01:06.995 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:01:07.027 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:01:07.027 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:01:07.053 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:01:07.053 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:01:07.054 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:01:07.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:01:07.143 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:01:07.143 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:01:07.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:01:07.269 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:01:07.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:01:07.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:01:07.355 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:01:07.355 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:01:07.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:01:07.381 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:01:07.446 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:01:07.446 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:01:07.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:01:07.570 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:01:07.570 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:01:07.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:01:07.657 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:01:07.658 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:01:07.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:01:07.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:01:07.747 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:01:07.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:01:07.776 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:01:07.846 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:01:07.846 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:01:07.872 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:01:07.935 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T23:01:08.041 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-24T23:01:08.042 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:08.042 INFO:tasks.cephadm:Cluster fsid is 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:01:08.042 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T23:01:08.043 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.38', 'mon.c': '172.21.15.117'} 2024-04-24T23:01:08.043 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-04-24T23:01:08.043 INFO:tasks.cephadm:First mgr is x 2024-04-24T23:01:08.043 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T23:01:08.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-04-24T23:01:08.068 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-04-24T23:01:08.098 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-04-24T23:01:08.122 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T23:01:08.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:08.349 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T23:01:08.503 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:08.505 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-24T23:01:08.505 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:08.897 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:01 /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:08.897 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.415 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:01 /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.415 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.822 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:01 /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.823 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.842 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.863 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:09.895 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-24T23:01:09.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:01:09.898 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:01:09.906 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:01:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:01:11.480 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:01:11.603 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:01:56.773 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:01:56.773 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:01:56.773 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:01:56.773 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-04-24T23:01:56.773 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:01:56.774 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:01:56.774 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-24T23:02:00.218 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-24T23:02:00.241 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-24T23:02:00.242 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:02:00.242 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:02:00.242 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-04-24T23:02:00.242 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:02:00.242 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-04-24T23:02:00.242 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-24T23:02:00.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-04-24T23:02:00.301 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-04-24T23:02:00.329 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-04-24T23:02:00.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-04-24T23:02:00.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-04-24T23:02:00.407 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-04-24T23:02:00.430 INFO:tasks.cephadm:Writing seed config... 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T23:02:00.434 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T23:02:00.435 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T23:02:00.436 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T23:02:00.437 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-24T23:02:00.438 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:02:00.438 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T23:02:00.455 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8917d4f4-028e-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T23:02:00.456 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T23:02:00.498 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T23:02:00.541 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T23:02:00.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:02:00.747 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:00.747 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '8917d4f4-028e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-04-24T23:02:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T23:02:00.747 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-04-24T23:02:00.783 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:02:00.783 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-04-24T23:02:00.784 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-04-24T23:02:00.791 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:02:00.791 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:02:00.797 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:02:00.798 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-24T23:02:00.804 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-24T23:02:00.810 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-24T23:02:00.810 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:02:00.811 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-04-24T23:02:00.835 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:02:00.835 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T23:02:00.835 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-04-24T23:02:00.835 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-04-24T23:02:00.842 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:02:00.842 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:02:00.848 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:02:00.848 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-24T23:02:00.854 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-24T23:02:00.861 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-24T23:02:00.861 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:02:00.861 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-04-24T23:02:00.861 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:00.861 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140050310311456 on /run/cephadm/8917d4f4-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:02:00.861 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140050310311456 acquired on /run/cephadm/8917d4f4-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:02:00.862 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-04-24T23:02:00.862 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-04-24T23:02:00.862 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-24T23:02:00.865 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-04-24T23:02:00.865 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-04-24T23:02:00.868 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T23:02:00.868 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-24T23:02:00.868 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-24T23:02:00.870 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T23:02:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T23:02:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:02:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-24T23:02:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-24T23:02:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:02:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-24T23:02:00.872 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-24T23:02:00.872 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T23:02:00.872 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T23:02:00.873 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:02:01.324 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T23:02:02.228 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:02:02.228 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:02:02.228 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T23:02:03.214 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-04-24T23:02:03.214 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-04-24T23:02:04.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBrjylmDfzBIBAAFzPRCYjR5rDkS6DwMd2gvg== 2024-04-24T23:02:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBsjylmgooUIxAA26JhmI9kPDZz8e6paOx50A== 2024-04-24T23:02:06.192 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBtjylmfD8MIhAAnXxp563m+aII7tzAp9iGHg== 2024-04-24T23:02:06.193 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-04-24T23:02:07.324 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:02:07.325 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-04-24T23:02:08.283 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-04-24T23:02:08.495 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T23:02:08.652 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T23:02:08.830 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8917d4f4-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-8917d4f4-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:02:08.830 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8917d4f4-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-8917d4f4-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:02:09.090 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:02:09.090 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T23:02:09.271 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8917d4f4-028e-11ef-bc93-c7b262605968.target.wants/ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-8917d4f4-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:02:09.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 systemd[1]: Starting Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:02:09.588 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 podman[28807]: 2024-04-24 23:02:09.489572386 +0000 UTC m=+0.087650366 container create 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:02:09.588 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 podman[28807]: 2024-04-24 23:02:09.420573689 +0000 UTC m=+0.018651689 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:02:09.769 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:02:09.769 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-24T23:02:09.776 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:02:09.776 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-24T23:02:09.777 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-24T23:02:09.777 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:02:09.777 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-04-24T23:02:09.777 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-04-24T23:02:09.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 podman[28807]: 2024-04-24 23:02:09.676023994 +0000 UTC m=+0.274101985 container init 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:02:09.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 podman[28807]: 2024-04-24 23:02:09.679405784 +0000 UTC m=+0.277483768 container start 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T23:02:09.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 ceph-mon[28826]: mkfs 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:09.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 ceph-mon[28826]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:02:09.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 bash[28807]: 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 2024-04-24T23:02:09.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:09 smithi033 systemd[1]: Started Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:02:10.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: monmap epoch 1 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: last_changed 2024-04-24T23:02:06.759607+0000 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: min_mon_release 19 (squid) 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: election_strategy: 1 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: fsmap 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: mgrmap e1: no daemons active 2024-04-24T23:02:10.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:10 smithi033 ceph-mon[28826]: from='client.? 172.21.15.33:0/2092086380' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T23:02:10.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.590855s) 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T23:02:10.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T23:02:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.240+0000 7fa63fd99700 1 Processor -- start 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.241+0000 7fa63fd99700 1 -- start start 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.241+0000 7fa63fd99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa638108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.241+0000 7fa63fd99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381086d0 con 0x7fa638107ce0 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.241+0000 7fa63db35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa638108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.241+0000 7fa63db35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa638108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34418/0 (socket says 172.21.15.33:34418) 2024-04-24T23:02:10.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.241+0000 7fa63db35700 1 -- 172.21.15.33:0/3397302524 learned_addr learned my addr 172.21.15.33:0/3397302524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63db35700 1 -- 172.21.15.33:0/3397302524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa638108e40 con 0x7fa638107ce0 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63db35700 1 --2- 172.21.15.33:0/3397302524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa638108100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098a0 tx=0x7fa628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a25ade016e966f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63cb33700 1 -- 172.21.15.33:0/3397302524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628004160 con 0x7fa638107ce0 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63cb33700 1 -- 172.21.15.33:0/3397302524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628004300 con 0x7fa638107ce0 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63cb33700 1 -- 172.21.15.33:0/3397302524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6280036a0 con 0x7fa638107ce0 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63fd99700 1 -- 172.21.15.33:0/3397302524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 msgr2=0x7fa638108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:10.867 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63fd99700 1 --2- 172.21.15.33:0/3397302524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa638108100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098a0 tx=0x7fa628009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63fd99700 1 -- 172.21.15.33:0/3397302524 shutdown_connections 2024-04-24T23:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63fd99700 1 --2- 172.21.15.33:0/3397302524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa638108100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.242+0000 7fa63fd99700 1 -- 172.21.15.33:0/3397302524 >> 172.21.15.33:0/3397302524 conn(0x7fa638103300 msgr2=0x7fa638105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63fd99700 1 -- 172.21.15.33:0/3397302524 shutdown_connections 2024-04-24T23:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63fd99700 1 -- 172.21.15.33:0/3397302524 wait complete. 2024-04-24T23:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63fd99700 1 Processor -- start 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63fd99700 1 -- start start 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63fd99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa63807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63fd99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa638109370 con 0x7fa638107ce0 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63db35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa63807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63db35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa63807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34430/0 (socket says 172.21.15.33:34430) 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.243+0000 7fa63db35700 1 -- 172.21.15.33:0/2092086380 learned_addr learned my addr 172.21.15.33:0/2092086380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:10.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa63db35700 1 -- 172.21.15.33:0/2092086380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63807e030 con 0x7fa638107ce0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa63db35700 1 --2- 172.21.15.33:0/2092086380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa63807daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa628000c00 tx=0x7fa628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628003db0 con 0x7fa638107ce0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628011630 con 0x7fa638107ce0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63807c220 con 0x7fa638107ce0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628011950 con 0x7fa638107ce0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63807c760 con 0x7fa638107ce0 2024-04-24T23:02:10.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa628011af0 con 0x7fa638107ce0 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.244+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa62801c450 con 0x7fa638107ce0 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.245+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638063890 con 0x7fa638107ce0 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.248+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa628004260 con 0x7fa638107ce0 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.321+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa638194b60 con 0x7fa638107ce0 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.321+0000 7fa62effd700 1 -- 172.21.15.33:0/2092086380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa62801b450 con 0x7fa638107ce0 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 msgr2=0x7fa63807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:10.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 --2- 172.21.15.33:0/2092086380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa63807daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa628000c00 tx=0x7fa628004750 comp rx=0 tx=0).stop 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 shutdown_connections 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 --2- 172.21.15.33:0/2092086380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa638107ce0 0x7fa63807daf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 >> 172.21.15.33:0/2092086380 conn(0x7fa638103300 msgr2=0x7fa638191fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 shutdown_connections 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:10.322+0000 7fa63fd99700 1 -- 172.21.15.33:0/2092086380 wait complete. 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-04-24T23:02:10.872 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T23:02:11.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:11.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:02:11.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:11.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-24T23:02:11.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:02:11.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:02:11.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:02:11.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.266+0000 7f7726549700 1 Processor -- start 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.266+0000 7f7726549700 1 -- start start 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.266+0000 7f7726549700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772010aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.266+0000 7f7726549700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f772007be30 con 0x7f7720108700 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772010aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772010aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34436/0 (socket says 172.21.15.33:34436) 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771ffff700 1 -- 172.21.15.33:0/194542253 learned_addr learned my addr 172.21.15.33:0/194542253 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771ffff700 1 -- 172.21.15.33:0/194542253 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772010b030 con 0x7f7720108700 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771ffff700 1 --2- 172.21.15.33:0/194542253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772010aaf0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f77100098a0 tx=0x7f7710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d646f155be043125 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771f7fe700 1 -- 172.21.15.33:0/194542253 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7710004160 con 0x7f7720108700 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771f7fe700 1 -- 172.21.15.33:0/194542253 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7710004300 con 0x7f7720108700 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.267+0000 7f771f7fe700 1 -- 172.21.15.33:0/194542253 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77100036a0 con 0x7f7720108700 2024-04-24T23:02:11.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 -- 172.21.15.33:0/194542253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 msgr2=0x7f772010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 --2- 172.21.15.33:0/194542253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772010aaf0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f77100098a0 tx=0x7f7710009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 -- 172.21.15.33:0/194542253 shutdown_connections 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 --2- 172.21.15.33:0/194542253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772010aaf0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 -- 172.21.15.33:0/194542253 >> 172.21.15.33:0/194542253 conn(0x7f7720102130 msgr2=0x7f7720104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 -- 172.21.15.33:0/194542253 shutdown_connections 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.268+0000 7f7726549700 1 -- 172.21.15.33:0/194542253 wait complete. 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f7726549700 1 Processor -- start 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f7726549700 1 -- start start 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f7726549700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f7726549700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f772010b500 con 0x7f7720108700 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:11.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34450/0 (socket says 172.21.15.33:34450) 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771ffff700 1 -- 172.21.15.33:0/1313899883 learned_addr learned my addr 172.21.15.33:0/1313899883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771ffff700 1 -- 172.21.15.33:0/1313899883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772019daf0 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771ffff700 1 --2- 172.21.15.33:0/1313899883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772019d5b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7710000c00 tx=0x7f7710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7710003db0 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7710011630 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772019dd80 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7710011950 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.269+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77201a0a70 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.270+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f771001a460 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.270+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f771001b780 con 0x7f7720108700 2024-04-24T23:02:11.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.270+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7720063890 con 0x7f7720108700 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.273+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f771001ba20 con 0x7f7720108700 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.344+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f772002dd80 con 0x7f7720108700 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.347+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f771001bc00 con 0x7f7720108700 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.347+0000 7f771dffb700 1 -- 172.21.15.33:0/1313899883 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f771001bde0 con 0x7f7720108700 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 msgr2=0x7f772019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 --2- 172.21.15.33:0/1313899883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772019d5b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7710000c00 tx=0x7f7710004750 comp rx=0 tx=0).stop 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 shutdown_connections 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 --2- 172.21.15.33:0/1313899883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720108700 0x7f772019d5b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 >> 172.21.15.33:0/1313899883 conn(0x7f7720102130 msgr2=0x7f7720103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 shutdown_connections 2024-04-24T23:02:11.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:11.348+0000 7f7726549700 1 -- 172.21.15.33:0/1313899883 wait complete. 2024-04-24T23:02:11.888 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-04-24T23:02:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:11 smithi033 ceph-mon[28826]: from='client.? 172.21.15.33:0/1313899883' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:02:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:11 smithi033 ceph-mon[28826]: from='client.? 172.21.15.33:0/1313899883' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T23:02:12.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:12 smithi033 ceph-mon[28826]: from='client.? 172.21.15.33:0/4059229434' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:02:12.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.304+0000 7f26bb5cc700 1 Processor -- start 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.304+0000 7f26bb5cc700 1 -- start start 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.304+0000 7f26bb5cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b410ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.304+0000 7f26bb5cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26b407bdf0 con 0x7f26b4108740 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26b9368700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b410ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26b9368700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b410ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34466/0 (socket says 172.21.15.33:34466) 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26b9368700 1 -- 172.21.15.33:0/1061862528 learned_addr learned my addr 172.21.15.33:0/1061862528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26b9368700 1 -- 172.21.15.33:0/1061862528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b410b070 con 0x7f26b4108740 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26b9368700 1 --2- 172.21.15.33:0/1061862528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b410ab30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f26a80098a0 tx=0x7f26a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dab551cc016dea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26a7fff700 1 -- 172.21.15.33:0/1061862528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8004160 con 0x7f26b4108740 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26a7fff700 1 -- 172.21.15.33:0/1061862528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26a8004300 con 0x7f26b4108740 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.305+0000 7f26a7fff700 1 -- 172.21.15.33:0/1061862528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8011b50 con 0x7f26b4108740 2024-04-24T23:02:12.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 -- 172.21.15.33:0/1061862528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 msgr2=0x7f26b410ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 --2- 172.21.15.33:0/1061862528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b410ab30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f26a80098a0 tx=0x7f26a8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 -- 172.21.15.33:0/1061862528 shutdown_connections 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 --2- 172.21.15.33:0/1061862528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b410ab30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 -- 172.21.15.33:0/1061862528 >> 172.21.15.33:0/1061862528 conn(0x7f26b4102130 msgr2=0x7f26b4104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 -- 172.21.15.33:0/1061862528 shutdown_connections 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.306+0000 7f26bb5cc700 1 -- 172.21.15.33:0/1061862528 wait complete. 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26bb5cc700 1 Processor -- start 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26bb5cc700 1 -- start start 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26bb5cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26bb5cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26b410c090 con 0x7f26b4108740 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26b9368700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26b9368700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b419bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34472/0 (socket says 172.21.15.33:34472) 2024-04-24T23:02:12.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26b9368700 1 -- 172.21.15.33:0/4059229434 learned_addr learned my addr 172.21.15.33:0/4059229434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26b9368700 1 -- 172.21.15.33:0/4059229434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b419c180 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26b9368700 1 --2- 172.21.15.33:0/4059229434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b419bc40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009fb0 tx=0x7f26a8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a80038b0 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.307+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26a8011e50 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.308+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b419c410 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.308+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8019420 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.308+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b419f100 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.308+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f26a8019ae0 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.308+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f26a8024730 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.309+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b4063890 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.311+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f26a8003a50 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.382+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f26b419f2d0 con 0x7f26b4108740 2024-04-24T23:02:12.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.382+0000 7f26a67fc700 1 -- 172.21.15.33:0/4059229434 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f26a8032030 con 0x7f26b4108740 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 msgr2=0x7f26b419bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 --2- 172.21.15.33:0/4059229434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b419bc40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009fb0 tx=0x7f26a8011bf0 comp rx=0 tx=0).stop 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 shutdown_connections 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 --2- 172.21.15.33:0/4059229434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b4108740 0x7f26b419bc40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 >> 172.21.15.33:0/4059229434 conn(0x7f26b4102130 msgr2=0x7f26b4102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 shutdown_connections 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:12.383+0000 7f26bb5cc700 1 -- 172.21.15.33:0/4059229434 wait complete. 2024-04-24T23:02:12.920 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-04-24T23:02:13.180 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:12 smithi033 systemd[1]: Stopping Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:02:13.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:13 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a[28822]: 2024-04-24T23:02:13.177+0000 7f7808244700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:02:13.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:13 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a[28822]: 2024-04-24T23:02:13.177+0000 7f7808244700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T23:02:13.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:13 smithi033 podman[29072]: 2024-04-24 23:02:13.250357014 +0000 UTC m=+0.157792692 container died 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:02:13.888 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:13 smithi033 podman[29072]: 2024-04-24 23:02:13.603313239 +0000 UTC m=+0.510748949 container cleanup 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:02:13.888 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:13 smithi033 bash[29072]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:02:14.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:13 smithi033 podman[29086]: 2024-04-24 23:02:13.887389399 +0000 UTC m=+0.629867287 container remove 77569e3c7687c9b555d3ab397875fc064e8ef14ca292555e7a458ef357f75d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:02:14.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T23:02:14.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 systemd[1]: Stopped Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:02:14.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 systemd[1]: Starting Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:02:14.541 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 podman[29182]: 2024-04-24 23:02:14.154985071 +0000 UTC m=+0.018174764 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:02:14.541 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 podman[29182]: 2024-04-24 23:02:14.25499653 +0000 UTC m=+0.118186269 container create f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 podman[29182]: 2024-04-24 23:02:14.465371473 +0000 UTC m=+0.328561146 container init f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 podman[29182]: 2024-04-24 23:02:14.468595911 +0000 UTC m=+0.331785585 container start f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: pidfile_write: ignore empty --pid-file 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: load: jerasure load: lrc 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Git sha 0 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:02:14.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: DB SUMMARY 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: DB Session ID: V0DXK0KYP3VDUVRDXAPR 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104870 ; 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:02:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.env: 0x55eb58f4dc60 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.info_log: 0x55eb5b243aa0 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.statistics: (nil) 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.use_fsync: 0 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:02:14.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.db_log_dir: 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.wal_dir: 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:02:14.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.write_buffer_manager: 0x55eb5b2161e0 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:02:14.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:02:14.547 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.unordered_write: 0 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.row_cache: None 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.wal_filter: None 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:02:14.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:02:14.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.wal_compression: 0 2024-04-24T23:02:14.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:02:14.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:02:14.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:02:14.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_open_files: -1 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:02:14.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Compression algorithms supported: 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kZSTD supported: 0 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:02:14.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.merge_operator: 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_filter: None 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eb5b243ba0) 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:02:14.553 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x55eb5b247090 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-04-24T23:02:14.554 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-04-24T23:02:14.555 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression: NoCompression 2024-04-24T23:02:14.556 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.num_levels: 7 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:02:14.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:02:14.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:02:14.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:02:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:02:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:02:14.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:02:14.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.ttl: 2592000 2024-04-24T23:02:14.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:02:14.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 04ff638e-009e-4a66-946e-63c94b3fe218 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999734498120, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T23:02:14.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999734499129, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97893, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14195, "raw_average_key_size": 49, "raw_value_size": 90180, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999734, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "04ff638e-009e-4a66-946e-63c94b3fe218", "db_session_id": "V0DXK0KYP3VDUVRDXAPR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999734499222, "job": 1, "event": "recovery_finished"} 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eb5b338000 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: DB pointer 0x55eb5b322000 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:02:14.568 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 99.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 99.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-24T23:02:14.569 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:02:14.570 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 31.10 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 31.10 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x55eb5b247090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???) e1 preinit fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:14.571 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).mds e1 new map 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).mds e1 print_map 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 bash[29182]: f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 2024-04-24T23:02:14.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 systemd[1]: Started Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:02:14.573 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: monmap epoch 1 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: last_changed 2024-04-24T23:02:06.759607+0000 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: min_mon_release 19 (squid) 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: election_strategy: 1 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: fsmap 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:02:14.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:14 smithi033 ceph-mon[29201]: mgrmap e1: no daemons active 2024-04-24T23:02:15.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.940+0000 7f94d925c700 1 Processor -- start 2024-04-24T23:02:15.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.940+0000 7f94d925c700 1 -- start start 2024-04-24T23:02:15.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d925c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d4108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d925c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41086d0 con 0x7f94d4107ce0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d4108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d4108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34486/0 (socket says 172.21.15.33:34486) 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d2d9d700 1 -- 172.21.15.33:0/3827031532 learned_addr learned my addr 172.21.15.33:0/3827031532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d2d9d700 1 -- 172.21.15.33:0/3827031532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d4108e40 con 0x7f94d4107ce0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.941+0000 7f94d2d9d700 1 --2- 172.21.15.33:0/3827031532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d4108100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f94c80098a0 tx=0x7f94c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03f2e92671b81e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d1d9b700 1 -- 172.21.15.33:0/3827031532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94c8004160 con 0x7f94d4107ce0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d1d9b700 1 -- 172.21.15.33:0/3827031532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94c8004300 con 0x7f94d4107ce0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d1d9b700 1 -- 172.21.15.33:0/3827031532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94c8011b50 con 0x7f94d4107ce0 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 -- 172.21.15.33:0/3827031532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 msgr2=0x7f94d4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 --2- 172.21.15.33:0/3827031532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d4108100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f94c80098a0 tx=0x7f94c8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 -- 172.21.15.33:0/3827031532 shutdown_connections 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 --2- 172.21.15.33:0/3827031532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d4108100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 -- 172.21.15.33:0/3827031532 >> 172.21.15.33:0/3827031532 conn(0x7f94d4103300 msgr2=0x7f94d4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 -- 172.21.15.33:0/3827031532 shutdown_connections 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.942+0000 7f94d925c700 1 -- 172.21.15.33:0/3827031532 wait complete. 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.943+0000 7f94d925c700 1 Processor -- start 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.943+0000 7f94d925c700 1 -- start start 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.943+0000 7f94d925c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.943+0000 7f94d925c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410c1f0 con 0x7f94d4107ce0 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:15.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34500/0 (socket says 172.21.15.33:34500) 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d2d9d700 1 -- 172.21.15.33:0/3121006249 learned_addr learned my addr 172.21.15.33:0/3121006249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d2d9d700 1 -- 172.21.15.33:0/3121006249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d407e030 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d2d9d700 1 --2- 172.21.15.33:0/3121006249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d407daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f94c8011670 tx=0x7f94c8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94c80039c0 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d407c220 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94c8011e50 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94c801a5c0 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.944+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d407c760 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.945+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f94c801a8c0 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.945+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f94c80193c0 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.945+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d4063890 con 0x7f94d4107ce0 2024-04-24T23:02:15.558 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:14.948+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f94c802de10 con 0x7f94d4107ce0 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.018+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f94d407cc90 con 0x7f94d4107ce0 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94bbfff700 1 -- 172.21.15.33:0/3121006249 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f94c8032030 con 0x7f94d4107ce0 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 msgr2=0x7f94d407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94d925c700 1 --2- 172.21.15.33:0/3121006249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d407daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f94c8011670 tx=0x7f94c8011b60 comp rx=0 tx=0).stop 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 shutdown_connections 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94d925c700 1 --2- 172.21.15.33:0/3121006249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d4107ce0 0x7f94d407daf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 >> 172.21.15.33:0/3121006249 conn(0x7f94d4103300 msgr2=0x7f94d4103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.023+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 shutdown_connections 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:15.024+0000 7f94d925c700 1 -- 172.21.15.33:0/3121006249 wait complete. 2024-04-24T23:02:15.559 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T23:02:15.582 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:02:15.582 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-04-24T23:02:15.583 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T23:02:15.583 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T23:02:15.816 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:02:15.816 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T23:02:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8917d4f4-028e-11ef-bc93-c7b262605968.target.wants/ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-8917d4f4-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:02:16.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:16 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3121006249' entity='client.admin' 2024-04-24T23:02:16.455 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:16 smithi033 podman[29421]: 2024-04-24 23:02:16.413762418 +0000 UTC m=+0.325580494 container start a72910c86a2e39aa381be45306e68979d371d2abea5a82739611ef3e7cd512e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:02:16.502 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:02:16.502 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-24T23:02:16.510 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:02:16.510 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-24T23:02:16.510 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-24T23:02:16.510 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:02:16.518 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:02:16.518 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-24T23:02:16.526 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:02:16.526 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-24T23:02:16.526 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-24T23:02:16.527 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T23:02:16.527 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-04-24T23:02:16.527 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-04-24T23:02:16.763 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:16 smithi033 bash[29421]: a72910c86a2e39aa381be45306e68979d371d2abea5a82739611ef3e7cd512e2 2024-04-24T23:02:16.764 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:16 smithi033 systemd[1]: Started Ceph mgr.x for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:02:16.764 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:16 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:16.570+0000 7f503fef7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:02:16.764 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:16 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:16.761+0000 7f503fef7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:02:17.190 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:16 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:16.867+0000 7f503fef7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:02:17.190 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:17 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:17.188+0000 7f503fef7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:02:17.455 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:17 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:17.276+0000 7f503fef7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:02:18.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3722089267' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:02:18.910 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:18 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:18.545+0000 7f503fef7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:02:18.911 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:18 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:18.725+0000 7f503fef7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:02:18.911 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:18 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:18.908+0000 7f503fef7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:02:19.518 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:19.207+0000 7f503fef7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:02:19.518 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:19.302+0000 7f503fef7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:02:19.518 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:19.516+0000 7f503fef7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:02:19.955 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:19.715+0000 7f503fef7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:02:20.705 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:20 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:20.369+0000 7f503fef7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:02:21.114 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:20 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:20.806+0000 7f503fef7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:02:21.115 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:20 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:20.906+0000 7f503fef7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:02:21.115 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:21 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:21.113+0000 7f503fef7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:02:21.955 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:21 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:21.661+0000 7f503fef7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:02:21.955 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:21 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:21.756+0000 7f503fef7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:02:22.301 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:22 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:22.006+0000 7f503fef7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:02:22.301 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:22 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:22.144+0000 7f503fef7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:02:22.301 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:22 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:22.242+0000 7f503fef7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:02:22.705 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:22 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:22.361+0000 7f503fef7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:02:22.705 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:22 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:22.455+0000 7f503fef7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:02:23.581 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:23.301+0000 7f503fef7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:02:23.581 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:23.525+0000 7f503fef7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:02:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: Activating manager daemon x 2024-04-24T23:02:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: mgrmap e2: x(active, starting, since 0.00301706s) 2024-04-24T23:02:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:02:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:02:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: Manager daemon x is now available 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' 2024-04-24T23:02:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:23 smithi033 ceph-mon[29201]: from='mgr.14100 172.21.15.33:0/577502411' entity='mgr.x' 2024-04-24T23:02:25.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:25.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:02:25.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:02:25.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:02:25.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:02:25.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:02:25.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:02:09.733038+0000", 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.651+0000 7fdc57649700 1 Processor -- start 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc57649700 1 -- start start 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc57649700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc57649700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5007c870 con 0x7fdc5007de40 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc553e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc553e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41844/0 (socket says 172.21.15.33:41844) 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc553e5700 1 -- 172.21.15.33:0/3394735055 learned_addr learned my addr 172.21.15.33:0/3394735055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.652+0000 7fdc553e5700 1 -- 172.21.15.33:0/3394735055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5007c9f0 con 0x7fdc5007de40 2024-04-24T23:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.653+0000 7fdc553e5700 1 --2- 172.21.15.33:0/3394735055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5007c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0098a0 tx=0x7fdc4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb16533c8144edf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.653+0000 7fdc47fff700 1 -- 172.21.15.33:0/3394735055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4c004160 con 0x7fdc5007de40 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.653+0000 7fdc47fff700 1 -- 172.21.15.33:0/3394735055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc4c004300 con 0x7fdc5007de40 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.653+0000 7fdc47fff700 1 -- 172.21.15.33:0/3394735055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4c011b50 con 0x7fdc5007de40 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.653+0000 7fdc57649700 1 -- 172.21.15.33:0/3394735055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 msgr2=0x7fdc5007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.653+0000 7fdc57649700 1 --2- 172.21.15.33:0/3394735055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5007c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0098a0 tx=0x7fdc4c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.654+0000 7fdc57649700 1 -- 172.21.15.33:0/3394735055 shutdown_connections 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.654+0000 7fdc57649700 1 --2- 172.21.15.33:0/3394735055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5007c2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.654+0000 7fdc57649700 1 -- 172.21.15.33:0/3394735055 >> 172.21.15.33:0/3394735055 conn(0x7fdc50103210 msgr2=0x7fdc50105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.654+0000 7fdc57649700 1 -- 172.21.15.33:0/3394735055 shutdown_connections 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.654+0000 7fdc57649700 1 -- 172.21.15.33:0/3394735055 wait complete. 2024-04-24T23:02:25.130 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.654+0000 7fdc57649700 1 Processor -- start 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc57649700 1 -- start start 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc57649700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc57649700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5010e260 con 0x7fdc5007de40 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc553e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc553e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41858/0 (socket says 172.21.15.33:41858) 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc553e5700 1 -- 172.21.15.33:0/3722089267 learned_addr learned my addr 172.21.15.33:0/3722089267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc553e5700 1 -- 172.21.15.33:0/3722089267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5019e3d0 con 0x7fdc5007de40 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc553e5700 1 --2- 172.21.15.33:0/3722089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5019de90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009fb0 tx=0x7fdc4c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.655+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4c0038b0 con 0x7fdc5007de40 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.656+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc4c011e50 con 0x7fdc5007de40 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.656+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5019e660 con 0x7fdc5007de40 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.656+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4c019550 con 0x7fdc5007de40 2024-04-24T23:02:25.131 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.656+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501a1350 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.656+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc4c019b90 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.656+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc4c024730 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.657+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc50063890 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.659+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdc4c003a50 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.733+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdc501a1520 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.733+0000 7fdc467fc700 1 -- 172.21.15.33:0/3722089267 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdc4c0249d0 con 0x7fdc5007de40 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 msgr2=0x7fdc5019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 --2- 172.21.15.33:0/3722089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5019de90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009fb0 tx=0x7fdc4c011bf0 comp rx=0 tx=0).stop 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 shutdown_connections 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 --2- 172.21.15.33:0/3722089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc5007de40 0x7fdc5019de90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 >> 172.21.15.33:0/3722089267 conn(0x7fdc50103210 msgr2=0x7fdc50103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:25.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 shutdown_connections 2024-04-24T23:02:25.133 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:18.734+0000 7fdc57649700 1 -- 172.21.15.33:0/3722089267 wait complete. 2024-04-24T23:02:25.133 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-04-24T23:02:25.837 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:25 smithi033 ceph-mon[29201]: mgrmap e3: x(active, since 1.00732s) 2024-04-24T23:02:26.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:26 smithi033 ceph-mon[29201]: mgrmap e4: x(active, since 2s) 2024-04-24T23:02:28.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1736233477' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:02:29.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:02:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:02:29.079 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:02:29.080 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:02:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:02:09.733038+0000", 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.130+0000 7f1d6ec9e700 1 Processor -- start 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.130+0000 7f1d6ec9e700 1 -- start start 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.130+0000 7f1d6ec9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d681080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d6ec9e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68108670 con 0x7f1d68107c80 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d6ca3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d681080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d6ca3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d681080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60954/0 (socket says 172.21.15.33:60954) 2024-04-24T23:02:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d6ca3a700 1 -- 172.21.15.33:0/1447032009 learned_addr learned my addr 172.21.15.33:0/1447032009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d6ca3a700 1 -- 172.21.15.33:0/1447032009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d68108e30 con 0x7f1d68107c80 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d6ca3a700 1 --2- 172.21.15.33:0/1447032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d681080a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d640098a0 tx=0x7f1d64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a712e954fb5b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d5f7fe700 1 -- 172.21.15.33:0/1447032009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d64004160 con 0x7f1d68107c80 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d5f7fe700 1 -- 172.21.15.33:0/1447032009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d64004300 con 0x7f1d68107c80 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.131+0000 7f1d5f7fe700 1 -- 172.21.15.33:0/1447032009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d64011b50 con 0x7f1d68107c80 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1447032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 msgr2=0x7f1d681080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 --2- 172.21.15.33:0/1447032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d681080a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d640098a0 tx=0x7f1d64009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1447032009 shutdown_connections 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 --2- 172.21.15.33:0/1447032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d681080a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1447032009 >> 172.21.15.33:0/1447032009 conn(0x7f1d68103260 msgr2=0x7f1d68105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1447032009 shutdown_connections 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.132+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1447032009 wait complete. 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ec9e700 1 Processor -- start 2024-04-24T23:02:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ec9e700 1 -- start start 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ec9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d6807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ec9e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810c1e0 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ca3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d6807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ca3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d6807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60970/0 (socket says 172.21.15.33:60970) 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ca3a700 1 -- 172.21.15.33:0/1736233477 learned_addr learned my addr 172.21.15.33:0/1736233477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.133+0000 7f1d6ca3a700 1 -- 172.21.15.33:0/1736233477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6807e030 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.134+0000 7f1d6ca3a700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d6807daf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d64000c00 tx=0x7f1d64011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.134+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d64003b20 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.134+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d64003cc0 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.134+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6807c220 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.134+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d64024910 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.134+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6807c760 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.135+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1d64022020 con 0x7f1d68107c80 2024-04-24T23:02:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.135+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d68063890 con 0x7f1d68107c80 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.135+0000 7f1d5dffb700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1d50039050 0x7f1d5003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.135+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d64011e00 con 0x7f1d68107c80 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.135+0000 7f1d5ffff700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1d50039050 0x7f1d5003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.136+0000 7f1d5ffff700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1d50039050 0x7f1d5003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d58006fd0 tx=0x7f1d58006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.141+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d6404cbb0 con 0x7f1d68107c80 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.430+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1d6807c9e0 con 0x7f1d68107c80 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.431+0000 7f1d5dffb700 1 -- 172.21.15.33:0/1736233477 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1d6401acd0 con 0x7f1d68107c80 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.434+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1d50039050 msgr2=0x7f1d5003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:29.085 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1d50039050 0x7f1d5003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d58006fd0 tx=0x7f1d58006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 msgr2=0x7f1d6807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d6807daf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d64000c00 tx=0x7f1d64011c10 comp rx=0 tx=0).stop 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 shutdown_connections 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1d50039050 0x7f1d5003b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 --2- 172.21.15.33:0/1736233477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68107c80 0x7f1d6807daf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 >> 172.21.15.33:0/1736233477 conn(0x7f1d68103260 msgr2=0x7f1d68103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 shutdown_connections 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:28.435+0000 7f1d6ec9e700 1 -- 172.21.15.33:0/1736233477 wait complete. 2024-04-24T23:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-04-24T23:02:30.010 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:29 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3631086922' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:02:30.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:30.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:02:30.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:30.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:02:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f261a9da700 1 Processor -- start 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f261a9da700 1 -- start start 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f261a9da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f261a9da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f261407c7a0 con 0x7f261407dd60 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f2613fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f2613fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60984/0 (socket says 172.21.15.33:60984) 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.475+0000 7f2613fff700 1 -- 172.21.15.33:0/3334315335 learned_addr learned my addr 172.21.15.33:0/3334315335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.476+0000 7f2613fff700 1 -- 172.21.15.33:0/3334315335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261407c920 con 0x7f261407dd60 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.476+0000 7f2613fff700 1 --2- 172.21.15.33:0/3334315335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261407c260 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f26040098a0 tx=0x7f2604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51ab331c36dcd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.476+0000 7f2612ffd700 1 -- 172.21.15.33:0/3334315335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2604004160 con 0x7f261407dd60 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.476+0000 7f2612ffd700 1 -- 172.21.15.33:0/3334315335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2604004300 con 0x7f261407dd60 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.476+0000 7f2612ffd700 1 -- 172.21.15.33:0/3334315335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2604011b50 con 0x7f261407dd60 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 -- 172.21.15.33:0/3334315335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 msgr2=0x7f261407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 --2- 172.21.15.33:0/3334315335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261407c260 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f26040098a0 tx=0x7f2604009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 -- 172.21.15.33:0/3334315335 shutdown_connections 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 --2- 172.21.15.33:0/3334315335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261407c260 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 -- 172.21.15.33:0/3334315335 >> 172.21.15.33:0/3334315335 conn(0x7f2614103300 msgr2=0x7f2614105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 -- 172.21.15.33:0/3334315335 shutdown_connections 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 -- 172.21.15.33:0/3334315335 wait complete. 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 Processor -- start 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.477+0000 7f261a9da700 1 -- start start 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f261a9da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f261a9da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f261410e250 con 0x7f261407dd60 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f2613fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f2613fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60994/0 (socket says 172.21.15.33:60994) 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f2613fff700 1 -- 172.21.15.33:0/3631086922 learned_addr learned my addr 172.21.15.33:0/3631086922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f2613fff700 1 -- 172.21.15.33:0/3631086922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261419e2e0 con 0x7f261407dd60 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f2613fff700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261419dda0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2604011670 tx=0x7f2604011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26040039c0 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.478+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261419e570 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.479+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2604011e50 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.479+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260401a5c0 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.479+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a1260 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.479+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f260401a8c0 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.480+0000 7f26117fa700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f25fc038fb0 0x7f25fc03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.480+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614063890 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.480+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f260404d5c0 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.480+0000 7f26137fe700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f25fc038fb0 0x7f25fc03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.480+0000 7f26137fe700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f25fc038fb0 0x7f25fc03b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2608006fd0 tx=0x7f2608006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.485+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2604019c70 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.690+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f26141a1430 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.691+0000 7f26117fa700 1 -- 172.21.15.33:0/3631086922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f26141a1430 con 0x7f261407dd60 2024-04-24T23:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.693+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f25fc038fb0 msgr2=0x7f25fc03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.693+0000 7f261a9da700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f25fc038fb0 0x7f25fc03b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2608006fd0 tx=0x7f2608006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 msgr2=0x7f261419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261419dda0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2604011670 tx=0x7f2604011b60 comp rx=0 tx=0).stop 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 shutdown_connections 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f25fc038fb0 0x7f25fc03b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 --2- 172.21.15.33:0/3631086922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261407dd60 0x7f261419dda0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 >> 172.21.15.33:0/3631086922 conn(0x7f2614103300 msgr2=0x7f2614103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 shutdown_connections 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:29.694+0000 7f261a9da700 1 -- 172.21.15.33:0/3631086922 wait complete. 2024-04-24T23:02:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-04-24T23:02:31.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/182343901' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T23:02:31.368 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: ignoring --setuser ceph since I am not root 2024-04-24T23:02:31.368 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: ignoring --setgroup ceph since I am not root 2024-04-24T23:02:31.368 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:31.235+0000 7f951748d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:02:31.642 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:31.428+0000 7f951748d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:02:31.642 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:31.534+0000 7f951748d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:02:31.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.724+0000 7f27b3edb700 1 Processor -- start 2024-04-24T23:02:31.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b3edb700 1 -- start start 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b3edb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac10abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b3edb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac07bd70 con 0x7f27ac1087c0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b1c77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac10abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b1c77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac10abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32774/0 (socket says 172.21.15.33:32774) 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b1c77700 1 -- 172.21.15.33:0/2848260026 learned_addr learned my addr 172.21.15.33:0/2848260026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.725+0000 7f27b1c77700 1 -- 172.21.15.33:0/2848260026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac10b0f0 con 0x7f27ac1087c0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.726+0000 7f27b1c77700 1 --2- 172.21.15.33:0/2848260026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac10abb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098a0 tx=0x7f27a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608686e53debd873 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.726+0000 7f27b0c75700 1 -- 172.21.15.33:0/2848260026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a0004160 con 0x7f27ac1087c0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.726+0000 7f27b0c75700 1 -- 172.21.15.33:0/2848260026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f27a0004300 con 0x7f27ac1087c0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.726+0000 7f27b0c75700 1 -- 172.21.15.33:0/2848260026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a0011b50 con 0x7f27ac1087c0 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.726+0000 7f27b3edb700 1 -- 172.21.15.33:0/2848260026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 msgr2=0x7f27ac10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.726+0000 7f27b3edb700 1 --2- 172.21.15.33:0/2848260026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac10abb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098a0 tx=0x7f27a0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:31.651 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 -- 172.21.15.33:0/2848260026 shutdown_connections 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 --2- 172.21.15.33:0/2848260026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac10abb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 -- 172.21.15.33:0/2848260026 >> 172.21.15.33:0/2848260026 conn(0x7f27ac102130 msgr2=0x7f27ac104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 -- 172.21.15.33:0/2848260026 shutdown_connections 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 -- 172.21.15.33:0/2848260026 wait complete. 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 Processor -- start 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 -- start start 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.727+0000 7f27b3edb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac10c060 con 0x7f27ac1087c0 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b1c77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b1c77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32788/0 (socket says 172.21.15.33:32788) 2024-04-24T23:02:31.652 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b1c77700 1 -- 172.21.15.33:0/182343901 learned_addr learned my addr 172.21.15.33:0/182343901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b1c77700 1 -- 172.21.15.33:0/182343901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac19c180 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b1c77700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac19bc40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009fb0 tx=0x7f27a0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a00038b0 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f27a0011e50 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ac19c410 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a0019420 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.728+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ac19f100 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.729+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f27a0019750 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.729+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ac063890 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.729+0000 7f279effd700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f2798039000 0x7f279803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.729+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f27a004d3d0 con 0x7f27ac1087c0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.729+0000 7f27b1476700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f2798039000 0x7f279803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.730+0000 7f27b1476700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f2798039000 0x7f279803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27a8006fd0 tx=0x7f27a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:31.653 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.734+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f27a000bc70 con 0x7f27ac1087c0 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:30.999+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f27ac19f2d0 con 0x7f27ac1087c0 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.051+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f27ac19f2d0 con 0x7f27ac1087c0 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.052+0000 7f279effd700 1 -- 172.21.15.33:0/182343901 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f27a002d7c0 con 0x7f27ac1087c0 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f2798039000 msgr2=0x7f279803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f2798039000 0x7f279803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27a8006fd0 tx=0x7f27a8006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 msgr2=0x7f27ac19bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac19bc40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009fb0 tx=0x7f27a0011bf0 comp rx=0 tx=0).stop 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 shutdown_connections 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f2798039000 0x7f279803b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 --2- 172.21.15.33:0/182343901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27ac1087c0 0x7f27ac19bc40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 >> 172.21.15.33:0/182343901 conn(0x7f27ac102130 msgr2=0x7f27ac102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 shutdown_connections 2024-04-24T23:02:31.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:31.054+0000 7f27b3edb700 1 -- 172.21.15.33:0/182343901 wait complete. 2024-04-24T23:02:31.916 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:31.862+0000 7f951748d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:02:32.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:31.955+0000 7f951748d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:02:32.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:32 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/182343901' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T23:02:32.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:32 smithi033 ceph-mon[29201]: mgrmap e5: x(active, since 7s) 2024-04-24T23:02:33.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:33 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1839387401' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:02:33.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:02:33.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T23:02:33.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:02:33.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:02:33.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.120+0000 7efd4f18c700 1 Processor -- start 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.120+0000 7efd4f18c700 1 -- start start 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.120+0000 7efd4f18c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd481063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.120+0000 7efd4f18c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd48106970 con 0x7efd48105f80 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.120+0000 7efd4cf28700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd481063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd4cf28700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd481063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32816/0 (socket says 172.21.15.33:32816) 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd4cf28700 1 -- 172.21.15.33:0/904451297 learned_addr learned my addr 172.21.15.33:0/904451297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd4cf28700 1 -- 172.21.15.33:0/904451297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd48108f10 con 0x7efd48105f80 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd4cf28700 1 --2- 172.21.15.33:0/904451297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd481063a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0098a0 tx=0x7efd3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356b3d89d17e21fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd477fe700 1 -- 172.21.15.33:0/904451297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd3c004160 con 0x7efd48105f80 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd477fe700 1 -- 172.21.15.33:0/904451297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd3c004300 con 0x7efd48105f80 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.121+0000 7efd477fe700 1 -- 172.21.15.33:0/904451297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd3c011b50 con 0x7efd48105f80 2024-04-24T23:02:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 -- 172.21.15.33:0/904451297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 msgr2=0x7efd481063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 --2- 172.21.15.33:0/904451297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd481063a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0098a0 tx=0x7efd3c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 -- 172.21.15.33:0/904451297 shutdown_connections 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 --2- 172.21.15.33:0/904451297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd481063a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 -- 172.21.15.33:0/904451297 >> 172.21.15.33:0/904451297 conn(0x7efd481018f0 msgr2=0x7efd48103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 -- 172.21.15.33:0/904451297 shutdown_connections 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.122+0000 7efd4f18c700 1 -- 172.21.15.33:0/904451297 wait complete. 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4f18c700 1 Processor -- start 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4f18c700 1 -- start start 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4f18c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd4819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4f18c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810c2c0 con 0x7efd48105f80 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4cf28700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd4819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4cf28700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd4819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32818/0 (socket says 172.21.15.33:32818) 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4cf28700 1 -- 172.21.15.33:0/1839387401 learned_addr learned my addr 172.21.15.33:0/1839387401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.123+0000 7efd4cf28700 1 -- 172.21.15.33:0/1839387401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4819c1b0 con 0x7efd48105f80 2024-04-24T23:02:33.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.124+0000 7efd4cf28700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd4819bc70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009fb0 tx=0x7efd3c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.124+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd3c0038b0 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.124+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd3c011e50 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.124+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd3c019550 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.124+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4819c440 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.124+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4819f130 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.125+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7efd3c019850 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.125+0000 7efd45ffb700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 0x7efd3803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.125+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd3c04d6c0 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.125+0000 7efd47fff700 1 -- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 msgr2=0x7efd3803b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.125+0000 7efd47fff700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 0x7efd3803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.125+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd48063890 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.130+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd3c04d0e0 con 0x7efd48105f80 2024-04-24T23:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.326+0000 7efd47fff700 1 -- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 msgr2=0x7efd3803b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.326+0000 7efd47fff700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 0x7efd3803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.488+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7efd4819f300 con 0x7efd48105f80 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.488+0000 7efd45ffb700 1 -- 172.21.15.33:0/1839387401 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7efd3c023c90 con 0x7efd48105f80 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 msgr2=0x7efd3803b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 0x7efd3803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 msgr2=0x7efd4819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd4819bc70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009fb0 tx=0x7efd3c011bf0 comp rx=0 tx=0).stop 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 shutdown_connections 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7efd38039000 0x7efd3803b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 --2- 172.21.15.33:0/1839387401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd48105f80 0x7efd4819bc70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 >> 172.21.15.33:0/1839387401 conn(0x7efd481018f0 msgr2=0x7efd481023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 shutdown_connections 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:32.491+0000 7efd4f18c700 1 -- 172.21.15.33:0/1839387401 wait complete. 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-24T23:02:33.219 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-04-24T23:02:33.475 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:33.251+0000 7f951748d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:02:33.476 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:33.429+0000 7f951748d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:02:33.858 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:33.609+0000 7f951748d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:02:34.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:33.913+0000 7f951748d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:02:34.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:34.009+0000 7f951748d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:02:34.705 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:34.220+0000 7f951748d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:02:34.705 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:34.420+0000 7f951748d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:02:35.310 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:35.049+0000 7f951748d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:02:35.592 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:35.490+0000 7f951748d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:02:35.593 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:35.590+0000 7f951748d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:02:35.903 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:35.798+0000 7f951748d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:02:36.662 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:36.349+0000 7f951748d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:02:36.662 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:36.444+0000 7f951748d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:02:36.936 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:36.696+0000 7f951748d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:02:36.936 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:36.835+0000 7f951748d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:02:36.936 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:36.934+0000 7f951748d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:02:37.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:37.053+0000 7f951748d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:02:37.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:37.148+0000 7f951748d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:02:38.269 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:37.993+0000 7f951748d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:02:38.269 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:38.214+0000 7f951748d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: Active manager daemon x restarted 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: Activating manager daemon x 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: mgrmap e6: x(active, starting, since 0.00385113s) 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:02:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: Manager daemon x is now available 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:38 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:39.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:39 smithi033 ceph-mon[29201]: Found migration_current of "None". Setting to last migration. 2024-04-24T23:02:39.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:39 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:02:39.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:39 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:02:39.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:39 smithi033 ceph-mon[29201]: mgrmap e7: x(active, since 1.00532s) 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.651+0000 7f148d4d4700 1 Processor -- start 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.651+0000 7f148d4d4700 1 -- start start 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.651+0000 7f148d4d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f14881069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f148d4d4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f148807bd90 con 0x7f14881065b0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1486ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f14881069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1486ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f14881069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32826/0 (socket says 172.21.15.33:32826) 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1486ffd700 1 -- 172.21.15.33:0/3934566549 learned_addr learned my addr 172.21.15.33:0/3934566549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1486ffd700 1 -- 172.21.15.33:0/3934566549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1488106f10 con 0x7f14881065b0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1486ffd700 1 --2- 172.21.15.33:0/3934566549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f14881069d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f14780098a0 tx=0x7f1478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b1d09a5126b588 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1485ffb700 1 -- 172.21.15.33:0/3934566549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1478004160 con 0x7f14881065b0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1485ffb700 1 -- 172.21.15.33:0/3934566549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1478004300 con 0x7f14881065b0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.652+0000 7f1485ffb700 1 -- 172.21.15.33:0/3934566549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1478011b50 con 0x7f14881065b0 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 -- 172.21.15.33:0/3934566549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 msgr2=0x7f14881069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 --2- 172.21.15.33:0/3934566549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f14881069d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f14780098a0 tx=0x7f1478009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 -- 172.21.15.33:0/3934566549 shutdown_connections 2024-04-24T23:02:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 --2- 172.21.15.33:0/3934566549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f14881069d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 -- 172.21.15.33:0/3934566549 >> 172.21.15.33:0/3934566549 conn(0x7f1488102170 msgr2=0x7f1488104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 -- 172.21.15.33:0/3934566549 shutdown_connections 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.653+0000 7f148d4d4700 1 -- 172.21.15.33:0/3934566549 wait complete. 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f148d4d4700 1 Processor -- start 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f148d4d4700 1 -- start start 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f148d4d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f148819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f148d4d4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1488107fa0 con 0x7f14881065b0 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f1486ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f148819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f1486ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f148819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32830/0 (socket says 172.21.15.33:32830) 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f1486ffd700 1 -- 172.21.15.33:0/4150063060 learned_addr learned my addr 172.21.15.33:0/4150063060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.654+0000 7f1486ffd700 1 -- 172.21.15.33:0/4150063060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819e390 con 0x7f14881065b0 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.655+0000 7f1486ffd700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f148819de50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1478011b60 tx=0x7f1478011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.655+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1478003970 con 0x7f14881065b0 2024-04-24T23:02:39.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.655+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819e620 con 0x7f14881065b0 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.655+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1478003b10 con 0x7f14881065b0 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.655+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1478024610 con 0x7f14881065b0 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.655+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a1310 con 0x7f14881065b0 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.656+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1478024910 con 0x7f14881065b0 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.656+0000 7f147ffff700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.656+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 --> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1488063890 con 0x7f1474039050 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.656+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1478011e00 con 0x7f14881065b0 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.656+0000 7f14867fc700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 msgr2=0x7f147403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.656+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.857+0000 7f14867fc700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 msgr2=0x7f147403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:33.857+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:34.258+0000 7f14867fc700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 msgr2=0x7f147403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:39.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:34.258+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:35.059+0000 7f14867fc700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 msgr2=0x7f147403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:35.059+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:36.660+0000 7f14867fc700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 msgr2=0x7f147403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1299073362 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:36.660+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:38.218+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f147802d910 con 0x7f14881065b0 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:38.218+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 msgr2=0x7f147403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:38.218+0000 7f147ffff700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.218+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f147804ecf0 con 0x7f14881065b0 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.218+0000 7f147ffff700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f147403c790 0x7f147403eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.218+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f147801bec0 con 0x7f147403c790 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.219+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f147403c790 0x7f147403eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.219+0000 7f14867fc700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f147403c790 0x7f147403eb80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1470003a10 tx=0x7f1470009290 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.220+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f147801bec0 con 0x7f147403c790 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1488062670 con 0x7f147403c790 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f147ffff700 1 -- 172.21.15.33:0/4150063060 <== mgr.14116 v2:172.21.15.33:6800/2813273186 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f1488062670 con 0x7f147403c790 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f147403c790 msgr2=0x7f147403eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f147403c790 0x7f147403eb80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1470003a10 tx=0x7f1470009290 comp rx=0 tx=0).stop 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 msgr2=0x7f148819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f148819de50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1478011b60 tx=0x7f1478011c40 comp rx=0 tx=0).stop 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 shutdown_connections 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f147403c790 0x7f147403eb80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881065b0 0x7f148819de50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 --2- 172.21.15.33:0/4150063060 >> [v2:172.21.15.33:6800/1299073362,v1:172.21.15.33:6801/1299073362] conn(0x7f1474039050 0x7f147403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.227+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 >> 172.21.15.33:0/4150063060 conn(0x7f1488102170 msgr2=0x7f1488194ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.228+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 shutdown_connections 2024-04-24T23:02:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:39.228+0000 7f148d4d4700 1 -- 172.21.15.33:0/4150063060 wait complete. 2024-04-24T23:02:39.779 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-04-24T23:02:39.779 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-04-24T23:02:40.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:40 smithi033 ceph-mon[29201]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:02:40.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:40 smithi033 ceph-mon[29201]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:02:40.993 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.182+0000 7fcb13f45700 1 Processor -- start 2024-04-24T23:02:40.993 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.182+0000 7fcb13f45700 1 -- start start 2024-04-24T23:02:40.993 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb13f45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c10aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.993 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb13f45700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c07be30 con 0x7fcb0c108700 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb11ce1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c10aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb11ce1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c10aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50744/0 (socket says 172.21.15.33:50744) 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb11ce1700 1 -- 172.21.15.33:0/2925583826 learned_addr learned my addr 172.21.15.33:0/2925583826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb11ce1700 1 -- 172.21.15.33:0/2925583826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c10b030 con 0x7fcb0c108700 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb11ce1700 1 --2- 172.21.15.33:0/2925583826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c10aaf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb080098a0 tx=0x7fcb08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0db31fd09f450db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.183+0000 7fcb10cdf700 1 -- 172.21.15.33:0/2925583826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08004160 con 0x7fcb0c108700 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb10cdf700 1 -- 172.21.15.33:0/2925583826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb08004300 con 0x7fcb0c108700 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb10cdf700 1 -- 172.21.15.33:0/2925583826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08011b50 con 0x7fcb0c108700 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 -- 172.21.15.33:0/2925583826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 msgr2=0x7fcb0c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 --2- 172.21.15.33:0/2925583826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c10aaf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb080098a0 tx=0x7fcb08009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 -- 172.21.15.33:0/2925583826 shutdown_connections 2024-04-24T23:02:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 --2- 172.21.15.33:0/2925583826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c10aaf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 -- 172.21.15.33:0/2925583826 >> 172.21.15.33:0/2925583826 conn(0x7fcb0c102110 msgr2=0x7fcb0c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 -- 172.21.15.33:0/2925583826 shutdown_connections 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.184+0000 7fcb13f45700 1 -- 172.21.15.33:0/2925583826 wait complete. 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb13f45700 1 Processor -- start 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb13f45700 1 -- start start 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb13f45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c107f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb13f45700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c10c060 con 0x7fcb0c108700 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb11ce1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb11ce1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50752/0 (socket says 172.21.15.33:50752) 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.185+0000 7fcb11ce1700 1 -- 172.21.15.33:0/4074323679 learned_addr learned my addr 172.21.15.33:0/4074323679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcb11ce1700 1 -- 172.21.15.33:0/4074323679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1084c0 con 0x7fcb0c108700 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcb11ce1700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c107f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fb0 tx=0x7fcb08011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb080038b0 con 0x7fcb0c108700 2024-04-24T23:02:40.995 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb08011e50 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1066b0 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08019420 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.186+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c106bf0 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.187+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c063890 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.187+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fcb08003a50 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.187+0000 7fcafeffd700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fcaf8038f40 0x7fcaf803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.187+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb0804d0b0 con 0x7fcb0c108700 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.187+0000 7fcb114e0700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fcaf8038f40 0x7fcaf803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.188+0000 7fcb114e0700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fcaf8038f40 0x7fcaf803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb00006fd0 tx=0x7fcb00006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.192+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcb0800fda0 con 0x7fcb0c108700 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.270+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcb080196f0 con 0x7fcb0c108700 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.419+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcb0c107450 con 0x7fcaf8038f40 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.424+0000 7fcafeffd700 1 -- 172.21.15.33:0/4074323679 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb0c107450 con 0x7fcaf8038f40 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fcaf8038f40 msgr2=0x7fcaf803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fcaf8038f40 0x7fcaf803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb00006fd0 tx=0x7fcb00006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 msgr2=0x7fcb0c107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c107f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fb0 tx=0x7fcb08011bf0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 shutdown_connections 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fcaf8038f40 0x7fcaf803b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 --2- 172.21.15.33:0/4074323679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb0c108700 0x7fcb0c107f80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.427+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 >> 172.21.15.33:0/4074323679 conn(0x7fcb0c102110 msgr2=0x7fcb0c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.428+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 shutdown_connections 2024-04-24T23:02:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:40.428+0000 7fcb13f45700 1 -- 172.21.15.33:0/4074323679 wait complete. 2024-04-24T23:02:41.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:41 smithi033 ceph-mon[29201]: mgrmap e8: x(active, since 2s) 2024-04-24T23:02:41.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:41 smithi033 ceph-mon[29201]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:41.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:41 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:41.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:41 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:41.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:41 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:41.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:41 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.400+0000 7f6f19267700 1 Processor -- start 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f19267700 1 -- start start 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f19267700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f141072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f19267700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f14107880 con 0x7f6f14106e90 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f141072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f141072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50756/0 (socket says 172.21.15.33:50756) 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f12ffd700 1 -- 172.21.15.33:0/101022558 learned_addr learned my addr 172.21.15.33:0/101022558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f12ffd700 1 -- 172.21.15.33:0/101022558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f14107a00 con 0x7f6f14106e90 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.401+0000 7f6f12ffd700 1 --2- 172.21.15.33:0/101022558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f141072b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f040098a0 tx=0x7f6f04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1e940d469d26ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.224 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f11ffb700 1 -- 172.21.15.33:0/101022558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f04004160 con 0x7f6f14106e90 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f11ffb700 1 -- 172.21.15.33:0/101022558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f04004300 con 0x7f6f14106e90 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f11ffb700 1 -- 172.21.15.33:0/101022558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f04011b50 con 0x7f6f14106e90 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 -- 172.21.15.33:0/101022558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 msgr2=0x7f6f141072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 --2- 172.21.15.33:0/101022558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f141072b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f040098a0 tx=0x7f6f04009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 -- 172.21.15.33:0/101022558 shutdown_connections 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 --2- 172.21.15.33:0/101022558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f141072b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 -- 172.21.15.33:0/101022558 >> 172.21.15.33:0/101022558 conn(0x7f6f14101490 msgr2=0x7f6f141038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 -- 172.21.15.33:0/101022558 shutdown_connections 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.402+0000 7f6f19267700 1 -- 172.21.15.33:0/101022558 wait complete. 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f19267700 1 Processor -- start 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f19267700 1 -- start start 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f19267700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f1419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f19267700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141067b0 con 0x7f6f14106e90 2024-04-24T23:02:42.225 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f1419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f1419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50764/0 (socket says 172.21.15.33:50764) 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.403+0000 7f6f12ffd700 1 -- 172.21.15.33:0/2247801464 learned_addr learned my addr 172.21.15.33:0/2247801464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6f12ffd700 1 -- 172.21.15.33:0/2247801464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1419e340 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6f12ffd700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f1419de00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f04011b60 tx=0x7f6f04011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f04003970 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f04003b10 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1419e630 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f04024610 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.404+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f141a12c0 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.405+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6f040248c0 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.405+0000 7f6efbfff700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6efc0390a0 0x7f6efc03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.405+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f14063890 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.405+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6f04011e00 con 0x7f6f14106e90 2024-04-24T23:02:42.226 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.405+0000 7f6f127fc700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6efc0390a0 0x7f6efc03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.406+0000 7f6f127fc700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6efc0390a0 0x7f6efc03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006fd0 tx=0x7f6f08006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.411+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f04029de0 con 0x7f6f14106e90 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.618+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6f14062670 con 0x7f6efc0390a0 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.619+0000 7f6efbfff700 1 -- 172.21.15.33:0/2247801464 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6f14062670 con 0x7f6efc0390a0 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6efc0390a0 msgr2=0x7f6efc03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6efc0390a0 0x7f6efc03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006fd0 tx=0x7f6f08006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 msgr2=0x7f6f1419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f1419de00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f04011b60 tx=0x7f6f04011c40 comp rx=0 tx=0).stop 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 shutdown_connections 2024-04-24T23:02:42.227 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6efc0390a0 0x7f6efc03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.228 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 --2- 172.21.15.33:0/2247801464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f14106e90 0x7f6f1419de00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.228 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.621+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 >> 172.21.15.33:0/2247801464 conn(0x7f6f14101490 msgr2=0x7f6f14194990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.228 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.622+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 shutdown_connections 2024-04-24T23:02:42.228 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:41.622+0000 7f6f19267700 1 -- 172.21.15.33:0/2247801464 wait complete. 2024-04-24T23:02:42.228 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-04-24T23:02:42.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:42 smithi033 ceph-mon[29201]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:43.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: [24/Apr/2024:23:02:42] ENGINE Bus STARTING 2024-04-24T23:02:43.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: [24/Apr/2024:23:02:42] ENGINE Serving on http://172.21.15.33:8765 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: [24/Apr/2024:23:02:42] ENGINE Serving on https://172.21.15.33:7150 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: [24/Apr/2024:23:02:42] ENGINE Bus STARTED 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-mon[29201]: Generating ssh key... 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: Generating public/private rsa key pair. 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: Your identification has been saved in /tmp/tmpjt27pqou/key. 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: Your public key has been saved in /tmp/tmpjt27pqou/key.pub. 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: The key fingerprint is: 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: SHA256:HfSmgxwEizoFT3WZMPJ5Zmh9goZARaxa2nCRebtfx7A ceph-8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:43.623 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: The key's randomart image is: 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: +---[RSA 3072]----+ 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | .+*=.=ooo. | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | +=o= Xo. . | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | +=.O B o o | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: |. +o.o =.* + | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | Bo . S++ | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: |o ... E o. | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | . . . | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | . | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: | | 2024-04-24T23:02:43.624 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: +----[SHA256]-----+ 2024-04-24T23:02:43.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092cadd700 1 Processor -- start 2024-04-24T23:02:43.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092cadd700 1 -- start start 2024-04-24T23:02:43.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092cadd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f09281080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092cadd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0928108690 con 0x7f0928107ca0 2024-04-24T23:02:43.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f09281080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f09281080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50766/0 (socket says 172.21.15.33:50766) 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092659c700 1 -- 172.21.15.33:0/1523934666 learned_addr learned my addr 172.21.15.33:0/1523934666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.694+0000 7f092659c700 1 -- 172.21.15.33:0/1523934666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0928108e50 con 0x7f0928107ca0 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092659c700 1 --2- 172.21.15.33:0/1523934666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f09281080c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09180098a0 tx=0x7f0918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a064213d9a10ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092559a700 1 -- 172.21.15.33:0/1523934666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0918004160 con 0x7f0928107ca0 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092559a700 1 -- 172.21.15.33:0/1523934666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0918004300 con 0x7f0928107ca0 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092559a700 1 -- 172.21.15.33:0/1523934666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0918011b50 con 0x7f0928107ca0 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092cadd700 1 -- 172.21.15.33:0/1523934666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 msgr2=0x7f09281080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092cadd700 1 --2- 172.21.15.33:0/1523934666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f09281080c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09180098a0 tx=0x7f0918009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092cadd700 1 -- 172.21.15.33:0/1523934666 shutdown_connections 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092cadd700 1 --2- 172.21.15.33:0/1523934666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f09281080c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092cadd700 1 -- 172.21.15.33:0/1523934666 >> 172.21.15.33:0/1523934666 conn(0x7f09281032a0 msgr2=0x7f09281056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.695+0000 7f092cadd700 1 -- 172.21.15.33:0/1523934666 shutdown_connections 2024-04-24T23:02:43.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.696+0000 7f092cadd700 1 -- 172.21.15.33:0/1523934666 wait complete. 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.696+0000 7f092cadd700 1 Processor -- start 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.696+0000 7f092cadd700 1 -- start start 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.696+0000 7f092cadd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f092819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.696+0000 7f092cadd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f092810c200 con 0x7f0928107ca0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f092819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f092819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50768/0 (socket says 172.21.15.33:50768) 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092659c700 1 -- 172.21.15.33:0/4285200124 learned_addr learned my addr 172.21.15.33:0/4285200124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092659c700 1 -- 172.21.15.33:0/4285200124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092819c1d0 con 0x7f0928107ca0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092659c700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f092819bc90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0918011850 tx=0x7f0918011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0918003af0 con 0x7f0928107ca0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0918020070 con 0x7f0928107ca0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09180195a0 con 0x7f0928107ca0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092819c460 con 0x7f0928107ca0 2024-04-24T23:02:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.697+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092819f150 con 0x7f0928107ca0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.698+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f09180198a0 con 0x7f0928107ca0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.698+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0928063890 con 0x7f0928107ca0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.698+0000 7f09177fe700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f0910039050 0x7f091003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.698+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0918011e00 con 0x7f0928107ca0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.698+0000 7f0925d9b700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f0910039050 0x7f091003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.699+0000 7f0925d9b700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f0910039050 0x7f091003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f091c006fd0 tx=0x7f091c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.703+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f091801a430 con 0x7f0928107ca0 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:42.910+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0928104560 con 0x7f0910039050 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.315+0000 7f09177fe700 1 -- 172.21.15.33:0/4285200124 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0928104560 con 0x7f0910039050 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f0910039050 msgr2=0x7f091003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f0910039050 0x7f091003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f091c006fd0 tx=0x7f091c006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 msgr2=0x7f092819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f092819bc90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0918011850 tx=0x7f0918011c10 comp rx=0 tx=0).stop 2024-04-24T23:02:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 shutdown_connections 2024-04-24T23:02:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f0910039050 0x7f091003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 --2- 172.21.15.33:0/4285200124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0928107ca0 0x7f092819bc90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 >> 172.21.15.33:0/4285200124 conn(0x7f09281032a0 msgr2=0x7f0928103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 shutdown_connections 2024-04-24T23:02:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:43.317+0000 7f092cadd700 1 -- 172.21.15.33:0/4285200124 wait complete. 2024-04-24T23:02:44.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:44 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:44.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:44 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:45.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95daf69700 1 Processor -- start 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95daf69700 1 -- start start 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95daf69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d4108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95daf69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41086d0 con 0x7f95d4107ce0 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95d8d05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d4108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95d8d05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d4108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50778/0 (socket says 172.21.15.33:50778) 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.379+0000 7f95d8d05700 1 -- 172.21.15.33:0/3633455105 learned_addr learned my addr 172.21.15.33:0/3633455105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:45.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95d8d05700 1 -- 172.21.15.33:0/3633455105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d4108e40 con 0x7f95d4107ce0 2024-04-24T23:02:45.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95d8d05700 1 --2- 172.21.15.33:0/3633455105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d4108100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098a0 tx=0x7f95c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0f81958be2aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:45.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95d37fe700 1 -- 172.21.15.33:0/3633455105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c8004160 con 0x7f95d4107ce0 2024-04-24T23:02:45.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95d37fe700 1 -- 172.21.15.33:0/3633455105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95c8004300 con 0x7f95d4107ce0 2024-04-24T23:02:45.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95d37fe700 1 -- 172.21.15.33:0/3633455105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c8011b50 con 0x7f95d4107ce0 2024-04-24T23:02:45.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95daf69700 1 -- 172.21.15.33:0/3633455105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 msgr2=0x7f95d4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95daf69700 1 --2- 172.21.15.33:0/3633455105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d4108100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098a0 tx=0x7f95c8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.380+0000 7f95daf69700 1 -- 172.21.15.33:0/3633455105 shutdown_connections 2024-04-24T23:02:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 --2- 172.21.15.33:0/3633455105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d4108100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 -- 172.21.15.33:0/3633455105 >> 172.21.15.33:0/3633455105 conn(0x7f95d4103300 msgr2=0x7f95d4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 -- 172.21.15.33:0/3633455105 shutdown_connections 2024-04-24T23:02:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 -- 172.21.15.33:0/3633455105 wait complete. 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 Processor -- start 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 -- start start 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.381+0000 7f95daf69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95d410c1f0 con 0x7f95d4107ce0 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d8d05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d8d05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50780/0 (socket says 172.21.15.33:50780) 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d8d05700 1 -- 172.21.15.33:0/2207392689 learned_addr learned my addr 172.21.15.33:0/2207392689 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d8d05700 1 -- 172.21.15.33:0/2207392689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d419c170 con 0x7f95d4107ce0 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d8d05700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d419bc30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95c8011670 tx=0x7f95c8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c80039c0 con 0x7f95d4107ce0 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95c8011e50 con 0x7f95d4107ce0 2024-04-24T23:02:45.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d419c400 con 0x7f95d4107ce0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c801a5c0 con 0x7f95d4107ce0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.382+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d419f0f0 con 0x7f95d4107ce0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.383+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f95c8024430 con 0x7f95d4107ce0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.383+0000 7f95d1ffb700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f95c4039050 0x7f95c403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.383+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d4063890 con 0x7f95d4107ce0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.384+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95c804d410 con 0x7f95d4107ce0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.384+0000 7f95d3fff700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f95c4039050 0x7f95c403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.384+0000 7f95d3fff700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f95c4039050 0x7f95c403b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:45.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.389+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95c8022b60 con 0x7f95d4107ce0 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.600+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f95d4104540 con 0x7f95c4039050 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.600+0000 7f95d1ffb700 1 -- 172.21.15.33:0/2207392689 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f95d4104540 con 0x7f95c4039050 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f95c4039050 msgr2=0x7f95c403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f95c4039050 0x7f95c403b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 msgr2=0x7f95d419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d419bc30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95c8011670 tx=0x7f95c8011b60 comp rx=0 tx=0).stop 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 shutdown_connections 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f95c4039050 0x7f95c403b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.602+0000 7f95daf69700 1 --2- 172.21.15.33:0/2207392689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95d4107ce0 0x7f95d419bc30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.603+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 >> 172.21.15.33:0/2207392689 conn(0x7f95d4103300 msgr2=0x7f95d4103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:45.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.603+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 shutdown_connections 2024-04-24T23:02:45.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:44.603+0000 7f95daf69700 1 -- 172.21.15.33:0/2207392689 wait complete. 2024-04-24T23:02:45.155 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:02:45.155 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T23:02:45.155 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-04-24T23:02:45.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:45 smithi033 ceph-mon[29201]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:46.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:46 smithi033 ceph-mon[29201]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:47 smithi033 ceph-mon[29201]: Deploying cephadm binary to smithi033 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.647+0000 7f2999c51700 1 Processor -- start 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.647+0000 7f2999c51700 1 -- start start 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f2999c51700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f2999c51700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f299407c7a0 con 0x7f299407dd60 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50786/0 (socket says 172.21.15.33:50786) 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29937fe700 1 -- 172.21.15.33:0/2272706441 learned_addr learned my addr 172.21.15.33:0/2272706441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29937fe700 1 -- 172.21.15.33:0/2272706441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299407c920 con 0x7f299407dd60 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29937fe700 1 --2- 172.21.15.33:0/2272706441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299407c260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29840098a0 tx=0x7f2984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d91a1e552084ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29927fc700 1 -- 172.21.15.33:0/2272706441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984004160 con 0x7f299407dd60 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29927fc700 1 -- 172.21.15.33:0/2272706441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2984004300 con 0x7f299407dd60 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.648+0000 7f29927fc700 1 -- 172.21.15.33:0/2272706441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984011b50 con 0x7f299407dd60 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 -- 172.21.15.33:0/2272706441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 msgr2=0x7f299407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 --2- 172.21.15.33:0/2272706441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299407c260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29840098a0 tx=0x7f2984009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 -- 172.21.15.33:0/2272706441 shutdown_connections 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 --2- 172.21.15.33:0/2272706441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299407c260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 -- 172.21.15.33:0/2272706441 >> 172.21.15.33:0/2272706441 conn(0x7f2994103300 msgr2=0x7f2994105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 -- 172.21.15.33:0/2272706441 shutdown_connections 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.649+0000 7f2999c51700 1 -- 172.21.15.33:0/2272706441 wait complete. 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f2999c51700 1 Processor -- start 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f2999c51700 1 -- start start 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f2999c51700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f2999c51700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f299410e250 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f29937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f29937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50800/0 (socket says 172.21.15.33:50800) 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.650+0000 7f29937fe700 1 -- 172.21.15.33:0/204701654 learned_addr learned my addr 172.21.15.33:0/204701654 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f29937fe700 1 -- 172.21.15.33:0/204701654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299419e2e0 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f29937fe700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299419dda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2984011b60 tx=0x7f2984011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984003b20 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2984003cc0 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299419e570 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984024840 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.651+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941a1260 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.652+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2984022020 con 0x7f299407dd60 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.652+0000 7f2990ff9700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f297c0390a0 0x7f297c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.652+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2984011e00 con 0x7f299407dd60 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.652+0000 7f2992ffd700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f297c0390a0 0x7f297c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.652+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2994063890 con 0x7f299407dd60 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.653+0000 7f2992ffd700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f297c0390a0 0x7f297c03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2988006fd0 tx=0x7f2988006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.658+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f29840172b0 con 0x7f299407dd60 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:45.869+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) -- 0x7f2994104530 con 0x7f297c0390a0 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.144+0000 7f2990ff9700 1 -- 172.21.15.33:0/204701654 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2994104530 con 0x7f297c0390a0 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f297c0390a0 msgr2=0x7f297c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f297c0390a0 0x7f297c03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2988006fd0 tx=0x7f2988006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 msgr2=0x7f299419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299419dda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2984011b60 tx=0x7f2984011c40 comp rx=0 tx=0).stop 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 shutdown_connections 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f297c0390a0 0x7f297c03b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 --2- 172.21.15.33:0/204701654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f299407dd60 0x7f299419dda0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:50.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 >> 172.21.15.33:0/204701654 conn(0x7f2994103300 msgr2=0x7f2994103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:50.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 shutdown_connections 2024-04-24T23:02:50.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:50.147+0000 7f2999c51700 1 -- 172.21.15.33:0/204701654 wait complete. 2024-04-24T23:02:50.716 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-04-24T23:02:51.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:51 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:51.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:51 smithi033 ceph-mon[29201]: Added host smithi033 2024-04-24T23:02:51.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:51 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:52.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:52 smithi033 ceph-mon[29201]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:52.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:52 smithi033 ceph-mon[29201]: Saving service mon spec with placement count:5 2024-04-24T23:02:52.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:52 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.293+0000 7f9ff878c700 1 Processor -- start 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.294+0000 7f9ff878c700 1 -- start start 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.294+0000 7f9ff878c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff010ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.294+0000 7f9ff878c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff007bdd0 con 0x7f9ff0108760 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.294+0000 7f9ff6528700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff010ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.294+0000 7f9ff6528700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff010ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55824/0 (socket says 172.21.15.33:55824) 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.294+0000 7f9ff6528700 1 -- 172.21.15.33:0/746994266 learned_addr learned my addr 172.21.15.33:0/746994266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff6528700 1 -- 172.21.15.33:0/746994266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff010b090 con 0x7f9ff0108760 2024-04-24T23:02:53.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff6528700 1 --2- 172.21.15.33:0/746994266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff010ab50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0098a0 tx=0x7f9fec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efccdc3125e88146 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:53.194 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff5526700 1 -- 172.21.15.33:0/746994266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec004160 con 0x7f9ff0108760 2024-04-24T23:02:53.194 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff5526700 1 -- 172.21.15.33:0/746994266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fec004300 con 0x7f9ff0108760 2024-04-24T23:02:53.194 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff5526700 1 -- 172.21.15.33:0/746994266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec011b50 con 0x7f9ff0108760 2024-04-24T23:02:53.194 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff878c700 1 -- 172.21.15.33:0/746994266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 msgr2=0x7f9ff010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:53.194 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.295+0000 7f9ff878c700 1 --2- 172.21.15.33:0/746994266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff010ab50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0098a0 tx=0x7f9fec009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:53.194 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 -- 172.21.15.33:0/746994266 shutdown_connections 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 --2- 172.21.15.33:0/746994266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff010ab50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 -- 172.21.15.33:0/746994266 >> 172.21.15.33:0/746994266 conn(0x7f9ff0102170 msgr2=0x7f9ff0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 -- 172.21.15.33:0/746994266 shutdown_connections 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 -- 172.21.15.33:0/746994266 wait complete. 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 Processor -- start 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 -- start start 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.296+0000 7f9ff878c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff010c060 con 0x7f9ff0108760 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff6528700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff6528700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff019bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55828/0 (socket says 172.21.15.33:55828) 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff6528700 1 -- 172.21.15.33:0/1145847897 learned_addr learned my addr 172.21.15.33:0/1145847897 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff6528700 1 -- 172.21.15.33:0/1145847897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019c1a0 con 0x7f9ff0108760 2024-04-24T23:02:53.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff6528700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff019bc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9fec011b60 tx=0x7f9fec011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec003970 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019c430 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fec003b10 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec024610 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.297+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff019f120 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.298+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9fec0248c0 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.298+0000 7f9fe77fe700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f9fdc0390a0 0x7f9fdc03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.298+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9fec011e00 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.298+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff0063890 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.298+0000 7f9ff5d27700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f9fdc0390a0 0x7f9fdc03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.299+0000 7f9ff5d27700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f9fdc0390a0 0x7f9fdc03b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0006fd0 tx=0x7f9fe0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.303+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9fec0172b0 con 0x7f9ff0108760 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.540+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9ff01033b0 con 0x7f9fdc0390a0 2024-04-24T23:02:53.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.545+0000 7f9fe77fe700 1 -- 172.21.15.33:0/1145847897 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9ff01033b0 con 0x7f9fdc0390a0 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f9fdc0390a0 msgr2=0x7f9fdc03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f9fdc0390a0 0x7f9fdc03b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0006fd0 tx=0x7f9fe0006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 msgr2=0x7f9ff019bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff019bc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9fec011b60 tx=0x7f9fec011c40 comp rx=0 tx=0).stop 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 shutdown_connections 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f9fdc0390a0 0x7f9fdc03b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 --2- 172.21.15.33:0/1145847897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff0108760 0x7f9ff019bc60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 >> 172.21.15.33:0/1145847897 conn(0x7f9ff0102170 msgr2=0x7f9ff0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 shutdown_connections 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:51.547+0000 7f9ff878c700 1 -- 172.21.15.33:0/1145847897 wait complete. 2024-04-24T23:02:53.197 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-04-24T23:02:53.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:53 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:53.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:53 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:54.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.616+0000 7ff148450700 1 Processor -- start 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.616+0000 7ff148450700 1 -- start start 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.616+0000 7ff148450700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff140106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.616+0000 7ff148450700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff140107590 con 0x7ff140106ba0 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1461ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff140106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1461ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff140106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55832/0 (socket says 172.21.15.33:55832) 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1461ec700 1 -- 172.21.15.33:0/595163281 learned_addr learned my addr 172.21.15.33:0/595163281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1461ec700 1 -- 172.21.15.33:0/595163281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff140107da0 con 0x7ff140106ba0 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1461ec700 1 --2- 172.21.15.33:0/595163281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff140106fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0098a0 tx=0x7ff13c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df63dbe1ac65aaf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1451ea700 1 -- 172.21.15.33:0/595163281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13c004160 con 0x7ff140106ba0 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1451ea700 1 -- 172.21.15.33:0/595163281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c004300 con 0x7ff140106ba0 2024-04-24T23:02:54.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.617+0000 7ff1451ea700 1 -- 172.21.15.33:0/595163281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13c011b20 con 0x7ff140106ba0 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 -- 172.21.15.33:0/595163281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 msgr2=0x7ff140106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 --2- 172.21.15.33:0/595163281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff140106fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0098a0 tx=0x7ff13c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 -- 172.21.15.33:0/595163281 shutdown_connections 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 --2- 172.21.15.33:0/595163281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff140106fc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 -- 172.21.15.33:0/595163281 >> 172.21.15.33:0/595163281 conn(0x7ff140102160 msgr2=0x7ff140104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 -- 172.21.15.33:0/595163281 shutdown_connections 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.618+0000 7ff148450700 1 -- 172.21.15.33:0/595163281 wait complete. 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff148450700 1 Processor -- start 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff148450700 1 -- start start 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff148450700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff14007ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff148450700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff14010b150 con 0x7ff140106ba0 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff1461ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff14007ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff1461ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff14007ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55848/0 (socket says 172.21.15.33:55848) 2024-04-24T23:02:54.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff1461ec700 1 -- 172.21.15.33:0/4076375369 learned_addr learned my addr 172.21.15.33:0/4076375369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff1461ec700 1 -- 172.21.15.33:0/4076375369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019e920 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.619+0000 7ff1461ec700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff14007ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff13c000c00 tx=0x7ff13c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.620+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13c003b60 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.620+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c01a910 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.620+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019eaf0 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.620+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13c024930 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.620+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14019ee90 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.621+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff13c022020 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.621+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff140063890 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.621+0000 7ff1377fe700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7ff12c0390a0 0x7ff12c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.621+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff13c04d1e0 con 0x7ff140106ba0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.621+0000 7ff1459eb700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7ff12c0390a0 0x7ff12c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.621+0000 7ff1459eb700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7ff12c0390a0 0x7ff12c03b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff130006fd0 tx=0x7ff130006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.626+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff13c02c510 con 0x7ff140106ba0 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.863+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff140103380 con 0x7ff12c0390a0 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.870+0000 7ff1377fe700 1 -- 172.21.15.33:0/4076375369 <== mgr.14116 v2:172.21.15.33:6800/2813273186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff140103380 con 0x7ff12c0390a0 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7ff12c0390a0 msgr2=0x7ff12c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7ff12c0390a0 0x7ff12c03b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff130006fd0 tx=0x7ff130006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 msgr2=0x7ff14007ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff14007ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff13c000c00 tx=0x7ff13c011c10 comp rx=0 tx=0).stop 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 shutdown_connections 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7ff12c0390a0 0x7ff12c03b560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 --2- 172.21.15.33:0/4076375369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff140106ba0 0x7ff14007ba30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.872+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 >> 172.21.15.33:0/4076375369 conn(0x7ff140102160 msgr2=0x7ff140102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.873+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 shutdown_connections 2024-04-24T23:02:54.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.873+0000 7ff148450700 1 -- 172.21.15.33:0/4076375369 wait complete. 2024-04-24T23:02:54.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:54 smithi033 ceph-mon[29201]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:02:54.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:54 smithi033 ceph-mon[29201]: Saving service mgr spec with placement count:2 2024-04-24T23:02:54.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:54 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:55.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ee057b700 1 Processor -- start 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ee057b700 1 -- start start 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ee057b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed810ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ee057b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed807bdf0 con 0x7f3ed8108750 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ede317700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed810ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ede317700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed810ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55854/0 (socket says 172.21.15.33:55854) 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.176+0000 7f3ede317700 1 -- 172.21.15.33:0/2232720800 learned_addr learned my addr 172.21.15.33:0/2232720800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.177+0000 7f3ede317700 1 -- 172.21.15.33:0/2232720800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed810b080 con 0x7f3ed8108750 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.177+0000 7f3ede317700 1 --2- 172.21.15.33:0/2232720800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed810ab40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40098a0 tx=0x7f3ed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8ccb8682dc1a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.177+0000 7f3edd315700 1 -- 172.21.15.33:0/2232720800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4004160 con 0x7f3ed8108750 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.177+0000 7f3edd315700 1 -- 172.21.15.33:0/2232720800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed4004300 con 0x7f3ed8108750 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.177+0000 7f3edd315700 1 -- 172.21.15.33:0/2232720800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4011b20 con 0x7f3ed8108750 2024-04-24T23:02:56.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 -- 172.21.15.33:0/2232720800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 msgr2=0x7f3ed810ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 --2- 172.21.15.33:0/2232720800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed810ab40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40098a0 tx=0x7f3ed4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 -- 172.21.15.33:0/2232720800 shutdown_connections 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 --2- 172.21.15.33:0/2232720800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed810ab40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 -- 172.21.15.33:0/2232720800 >> 172.21.15.33:0/2232720800 conn(0x7f3ed8102160 msgr2=0x7f3ed8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 -- 172.21.15.33:0/2232720800 shutdown_connections 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 -- 172.21.15.33:0/2232720800 wait complete. 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 Processor -- start 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.178+0000 7f3ee057b700 1 -- start start 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ee057b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ee057b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810c0e0 con 0x7f3ed8108750 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ede317700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ede317700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55870/0 (socket says 172.21.15.33:55870) 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ede317700 1 -- 172.21.15.33:0/4030993961 learned_addr learned my addr 172.21.15.33:0/4030993961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:56.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ede317700 1 -- 172.21.15.33:0/4030993961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed819c140 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ede317700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed819bc00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4011b60 tx=0x7f3ed4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4003b60 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed401a910 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed819c3d0 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.179+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4024930 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.180+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed819c830 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.180+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3ed4022020 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.181+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed8063890 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.181+0000 7f3ecf7fe700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f3ec4039050 0x7f3ec403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.181+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ed404d2d0 con 0x7f3ed8108750 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.181+0000 7f3eddb16700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f3ec4039050 0x7f3ec403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.181+0000 7f3eddb16700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f3ec4039050 0x7f3ec403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006fd0 tx=0x7f3ec8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.002 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.186+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ed4019c70 con 0x7f3ed8108750 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.400+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3ed819f0c0 con 0x7f3ed8108750 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.402+0000 7f3ecf7fe700 1 -- 172.21.15.33:0/4030993961 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3ed40172b0 con 0x7f3ed8108750 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f3ec4039050 msgr2=0x7f3ec403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f3ec4039050 0x7f3ec403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006fd0 tx=0x7f3ec8006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 msgr2=0x7f3ed819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed819bc00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4011b60 tx=0x7f3ed4011c40 comp rx=0 tx=0).stop 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 shutdown_connections 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f3ec4039050 0x7f3ec403b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 --2- 172.21.15.33:0/4030993961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ed8108750 0x7f3ed819bc00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.404+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 >> 172.21.15.33:0/4030993961 conn(0x7f3ed8102160 msgr2=0x7f3ed8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.405+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 shutdown_connections 2024-04-24T23:02:56.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.405+0000 7f3ee057b700 1 -- 172.21.15.33:0/4030993961 wait complete. 2024-04-24T23:02:56.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:56 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4030993961' entity='client.admin' 2024-04-24T23:02:57.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.423+0000 7fbfab8c4700 1 Processor -- start 2024-04-24T23:02:57.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.423+0000 7fbfab8c4700 1 -- start start 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfab8c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa410ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfab8c4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa407be10 con 0x7fbfa4108720 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfa9660700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa410ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfa9660700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa410ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57406/0 (socket says 172.21.15.33:57406) 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfa9660700 1 -- 172.21.15.33:0/2932462196 learned_addr learned my addr 172.21.15.33:0/2932462196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfa9660700 1 -- 172.21.15.33:0/2932462196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa410b050 con 0x7fbfa4108720 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbfa9660700 1 --2- 172.21.15.33:0/2932462196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa410ab10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00098a0 tx=0x7fbfa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b85f9838369cefd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.424+0000 7fbf9bfff700 1 -- 172.21.15.33:0/2932462196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa0004160 con 0x7fbfa4108720 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbf9bfff700 1 -- 172.21.15.33:0/2932462196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa0004300 con 0x7fbfa4108720 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbf9bfff700 1 -- 172.21.15.33:0/2932462196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa0011b20 con 0x7fbfa4108720 2024-04-24T23:02:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 -- 172.21.15.33:0/2932462196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 msgr2=0x7fbfa410ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 --2- 172.21.15.33:0/2932462196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa410ab10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00098a0 tx=0x7fbfa0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 -- 172.21.15.33:0/2932462196 shutdown_connections 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 --2- 172.21.15.33:0/2932462196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa410ab10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 -- 172.21.15.33:0/2932462196 >> 172.21.15.33:0/2932462196 conn(0x7fbfa4102110 msgr2=0x7fbfa4104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 -- 172.21.15.33:0/2932462196 shutdown_connections 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.425+0000 7fbfab8c4700 1 -- 172.21.15.33:0/2932462196 wait complete. 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.426+0000 7fbfab8c4700 1 Processor -- start 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfab8c4700 1 -- start start 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfab8c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa4107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfab8c4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa410c0b0 con 0x7fbfa4108720 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfa9660700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa4107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfa9660700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa4107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57420/0 (socket says 172.21.15.33:57420) 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfa9660700 1 -- 172.21.15.33:0/4012873735 learned_addr learned my addr 172.21.15.33:0/4012873735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfa9660700 1 -- 172.21.15.33:0/4012873735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa41084e0 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbfa9660700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa4107fa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00116a0 tx=0x7fbfa0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.427+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa0003a40 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.428+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa41066d0 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.428+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa0003be0 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.428+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa001a750 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.428+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa4106b90 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.428+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbfa001aa50 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.429+0000 7fbf9a7fc700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fbf90039000 0x7fbf9003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.429+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbfa004e610 con 0x7fbfa4108720 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.429+0000 7fbfa8e5f700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fbf90039000 0x7fbf9003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.429+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa4063890 con 0x7fbfa4108720 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.429+0000 7fbfa8e5f700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fbf90039000 0x7fbf9003b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006fd0 tx=0x7fbf94006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.434+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfa0011e20 con 0x7fbfa4108720 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.656+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbfa4107520 con 0x7fbfa4108720 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.660+0000 7fbf9a7fc700 1 -- 172.21.15.33:0/4012873735 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbfa0016df0 con 0x7fbfa4108720 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fbf90039000 msgr2=0x7fbf9003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fbf90039000 0x7fbf9003b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006fd0 tx=0x7fbf94006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 msgr2=0x7fbfa4107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa4107fa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00116a0 tx=0x7fbfa0011ba0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 shutdown_connections 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7fbf90039000 0x7fbf9003b4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 --2- 172.21.15.33:0/4012873735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa4108720 0x7fbfa4107fa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 >> 172.21.15.33:0/4012873735 conn(0x7fbfa4102110 msgr2=0x7fbfa4102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 shutdown_connections 2024-04-24T23:02:57.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:56.662+0000 7fbfab8c4700 1 -- 172.21.15.33:0/4012873735 wait complete. 2024-04-24T23:02:57.261 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-04-24T23:02:57.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:57 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:57.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:57 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4012873735' entity='client.admin' 2024-04-24T23:02:58.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:58 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/815075775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T23:02:58.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:58 smithi033 ceph-mon[29201]: from='mgr.14116 172.21.15.33:0/1733926180' entity='mgr.x' 2024-04-24T23:02:58.855 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: ignoring --setuser ceph since I am not root 2024-04-24T23:02:58.855 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: ignoring --setgroup ceph since I am not root 2024-04-24T23:02:58.855 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:58.758+0000 7f7bb1cb4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:02:59.163 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:58.951+0000 7f7bb1cb4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:02:59.164 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:59.056+0000 7f7bb1cb4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:02:59.173 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33ef163700 1 Processor -- start 2024-04-24T23:02:59.173 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33ef163700 1 -- start start 2024-04-24T23:02:59.173 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33ef163700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e8106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.173 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33ef163700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33e8107580 con 0x7f33e8106b90 2024-04-24T23:02:59.173 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33eceff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e8106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33eceff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e8106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57434/0 (socket says 172.21.15.33:57434) 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33eceff700 1 -- 172.21.15.33:0/680059574 learned_addr learned my addr 172.21.15.33:0/680059574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.664+0000 7f33eceff700 1 -- 172.21.15.33:0/680059574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e8107d90 con 0x7f33e8106b90 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33eceff700 1 --2- 172.21.15.33:0/680059574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e8106fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0098a0 tx=0x7f33dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1997573dd7912e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33e77fe700 1 -- 172.21.15.33:0/680059574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33dc004160 con 0x7f33e8106b90 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33e77fe700 1 -- 172.21.15.33:0/680059574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc004300 con 0x7f33e8106b90 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33e77fe700 1 -- 172.21.15.33:0/680059574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33dc011b20 con 0x7f33e8106b90 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33ef163700 1 -- 172.21.15.33:0/680059574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 msgr2=0x7f33e8106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33ef163700 1 --2- 172.21.15.33:0/680059574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e8106fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0098a0 tx=0x7f33dc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33ef163700 1 -- 172.21.15.33:0/680059574 shutdown_connections 2024-04-24T23:02:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33ef163700 1 --2- 172.21.15.33:0/680059574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e8106fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33ef163700 1 -- 172.21.15.33:0/680059574 >> 172.21.15.33:0/680059574 conn(0x7f33e8102110 msgr2=0x7f33e8104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.665+0000 7f33ef163700 1 -- 172.21.15.33:0/680059574 shutdown_connections 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.666+0000 7f33ef163700 1 -- 172.21.15.33:0/680059574 wait complete. 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.666+0000 7f33ef163700 1 Processor -- start 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.666+0000 7f33ef163700 1 -- start start 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.666+0000 7f33ef163700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.666+0000 7f33ef163700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33e810b140 con 0x7f33e8106b90 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33eceff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33eceff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57448/0 (socket says 172.21.15.33:57448) 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33eceff700 1 -- 172.21.15.33:0/815075775 learned_addr learned my addr 172.21.15.33:0/815075775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33eceff700 1 -- 172.21.15.33:0/815075775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e819b060 con 0x7f33e8106b90 2024-04-24T23:02:59.175 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33eceff700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e819ab20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000c00 tx=0x7f33dc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33dc003b60 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc01a910 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33dc024930 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e819b2f0 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.667+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e819b750 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.668+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f33dc022020 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.668+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e8063890 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.668+0000 7f33e5ffb700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f33d8039050 0x7f33d803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.669+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f33dc04d240 con 0x7f33e8106b90 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.669+0000 7f33e7fff700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f33d8039050 0x7f33d803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.669+0000 7f33e7fff700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f33d8039050 0x7f33d803b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33d4006fd0 tx=0x7f33d4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.675+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f33dc01ab70 con 0x7f33e8106b90 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:57.944+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f33e819e120 con 0x7f33e8106b90 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.553+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f33dc02d3f0 con 0x7f33e8106b90 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.554+0000 7f33e5ffb700 1 -- 172.21.15.33:0/815075775 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f33dc01a430 con 0x7f33e8106b90 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f33d8039050 msgr2=0x7f33d803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f33d8039050 0x7f33d803b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33d4006fd0 tx=0x7f33d4006e40 comp rx=0 tx=0).stop 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 msgr2=0x7f33e819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e819ab20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000c00 tx=0x7f33dc011c10 comp rx=0 tx=0).stop 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 shutdown_connections 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f33d8039050 0x7f33d803b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 --2- 172.21.15.33:0/815075775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e8106b90 0x7f33e819ab20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.178 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.556+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 >> 172.21.15.33:0/815075775 conn(0x7f33e8102110 msgr2=0x7f33e8102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:59.178 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.557+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 shutdown_connections 2024-04-24T23:02:59.178 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:58.557+0000 7f33ef163700 1 -- 172.21.15.33:0/815075775 wait complete. 2024-04-24T23:02:59.414 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:59.380+0000 7f7bb1cb4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:02:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:02:59.471+0000 7f7bb1cb4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:59 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/815075775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:02:59 smithi033 ceph-mon[29201]: mgrmap e9: x(active, since 20s) 2024-04-24T23:03:00.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:00 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/941876875' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:03:00.914 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:00.739+0000 7f7bb1cb4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:03:01.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:00.916+0000 7f7bb1cb4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:01.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:01 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:01.093+0000 7f7bb1cb4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.609+0000 7fabc2765700 1 Processor -- start 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.609+0000 7fabc2765700 1 -- start start 2024-04-24T23:03:01.519 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.609+0000 7fabc2765700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.609+0000 7fabc2765700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc07c850 con 0x7fabbc07de20 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57482/0 (socket says 172.21.15.33:57482) 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbbfff700 1 -- 172.21.15.33:0/2894807900 learned_addr learned my addr 172.21.15.33:0/2894807900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbbfff700 1 -- 172.21.15.33:0/2894807900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc07c9d0 con 0x7fabbc07de20 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbbfff700 1 --2- 172.21.15.33:0/2894807900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc07c280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098a0 tx=0x7fabac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0b04fdd2c0dfac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbaffd700 1 -- 172.21.15.33:0/2894807900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac004160 con 0x7fabbc07de20 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbaffd700 1 -- 172.21.15.33:0/2894807900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac004300 con 0x7fabbc07de20 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.610+0000 7fabbaffd700 1 -- 172.21.15.33:0/2894807900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac011b20 con 0x7fabbc07de20 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 -- 172.21.15.33:0/2894807900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 msgr2=0x7fabbc07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 --2- 172.21.15.33:0/2894807900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc07c280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098a0 tx=0x7fabac009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 -- 172.21.15.33:0/2894807900 shutdown_connections 2024-04-24T23:03:01.520 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 --2- 172.21.15.33:0/2894807900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc07c280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 -- 172.21.15.33:0/2894807900 >> 172.21.15.33:0/2894807900 conn(0x7fabbc103210 msgr2=0x7fabbc105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 -- 172.21.15.33:0/2894807900 shutdown_connections 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 -- 172.21.15.33:0/2894807900 wait complete. 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.611+0000 7fabc2765700 1 Processor -- start 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabc2765700 1 -- start start 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabc2765700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabc2765700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10e2d0 con 0x7fabbc07de20 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57496/0 (socket says 172.21.15.33:57496) 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabbbfff700 1 -- 172.21.15.33:0/941876875 learned_addr learned my addr 172.21.15.33:0/941876875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabbbfff700 1 -- 172.21.15.33:0/941876875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19e340 con 0x7fabbc07de20 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.612+0000 7fabbbfff700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc19de00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabac0116a0 tx=0x7fabac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.613+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac003a40 con 0x7fabbc07de20 2024-04-24T23:03:01.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.613+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac003be0 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.613+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc19e5d0 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.613+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac01a720 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.613+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc19ea30 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.614+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fabac024aa0 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.614+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc063890 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.614+0000 7fabb97fa700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 0x7faba403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.614+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabac04d620 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.614+0000 7fabbb7fe700 1 -- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 msgr2=0x7faba403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.614+0000 7fabbb7fe700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 0x7faba403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.619+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fabac011e20 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.815+0000 7fabbb7fe700 1 -- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 msgr2=0x7faba403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.815+0000 7fabbb7fe700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 0x7faba403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.955+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fabbc1a13e0 con 0x7fabbc07de20 2024-04-24T23:03:01.522 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.956+0000 7fabb97fa700 1 -- 172.21.15.33:0/941876875 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fabac02d770 con 0x7fabbc07de20 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 msgr2=0x7faba403b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 0x7faba403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 msgr2=0x7fabbc19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc19de00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabac0116a0 tx=0x7fabac011ba0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 shutdown_connections 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7faba40390b0 0x7faba403b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 --2- 172.21.15.33:0/941876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabbc07de20 0x7fabbc19de00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 >> 172.21.15.33:0/941876875 conn(0x7fabbc103210 msgr2=0x7fabbc103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 shutdown_connections 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.958+0000 7fabc2765700 1 -- 172.21.15.33:0/941876875 wait complete. 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-24T23:03:01.523 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-04-24T23:03:01.637 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:01 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:01.395+0000 7f7bb1cb4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:03:01.637 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:01 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:01.493+0000 7f7bb1cb4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:03:01.913 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:01 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:01.708+0000 7f7bb1cb4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:03:02.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:01 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:01.912+0000 7f7bb1cb4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:02.955 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:02 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:02.538+0000 7f7bb1cb4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:03:03.284 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:02 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:02.976+0000 7f7bb1cb4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:03:03.285 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:03 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:03.075+0000 7f7bb1cb4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:03:03.285 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:03 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:03.282+0000 7f7bb1cb4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:03:04.174 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:03 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:03.825+0000 7f7bb1cb4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:03:04.175 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:03 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:03.920+0000 7f7bb1cb4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:03:04.455 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:04 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:04.173+0000 7f7bb1cb4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:03:04.455 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:04 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:04.310+0000 7f7bb1cb4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:03:04.455 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:04 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:04.408+0000 7f7bb1cb4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:03:04.950 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:04 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:04.528+0000 7f7bb1cb4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:04.950 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:04 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:04.623+0000 7f7bb1cb4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:03:05.744 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:05.464+0000 7f7bb1cb4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:03:05.744 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:05.690+0000 7f7bb1cb4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:03:06.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: Active manager daemon x restarted 2024-04-24T23:03:06.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: Activating manager daemon x 2024-04-24T23:03:06.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T23:03:06.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: mgrmap e10: x(active, starting, since 0.00342492s) 2024-04-24T23:03:06.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: Manager daemon x is now available 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:06.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:03:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:03:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:06 smithi033 ceph-mon[29201]: [24/Apr/2024:23:03:06] ENGINE Bus STARTING 2024-04-24T23:03:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:06 smithi033 ceph-mon[29201]: [24/Apr/2024:23:03:06] ENGINE Serving on http://172.21.15.33:8765 2024-04-24T23:03:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:06 smithi033 ceph-mon[29201]: mgrmap e11: x(active, since 1.00584s) 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.939+0000 7f6ad09bb700 1 Processor -- start 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.939+0000 7f6ad09bb700 1 -- start start 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.939+0000 7f6ad09bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc10ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.654 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.939+0000 7f6ad09bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc07bdf0 con 0x7f6acc108750 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6aca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc10ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6aca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc10ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57498/0 (socket says 172.21.15.33:57498) 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6aca59c700 1 -- 172.21.15.33:0/2116727145 learned_addr learned my addr 172.21.15.33:0/2116727145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6aca59c700 1 -- 172.21.15.33:0/2116727145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6acc10b080 con 0x7f6acc108750 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6aca59c700 1 --2- 172.21.15.33:0/2116727145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc10ab40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40098a0 tx=0x7f6ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce4457e698fa79b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6ac959a700 1 -- 172.21.15.33:0/2116727145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4004160 con 0x7f6acc108750 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6ac959a700 1 -- 172.21.15.33:0/2116727145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab4004300 con 0x7f6acc108750 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.940+0000 7f6ac959a700 1 -- 172.21.15.33:0/2116727145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4011b20 con 0x7f6acc108750 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2116727145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 msgr2=0x7f6acc10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2116727145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc10ab40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40098a0 tx=0x7f6ab4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2116727145 shutdown_connections 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2116727145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc10ab40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2116727145 >> 172.21.15.33:0/2116727145 conn(0x7f6acc102160 msgr2=0x7f6acc104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2116727145 shutdown_connections 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.941+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2116727145 wait complete. 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6ad09bb700 1 Processor -- start 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6ad09bb700 1 -- start start 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6ad09bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6ad09bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc10c0e0 con 0x7f6acc108750 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6aca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6aca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57502/0 (socket says 172.21.15.33:57502) 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.942+0000 7f6aca59c700 1 -- 172.21.15.33:0/2378253446 learned_addr learned my addr 172.21.15.33:0/2378253446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6aca59c700 1 -- 172.21.15.33:0/2378253446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6acc19c120 con 0x7f6acc108750 2024-04-24T23:03:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6aca59c700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc19bbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40116a0 tx=0x7f6ab4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4003a40 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6acc19c3b0 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab4003be0 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab401a750 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.943+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6acc19c810 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.944+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f6ab401aa90 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.944+0000 7f6ac37fe700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.944+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 --> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6acc063890 con 0x7f6ab8039050 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.944+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ab404d900 con 0x7f6acc108750 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.944+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 msgr2=0x7f6ab803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.944+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.145+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 msgr2=0x7f6ab803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.145+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:03:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.546+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 msgr2=0x7f6ab803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.546+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:03.347+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 msgr2=0x7f6ab803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:03.347+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:04.948+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 msgr2=0x7f6ab803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2813273186 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:04.948+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.693+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f6ab402d910 con 0x7f6acc108750 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.693+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 msgr2=0x7f6ab803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.693+0000 7f6ac37fe700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.695+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6ab404f5a0 con 0x7f6acc108750 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.695+0000 7f6ac37fe700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6ab803c920 0x7f6ab803ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.695+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6acc063890 con 0x7f6ab803c920 2024-04-24T23:03:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.696+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6ab803c920 0x7f6ab803ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.696+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6ab803c920 0x7f6ab803ed10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6abc003a10 tx=0x7f6abc0092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.697+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f6acc063890 con 0x7f6ab803c920 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6acc103330 con 0x7f6ab803c920 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ac37fe700 1 -- 172.21.15.33:0/2378253446 <== mgr.14148 v2:172.21.15.33:6800/3494091302 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6acc103330 con 0x7f6ab803c920 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6ab803c920 msgr2=0x7f6ab803ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6ab803c920 0x7f6ab803ed10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6abc003a10 tx=0x7f6abc0092b0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 msgr2=0x7f6acc19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc19bbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40116a0 tx=0x7f6ab4011ba0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.659 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 shutdown_connections 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6ab803c920 0x7f6ab803ed10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc108750 0x7f6acc19bbe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 --2- 172.21.15.33:0/2378253446 >> [v2:172.21.15.33:6800/2813273186,v1:172.21.15.33:6801/2813273186] conn(0x7f6ab8039050 0x7f6ab803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 >> 172.21.15.33:0/2378253446 conn(0x7f6acc102160 msgr2=0x7f6acc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 shutdown_connections 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:06.704+0000 7f6ad09bb700 1 -- 172.21.15.33:0/2378253446 wait complete. 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-04-24T23:03:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-04-24T23:03:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:08 smithi033 ceph-mon[29201]: [24/Apr/2024:23:03:06] ENGINE Serving on https://172.21.15.33:7150 2024-04-24T23:03:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:08 smithi033 ceph-mon[29201]: [24/Apr/2024:23:03:06] ENGINE Bus STARTED 2024-04-24T23:03:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:08 smithi033 ceph-mon[29201]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:03:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:08 smithi033 ceph-mon[29201]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:03:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.140+0000 7f3b4584b700 1 Processor -- start 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.140+0000 7f3b4584b700 1 -- start start 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b4584b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b401069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b4584b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4007bdb0 con 0x7f3b40106580 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b401069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b401069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36196/0 (socket says 172.21.15.33:36196) 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3effd700 1 -- 172.21.15.33:0/3436963676 learned_addr learned my addr 172.21.15.33:0/3436963676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:09.125 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3effd700 1 -- 172.21.15.33:0/3436963676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b40106ee0 con 0x7f3b40106580 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3effd700 1 --2- 172.21.15.33:0/3436963676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b401069a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b300098a0 tx=0x7f3b30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08c4967b21fe04a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3dffb700 1 -- 172.21.15.33:0/3436963676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b30004160 con 0x7f3b40106580 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.141+0000 7f3b3dffb700 1 -- 172.21.15.33:0/3436963676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30004300 con 0x7f3b40106580 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b3dffb700 1 -- 172.21.15.33:0/3436963676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b30011b20 con 0x7f3b40106580 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 -- 172.21.15.33:0/3436963676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 msgr2=0x7f3b401069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 --2- 172.21.15.33:0/3436963676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b401069a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b300098a0 tx=0x7f3b30009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 -- 172.21.15.33:0/3436963676 shutdown_connections 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 --2- 172.21.15.33:0/3436963676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b401069a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 -- 172.21.15.33:0/3436963676 >> 172.21.15.33:0/3436963676 conn(0x7f3b40102160 msgr2=0x7f3b40104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 -- 172.21.15.33:0/3436963676 shutdown_connections 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.142+0000 7f3b4584b700 1 -- 172.21.15.33:0/3436963676 wait complete. 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b4584b700 1 Processor -- start 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b4584b700 1 -- start start 2024-04-24T23:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b4584b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b4019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b4584b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40107fb0 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b4019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b4019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36202/0 (socket says 172.21.15.33:36202) 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.143+0000 7f3b3effd700 1 -- 172.21.15.33:0/2669266040 learned_addr learned my addr 172.21.15.33:0/2669266040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b3effd700 1 -- 172.21.15.33:0/2669266040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4019e330 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b3effd700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b4019ddf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b300116a0 tx=0x7f3b30011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b30003a40 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4019e5c0 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30003be0 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b3001a750 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.144+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4019ea20 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.145+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f3b3001aa50 con 0x7f3b40106580 2024-04-24T23:03:09.127 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.145+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c005360 con 0x7f3b40106580 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.145+0000 7f3b44849700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3b28038f40 0x7f3b2803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.145+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b3004d620 con 0x7f3b40106580 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.145+0000 7f3b3e7fc700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3b28038f40 0x7f3b2803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.146+0000 7f3b3e7fc700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3b28038f40 0x7f3b2803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.150+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b30011e20 con 0x7f3b40106580 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.350+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3b3002d910 con 0x7f3b40106580 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.366+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f3b2c000bf0 con 0x7f3b28038f40 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.509+0000 7f3b44849700 1 -- 172.21.15.33:0/2669266040 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f3b2c000bf0 con 0x7f3b28038f40 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3b28038f40 msgr2=0x7f3b2803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3b28038f40 0x7f3b2803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 msgr2=0x7f3b4019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b4019ddf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b300116a0 tx=0x7f3b30011ba0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 shutdown_connections 2024-04-24T23:03:09.128 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3b28038f40 0x7f3b2803b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 --2- 172.21.15.33:0/2669266040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40106580 0x7f3b4019ddf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 >> 172.21.15.33:0/2669266040 conn(0x7f3b40102160 msgr2=0x7f3b40194a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:09.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 shutdown_connections 2024-04-24T23:03:09.129 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.512+0000 7f3b4584b700 1 -- 172.21.15.33:0/2669266040 wait complete. 2024-04-24T23:03:09.129 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-04-24T23:03:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:09 smithi033 ceph-mon[29201]: mgrmap e12: x(active, since 2s) 2024-04-24T23:03:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:09 smithi033 ceph-mon[29201]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:10.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8jD7B/qc8zwNvo1PRnHif.K3ulSb19UZcols/7zQmfMEuuPON7wKm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713999790, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.579+0000 7f903aaf3700 1 Processor -- start 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.579+0000 7f903aaf3700 1 -- start start 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.579+0000 7f903aaf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f9034106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.579+0000 7f903aaf3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9034106940 con 0x7f9034105f50 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f903888f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f9034106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f903888f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f9034106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36216/0 (socket says 172.21.15.33:36216) 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f903888f700 1 -- 172.21.15.33:0/2213882781 learned_addr learned my addr 172.21.15.33:0/2213882781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f903888f700 1 -- 172.21.15.33:0/2213882781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9034108f30 con 0x7f9034105f50 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f903888f700 1 --2- 172.21.15.33:0/2213882781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f9034106370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9024009e80 tx=0x7f902400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619590e5be68203 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f90337fe700 1 -- 172.21.15.33:0/2213882781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9024017070 con 0x7f9034105f50 2024-04-24T23:03:10.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f90337fe700 1 -- 172.21.15.33:0/2213882781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902400b8c0 con 0x7f9034105f50 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.580+0000 7f90337fe700 1 -- 172.21.15.33:0/2213882781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90240045c0 con 0x7f9034105f50 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 -- 172.21.15.33:0/2213882781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 msgr2=0x7f9034106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 --2- 172.21.15.33:0/2213882781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f9034106370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9024009e80 tx=0x7f902400b330 comp rx=0 tx=0).stop 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 -- 172.21.15.33:0/2213882781 shutdown_connections 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 --2- 172.21.15.33:0/2213882781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f9034106370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 -- 172.21.15.33:0/2213882781 >> 172.21.15.33:0/2213882781 conn(0x7f90341018f0 msgr2=0x7f9034103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 -- 172.21.15.33:0/2213882781 shutdown_connections 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.581+0000 7f903aaf3700 1 -- 172.21.15.33:0/2213882781 wait complete. 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903aaf3700 1 Processor -- start 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903aaf3700 1 -- start start 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903aaf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f903419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903aaf3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f903410c2e0 con 0x7f9034105f50 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903888f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f903419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:10.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903888f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f903419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36232/0 (socket says 172.21.15.33:36232) 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903888f700 1 -- 172.21.15.33:0/1038282497 learned_addr learned my addr 172.21.15.33:0/1038282497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.582+0000 7f903888f700 1 -- 172.21.15.33:0/1038282497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903419c1a0 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.583+0000 7f903888f700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f903419bc60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9024009530 tx=0x7f9024003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.583+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9024020020 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.583+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903419c430 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.583+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902400bba0 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.583+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9024017040 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.583+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903419f120 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.584+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f9024023020 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.584+0000 7f9031ffb700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f90200390a0 0x7f902003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.584+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9034063890 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.584+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f902400f050 con 0x7f9034105f50 2024-04-24T23:03:10.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.585+0000 7f9033fff700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f90200390a0 0x7f902003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.585+0000 7f9033fff700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f90200390a0 0x7f902003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f901c006fd0 tx=0x7f901c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.594+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f902402a020 con 0x7f9034105f50 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:09.826+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f903419f3b0 con 0x7f90200390a0 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.074+0000 7f9031ffb700 1 -- 172.21.15.33:0/1038282497 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f903419f3b0 con 0x7f90200390a0 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f90200390a0 msgr2=0x7f902003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f90200390a0 0x7f902003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f901c006fd0 tx=0x7f901c006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 msgr2=0x7f903419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f903419bc60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9024009530 tx=0x7f9024003d00 comp rx=0 tx=0).stop 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 shutdown_connections 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f90200390a0 0x7f902003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 --2- 172.21.15.33:0/1038282497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034105f50 0x7f903419bc60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 >> 172.21.15.33:0/1038282497 conn(0x7f90341018f0 msgr2=0x7f90341024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 shutdown_connections 2024-04-24T23:03:10.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:10.076+0000 7f903aaf3700 1 -- 172.21.15.33:0/1038282497 wait complete. 2024-04-24T23:03:10.764 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-04-24T23:03:11.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:11 smithi033 ceph-mon[29201]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:11.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:12.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:12 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3834335915' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T23:03:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T23:03:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484efbf700 1 Processor -- start 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484efbf700 1 -- start start 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484efbf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f4848108640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484efbf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4848108c10 con 0x7f4848108220 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484cd5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f4848108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484cd5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f4848108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36246/0 (socket says 172.21.15.33:36246) 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.279+0000 7f484cd5b700 1 -- 172.21.15.33:0/1315716003 learned_addr learned my addr 172.21.15.33:0/1315716003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484cd5b700 1 -- 172.21.15.33:0/1315716003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4848105d20 con 0x7f4848108220 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484cd5b700 1 --2- 172.21.15.33:0/1315716003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f4848108640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098a0 tx=0x7f483c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27805dd19b1dfee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f48477fe700 1 -- 172.21.15.33:0/1315716003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f483c004160 con 0x7f4848108220 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f48477fe700 1 -- 172.21.15.33:0/1315716003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c004300 con 0x7f4848108220 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f48477fe700 1 -- 172.21.15.33:0/1315716003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f483c011b20 con 0x7f4848108220 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484efbf700 1 -- 172.21.15.33:0/1315716003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 msgr2=0x7f4848108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:12.243 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484efbf700 1 --2- 172.21.15.33:0/1315716003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f4848108640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098a0 tx=0x7f483c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484efbf700 1 -- 172.21.15.33:0/1315716003 shutdown_connections 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484efbf700 1 --2- 172.21.15.33:0/1315716003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f4848108640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.280+0000 7f484efbf700 1 -- 172.21.15.33:0/1315716003 >> 172.21.15.33:0/1315716003 conn(0x7f4848101940 msgr2=0x7f4848103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484efbf700 1 -- 172.21.15.33:0/1315716003 shutdown_connections 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484efbf700 1 -- 172.21.15.33:0/1315716003 wait complete. 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484efbf700 1 Processor -- start 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484efbf700 1 -- start start 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484efbf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f484819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484efbf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4848106de0 con 0x7f4848108220 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484cd5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f484819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.281+0000 7f484cd5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f484819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36256/0 (socket says 172.21.15.33:36256) 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f484cd5b700 1 -- 172.21.15.33:0/3834335915 learned_addr learned my addr 172.21.15.33:0/3834335915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f484cd5b700 1 -- 172.21.15.33:0/3834335915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484819e3c0 con 0x7f4848108220 2024-04-24T23:03:12.244 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f484cd5b700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f484819de80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f483c0116a0 tx=0x7f483c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f483c003a40 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c003be0 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484819e650 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f483c01a750 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.282+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484819eab0 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.283+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848063890 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.283+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f483c01aa00 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.284+0000 7f4845ffb700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48380390b0 0x7f483803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.284+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f483c04d900 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.284+0000 7f4847fff700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48380390b0 0x7f483803b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.284+0000 7f4847fff700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48380390b0 0x7f483803b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4834006fd0 tx=0x7f4834006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.289+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f483c011e20 con 0x7f4848108220 2024-04-24T23:03:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.497+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f4848062e40 con 0x7f4848108220 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.497+0000 7f4845ffb700 1 -- 172.21.15.33:0/3834335915 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f483c01adf0 con 0x7f4848108220 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48380390b0 msgr2=0x7f483803b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48380390b0 0x7f483803b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4834006fd0 tx=0x7f4834006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 msgr2=0x7f484819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f484819de80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f483c0116a0 tx=0x7f483c011ba0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 shutdown_connections 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48380390b0 0x7f483803b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 --2- 172.21.15.33:0/3834335915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848108220 0x7f484819de80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 >> 172.21.15.33:0/3834335915 conn(0x7f4848101940 msgr2=0x7f48481949c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 shutdown_connections 2024-04-24T23:03:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.500+0000 7f484efbf700 1 -- 172.21.15.33:0/3834335915 wait complete. 2024-04-24T23:03:12.250 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:12.250 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-24T23:03:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-24T23:03:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-24T23:03:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T23:03:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-04-24T23:03:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033:8443/ 2024-04-24T23:03:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-04-24T23:03:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: Password: giuxrowv8m 2024-04-24T23:03:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config directory 2024-04-24T23:03:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-04-24T23:03:13.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:13 smithi033 ceph-mon[29201]: mgrmap e13: x(active, since 6s) 2024-04-24T23:03:13.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.727+0000 7f9595303700 1 Processor -- start 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.727+0000 7f9595303700 1 -- start start 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.727+0000 7f9595303700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.727+0000 7f9595303700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9590107580 con 0x7f9590106b90 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.727+0000 7f958effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36260/0 (socket says 172.21.15.33:36260) 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958effd700 1 -- 172.21.15.33:0/2788861665 learned_addr learned my addr 172.21.15.33:0/2788861665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958effd700 1 -- 172.21.15.33:0/2788861665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9590107d90 con 0x7f9590106b90 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958effd700 1 --2- 172.21.15.33:0/2788861665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590106fb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f958800d180 tx=0x7f958800d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662acfd170b9f81a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958dffb700 1 -- 172.21.15.33:0/2788861665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9588019070 con 0x7f9590106b90 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958dffb700 1 -- 172.21.15.33:0/2788861665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958800de20 con 0x7f9590106b90 2024-04-24T23:03:13.406 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.728+0000 7f958dffb700 1 -- 172.21.15.33:0/2788861665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9588009af0 con 0x7f9590106b90 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 -- 172.21.15.33:0/2788861665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 msgr2=0x7f9590106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 --2- 172.21.15.33:0/2788861665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590106fb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f958800d180 tx=0x7f958800d490 comp rx=0 tx=0).stop 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 -- 172.21.15.33:0/2788861665 shutdown_connections 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 --2- 172.21.15.33:0/2788861665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590106fb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 -- 172.21.15.33:0/2788861665 >> 172.21.15.33:0/2788861665 conn(0x7f9590102110 msgr2=0x7f9590104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 -- 172.21.15.33:0/2788861665 shutdown_connections 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 -- 172.21.15.33:0/2788861665 wait complete. 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.729+0000 7f9595303700 1 Processor -- start 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f9595303700 1 -- start start 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f9595303700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f9595303700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f959007af50 con 0x7f9590106b90 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f958effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:13.407 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f958effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36274/0 (socket says 172.21.15.33:36274) 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f958effd700 1 -- 172.21.15.33:0/3544345676 learned_addr learned my addr 172.21.15.33:0/3544345676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f958effd700 1 -- 172.21.15.33:0/3544345676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9590199f80 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.730+0000 7f958effd700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590199a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9588003c50 tx=0x7f9588003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.731+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9588023020 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.731+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958800b200 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.731+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959019a210 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.731+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9588019040 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.731+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959019cf00 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.731+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9588026020 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.732+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9590063890 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.732+0000 7f9587fff700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f957c039150 0x7f957c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.732+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9588011050 con 0x7f9590106b90 2024-04-24T23:03:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.732+0000 7f958e7fc700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f957c039150 0x7f957c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.732+0000 7f958e7fc700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f957c039150 0x7f957c03b610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9578006fd0 tx=0x7f9578006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.737+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f958802d020 con 0x7f9590106b90 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.943+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f959002ddf0 con 0x7f9590106b90 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.943+0000 7f9587fff700 1 -- 172.21.15.33:0/3544345676 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f958801ec10 con 0x7f9590106b90 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f957c039150 msgr2=0x7f957c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f957c039150 0x7f957c03b610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9578006fd0 tx=0x7f9578006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 msgr2=0x7f9590199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590199a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9588003c50 tx=0x7f9588003d30 comp rx=0 tx=0).stop 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 shutdown_connections 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f957c039150 0x7f957c03b610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 --2- 172.21.15.33:0/3544345676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590106b90 0x7f9590199a40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 >> 172.21.15.33:0/3544345676 conn(0x7f9590102110 msgr2=0x7f9590190560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 shutdown_connections 2024-04-24T23:03:13.409 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.946+0000 7f9595303700 1 -- 172.21.15.33:0/3544345676 wait complete. 2024-04-24T23:03:14.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.848+0000 7fb4798b3700 1 Processor -- start 2024-04-24T23:03:14.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb4798b3700 1 -- start start 2024-04-24T23:03:14.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb4798b3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb474108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb4798b3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741086d0 con 0x7fb474107ce0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb472ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb474108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb472ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb474108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36290/0 (socket says 172.21.15.33:36290) 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb472ffd700 1 -- 172.21.15.33:0/3243475870 learned_addr learned my addr 172.21.15.33:0/3243475870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb472ffd700 1 -- 172.21.15.33:0/3243475870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb474108e40 con 0x7fb474107ce0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb472ffd700 1 --2- 172.21.15.33:0/3243475870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb474108100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4640098a0 tx=0x7fb464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f506910f3024aa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.849+0000 7fb471ffb700 1 -- 172.21.15.33:0/3243475870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb464004160 con 0x7fb474107ce0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb471ffb700 1 -- 172.21.15.33:0/3243475870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb464004300 con 0x7fb474107ce0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb471ffb700 1 -- 172.21.15.33:0/3243475870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb464011b20 con 0x7fb474107ce0 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 -- 172.21.15.33:0/3243475870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 msgr2=0x7fb474108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:14.758 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 --2- 172.21.15.33:0/3243475870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb474108100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4640098a0 tx=0x7fb464009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 -- 172.21.15.33:0/3243475870 shutdown_connections 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 --2- 172.21.15.33:0/3243475870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb474108100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 -- 172.21.15.33:0/3243475870 >> 172.21.15.33:0/3243475870 conn(0x7fb474103300 msgr2=0x7fb474105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 -- 172.21.15.33:0/3243475870 shutdown_connections 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.850+0000 7fb4798b3700 1 -- 172.21.15.33:0/3243475870 wait complete. 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb4798b3700 1 Processor -- start 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb4798b3700 1 -- start start 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb4798b3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb47407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb4798b3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb47410c1f0 con 0x7fb474107ce0 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb472ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb47407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:14.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb472ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb47407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36304/0 (socket says 172.21.15.33:36304) 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb472ffd700 1 -- 172.21.15.33:0/3111888223 learned_addr learned my addr 172.21.15.33:0/3111888223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.851+0000 7fb472ffd700 1 -- 172.21.15.33:0/3111888223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47407e030 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.852+0000 7fb472ffd700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb47407daf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4640117e0 tx=0x7fb464011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.852+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb464003bf0 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.852+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46401a910 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.852+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4640249a0 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.852+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47407c220 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.852+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47407c6e0 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.853+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb464022020 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.853+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb474063890 con 0x7fb474107ce0 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.853+0000 7fb4788b1700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb45c0390f0 0x7fb45c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:14.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.853+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb46404d540 con 0x7fb474107ce0 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.853+0000 7fb4727fc700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb45c0390f0 0x7fb45c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.854+0000 7fb4727fc700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb45c0390f0 0x7fb45c03b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb468006fd0 tx=0x7fb468006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:13.858+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb464011e20 con 0x7fb474107ce0 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.181+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb47407c960 con 0x7fb474107ce0 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.184+0000 7fb4788b1700 1 -- 172.21.15.33:0/3111888223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb464004300 con 0x7fb474107ce0 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb45c0390f0 msgr2=0x7fb45c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb45c0390f0 0x7fb45c03b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb468006fd0 tx=0x7fb468006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 msgr2=0x7fb47407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb47407daf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4640117e0 tx=0x7fb464011c10 comp rx=0 tx=0).stop 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 shutdown_connections 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb45c0390f0 0x7fb45c03b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 --2- 172.21.15.33:0/3111888223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb474107ce0 0x7fb47407daf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 >> 172.21.15.33:0/3111888223 conn(0x7fb474103300 msgr2=0x7fb474103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 shutdown_connections 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.186+0000 7fb4798b3700 1 -- 172.21.15.33:0/3111888223 wait complete. 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T23:03:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:14.763 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-04-24T23:03:14.794 INFO:tasks.cephadm:Fetching config... 2024-04-24T23:03:14.795 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:03:14.795 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T23:03:14.811 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T23:03:14.811 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:03:14.811 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T23:03:14.866 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T23:03:14.867 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:03:14.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T23:03:14.931 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T23:03:14.931 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:03:14.931 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T23:03:14.987 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T23:03:14.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:03:15.061 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:15.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:03:15.113 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:15.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:03:15.164 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZDmlaAuMI/z7FuINpJa7By2t6T9/c4VyfBl1S7IadVoDoJV2bn3fIxEC8FmKkpwmAdByeWBaM1oz/fUjl07Vxc2cTlQHd/ZY3q1ydo6D4uZ2TotqNuSdDG6q2BMbA6v0M8SOAR8zyvq/RDfU1G6w/xeuC57eJmTGq2Pl03ea2lHaHEwFjVYasrndfyXaD31FjwaS+UtAIpF5L5+ldaTQgnDWdY3XOi2vffo09OW/a7ElyihsDKYhsKNdGw8upt0qqPJbqGNGvgtKbP4asRxZRq/yZqd5x5FTTKYgE+83oTBnKAHg94aVjcdKK6PkIK7s7h6MyzeYADWwDyWFslZASYNxRlN7yt7y5yc2/gLT8TlyFUxmSzWfIl/QgYqXm8foHzrrhdNcJriq0kOO5urWzPDBUpTJG8SwDQaxUFHh2l68j6mhKg8045icaaF9Vok2yRcMZHcS2Moze0VUtwP4UonTEfAD2KvKWgr99e37IFH1WeT+JgPlRcrWGW2Rujz0= ceph-8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:15.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T23:03:15.302 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:15 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3111888223' entity='client.admin' 2024-04-24T23:03:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 -- 172.21.15.33:0/90680204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 msgr2=0x7f0ce4104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 --2- 172.21.15.33:0/90680204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4104bd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009910 tx=0x7f0cd4009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 -- 172.21.15.33:0/90680204 shutdown_connections 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 --2- 172.21.15.33:0/90680204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4104bd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 -- 172.21.15.33:0/90680204 >> 172.21.15.33:0/90680204 conn(0x7f0ce40fc130 msgr2=0x7f0ce40fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 -- 172.21.15.33:0/90680204 shutdown_connections 2024-04-24T23:03:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.639+0000 7f0ce9186700 1 -- 172.21.15.33:0/90680204 wait complete. 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce9186700 1 Processor -- start 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce9186700 1 -- start start 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce9186700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce9186700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4107d20 con 0x7f0ce41027e0 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36962/0 (socket says 172.21.15.33:36962) 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.640+0000 7f0ce2d9d700 1 -- 172.21.15.33:0/534553139 learned_addr learned my addr 172.21.15.33:0/534553139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ce2d9d700 1 -- 172.21.15.33:0/534553139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce4197850 con 0x7f0ce41027e0 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ce2d9d700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4197310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4005950 tx=0x7f0cd4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd4026070 con 0x7f0ce41027e0 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd400bc30 con 0x7f0ce41027e0 2024-04-24T23:03:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce4197ae0 con 0x7f0ce41027e0 2024-04-24T23:03:17.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd4003b30 con 0x7f0ce41027e0 2024-04-24T23:03:17.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.641+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce419a7d0 con 0x7f0ce41027e0 2024-04-24T23:03:17.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.642+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0cd4029070 con 0x7f0ce41027e0 2024-04-24T23:03:17.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.642+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce41084a0 con 0x7f0ce41027e0 2024-04-24T23:03:17.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.642+0000 7f0ccbfff700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ccc0390f0 0x7f0ccc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:17.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.642+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0cd401e070 con 0x7f0ce41027e0 2024-04-24T23:03:17.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.642+0000 7f0ce259c700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ccc0390f0 0x7f0ccc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:17.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.643+0000 7f0ce259c700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ccc0390f0 0x7f0ccc03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006fd0 tx=0x7f0cd8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:17.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.647+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0cd4013070 con 0x7f0ce41027e0 2024-04-24T23:03:17.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.853+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0ce402dc40 con 0x7f0ce41027e0 2024-04-24T23:03:17.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.855+0000 7f0ccbfff700 1 -- 172.21.15.33:0/534553139 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0cd4030070 con 0x7f0ce41027e0 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ccc0390f0 msgr2=0x7f0ccc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ccc0390f0 0x7f0ccc03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006fd0 tx=0x7f0cd8006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 msgr2=0x7f0ce4197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4197310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4005950 tx=0x7f0cd4005030 comp rx=0 tx=0).stop 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 shutdown_connections 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ccc0390f0 0x7f0ccc03b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 --2- 172.21.15.33:0/534553139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce41027e0 0x7f0ce4197310 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 >> 172.21.15.33:0/534553139 conn(0x7f0ce40fc130 msgr2=0x7f0ce40fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 shutdown_connections 2024-04-24T23:03:17.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:17.858+0000 7f0ce9186700 1 -- 172.21.15.33:0/534553139 wait complete. 2024-04-24T23:03:18.734 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T23:03:18.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T23:03:18.973 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/534553139' entity='client.admin' 2024-04-24T23:03:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 -- 172.21.15.33:0/2184047739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 msgr2=0x7f18b8104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 --2- 172.21.15.33:0/2184047739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8104b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009910 tx=0x7f18b0009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 -- 172.21.15.33:0/2184047739 shutdown_connections 2024-04-24T23:03:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 --2- 172.21.15.33:0/2184047739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8104b10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 -- 172.21.15.33:0/2184047739 >> 172.21.15.33:0/2184047739 conn(0x7f18b80fc130 msgr2=0x7f18b80fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 -- 172.21.15.33:0/2184047739 shutdown_connections 2024-04-24T23:03:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.076+0000 7f18bd707700 1 -- 172.21.15.33:0/2184047739 wait complete. 2024-04-24T23:03:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18bd707700 1 Processor -- start 2024-04-24T23:03:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18bd707700 1 -- start start 2024-04-24T23:03:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18bd707700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18bd707700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8107cc0 con 0x7f18b8102720 2024-04-24T23:03:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36980/0 (socket says 172.21.15.33:36980) 2024-04-24T23:03:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18b6ffd700 1 -- 172.21.15.33:0/2683523215 learned_addr learned my addr 172.21.15.33:0/2683523215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.077+0000 7f18b6ffd700 1 -- 172.21.15.33:0/2683523215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b8199a60 con 0x7f18b8102720 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.078+0000 7f18b6ffd700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8199520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18b0004fa0 tx=0x7f18b0005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.078+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b0027070 con 0x7f18b8102720 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.078+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b000be40 con 0x7f18b8102720 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.078+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b8199d50 con 0x7f18b8102720 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.078+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b000f460 con 0x7f18b8102720 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.078+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b819c9e0 con 0x7f18b8102720 2024-04-24T23:03:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.079+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f18b002a070 con 0x7f18b8102720 2024-04-24T23:03:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.079+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8108440 con 0x7f18b8102720 2024-04-24T23:03:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.079+0000 7f18affff700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18a40390f0 0x7f18a403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.079+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f18b001e030 con 0x7f18b8102720 2024-04-24T23:03:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.079+0000 7f18b67fc700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18a40390f0 0x7f18a403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.079+0000 7f18b67fc700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18a40390f0 0x7f18a403b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006fd0 tx=0x7f18a0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.084+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f18b0013070 con 0x7f18b8102720 2024-04-24T23:03:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.295+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f18b802dd20 con 0x7f18a40390f0 2024-04-24T23:03:21.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.298+0000 7f18affff700 1 -- 172.21.15.33:0/2683523215 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18b802dd20 con 0x7f18a40390f0 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18a40390f0 msgr2=0x7f18a403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18a40390f0 0x7f18a403b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006fd0 tx=0x7f18a0006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 msgr2=0x7f18b8199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8199520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18b0004fa0 tx=0x7f18b0005080 comp rx=0 tx=0).stop 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 shutdown_connections 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18a40390f0 0x7f18a403b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 --2- 172.21.15.33:0/2683523215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b8102720 0x7f18b8199520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 >> 172.21.15.33:0/2683523215 conn(0x7f18b80fc130 msgr2=0x7f18b80fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 shutdown_connections 2024-04-24T23:03:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:21.300+0000 7f18bd707700 1 -- 172.21.15.33:0/2683523215 wait complete. 2024-04-24T23:03:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:03:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-24T23:03:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:21 smithi033 ceph-mon[29201]: Updating smithi033:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:22.005 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi038 2024-04-24T23:03:22.005 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:03:22.005 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:03:22.025 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:03:22.025 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:22.084 INFO:tasks.cephadm:Adding host smithi038 to orchestrator... 2024-04-24T23:03:22.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi038 2024-04-24T23:03:22.327 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:23.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:23 smithi033 ceph-mon[29201]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:23.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:23 smithi033 ceph-mon[29201]: Updating smithi033:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:03:23.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:23.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:23.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.852+0000 7f4674d20700 1 -- 172.21.15.33:0/873531472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 msgr2=0x7f4670102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.852+0000 7f4674d20700 1 --2- 172.21.15.33:0/873531472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f4670102280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4660009910 tx=0x7f4660009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.852+0000 7f4674d20700 1 -- 172.21.15.33:0/873531472 shutdown_connections 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.852+0000 7f4674d20700 1 --2- 172.21.15.33:0/873531472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f4670102280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.852+0000 7f4674d20700 1 -- 172.21.15.33:0/873531472 >> 172.21.15.33:0/873531472 conn(0x7f46700fd770 msgr2=0x7f46700ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.852+0000 7f4674d20700 1 -- 172.21.15.33:0/873531472 shutdown_connections 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.853+0000 7f4674d20700 1 -- 172.21.15.33:0/873531472 wait complete. 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.853+0000 7f4674d20700 1 Processor -- start 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.853+0000 7f4674d20700 1 -- start start 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.853+0000 7f4674d20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f467019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.853+0000 7f4674d20700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f467010c270 con 0x7f4670101ea0 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f466e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f467019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f466e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f467019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36998/0 (socket says 172.21.15.33:36998) 2024-04-24T23:03:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f466e59c700 1 -- 172.21.15.33:0/1272290282 learned_addr learned my addr 172.21.15.33:0/1272290282 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f466e59c700 1 -- 172.21.15.33:0/1272290282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467019dab0 con 0x7f4670101ea0 2024-04-24T23:03:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f466e59c700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f467019d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4660006010 tx=0x7f4660005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4660026070 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467019dda0 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466000bbd0 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4660003910 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.854+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4670197810 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.855+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4660029070 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.855+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4670103380 con 0x7f4670101ea0 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.855+0000 7f465f7fe700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46580390f0 0x7f465803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.855+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f466001e070 con 0x7f4670101ea0 2024-04-24T23:03:23.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.855+0000 7f466dd9b700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46580390f0 0x7f465803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:23.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.856+0000 7f466dd9b700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46580390f0 0x7f465803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4664006fd0 tx=0x7f4664006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:23.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:23.860+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4660013070 con 0x7f4670101ea0 2024-04-24T23:03:24.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:24.070+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "target": ["mon-mgr", ""]}) -- 0x7f4670198b70 con 0x7f46580390f0 2024-04-24T23:03:24.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:24 smithi033 ceph-mon[29201]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:25.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:25 smithi033 ceph-mon[29201]: Deploying cephadm binary to smithi038 2024-04-24T23:03:26.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:26 smithi033 ceph-mon[29201]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:28.301 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi038' with addr '172.21.15.38' 2024-04-24T23:03:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.298+0000 7f465f7fe700 1 -- 172.21.15.33:0/1272290282 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4670198b70 con 0x7f46580390f0 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46580390f0 msgr2=0x7f465803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46580390f0 0x7f465803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4664006fd0 tx=0x7f4664006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 msgr2=0x7f467019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f467019d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4660006010 tx=0x7f4660005030 comp rx=0 tx=0).stop 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 shutdown_connections 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46580390f0 0x7f465803b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 --2- 172.21.15.33:0/1272290282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4670101ea0 0x7f467019d570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 >> 172.21.15.33:0/1272290282 conn(0x7f46700fd770 msgr2=0x7f4670104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 shutdown_connections 2024-04-24T23:03:28.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:28.302+0000 7f4674d20700 1 -- 172.21.15.33:0/1272290282 wait complete. 2024-04-24T23:03:28.887 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:03:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:29.533 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:29 smithi033 ceph-mon[29201]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:29.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:29 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:29.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:29 smithi033 ceph-mon[29201]: Added host smithi038 2024-04-24T23:03:29.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:29 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:29.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:29 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:30.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 -- 172.21.15.33:0/4264417884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 msgr2=0x7f76b81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 --2- 172.21.15.33:0/4264417884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b81011c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009910 tx=0x7f76a8009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 -- 172.21.15.33:0/4264417884 shutdown_connections 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 --2- 172.21.15.33:0/4264417884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b81011c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 -- 172.21.15.33:0/4264417884 >> 172.21.15.33:0/4264417884 conn(0x7f76b80fc690 msgr2=0x7f76b80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 -- 172.21.15.33:0/4264417884 shutdown_connections 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.729+0000 7f76bd462700 1 -- 172.21.15.33:0/4264417884 wait complete. 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76bd462700 1 Processor -- start 2024-04-24T23:03:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76bd462700 1 -- start start 2024-04-24T23:03:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76bd462700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76bd462700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810b1b0 con 0x7f76b8100de0 2024-04-24T23:03:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51896/0 (socket says 172.21.15.33:51896) 2024-04-24T23:03:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.730+0000 7f76b6ffd700 1 -- 172.21.15.33:0/4181721605 learned_addr learned my addr 172.21.15.33:0/4181721605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f76b6ffd700 1 -- 172.21.15.33:0/4181721605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b819d920 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f76b6ffd700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b819d3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005850 tx=0x7f76a8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76a8026070 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b819dc10 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a800bbd0 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76a8003970 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.731+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81975a0 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.732+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f76a8029070 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.732+0000 7f769ffff700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76a00390f0 0x7f76a003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.732+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b81022c0 con 0x7f76b8100de0 2024-04-24T23:03:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.732+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f76a801e070 con 0x7f76b8100de0 2024-04-24T23:03:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.732+0000 7f76b67fc700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76a00390f0 0x7f76a003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.732+0000 7f76b67fc700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76a00390f0 0x7f76a003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f76ac006fd0 tx=0x7f76ac006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.737+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76a8030070 con 0x7f76b8100de0 2024-04-24T23:03:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.947+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76b8198ad0 con 0x7f76a00390f0 2024-04-24T23:03:30.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.948+0000 7f769ffff700 1 -- 172.21.15.33:0/4181721605 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f76b8198ad0 con 0x7f76a00390f0 2024-04-24T23:03:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:30.949 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}] 2024-04-24T23:03:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.950+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76a00390f0 msgr2=0x7f76a003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.950+0000 7f76bd462700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76a00390f0 0x7f76a003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f76ac006fd0 tx=0x7f76ac006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.950+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 msgr2=0x7f76b819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.950+0000 7f76bd462700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b819d3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005850 tx=0x7f76a8005030 comp rx=0 tx=0).stop 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.951+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 shutdown_connections 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.951+0000 7f76bd462700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76a00390f0 0x7f76a003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.951+0000 7f76bd462700 1 --2- 172.21.15.33:0/4181721605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b8100de0 0x7f76b819d3e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.951+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 >> 172.21.15.33:0/4181721605 conn(0x7f76b80fc690 msgr2=0x7f76b806a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.951+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 shutdown_connections 2024-04-24T23:03:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:30.951+0000 7f76bd462700 1 -- 172.21.15.33:0/4181721605 wait complete. 2024-04-24T23:03:31.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:31 smithi033 ceph-mon[29201]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:31.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:31.624 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-04-24T23:03:31.624 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:03:31.624 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:03:31.642 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:03:31.643 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:31.699 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-04-24T23:03:31.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi117 2024-04-24T23:03:31.945 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:32.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:32 smithi033 ceph-mon[29201]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:03:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 -- 172.21.15.33:0/1124050558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 msgr2=0x7f34001011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 --2- 172.21.15.33:0/1124050558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f34001011c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009910 tx=0x7f33f0009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 -- 172.21.15.33:0/1124050558 shutdown_connections 2024-04-24T23:03:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 --2- 172.21.15.33:0/1124050558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f34001011c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 -- 172.21.15.33:0/1124050558 >> 172.21.15.33:0/1124050558 conn(0x7f34000fc690 msgr2=0x7f34000feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 -- 172.21.15.33:0/1124050558 shutdown_connections 2024-04-24T23:03:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.443+0000 7f34051cb700 1 -- 172.21.15.33:0/1124050558 wait complete. 2024-04-24T23:03:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.444+0000 7f34051cb700 1 Processor -- start 2024-04-24T23:03:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.444+0000 7f34051cb700 1 -- start start 2024-04-24T23:03:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f34051cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f3400197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f34051cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f340010b1b0 con 0x7f3400100de0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f33fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f3400197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f33fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f3400197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51910/0 (socket says 172.21.15.33:51910) 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f33fed9d700 1 -- 172.21.15.33:0/3189106986 learned_addr learned my addr 172.21.15.33:0/3189106986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f33fed9d700 1 -- 172.21.15.33:0/3189106986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3400197c90 con 0x7f3400100de0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f33fed9d700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f3400197750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005850 tx=0x7f33f0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.445+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33f0026070 con 0x7f3400100de0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.446+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f000bbd0 con 0x7f3400100de0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.446+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33f0003970 con 0x7f3400100de0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.446+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3400197f80 con 0x7f3400100de0 2024-04-24T23:03:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.446+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340019c020 con 0x7f3400100de0 2024-04-24T23:03:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.446+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f33f0029070 con 0x7f3400100de0 2024-04-24T23:03:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.447+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34001022c0 con 0x7f3400100de0 2024-04-24T23:03:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.447+0000 7f33e7fff700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33e80390f0 0x7f33e803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.447+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f33f001e070 con 0x7f3400100de0 2024-04-24T23:03:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.447+0000 7f33fe59c700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33e80390f0 0x7f33e803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.447+0000 7f33fe59c700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33e80390f0 0x7f33e803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:33.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.452+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f33f0013070 con 0x7f3400100de0 2024-04-24T23:03:33.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:33.662+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) -- 0x7f3400198ab0 con 0x7f33e80390f0 2024-04-24T23:03:33.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:33 smithi033 ceph-mon[29201]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:35.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:35.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: Deploying cephadm binary to smithi117 2024-04-24T23:03:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:35 smithi033 ceph-mon[29201]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:36.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:36 smithi033 ceph-mon[29201]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:37 smithi033 ceph-mon[29201]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.931+0000 7f33e7fff700 1 -- 172.21.15.33:0/3189106986 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3400198ab0 con 0x7f33e80390f0 2024-04-24T23:03:37.932 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33e80390f0 msgr2=0x7f33e803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33e80390f0 0x7f33e803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f4006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 msgr2=0x7f3400197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f3400197750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005850 tx=0x7f33f0005030 comp rx=0 tx=0).stop 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 shutdown_connections 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33e80390f0 0x7f33e803b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 --2- 172.21.15.33:0/3189106986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3400100de0 0x7f3400197750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 >> 172.21.15.33:0/3189106986 conn(0x7f34000fc690 msgr2=0x7f340006a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 shutdown_connections 2024-04-24T23:03:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:37.934+0000 7f34051cb700 1 -- 172.21.15.33:0/3189106986 wait complete. 2024-04-24T23:03:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:03:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:38.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:38.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:03:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:39.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:39 smithi033 ceph-mon[29201]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:39.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:39 smithi033 ceph-mon[29201]: Added host smithi117 2024-04-24T23:03:39.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 -- 172.21.15.33:0/1905628415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 msgr2=0x7f3f20102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 --2- 172.21.15.33:0/1905628415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f20102280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009910 tx=0x7f3f10009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 -- 172.21.15.33:0/1905628415 shutdown_connections 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 --2- 172.21.15.33:0/1905628415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f20102280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 -- 172.21.15.33:0/1905628415 >> 172.21.15.33:0/1905628415 conn(0x7f3f200fd770 msgr2=0x7f3f200ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 -- 172.21.15.33:0/1905628415 shutdown_connections 2024-04-24T23:03:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.345+0000 7f3f26467700 1 -- 172.21.15.33:0/1905628415 wait complete. 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f26467700 1 Processor -- start 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f26467700 1 -- start start 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f26467700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f2019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f26467700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010c270 con 0x7f3f20101ea0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f2019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f2019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35876/0 (socket says 172.21.15.33:35876) 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.346+0000 7f3f1ffff700 1 -- 172.21.15.33:0/3144581944 learned_addr learned my addr 172.21.15.33:0/3144581944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f1ffff700 1 -- 172.21.15.33:0/3144581944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2019d930 con 0x7f3f20101ea0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f1ffff700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f2019d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005850 tx=0x7f3f10005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f10026070 con 0x7f3f20101ea0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2019dc20 con 0x7f3f20101ea0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f1000bbd0 con 0x7f3f20101ea0 2024-04-24T23:03:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f10003970 con 0x7f3f20101ea0 2024-04-24T23:03:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.347+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201975b0 con 0x7f3f20101ea0 2024-04-24T23:03:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.348+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3f10029070 con 0x7f3f20101ea0 2024-04-24T23:03:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.348+0000 7f3f1d7fa700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f0c0390f0 0x7f3f0c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.348+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f1001e070 con 0x7f3f20101ea0 2024-04-24T23:03:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.348+0000 7f3f1f7fe700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f0c0390f0 0x7f3f0c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.349+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f20103380 con 0x7f3f20101ea0 2024-04-24T23:03:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.349+0000 7f3f1f7fe700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f0c0390f0 0x7f3f0c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f08006fd0 tx=0x7f3f08006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:40.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.354+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f10013070 con 0x7f3f20101ea0 2024-04-24T23:03:40.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.566+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f20198ae0 con 0x7f3f0c0390f0 2024-04-24T23:03:40.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.567+0000 7f3f1d7fa700 1 -- 172.21.15.33:0/3144581944 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f3f20198ae0 con 0x7f3f0c0390f0 2024-04-24T23:03:40.569 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:03:40.569 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f0c0390f0 msgr2=0x7f3f0c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f0c0390f0 0x7f3f0c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f08006fd0 tx=0x7f3f08006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 msgr2=0x7f3f2019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f2019d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005850 tx=0x7f3f10005030 comp rx=0 tx=0).stop 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 shutdown_connections 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f0c0390f0 0x7f3f0c03b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 --2- 172.21.15.33:0/3144581944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f20101ea0 0x7f3f2019d3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:40.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 >> 172.21.15.33:0/3144581944 conn(0x7f3f200fd770 msgr2=0x7f3f20103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:40.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 shutdown_connections 2024-04-24T23:03:40.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:40.569+0000 7f3f26467700 1 -- 172.21.15.33:0/3144581944 wait complete. 2024-04-24T23:03:41.156 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T23:03:41.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T23:03:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:41.788 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:41 smithi033 ceph-mon[29201]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:41.788 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:41 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:41.788 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:41 smithi033 ceph-mon[29201]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 -- 172.21.15.33:0/1969284589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 msgr2=0x7f8f0c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 --2- 172.21.15.33:0/1969284589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c1022b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009910 tx=0x7f8f08009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 -- 172.21.15.33:0/1969284589 shutdown_connections 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 --2- 172.21.15.33:0/1969284589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c1022b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 -- 172.21.15.33:0/1969284589 >> 172.21.15.33:0/1969284589 conn(0x7f8f0c0fd760 msgr2=0x7f8f0c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 -- 172.21.15.33:0/1969284589 shutdown_connections 2024-04-24T23:03:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 -- 172.21.15.33:0/1969284589 wait complete. 2024-04-24T23:03:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 Processor -- start 2024-04-24T23:03:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.905+0000 7f8f14753700 1 -- start start 2024-04-24T23:03:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f14753700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f14753700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10c2a0 con 0x7f8f0c101ed0 2024-04-24T23:03:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f124ef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f124ef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35894/0 (socket says 172.21.15.33:35894) 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f124ef700 1 -- 172.21.15.33:0/1260641573 learned_addr learned my addr 172.21.15.33:0/1260641573 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f124ef700 1 -- 172.21.15.33:0/1260641573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c076420 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f124ef700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c074530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006010 tx=0x7f8f08005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f08026070 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0800bc30 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c074ad0 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f080039d0 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.906+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c074f90 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.907+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8f08029070 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.907+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c1033b0 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.907+0000 7f8f037fe700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ef80390f0 0x7f8ef803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.908+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f0801e070 con 0x7f8f0c101ed0 2024-04-24T23:03:42.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.908+0000 7f8f11cee700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ef80390f0 0x7f8ef803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.908+0000 7f8f11cee700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ef80390f0 0x7f8ef803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8efc006fd0 tx=0x7f8efc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:42.912+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f08013070 con 0x7f8f0c101ed0 2024-04-24T23:03:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.125+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8f0c075b00 con 0x7f8f0c101ed0 2024-04-24T23:03:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.557+0000 7f8f037fe700 1 -- 172.21.15.33:0/1260641573 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8f08030090 con 0x7f8f0c101ed0 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ef80390f0 msgr2=0x7f8ef803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ef80390f0 0x7f8ef803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8efc006fd0 tx=0x7f8efc006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 msgr2=0x7f8f0c074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c074530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006010 tx=0x7f8f08005030 comp rx=0 tx=0).stop 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 shutdown_connections 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ef80390f0 0x7f8ef803b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 --2- 172.21.15.33:0/1260641573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c101ed0 0x7f8f0c074530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 >> 172.21.15.33:0/1260641573 conn(0x7f8f0c0fd760 msgr2=0x7f8f0c104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 shutdown_connections 2024-04-24T23:03:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:03:43.560+0000 7f8f14753700 1 -- 172.21.15.33:0/1260641573 wait complete. 2024-04-24T23:03:43.563 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-04-24T23:03:43.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:43 smithi033 ceph-mon[29201]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:43.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:43 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1260641573' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:03:44.254 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-04-24T23:03:44.254 INFO:tasks.cephadm:Adding mon.b on smithi038 2024-04-24T23:03:44.254 INFO:tasks.cephadm:Adding mon.c on smithi117 2024-04-24T23:03:44.254 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c' 2024-04-24T23:03:44.485 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1260641573' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:03:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:44.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:45.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.936+0000 7f262dec9700 1 -- 172.21.15.117:0/4040595904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 msgr2=0x7f2628102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.936+0000 7f262dec9700 1 --2- 172.21.15.117:0/4040595904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f2628102280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f261800b0a0 tx=0x7f261800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.936+0000 7f262dec9700 1 -- 172.21.15.117:0/4040595904 shutdown_connections 2024-04-24T23:03:45.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.936+0000 7f262dec9700 1 --2- 172.21.15.117:0/4040595904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f2628102280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.936+0000 7f262dec9700 1 -- 172.21.15.117:0/4040595904 >> 172.21.15.117:0/4040595904 conn(0x7f26280fd770 msgr2=0x7f26280ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.936+0000 7f262dec9700 1 -- 172.21.15.117:0/4040595904 shutdown_connections 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.937+0000 7f262dec9700 1 -- 172.21.15.117:0/4040595904 wait complete. 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.937+0000 7f262dec9700 1 Processor -- start 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.937+0000 7f262dec9700 1 -- start start 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.937+0000 7f262dec9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f262819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.937+0000 7f262dec9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f262810c270 con 0x7f2628101ea0 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f26277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f262819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f26277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f262819d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.117:42296/0 (socket says 172.21.15.117:42296) 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f26277fe700 1 -- 172.21.15.117:0/1744114054 learned_addr learned my addr 172.21.15.117:0/1744114054 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:03:45.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f26277fe700 1 -- 172.21.15.117:0/1744114054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262819d910 con 0x7f2628101ea0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f26277fe700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f262819d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26180093e0 tx=0x7f2618003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2618017040 con 0x7f2628101ea0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262819dba0 con 0x7f2628101ea0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2618022070 con 0x7f2628101ea0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f261801d440 con 0x7f2628101ea0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.938+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2628197590 con 0x7f2628101ea0 2024-04-24T23:03:45.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.939+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2618004000 con 0x7f2628101ea0 2024-04-24T23:03:45.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.939+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2628103380 con 0x7f2628101ea0 2024-04-24T23:03:45.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.939+0000 7f2624ff9700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2610039140 0x7f261003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.939+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f261800f050 con 0x7f2628101ea0 2024-04-24T23:03:45.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.940+0000 7f2626ffd700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2610039140 0x7f261003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.940+0000 7f2626ffd700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2610039140 0x7f261003b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:45.945+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2618007e20 con 0x7f2628101ea0 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:45 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.150+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}) -- 0x7f2628198ac0 con 0x7f2610039140 2024-04-24T23:03:46.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.157+0000 7f2624ff9700 1 -- 172.21.15.117:0/1744114054 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2628198ac0 con 0x7f2610039140 2024-04-24T23:03:46.159 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mon update... 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2610039140 msgr2=0x7f261003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2610039140 0x7f261003b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 msgr2=0x7f262819d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f262819d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26180093e0 tx=0x7f2618003ce0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 shutdown_connections 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2610039140 0x7f261003b600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 --2- 172.21.15.117:0/1744114054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2628101ea0 0x7f262819d3d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 >> 172.21.15.117:0/1744114054 conn(0x7f26280fd770 msgr2=0x7f2628103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 shutdown_connections 2024-04-24T23:03:46.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:46.159+0000 7f262dec9700 1 -- 172.21.15.117:0/1744114054 wait complete. 2024-04-24T23:03:46.759 DEBUG:teuthology.orchestra.run.smithi038:mon.b> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T23:03:46.762 DEBUG:teuthology.orchestra.run.smithi117:mon.c> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T23:03:46.764 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:03:46.764 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:03:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c;count:3 2024-04-24T23:03:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:47.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:47 smithi033 ceph-mon[29201]: Deploying daemon mon.c on smithi117 2024-04-24T23:03:47.929 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:49.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:49 smithi033 ceph-mon[29201]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:50.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- 172.21.15.117:0/2953327998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 msgr2=0x7f5ccc1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 --2- 172.21.15.117:0/2953327998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc1080b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009910 tx=0x7f5cc8009c20 comp rx=0 tx=0).stop 2024-04-24T23:03:50.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- 172.21.15.117:0/2953327998 shutdown_connections 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 --2- 172.21.15.117:0/2953327998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc1080b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- 172.21.15.117:0/2953327998 >> 172.21.15.117:0/2953327998 conn(0x7f5ccc076690 msgr2=0x7f5ccc076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- 172.21.15.117:0/2953327998 shutdown_connections 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- 172.21.15.117:0/2953327998 wait complete. 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 Processor -- start 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- start start 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.671+0000 7f5cd2e93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10c260 con 0x7f5ccc107cd0 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cd0c2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cd0c2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.117:45330/0 (socket says 172.21.15.117:45330) 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cd0c2f700 1 -- 172.21.15.117:0/401934189 learned_addr learned my addr 172.21.15.117:0/401934189 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:03:50.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cd0c2f700 1 -- 172.21.15.117:0/401934189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc19d910 con 0x7f5ccc107cd0 2024-04-24T23:03:50.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cd0c2f700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc19d3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005850 tx=0x7f5cc8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:50.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8026070 con 0x7f5ccc107cd0 2024-04-24T23:03:50.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc19dc00 con 0x7f5ccc107cd0 2024-04-24T23:03:50.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc800bbd0 con 0x7f5ccc107cd0 2024-04-24T23:03:50.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.672+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8003970 con 0x7f5ccc107cd0 2024-04-24T23:03:50.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.673+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc197590 con 0x7f5ccc107cd0 2024-04-24T23:03:50.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.673+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc103380 con 0x7f5ccc107cd0 2024-04-24T23:03:50.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.673+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5cc8029070 con 0x7f5ccc107cd0 2024-04-24T23:03:50.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.674+0000 7f5cc1ffb700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cb40390f0 0x7f5cb403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.674+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5cc801e070 con 0x7f5ccc107cd0 2024-04-24T23:03:50.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.674+0000 7f5cc3fff700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cb40390f0 0x7f5cb403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.674+0000 7f5cc3fff700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cb40390f0 0x7f5cb403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006fd0 tx=0x7f5cbc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:50.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.679+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5cc8013070 con 0x7f5ccc107cd0 2024-04-24T23:03:50.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:50 smithi117 podman[32497]: 2024-04-24 23:03:50.865367571 +0000 UTC m=+0.128555324 container create 266d12945107cf0b688b0f80da17a8bed1535bb40976ec93c3b99951ebdda814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T23:03:50.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.980+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ccc063890 con 0x7f5ccc107cd0 2024-04-24T23:03:50.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.981+0000 7f5cc1ffb700 1 -- 172.21.15.117:0/401934189 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f5cc8030020 con 0x7f5ccc107cd0 2024-04-24T23:03:50.983 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:03:50.983 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:02:06.759607Z","created":"2024-04-24T23:02:06.759607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cb40390f0 msgr2=0x7f5cb403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cb40390f0 0x7f5cb403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006fd0 tx=0x7f5cbc006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 msgr2=0x7f5ccc19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc19d3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005850 tx=0x7f5cc8005030 comp rx=0 tx=0).stop 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 shutdown_connections 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cb40390f0 0x7f5cb403b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 --2- 172.21.15.117:0/401934189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc107cd0 0x7f5ccc19d3d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 >> 172.21.15.117:0/401934189 conn(0x7f5ccc076690 msgr2=0x7f5ccc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 shutdown_connections 2024-04-24T23:03:50.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:50.983+0000 7f5cd2e93700 1 -- 172.21.15.117:0/401934189 wait complete. 2024-04-24T23:03:50.986 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-04-24T23:03:51.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 podman[32497]: 2024-04-24 23:03:51.043406418 +0000 UTC m=+0.306594177 container init 266d12945107cf0b688b0f80da17a8bed1535bb40976ec93c3b99951ebdda814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 podman[32497]: 2024-04-24 23:03:51.04664952 +0000 UTC m=+0.309837277 container start 266d12945107cf0b688b0f80da17a8bed1535bb40976ec93c3b99951ebdda814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pidfile_write: ignore empty --pid-file 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: load: jerasure load: lrc 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Git sha 0 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: DB SUMMARY 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: DB Session ID: YKJBGVZ65NJ0FVTVEWNN 2024-04-24T23:03:51.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.env: 0x55809e572c60 2024-04-24T23:03:51.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.info_log: 0x5580a0357aa0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.statistics: (nil) 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.use_fsync: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:03:51.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.db_log_dir: 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.wal_dir: 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.write_buffer_manager: 0x5580a032a1e0 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:03:51.430 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.unordered_write: 0 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.row_cache: None 2024-04-24T23:03:51.431 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.wal_filter: None 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.wal_compression: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:03:51.433 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:03:51.434 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_open_files: -1 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:03:51.435 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Compression algorithms supported: 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kZSTD supported: 0 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T23:03:51.436 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.merge_operator: 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_filter: None 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5580a0357ba0) 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout: index_type: 0 2024-04-24T23:03:51.437 INFO:journalctl@ceph.mon.c.smithi117.stdout: data_block_index_type: 0 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: index_shortening: 1 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: checksum: 4 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: no_block_cache: 0 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache: 0x5580a035b090 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache_options: 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: capacity : 536870912 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: num_shard_bits : 4 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: strict_capacity_limit : 0 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache_compressed: (nil) 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: persistent_cache: (nil) 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_size: 4096 2024-04-24T23:03:51.438 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_size_deviation: 10 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_restart_interval: 16 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: index_block_restart_interval: 1 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: metadata_block_size: 4096 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: partition_filters: 0 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: use_delta_encoding: 1 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: filter_policy: bloomfilter 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: whole_key_filtering: 1 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: verify_compression: 0 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: format_version: 5 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: enable_index_compression: 1 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_align: 0 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: max_auto_readahead_size: 262144 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: prepopulate_block_cache: 0 2024-04-24T23:03:51.439 INFO:journalctl@ceph.mon.c.smithi117.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression: NoCompression 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.num_levels: 7 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:03:51.440 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:03:51.441 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:03:51.442 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:03:51.443 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:03:51.444 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.ttl: 2592000 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:03:51.445 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 976dfd62-85a7-477d-9a95-c0850f9a803c 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999831086985, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999831087405, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999831, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "976dfd62-85a7-477d-9a95-c0850f9a803c", "db_session_id": "YKJBGVZ65NJ0FVTVEWNN", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999831087500, "job": 1, "event": "recovery_finished"} 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:03:51.446 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5580a044c000 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: DB pointer 0x5580a0436000 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** DB Stats ** 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-24T23:03:51.447 INFO:journalctl@ceph.mon.c.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-24T23:03:51.448 INFO:journalctl@ceph.mon.c.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: Block cache BinnedLRUCache@0x5580a035b090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:03:51.449 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: using public_addr v2:172.21.15.117:0/0 -> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: starting mon.c rank -1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(???) e0 preinit fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout: e1 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout: legacy client fscid: -1 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout: No filesystems configured 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:03:51.450 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.451 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Deploying cephadm binary to smithi117 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:03:51.452 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Added host smithi117 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1260641573' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1260641573' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:51.454 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c;count:3 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:51.455 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: Deploying daemon mon.c on smithi117 2024-04-24T23:03:51.456 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:51.456 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 ceph-mon[32529]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T23:03:51.456 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 bash[32497]: 266d12945107cf0b688b0f80da17a8bed1535bb40976ec93c3b99951ebdda814 2024-04-24T23:03:51.456 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:51 smithi117 systemd[1]: Started Ceph mon.c for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:03:52.835 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:03:52.836 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:03:53.081 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:03:56.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: mon.a calling monitor election 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: mon.c calling monitor election 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:03:56.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: monmap epoch 2 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: last_changed 2024-04-24T23:03:51.110317+0000 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: min_mon_release 19 (squid) 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: election_strategy: 1 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: fsmap 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: mgrmap e13: x(active, since 50s) 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: overall HEALTH_OK 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:03:56.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: mon.a calling monitor election 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: mon.c calling monitor election 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: monmap epoch 2 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: last_changed 2024-04-24T23:03:51.110317+0000 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: min_mon_release 19 (squid) 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: election_strategy: 1 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: fsmap 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: mgrmap e13: x(active, since 50s) 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: overall HEALTH_OK 2024-04-24T23:03:56.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:56.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:56.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:03:56.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:03:56.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:57.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 -- 172.21.15.117:0/948653549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cec0035c0 msgr2=0x7f1cec005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 --2- 172.21.15.117:0/948653549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cec0035c0 0x7f1cec005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d04077370 tx=0x7f1cf4010ec0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 -- 172.21.15.117:0/948653549 shutdown_connections 2024-04-24T23:03:57.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 --2- 172.21.15.117:0/948653549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cec0035c0 0x7f1cec005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 -- 172.21.15.117:0/948653549 >> 172.21.15.117:0/948653549 conn(0x7f1d04075f90 msgr2=0x7f1d040763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 -- 172.21.15.117:0/948653549 shutdown_connections 2024-04-24T23:03:57.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.349+0000 7f1d09260700 1 -- 172.21.15.117:0/948653549 wait complete. 2024-04-24T23:03:57.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.351+0000 7f1d09260700 1 Processor -- start 2024-04-24T23:03:57.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.351+0000 7f1d09260700 1 -- start start 2024-04-24T23:03:57.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.351+0000 7f1d09260700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.351+0000 7f1d09260700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 0x7f1d04197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.351+0000 7f1d09260700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04107ca0 con 0x7f1d0419dbe0 2024-04-24T23:03:57.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.351+0000 7f1d09260700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04107b20 con 0x7f1cec0035c0 2024-04-24T23:03:57.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 0x7f1d04197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 0x7f1d04197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.117:45400/0 (socket says 172.21.15.117:45400) 2024-04-24T23:03:57.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 -- 172.21.15.117:0/2120755040 learned_addr learned my addr 172.21.15.117:0/2120755040 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:03:57.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d02d9d700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57028/0 (socket says 172.21.15.117:57028) 2024-04-24T23:03:57.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d02d9d700 1 -- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 msgr2=0x7f1d0419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d02d9d700 1 -- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 msgr2=0x7f1d0419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d02d9d700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d02d9d700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 -- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 msgr2=0x7f1d0419d6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 -- 172.21.15.117:0/2120755040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0419e160 con 0x7f1d0419dbe0 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d0259c700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 0x7f1d04197770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800b700 tx=0x7f1cf800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1cf8011630 con 0x7f1d0419dbe0 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf8011cb0 con 0x7f1d0419dbe0 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.352+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d04197d10 con 0x7f1d0419dbe0 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.353+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1cf8010e60 con 0x7f1d0419dbe0 2024-04-24T23:03:57.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.353+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041982a0 con 0x7f1d0419dbe0 2024-04-24T23:03:57.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.353+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1cf8018020 con 0x7f1d0419dbe0 2024-04-24T23:03:57.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.354+0000 7f1cebfff700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cf0038e90 0x7f1cf003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.354+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1cf804cc90 con 0x7f1d0419dbe0 2024-04-24T23:03:57.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.354+0000 7f1d02d9d700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cf0038e90 0x7f1cf003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.354+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d04101b00 con 0x7f1d0419dbe0 2024-04-24T23:03:57.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.354+0000 7f1d02d9d700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cf0038e90 0x7f1cf003b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005d10 tx=0x7f1cf4005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.359+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1cf801f020 con 0x7f1d0419dbe0 2024-04-24T23:03:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:57 smithi033 ceph-mon[29201]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:57 smithi033 ceph-mon[29201]: Deploying daemon mon.b on smithi038 2024-04-24T23:03:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:57.518 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:57 smithi117 ceph-mon[32529]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:57.518 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:57 smithi117 ceph-mon[32529]: Deploying daemon mon.b on smithi038 2024-04-24T23:03:57.518 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:03:57.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.643+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1d04198610 con 0x7f1d0419dbe0 2024-04-24T23:03:57.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.643+0000 7f1cebfff700 1 -- 172.21.15.117:0/2120755040 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1cf8018310 con 0x7f1d0419dbe0 2024-04-24T23:03:57.645 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:03:57.645 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:51.110317Z","created":"2024-04-24T23:02:06.759607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cf0038e90 msgr2=0x7f1cf003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cf0038e90 0x7f1cf003b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005d10 tx=0x7f1cf4005bc0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 msgr2=0x7f1d04197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 0x7f1d04197770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800b700 tx=0x7f1cf800bac0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 shutdown_connections 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cf0038e90 0x7f1cf003b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cec0035c0 0x7f1d0419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 --2- 172.21.15.117:0/2120755040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419dbe0 0x7f1d04197770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 >> 172.21.15.117:0/2120755040 conn(0x7f1d04075f90 msgr2=0x7f1d04103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 shutdown_connections 2024-04-24T23:03:57.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:03:57.645+0000 7f1d09260700 1 -- 172.21.15.117:0/2120755040 wait complete. 2024-04-24T23:03:57.648 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 2 2024-04-24T23:03:58.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:58 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/2120755040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:03:58.455 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:03:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:03:58.110+0000 7f7b6f15a700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T23:03:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:58 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/2120755040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:03:59.223 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:03:59.223 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:03:59.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:03:59 smithi033 ceph-mon[29201]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:59.479 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:03:59.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:03:59 smithi117 ceph-mon[32529]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:03:59.792 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:03:59 smithi038 podman[32431]: 2024-04-24 23:03:59.695718685 +0000 UTC m=+0.019336448 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:04:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:03:59 smithi038 podman[32431]: 2024-04-24 23:03:59.968362117 +0000 UTC m=+0.291979861 container init dfde4ac5135b9cc8f7f56ce4aafc4624cc7a1b9ce213fa45eb809f9672f79cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:04:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:03:59 smithi038 podman[32431]: 2024-04-24 23:03:59.971695571 +0000 UTC m=+0.295313323 container start dfde4ac5135b9cc8f7f56ce4aafc4624cc7a1b9ce213fa45eb809f9672f79cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T23:04:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:03:59 smithi038 ceph-mon[32450]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:04:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:03:59 smithi038 ceph-mon[32450]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:04:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:03:59 smithi038 ceph-mon[32450]: pidfile_write: ignore empty --pid-file 2024-04-24T23:04:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: load: jerasure load: lrc 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Git sha 0 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: DB SUMMARY 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: DB Session ID: 2W38OQ1NMJXK6R3IR839 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:04:00.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.env: 0x55d07c2dcc60 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.info_log: 0x55d07e2a1aa0 2024-04-24T23:04:00.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:04:00.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.statistics: (nil) 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.use_fsync: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:04:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.db_log_dir: 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.wal_dir: 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.write_buffer_manager: 0x55d07e2741e0 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:04:00.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.unordered_write: 0 2024-04-24T23:04:00.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.row_cache: None 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.wal_filter: None 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.wal_compression: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:04:00.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:04:00.179 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_open_files: -1 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Compression algorithms supported: 2024-04-24T23:04:00.180 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kZSTD supported: 0 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.merge_operator: 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_filter: None 2024-04-24T23:04:00.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d07e2a1ba0) 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: index_type: 0 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: data_block_index_type: 0 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: index_shortening: 1 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: checksum: 4 2024-04-24T23:04:00.182 INFO:journalctl@ceph.mon.b.smithi038.stdout: no_block_cache: 0 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_cache: 0x55d07e2a5090 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_cache_options: 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: capacity : 536870912 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: num_shard_bits : 4 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: strict_capacity_limit : 0 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_cache_compressed: (nil) 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: persistent_cache: (nil) 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_size: 4096 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_size_deviation: 10 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_restart_interval: 16 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: index_block_restart_interval: 1 2024-04-24T23:04:00.183 INFO:journalctl@ceph.mon.b.smithi038.stdout: metadata_block_size: 4096 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: partition_filters: 0 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: use_delta_encoding: 1 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: filter_policy: bloomfilter 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: whole_key_filtering: 1 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: verify_compression: 0 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: format_version: 5 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: enable_index_compression: 1 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: block_align: 0 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: max_auto_readahead_size: 262144 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: prepopulate_block_cache: 0 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:04:00.184 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression: NoCompression 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.num_levels: 7 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:04:00.185 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:04:00.186 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:04:00.187 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:04:00.188 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:04:00.189 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.ttl: 2592000 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:04:00.190 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 74dec309-f53f-4097-b5b3-e6a57efe7704 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999840002634, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T23:04:00.191 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999840003276, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999840, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "74dec309-f53f-4097-b5b3-e6a57efe7704", "db_session_id": "2W38OQ1NMJXK6R3IR839", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999840003390, "job": 1, "event": "recovery_finished"} 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d07e396000 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: DB pointer 0x55d07e380000 2024-04-24T23:04:00.192 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: ** DB Stats ** 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: ** Compaction Stats [default] ** 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:04:00.193 INFO:journalctl@ceph.mon.b.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: ** Compaction Stats [default] ** 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:04:00.194 INFO:journalctl@ceph.mon.b.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: Cumulative compaction: 0.00 GB write, 0.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: Interval compaction: 0.00 GB write, 0.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: Block cache BinnedLRUCache@0x55d07e2a5090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: using public_addr v2:172.21.15.38:0/0 -> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: starting mon.b rank -1 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:00.195 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(???) e0 preinit fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).mds e1 new map 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout: e1 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout: legacy client fscid: -1 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout: 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout: No filesystems configured 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:04:00.196 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:04:00.197 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-24T23:04:00.199 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Deploying cephadm binary to smithi117 2024-04-24T23:04:00.199 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:00.199 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:00.199 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Added host smithi117 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.200 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1260641573' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1260641573' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-24T23:04:00.201 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c;count:3 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.202 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Deploying daemon mon.c on smithi117 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.a calling monitor election 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.c calling monitor election 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.203 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: monmap epoch 2 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: last_changed 2024-04-24T23:03:51.110317+0000 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: min_mon_release 19 (squid) 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: election_strategy: 1 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: fsmap 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mgrmap e13: x(active, since 50s) 2024-04-24T23:04:00.204 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: overall HEALTH_OK 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: Deploying daemon mon.b on smithi038 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/2120755040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 ceph-mon[32450]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 bash[32431]: dfde4ac5135b9cc8f7f56ce4aafc4624cc7a1b9ce213fa45eb809f9672f79cc0 2024-04-24T23:04:00.205 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:00 smithi038 systemd[1]: Started Ceph mon.b for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.297+0000 7f96d6806700 1 -- 172.21.15.117:0/988931730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b80035c0 msgr2=0x7f96b8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.297+0000 7f96d6806700 1 --2- 172.21.15.117:0/988931730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b80035c0 0x7f96b8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96d0101c50 tx=0x7f96c400fe60 comp rx=0 tx=0).stop 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 -- 172.21.15.117:0/988931730 shutdown_connections 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 --2- 172.21.15.117:0/988931730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b80035c0 0x7f96b8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 -- 172.21.15.117:0/988931730 >> 172.21.15.117:0/988931730 conn(0x7f96d00fd830 msgr2=0x7f96d00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 -- 172.21.15.117:0/988931730 shutdown_connections 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 -- 172.21.15.117:0/988931730 wait complete. 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 Processor -- start 2024-04-24T23:04:05.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.298+0000 7f96d6806700 1 -- start start 2024-04-24T23:04:05.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d6806700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d6806700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d0074a70 0x7f96d0074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d6806700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 0x7f96d01a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d6806700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0109d60 con 0x7f96d0074a70 2024-04-24T23:04:05.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d6806700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0109be0 con 0x7f96d01a3ba0 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d6806700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0109ee0 con 0x7f96b80035c0 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52420/0 (socket says 172.21.15.117:52420) 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 -- 172.21.15.117:0/3645385708 learned_addr learned my addr 172.21.15.117:0/3645385708 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cf7fe700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d0074a70 0x7f96d0074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 msgr2=0x7f96d0074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 msgr2=0x7f96d0074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96d4da3700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 0x7f96d01a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.299+0000 7f96cffff700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96d4da3700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 msgr2=0x7f96d01a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96d4da3700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 msgr2=0x7f96d01a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96d4da3700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 0x7f96d01a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:04:05.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cf7fe700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 msgr2=0x7f96d0074530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cf7fe700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96d4da3700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 0x7f96d01a5f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cf7fe700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 msgr2=0x7f96d01a5f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cf7fe700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 0x7f96d01a5f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cf7fe700 1 -- 172.21.15.117:0/3645385708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d01a66c0 con 0x7f96d0074a70 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cf7fe700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d0074a70 0x7f96d0074ef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f96c000b550 tx=0x7f96c000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0004230 con 0x7f96d0074a70 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c00043d0 con 0x7f96d0074a70 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d01a69b0 con 0x7f96d0074a70 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0011c20 con 0x7f96d0074a70 2024-04-24T23:04:05.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.300+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d01a6f40 con 0x7f96d0074a70 2024-04-24T23:04:05.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.301+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f96c001a400 con 0x7f96d0074a70 2024-04-24T23:04:05.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.301+0000 7f96cd7fa700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96bc039580 0x7f96bc03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.301+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d0063890 con 0x7f96d0074a70 2024-04-24T23:04:05.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.302+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f96c004bdb0 con 0x7f96d0074a70 2024-04-24T23:04:05.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.302+0000 7f96cffff700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96bc039580 0x7f96bc03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.302+0000 7f96cffff700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96bc039580 0x7f96bc03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f96c4001eb0 tx=0x7f96c4005de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.307+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f96c0003c80 con 0x7f96d0074a70 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: mon.c calling monitor election 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: mon.a calling monitor election 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:05.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: monmap epoch 3 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: last_changed 2024-04-24T23:04:00.028874+0000 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: min_mon_release 19 (squid) 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: election_strategy: 1 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: fsmap 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:05.428 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: mgrmap e13: x(active, since 59s) 2024-04-24T23:04:05.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: overall HEALTH_OK 2024-04-24T23:04:05.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:05.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:05.429 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: mon.c calling monitor election 2024-04-24T23:04:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: mon.a calling monitor election 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: monmap epoch 3 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: last_changed 2024-04-24T23:04:00.028874+0000 2024-04-24T23:04:05.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: min_mon_release 19 (squid) 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: election_strategy: 1 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: fsmap 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: mgrmap e13: x(active, since 59s) 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: overall HEALTH_OK 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:05.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:05.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.598+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f96d002db30 con 0x7f96d0074a70 2024-04-24T23:04:05.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.599+0000 7f96cd7fa700 1 -- 172.21.15.117:0/3645385708 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f96c00193a0 con 0x7f96d0074a70 2024-04-24T23:04:05.601 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-24T23:04:05.602 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":3,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:04:00.028874Z","created":"2024-04-24T23:02:06.759607Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96bc039580 msgr2=0x7f96bc03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96bc039580 0x7f96bc03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f96c4001eb0 tx=0x7f96c4005de0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d0074a70 msgr2=0x7f96d0074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d0074a70 0x7f96d0074ef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f96c000b550 tx=0x7f96c000b910 comp rx=0 tx=0).stop 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 shutdown_connections 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96bc039580 0x7f96bc03ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96d01a3ba0 0x7f96d01a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96b80035c0 0x7f96d0074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 --2- 172.21.15.117:0/3645385708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d0074a70 0x7f96d0074ef0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 >> 172.21.15.117:0/3645385708 conn(0x7f96d00fd830 msgr2=0x7f96d00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 shutdown_connections 2024-04-24T23:04:05.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:05.601+0000 7f96d6806700 1 -- 172.21.15.117:0/3645385708 wait complete. 2024-04-24T23:04:05.605 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 3 2024-04-24T23:04:06.152 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:06.152 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:06.152 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:06 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/3645385708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:06.153 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:06.153 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:06.153 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T23:04:06.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T23:04:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:06 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/3645385708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.c calling monitor election 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.a calling monitor election 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: monmap epoch 3 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: last_changed 2024-04-24T23:04:00.028874+0000 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: min_mon_release 19 (squid) 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: election_strategy: 1 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:07.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: fsmap 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mgrmap e13: x(active, since 59s) 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: overall HEALTH_OK 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/3645385708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:07.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 -- 172.21.15.33:0/2595728387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0107dc0 msgr2=0x7f13b01081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 --2- 172.21.15.33:0/2595728387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0107dc0 0x7f13b01081a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009910 tx=0x7f13ac009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 -- 172.21.15.33:0/2595728387 shutdown_connections 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 --2- 172.21.15.33:0/2595728387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0107dc0 0x7f13b01081a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 -- 172.21.15.33:0/2595728387 >> 172.21.15.33:0/2595728387 conn(0x7f13b00772b0 msgr2=0x7f13b00776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 -- 172.21.15.33:0/2595728387 shutdown_connections 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.036+0000 7f13b749b700 1 -- 172.21.15.33:0/2595728387 wait complete. 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 Processor -- start 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 -- start start 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b0107dc0 0x7f13b019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b019dcf0 0x7f13b0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 0x7f13b0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010c370 con 0x7f13b0197eb0 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010c1f0 con 0x7f13b019dcf0 2024-04-24T23:04:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.037+0000 7f13b749b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010c4f0 con 0x7f13b0107dc0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 0x7f13b0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 0x7f13b0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34718/0 (socket says 172.21.15.33:34718) 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 -- 172.21.15.33:0/1934220577 learned_addr learned my addr 172.21.15.33:0/1934220577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5237700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b0107dc0 0x7f13b019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b4a36700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b019dcf0 0x7f13b0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 -- 172.21.15.33:0/1934220577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b0107dc0 msgr2=0x7f13b019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b0107dc0 0x7f13b019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 -- 172.21.15.33:0/1934220577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b019dcf0 msgr2=0x7f13b0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b019dcf0 0x7f13b0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 -- 172.21.15.33:0/1934220577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b01a4680 con 0x7f13b0197eb0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b5a38700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 0x7f13b0198360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13a800b9e0 tx=0x7f13a800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8019070 con 0x7f13b0197eb0 2024-04-24T23:04:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b01a48b0 con 0x7f13b0197eb0 2024-04-24T23:04:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a800ce10 con 0x7f13b0197eb0 2024-04-24T23:04:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8007350 con 0x7f13b0197eb0 2024-04-24T23:04:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.038+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b01a4df0 con 0x7f13b0197eb0 2024-04-24T23:04:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.039+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f13a801c070 con 0x7f13b0197eb0 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.039+0000 7f13a67fc700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f139c039580 0x7f139c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.039+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b0198ea0 con 0x7f13b0197eb0 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.040+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f13a804b7b0 con 0x7f13b0197eb0 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.040+0000 7f13b5237700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f139c039580 0x7f139c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.040+0000 7f13b5237700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f139c039580 0x7f139c03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009750 tx=0x7f13ac00b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:08.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: mon.b calling monitor election 2024-04-24T23:04:08.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-24T23:04:08.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-24T23:04:08.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-24T23:04:08.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: mon.b calling monitor election 2024-04-24T23:04:08.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: mon.c calling monitor election 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: mon.a calling monitor election 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: monmap epoch 3 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: last_changed 2024-04-24T23:04:00.028874+0000 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: min_mon_release 19 (squid) 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: election_strategy: 1 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: fsmap 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:08.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: mgrmap e13: x(active, since 61s) 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: overall HEALTH_OK 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Updating smithi033:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:04:08.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:08.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:08.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:08.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:07 smithi033 ceph-mon[29201]: Reconfiguring daemon mon.a on smithi033 2024-04-24T23:04:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.046+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f13a8012500 con 0x7f13b0197eb0 2024-04-24T23:04:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.b calling monitor election 2024-04-24T23:04:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-24T23:04:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-24T23:04:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-24T23:04:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.b calling monitor election 2024-04-24T23:04:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.c calling monitor election 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.a calling monitor election 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: monmap epoch 3 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: last_changed 2024-04-24T23:04:00.028874+0000 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: min_mon_release 19 (squid) 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: election_strategy: 1 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-04-24T23:04:08.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: fsmap 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: mgrmap e13: x(active, since 61s) 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: overall HEALTH_OK 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Updating smithi033:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.173 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:08.174 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:07 smithi038 ceph-mon[32450]: Reconfiguring daemon mon.a on smithi033 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: mon.b calling monitor election 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Updating smithi038:/etc/ceph/ceph.conf 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: mon.b calling monitor election 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: mon.c calling monitor election 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: mon.a calling monitor election 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: monmap epoch 3 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: last_changed 2024-04-24T23:04:00.028874+0000 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: created 2024-04-24T23:02:06.759607+0000 2024-04-24T23:04:08.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: min_mon_release 19 (squid) 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: election_strategy: 1 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: fsmap 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: mgrmap e13: x(active, since 61s) 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: overall HEALTH_OK 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Updating smithi033:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Updating smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Updating smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:08.179 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:07 smithi117 ceph-mon[32529]: Reconfiguring daemon mon.a on smithi033 2024-04-24T23:04:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.256+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f13b0101940 con 0x7f13b0197eb0 2024-04-24T23:04:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.257+0000 7f13a67fc700 1 -- 172.21.15.33:0/1934220577 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f13a8012760 con 0x7f13b0197eb0 2024-04-24T23:04:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-04-24T23:04:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-04-24T23:04:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f139c039580 msgr2=0x7f139c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f139c039580 0x7f139c03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009750 tx=0x7f13ac00b540 comp rx=0 tx=0).stop 2024-04-24T23:04:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 msgr2=0x7f13b0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 0x7f13b0198360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13a800b9e0 tx=0x7f13a800bcf0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 shutdown_connections 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f139c039580 0x7f139c03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b0197eb0 0x7f13b0198360 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b0107dc0 0x7f13b019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 --2- 172.21.15.33:0/1934220577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13b019dcf0 0x7f13b0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 >> 172.21.15.33:0/1934220577 conn(0x7f13b00772b0 msgr2=0x7f13b0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 shutdown_connections 2024-04-24T23:04:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:08.259+0000 7f13b749b700 1 -- 172.21.15.33:0/1934220577 wait complete. 2024-04-24T23:04:08.958 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T23:04:08.958 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:04:08.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:08.989 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:04:08.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:09.053 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:04:09.054 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:09.073 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:08 smithi033 ceph-mon[29201]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:09.073 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:09.073 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:08 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1934220577' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:09.081 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:08 smithi038 ceph-mon[32450]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:09.081 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:09.081 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:08 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1934220577' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:09.087 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:04:09.087 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:09.159 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:04:09.159 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:09.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:08 smithi117 ceph-mon[32529]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:09.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:09.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:08 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1934220577' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:09.190 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:04:09.191 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:09.258 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-04-24T23:04:09.258 INFO:tasks.cephadm:Adding mgr.y on smithi038 2024-04-24T23:04:09.258 INFO:tasks.cephadm:Adding mgr.z on smithi117 2024-04-24T23:04:09.259 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi033=x;smithi038=y;smithi117=z' 2024-04-24T23:04:09.532 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:04:10.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:10 smithi038 ceph-mon[32450]: Reconfiguring daemon mon.b on smithi038 2024-04-24T23:04:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 -- 172.21.15.117:0/3347438836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f501028a0 msgr2=0x7f4f50104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 --2- 172.21.15.117:0/3347438836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f501028a0 0x7f4f50104c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009910 tx=0x7f4f40009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 -- 172.21.15.117:0/3347438836 shutdown_connections 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 --2- 172.21.15.117:0/3347438836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f501028a0 0x7f4f50104c90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 -- 172.21.15.117:0/3347438836 >> 172.21.15.117:0/3347438836 conn(0x7f4f500fc270 msgr2=0x7f4f500fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 -- 172.21.15.117:0/3347438836 shutdown_connections 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 -- 172.21.15.117:0/3347438836 wait complete. 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 Processor -- start 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.970+0000 7f4f5854e700 1 -- start start 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f5854e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 0x7f4f50102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f5854e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 0x7f4f50100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f5854e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f50101210 0x7f4f501a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f5854e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50107e10 con 0x7f4f50100850 2024-04-24T23:04:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f5854e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50107c90 con 0x7f4f50101210 2024-04-24T23:04:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f5854e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50107f90 con 0x7f4f501028a0 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f562ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 0x7f4f50102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 0x7f4f50100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 0x7f4f50100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.117:45528/0 (socket says 172.21.15.117:45528) 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 -- 172.21.15.117:0/2012473560 learned_addr learned my addr 172.21.15.117:0/2012473560 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f562ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 0x7f4f50102200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57592/0 (socket says 172.21.15.117:57592) 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f56aeb700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f50101210 0x7f4f501a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 -- 172.21.15.117:0/2012473560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 msgr2=0x7f4f50102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 0x7f4f50102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 -- 172.21.15.117:0/2012473560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f50101210 msgr2=0x7f4f501a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f50101210 0x7f4f501a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 -- 172.21.15.117:0/2012473560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f501a1c40 con 0x7f4f50100850 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f562ea700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 0x7f4f50102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.971+0000 7f4f55ae9700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 0x7f4f50100cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b780 tx=0x7f4f4c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.972+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c004230 con 0x7f4f50100850 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.972+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f501a1f30 con 0x7f4f50100850 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.972+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f501a24b0 con 0x7f4f50100850 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.972+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c0043d0 con 0x7f4f50100850 2024-04-24T23:04:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.972+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c011b90 con 0x7f4f50100850 2024-04-24T23:04:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.972+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f50063890 con 0x7f4f50100850 2024-04-24T23:04:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.973+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4f4c0036a0 con 0x7f4f50100850 2024-04-24T23:04:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.973+0000 7f4f477fe700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f3c039580 0x7f4f3c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.973+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4f4c04bab0 con 0x7f4f50100850 2024-04-24T23:04:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.973+0000 7f4f562ea700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f3c039580 0x7f4f3c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.973+0000 7f4f562ea700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f3c039580 0x7f4f3c03ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f40000c00 tx=0x7f4f4001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:10.978+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f4c01f070 con 0x7f4f50100850 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:10 smithi117 ceph-mon[32529]: Reconfiguring daemon mon.b on smithi038 2024-04-24T23:04:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.185+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi038=y;smithi117=z", "target": ["mon-mgr", ""]}) -- 0x7f4f500627c0 con 0x7f4f3c039580 2024-04-24T23:04:11.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.192+0000 7f4f477fe700 1 -- 172.21.15.117:0/2012473560 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4f500627c0 con 0x7f4f3c039580 2024-04-24T23:04:11.194 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mgr update... 2024-04-24T23:04:11.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.194+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f3c039580 msgr2=0x7f4f3c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:11.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.194+0000 7f4f5854e700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f3c039580 0x7f4f3c03ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f40000c00 tx=0x7f4f4001a040 comp rx=0 tx=0).stop 2024-04-24T23:04:11.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.194+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 msgr2=0x7f4f50100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:11.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.194+0000 7f4f5854e700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 0x7f4f50100cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b780 tx=0x7f4f4c00ba90 comp rx=0 tx=0).stop 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 shutdown_connections 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f3c039580 0x7f4f3c03ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f50101210 0x7f4f501a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f501028a0 0x7f4f50102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 --2- 172.21.15.117:0/2012473560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f50100850 0x7f4f50100cd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 >> 172.21.15.117:0/2012473560 conn(0x7f4f500fc270 msgr2=0x7f4f500fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 shutdown_connections 2024-04-24T23:04:11.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:04:11.195+0000 7f4f5854e700 1 -- 172.21.15.117:0/2012473560 wait complete. 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:10 smithi033 ceph-mon[29201]: Reconfiguring daemon mon.b on smithi038 2024-04-24T23:04:11.784 DEBUG:teuthology.orchestra.run.smithi038:mgr.y> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T23:04:11.786 DEBUG:teuthology.orchestra.run.smithi117:mgr.z> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T23:04:11.788 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T23:04:11.789 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:04:11.789 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:04:11.809 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:04:11.809 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-04-24T23:04:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-24 23:03:18.692294896 +0000 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-24 22:58:14.145126053 +0000 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-24 22:58:14.145126053 +0000 2024-04-24T23:04:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-24 22:58:14.145126053 +0000 2024-04-24T23:04:11.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:04:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-24T23:04:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-24T23:04:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000236465 s, 2.2 MB/s 2024-04-24T23:04:11.939 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:04:11.999 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-04-24T23:04:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:04:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:12.058 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T23:04:12.059 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:12.059 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:12.059 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-24 23:03:18.729294362 +0000 2024-04-24T23:04:12.059 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-24 22:58:14.504120344 +0000 2024-04-24T23:04:12.059 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-24 22:58:14.504120344 +0000 2024-04-24T23:04:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-24 22:58:14.504120344 +0000 2024-04-24T23:04:12.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:04:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-24T23:04:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-24T23:04:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000205178 s, 2.5 MB/s 2024-04-24T23:04:12.126 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:04:12.184 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-04-24T23:04:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-24 23:03:18.766293828 +0000 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-24 22:58:14.860114682 +0000 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-24 22:58:14.860114682 +0000 2024-04-24T23:04:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-24 22:58:14.860114682 +0000 2024-04-24T23:04:12.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:04:12.306 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-24T23:04:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-24T23:04:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000245743 s, 2.1 MB/s 2024-04-24T23:04:12.308 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:04:12.366 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-24 23:03:18.800293337 +0000 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-24 22:58:15.209109132 +0000 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-24 22:58:15.209109132 +0000 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-24 22:58:15.209109132 +0000 2024-04-24T23:04:12.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:04:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi038=y;smithi117=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: Saving service mgr spec with placement smithi033=x;smithi038=y;smithi117=z;count:3 2024-04-24T23:04:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:12 smithi033 ceph-mon[29201]: Reconfiguring daemon mon.c on smithi117 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000315319 s, 1.6 MB/s 2024-04-24T23:04:12.480 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:04:12.537 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-24T23:04:12.537 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:04:12.556 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:04:12.557 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-04-24T23:04:12.614 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:04:12.614 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:12.614 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:04:12.614 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:12.614 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:12.614 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-24 23:03:59.469606562 +0000 2024-04-24T23:04:12.615 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-24 22:58:59.995879278 +0000 2024-04-24T23:04:12.615 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-24 22:58:59.995879278 +0000 2024-04-24T23:04:12.615 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-24 22:58:59.995879278 +0000 2024-04-24T23:04:12.615 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:04:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi038=y;smithi117=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: Saving service mgr spec with placement smithi033=x;smithi038=y;smithi117=z;count:3 2024-04-24T23:04:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:12 smithi038 ceph-mon[32450]: Reconfiguring daemon mon.c on smithi117 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi038=y;smithi117=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: Saving service mgr spec with placement smithi033=x;smithi038=y;smithi117=z;count:3 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:12.673 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:12 smithi117 ceph-mon[32529]: Reconfiguring daemon mon.c on smithi117 2024-04-24T23:04:12.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-24T23:04:12.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-24T23:04:12.681 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000314464 s, 1.6 MB/s 2024-04-24T23:04:12.682 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:04:12.740 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-04-24T23:04:12.797 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:04:12.797 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:12.797 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T23:04:12.797 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:12.797 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:12.797 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-24 23:03:59.470606544 +0000 2024-04-24T23:04:12.798 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-24 22:59:00.340873034 +0000 2024-04-24T23:04:12.798 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-24 22:59:00.340873034 +0000 2024-04-24T23:04:12.798 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-24 22:59:00.340873034 +0000 2024-04-24T23:04:12.798 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:04:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-24T23:04:12.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-24T23:04:12.863 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000278385 s, 1.8 MB/s 2024-04-24T23:04:12.864 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:04:12.922 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-04-24T23:04:12.978 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:04:12.978 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:12.978 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-24T23:04:12.979 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:12.979 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:12.979 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-24 23:03:59.470606544 +0000 2024-04-24T23:04:12.979 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-24 22:59:00.695866609 +0000 2024-04-24T23:04:12.979 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-24 22:59:00.695866609 +0000 2024-04-24T23:04:12.979 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-24 22:59:00.695866609 +0000 2024-04-24T23:04:12.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:04:13.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-24T23:04:13.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-24T23:04:13.043 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000290184 s, 1.8 MB/s 2024-04-24T23:04:13.044 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:04:13.101 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-24 23:03:59.471606526 +0000 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-24 22:59:01.046860257 +0000 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-24 22:59:01.046860257 +0000 2024-04-24T23:04:13.157 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-24 22:59:01.046860257 +0000 2024-04-24T23:04:13.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:04:13.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-24T23:04:13.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-24T23:04:13.221 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000445334 s, 1.1 MB/s 2024-04-24T23:04:13.222 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:04:13.279 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-24T23:04:13.279 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:04:13.296 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:04:13.296 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-24 23:03:49.978276586 +0000 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-24 22:58:34.905590657 +0000 2024-04-24T23:04:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-24 22:58:34.905590657 +0000 2024-04-24T23:04:13.353 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-24 22:58:34.905590657 +0000 2024-04-24T23:04:13.353 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:04:13.416 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-24T23:04:13.417 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-24T23:04:13.417 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000279152 s, 1.8 MB/s 2024-04-24T23:04:13.418 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:04:13.474 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-24 23:03:49.979276567 +0000 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-24 22:58:35.243584053 +0000 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-24 22:58:35.243584053 +0000 2024-04-24T23:04:13.530 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-24 22:58:35.243584053 +0000 2024-04-24T23:04:13.531 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:04:13.591 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:13 smithi117 ceph-mon[32529]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:13.591 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:13.591 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:13.591 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:13.594 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-24T23:04:13.594 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-24T23:04:13.594 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000196774 s, 2.6 MB/s 2024-04-24T23:04:13.595 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:04:13.652 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-04-24T23:04:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:13 smithi038 ceph-mon[32450]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:13 smithi033 ceph-mon[29201]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:13.707 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:04:13.707 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:13.707 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-24T23:04:13.707 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:13.707 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:13.708 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-24 23:03:49.979276567 +0000 2024-04-24T23:04:13.708 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-24 22:58:35.578577508 +0000 2024-04-24T23:04:13.708 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-24 22:58:35.578577508 +0000 2024-04-24T23:04:13.708 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-24 22:58:35.578577508 +0000 2024-04-24T23:04:13.709 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:04:13.769 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-24T23:04:13.770 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-24T23:04:13.770 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00021852 s, 2.3 MB/s 2024-04-24T23:04:13.771 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:04:13.827 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-04-24T23:04:13.882 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:04:13.882 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:13.882 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:04:13.882 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:13.882 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:13.882 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-24 23:03:49.979276567 +0000 2024-04-24T23:04:13.883 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-24 22:58:35.911571001 +0000 2024-04-24T23:04:13.883 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-24 22:58:35.911571001 +0000 2024-04-24T23:04:13.883 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-24 22:58:35.911571001 +0000 2024-04-24T23:04:13.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:04:13.944 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-24T23:04:13.944 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-24T23:04:13.945 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000216677 s, 2.4 MB/s 2024-04-24T23:04:13.946 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:04:14.002 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-04-24T23:04:14.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:04:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:14.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:14.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:14 smithi033 ceph-mon[29201]: Deploying daemon mgr.z on smithi117 2024-04-24T23:04:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:14 smithi038 ceph-mon[32450]: Deploying daemon mgr.z on smithi117 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:14 smithi117 ceph-mon[32529]: Deploying daemon mgr.z on smithi117 2024-04-24T23:04:15.426 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 systemd[1]: Starting Ceph mgr.z for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:04:15.742 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:15 smithi117 ceph-mon[32529]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:15.742 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 podman[34364]: 2024-04-24 23:04:15.403223238 +0000 UTC m=+0.017973012 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:04:15.742 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 podman[34364]: 2024-04-24 23:04:15.52884637 +0000 UTC m=+0.143596129 container create 56296d78a223bc382881535ea8a6c4fd9bd33125f7038564fa678f9ae18ccb8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:04:15.743 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 podman[34364]: 2024-04-24 23:04:15.714354803 +0000 UTC m=+0.329104565 container init 56296d78a223bc382881535ea8a6c4fd9bd33125f7038564fa678f9ae18ccb8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:04:15.743 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 podman[34364]: 2024-04-24 23:04:15.717056953 +0000 UTC m=+0.331806716 container start 56296d78a223bc382881535ea8a6c4fd9bd33125f7038564fa678f9ae18ccb8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:04:15.743 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 ceph-mgr[34382]: -- 172.21.15.117:0/3456467219 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8756a25a0 con 0x55e875677000 2024-04-24T23:04:15.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:15 smithi038 ceph-mon[32450]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:15.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:15 smithi033 ceph-mon[29201]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:16.058 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 bash[34364]: 56296d78a223bc382881535ea8a6c4fd9bd33125f7038564fa678f9ae18ccb8e 2024-04-24T23:04:16.058 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 systemd[1]: Started Ceph mgr.z for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:16.058 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:15.872+0000 7f98c3431200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:04:16.059 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:16.056+0000 7f98c3431200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:04:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:04:16.087 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-24T23:04:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-24T23:04:16.087 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118363 s, 886 MB/s 2024-04-24T23:04:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-24T23:04:16.426 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:16.158+0000 7f98c3431200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:04:16.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-04-24T23:04:16.852 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:16.466+0000 7f98c3431200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:04:16.852 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:16.551+0000 7f98c3431200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:16.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:16.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:16 smithi033 ceph-mon[29201]: Deploying daemon mgr.y on smithi038 2024-04-24T23:04:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:17.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:16 smithi038 ceph-mon[32450]: Deploying daemon mgr.y on smithi038 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:16 smithi117 ceph-mon[32529]: Deploying daemon mgr.y on smithi038 2024-04-24T23:04:18.097 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:17 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:17.750+0000 7f98c3431200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:04:18.097 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:17 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:17.921+0000 7f98c3431200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:18.097 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:18.095+0000 7f98c3431200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:04:18.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/3635516357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 msgr2=0x7fb5f81026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/3635516357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81026b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0099c0 tx=0x7fb5ec009cd0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/3635516357 shutdown_connections 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/3635516357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f8103e30 0x7fb5f810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/3635516357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81026b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/3635516357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/3635516357 >> 172.21.15.33:0/3635516357 conn(0x7fb5f80fd830 msgr2=0x7fb5f80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/3635516357 shutdown_connections 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/3635516357 wait complete. 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.613+0000 7fb5ff0c1700 1 Processor -- start 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 -- start start 2024-04-24T23:04:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f8103e30 0x7fb5f819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810de50 con 0x7fb5f8102290 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810dcd0 con 0x7fb5f8103490 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5ff0c1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810dfd0 con 0x7fb5f8103e30 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5fce5d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36440/0 (socket says 172.21.15.33:36440) 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5fce5d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58542/0 (socket says 172.21.15.33:58542) 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 -- 172.21.15.33:0/2644531345 learned_addr learned my addr 172.21.15.33:0/2644531345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5fd65e700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f8103e30 0x7fb5f819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 -- 172.21.15.33:0/2644531345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f8103e30 msgr2=0x7fb5f819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f8103e30 0x7fb5f819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 -- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 msgr2=0x7fb5f81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 -- 172.21.15.33:0/2644531345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81a0120 con 0x7fb5f8103490 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5fce5d700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.614+0000 7fb5f7fff700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f8198a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400e9a0 tx=0x7fb5e400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.615+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400ca10 con 0x7fb5f8103490 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.615+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81a0410 con 0x7fb5f8103490 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.615+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e400cbb0 con 0x7fb5f8103490 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.615+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4018710 con 0x7fb5f8103490 2024-04-24T23:04:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.615+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81a09a0 con 0x7fb5f8103490 2024-04-24T23:04:18.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.616+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb5e40188b0 con 0x7fb5f8103490 2024-04-24T23:04:18.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.616+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f8068f20 con 0x7fb5f8103490 2024-04-24T23:04:18.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.616+0000 7fb5f5ffb700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5e8039330 0x7fb5e803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.616+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb5e404c540 con 0x7fb5f8103490 2024-04-24T23:04:18.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.616+0000 7fb5fce5d700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5e8039330 0x7fb5e803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.617+0000 7fb5fce5d700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5e8039330 0x7fb5e803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b580 tx=0x7fb5ec005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.621+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb5e40071c0 con 0x7fb5f8103490 2024-04-24T23:04:18.670 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:18 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:18.630+0000 7f9559747200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:18.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:18.394+0000 7f98c3431200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:04:18.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:18.490+0000 7f98c3431200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:04:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:18.860+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb5f8062730 con 0x7fb5e8039330 2024-04-24T23:04:19.082 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:18 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:18.744+0000 7f9559747200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:04:19.094 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:18.697+0000 7f98c3431200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:04:19.094 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:18.905+0000 7f98c3431200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:19.416 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:19.080+0000 7f9559747200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:04:19.416 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:19.189+0000 7f9559747200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:04:19.667 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:04:19.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:04:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:04:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:19.533+0000 7f98c3431200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:04:19.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:04:19.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:20.271 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:19 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:19.969+0000 7f98c3431200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:04:20.271 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:20.067+0000 7f98c3431200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:04:20.271 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:20.269+0000 7f98c3431200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:20.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:20.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:20.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:20.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:20.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:20.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:20.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:20.485+0000 7f9559747200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:20.660+0000 7f9559747200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:20.837+0000 7f9559747200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.839 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:20.840 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:20.840 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:20.840 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:20.840 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:20.840 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:21.125 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:20.790+0000 7f98c3431200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:04:21.125 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:20.881+0000 7f98c3431200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:04:21.420 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:21.134+0000 7f9559747200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:04:21.420 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:21.229+0000 7f9559747200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:04:21.421 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:21.123+0000 7f98c3431200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:04:21.421 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:21.255+0000 7f98c3431200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:04:21.421 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:21.349+0000 7f98c3431200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:04:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-mon[32529]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-mon[32529]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:04:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-mon[32529]: Reconfiguring daemon mgr.x on smithi033 2024-04-24T23:04:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:21.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:21.464+0000 7f98c3431200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:21.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:21.555+0000 7f98c3431200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:04:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:21 smithi033 ceph-mon[29201]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:21 smithi033 ceph-mon[29201]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:04:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:21 smithi033 ceph-mon[29201]: Reconfiguring daemon mgr.x on smithi033 2024-04-24T23:04:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-mon[32450]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-mon[32450]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:04:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-mon[32450]: Reconfiguring daemon mgr.x on smithi033 2024-04-24T23:04:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:21.921 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:21.438+0000 7f9559747200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:04:21.921 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:21.636+0000 7f9559747200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:22.670 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:22.263+0000 7f9559747200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:04:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1581938074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93333af-c06d-4be4-91c8-bd832217bd8b"}]: dispatch 2024-04-24T23:04:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1581938074' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d93333af-c06d-4be4-91c8-bd832217bd8b"}]': finished 2024-04-24T23:04:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-mon[32450]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:04:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:22.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:22 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1581938074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93333af-c06d-4be4-91c8-bd832217bd8b"}]: dispatch 2024-04-24T23:04:22.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:22 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1581938074' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d93333af-c06d-4be4-91c8-bd832217bd8b"}]': finished 2024-04-24T23:04:22.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:22 smithi117 ceph-mon[32529]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:04:22.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:22 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:22.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:22.360+0000 7f98c3431200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:04:22.676 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:04:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z[34378]: 2024-04-24T23:04:22.574+0000 7f98c3431200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:04:22.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:22 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1581938074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93333af-c06d-4be4-91c8-bd832217bd8b"}]: dispatch 2024-04-24T23:04:22.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:22 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1581938074' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d93333af-c06d-4be4-91c8-bd832217bd8b"}]': finished 2024-04-24T23:04:22.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:22 smithi033 ceph-mon[29201]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:04:22.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:22 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:23.002 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:22.697+0000 7f9559747200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:04:23.003 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:22 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:22.796+0000 7f9559747200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:04:23.003 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:23.001+0000 7f9559747200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:04:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:23.434+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fb5e40178f0 con 0x7fb5f8103490 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1181755733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: Standby manager daemon z started 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:04:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:23 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:04:23.886 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:23.544+0000 7f9559747200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:23.637+0000 7f9559747200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1181755733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: Standby manager daemon z started 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:04:23.887 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:04:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1181755733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:04:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: Standby manager daemon z started 2024-04-24T23:04:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:04:23.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:04:23.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:04:23.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:23 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.117:0/863824255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:04:24.170 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:23.885+0000 7f9559747200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:04:24.170 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:24.022+0000 7f9559747200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:04:24.170 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:24.119+0000 7f9559747200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:04:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:24 smithi038 ceph-mon[32450]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:04:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:04:24.671 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:24.238+0000 7f9559747200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:24.671 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:24.331+0000 7f9559747200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:04:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:24 smithi033 ceph-mon[29201]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:04:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:04:24.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:24 smithi117 ceph-mon[32529]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:04:24.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:04:25.440 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:25.173+0000 7f9559747200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:04:25.440 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y[33824]: 2024-04-24T23:04:25.392+0000 7f9559747200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:04:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:25.449+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mon.2 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fb5e40173d0 con 0x7fb5f8103490 2024-04-24T23:04:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:25 smithi033 ceph-mon[29201]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:25 smithi033 ceph-mon[29201]: Standby manager daemon y started 2024-04-24T23:04:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:25 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:04:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:25 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:04:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:25 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:04:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:25 smithi033 ceph-mon[29201]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:04:25.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-mon[32450]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:25.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-mon[32450]: Standby manager daemon y started 2024-04-24T23:04:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:04:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:04:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:04:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:25 smithi038 ceph-mon[32450]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:04:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:25 smithi117 ceph-mon[32529]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:25 smithi117 ceph-mon[32529]: Standby manager daemon y started 2024-04-24T23:04:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:25 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:04:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:25 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:04:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:25 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:04:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:25 smithi117 ceph-mon[32529]: from='mgr.? 172.21.15.38:0/740079255' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:04:26.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:26 smithi033 ceph-mon[29201]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-24T23:04:26.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:04:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:26 smithi038 ceph-mon[32450]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-24T23:04:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:04:26.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:26 smithi117 ceph-mon[32529]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-24T23:04:26.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:04:27.715 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:27 smithi033 ceph-mon[29201]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:27.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:27 smithi038 ceph-mon[32450]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:27 smithi117 ceph-mon[32529]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.867 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.867 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.867 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:28.867 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:28.867 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:28.867 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:28.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:28.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:28.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:28.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:29.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:29 smithi033 ceph-mon[29201]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:29.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:29 smithi038 ceph-mon[32450]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:29.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:29 smithi117 ceph-mon[32529]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:30.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:04:30.802 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:30.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:04:30.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:04:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:31 smithi033 ceph-mon[29201]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:31 smithi033 ceph-mon[29201]: Deploying daemon osd.0 on smithi033 2024-04-24T23:04:31.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:31 smithi038 ceph-mon[32450]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:31.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:31 smithi038 ceph-mon[32450]: Deploying daemon osd.0 on smithi033 2024-04-24T23:04:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:31 smithi117 ceph-mon[32529]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:31 smithi117 ceph-mon[32529]: Deploying daemon osd.0 on smithi033 2024-04-24T23:04:33.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:33 smithi033 ceph-mon[29201]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:33 smithi038 ceph-mon[32450]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:33 smithi117 ceph-mon[32529]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:35 smithi038 ceph-mon[32450]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:35 smithi117 ceph-mon[32529]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:35.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:35 smithi033 ceph-mon[29201]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:35.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:35.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:35.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:37 smithi033 ceph-mon[29201]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:37 smithi038 ceph-mon[32450]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:37 smithi117 ceph-mon[32529]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:38 smithi033 ceph-mon[29201]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:39.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:39.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-04-24T23:04:39.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.022+0000 7fb5f5ffb700 1 -- 172.21.15.33:0/2644531345 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb5f8062730 con 0x7fb5e8039330 2024-04-24T23:04:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.024+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5e8039330 msgr2=0x7fb5e803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.024+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5e8039330 0x7fb5e803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b580 tx=0x7fb5ec005fb0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 msgr2=0x7fb5f8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f8198a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400e9a0 tx=0x7fb5e400ecb0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 shutdown_connections 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5e8039330 0x7fb5e803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f8103e30 0x7fb5f819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f8102290 0x7fb5f81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 --2- 172.21.15.33:0/2644531345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5f8103490 0x7fb5f8198a00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 >> 172.21.15.33:0/2644531345 conn(0x7fb5f80fd830 msgr2=0x7fb5f8106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 shutdown_connections 2024-04-24T23:04:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:04:39.025+0000 7fb5ff0c1700 1 -- 172.21.15.33:0/2644531345 wait complete. 2024-04-24T23:04:39.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:38 smithi038 ceph-mon[32450]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:38 smithi117 ceph-mon[32529]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:39.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:39.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:39.715 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T23:04:39.718 INFO:tasks.cephadm:Deploying osd.1 on smithi038 with /dev/vg_nvme/lv_4... 2024-04-24T23:04:39.718 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:04:39.973 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:04:40.098 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:40.099 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:40.099 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:40.099 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:39 smithi038 ceph-mon[32450]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:04:40.283 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:40.283 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:40.284 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:40.284 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:39 smithi033 ceph-mon[29201]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:04:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:39 smithi117 ceph-mon[32529]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:04:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:40 smithi033 ceph-mon[29201]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:40 smithi033 ceph-mon[29201]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:04:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:40 smithi033 ceph-mon[29201]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:04:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:40 smithi033 ceph-mon[29201]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:04:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:41.205 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:04:40 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0[44302]: 2024-04-24T23:04:40.997+0000 7f95336b3700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:04:41.206 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:04:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0[44302]: 2024-04-24T23:04:41.003+0000 7f952f4a6700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:04:41.254 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:40 smithi038 ceph-mon[32450]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:41.254 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:40 smithi038 ceph-mon[32450]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:04:41.254 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:40 smithi038 ceph-mon[32450]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:04:41.255 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:40 smithi038 ceph-mon[32450]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:04:41.255 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:40 smithi117 ceph-mon[32529]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:41.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:40 smithi117 ceph-mon[32529]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:04:41.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:40 smithi117 ceph-mon[32529]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:04:41.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:40 smithi117 ceph-mon[32529]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:04:41.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:41.940 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:04:41.940 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:04:41.940 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-24T23:04:41.940 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-24T23:04:41.940 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142271 s, 737 MB/s 2024-04-24T23:04:41.940 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-24T23:04:42.208 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:42 smithi038 ceph-mon[32450]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:04:42.208 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:42 smithi038 ceph-mon[32450]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:04:42.208 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.208 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.208 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:42 smithi117 ceph-mon[32529]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:04:42.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:42 smithi117 ceph-mon[32529]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:04:42.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:42 smithi033 ceph-mon[29201]: from='osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:04:42.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:42 smithi033 ceph-mon[29201]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:04:42.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:42.495 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-04-24T23:04:42.751 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] boot 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:43.103 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.104 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.104 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:43.104 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.104 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:43.104 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:43.104 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:04:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:04:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] boot 2024-04-24T23:04:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:04:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:43.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: osd.0 [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] boot 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:04:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:43.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:43.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:43.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:43.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:04:44.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:44 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi033 2024-04-24T23:04:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:44 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-24T23:04:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:44 smithi038 ceph-mon[32450]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:04:44.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.229+0000 7f43f9a72700 1 -- 172.21.15.38:0/3319361944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 msgr2=0x7f43f4103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.229+0000 7f43f9a72700 1 --2- 172.21.15.38:0/3319361944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4103910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43e80099c0 tx=0x7f43e8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 -- 172.21.15.38:0/3319361944 shutdown_connections 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 --2- 172.21.15.38:0/3319361944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f4103e50 0x7f43f410a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 --2- 172.21.15.38:0/3319361944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 0x7f43f41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 --2- 172.21.15.38:0/3319361944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4103910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 -- 172.21.15.38:0/3319361944 >> 172.21.15.38:0/3319361944 conn(0x7f43f40fd850 msgr2=0x7f43f40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 -- 172.21.15.38:0/3319361944 shutdown_connections 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 -- 172.21.15.38:0/3319361944 wait complete. 2024-04-24T23:04:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 Processor -- start 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.230+0000 7f43f9a72700 1 -- start start 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 0x7f43f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f4103e50 0x7f43f419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410df80 con 0x7f43f41022b0 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410de00 con 0x7f43f41034b0 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410e100 con 0x7f43f4103e50 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60038/0 (socket says 172.21.15.38:60038) 2024-04-24T23:04:44.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 -- 172.21.15.38:0/1860565441 learned_addr learned my addr 172.21.15.38:0/1860565441 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f37fe700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 0x7f43f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 -- 172.21.15.38:0/1860565441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f4103e50 msgr2=0x7f43f419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f4103e50 0x7f43f419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 -- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 msgr2=0x7f43f4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 0x7f43f4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 -- 172.21.15.38:0/1860565441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f41a01a0 con 0x7f43f41034b0 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f37fe700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 0x7f43f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f2ffd700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4198a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c010 tx=0x7f43e8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8026070 con 0x7f43f41034b0 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e8004210 con 0x7f43f41034b0 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f41a0430 con 0x7f43f41034b0 2024-04-24T23:04:44.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8021410 con 0x7f43f41034b0 2024-04-24T23:04:44.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.231+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41a0960 con 0x7f43f41034b0 2024-04-24T23:04:44.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.232+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f4068f20 con 0x7f43f41034b0 2024-04-24T23:04:44.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.234+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f43e8029070 con 0x7f43f41034b0 2024-04-24T23:04:44.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.234+0000 7f43f0ff9700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43dc0a2c10 0x7f43dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.234+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f43e80ce090 con 0x7f43f41034b0 2024-04-24T23:04:44.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.234+0000 7f43f37fe700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43dc0a2c10 0x7f43dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.235+0000 7f43f37fe700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43dc0a2c10 0x7f43dc0a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f43f4103310 tx=0x7f43e4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.239+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43e805d740 con 0x7f43f41034b0 2024-04-24T23:04:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:44 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi033 2024-04-24T23:04:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:44 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-24T23:04:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:44 smithi117 ceph-mon[32529]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:04:44.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:44 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi033 2024-04-24T23:04:44.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:44 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-24T23:04:44.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:44 smithi033 ceph-mon[29201]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:04:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:04:44.455+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f43f4062670 con 0x7f43dc0a2c10 2024-04-24T23:04:45.295 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:45 smithi038 ceph-mon[32450]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:45.295 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:45 smithi038 ceph-mon[32450]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:45.295 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:45 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:04:45.295 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:45 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:04:45.295 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:45 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:45.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:45 smithi117 ceph-mon[32529]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:45.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:45 smithi117 ceph-mon[32529]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:45.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:45 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:04:45.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:45 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:04:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:45 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:45 smithi033 ceph-mon[29201]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:45 smithi033 ceph-mon[29201]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:45.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:45 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:04:45.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:45 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:04:45.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:45 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:47.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:47 smithi038 ceph-mon[32450]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:47.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:47 smithi117 ceph-mon[32529]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:47 smithi033 ceph-mon[29201]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:48.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:48 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/139237649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]: dispatch 2024-04-24T23:04:48.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:48 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]: dispatch 2024-04-24T23:04:48.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:48 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]': finished 2024-04-24T23:04:48.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:48 smithi038 ceph-mon[32450]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:04:48.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:04:48.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:48 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/3842123473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:04:48.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:48 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/139237649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]: dispatch 2024-04-24T23:04:48.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:48 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]: dispatch 2024-04-24T23:04:48.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:48 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]': finished 2024-04-24T23:04:48.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:48 smithi117 ceph-mon[32529]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:04:48.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:04:48.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:48 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/3842123473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:04:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:48 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/139237649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]: dispatch 2024-04-24T23:04:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:48 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]: dispatch 2024-04-24T23:04:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:48 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c497d09-e263-481e-a4dc-5352cb3ae2d6"}]': finished 2024-04-24T23:04:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:48 smithi033 ceph-mon[29201]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:04:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:04:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:48 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/3842123473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:04:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:49 smithi038 ceph-mon[32450]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:49 smithi117 ceph-mon[32529]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:49.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:49 smithi033 ceph-mon[29201]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:51.380 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:51 smithi038 ceph-mon[32450]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:51.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:51 smithi117 ceph-mon[32529]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:51.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:51 smithi033 ceph-mon[29201]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:53.215 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:53 smithi038 ceph-mon[32450]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:53 smithi117 ceph-mon[32529]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:53 smithi033 ceph-mon[29201]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:55.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:55 smithi038 ceph-mon[32450]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:04:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:55 smithi038 ceph-mon[32450]: Deploying daemon osd.1 on smithi038 2024-04-24T23:04:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:55 smithi117 ceph-mon[32529]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:04:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:55 smithi117 ceph-mon[32529]: Deploying daemon osd.1 on smithi038 2024-04-24T23:04:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:55 smithi033 ceph-mon[29201]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:04:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:55 smithi033 ceph-mon[29201]: Deploying daemon osd.1 on smithi038 2024-04-24T23:04:57.381 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:57 smithi038 ceph-mon[32450]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:57 smithi117 ceph-mon[32529]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:57 smithi033 ceph-mon[29201]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:59.333 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:04:59 smithi038 ceph-mon[32450]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:59.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:04:59 smithi117 ceph-mon[32529]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:04:59.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:04:59 smithi033 ceph-mon[29201]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:00.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:00.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:00.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:00.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:01.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:01 smithi038 ceph-mon[32450]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:01 smithi117 ceph-mon[32529]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:01 smithi033 ceph-mon[29201]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:03.140 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:02 smithi038 ceph-mon[32450]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:03.141 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.141 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.141 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:03.141 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:03.141 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.223 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 1 on host 'smithi038' 2024-04-24T23:05:03.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.221+0000 7f43f0ff9700 1 -- 172.21.15.38:0/1860565441 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f43f4062670 con 0x7f43dc0a2c10 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43dc0a2c10 msgr2=0x7f43dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43dc0a2c10 0x7f43dc0a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f43f4103310 tx=0x7f43e4008040 comp rx=0 tx=0).stop 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 msgr2=0x7f43f4198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4198a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c010 tx=0x7f43e8009580 comp rx=0 tx=0).stop 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 shutdown_connections 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43dc0a2c10 0x7f43dc0a50d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f4103e50 0x7f43f419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f41022b0 0x7f43f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 --2- 172.21.15.38:0/1860565441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f41034b0 0x7f43f4198a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 >> 172.21.15.38:0/1860565441 conn(0x7f43f40fd850 msgr2=0x7f43f41067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.223+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 shutdown_connections 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:03.224+0000 7f43f9a72700 1 -- 172.21.15.38:0/1860565441 wait complete. 2024-04-24T23:05:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:02 smithi117 ceph-mon[32529]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:02 smithi033 ceph-mon[29201]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:03.838 DEBUG:teuthology.orchestra.run.smithi038:osd.1> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T23:05:03.840 INFO:tasks.cephadm:Deploying osd.2 on smithi117 with /dev/vg_nvme/lv_4... 2024-04-24T23:05:03.840 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:05:04.082 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:05:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:04 smithi038 ceph-mon[32450]: from='osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:04 smithi038 ceph-mon[32450]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:04 smithi033 ceph-mon[29201]: from='osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:04 smithi033 ceph-mon[29201]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:04.464 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:04.464 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:04.464 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:04.464 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:04 smithi117 ceph-mon[32529]: from='osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:04.464 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:04 smithi117 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:05.243 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:05 smithi117 ceph-mon[32529]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:05.244 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:05 smithi117 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:05:05.244 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:05 smithi117 ceph-mon[32529]: from='osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:05:05.244 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:05 smithi117 ceph-mon[32529]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:05:05.244 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:05.244 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:05 smithi117 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:05:05.326 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-mon[32450]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:05.326 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-mon[32450]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:05:05.326 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-mon[32450]: from='osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:05:05.326 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-mon[32450]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:05:05.326 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:05.326 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-mon[32450]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:05:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:05 smithi033 ceph-mon[29201]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:05 smithi033 ceph-mon[29201]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:05:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:05 smithi033 ceph-mon[29201]: from='osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:05:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:05 smithi033 ceph-mon[29201]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:05:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:05 smithi033 ceph-mon[29201]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:05:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:05:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:05:05.765 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-24T23:05:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-24T23:05:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114341 s, 917 MB/s 2024-04-24T23:05:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-24T23:05:05.921 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1[37044]: 2024-04-24T23:05:05.623+0000 7f0fcbc0a700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:05:05.921 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1[37044]: 2024-04-24T23:05:05.629+0000 7f0fc39f5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:05:06.317 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-04-24T23:05:06.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:06 smithi038 ceph-mon[32450]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:05:06.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:06 smithi038 ceph-mon[32450]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:05:06.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:06.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:06.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:06 smithi038 ceph-mon[32450]: from='osd.1 ' entity='osd.1' 2024-04-24T23:05:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:06 smithi033 ceph-mon[29201]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:05:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:06 smithi033 ceph-mon[29201]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:05:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:06.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:06 smithi033 ceph-mon[29201]: from='osd.1 ' entity='osd.1' 2024-04-24T23:05:06.552 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:05:06.594 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:06 smithi117 ceph-mon[32529]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:05:06.595 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:06 smithi117 ceph-mon[32529]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:05:06.595 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:06.595 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:06.595 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:06 smithi117 ceph-mon[32529]: from='osd.1 ' entity='osd.1' 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] boot 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:05:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:07.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] boot 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:07.530 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.531 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.531 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:07.531 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.531 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:07.531 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:07.531 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: osd.1 [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] boot 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:07.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 -- 172.21.15.117:0/751420613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b01013d0 msgr2=0x7f01b0101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 --2- 172.21.15.117:0/751420613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b01013d0 0x7f01b0101850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0198009990 tx=0x7f0198009ca0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 -- 172.21.15.117:0/751420613 shutdown_connections 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 --2- 172.21.15.117:0/751420613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0101d90 0x7f01b0109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 --2- 172.21.15.117:0/751420613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b0102800 0x7f01b0102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 --2- 172.21.15.117:0/751420613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b01013d0 0x7f01b0101850 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 -- 172.21.15.117:0/751420613 >> 172.21.15.117:0/751420613 conn(0x7f01b00fbec0 msgr2=0x7f01b00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:08.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.042+0000 7f01b64ac700 1 -- 172.21.15.117:0/751420613 shutdown_connections 2024-04-24T23:05:08.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.043+0000 7f01b64ac700 1 -- 172.21.15.117:0/751420613 wait complete. 2024-04-24T23:05:08.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.043+0000 7f01b64ac700 1 Processor -- start 2024-04-24T23:05:08.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.043+0000 7f01b64ac700 1 -- start start 2024-04-24T23:05:08.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.043+0000 7f01b64ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b01013d0 0x7f01b0100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.043+0000 7f01b64ac700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 0x7f01b019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.043+0000 7f01b64ac700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0102800 0x7f01b019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b64ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010cdc0 con 0x7f01b01013d0 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b64ac700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010cc40 con 0x7f01b0102800 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b64ac700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010cf40 con 0x7f01b0101d90 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 0x7f01b019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 0x7f01b019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:32884/0 (socket says 172.21.15.117:32884) 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 -- 172.21.15.117:0/4034356697 learned_addr learned my addr 172.21.15.117:0/4034356697 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b4a49700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0102800 0x7f01b019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01affff700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b01013d0 0x7f01b0100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 -- 172.21.15.117:0/4034356697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0102800 msgr2=0x7f01b019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0102800 0x7f01b019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 -- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b01013d0 msgr2=0x7f01b0100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b01013d0 0x7f01b0100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 -- 172.21.15.117:0/4034356697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b01a0110 con 0x7f01b0101d90 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b4a49700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0102800 0x7f01b019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01a7fff700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 0x7f01b019d200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0198003890 tx=0x7f01980047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198027070 con 0x7f01b0101d90 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0198004150 con 0x7f01b0101d90 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019800f460 con 0x7f01b0101d90 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b01a03a0 con 0x7f01b0101d90 2024-04-24T23:05:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.044+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b01a07a0 con 0x7f01b0101d90 2024-04-24T23:05:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.045+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b0068f20 con 0x7f01b0101d90 2024-04-24T23:05:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.046+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f019802a070 con 0x7f01b0101d90 2024-04-24T23:05:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.047+0000 7f01adffb700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f019c0a2c10 0x7f019c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.047+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f01980ce4f0 con 0x7f01b0101d90 2024-04-24T23:05:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.047+0000 7f01affff700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f019c0a2c10 0x7f019c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.048+0000 7f01affff700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f019c0a2c10 0x7f019c0a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005950 tx=0x7f01a00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:08.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.052+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0198013070 con 0x7f01b0101d90 2024-04-24T23:05:08.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:08.258+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f01b0062670 con 0x7f019c0a2c10 2024-04-24T23:05:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:08 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi038 2024-04-24T23:05:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:08 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi038 to 17093M 2024-04-24T23:05:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:08 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi038 2024-04-24T23:05:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:08 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi038 to 17093M 2024-04-24T23:05:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:08 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi038 2024-04-24T23:05:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:08 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi038 to 17093M 2024-04-24T23:05:09.388 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:09 smithi117 ceph-mon[32529]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:09.388 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:09 smithi117 ceph-mon[32529]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:05:09.388 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:09 smithi117 ceph-mon[32529]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:09.388 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:09.388 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:09.388 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:09 smithi038 ceph-mon[32450]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:09 smithi038 ceph-mon[32450]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:05:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:09 smithi038 ceph-mon[32450]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:09 smithi033 ceph-mon[29201]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:09 smithi033 ceph-mon[29201]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:05:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:09 smithi033 ceph-mon[29201]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:11 smithi117 ceph-mon[32529]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:11 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/3666949758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]: dispatch 2024-04-24T23:05:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:11 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]: dispatch 2024-04-24T23:05:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:11 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]': finished 2024-04-24T23:05:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:11 smithi117 ceph-mon[32529]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:05:11.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:11 smithi038 ceph-mon[32450]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:11 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/3666949758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]: dispatch 2024-04-24T23:05:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:11 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]: dispatch 2024-04-24T23:05:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:11 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]': finished 2024-04-24T23:05:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:11 smithi038 ceph-mon[32450]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:05:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:11 smithi033 ceph-mon[29201]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:11 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/3666949758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]: dispatch 2024-04-24T23:05:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:11 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]: dispatch 2024-04-24T23:05:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:11 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "956cb2b5-6d5f-4817-b066-74c8227c7960"}]': finished 2024-04-24T23:05:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:11 smithi033 ceph-mon[29201]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:05:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:12 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/2851343654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:12.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:12 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/2851343654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:12.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:12 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/2851343654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:13 smithi038 ceph-mon[32450]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:13 smithi117 ceph-mon[32529]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:13 smithi033 ceph-mon[29201]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:15.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:15 smithi117 ceph-mon[32529]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:15 smithi038 ceph-mon[32450]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:15 smithi033 ceph-mon[29201]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:17.488 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:17 smithi038 ceph-mon[32450]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:17.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:17 smithi117 ceph-mon[32529]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:17 smithi033 ceph-mon[29201]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:18.532 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:05:18.532 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:18.673 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:05:18.673 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:05:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:19.532 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:19 smithi117 ceph-mon[32529]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:19.532 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:19 smithi117 ceph-mon[32529]: Deploying daemon osd.2 on smithi117 2024-04-24T23:05:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:19 smithi038 ceph-mon[32450]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:19 smithi038 ceph-mon[32450]: Deploying daemon osd.2 on smithi117 2024-04-24T23:05:19.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:19 smithi033 ceph-mon[29201]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:19.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:19 smithi033 ceph-mon[29201]: Deploying daemon osd.2 on smithi117 2024-04-24T23:05:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:21 smithi038 ceph-mon[32450]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:21 smithi117 ceph-mon[32529]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:21 smithi033 ceph-mon[29201]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:23 smithi117 ceph-mon[32529]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:23.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:23.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:23.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:23.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:23 smithi038 ceph-mon[32450]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:23 smithi033 ceph-mon[29201]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:25.276 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:25 smithi117 ceph-mon[32529]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:25 smithi038 ceph-mon[32450]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:25 smithi033 ceph-mon[29201]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:26.278 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 2 on host 'smithi117' 2024-04-24T23:05:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.277+0000 7f01adffb700 1 -- 172.21.15.117:0/4034356697 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f01b0062670 con 0x7f019c0a2c10 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f019c0a2c10 msgr2=0x7f019c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f019c0a2c10 0x7f019c0a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005950 tx=0x7f01a00058e0 comp rx=0 tx=0).stop 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 msgr2=0x7f01b019d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 0x7f01b019d200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0198003890 tx=0x7f01980047b0 comp rx=0 tx=0).stop 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 shutdown_connections 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f019c0a2c10 0x7f019c0a50d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01b0102800 0x7f01b019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b01013d0 0x7f01b0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 --2- 172.21.15.117:0/4034356697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b0101d90 0x7f01b019d200 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 >> 172.21.15.117:0/4034356697 conn(0x7f01b00fbec0 msgr2=0x7f01b01056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 shutdown_connections 2024-04-24T23:05:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:05:26.279+0000 7f01b64ac700 1 -- 172.21.15.117:0/4034356697 wait complete. 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:26.919 DEBUG:teuthology.orchestra.run.smithi117:osd.2> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T23:05:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:26.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:26.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:26.921 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-04-24T23:05:26.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:05:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:27.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:27 smithi033 ceph-mon[29201]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:27.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:27 smithi033 ceph-mon[29201]: from='osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:05:27.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:27 smithi033 ceph-mon[29201]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:05:27.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:27 smithi038 ceph-mon[32450]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:27.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:27 smithi038 ceph-mon[32450]: from='osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:05:27.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:27 smithi038 ceph-mon[32450]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:05:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:27 smithi117 ceph-mon[32529]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:27 smithi117 ceph-mon[32529]: from='osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:05:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:27 smithi117 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:05:28.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:28 smithi033 ceph-mon[29201]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:05:28.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:28 smithi033 ceph-mon[29201]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:05:28.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:28.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:28 smithi033 ceph-mon[29201]: from='osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:05:28.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:28 smithi033 ceph-mon[29201]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:05:28.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:28 smithi038 ceph-mon[32450]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:05:28.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:28 smithi038 ceph-mon[32450]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:05:28.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:28.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:28 smithi038 ceph-mon[32450]: from='osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:05:28.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:28 smithi038 ceph-mon[32450]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:05:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:05:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-mon[32529]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:05:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-mon[32529]: from='osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:05:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:05:29.176 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2[37657]: 2024-04-24T23:05:28.916+0000 7f784ee79700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:05:29.176 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:28 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2[37657]: 2024-04-24T23:05:28.920+0000 7f7846c64700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:05:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:05:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:05:29.579 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-24T23:05:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-24T23:05:29.580 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115184 s, 910 MB/s 2024-04-24T23:05:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-24T23:05:29.604 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:29 smithi117 ceph-mon[32529]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:29.604 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:29 smithi117 ceph-mon[32529]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:05:29.604 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:29 smithi117 ceph-mon[32529]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:05:29.604 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:29 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:29.604 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:29 smithi117 ceph-mon[32529]: from='osd.2 ' entity='osd.2' 2024-04-24T23:05:29.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:29 smithi038 ceph-mon[32450]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:29.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:29 smithi038 ceph-mon[32450]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:05:29.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:29 smithi038 ceph-mon[32450]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:05:29.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:29 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:29.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:29 smithi038 ceph-mon[32450]: from='osd.2 ' entity='osd.2' 2024-04-24T23:05:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:29 smithi033 ceph-mon[29201]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:29 smithi033 ceph-mon[29201]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:05:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:29 smithi033 ceph-mon[29201]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:05:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:29 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:29 smithi033 ceph-mon[29201]: from='osd.2 ' entity='osd.2' 2024-04-24T23:05:30.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-04-24T23:05:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] boot 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:30.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:30.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] boot 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:30.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: osd.2 [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] boot 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:30.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:30.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:30.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:31 smithi033 ceph-mon[29201]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:31 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi117 2024-04-24T23:05:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:31 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi117 to 17088M 2024-04-24T23:05:31.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:31 smithi038 ceph-mon[32450]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:31.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:31 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi117 2024-04-24T23:05:31.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:31 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi117 to 17088M 2024-04-24T23:05:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:31 smithi117 ceph-mon[32529]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:31 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi117 2024-04-24T23:05:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:31 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi117 to 17088M 2024-04-24T23:05:32.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 -- 172.21.15.33:0/4022826339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102390 msgr2=0x7f5dd81027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4022826339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102390 0x7f5dd81027f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80099c0 tx=0x7f5dc8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 -- 172.21.15.33:0/4022826339 shutdown_connections 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4022826339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd8102d30 0x7f5dd81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4022826339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dd8101190 0x7f5dd81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4022826339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102390 0x7f5dd81027f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.390+0000 7f5ddff14700 1 -- 172.21.15.33:0/4022826339 >> 172.21.15.33:0/4022826339 conn(0x7f5dd80fc730 msgr2=0x7f5dd80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.391+0000 7f5ddff14700 1 -- 172.21.15.33:0/4022826339 shutdown_connections 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.391+0000 7f5ddff14700 1 -- 172.21.15.33:0/4022826339 wait complete. 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.391+0000 7f5ddff14700 1 Processor -- start 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.391+0000 7f5ddff14700 1 -- start start 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddff14700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd8101190 0x7f5dd81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddff14700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dd8102390 0x7f5dd8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddff14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 0x7f5dd819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddff14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810cd90 con 0x7f5dd8102d30 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddff14700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810cc10 con 0x7f5dd8102390 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddff14700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810cf10 con 0x7f5dd8101190 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 0x7f5dd819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 0x7f5dd819fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60310/0 (socket says 172.21.15.33:60310) 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 -- 172.21.15.33:0/4134068825 learned_addr learned my addr 172.21.15.33:0/4134068825 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5ddd4af700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dd8102390 0x7f5dd8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dddcb0700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd8101190 0x7f5dd81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 -- 172.21.15.33:0/4134068825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd8101190 msgr2=0x7f5dd81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd8101190 0x7f5dd81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 -- 172.21.15.33:0/4134068825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dd8102390 msgr2=0x7f5dd8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dd8102390 0x7f5dd8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 -- 172.21.15.33:0/4134068825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd81a0120 con 0x7f5dd8102d30 2024-04-24T23:05:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dde4b1700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 0x7f5dd819fa80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000ef60 tx=0x7f5dd000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00092e0 con 0x7f5dd8102d30 2024-04-24T23:05:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd0004bb0 con 0x7f5dd8102d30 2024-04-24T23:05:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.392+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0004280 con 0x7f5dd8102d30 2024-04-24T23:05:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.393+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd81a03b0 con 0x7f5dd8102d30 2024-04-24T23:05:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.393+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd81a0940 con 0x7f5dd8102d30 2024-04-24T23:05:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.393+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd8068f20 con 0x7f5dd8102d30 2024-04-24T23:05:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.395+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5dd001b030 con 0x7f5dd8102d30 2024-04-24T23:05:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.396+0000 7f5dceffd700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5dc40a2c10 0x7f5dc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.396+0000 7f5dddcb0700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5dc40a2c10 0x7f5dc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.396+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f5dd00cdd50 con 0x7f5dd8102d30 2024-04-24T23:05:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.396+0000 7f5dddcb0700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5dc40a2c10 0x7f5dc40a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dd81021f0 tx=0x7f5dd4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.400+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5dd0022020 con 0x7f5dd8102d30 2024-04-24T23:05:32.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:32.621+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5dd81a0c60 con 0x7f5dc40a2c10 2024-04-24T23:05:32.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:32 smithi033 ceph-mon[29201]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:05:32.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:05:32.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:32 smithi038 ceph-mon[32450]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:05:32.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:05:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:32 smithi117 ceph-mon[32529]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:05:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:05:33.530 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:33.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:33.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:34.287 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[48981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:05:34.287 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[48981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:34.287 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[48981]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:34.287 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[48981]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:34.500+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5dd0012050 con 0x7f5dd8102d30 2024-04-24T23:05:34.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:05:34.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:34 smithi033 ceph-mon[29201]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:05:34.594 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[49005]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:05:34.594 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[49005]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:34.594 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[49005]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:34.594 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:05:34 smithi033 sudo[49005]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:05:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:34 smithi038 ceph-mon[32450]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:05:34.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:05:34.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:34 smithi117 ceph-mon[32529]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:05:35.420 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:34 smithi038 sudo[40742]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:05:35.420 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:34 smithi038 sudo[40742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:35.421 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:34 smithi038 sudo[40742]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:35.421 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:34 smithi038 sudo[40742]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:35.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3868352907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec098a5-9343-4caf-931d-f4c79d4fbdf3"}]: dispatch 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3868352907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ec098a5-9343-4caf-931d-f4c79d4fbdf3"}]': finished 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:05:35.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:35.921 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:35 smithi038 sudo[40745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:05:35.921 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:35 smithi038 sudo[40745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:35.921 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:35 smithi038 sudo[40745]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:35.921 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:05:35 smithi038 sudo[40745]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:05:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T23:05:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3868352907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec098a5-9343-4caf-931d-f4c79d4fbdf3"}]: dispatch 2024-04-24T23:05:35.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3868352907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ec098a5-9343-4caf-931d-f4c79d4fbdf3"}]': finished 2024-04-24T23:05:35.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:05:35.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:35.926 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41716]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:05:35.926 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41716]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:35.926 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41716]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:35.926 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41716]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:05:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T23:05:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:35.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3868352907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec098a5-9343-4caf-931d-f4c79d4fbdf3"}]: dispatch 2024-04-24T23:05:35.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3868352907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ec098a5-9343-4caf-931d-f4c79d4fbdf3"}]': finished 2024-04-24T23:05:35.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:05:35.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:36.426 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:05:36.426 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:36.426 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41719]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:36.426 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:05:35 smithi117 sudo[41719]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49139]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:05:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49139]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49139]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49327]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:05:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49327]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49327]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:36 smithi033 sudo[49327]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:37.395 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:05:37.395 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:37.395 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:37.396 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41722]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1236593937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:37.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41725]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 sudo[41725]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1236593937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:37.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1236593937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:38.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 sudo[40748]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:05:38.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 sudo[40748]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:38.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 sudo[40748]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:38.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:37 smithi038 sudo[40748]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:38.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 sudo[40751]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 sudo[40751]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 sudo[40751]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 sudo[40751]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:38 smithi038 ceph-mon[32450]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:38 smithi117 ceph-mon[32529]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:05:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:05:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:05:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:38 smithi033 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:05:39.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:39 smithi033 ceph-mon[29201]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:39 smithi038 ceph-mon[32450]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:39.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:39 smithi117 ceph-mon[32529]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:41.665 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:41 smithi033 ceph-mon[29201]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:41.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:41 smithi038 ceph-mon[32450]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:41.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:41 smithi117 ceph-mon[32529]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:43.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:43 smithi117 ceph-mon[32529]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:43.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:05:43.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:43 smithi033 ceph-mon[29201]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:05:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:43 smithi038 ceph-mon[32450]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:05:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:44.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:44 smithi033 ceph-mon[29201]: Deploying daemon osd.3 on smithi033 2024-04-24T23:05:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:44 smithi038 ceph-mon[32450]: Deploying daemon osd.3 on smithi033 2024-04-24T23:05:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:44 smithi117 ceph-mon[32529]: Deploying daemon osd.3 on smithi033 2024-04-24T23:05:45.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:45 smithi033 ceph-mon[29201]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-24T23:05:45.826 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:45 smithi117 ceph-mon[32529]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-24T23:05:45.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:45 smithi038 ceph-mon[32450]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-24T23:05:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:47 smithi033 ceph-mon[29201]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-24T23:05:47.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:47 smithi038 ceph-mon[32450]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-24T23:05:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:47 smithi117 ceph-mon[32529]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-24T23:05:49.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:49 smithi033 ceph-mon[29201]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-24T23:05:49.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:49.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:49.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:49 smithi038 ceph-mon[32450]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-24T23:05:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:49.943 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:49 smithi117 ceph-mon[32529]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-24T23:05:49.944 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:49.944 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:49.944 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:51.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:51 smithi033 ceph-mon[29201]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-24T23:05:51.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:51 smithi038 ceph-mon[32450]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-24T23:05:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:51 smithi117 ceph-mon[32529]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-24T23:05:52.129 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-04-24T23:05:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.127+0000 7f5dceffd700 1 -- 172.21.15.33:0/4134068825 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5dd81a0c60 con 0x7f5dc40a2c10 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5dc40a2c10 msgr2=0x7f5dc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5dc40a2c10 0x7f5dc40a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dd81021f0 tx=0x7f5dd4009260 comp rx=0 tx=0).stop 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 msgr2=0x7f5dd819fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 0x7f5dd819fa80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000ef60 tx=0x7f5dd000ef90 comp rx=0 tx=0).stop 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 shutdown_connections 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5dc40a2c10 0x7f5dc40a50d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd8102d30 0x7f5dd819fa80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd8101190 0x7f5dd81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 --2- 172.21.15.33:0/4134068825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dd8102390 0x7f5dd8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 >> 172.21.15.33:0/4134068825 conn(0x7f5dd80fc730 msgr2=0x7f5dd80fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 shutdown_connections 2024-04-24T23:05:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:05:52.129+0000 7f5ddff14700 1 -- 172.21.15.33:0/4134068825 wait complete. 2024-04-24T23:05:52.801 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T23:05:52.803 INFO:tasks.cephadm:Deploying osd.4 on smithi038 with /dev/vg_nvme/lv_3... 2024-04-24T23:05:52.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:05:53.059 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:05:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:53 smithi038 ceph-mon[32450]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-24T23:05:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:53 smithi038 ceph-mon[32450]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:05:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:53 smithi117 ceph-mon[32529]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-24T23:05:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:53 smithi117 ceph-mon[32529]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:05:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:53 smithi033 ceph-mon[29201]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-24T23:05:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:53 smithi033 ceph-mon[29201]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:05:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:54.747 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:54 smithi038 ceph-mon[32450]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:05:54.747 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:54 smithi038 ceph-mon[32450]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:05:54.747 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:54 smithi038 ceph-mon[32450]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:05:54.747 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:54.776 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:54 smithi033 ceph-mon[29201]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:05:54.776 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:54 smithi033 ceph-mon[29201]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:05:54.776 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:54 smithi033 ceph-mon[29201]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:05:54.777 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:54.777 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:05:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3[51096]: 2024-04-24T23:05:54.469+0000 7ff1b0f38700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:05:54.777 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:05:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3[51096]: 2024-04-24T23:05:54.477+0000 7ff1acd2b700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:05:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:54 smithi117 ceph-mon[32529]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:05:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:54 smithi117 ceph-mon[32529]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:05:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:54 smithi117 ceph-mon[32529]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:05:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.114 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:05:55.114 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:05:55.114 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-24T23:05:55.114 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-24T23:05:55.114 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0070721 s, 1.5 GB/s 2024-04-24T23:05:55.114 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-24T23:05:55.668 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] boot 2024-04-24T23:05:55.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:05:55.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:55.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:05:55.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T23:05:55.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] boot 2024-04-24T23:05:55.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:05:55.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.923 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:05:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:05:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: from='osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:05:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T23:05:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: osd.3 [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] boot 2024-04-24T23:05:55.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:05:55.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:05:56.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:05:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:05:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:05:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 -- 172.21.15.38:0/2966311298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103490 msgr2=0x7f58401038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 --2- 172.21.15.38:0/2966311298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103490 0x7f58401038f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f58340099c0 tx=0x7f5834009cd0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 -- 172.21.15.38:0/2966311298 shutdown_connections 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 --2- 172.21.15.38:0/2966311298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5840103e30 0x7f584010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 --2- 172.21.15.38:0/2966311298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 0x7f58401026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 --2- 172.21.15.38:0/2966311298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103490 0x7f58401038f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 -- 172.21.15.38:0/2966311298 >> 172.21.15.38:0/2966311298 conn(0x7f58400fd830 msgr2=0x7f58400ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 -- 172.21.15.38:0/2966311298 shutdown_connections 2024-04-24T23:05:57.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 -- 172.21.15.38:0/2966311298 wait complete. 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.635+0000 7f5845318700 1 Processor -- start 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 -- start start 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 0x7f5840075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5840103490 0x7f5840074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 0x7f5840074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f584010de50 con 0x7f5840103490 2024-04-24T23:05:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f584010dcd0 con 0x7f5840103e30 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f5845318700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f584010dfd0 con 0x7f5840102290 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 0x7f5840074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 0x7f5840074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46464/0 (socket says 172.21.15.38:46464) 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 -- 172.21.15.38:0/3269383421 learned_addr learned my addr 172.21.15.38:0/3269383421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583e7fc700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5840103490 0x7f5840074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583effd700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 0x7f5840075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 -- 172.21.15.38:0/3269383421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 msgr2=0x7f5840075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 0x7f5840075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 -- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5840103490 msgr2=0x7f5840074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5840103490 0x7f5840074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 -- 172.21.15.38:0/3269383421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5840075410 con 0x7f5840103e30 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583effd700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 0x7f5840075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.636+0000 7f583f7fe700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 0x7f5840074b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5838007d70 tx=0x7f583800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.637+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838016020 con 0x7f5840103e30 2024-04-24T23:05:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.637+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58401a2510 con 0x7f5840103e30 2024-04-24T23:05:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.637+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583800ad40 con 0x7f5840103e30 2024-04-24T23:05:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.637+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838011600 con 0x7f5840103e30 2024-04-24T23:05:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.637+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58401a29d0 con 0x7f5840103e30 2024-04-24T23:05:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.638+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5840068f20 con 0x7f5840103e30 2024-04-24T23:05:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.639+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5838019020 con 0x7f5840103e30 2024-04-24T23:05:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.639+0000 7f5827fff700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58280a2c20 0x7f58280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.639+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f58380ce180 con 0x7f5840103e30 2024-04-24T23:05:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.640+0000 7f583effd700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58280a2c20 0x7f58280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.640+0000 7f583effd700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58280a2c20 0x7f58280a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58401032f0 tx=0x7f583000a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.643+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5838020020 con 0x7f5840103e30 2024-04-24T23:05:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:57 smithi038 ceph-mon[32450]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:05:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:57 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi033 2024-04-24T23:05:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:57 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-24T23:05:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:57 smithi038 ceph-mon[32450]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:05:57.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:57 smithi033 ceph-mon[29201]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:05:57.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:57 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi033 2024-04-24T23:05:57.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:57 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-24T23:05:57.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:57 smithi033 ceph-mon[29201]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:05:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:05:57.858+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5840062670 con 0x7f58280a2c20 2024-04-24T23:05:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:57 smithi117 ceph-mon[32529]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:05:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:57 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi033 2024-04-24T23:05:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:57 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-24T23:05:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:57 smithi117 ceph-mon[32529]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:05:58.813 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:58 smithi038 ceph-mon[32450]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:05:58.814 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:58.814 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:58.814 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:58 smithi117 ceph-mon[32529]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:05:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:58 smithi033 ceph-mon[29201]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:05:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:59.703 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:59 smithi038 ceph-mon[32450]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-04-24T23:05:59.703 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:59 smithi038 ceph-mon[32450]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:59.703 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:05:59 smithi038 ceph-mon[32450]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T23:05:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:59 smithi117 ceph-mon[32529]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-04-24T23:05:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:59 smithi117 ceph-mon[32529]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:59.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:05:59 smithi117 ceph-mon[32529]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T23:05:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:59 smithi033 ceph-mon[29201]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-04-24T23:05:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:59 smithi033 ceph-mon[29201]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:05:59 smithi033 ceph-mon[29201]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/818713260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]: dispatch 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]: dispatch 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]': finished 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/2954715249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/818713260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]: dispatch 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]: dispatch 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]': finished 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/2954715249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/818713260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]: dispatch 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]: dispatch 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34a597a4-f838-4edf-9198-7d674b4aa9d1"}]': finished 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/2954715249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:03 smithi038 ceph-mon[32450]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:06:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:03 smithi117 ceph-mon[32529]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:06:03.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:03 smithi033 ceph-mon[29201]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:06:05.894 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:05 smithi038 ceph-mon[32450]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:05 smithi117 ceph-mon[32529]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:05.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:05 smithi033 ceph-mon[29201]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:07.873 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:07 smithi038 ceph-mon[32450]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:07 smithi117 ceph-mon[32529]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:07 smithi033 ceph-mon[29201]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:08.852 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:06:08.853 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:08.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:06:08.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:08.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:06:08.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:09.884 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:09 smithi038 ceph-mon[32450]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:09.884 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:09 smithi038 ceph-mon[32450]: Deploying daemon osd.4 on smithi038 2024-04-24T23:06:09.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:09 smithi033 ceph-mon[29201]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:09.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:09 smithi033 ceph-mon[29201]: Deploying daemon osd.4 on smithi038 2024-04-24T23:06:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:09 smithi117 ceph-mon[32529]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:09 smithi117 ceph-mon[32529]: Deploying daemon osd.4 on smithi038 2024-04-24T23:06:11.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:11 smithi038 ceph-mon[32450]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:11.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:11 smithi117 ceph-mon[32529]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:11.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:11 smithi033 ceph-mon[29201]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:13 smithi038 ceph-mon[32450]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:13 smithi117 ceph-mon[32529]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:13.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:13 smithi033 ceph-mon[29201]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:15.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:14 smithi038 ceph-mon[32450]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:15.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:15.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:15.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:14 smithi117 ceph-mon[32529]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:14 smithi033 ceph-mon[29201]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:16.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:16 smithi038 ceph-mon[32450]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:16 smithi117 ceph-mon[32529]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:16 smithi033 ceph-mon[29201]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:17.733 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 4 on host 'smithi038' 2024-04-24T23:06:17.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.732+0000 7f5827fff700 1 -- 172.21.15.38:0/3269383421 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5840062670 con 0x7f58280a2c20 2024-04-24T23:06:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.734+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58280a2c20 msgr2=0x7f58280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.734+0000 7f5845318700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58280a2c20 0x7f58280a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58401032f0 tx=0x7f583000a680 comp rx=0 tx=0).stop 2024-04-24T23:06:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 msgr2=0x7f5840074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 0x7f5840074b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5838007d70 tx=0x7f583800a340 comp rx=0 tx=0).stop 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 shutdown_connections 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58280a2c20 0x7f58280a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5840103e30 0x7f5840074b60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5840102290 0x7f5840075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 --2- 172.21.15.38:0/3269383421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5840103490 0x7f5840074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 >> 172.21.15.38:0/3269383421 conn(0x7f58400fd830 msgr2=0x7f5840106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 shutdown_connections 2024-04-24T23:06:17.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:06:17.735+0000 7f5845318700 1 -- 172.21.15.38:0/3269383421 wait complete. 2024-04-24T23:06:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:06:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:17 smithi038 ceph-mon[32450]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:06:18.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:18.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:18.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:06:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:17 smithi117 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:06:18.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:06:18.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:17 smithi033 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:06:18.448 DEBUG:teuthology.orchestra.run.smithi038:osd.4> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T23:06:18.491 INFO:tasks.cephadm:Deploying osd.5 on smithi117 with /dev/vg_nvme/lv_3... 2024-04-24T23:06:18.491 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:06:18.738 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:06:19.000 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:18 smithi038 ceph-mon[32450]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:19.000 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:18 smithi038 ceph-mon[32450]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:06:19.000 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:18 smithi038 ceph-mon[32450]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:06:19.000 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:19.000 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:18 smithi038 ceph-mon[32450]: from='osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:06:19.000 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:18 smithi038 ceph-mon[32450]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:06:19.136 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:18 smithi117 ceph-mon[32529]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:19.136 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:18 smithi117 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:06:19.136 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:18 smithi117 ceph-mon[32529]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:06:19.136 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:19.136 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:18 smithi117 ceph-mon[32529]: from='osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:06:19.136 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:18 smithi117 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:06:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:18 smithi033 ceph-mon[29201]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:18 smithi033 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:06:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:18 smithi033 ceph-mon[29201]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:06:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:18 smithi033 ceph-mon[29201]: from='osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:06:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:18 smithi033 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:06:19.914 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:06:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-24T23:06:19.706+0000 7f8f242ca700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:06:19.914 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:06:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-24T23:06:19.713+0000 7f8f200bd700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:06:20.699 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:06:20.700 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:06:20.700 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-24T23:06:20.700 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-24T23:06:20.700 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113307 s, 925 MB/s 2024-04-24T23:06:20.700 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-24T23:06:21.029 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:20 smithi117 ceph-mon[32529]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:06:21.029 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:20 smithi117 ceph-mon[32529]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T23:06:21.029 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.030 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.030 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:20 smithi117 ceph-mon[32529]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:21.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:20 smithi038 ceph-mon[32450]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:06:21.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:20 smithi038 ceph-mon[32450]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T23:06:21.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:20 smithi038 ceph-mon[32450]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:20 smithi033 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:06:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:20 smithi033 ceph-mon[29201]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T23:06:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:20 smithi033 ceph-mon[29201]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:21.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-04-24T23:06:21.558 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:06:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] boot 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi038 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:21.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:21.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.941 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:06:21.941 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:06:21.941 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] boot 2024-04-24T23:06:21.941 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi038 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:21.942 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:06:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:06:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: osd.4 [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] boot 2024-04-24T23:06:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi038 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:22.989 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:22 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi038 to 8546M 2024-04-24T23:06:22.990 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:22 smithi117 ceph-mon[32529]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T23:06:22.990 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:22 smithi117 ceph-mon[32529]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:22 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi038 to 8546M 2024-04-24T23:06:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:22 smithi038 ceph-mon[32450]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T23:06:23.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:22 smithi038 ceph-mon[32450]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:22 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi038 to 8546M 2024-04-24T23:06:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:22 smithi033 ceph-mon[29201]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T23:06:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:22 smithi033 ceph-mon[29201]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.198+0000 7f924acfe700 1 -- 172.21.15.117:0/201078024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441011b0 msgr2=0x7f92441015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.198+0000 7f924acfe700 1 --2- 172.21.15.117:0/201078024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441011b0 0x7f92441015d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238009ca0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- 172.21.15.117:0/201078024 shutdown_connections 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 --2- 172.21.15.117:0/201078024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9244102d50 0x7f9244109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 --2- 172.21.15.117:0/201078024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441011b0 0x7f92441015d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 --2- 172.21.15.117:0/201078024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92441023b0 0x7f9244102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- 172.21.15.117:0/201078024 >> 172.21.15.117:0/201078024 conn(0x7f92440fc790 msgr2=0x7f92440febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- 172.21.15.117:0/201078024 shutdown_connections 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- 172.21.15.117:0/201078024 wait complete. 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 Processor -- start 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- start start 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92441011b0 0x7f924406b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 0x7f924419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9244102d50 0x7f924419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f924410ce70 con 0x7f92441011b0 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f924410ccf0 con 0x7f9244102d50 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f924acfe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f924410cff0 con 0x7f92441023b0 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f9243fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 0x7f924419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f9243fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 0x7f924419d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53014/0 (socket says 172.21.15.117:53014) 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.199+0000 7f9243fff700 1 -- 172.21.15.117:0/3338923473 learned_addr learned my addr 172.21.15.117:0/3338923473 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9248a9a700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92441011b0 0x7f924406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f924929b700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9244102d50 0x7f924419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9243fff700 1 -- 172.21.15.117:0/3338923473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9244102d50 msgr2=0x7f924419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9243fff700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9244102d50 0x7f924419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9243fff700 1 -- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92441011b0 msgr2=0x7f924406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9243fff700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92441011b0 0x7f924406b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9243fff700 1 -- 172.21.15.117:0/3338923473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441a02b0 con 0x7f92441023b0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9248a9a700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92441011b0 0x7f924406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f924929b700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9244102d50 0x7f924419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9243fff700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 0x7f924419d3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f923000b530 tx=0x7f923000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230015020 con 0x7f92441023b0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92441a05a0 con 0x7f92441023b0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230011d10 con 0x7f92441023b0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230010e50 con 0x7f92441023b0 2024-04-24T23:06:23.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.200+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441a0b10 con 0x7f92441023b0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.201+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244068f20 con 0x7f92441023b0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.202+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9230018020 con 0x7f92441023b0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.203+0000 7f9241ffb700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92340a2dc0 0x7f92340a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.203+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f92300cd8b0 con 0x7f92441023b0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.203+0000 7f9248a9a700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92340a2dc0 0x7f92340a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.203+0000 7f9248a9a700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92340a2dc0 0x7f92340a5280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92380039c0 tx=0x7f9238005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.208+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f923001f020 con 0x7f92441023b0 2024-04-24T23:06:23.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:23.409+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9244062670 con 0x7f92340a2dc0 2024-04-24T23:06:23.998 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:23 smithi117 ceph-mon[32529]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:23.998 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:23.998 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:23.998 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:24.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:23 smithi038 ceph-mon[32450]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:24.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:24.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:24.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:24.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:23 smithi033 ceph-mon[29201]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:24.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:24.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:24.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:25.001 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:24 smithi117 ceph-mon[32529]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:25.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:24 smithi038 ceph-mon[32450]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:25.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:24 smithi033 ceph-mon[29201]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:27.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:27.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/2823431441' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]: dispatch 2024-04-24T23:06:27.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]: dispatch 2024-04-24T23:06:27.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]': finished 2024-04-24T23:06:27.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:06:27.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:27.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/1534824067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:27.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/2823431441' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]: dispatch 2024-04-24T23:06:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]: dispatch 2024-04-24T23:06:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]': finished 2024-04-24T23:06:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:06:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/1534824067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/2823431441' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]: dispatch 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]: dispatch 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4fef747-c797-4d33-b6bd-c1f7d55e052b"}]': finished 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/1534824067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:29.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:28 smithi038 ceph-mon[32450]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:28 smithi117 ceph-mon[32529]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:29.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:28 smithi033 ceph-mon[29201]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:30.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:30 smithi038 ceph-mon[32450]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:31.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:30 smithi117 ceph-mon[32529]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:31.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:30 smithi033 ceph-mon[29201]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:32 smithi117 ceph-mon[32529]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:33.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:32 smithi038 ceph-mon[32450]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:33.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:32 smithi033 ceph-mon[29201]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:34.029 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:06:34.030 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:33 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:34.030 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:33 smithi117 ceph-mon[32529]: Deploying daemon osd.5 on smithi117 2024-04-24T23:06:34.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:06:34.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:33 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:34.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:33 smithi038 ceph-mon[32450]: Deploying daemon osd.5 on smithi117 2024-04-24T23:06:34.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:06:34.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:33 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:34.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:33 smithi033 ceph-mon[29201]: Deploying daemon osd.5 on smithi117 2024-04-24T23:06:34.833 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:34 smithi117 ceph-mon[32529]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:35.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:34 smithi038 ceph-mon[32450]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:35.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:34 smithi033 ceph-mon[29201]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:37.011 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:36 smithi117 ceph-mon[32529]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:37.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:36 smithi038 ceph-mon[32450]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:36 smithi033 ceph-mon[29201]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:39.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:38 smithi117 ceph-mon[32529]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:39.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:39.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:39.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:39.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:38 smithi038 ceph-mon[32450]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:39.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:39.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:39.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:38 smithi033 ceph-mon[29201]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:39.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:39.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:39.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:41.130 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:40 smithi117 ceph-mon[32529]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:40 smithi038 ceph-mon[32450]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:40 smithi033 ceph-mon[29201]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:41.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.978+0000 7f9241ffb700 1 -- 172.21.15.117:0/3338923473 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9244062670 con 0x7f92340a2dc0 2024-04-24T23:06:41.980 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 5 on host 'smithi117' 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92340a2dc0 msgr2=0x7f92340a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92340a2dc0 0x7f92340a5280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92380039c0 tx=0x7f9238005fb0 comp rx=0 tx=0).stop 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 msgr2=0x7f924419d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 0x7f924419d3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f923000b530 tx=0x7f923000b8f0 comp rx=0 tx=0).stop 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 shutdown_connections 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92340a2dc0 0x7f92340a5280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:41.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9244102d50 0x7f924419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:41.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92441011b0 0x7f924406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:41.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.979+0000 7f924acfe700 1 --2- 172.21.15.117:0/3338923473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92441023b0 0x7f924419d3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:41.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.980+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 >> 172.21.15.117:0/3338923473 conn(0x7f92440fc790 msgr2=0x7f9244105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:41.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.980+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 shutdown_connections 2024-04-24T23:06:41.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:06:41.980+0000 7f924acfe700 1 -- 172.21.15.117:0/3338923473 wait complete. 2024-04-24T23:06:42.698 DEBUG:teuthology.orchestra.run.smithi117:osd.5> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T23:06:42.701 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-04-24T23:06:42.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:06:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:43.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:43.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: from='osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:06:43.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:06:43.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:42 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.406 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:43.407 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.407 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.407 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: from='osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:06:43.407 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:06:43.407 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.407 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:42 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: from='osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:06:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:06:43.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:42 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:43.986 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:06:43.986 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-mon[32529]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:06:43.986 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:43.986 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-mon[32529]: from='osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:06:43.986 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:06:43.987 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:06:43.978+0000 7fe26facd700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:06:43.987 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:06:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:06:43.984+0000 7fe26b8c0700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:06:44.070 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:43 smithi033 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:06:44.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:43 smithi033 ceph-mon[29201]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:06:44.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:44.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:43 smithi033 ceph-mon[29201]: from='osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:06:44.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:43 smithi033 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:06:44.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:43 smithi038 ceph-mon[32450]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:06:44.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:43 smithi038 ceph-mon[32450]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:06:44.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:44.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:43 smithi038 ceph-mon[32450]: from='osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:06:44.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:43 smithi038 ceph-mon[32450]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:06:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:06:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:06:44.920 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-24T23:06:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-24T23:06:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112111 s, 935 MB/s 2024-04-24T23:06:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-24T23:06:45.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] boot 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:06:45.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:44 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:06:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:06:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] boot 2024-04-24T23:06:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:06:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:44 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: osd.5 [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] boot 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:06:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:44 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:06:45.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-04-24T23:06:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:46.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:06:46.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:06:46.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:06:46.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:46.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:46.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:46.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:46.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:46.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:46.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:06:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 -- 172.21.15.33:0/3082652886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 msgr2=0x7f863c1026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 --2- 172.21.15.33:0/3082652886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c1026d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f862c0099c0 tx=0x7f862c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:06:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 -- 172.21.15.33:0/3082652886 shutdown_connections 2024-04-24T23:06:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 --2- 172.21.15.33:0/3082652886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f863c103e50 0x7f863c10a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 --2- 172.21.15.33:0/3082652886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c1026d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 --2- 172.21.15.33:0/3082652886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c1034b0 0x7f863c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:47.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 -- 172.21.15.33:0/3082652886 >> 172.21.15.33:0/3082652886 conn(0x7f863c0fd850 msgr2=0x7f863c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:47.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 -- 172.21.15.33:0/3082652886 shutdown_connections 2024-04-24T23:06:47.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.492+0000 7f8640c69700 1 -- 172.21.15.33:0/3082652886 wait complete. 2024-04-24T23:06:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 Processor -- start 2024-04-24T23:06:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 -- start start 2024-04-24T23:06:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f863c1034b0 0x7f863c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c103e50 0x7f863c19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f863c10def0 con 0x7f863c1022b0 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f863c10dd70 con 0x7f863c103e50 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8640c69700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f863c10e070 con 0x7f863c1034b0 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50974/0 (socket says 172.21.15.33:50974) 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 -- 172.21.15.33:0/3881360241 learned_addr learned my addr 172.21.15.33:0/3881360241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f8639d9b700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f863c1034b0 0x7f863c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863ad9d700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c103e50 0x7f863c19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:47.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 -- 172.21.15.33:0/3881360241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f863c1034b0 msgr2=0x7f863c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f863c1034b0 0x7f863c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 -- 172.21.15.33:0/3881360241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c103e50 msgr2=0x7f863c19fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c103e50 0x7f863c19fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863a59c700 1 -- 172.21.15.33:0/3881360241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c1a00e0 con 0x7f863c1022b0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.493+0000 7f863ad9d700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c103e50 0x7f863c19fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.494+0000 7f863a59c700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c198480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f862c005e80 tx=0x7f862c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.494+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c027070 con 0x7f863c1022b0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.494+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c022470 con 0x7f863c1022b0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.494+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c1a0370 con 0x7f863c1022b0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.494+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c00f6a0 con 0x7f863c1022b0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.494+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1a08d0 con 0x7f863c1022b0 2024-04-24T23:06:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.495+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c068f20 con 0x7f863c1022b0 2024-04-24T23:06:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.496+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f862c02a070 con 0x7f863c1022b0 2024-04-24T23:06:47.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.497+0000 7f86337fe700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86280a2c20 0x7f86280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:47.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.497+0000 7f8639d9b700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86280a2c20 0x7f86280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:47.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.497+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f862c0cefe0 con 0x7f863c1022b0 2024-04-24T23:06:47.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.497+0000 7f8639d9b700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86280a2c20 0x7f86280a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f862400ac50 tx=0x7f862400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:47.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.501+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f862c013070 con 0x7f863c1022b0 2024-04-24T23:06:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:06:47.719+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f863c062670 con 0x7f86280a2c20 2024-04-24T23:06:47.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:47 smithi033 ceph-mon[29201]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:47.721 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:47 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi117 2024-04-24T23:06:47.721 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:47 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi117 to 8544M 2024-04-24T23:06:47.721 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:47 smithi033 ceph-mon[29201]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:06:47.739 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:47 smithi117 ceph-mon[32529]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:47.739 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:47 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi117 2024-04-24T23:06:47.739 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:47 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi117 to 8544M 2024-04-24T23:06:47.739 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:47 smithi117 ceph-mon[32529]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:06:47.823 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:47 smithi038 ceph-mon[32450]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:47.823 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:47 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi117 2024-04-24T23:06:47.823 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:47 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi117 to 8544M 2024-04-24T23:06:47.823 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:47 smithi038 ceph-mon[32450]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:06:48.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:48 smithi033 ceph-mon[29201]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:06:48.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:48.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:48.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:48.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:48 smithi038 ceph-mon[32450]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:06:48.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:48.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:48.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:48.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:48 smithi117 ceph-mon[32529]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:06:48.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:48.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:48.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:49.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:49 smithi033 ceph-mon[29201]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:49.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:49 smithi033 ceph-mon[29201]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-04-24T23:06:49.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:49 smithi033 ceph-mon[29201]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:06:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:49 smithi038 ceph-mon[32450]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:49 smithi038 ceph-mon[32450]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-04-24T23:06:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:49 smithi038 ceph-mon[32450]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:06:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:49 smithi117 ceph-mon[32529]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:49 smithi117 ceph-mon[32529]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-04-24T23:06:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:49 smithi117 ceph-mon[32529]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:06:51.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:51 smithi038 ceph-mon[32450]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:06:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:51 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1933230611' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeb1c83d-c2cd-4d7b-95dc-dff2b00400db"}]: dispatch 2024-04-24T23:06:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:51 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1933230611' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeb1c83d-c2cd-4d7b-95dc-dff2b00400db"}]': finished 2024-04-24T23:06:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:51 smithi038 ceph-mon[32450]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:06:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:51 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:06:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:51 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2596838391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:51 smithi117 ceph-mon[32529]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:06:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1933230611' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeb1c83d-c2cd-4d7b-95dc-dff2b00400db"}]: dispatch 2024-04-24T23:06:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1933230611' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeb1c83d-c2cd-4d7b-95dc-dff2b00400db"}]': finished 2024-04-24T23:06:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:51 smithi117 ceph-mon[32529]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:06:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:06:51.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2596838391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:51 smithi033 ceph-mon[29201]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:06:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:51 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1933230611' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeb1c83d-c2cd-4d7b-95dc-dff2b00400db"}]: dispatch 2024-04-24T23:06:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:51 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1933230611' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeb1c83d-c2cd-4d7b-95dc-dff2b00400db"}]': finished 2024-04-24T23:06:51.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:51 smithi033 ceph-mon[29201]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:06:51.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:51 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:06:51.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:51 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2596838391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:53.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:53 smithi038 ceph-mon[32450]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:06:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:53 smithi117 ceph-mon[32529]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:06:53.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:53 smithi033 ceph-mon[29201]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:06:55.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:55 smithi033 ceph-mon[29201]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:55.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:55 smithi038 ceph-mon[32450]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:55 smithi117 ceph-mon[32529]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:57.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:57 smithi038 ceph-mon[32450]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:57 smithi117 ceph-mon[32529]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:57 smithi033 ceph-mon[29201]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:58.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:06:58.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:58.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:06:58.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:06:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:59.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:59 smithi038 ceph-mon[32450]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:59.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:06:59 smithi038 ceph-mon[32450]: Deploying daemon osd.6 on smithi033 2024-04-24T23:06:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:59 smithi117 ceph-mon[32529]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:06:59 smithi117 ceph-mon[32529]: Deploying daemon osd.6 on smithi033 2024-04-24T23:06:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:59 smithi033 ceph-mon[29201]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:06:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:06:59 smithi033 ceph-mon[29201]: Deploying daemon osd.6 on smithi033 2024-04-24T23:07:01.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:01 smithi033 ceph-mon[29201]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:01.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:01 smithi038 ceph-mon[32450]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:01 smithi117 ceph-mon[32529]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:03.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:03 smithi033 ceph-mon[29201]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:03 smithi038 ceph-mon[32450]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:03 smithi117 ceph-mon[32529]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:05 smithi038 ceph-mon[32450]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:05 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:05.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:05 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:05.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:05 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:05 smithi117 ceph-mon[32529]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:05 smithi033 ceph-mon[29201]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:07 smithi038 ceph-mon[32450]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:07 smithi117 ceph-mon[32529]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:07 smithi033 ceph-mon[29201]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.050 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-04-24T23:07:08.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.048+0000 7f86337fe700 1 -- 172.21.15.33:0/3881360241 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f863c062670 con 0x7f86280a2c20 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86280a2c20 msgr2=0x7f86280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86280a2c20 0x7f86280a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f862400ac50 tx=0x7f862400a380 comp rx=0 tx=0).stop 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 msgr2=0x7f863c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c198480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f862c005e80 tx=0x7f862c005eb0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 shutdown_connections 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86280a2c20 0x7f86280a50e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f863c103e50 0x7f863c19fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1022b0 0x7f863c198480 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 --2- 172.21.15.33:0/3881360241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f863c1034b0 0x7f863c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 >> 172.21.15.33:0/3881360241 conn(0x7f863c0fd850 msgr2=0x7f863c106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 shutdown_connections 2024-04-24T23:07:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:07:08.051+0000 7f8640c69700 1 -- 172.21.15.33:0/3881360241 wait complete. 2024-04-24T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:08 smithi033 ceph-mon[29201]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:08.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:08 smithi038 ceph-mon[32450]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:07:08.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:08 smithi117 ceph-mon[32529]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:08.677 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T23:07:08.682 INFO:tasks.cephadm:Deploying osd.7 on smithi038 with /dev/vg_nvme/lv_2... 2024-04-24T23:07:08.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:07:08.938 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:07:09.381 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:09 smithi038 ceph-mon[32450]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:09.381 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:09 smithi038 ceph-mon[32450]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:07:09.381 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:09 smithi038 ceph-mon[32450]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:07:09.381 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:09 smithi038 ceph-mon[32450]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:07:09.381 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:09 smithi117 ceph-mon[32529]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:09 smithi117 ceph-mon[32529]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:07:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:09 smithi117 ceph-mon[32529]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:07:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:09 smithi117 ceph-mon[32529]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:07:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:09 smithi033 ceph-mon[29201]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:09 smithi033 ceph-mon[29201]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:07:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:09 smithi033 ceph-mon[29201]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:07:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:09 smithi033 ceph-mon[29201]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:07:09.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:10.170 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:07:10 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:07:10.003+0000 7f62c8ae8700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:07:10.170 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:07:10 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:07:10.012+0000 7f62c48db700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:07:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:07:11.069 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-24T23:07:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-24T23:07:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0148986 s, 704 MB/s 2024-04-24T23:07:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-24T23:07:11.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:07:11.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:07:11.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:11.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:07:11.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:07:11.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:11.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:11.382 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:07:11.382 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:07:11.382 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:11.382 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:07:11.382 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:07:11.382 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:11.383 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: from='osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:11.800 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-04-24T23:07:12.017 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] boot 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:12.018 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.055 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:07:12.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] boot 2024-04-24T23:07:12.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:07:12.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:12.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:12.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:12.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: osd.6 [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] boot 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:12.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:12.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:13.242 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:13 smithi038 ceph-mon[32450]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:13.243 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:13 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi033 2024-04-24T23:07:13.243 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:13 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-24T23:07:13.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:13 smithi117 ceph-mon[32529]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:13.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:13 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi033 2024-04-24T23:07:13.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:13 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-24T23:07:13.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:13 smithi033 ceph-mon[29201]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:13.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:13 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi033 2024-04-24T23:07:13.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:13 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 -- 172.21.15.38:0/666612454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581034b0 msgr2=0x7fb758103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 --2- 172.21.15.38:0/666612454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581034b0 0x7fb758103910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb7400099c0 tx=0x7fb740009cd0 comp rx=0 tx=0).stop 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 -- 172.21.15.38:0/666612454 shutdown_connections 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 --2- 172.21.15.38:0/666612454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb758103e50 0x7fb75810a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 --2- 172.21.15.38:0/666612454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7581022b0 0x7fb7581026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 --2- 172.21.15.38:0/666612454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581034b0 0x7fb758103910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 -- 172.21.15.38:0/666612454 >> 172.21.15.38:0/666612454 conn(0x7fb7580fd850 msgr2=0x7fb7580ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 -- 172.21.15.38:0/666612454 shutdown_connections 2024-04-24T23:07:13.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.775+0000 7fb75db67700 1 -- 172.21.15.38:0/666612454 wait complete. 2024-04-24T23:07:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 Processor -- start 2024-04-24T23:07:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 -- start start 2024-04-24T23:07:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 0x7fb758198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7581034b0 0x7fb758198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb758103e50 0x7fb75819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:13.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810de80 con 0x7fb758103e50 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810dd00 con 0x7fb7581022b0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb75db67700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810e000 con 0x7fb7581034b0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb7577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 0x7fb758198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb7577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 0x7fb758198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56124/0 (socket says 172.21.15.38:56124) 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb7577fe700 1 -- 172.21.15.38:0/1181674007 learned_addr learned my addr 172.21.15.38:0/1181674007 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.776+0000 7fb756ffd700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7581034b0 0x7fb758198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb757fff700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb758103e50 0x7fb75819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb7577fe700 1 -- 172.21.15.38:0/1181674007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7581034b0 msgr2=0x7fb758198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb7577fe700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7581034b0 0x7fb758198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb7577fe700 1 -- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb758103e50 msgr2=0x7fb75819f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb7577fe700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb758103e50 0x7fb75819f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb7577fe700 1 -- 172.21.15.38:0/1181674007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7581a0060 con 0x7fb7581022b0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb7577fe700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 0x7fb758198400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb748007e00 tx=0x7fb74800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:13.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748016020 con 0x7fb7581022b0 2024-04-24T23:07:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7581a0350 con 0x7fb7581022b0 2024-04-24T23:07:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74800ad40 con 0x7fb7581022b0 2024-04-24T23:07:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748011650 con 0x7fb7581022b0 2024-04-24T23:07:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.777+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a08e0 con 0x7fb7581022b0 2024-04-24T23:07:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.778+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758068f20 con 0x7fb7581022b0 2024-04-24T23:07:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.779+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb748019020 con 0x7fb7581022b0 2024-04-24T23:07:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.780+0000 7fb754ff9700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7440a2c20 0x7fb7440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.780+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fb7480cdfe0 con 0x7fb7581022b0 2024-04-24T23:07:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.780+0000 7fb756ffd700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7440a2c20 0x7fb7440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.781+0000 7fb756ffd700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7440a2c20 0x7fb7440a50e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb740000c00 tx=0x7fb740005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:13.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.784+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb748020030 con 0x7fb7581022b0 2024-04-24T23:07:14.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:13.998+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb758062670 con 0x7fb7440a2c20 2024-04-24T23:07:14.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:14 smithi038 ceph-mon[32450]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:07:14.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:14.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:14.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:14.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:14 smithi117 ceph-mon[32529]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:07:14.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:14.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:14.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:14.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:14 smithi033 ceph-mon[29201]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:07:14.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:14.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:14.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:15.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:15 smithi117 ceph-mon[32529]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:15.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:15 smithi117 ceph-mon[32529]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:15.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:15 smithi038 ceph-mon[32450]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:15.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:15 smithi038 ceph-mon[32450]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:15.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:15 smithi033 ceph-mon[29201]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:15.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:15 smithi033 ceph-mon[29201]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:17.322 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:17 smithi038 ceph-mon[32450]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:17.322 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:17 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/1560743192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]: dispatch 2024-04-24T23:07:17.322 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:17 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]: dispatch 2024-04-24T23:07:17.323 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:17 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]': finished 2024-04-24T23:07:17.323 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:17 smithi038 ceph-mon[32450]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:07:17.323 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:17 smithi117 ceph-mon[32529]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:17 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/1560743192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]: dispatch 2024-04-24T23:07:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:17 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]: dispatch 2024-04-24T23:07:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:17 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]': finished 2024-04-24T23:07:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:17 smithi117 ceph-mon[32529]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:07:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:17 smithi033 ceph-mon[29201]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:17 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/1560743192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]: dispatch 2024-04-24T23:07:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:17 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]: dispatch 2024-04-24T23:07:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:17 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e464d3b-4514-43db-9449-25f51b362bab"}]': finished 2024-04-24T23:07:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:17 smithi033 ceph-mon[29201]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:07:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:18.333 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:18 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/4081595016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:18.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:18 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/4081595016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:18.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/4081595016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:19.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:19 smithi038 ceph-mon[32450]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:19.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:19 smithi117 ceph-mon[32529]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:19.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:19 smithi033 ceph-mon[29201]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:21.254 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:21 smithi038 ceph-mon[32450]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:21.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:21 smithi117 ceph-mon[32529]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:21.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:21 smithi033 ceph-mon[29201]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:23.282 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:23 smithi038 ceph-mon[32450]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:23.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:23 smithi117 ceph-mon[32529]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:23.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:23 smithi033 ceph-mon[29201]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:25 smithi038 ceph-mon[32450]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:25 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:25 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:25 smithi038 ceph-mon[32450]: Deploying daemon osd.7 on smithi038 2024-04-24T23:07:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:25 smithi117 ceph-mon[32529]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:25 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:07:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:25 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:25 smithi117 ceph-mon[32529]: Deploying daemon osd.7 on smithi038 2024-04-24T23:07:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:25 smithi033 ceph-mon[29201]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:25 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:07:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:25 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:25 smithi033 ceph-mon[29201]: Deploying daemon osd.7 on smithi038 2024-04-24T23:07:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:27 smithi038 ceph-mon[32450]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:27 smithi117 ceph-mon[32529]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:27 smithi033 ceph-mon[29201]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:29.377 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:29 smithi038 ceph-mon[32450]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:29 smithi117 ceph-mon[32529]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:29 smithi033 ceph-mon[29201]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:31 smithi038 ceph-mon[32450]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:31.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:31.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:31 smithi117 ceph-mon[32529]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:31 smithi033 ceph-mon[29201]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:33.563 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:33 smithi038 ceph-mon[32450]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:33 smithi117 ceph-mon[32529]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:33 smithi033 ceph-mon[29201]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:34.272 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 7 on host 'smithi038' 2024-04-24T23:07:34.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.270+0000 7fb754ff9700 1 -- 172.21.15.38:0/1181674007 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb758062670 con 0x7fb7440a2c20 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7440a2c20 msgr2=0x7fb7440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7440a2c20 0x7fb7440a50e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb740000c00 tx=0x7fb740005fb0 comp rx=0 tx=0).stop 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 msgr2=0x7fb758198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 0x7fb758198400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb748007e00 tx=0x7fb74800a340 comp rx=0 tx=0).stop 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 shutdown_connections 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7440a2c20 0x7fb7440a50e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb758103e50 0x7fb75819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7581022b0 0x7fb758198400 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 --2- 172.21.15.38:0/1181674007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7581034b0 0x7fb758198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:34.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 >> 172.21.15.38:0/1181674007 conn(0x7fb7580fd850 msgr2=0x7fb7581067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:34.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 shutdown_connections 2024-04-24T23:07:34.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:07:34.273+0000 7fb75db67700 1 -- 172.21.15.38:0/1181674007 wait complete. 2024-04-24T23:07:34.605 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.605 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.605 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:34.605 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:34.605 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.605 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:07:34.606 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:07:34.606 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.606 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.606 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:34 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:34.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:07:34.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:07:34.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:34 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:34.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:34.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:34 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:34.945 DEBUG:teuthology.orchestra.run.smithi038:osd.7> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T23:07:34.947 INFO:tasks.cephadm:Deploying osd.8 on smithi117 with /dev/vg_nvme/lv_2... 2024-04-24T23:07:34.948 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:07:35.193 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:07:35.526 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-mon[32450]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:35.526 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-mon[32450]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:07:35.527 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-mon[32450]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:07:35.527 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:35.527 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-mon[32450]: from='osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:07:35.527 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-mon[32450]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:07:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:35 smithi117 ceph-mon[32529]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:35 smithi117 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:07:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:35 smithi117 ceph-mon[32529]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:07:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:35 smithi117 ceph-mon[32529]: from='osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:07:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:35 smithi117 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:07:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:35 smithi033 ceph-mon[29201]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:35 smithi033 ceph-mon[29201]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:07:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:35 smithi033 ceph-mon[29201]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:07:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:35.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:35 smithi033 ceph-mon[29201]: from='osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:07:35.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:35 smithi033 ceph-mon[29201]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:07:36.163 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:07:35.816+0000 7fb1fe8b2700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:07:36.163 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:07:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:07:35.826+0000 7fb1f5e9c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:36.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:07:36.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:07:36.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: from='osd.7 ' entity='osd.7' 2024-04-24T23:07:36.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] boot 2024-04-24T23:07:36.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:07:36.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:36 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:07:36.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:07:36.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: from='osd.7 ' entity='osd.7' 2024-04-24T23:07:36.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] boot 2024-04-24T23:07:36.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:07:36.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:36 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: from='osd.7 ' entity='osd.7' 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: osd.7 [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] boot 2024-04-24T23:07:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:07:36.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:36 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:07:37.310 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:07:37.310 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:07:37.310 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-24T23:07:37.310 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-24T23:07:37.310 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136182 s, 770 MB/s 2024-04-24T23:07:37.310 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-24T23:07:37.601 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:37 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:07:37.602 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:37 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:07:37.602 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:37 smithi117 ceph-mon[32529]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:37.602 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:37 smithi117 ceph-mon[32529]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:07:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:37 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:07:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:37 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:07:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:37 smithi038 ceph-mon[32450]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:37 smithi038 ceph-mon[32450]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:07:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:37 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:07:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:37 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:07:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:37 smithi033 ceph-mon[29201]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:37 smithi033 ceph-mon[29201]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:07:38.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-04-24T23:07:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi038 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.600 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi038 to 5697M 2024-04-24T23:07:39.601 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.601 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:39.601 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:39.601 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.601 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:39 smithi117 ceph-mon[32529]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:07:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi038 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi038 to 5697M 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:39 smithi038 ceph-mon[32450]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi038 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi038 to 5697M 2024-04-24T23:07:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:39 smithi033 ceph-mon[29201]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:07:39.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 -- 172.21.15.117:0/1912218641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 msgr2=0x7f6b84103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:39.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1912218641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84103920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009990 tx=0x7f6b74009ca0 comp rx=0 tx=0).stop 2024-04-24T23:07:39.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 -- 172.21.15.117:0/1912218641 shutdown_connections 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1912218641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b84103e60 0x7f6b8410a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1912218641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 0x7f6b841026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1912218641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84103920 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 -- 172.21.15.117:0/1912218641 >> 172.21.15.117:0/1912218641 conn(0x7f6b840fd880 msgr2=0x7f6b840ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 -- 172.21.15.117:0/1912218641 shutdown_connections 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.831+0000 7f6b8b890700 1 -- 172.21.15.117:0/1912218641 wait complete. 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 Processor -- start 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 -- start start 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 0x7f6b84198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b84103e60 0x7f6b8419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8410dfa0 con 0x7f6b84103e60 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8410de20 con 0x7f6b841022c0 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8b890700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8410e120 con 0x7f6b841034c0 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42496/0 (socket says 172.21.15.117:42496) 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 -- 172.21.15.117:0/1067340531 learned_addr learned my addr 172.21.15.117:0/1067340531 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8962c700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 0x7f6b84198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b89e2d700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b84103e60 0x7f6b8419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 -- 172.21.15.117:0/1067340531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 msgr2=0x7f6b84198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 0x7f6b84198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 -- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b84103e60 msgr2=0x7f6b8419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b84103e60 0x7f6b8419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:39.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b88e2b700 1 -- 172.21.15.117:0/1067340531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b841a0270 con 0x7f6b841034c0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.832+0000 7f6b8962c700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 0x7f6b84198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.833+0000 7f6b88e2b700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84198b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6b740047b0 tx=0x7f6b74003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.833+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74027070 con 0x7f6b841034c0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.833+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b74022470 con 0x7f6b841034c0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.833+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b841a0500 con 0x7f6b841034c0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.833+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400f6c0 con 0x7f6b841034c0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.833+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b841a09d0 con 0x7f6b841034c0 2024-04-24T23:07:39.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.834+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b84068f20 con 0x7f6b841034c0 2024-04-24T23:07:39.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.835+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6b7402a070 con 0x7f6b841034c0 2024-04-24T23:07:39.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.835+0000 7f6b7a7fc700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b700a2c20 0x7f6b700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:39.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.836+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f6b740cfb00 con 0x7f6b841034c0 2024-04-24T23:07:39.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.836+0000 7f6b8962c700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b700a2c20 0x7f6b700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:39.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.836+0000 7f6b8962c700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b700a2c20 0x7f6b700a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b84103320 tx=0x7f6b80008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:39.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:39.839+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6b74031070 con 0x7f6b841034c0 2024-04-24T23:07:40.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:40.044+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6b84062670 con 0x7f6b700a2c20 2024-04-24T23:07:40.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:40 smithi033 ceph-mon[29201]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:07:40.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:40.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:40.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:40.788 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:40 smithi117 ceph-mon[32529]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:07:40.788 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:40.789 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:40.789 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:40.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:40 smithi038 ceph-mon[32450]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:07:40.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:40.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:40.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:41.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:41 smithi033 ceph-mon[29201]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:41.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:41 smithi033 ceph-mon[29201]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:41.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:41 smithi033 ceph-mon[29201]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:07:41.801 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:41 smithi117 ceph-mon[32529]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:41.801 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:41 smithi117 ceph-mon[32529]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:41.801 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:41 smithi117 ceph-mon[32529]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:07:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:41 smithi038 ceph-mon[32450]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:41 smithi038 ceph-mon[32450]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:41 smithi038 ceph-mon[32450]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:07:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/3150030432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]: dispatch 2024-04-24T23:07:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]: dispatch 2024-04-24T23:07:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]': finished 2024-04-24T23:07:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:07:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:07:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:43 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/1156541121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/3150030432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]: dispatch 2024-04-24T23:07:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]: dispatch 2024-04-24T23:07:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]': finished 2024-04-24T23:07:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:07:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:07:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:43 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/1156541121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/3150030432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]: dispatch 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]: dispatch 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69890093-ead1-4825-8a46-26a2b2c1b401"}]': finished 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:43 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/1156541121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:45.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:45 smithi033 ceph-mon[29201]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:45.728 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:45 smithi117 ceph-mon[32529]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:45.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:45 smithi038 ceph-mon[32450]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:47 smithi033 ceph-mon[29201]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:47.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:47 smithi038 ceph-mon[32450]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:47.766 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:47 smithi117 ceph-mon[32529]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:49.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:49 smithi033 ceph-mon[29201]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:49.851 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:49 smithi117 ceph-mon[32529]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:49 smithi038 ceph-mon[32450]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:50.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:50 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:07:50.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:50 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:50.791 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:50 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:07:50.791 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:50 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:50.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:50 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:07:50.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:50 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:51 smithi033 ceph-mon[29201]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:51 smithi033 ceph-mon[29201]: Deploying daemon osd.8 on smithi117 2024-04-24T23:07:51.800 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:51 smithi117 ceph-mon[32529]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:51.800 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:51 smithi117 ceph-mon[32529]: Deploying daemon osd.8 on smithi117 2024-04-24T23:07:51.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:51 smithi038 ceph-mon[32450]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:51.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:51 smithi038 ceph-mon[32450]: Deploying daemon osd.8 on smithi117 2024-04-24T23:07:53.646 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:53 smithi117 ceph-mon[32529]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:53 smithi033 ceph-mon[29201]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:53.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:53 smithi038 ceph-mon[32450]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:55.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:55 smithi117 ceph-mon[32529]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:55.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:55 smithi038 ceph-mon[32450]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:55 smithi033 ceph-mon[29201]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:56 smithi117 ceph-mon[32529]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:57.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:56 smithi038 ceph-mon[32450]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:57.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:57.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:57.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:57.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:56 smithi033 ceph-mon[29201]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:57.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:57.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:07:57.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:58 smithi117 ceph-mon[32529]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:59.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:58 smithi038 ceph-mon[32450]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:59.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:58 smithi033 ceph-mon[29201]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:07:59.866 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 8 on host 'smithi117' 2024-04-24T23:07:59.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.865+0000 7f6b7a7fc700 1 -- 172.21.15.117:0/1067340531 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b84062670 con 0x7f6b700a2c20 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b700a2c20 msgr2=0x7f6b700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b700a2c20 0x7f6b700a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b84103320 tx=0x7f6b80008040 comp rx=0 tx=0).stop 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 msgr2=0x7f6b84198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84198b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6b740047b0 tx=0x7f6b74003680 comp rx=0 tx=0).stop 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 shutdown_connections 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b700a2c20 0x7f6b700a50e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b84103e60 0x7f6b8419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b841022c0 0x7f6b84198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 --2- 172.21.15.117:0/1067340531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841034c0 0x7f6b84198b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 >> 172.21.15.117:0/1067340531 conn(0x7f6b840fd880 msgr2=0x7f6b84106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:59.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 shutdown_connections 2024-04-24T23:07:59.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:07:59.867+0000 7f6b8b890700 1 -- 172.21.15.117:0/1067340531 wait complete. 2024-04-24T23:08:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:00.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:00.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:07:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:07:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:00.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:07:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:00.526 DEBUG:teuthology.orchestra.run.smithi117:osd.8> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T23:08:00.529 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-04-24T23:08:00.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:08:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:08:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-mon[32529]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:08:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-mon[32529]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:08:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-mon[32529]: from='osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:08:01.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:01.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:08:01.177 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:08:00.971+0000 7fe7fb880700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T23:08:01.177 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:08:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:08:00.979+0000 7fe7f7673700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:00 smithi033 ceph-mon[29201]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:00 smithi033 ceph-mon[29201]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:08:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:00 smithi033 ceph-mon[29201]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:08:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:00 smithi033 ceph-mon[29201]: from='osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:08:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:00 smithi033 ceph-mon[29201]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:08:01.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:00 smithi038 ceph-mon[32450]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:01.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:00 smithi038 ceph-mon[32450]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:08:01.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:00 smithi038 ceph-mon[32450]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:08:01.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:00 smithi038 ceph-mon[32450]: from='osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:08:01.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:01.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:00 smithi038 ceph-mon[32450]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:08:02.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:01 smithi033 ceph-mon[29201]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:08:02.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:01 smithi033 ceph-mon[29201]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:08:02.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:02.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:02.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:01 smithi038 ceph-mon[32450]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:08:02.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:01 smithi038 ceph-mon[32450]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:08:02.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:02.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:02.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:01 smithi117 ceph-mon[32529]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:08:02.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:01 smithi117 ceph-mon[32529]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:08:02.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:02.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:08:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:08:02.958 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-24T23:08:02.962 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-24T23:08:02.962 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112578 s, 931 MB/s 2024-04-24T23:08:02.962 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] boot 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:08:03.165 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] boot 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:08:03.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:08:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:08:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: osd.8 [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] boot 2024-04-24T23:08:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:08:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:03.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-04-24T23:08:03.863 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:08:04.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:08:04.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi117 2024-04-24T23:08:04.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi117 to 5696M 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:04.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi117 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:03 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi117 to 5696M 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:04.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:08:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi117 2024-04-24T23:08:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi117 to 5696M 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:04.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:04 smithi033 ceph-mon[29201]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:04 smithi033 ceph-mon[29201]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:08:05.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:04 smithi038 ceph-mon[32450]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:05.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:04 smithi038 ceph-mon[32450]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:08:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:04 smithi117 ceph-mon[32529]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:04 smithi117 ceph-mon[32529]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:08:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.583+0000 7fa8a41fa700 1 -- 172.21.15.33:0/4071262605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c103490 msgr2=0x7fa89c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.583+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/4071262605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c103490 0x7fa89c1038f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0099c0 tx=0x7fa88c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:08:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- 172.21.15.33:0/4071262605 shutdown_connections 2024-04-24T23:08:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/4071262605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa89c103e30 0x7fa89c10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/4071262605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa89c102290 0x7fa89c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/4071262605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c103490 0x7fa89c1038f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- 172.21.15.33:0/4071262605 >> 172.21.15.33:0/4071262605 conn(0x7fa89c0fd830 msgr2=0x7fa89c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- 172.21.15.33:0/4071262605 shutdown_connections 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- 172.21.15.33:0/4071262605 wait complete. 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 Processor -- start 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- start start 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 0x7fa89c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa89c103490 0x7fa89c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa89c103e30 0x7fa89c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c10def0 con 0x7fa89c102290 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c10dd70 con 0x7fa89c103e30 2024-04-24T23:08:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.584+0000 7fa8a41fa700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c10e070 con 0x7fa89c103490 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 0x7fa89c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1795700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa89c103490 0x7fa89c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 0x7fa89c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47730/0 (socket says 172.21.15.33:47730) 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 -- 172.21.15.33:0/219855179 learned_addr learned my addr 172.21.15.33:0/219855179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a2797700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa89c103e30 0x7fa89c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 -- 172.21.15.33:0/219855179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa89c103490 msgr2=0x7fa89c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa89c103490 0x7fa89c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 -- 172.21.15.33:0/219855179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa89c103e30 msgr2=0x7fa89c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa89c103e30 0x7fa89c19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 -- 172.21.15.33:0/219855179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c1a0190 con 0x7fa89c102290 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a1f96700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 0x7fa89c198530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b530 tx=0x7fa89800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898015020 con 0x7fa89c102290 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c1a0480 con 0x7fa89c102290 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa898011d10 con 0x7fa89c102290 2024-04-24T23:08:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898010d90 con 0x7fa89c102290 2024-04-24T23:08:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.585+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1a0a40 con 0x7fa89c102290 2024-04-24T23:08:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.586+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c068f20 con 0x7fa89c102290 2024-04-24T23:08:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.588+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa898018020 con 0x7fa89c102290 2024-04-24T23:08:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.589+0000 7fa892ffd700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8880a2c20 0x7fa8880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.589+0000 7fa8a1795700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8880a2c20 0x7fa8880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.589+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fa8980ce310 con 0x7fa89c102290 2024-04-24T23:08:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.589+0000 7fa8a1795700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8880a2c20 0x7fa8880a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa88c000c00 tx=0x7fa88c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:05.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.593+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa89801f020 con 0x7fa89c102290 2024-04-24T23:08:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:05.811+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa89c062670 con 0x7fa8880a2c20 2024-04-24T23:08:06.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:06 smithi033 ceph-mon[29201]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:08:06.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:06.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:06.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:06 smithi038 ceph-mon[32450]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:08:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:06 smithi117 ceph-mon[32529]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:08:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:07.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:07 smithi033 ceph-mon[29201]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:07.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:07 smithi033 ceph-mon[29201]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:07.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:07 smithi033 ceph-mon[29201]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:08:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:07 smithi038 ceph-mon[32450]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:07 smithi038 ceph-mon[32450]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:07 smithi038 ceph-mon[32450]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:08:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:07 smithi117 ceph-mon[32529]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:07 smithi117 ceph-mon[32529]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:07 smithi117 ceph-mon[32529]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:08:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:09 smithi038 ceph-mon[32450]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T23:08:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:09 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2386215287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8293c811-4410-4f09-9ea7-4de698fc76c0"}]: dispatch 2024-04-24T23:08:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:09 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2386215287' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8293c811-4410-4f09-9ea7-4de698fc76c0"}]': finished 2024-04-24T23:08:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:09 smithi038 ceph-mon[32450]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:08:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:09 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/348880985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:09 smithi117 ceph-mon[32529]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T23:08:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:09 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2386215287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8293c811-4410-4f09-9ea7-4de698fc76c0"}]: dispatch 2024-04-24T23:08:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:09 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2386215287' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8293c811-4410-4f09-9ea7-4de698fc76c0"}]': finished 2024-04-24T23:08:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:09 smithi117 ceph-mon[32529]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:08:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:09 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/348880985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:09 smithi033 ceph-mon[29201]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:09 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2386215287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8293c811-4410-4f09-9ea7-4de698fc76c0"}]: dispatch 2024-04-24T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:09 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2386215287' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8293c811-4410-4f09-9ea7-4de698fc76c0"}]': finished 2024-04-24T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:09 smithi033 ceph-mon[29201]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:09 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/348880985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:11.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:11 smithi038 ceph-mon[32450]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T23:08:11.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:11 smithi117 ceph-mon[32529]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T23:08:11.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:11 smithi033 ceph-mon[29201]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T23:08:13.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:13 smithi033 ceph-mon[29201]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-24T23:08:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:13 smithi038 ceph-mon[32450]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-24T23:08:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:13 smithi117 ceph-mon[32529]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-24T23:08:15.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:15 smithi038 ceph-mon[32450]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:08:15.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:15 smithi117 ceph-mon[32529]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:08:15.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:15 smithi033 ceph-mon[29201]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:08:16.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:08:16.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:16.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:08:16.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:16.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:08:16.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:17 smithi033 ceph-mon[29201]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:17 smithi033 ceph-mon[29201]: Deploying daemon osd.9 on smithi033 2024-04-24T23:08:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:17 smithi038 ceph-mon[32450]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:17 smithi038 ceph-mon[32450]: Deploying daemon osd.9 on smithi033 2024-04-24T23:08:17.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:17 smithi117 ceph-mon[32529]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:17.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:17 smithi117 ceph-mon[32529]: Deploying daemon osd.9 on smithi033 2024-04-24T23:08:19.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:19 smithi033 ceph-mon[29201]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:19.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:19 smithi038 ceph-mon[32450]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:19.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:19 smithi117 ceph-mon[32529]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:21 smithi038 ceph-mon[32450]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:21.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:21 smithi117 ceph-mon[32529]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:21 smithi033 ceph-mon[29201]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:23.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:23 smithi038 ceph-mon[32450]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:23.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:23.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:23.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:23 smithi117 ceph-mon[32529]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:23 smithi033 ceph-mon[29201]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:25 smithi033 ceph-mon[29201]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:25.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:25 smithi038 ceph-mon[32450]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:25 smithi117 ceph-mon[32529]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.168+0000 7fa892ffd700 1 -- 172.21.15.33:0/219855179 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa89c062670 con 0x7fa8880a2c20 2024-04-24T23:08:26.171 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-04-24T23:08:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8880a2c20 msgr2=0x7fa8880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8880a2c20 0x7fa8880a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa88c000c00 tx=0x7fa88c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 msgr2=0x7fa89c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 0x7fa89c198530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b530 tx=0x7fa89800b840 comp rx=0 tx=0).stop 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 shutdown_connections 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8880a2c20 0x7fa8880a50e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa89c103e30 0x7fa89c19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa89c102290 0x7fa89c198530 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 --2- 172.21.15.33:0/219855179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa89c103490 0x7fa89c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.170+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 >> 172.21.15.33:0/219855179 conn(0x7fa89c0fd830 msgr2=0x7fa89c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.171+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 shutdown_connections 2024-04-24T23:08:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:08:26.171+0000 7fa8a41fa700 1 -- 172.21.15.33:0/219855179 wait complete. 2024-04-24T23:08:26.805 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T23:08:26.808 INFO:tasks.cephadm:Deploying osd.10 on smithi038 with /dev/vg_nvme/lv_1... 2024-04-24T23:08:26.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:08:27.067 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:08:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:27 smithi038 ceph-mon[32450]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:27 smithi038 ceph-mon[32450]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:08:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:27 smithi117 ceph-mon[32529]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:27 smithi117 ceph-mon[32529]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:27 smithi033 ceph-mon[29201]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:27 smithi033 ceph-mon[29201]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:27.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:28.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:28 smithi038 ceph-mon[32450]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:08:28.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:28 smithi038 ceph-mon[32450]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:08:28.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:28 smithi038 ceph-mon[32450]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:08:28.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:28.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:28 smithi117 ceph-mon[32529]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:08:28.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:28 smithi117 ceph-mon[32529]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:08:28.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:28 smithi117 ceph-mon[32529]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:08:28.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:28 smithi033 ceph-mon[29201]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:08:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:28 smithi033 ceph-mon[29201]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:08:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:28 smithi033 ceph-mon[29201]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-24T23:08:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:28.888 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:08:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:08:28.584+0000 7f9e3483b700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T23:08:28.888 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:08:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:08:28.597+0000 7f9e2be25700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:29.166 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:08:29.166 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:08:29.166 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-04-24T23:08:29.166 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-04-24T23:08:29.167 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0167005 s, 628 MB/s 2024-04-24T23:08:29.167 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-04-24T23:08:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:29 smithi038 ceph-mon[32450]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:29 smithi038 ceph-mon[32450]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:08:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:29 smithi038 ceph-mon[32450]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:08:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:29 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:29 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:29 smithi038 ceph-mon[32450]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' 2024-04-24T23:08:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:29 smithi117 ceph-mon[32529]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:29 smithi117 ceph-mon[32529]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:08:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:29 smithi117 ceph-mon[32529]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:08:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:29 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:29 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:29 smithi117 ceph-mon[32529]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' 2024-04-24T23:08:29.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:29 smithi033 ceph-mon[29201]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:29.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:29 smithi033 ceph-mon[29201]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-24T23:08:29.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:29 smithi033 ceph-mon[29201]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:08:29.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:29 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:29.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:29 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:29.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:29 smithi033 ceph-mon[29201]: from='osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081]' entity='osd.9' 2024-04-24T23:08:29.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-04-24T23:08:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:08:30.359 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:30 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:08:30.359 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:30 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:08:30.359 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:30.359 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:30 smithi033 ceph-mon[29201]: osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] boot 2024-04-24T23:08:30.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:30 smithi033 ceph-mon[29201]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:08:30.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:30.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:30 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:08:30.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:30 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:08:30.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:30.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:30 smithi038 ceph-mon[32450]: osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] boot 2024-04-24T23:08:30.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:30 smithi038 ceph-mon[32450]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:08:30.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:30 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:08:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:30 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:08:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:30 smithi117 ceph-mon[32529]: osd.9 [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] boot 2024-04-24T23:08:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:30 smithi117 ceph-mon[32529]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:08:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:31.859 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:31.859 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:08:31.859 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.859 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.859 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.859 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.860 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:08:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 -- 172.21.15.38:0/3766663982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 msgr2=0x7f43f01051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 --2- 172.21.15.38:0/3766663982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f01051d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b0a0 tx=0x7f43e000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 -- 172.21.15.38:0/3766663982 shutdown_connections 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 --2- 172.21.15.38:0/3766663982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0108040 0x7f43f010a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 --2- 172.21.15.38:0/3766663982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f01051d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 --2- 172.21.15.38:0/3766663982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0105710 0x7f43f0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 -- 172.21.15.38:0/3766663982 >> 172.21.15.38:0/3766663982 conn(0x7f43f00fc750 msgr2=0x7f43f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 -- 172.21.15.38:0/3766663982 shutdown_connections 2024-04-24T23:08:32.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 -- 172.21.15.38:0/3766663982 wait complete. 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.106+0000 7f43f582c700 1 Processor -- start 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 -- start start 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0105710 0x7f43f01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0108040 0x7f43f019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010deb0 con 0x7f43f0105710 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010dd30 con 0x7f43f0102de0 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43f582c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010e030 con 0x7f43f0108040 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36124/0 (socket says 172.21.15.38:36124) 2024-04-24T23:08:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 -- 172.21.15.38:0/3343318172 learned_addr learned my addr 172.21.15.38:0/3343318172 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43ee7fc700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0105710 0x7f43f01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43ef7fe700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0108040 0x7f43f019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 -- 172.21.15.38:0/3343318172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0108040 msgr2=0x7f43f019fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0108040 0x7f43f019fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 -- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0105710 msgr2=0x7f43f01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0105710 0x7f43f01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43eeffd700 1 -- 172.21.15.38:0/3343318172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f01a00c0 con 0x7f43f0102de0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.107+0000 7f43ee7fc700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0105710 0x7f43f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.108+0000 7f43eeffd700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f0198460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43e0007fa0 tx=0x7f43e000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.108+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0021070 con 0x7f43f0102de0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.108+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f0100bd0 con 0x7f43f0102de0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.108+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e001d930 con 0x7f43f0102de0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.108+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0005710 con 0x7f43f0102de0 2024-04-24T23:08:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.108+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f0101110 con 0x7f43f0102de0 2024-04-24T23:08:32.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.109+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f0068f20 con 0x7f43f0102de0 2024-04-24T23:08:32.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.110+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f43e0024070 con 0x7f43f0102de0 2024-04-24T23:08:32.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.111+0000 7f43f482a700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43d80a2c20 0x7f43d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:32.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.111+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f43e000f020 con 0x7f43f0102de0 2024-04-24T23:08:32.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.111+0000 7f43ee7fc700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43d80a2c20 0x7f43d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:32.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.111+0000 7f43ee7fc700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43d80a2c20 0x7f43d80a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43e40099d0 tx=0x7f43e4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.115+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43e002b020 con 0x7f43f0102de0 2024-04-24T23:08:32.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:32.331+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f43f0062670 con 0x7f43d80a2c20 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi033 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:32.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi033 2024-04-24T23:08:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-24T23:08:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:32.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:32.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:32.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi033 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:33.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:33 smithi038 ceph-mon[32450]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:33.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:33 smithi038 ceph-mon[32450]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:33 smithi117 ceph-mon[32529]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:33 smithi117 ceph-mon[32529]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:33 smithi033 ceph-mon[29201]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:33 smithi033 ceph-mon[29201]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:35 smithi038 ceph-mon[32450]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/1509484379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]: dispatch 2024-04-24T23:08:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:35 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]: dispatch 2024-04-24T23:08:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:35 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]': finished 2024-04-24T23:08:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:35 smithi038 ceph-mon[32450]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:08:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:35 smithi117 ceph-mon[32529]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/1509484379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]: dispatch 2024-04-24T23:08:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:35 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]: dispatch 2024-04-24T23:08:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:35 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]': finished 2024-04-24T23:08:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:35 smithi117 ceph-mon[32529]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:08:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:35.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:35 smithi033 ceph-mon[29201]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:35.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/1509484379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]: dispatch 2024-04-24T23:08:35.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:35 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]: dispatch 2024-04-24T23:08:35.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:35 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57c8391d-387d-400c-80da-ac841c5d4608"}]': finished 2024-04-24T23:08:35.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:35 smithi033 ceph-mon[29201]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:08:35.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:36.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.38:0/3156658611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:36.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.38:0/3156658611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:36.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.38:0/3156658611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:37 smithi038 ceph-mon[32450]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:37 smithi117 ceph-mon[32529]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:37.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:37 smithi033 ceph-mon[29201]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:39 smithi038 ceph-mon[32450]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:39 smithi117 ceph-mon[32529]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:39.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:39 smithi033 ceph-mon[29201]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:41 smithi038 ceph-mon[32450]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:41.949 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:41 smithi117 ceph-mon[32529]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:41.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:41 smithi033 ceph-mon[29201]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:43 smithi038 ceph-mon[32450]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:08:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:43 smithi117 ceph-mon[32529]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:08:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:43.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:43 smithi033 ceph-mon[29201]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:43.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:08:43.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:44.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:44 smithi038 ceph-mon[32450]: Deploying daemon osd.10 on smithi038 2024-04-24T23:08:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:44 smithi117 ceph-mon[32529]: Deploying daemon osd.10 on smithi038 2024-04-24T23:08:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:44 smithi033 ceph-mon[29201]: Deploying daemon osd.10 on smithi038 2024-04-24T23:08:45.764 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:45 smithi038 ceph-mon[32450]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:45.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:45 smithi117 ceph-mon[32529]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:45 smithi033 ceph-mon[29201]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:47 smithi038 ceph-mon[32450]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:47 smithi117 ceph-mon[32529]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:47.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:47 smithi033 ceph-mon[29201]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:49 smithi038 ceph-mon[32450]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:49 smithi117 ceph-mon[32529]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:49.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:49 smithi033 ceph-mon[29201]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:49.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:49.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:49.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:51 smithi038 ceph-mon[32450]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:51 smithi117 ceph-mon[32529]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:51 smithi033 ceph-mon[29201]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:53.257 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 10 on host 'smithi038' 2024-04-24T23:08:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.254+0000 7f43f482a700 1 -- 172.21.15.38:0/3343318172 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f43f0062670 con 0x7f43d80a2c20 2024-04-24T23:08:53.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43d80a2c20 msgr2=0x7f43d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43d80a2c20 0x7f43d80a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43e40099d0 tx=0x7f43e4008040 comp rx=0 tx=0).stop 2024-04-24T23:08:53.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 msgr2=0x7f43f0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f0198460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43e0007fa0 tx=0x7f43e000bf10 comp rx=0 tx=0).stop 2024-04-24T23:08:53.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 shutdown_connections 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f43d80a2c20 0x7f43d80a50e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0108040 0x7f43f019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f0102de0 0x7f43f0198460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 --2- 172.21.15.38:0/3343318172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f0105710 0x7f43f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.256+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 >> 172.21.15.38:0/3343318172 conn(0x7f43f00fc750 msgr2=0x7f43f0103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.257+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 shutdown_connections 2024-04-24T23:08:53.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-24T23:08:53.257+0000 7f43f582c700 1 -- 172.21.15.38:0/3343318172 wait complete. 2024-04-24T23:08:53.635 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:53.635 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.636 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:53 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:08:53.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:53 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:53 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:53.989 DEBUG:teuthology.orchestra.run.smithi038:osd.10> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T23:08:53.992 INFO:tasks.cephadm:Deploying osd.11 on smithi117 with /dev/vg_nvme/lv_1... 2024-04-24T23:08:53.992 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:08:54.240 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:08:54.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:54 smithi033 ceph-mon[29201]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:08:54.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:54 smithi033 ceph-mon[29201]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:08:54.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:54.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:54 smithi033 ceph-mon[29201]: from='osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:08:54.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:54 smithi033 ceph-mon[29201]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:08:55.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-mon[32450]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:08:55.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-mon[32450]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:08:55.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:55.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-mon[32450]: from='osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:08:55.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-mon[32450]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:08:55.163 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:08:54.683+0000 7fa4bd683700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T23:08:55.163 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:08:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:08:54.695+0000 7fa4b9476700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:55.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:54 smithi117 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:08:55.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:54 smithi117 ceph-mon[32529]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:08:55.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:55.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:54 smithi117 ceph-mon[32529]: from='osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:08:55.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:54 smithi117 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-04-24T23:08:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:55 smithi038 ceph-mon[32450]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:55 smithi038 ceph-mon[32450]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:08:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:55 smithi038 ceph-mon[32450]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:08:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:55 smithi033 ceph-mon[29201]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:55 smithi033 ceph-mon[29201]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:08:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:55 smithi033 ceph-mon[29201]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:08:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:55 smithi117 ceph-mon[32529]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:56.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:55 smithi117 ceph-mon[32529]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-04-24T23:08:56.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:55 smithi117 ceph-mon[32529]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:08:56.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:08:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:08:56.286 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-24T23:08:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-24T23:08:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112734 s, 930 MB/s 2024-04-24T23:08:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-24T23:08:56.897 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:08:56.897 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:08:56.897 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.897 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] boot 2024-04-24T23:08:56.897 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:08:56.897 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.898 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:56 smithi038 ceph-mon[32450]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:08:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:08:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] boot 2024-04-24T23:08:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:08:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:56 smithi117 ceph-mon[32529]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:08:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:08:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: osd.10 [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] boot 2024-04-24T23:08:56.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:08:56.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:08:56.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:56 smithi033 ceph-mon[29201]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:08:57.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-04-24T23:08:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:08:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:08:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi038 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi038 to 4273M 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.082 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:08:58.082 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi038 2024-04-24T23:08:58.082 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi038 to 4273M 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:58.083 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi038 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi038 to 4273M 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:08:58.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 -- 172.21.15.117:0/881514997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640102d30 msgr2=0x7f36401095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 --2- 172.21.15.117:0/881514997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640102d30 0x7f36401095e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f363800b0a0 tx=0x7f363800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 -- 172.21.15.117:0/881514997 shutdown_connections 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 --2- 172.21.15.117:0/881514997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640102d30 0x7f36401095e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 --2- 172.21.15.117:0/881514997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3640101190 0x7f36401015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 --2- 172.21.15.117:0/881514997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 0x7f36401027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 -- 172.21.15.117:0/881514997 >> 172.21.15.117:0/881514997 conn(0x7f36400fc730 msgr2=0x7f36400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 -- 172.21.15.117:0/881514997 shutdown_connections 2024-04-24T23:08:58.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 -- 172.21.15.117:0/881514997 wait complete. 2024-04-24T23:08:58.992 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:58 smithi117 ceph-mon[32529]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:08:58.992 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:58 smithi117 ceph-mon[32529]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:08:58.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 Processor -- start 2024-04-24T23:08:58.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.989+0000 7f3646c1c700 1 -- start start 2024-04-24T23:08:58.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 0x7f3640105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 0x7f3640103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3640102d30 0x7f3640103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f364010cd60 con 0x7f3640102390 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f364010cbe0 con 0x7f3640102d30 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f364010cee0 con 0x7f3640101190 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 0x7f3640105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 0x7f3640105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54346/0 (socket says 172.21.15.117:54346) 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 -- 172.21.15.117:0/1229076685 learned_addr learned my addr 172.21.15.117:0/1229076685 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36451b9700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3640102d30 0x7f3640103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f363ffff700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 0x7f3640103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 -- 172.21.15.117:0/1229076685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3640102d30 msgr2=0x7f3640103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3640102d30 0x7f3640103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 -- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 msgr2=0x7f3640103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 0x7f3640103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 -- 172.21.15.117:0/1229076685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3640104560 con 0x7f3640101190 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f363ffff700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 0x7f3640103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f36449b8700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 0x7f3640105120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f363400c990 tx=0x7f363400cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634007b00 con 0x7f3640101190 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634007ca0 con 0x7f3640101190 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36401a24f0 con 0x7f3640101190 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.990+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634018680 con 0x7f3640101190 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.991+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401a2a30 con 0x7f3640101190 2024-04-24T23:08:58.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.992+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3640068f20 con 0x7f3640101190 2024-04-24T23:08:58.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.993+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f363400f450 con 0x7f3640101190 2024-04-24T23:08:58.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.993+0000 7f363dffb700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36300a2ca0 0x7f36300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.993+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f36340ce980 con 0x7f3640101190 2024-04-24T23:08:58.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.994+0000 7f363ffff700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36300a2ca0 0x7f36300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.994+0000 7f363ffff700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36300a2ca0 0x7f36300a5160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f362c005950 tx=0x7f362c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:58.998+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3634014030 con 0x7f3640101190 2024-04-24T23:08:59.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:58 smithi038 ceph-mon[32450]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:08:59.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:58 smithi038 ceph-mon[32450]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:08:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:08:59.202+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f36400fe7f0 con 0x7f36300a2ca0 2024-04-24T23:08:59.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:58 smithi033 ceph-mon[29201]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:08:59.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:58 smithi033 ceph-mon[29201]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:00.034 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:59 smithi117 ceph-mon[32529]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:09:00.034 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:59 smithi117 ceph-mon[32529]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:00.034 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:00.034 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:00.035 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:08:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:00.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:59 smithi033 ceph-mon[29201]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:09:00.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:59 smithi033 ceph-mon[29201]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:00.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:00.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:00.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:08:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:59 smithi038 ceph-mon[32450]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:09:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:59 smithi038 ceph-mon[32450]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:08:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:01.089 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:00 smithi117 ceph-mon[32529]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:09:01.090 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:00 smithi117 ceph-mon[32529]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:01.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:00 smithi038 ceph-mon[32450]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:09:01.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:00 smithi038 ceph-mon[32450]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:00 smithi033 ceph-mon[29201]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:09:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:00 smithi033 ceph-mon[29201]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:03.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-24T23:09:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/3709592109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]: dispatch 2024-04-24T23:09:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]: dispatch 2024-04-24T23:09:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]': finished 2024-04-24T23:09:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:09:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:02 smithi038 ceph-mon[32450]: from='client.? 172.21.15.117:0/4209908102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/3709592109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]: dispatch 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]: dispatch 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]': finished 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:03.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:02 smithi117 ceph-mon[32529]: from='client.? 172.21.15.117:0/4209908102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/3709592109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]: dispatch 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]: dispatch 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8842cc2-f8eb-4b78-9439-9aea72e0ae4b"}]': finished 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:02 smithi033 ceph-mon[29201]: from='client.? 172.21.15.117:0/4209908102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:05.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:04 smithi038 ceph-mon[32450]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:09:05.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:04 smithi117 ceph-mon[32529]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:09:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:04 smithi033 ceph-mon[29201]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:09:07.154 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:06 smithi117 ceph-mon[32529]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-24T23:09:07.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:06 smithi038 ceph-mon[32450]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-24T23:09:07.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:06 smithi033 ceph-mon[29201]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-24T23:09:08.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:08 smithi117 ceph-mon[32529]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:09.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:08 smithi038 ceph-mon[32450]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:09.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:08 smithi033 ceph-mon[29201]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:10.127 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:09:10.128 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:10.128 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:09 smithi117 ceph-mon[32529]: Deploying daemon osd.11 on smithi117 2024-04-24T23:09:10.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:09:10.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:10.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:09 smithi038 ceph-mon[32450]: Deploying daemon osd.11 on smithi117 2024-04-24T23:09:10.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:09:10.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:10.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:09 smithi033 ceph-mon[29201]: Deploying daemon osd.11 on smithi117 2024-04-24T23:09:11.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:10 smithi038 ceph-mon[32450]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:10 smithi117 ceph-mon[32529]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:10 smithi033 ceph-mon[29201]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:12 smithi038 ceph-mon[32450]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:13.171 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:12 smithi117 ceph-mon[32529]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:13.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:12 smithi033 ceph-mon[29201]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:15.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:14 smithi117 ceph-mon[32529]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:15.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:14 smithi038 ceph-mon[32450]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:14 smithi033 ceph-mon[29201]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:16.360 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:16.361 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:16.361 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:16.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:16.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:16.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:16.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:16.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:16.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:17 smithi117 ceph-mon[32529]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:17 smithi033 ceph-mon[29201]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:17.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:17 smithi038 ceph-mon[32450]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:19.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.075+0000 7f363dffb700 1 -- 172.21.15.117:0/1229076685 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f36400fe7f0 con 0x7f36300a2ca0 2024-04-24T23:09:19.078 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 11 on host 'smithi117' 2024-04-24T23:09:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36300a2ca0 msgr2=0x7f36300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36300a2ca0 0x7f36300a5160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f362c005950 tx=0x7f362c0058e0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 msgr2=0x7f3640105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 0x7f3640105120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f363400c990 tx=0x7f363400cca0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 shutdown_connections 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36300a2ca0 0x7f36300a5160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3640102d30 0x7f3640103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3640101190 0x7f3640105120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 --2- 172.21.15.117:0/1229076685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3640102390 0x7f3640103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 >> 172.21.15.117:0/1229076685 conn(0x7f36400fc730 msgr2=0x7f36400fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 shutdown_connections 2024-04-24T23:09:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:19.078+0000 7f3646c1c700 1 -- 172.21.15.117:0/1229076685 wait complete. 2024-04-24T23:09:19.567 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:19.567 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.568 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:19 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:19.642 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.643 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:19 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:19.668 DEBUG:teuthology.orchestra.run.smithi117:osd.11> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T23:09:19.670 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T23:09:19.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:09:19.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:09:19.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:19.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:19 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:19.931 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:09:20.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:20 smithi033 ceph-mon[29201]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:09:20.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:20 smithi033 ceph-mon[29201]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:09:20.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:20 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:20.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:20 smithi033 ceph-mon[29201]: from='osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:09:20.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:20 smithi033 ceph-mon[29201]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:09:20.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:20 smithi038 ceph-mon[32450]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:09:20.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:20 smithi038 ceph-mon[32450]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:09:20.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:20 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:20.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:20 smithi038 ceph-mon[32450]: from='osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:09:20.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:20 smithi038 ceph-mon[32450]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:09:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:09:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-mon[32529]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:09:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-mon[32529]: from='osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:09:20.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-24T23:09:20.678 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:09:20.304+0000 7f89268f8700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T23:09:20.678 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:09:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:09:20.316+0000 7f89226eb700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:21.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:21 smithi033 ceph-mon[29201]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:21.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:21 smithi033 ceph-mon[29201]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:09:21.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:21 smithi033 ceph-mon[29201]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:09:21.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:21.588 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:21 smithi038 ceph-mon[32450]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:21.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:21 smithi038 ceph-mon[32450]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:09:21.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:21 smithi038 ceph-mon[32450]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:09:21.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:21.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:21 smithi117 ceph-mon[32529]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:21 smithi117 ceph-mon[32529]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-24T23:09:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:21 smithi117 ceph-mon[32529]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:09:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:21.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.820+0000 7f8c5de20700 1 -- 172.21.15.33:0/776675927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 msgr2=0x7f8c581026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.820+0000 7f8c5de20700 1 --2- 172.21.15.33:0/776675927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c581026d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8c480099c0 tx=0x7f8c48009cd0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 -- 172.21.15.33:0/776675927 shutdown_connections 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 --2- 172.21.15.33:0/776675927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c58103e50 0x7f8c5810a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 --2- 172.21.15.33:0/776675927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c581026d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 --2- 172.21.15.33:0/776675927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c581034b0 0x7f8c58103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 -- 172.21.15.33:0/776675927 >> 172.21.15.33:0/776675927 conn(0x7f8c580fd850 msgr2=0x7f8c580ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 -- 172.21.15.33:0/776675927 shutdown_connections 2024-04-24T23:09:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 -- 172.21.15.33:0/776675927 wait complete. 2024-04-24T23:09:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.821+0000 7f8c5de20700 1 Processor -- start 2024-04-24T23:09:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 -- start start 2024-04-24T23:09:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c58198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c581034b0 0x7f8c58198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c58103e50 0x7f8c5819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5810dec0 con 0x7f8c581022b0 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c58198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c58198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42564/0 (socket says 172.21.15.33:42564) 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 -- 172.21.15.33:0/2805377687 learned_addr learned my addr 172.21.15.33:0/2805377687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5810dd40 con 0x7f8c58103e50 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5810e040 con 0x7f8c581034b0 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c56ffd700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c581034b0 0x7f8c58198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c57fff700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c58103e50 0x7f8c5819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 -- 172.21.15.33:0/2805377687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c581034b0 msgr2=0x7f8c58198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c581034b0 0x7f8c58198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 -- 172.21.15.33:0/2805377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c58103e50 msgr2=0x7f8c5819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c58103e50 0x7f8c5819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 -- 172.21.15.33:0/2805377687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c581a0160 con 0x7f8c581022b0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c577fe700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c58198500 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8c480057c0 tx=0x7f8c48009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.822+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48027070 con 0x7f8c581022b0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.823+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48004150 con 0x7f8c581022b0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.823+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c581a03f0 con 0x7f8c581022b0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.823+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48022e20 con 0x7f8c581022b0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.823+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581a08a0 con 0x7f8c581022b0 2024-04-24T23:09:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.824+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c58068f20 con 0x7f8c581022b0 2024-04-24T23:09:21.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.825+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c4802a070 con 0x7f8c581022b0 2024-04-24T23:09:21.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.826+0000 7f8c54ff9700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c400a2ca0 0x7f8c400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.826+0000 7f8c56ffd700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c400a2ca0 0x7f8c400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.826+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8c480cfb30 con 0x7f8c581022b0 2024-04-24T23:09:21.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.826+0000 7f8c56ffd700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c400a2ca0 0x7f8c400a5160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005950 tx=0x7f8c4c00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:21.830+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8c48013070 con 0x7f8c581022b0 2024-04-24T23:09:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.038+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8c58062e40 con 0x7f8c581022b0 2024-04-24T23:09:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.038+0000 7f8c54ff9700 1 -- 172.21.15.33:0/2805377687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f8c4809cfc0 con 0x7f8c581022b0 2024-04-24T23:09:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:22.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.040+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c400a2ca0 msgr2=0x7f8c400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.040+0000 7f8c5de20700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c400a2ca0 0x7f8c400a5160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005950 tx=0x7f8c4c00a300 comp rx=0 tx=0).stop 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 msgr2=0x7f8c58198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c58198500 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8c480057c0 tx=0x7f8c48009580 comp rx=0 tx=0).stop 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 shutdown_connections 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c400a2ca0 0x7f8c400a5160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c58103e50 0x7f8c5819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c581022b0 0x7f8c58198500 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 --2- 172.21.15.33:0/2805377687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c581034b0 0x7f8c58198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 >> 172.21.15.33:0/2805377687 conn(0x7f8c580fd850 msgr2=0x7f8c58106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 shutdown_connections 2024-04-24T23:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:22.041+0000 7f8c5de20700 1 -- 172.21.15.33:0/2805377687 wait complete. 2024-04-24T23:09:22.635 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: purged_snaps scrub starts 2024-04-24T23:09:22.635 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: purged_snaps scrub ok 2024-04-24T23:09:22.635 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:22.635 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] boot 2024-04-24T23:09:22.636 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:09:22.636 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:22.636 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:22 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2805377687' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:22.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: purged_snaps scrub starts 2024-04-24T23:09:22.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: purged_snaps scrub ok 2024-04-24T23:09:22.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:22.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] boot 2024-04-24T23:09:22.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:09:22.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:22.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:22 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2805377687' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:22.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: purged_snaps scrub starts 2024-04-24T23:09:22.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: purged_snaps scrub ok 2024-04-24T23:09:22.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:22.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: osd.11 [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] boot 2024-04-24T23:09:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:09:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:22 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2805377687' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:22.683 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714000161,"num_in_osds":12,"osd_in_since":1714000141,"num_remapped_pgs":0} 2024-04-24T23:09:22.684 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:09:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: Detected new or changed devices on smithi117 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:23.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:23.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:09:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: Detected new or changed devices on smithi117 2024-04-24T23:09:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:23.673 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: Detected new or changed devices on smithi117 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:23.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:23.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:23.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:23.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:24.389 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:24 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi117 to 4272M 2024-04-24T23:09:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:24 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi117 to 4272M 2024-04-24T23:09:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:24 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi117 to 4272M 2024-04-24T23:09:24.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.748+0000 7f8e9342a700 1 -- 172.21.15.33:0/2736929805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c102510 msgr2=0x7f8e8c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.748+0000 7f8e9342a700 1 --2- 172.21.15.33:0/2736929805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c102510 0x7f8e8c1028f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009990 tx=0x7f8e88009ca0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 -- 172.21.15.33:0/2736929805 shutdown_connections 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 --2- 172.21.15.33:0/2736929805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c107590 0x7f8e8c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 --2- 172.21.15.33:0/2736929805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c102510 0x7f8e8c1028f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 --2- 172.21.15.33:0/2736929805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e8c102ec0 0x7f8e8c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 -- 172.21.15.33:0/2736929805 >> 172.21.15.33:0/2736929805 conn(0x7f8e8c0793a0 msgr2=0x7f8e8c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 -- 172.21.15.33:0/2736929805 shutdown_connections 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 -- 172.21.15.33:0/2736929805 wait complete. 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 Processor -- start 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.749+0000 7f8e9342a700 1 -- start start 2024-04-24T23:09:24.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e9342a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e8c102510 0x7f8e8c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e9342a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c102ec0 0x7f8e8c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e9342a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 0x7f8e8c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e9342a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c112570 con 0x7f8e8c107590 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e9342a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1123f0 con 0x7f8e8c102ec0 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e9342a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1126f0 con 0x7f8e8c102510 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 0x7f8e8c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 0x7f8e8c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42586/0 (socket says 172.21.15.33:42586) 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 -- 172.21.15.33:0/1762130426 learned_addr learned my addr 172.21.15.33:0/1762130426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e911c6700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e8c102510 0x7f8e8c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e909c5700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c102ec0 0x7f8e8c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 -- 172.21.15.33:0/1762130426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e8c102510 msgr2=0x7f8e8c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e8c102510 0x7f8e8c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 -- 172.21.15.33:0/1762130426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c102ec0 msgr2=0x7f8e8c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c102ec0 0x7f8e8c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 -- 172.21.15.33:0/1762130426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c199090 con 0x7f8e8c107590 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e909c5700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c102ec0 0x7f8e8c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e919c7700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 0x7f8e8c1987b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8e84007d70 tx=0x7f8e8400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.750+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84016020 con 0x7f8e8c107590 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.751+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c199380 con 0x7f8e8c107590 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.751+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8400ad40 con 0x7f8e8c107590 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.751+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84011600 con 0x7f8e8c107590 2024-04-24T23:09:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.751+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a5610 con 0x7f8e8c107590 2024-04-24T23:09:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.751+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c10a7b0 con 0x7f8e8c107590 2024-04-24T23:09:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.753+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e84019020 con 0x7f8e8c107590 2024-04-24T23:09:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.753+0000 7f8e827fc700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e780a2c20 0x7f8e780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.754+0000 7f8e911c6700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e780a2c20 0x7f8e780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.754+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e840cef10 con 0x7f8e8c107590 2024-04-24T23:09:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.754+0000 7f8e911c6700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e780a2c20 0x7f8e780a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b5c0 tx=0x7f8e88005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.757+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8e84020020 con 0x7f8e8c107590 2024-04-24T23:09:24.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.962+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e8c199c90 con 0x7f8e8c107590 2024-04-24T23:09:24.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.963+0000 7f8e827fc700 1 -- 172.21.15.33:0/1762130426 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18420 (secure 0 0 0) 0x7f8e8409c310 con 0x7f8e8c107590 2024-04-24T23:09:24.965 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:24.965 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:09:22.310342+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e780a2c20 msgr2=0x7f8e780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e780a2c20 0x7f8e780a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b5c0 tx=0x7f8e88005960 comp rx=0 tx=0).stop 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 msgr2=0x7f8e8c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 0x7f8e8c1987b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8e84007d70 tx=0x7f8e8400a340 comp rx=0 tx=0).stop 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 shutdown_connections 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e780a2c20 0x7f8e780a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e8c107590 0x7f8e8c1987b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e8c102510 0x7f8e8c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 --2- 172.21.15.33:0/1762130426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e8c102ec0 0x7f8e8c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 >> 172.21.15.33:0/1762130426 conn(0x7f8e8c0793a0 msgr2=0x7f8e8c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 shutdown_connections 2024-04-24T23:09:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:24.965+0000 7f8e9342a700 1 -- 172.21.15.33:0/1762130426 wait complete. 2024-04-24T23:09:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:25 smithi038 ceph-mon[32450]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:25.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1762130426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:25 smithi117 ceph-mon[32529]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1762130426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:25 smithi033 ceph-mon[29201]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1762130426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:25.772 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:09:25.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T23:09:26.022 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:09:27.415 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:27 smithi033 ceph-mon[29201]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:27 smithi038 ceph-mon[32450]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:27 smithi117 ceph-mon[32529]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 -- 172.21.15.33:0/2161634092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48100e50 msgr2=0x7fda48101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 --2- 172.21.15.33:0/2161634092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48100e50 0x7fda48101230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda380099c0 tx=0x7fda38009cd0 comp rx=0 tx=0).stop 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 -- 172.21.15.33:0/2161634092 shutdown_connections 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 --2- 172.21.15.33:0/2161634092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda48107670 0x7fda4810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 --2- 172.21.15.33:0/2161634092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48100e50 0x7fda48101230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 --2- 172.21.15.33:0/2161634092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda48101800 0x7fda48106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 -- 172.21.15.33:0/2161634092 >> 172.21.15.33:0/2161634092 conn(0x7fda48076450 msgr2=0x7fda48076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 -- 172.21.15.33:0/2161634092 shutdown_connections 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 -- 172.21.15.33:0/2161634092 wait complete. 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 Processor -- start 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.785+0000 7fda4e102700 1 -- start start 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda4e102700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda48100e50 0x7fda48101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda4e102700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 0x7fda48102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda4e102700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda48107670 0x7fda48102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda4e102700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda48112610 con 0x7fda48101800 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda4e102700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda48112490 con 0x7fda48107670 2024-04-24T23:09:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda4e102700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda48112790 con 0x7fda48100e50 2024-04-24T23:09:27.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 0x7fda48102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:27.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 0x7fda48102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57002/0 (socket says 172.21.15.33:57002) 2024-04-24T23:09:27.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 -- 172.21.15.33:0/3219709963 learned_addr learned my addr 172.21.15.33:0/3219709963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:27.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda477fe700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda48100e50 0x7fda48101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:27.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda47fff700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda48107670 0x7fda48102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 -- 172.21.15.33:0/3219709963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda48100e50 msgr2=0x7fda48101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda48100e50 0x7fda48101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 -- 172.21.15.33:0/3219709963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda48107670 msgr2=0x7fda48102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda48107670 0x7fda48102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 -- 172.21.15.33:0/3219709963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481a6ed0 con 0x7fda48101800 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.786+0000 7fda46ffd700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 0x7fda48102310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00d980 tx=0x7fda3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.787+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c017070 con 0x7fda48101800 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.787+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481a71c0 con 0x7fda48101800 2024-04-24T23:09:27.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.787+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda3c005020 con 0x7fda48101800 2024-04-24T23:09:27.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.787+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00fbf0 con 0x7fda48101800 2024-04-24T23:09:27.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.787+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481a76b0 con 0x7fda48101800 2024-04-24T23:09:27.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.788+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4810a820 con 0x7fda48101800 2024-04-24T23:09:27.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.789+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fda3c0051d0 con 0x7fda48101800 2024-04-24T23:09:27.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.789+0000 7fda44ff9700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fda300a2c20 0x7fda300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:27.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.789+0000 7fda477fe700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fda300a2c20 0x7fda300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:27.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.789+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fda3c0ce9d0 con 0x7fda48101800 2024-04-24T23:09:27.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.790+0000 7fda477fe700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fda300a2c20 0x7fda300a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda38005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:27.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:27.794+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fda3c020020 con 0x7fda48101800 2024-04-24T23:09:28.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.004+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda48103300 con 0x7fda48101800 2024-04-24T23:09:28.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.004+0000 7fda44ff9700 1 -- 172.21.15.33:0/3219709963 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fda3c09bdd0 con 0x7fda48101800 2024-04-24T23:09:28.006 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:09:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fda300a2c20 msgr2=0x7fda300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fda300a2c20 0x7fda300a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda38005fb0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 msgr2=0x7fda48102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 0x7fda48102310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00d980 tx=0x7fda3c00dc90 comp rx=0 tx=0).stop 2024-04-24T23:09:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 shutdown_connections 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fda300a2c20 0x7fda300a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda48107670 0x7fda48102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda48100e50 0x7fda48101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 --2- 172.21.15.33:0/3219709963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda48101800 0x7fda48102310 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 >> 172.21.15.33:0/3219709963 conn(0x7fda48076450 msgr2=0x7fda480fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 shutdown_connections 2024-04-24T23:09:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:28.006+0000 7fda4e102700 1 -- 172.21.15.33:0/3219709963 wait complete. 2024-04-24T23:09:28.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3219709963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3219709963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3219709963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.b on smithi038 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.e on smithi038 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.h on smithi038 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.k on smithi038 2024-04-24T23:09:28.722 INFO:tasks.cephadm:Adding mds.c on smithi117 2024-04-24T23:09:28.723 INFO:tasks.cephadm:Adding mds.f on smithi117 2024-04-24T23:09:28.723 INFO:tasks.cephadm:Adding mds.i on smithi117 2024-04-24T23:09:28.723 INFO:tasks.cephadm:Adding mds.l on smithi117 2024-04-24T23:09:28.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l' 2024-04-24T23:09:28.972 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:09:29.641 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:29 smithi038 ceph-mon[32450]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:29.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:29 smithi117 ceph-mon[32529]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:29.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:29 smithi033 ceph-mon[29201]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:30.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.630+0000 7fa7d9267700 1 -- 172.21.15.117:0/405838790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d41011b0 msgr2=0x7fa7d41015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.630+0000 7fa7d9267700 1 --2- 172.21.15.117:0/405838790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d41011b0 0x7fa7d41015d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009990 tx=0x7fa7c4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.630+0000 7fa7d9267700 1 -- 172.21.15.117:0/405838790 shutdown_connections 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.630+0000 7fa7d9267700 1 --2- 172.21.15.117:0/405838790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d4102d50 0x7fa7d4109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.630+0000 7fa7d9267700 1 --2- 172.21.15.117:0/405838790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d41011b0 0x7fa7d41015d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 --2- 172.21.15.117:0/405838790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d41023b0 0x7fa7d4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 -- 172.21.15.117:0/405838790 >> 172.21.15.117:0/405838790 conn(0x7fa7d40fc790 msgr2=0x7fa7d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 -- 172.21.15.117:0/405838790 shutdown_connections 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 -- 172.21.15.117:0/405838790 wait complete. 2024-04-24T23:09:30.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 Processor -- start 2024-04-24T23:09:30.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 -- start start 2024-04-24T23:09:30.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.631+0000 7fa7d9267700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d41011b0 0x7fa7d4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:30.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d41023b0 0x7fa7d4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:30.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 0x7fa7d419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:30.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d410dee0 con 0x7fa7d41023b0 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d410dd60 con 0x7fa7d41011b0 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d410e060 con 0x7fa7d4102d50 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 0x7fa7d419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 0x7fa7d419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54068/0 (socket says 172.21.15.117:54068) 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 -- 172.21.15.117:0/1834645197 learned_addr learned my addr 172.21.15.117:0/1834645197 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d27fc700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d41023b0 0x7fa7d4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d2ffd700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d41011b0 0x7fa7d4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 -- 172.21.15.117:0/1834645197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d41011b0 msgr2=0x7fa7d4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d41011b0 0x7fa7d4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 -- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d41023b0 msgr2=0x7fa7d4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d41023b0 0x7fa7d4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 -- 172.21.15.117:0/1834645197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d41a0090 con 0x7fa7d4102d50 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d37fe700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 0x7fa7d419f9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc007d40 tx=0x7fa7cc00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:30.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc016020 con 0x7fa7d4102d50 2024-04-24T23:09:30.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc00ad60 con 0x7fa7d4102d50 2024-04-24T23:09:30.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d41a0380 con 0x7fa7d4102d50 2024-04-24T23:09:30.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc011600 con 0x7fa7d4102d50 2024-04-24T23:09:30.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.632+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d41a08c0 con 0x7fa7d4102d50 2024-04-24T23:09:30.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.633+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d4068f20 con 0x7fa7d4102d50 2024-04-24T23:09:30.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.634+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa7cc019020 con 0x7fa7d4102d50 2024-04-24T23:09:30.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.635+0000 7fa7bbfff700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7bc0a2ca0 0x7fa7bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:30.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.635+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa7cc0ceb20 con 0x7fa7d4102d50 2024-04-24T23:09:30.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.635+0000 7fa7d2ffd700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7bc0a2ca0 0x7fa7bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:30.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.636+0000 7fa7d2ffd700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7bc0a2ca0 0x7fa7bc0a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40039c0 tx=0x7fa7c4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:30.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.640+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa7cc020020 con 0x7fa7d4102d50 2024-04-24T23:09:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.845+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}) -- 0x7fa7d4062870 con 0x7fa7bc0a2ca0 2024-04-24T23:09:30.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.854+0000 7fa7bbfff700 1 -- 172.21.15.117:0/1834645197 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fa7d4062870 con 0x7fa7bc0a2ca0 2024-04-24T23:09:30.855 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mds.all update... 2024-04-24T23:09:30.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7bc0a2ca0 msgr2=0x7fa7bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7bc0a2ca0 0x7fa7bc0a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40039c0 tx=0x7fa7c4005fb0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 msgr2=0x7fa7d419f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 0x7fa7d419f9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc007d40 tx=0x7fa7cc00a340 comp rx=0 tx=0).stop 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 shutdown_connections 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7bc0a2ca0 0x7fa7bc0a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4102d50 0x7fa7d419f9f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7d41011b0 0x7fa7d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 --2- 172.21.15.117:0/1834645197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d41023b0 0x7fa7d4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 >> 172.21.15.117:0/1834645197 conn(0x7fa7d40fc790 msgr2=0x7fa7d4105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 shutdown_connections 2024-04-24T23:09:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-24T23:09:30.856+0000 7fa7d9267700 1 -- 172.21.15.117:0/1834645197 wait complete. 2024-04-24T23:09:31.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:31 smithi117 ceph-mon[32529]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:31.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:31.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:31.567 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T23:09:31.570 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T23:09:31.614 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T23:09:31.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:31 smithi033 ceph-mon[29201]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:31.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:31.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:31.635 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T23:09:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:31 smithi038 ceph-mon[32450]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:31.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:31.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:31.677 DEBUG:teuthology.orchestra.run.smithi038:mds.b> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T23:09:31.680 DEBUG:teuthology.orchestra.run.smithi038:mds.e> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T23:09:31.724 DEBUG:teuthology.orchestra.run.smithi038:mds.h> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T23:09:31.767 DEBUG:teuthology.orchestra.run.smithi038:mds.k> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T23:09:31.810 DEBUG:teuthology.orchestra.run.smithi117:mds.c> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T23:09:31.813 DEBUG:teuthology.orchestra.run.smithi117:mds.f> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T23:09:31.858 DEBUG:teuthology.orchestra.run.smithi117:mds.i> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T23:09:31.901 DEBUG:teuthology.orchestra.run.smithi117:mds.l> sudo journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T23:09:31.944 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T23:09:31.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:09:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3367190510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0102250 msgr2=0x7fd3f0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3367190510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0102250 0x7fd3f0108ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009ca0 tx=0x7fd3e802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3367190510 shutdown_connections 2024-04-24T23:09:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3367190510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0102250 0x7fd3f0108ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3367190510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f0101890 0x7fd3f0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3367190510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f01006a0 0x7fd3f0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3367190510 >> 172.21.15.33:0/3367190510 conn(0x7fd3f00fbc30 msgr2=0x7fd3f00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3367190510 shutdown_connections 2024-04-24T23:09:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.051+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3367190510 wait complete. 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 Processor -- start 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 -- start start 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 0x7fd3f006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0101890 0x7fd3f019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f0102250 0x7fd3f01a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f010c390 con 0x7fd3f01006a0 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f010c210 con 0x7fd3f0102250 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f6bd5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f010c510 con 0x7fd3f0101890 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 0x7fd3f006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 0x7fd3f006a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57024/0 (socket says 172.21.15.33:57024) 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 -- 172.21.15.33:0/3627953747 learned_addr learned my addr 172.21.15.33:0/3627953747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3effff640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0101890 0x7fd3f019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 -- 172.21.15.33:0/3627953747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0101890 msgr2=0x7fd3f019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0101890 0x7fd3f019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 -- 172.21.15.33:0/3627953747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f0102250 msgr2=0x7fd3f01a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f0102250 0x7fd3f01a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.052+0000 7fd3f4971640 1 -- 172.21.15.33:0/3627953747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f01a1970 con 0x7fd3f01006a0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.053+0000 7fd3f4971640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 0x7fd3f006a910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b450 tx=0x7fd3e400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.053+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4015020 con 0x7fd3f01006a0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.053+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e4011d70 con 0x7fd3f01006a0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.053+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f01a1c00 con 0x7fd3f01006a0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.053+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400f340 con 0x7fd3f01006a0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.053+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f01a2220 con 0x7fd3f01006a0 2024-04-24T23:09:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.054+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f0107a10 con 0x7fd3f01006a0 2024-04-24T23:09:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.055+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3e4018020 con 0x7fd3f01006a0 2024-04-24T23:09:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.055+0000 7fd3edffb640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd3e00a2c40 0x7fd3e00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.056+0000 7fd3effff640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd3e00a2c40 0x7fd3e00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.056+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd3e40ce6e0 con 0x7fd3f01006a0 2024-04-24T23:09:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.056+0000 7fd3effff640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd3e00a2c40 0x7fd3e00a5100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005fd0 tx=0x7fd3dc005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.060+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd3e401f020 con 0x7fd3f01006a0 2024-04-24T23:09:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.247+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3f00623e0 con 0x7fd3f01006a0 2024-04-24T23:09:32.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.248+0000 7fd3edffb640 1 -- 172.21.15.33:0/3627953747 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18420 (secure 0 0 0) 0x7fd3e400d040 con 0x7fd3f01006a0 2024-04-24T23:09:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:32.251 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:09:22.310342+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd3e00a2c40 msgr2=0x7fd3e00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd3e00a2c40 0x7fd3e00a5100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005fd0 tx=0x7fd3dc005850 comp rx=0 tx=0).stop 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 msgr2=0x7fd3f006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 0x7fd3f006a910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b450 tx=0x7fd3e400b910 comp rx=0 tx=0).stop 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 shutdown_connections 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd3e00a2c40 0x7fd3e00a5100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f0102250 0x7fd3f01a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0101890 0x7fd3f019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 --2- 172.21.15.33:0/3627953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f01006a0 0x7fd3f006a910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 >> 172.21.15.33:0/3627953747 conn(0x7fd3f00fbc30 msgr2=0x7fd3f0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 shutdown_connections 2024-04-24T23:09:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.251+0000 7fd3f6bd5640 1 -- 172.21.15.33:0/3627953747 wait complete. 2024-04-24T23:09:32.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:09:32.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:09:32.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 -- 172.21.15.33:0/2558057278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 msgr2=0x7f12b8106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 --2- 172.21.15.33:0/2558057278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8106fa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009900 tx=0x7f12a002a410 comp rx=0 tx=0).stop 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 -- 172.21.15.33:0/2558057278 shutdown_connections 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 --2- 172.21.15.33:0/2558057278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 0x7f12b81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 --2- 172.21.15.33:0/2558057278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8106fa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 --2- 172.21.15.33:0/2558057278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 0x7f12b8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 -- 172.21.15.33:0/2558057278 >> 172.21.15.33:0/2558057278 conn(0x7f12b80fbc30 msgr2=0x7f12b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 -- 172.21.15.33:0/2558057278 shutdown_connections 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.370+0000 7f12be6ba640 1 -- 172.21.15.33:0/2558057278 wait complete. 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 Processor -- start 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 -- start start 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 0x7f12b8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 0x7f12b819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810d420 con 0x7f12b8104bb0 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810d2a0 con 0x7f12b8102280 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12be6ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810d5a0 con 0x7f12b81074e0 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57032/0 (socket says 172.21.15.33:57032) 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12b77fe640 1 -- 172.21.15.33:0/3011759602 learned_addr learned my addr 172.21.15.33:0/3011759602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.371+0000 7f12bcc57640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 0x7f12b819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b7fff640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 0x7f12b8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b77fe640 1 -- 172.21.15.33:0/3011759602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 msgr2=0x7f12b819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b77fe640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 0x7f12b819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b77fe640 1 -- 172.21.15.33:0/3011759602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 msgr2=0x7f12b8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b77fe640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 0x7f12b8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b77fe640 1 -- 172.21.15.33:0/3011759602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b819f710 con 0x7f12b8104bb0 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12bcc57640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 0x7f12b819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:09:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b7fff640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 0x7f12b8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b77fe640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8198030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12a0031610 tx=0x7f12a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a003f030 con 0x7f12b8104bb0 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b819f9a0 con 0x7f12b8104bb0 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a0003ad0 con 0x7f12b8104bb0 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0035070 con 0x7f12b8104bb0 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.372+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b819fe90 con 0x7f12b8104bb0 2024-04-24T23:09:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.373+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b80fd720 con 0x7f12b8104bb0 2024-04-24T23:09:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.374+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12a0042030 con 0x7f12b8104bb0 2024-04-24T23:09:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.375+0000 7f12b57fa640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12a40a2b90 0x7f12a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.375+0000 7f12b7fff640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12a40a2b90 0x7f12a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.375+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f12a00ef4f0 con 0x7f12b8104bb0 2024-04-24T23:09:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.375+0000 7f12b7fff640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12a40a2b90 0x7f12a40a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009b40 tx=0x7f12a8005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.378+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12a00bc8f0 con 0x7f12b8104bb0 2024-04-24T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-24T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: Deploying daemon mds.l on smithi117 2024-04-24T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:32 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3627953747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:32.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.564+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12b802d300 con 0x7f12b8104bb0 2024-04-24T23:09:32.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.564+0000 7f12b57fa640 1 -- 172.21.15.33:0/3011759602 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f12a00bc1f0 con 0x7f12b8104bb0 2024-04-24T23:09:32.566 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:09:32.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.566+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12a40a2b90 msgr2=0x7f12a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.566+0000 7f12be6ba640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12a40a2b90 0x7f12a40a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009b40 tx=0x7f12a8005e50 comp rx=0 tx=0).stop 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.566+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 msgr2=0x7f12b8198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.566+0000 7f12be6ba640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8198030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12a0031610 tx=0x7f12a0004270 comp rx=0 tx=0).stop 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 shutdown_connections 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12a40a2b90 0x7f12a40a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b81074e0 0x7f12b819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12b8104bb0 0x7f12b8198030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 --2- 172.21.15.33:0/3011759602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12b8102280 0x7f12b8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 >> 172.21.15.33:0/3011759602 conn(0x7f12b80fbc30 msgr2=0x7f12b8102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 shutdown_connections 2024-04-24T23:09:32.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.567+0000 7f12be6ba640 1 -- 172.21.15.33:0/3011759602 wait complete. 2024-04-24T23:09:32.578 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:32.578 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-24T23:09:32.578 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: Deploying daemon mds.l on smithi117 2024-04-24T23:09:32.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:32 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3627953747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:32.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:32.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: Deploying daemon mds.l on smithi117 2024-04-24T23:09:32.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:32 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3627953747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:32.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 -- 172.21.15.33:0/3492310575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 msgr2=0x7f4b70106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 --2- 172.21.15.33:0/3492310575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b70106f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009900 tx=0x7f4b5802a410 comp rx=0 tx=0).stop 2024-04-24T23:09:32.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 -- 172.21.15.33:0/3492310575 shutdown_connections 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 --2- 172.21.15.33:0/3492310575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b70107460 0x7f4b701098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 --2- 172.21.15.33:0/3492310575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b70106f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 --2- 172.21.15.33:0/3492310575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 0x7f4b701045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 -- 172.21.15.33:0/3492310575 >> 172.21.15.33:0/3492310575 conn(0x7f4b700fbbf0 msgr2=0x7f4b700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 -- 172.21.15.33:0/3492310575 shutdown_connections 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.680+0000 7f4b74809640 1 -- 172.21.15.33:0/3492310575 wait complete. 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 Processor -- start 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 -- start start 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 0x7f4b70101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b701001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b70107460 0x7f4b701006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7010d3f0 con 0x7f4b70104b30 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7010d270 con 0x7f4b70102200 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b74809640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7010d570 con 0x7f4b70107460 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b701001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b701001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57052/0 (socket says 172.21.15.33:57052) 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b67fff640 1 -- 172.21.15.33:0/4211603676 learned_addr learned my addr 172.21.15.33:0/4211603676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.681+0000 7f4b6ed9d640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b70107460 0x7f4b701006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b6e59c640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 0x7f4b70101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b67fff640 1 -- 172.21.15.33:0/4211603676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b70107460 msgr2=0x7f4b701006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b67fff640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b70107460 0x7f4b701006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b67fff640 1 -- 172.21.15.33:0/4211603676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 msgr2=0x7f4b70101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b67fff640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 0x7f4b70101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b67fff640 1 -- 172.21.15.33:0/4211603676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70100f80 con 0x7f4b70104b30 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b6e59c640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 0x7f4b70101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b67fff640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b701001b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b58031610 tx=0x7f4b580043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5803f030 con 0x7f4b70104b30 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b58003ce0 con 0x7f4b70104b30 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b701a1b90 con 0x7f4b70104b30 2024-04-24T23:09:32.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58035070 con 0x7f4b70104b30 2024-04-24T23:09:32.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.682+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b701a20e0 con 0x7f4b70104b30 2024-04-24T23:09:32.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.683+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700fd6e0 con 0x7f4b70104b30 2024-04-24T23:09:32.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.684+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b58042030 con 0x7f4b70104b30 2024-04-24T23:09:32.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.685+0000 7f4b677fe640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b5c0a2b90 0x7f4b5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.685+0000 7f4b6e59c640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b5c0a2b90 0x7f4b5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.685+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4b580ef170 con 0x7f4b70104b30 2024-04-24T23:09:32.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.685+0000 7f4b6e59c640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b5c0a2b90 0x7f4b5c0a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b600058c0 tx=0x7f4b60005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.689+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b580bc570 con 0x7f4b70104b30 2024-04-24T23:09:32.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.920+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b7002d3c0 con 0x7f4b70104b30 2024-04-24T23:09:32.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.920+0000 7f4b677fe640 1 -- 172.21.15.33:0/4211603676 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f4b580bbe70 con 0x7f4b70104b30 2024-04-24T23:09:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:32.922 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T23:09:32.922 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b5c0a2b90 msgr2=0x7f4b5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b5c0a2b90 0x7f4b5c0a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b600058c0 tx=0x7f4b60005d30 comp rx=0 tx=0).stop 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 msgr2=0x7f4b701001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b701001b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b58031610 tx=0x7f4b580043b0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 shutdown_connections 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b5c0a2b90 0x7f4b5c0a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b70107460 0x7f4b701006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b70104b30 0x7f4b701001b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 --2- 172.21.15.33:0/4211603676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102200 0x7f4b70101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 >> 172.21.15.33:0/4211603676 conn(0x7f4b700fbbf0 msgr2=0x7f4b70102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 shutdown_connections 2024-04-24T23:09:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:32.922+0000 7f4b74809640 1 -- 172.21.15.33:0/4211603676 wait complete. 2024-04-24T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:33 smithi117 ceph-mon[32529]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:33 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3011759602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:33 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4211603676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:33.677 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:33 smithi117 systemd[1]: Starting Ceph mds.l for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:09:33.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:33 smithi038 ceph-mon[32450]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:33.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:33 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3011759602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:33 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4211603676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:33 smithi033 ceph-mon[29201]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:33 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3011759602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:33 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4211603676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:34.048 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:33 smithi117 podman[62437]: 2024-04-24 23:09:33.687591039 +0000 UTC m=+0.017941598 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:09:34.049 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:33 smithi117 podman[62437]: 2024-04-24 23:09:33.803443701 +0000 UTC m=+0.133794249 container create ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:09:34.049 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:34 smithi117 podman[62437]: 2024-04-24 23:09:34.047982227 +0000 UTC m=+0.378332781 container init ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T23:09:34.426 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:34 smithi117 podman[62437]: 2024-04-24 23:09:34.050877808 +0000 UTC m=+0.381228354 container start ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T23:09:34.426 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:34 smithi117 ceph-mds[62456]: -- 172.21.15.117:0/287484057 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600328cba40 con 0x56003289e800 2024-04-24T23:09:34.426 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:34 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l[62452]: starting mds.l at 2024-04-24T23:09:34.426 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:34 smithi117 bash[62437]: ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 2024-04-24T23:09:34.426 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:09:34 smithi117 systemd[1]: Started Ceph mds.l for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:34.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 -- 172.21.15.33:0/3918879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a30101890 msgr2=0x7f3a30101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 --2- 172.21.15.33:0/3918879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a30101890 0x7f3a30101d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02a410 comp rx=0 tx=0).stop 2024-04-24T23:09:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 -- 172.21.15.33:0/3918879979 shutdown_connections 2024-04-24T23:09:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 --2- 172.21.15.33:0/3918879979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a30102250 0x7f3a30108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 --2- 172.21.15.33:0/3918879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a30101890 0x7f3a30101d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 --2- 172.21.15.33:0/3918879979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a301006a0 0x7f3a30100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 -- 172.21.15.33:0/3918879979 >> 172.21.15.33:0/3918879979 conn(0x7f3a300fbc30 msgr2=0x7f3a300fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 -- 172.21.15.33:0/3918879979 shutdown_connections 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.048+0000 7f3a37184640 1 -- 172.21.15.33:0/3918879979 wait complete. 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 Processor -- start 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 -- start start 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 0x7f3a30197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a30101890 0x7f3a30198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a30102250 0x7f3a3019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010c300 con 0x7f3a301006a0 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010c180 con 0x7f3a30101890 2024-04-24T23:09:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a37184640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010c480 con 0x7f3a30102250 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a34f20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 0x7f3a30197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a34f20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 0x7f3a30197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57074/0 (socket says 172.21.15.33:57074) 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a34f20640 1 -- 172.21.15.33:0/1283090425 learned_addr learned my addr 172.21.15.33:0/1283090425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a2ffff640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a30101890 0x7f3a30198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.049+0000 7f3a35721640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a30102250 0x7f3a3019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a34f20640 1 -- 172.21.15.33:0/1283090425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a30102250 msgr2=0x7f3a3019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a34f20640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a30102250 0x7f3a3019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a34f20640 1 -- 172.21.15.33:0/1283090425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a30101890 msgr2=0x7f3a30198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a34f20640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a30101890 0x7f3a30198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a34f20640 1 -- 172.21.15.33:0/1283090425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3019f710 con 0x7f3a301006a0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a34f20640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 0x7f3a30197b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d770 tx=0x7f3a2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24017070 con 0x7f3a301006a0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a24009c30 con 0x7f3a301006a0 2024-04-24T23:09:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3019fa00 con 0x7f3a301006a0 2024-04-24T23:09:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24003dd0 con 0x7f3a301006a0 2024-04-24T23:09:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.050+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3019ff20 con 0x7f3a301006a0 2024-04-24T23:09:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.051+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a301a0150 con 0x7f3a301006a0 2024-04-24T23:09:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.052+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3a24003fb0 con 0x7f3a301006a0 2024-04-24T23:09:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.053+0000 7f3a2dffb640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2b90 0x7f3a200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.053+0000 7f3a2ffff640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2b90 0x7f3a200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.053+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3a240ce530 con 0x7f3a301006a0 2024-04-24T23:09:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.054+0000 7f3a2ffff640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2b90 0x7f3a200a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0314e0 tx=0x7f3a1c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.058+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a2409b930 con 0x7f3a301006a0 2024-04-24T23:09:35.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6832/2542827674,v1:172.21.15.117:6833/2542827674] up:boot 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: fsmap 1 up:standby 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:35.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:35 smithi038 ceph-mon[32450]: Deploying daemon mds.k on smithi038 2024-04-24T23:09:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.281+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a301a0320 con 0x7f3a301006a0 2024-04-24T23:09:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.281+0000 7f3a2dffb640 1 -- 172.21.15.33:0/1283090425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f3a24009de0 con 0x7f3a301006a0 2024-04-24T23:09:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:35.283 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:09:35.284 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-04-24T23:09:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2b90 msgr2=0x7f3a200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2b90 0x7f3a200a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0314e0 tx=0x7f3a1c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 msgr2=0x7f3a30197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 0x7f3a30197b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d770 tx=0x7f3a2400dc30 comp rx=0 tx=0).stop 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 shutdown_connections 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2b90 0x7f3a200a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a30102250 0x7f3a3019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a30101890 0x7f3a30198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 --2- 172.21.15.33:0/1283090425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a301006a0 0x7f3a30197b40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.283+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 >> 172.21.15.33:0/1283090425 conn(0x7f3a300fbc30 msgr2=0x7f3a300fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.284+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 shutdown_connections 2024-04-24T23:09:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:35.284+0000 7f3a37184640 1 -- 172.21.15.33:0/1283090425 wait complete. 2024-04-24T23:09:35.295 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:35.295 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6832/2542827674,v1:172.21.15.117:6833/2542827674] up:boot 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: fsmap 1 up:standby 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:35.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:35 smithi033 ceph-mon[29201]: Deploying daemon mds.k on smithi038 2024-04-24T23:09:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6832/2542827674,v1:172.21.15.117:6833/2542827674] up:boot 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: fsmap 1 up:standby 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:35.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:35 smithi117 ceph-mon[32529]: Deploying daemon mds.k on smithi038 2024-04-24T23:09:36.297 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1283090425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:36.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1283090425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:36.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1283090425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:36.671 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 systemd[1]: Starting Ceph mds.k for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:09:36.962 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 podman[61890]: 2024-04-24 23:09:36.596967191 +0000 UTC m=+0.018186682 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:09:36.963 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 podman[61890]: 2024-04-24 23:09:36.70761921 +0000 UTC m=+0.128838699 container create e0d23058e92023cbefbd68282b6ea8781eb0b9fc4f518b78c709021048fce833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T23:09:36.963 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 podman[61890]: 2024-04-24 23:09:36.935573092 +0000 UTC m=+0.356792580 container init e0d23058e92023cbefbd68282b6ea8781eb0b9fc4f518b78c709021048fce833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:09:36.963 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 podman[61890]: 2024-04-24 23:09:36.93837977 +0000 UTC m=+0.359599262 container start e0d23058e92023cbefbd68282b6ea8781eb0b9fc4f518b78c709021048fce833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T23:09:37.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 -- 172.21.15.33:0/2975348701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928102230 msgr2=0x7f9928108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 --2- 172.21.15.33:0/2975348701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928102230 0x7f9928108ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9920009ca0 tx=0x7f992002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 -- 172.21.15.33:0/2975348701 shutdown_connections 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 --2- 172.21.15.33:0/2975348701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928102230 0x7f9928108ac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 --2- 172.21.15.33:0/2975348701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928101870 0x7f9928101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 --2- 172.21.15.33:0/2975348701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928100680 0x7f9928100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 -- 172.21.15.33:0/2975348701 >> 172.21.15.33:0/2975348701 conn(0x7f99280fbbf0 msgr2=0x7f99280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.399+0000 7f992f2ba640 1 -- 172.21.15.33:0/2975348701 shutdown_connections 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 -- 172.21.15.33:0/2975348701 wait complete. 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 Processor -- start 2024-04-24T23:09:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 -- start start 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928100680 0x7f9928197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 0x7f9928197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928102230 0x7f992819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f992810c2e0 con 0x7f9928101870 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f992810c160 con 0x7f9928102230 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992f2ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f992810c460 con 0x7f9928100680 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992c855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 0x7f9928197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992c855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 0x7f9928197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56548/0 (socket says 172.21.15.33:56548) 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.400+0000 7f992c855640 1 -- 172.21.15.33:0/3949774974 learned_addr learned my addr 172.21.15.33:0/3949774974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992d857640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928102230 0x7f992819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992d056640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928100680 0x7f9928197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992c855640 1 -- 172.21.15.33:0/3949774974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928100680 msgr2=0x7f9928197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992c855640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928100680 0x7f9928197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992c855640 1 -- 172.21.15.33:0/3949774974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928102230 msgr2=0x7f992819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992c855640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928102230 0x7f992819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992c855640 1 -- 172.21.15.33:0/3949774974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992819f640 con 0x7f9928101870 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992d857640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928102230 0x7f992819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992d056640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928100680 0x7f9928197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992c855640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 0x7f9928197f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c890 tx=0x7f991c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c004220 con 0x7f9928101870 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991c0043c0 con 0x7f9928101870 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992819f930 con 0x7f9928101870 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0049e0 con 0x7f9928101870 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.401+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992819fed0 con 0x7f9928101870 2024-04-24T23:09:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.402+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99281a0100 con 0x7f9928101870 2024-04-24T23:09:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.404+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f991c0079b0 con 0x7f9928101870 2024-04-24T23:09:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.405+0000 7f991a7fc640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99140a2c10 0x7f99140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.405+0000 7f992d056640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99140a2c10 0x7f99140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.405+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f991c0cec70 con 0x7f9928101870 2024-04-24T23:09:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.406+0000 7f992d056640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99140a2c10 0x7f99140a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99281016d0 tx=0x7f9924005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.410+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f991c014030 con 0x7f9928101870 2024-04-24T23:09:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:37 smithi038 ceph-mon[32450]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:37 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:37.421 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 ceph-mds[61909]: -- 172.21.15.38:0/3338552188 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637aafe3a40 con 0x5637aafb6c00 2024-04-24T23:09:37.421 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k[61905]: starting mds.k at 2024-04-24T23:09:37.421 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:37 smithi038 bash[61890]: e0d23058e92023cbefbd68282b6ea8781eb0b9fc4f518b78c709021048fce833 2024-04-24T23:09:37.421 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:09:37 smithi038 systemd[1]: Started Ceph mds.k for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:37 smithi117 ceph-mon[32529]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:37 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:37 smithi033 ceph-mon[29201]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:37 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.640+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99281a02d0 con 0x7f9928101870 2024-04-24T23:09:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.640+0000 7f991a7fc640 1 -- 172.21.15.33:0/3949774974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f991c09c070 con 0x7f9928101870 2024-04-24T23:09:37.642 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:37.642 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:09:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-04-24T23:09:37.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99140a2c10 msgr2=0x7f99140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99140a2c10 0x7f99140a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99281016d0 tx=0x7f9924005d30 comp rx=0 tx=0).stop 2024-04-24T23:09:37.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 msgr2=0x7f9928197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 0x7f9928197f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c890 tx=0x7f991c00cd50 comp rx=0 tx=0).stop 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 shutdown_connections 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99140a2c10 0x7f99140a50d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9928102230 0x7f992819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9928101870 0x7f9928197f60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 --2- 172.21.15.33:0/3949774974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9928100680 0x7f9928197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.642+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 >> 172.21.15.33:0/3949774974 conn(0x7f99280fbbf0 msgr2=0x7f99280fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.643+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 shutdown_connections 2024-04-24T23:09:37.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:37.643+0000 7f992f2ba640 1 -- 172.21.15.33:0/3949774974 wait complete. 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6832/2605742679,v1:172.21.15.38:6833/2605742679] up:boot 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: fsmap 2 up:standby 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: Deploying daemon mds.j on smithi033 2024-04-24T23:09:38.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3949774974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6832/2605742679,v1:172.21.15.38:6833/2605742679] up:boot 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: fsmap 2 up:standby 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: Deploying daemon mds.j on smithi033 2024-04-24T23:09:38.365 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3949774974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6832/2605742679,v1:172.21.15.38:6833/2605742679] up:boot 2024-04-24T23:09:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: fsmap 2 up:standby 2024-04-24T23:09:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:09:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:38.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:38.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:38.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:38.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: Deploying daemon mds.j on smithi033 2024-04-24T23:09:38.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3949774974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:39 smithi038 ceph-mon[32450]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:39 smithi117 ceph-mon[32529]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:39.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:39 smithi033 ceph-mon[29201]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:39.456 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 systemd[1]: Starting Ceph mds.j for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:09:39.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 -- 172.21.15.33:0/1296973758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0818103350 msgr2=0x7f0818109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 --2- 172.21.15.33:0/1296973758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0818103350 0x7f0818109be0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f081002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 -- 172.21.15.33:0/1296973758 shutdown_connections 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 --2- 172.21.15.33:0/1296973758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0818103350 0x7f0818109be0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 --2- 172.21.15.33:0/1296973758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818102990 0x7f0818102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 --2- 172.21.15.33:0/1296973758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08181017a0 0x7f0818101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 -- 172.21.15.33:0/1296973758 >> 172.21.15.33:0/1296973758 conn(0x7f08180fccf0 msgr2=0x7f08180ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 -- 172.21.15.33:0/1296973758 shutdown_connections 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 -- 172.21.15.33:0/1296973758 wait complete. 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 Processor -- start 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.762+0000 7f081ea8c640 1 -- start start 2024-04-24T23:09:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 0x7f0818197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0818102990 0x7f0818198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818103350 0x7f081819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f081810d510 con 0x7f08181017a0 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f081810d390 con 0x7f0818103350 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f081810d690 con 0x7f0818102990 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 0x7f0818197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 0x7f0818197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56558/0 (socket says 172.21.15.33:56558) 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 -- 172.21.15.33:0/6132794 learned_addr learned my addr 172.21.15.33:0/6132794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081d029640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818103350 0x7f081819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f080ffff640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0818102990 0x7f0818198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 -- 172.21.15.33:0/6132794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0818102990 msgr2=0x7f0818198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0818102990 0x7f0818198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 -- 172.21.15.33:0/6132794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818103350 msgr2=0x7f081819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818103350 0x7f081819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 -- 172.21.15.33:0/6132794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081819f800 con 0x7f08181017a0 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081d029640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818103350 0x7f081819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f080ffff640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0818102990 0x7f0818198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081c828640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 0x7f0818197be0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f081400b450 tx=0x7f081400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814015020 con 0x7f08181017a0 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081819faf0 con 0x7f08181017a0 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814011d70 con 0x7f08181017a0 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.763+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400f340 con 0x7f08181017a0 2024-04-24T23:09:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.764+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081819fff0 con 0x7f08181017a0 2024-04-24T23:09:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.765+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0818108b10 con 0x7f08181017a0 2024-04-24T23:09:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.766+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0814018020 con 0x7f08181017a0 2024-04-24T23:09:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.767+0000 7f080dffb640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08000a2b90 0x7f08000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.767+0000 7f080ffff640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08000a2b90 0x7f08000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.767+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f08140ce7d0 con 0x7f08181017a0 2024-04-24T23:09:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.768+0000 7f080ffff640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08000a2b90 0x7f08000a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0808006fd0 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:39.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:39.771+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f081401f020 con 0x7f08181017a0 2024-04-24T23:09:39.794 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 podman[69865]: 2024-04-24 23:09:39.422823336 +0000 UTC m=+0.018426791 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:09:39.794 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 podman[69865]: 2024-04-24 23:09:39.537570105 +0000 UTC m=+0.133173533 container create 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T23:09:39.794 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 podman[69865]: 2024-04-24 23:09:39.765326725 +0000 UTC m=+0.360930152 container init 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:09:39.795 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 podman[69865]: 2024-04-24 23:09:39.769044707 +0000 UTC m=+0.364648135 container start 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:09:40.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.006+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08180623e0 con 0x7f08181017a0 2024-04-24T23:09:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.006+0000 7f080dffb640 1 -- 172.21.15.33:0/6132794 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f081400d040 con 0x7f08181017a0 2024-04-24T23:09:40.008 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:40.008 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:09:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08000a2b90 msgr2=0x7f08000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08000a2b90 0x7f08000a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0808006fd0 tx=0x7f0808008040 comp rx=0 tx=0).stop 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 msgr2=0x7f0818197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 0x7f0818197be0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f081400b450 tx=0x7f081400b910 comp rx=0 tx=0).stop 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 shutdown_connections 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08000a2b90 0x7f08000a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0818103350 0x7f081819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0818102990 0x7f0818198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 --2- 172.21.15.33:0/6132794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08181017a0 0x7f0818197be0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 >> 172.21.15.33:0/6132794 conn(0x7f08180fccf0 msgr2=0x7f0818105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 shutdown_connections 2024-04-24T23:09:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:40.009+0000 7f081ea8c640 1 -- 172.21.15.33:0/6132794 wait complete. 2024-04-24T23:09:40.205 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 ceph-mds[69924]: -- 172.21.15.33:0/3043396844 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563491a81a40 con 0x563491a54c00 2024-04-24T23:09:40.205 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j[69880]: starting mds.j at 2024-04-24T23:09:40.205 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 bash[69865]: 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed 2024-04-24T23:09:40.205 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:09:39 smithi033 systemd[1]: Started Ceph mds.j for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:41.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:41.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: Deploying daemon mds.d on smithi033 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/6132794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.33:6834/1264288095,v1:172.21.15.33:6835/1264288095] up:boot 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: fsmap 3 up:standby 2024-04-24T23:09:41.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:40 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:09:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:41.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:41.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: Deploying daemon mds.d on smithi033 2024-04-24T23:09:41.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/6132794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:41.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.33:6834/1264288095,v1:172.21.15.33:6835/1264288095] up:boot 2024-04-24T23:09:41.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: fsmap 3 up:standby 2024-04-24T23:09:41.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:40 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: Deploying daemon mds.d on smithi033 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/6132794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.33:6834/1264288095,v1:172.21.15.33:6835/1264288095] up:boot 2024-04-24T23:09:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: fsmap 3 up:standby 2024-04-24T23:09:41.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:40 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:09:42.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2626267683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 msgr2=0x7fdde41045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2626267683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde41045f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009900 tx=0x7fdddc02a410 comp rx=0 tx=0).stop 2024-04-24T23:09:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2626267683 shutdown_connections 2024-04-24T23:09:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2626267683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4107460 0x7fdde41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2626267683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde4104b30 0x7fdde4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2626267683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde41045f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2626267683 >> 172.21.15.33:0/2626267683 conn(0x7fdde40fbc30 msgr2=0x7fdde40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2626267683 shutdown_connections 2024-04-24T23:09:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.262+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2626267683 wait complete. 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 Processor -- start 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 -- start start 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde4101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4104b30 0x7fdde41001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde4107460 0x7fdde41006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410d480 con 0x7fdde4102200 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410d300 con 0x7fdde4104b30 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde8c7d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410d600 con 0x7fdde4107460 2024-04-24T23:09:42.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde4101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde4101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56592/0 (socket says 172.21.15.33:56592) 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde259c640 1 -- 172.21.15.33:0/2684325485 learned_addr learned my addr 172.21.15.33:0/2684325485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.263+0000 7fdde2d9d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde4107460 0x7fdde41006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde1d9b640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4104b30 0x7fdde41001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde259c640 1 -- 172.21.15.33:0/2684325485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde4107460 msgr2=0x7fdde41006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde259c640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde4107460 0x7fdde41006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde259c640 1 -- 172.21.15.33:0/2684325485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4104b30 msgr2=0x7fdde41001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde259c640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4104b30 0x7fdde41001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde259c640 1 -- 172.21.15.33:0/2684325485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde4100f80 con 0x7fdde4102200 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde259c640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde4101b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdddc031440 tx=0x7fdddc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc03f030 con 0x7fdde4102200 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdddc003ce0 con 0x7fdde4102200 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde41a1be0 con 0x7fdde4102200 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc035070 con 0x7fdde4102200 2024-04-24T23:09:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.264+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a2130 con 0x7fdde4102200 2024-04-24T23:09:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.265+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde40fd720 con 0x7fdde4102200 2024-04-24T23:09:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.267+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdddc042030 con 0x7fdde4102200 2024-04-24T23:09:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.267+0000 7fddd37fe640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fddcc0a2b90 0x7fddcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.267+0000 7fdde1d9b640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fddcc0a2b90 0x7fddcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.267+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdddc0ef220 con 0x7fdde4102200 2024-04-24T23:09:42.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.268+0000 7fdde1d9b640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fddcc0a2b90 0x7fddcc0a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005fd0 tx=0x7fddd8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.271+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdddc0bc600 con 0x7fdde4102200 2024-04-24T23:09:42.455 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 podman[70256]: 2024-04-24 23:09:42.326514955 +0000 UTC m=+0.019194315 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:09:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.500+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdde402d170 con 0x7fdde4102200 2024-04-24T23:09:42.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.501+0000 7fddd37fe640 1 -- 172.21.15.33:0/2684325485 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fdddc0bbf00 con 0x7fdde4102200 2024-04-24T23:09:42.503 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:42.503 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:09:42.503 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fddcc0a2b90 msgr2=0x7fddcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fddcc0a2b90 0x7fddcc0a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005fd0 tx=0x7fddd8005850 comp rx=0 tx=0).stop 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 msgr2=0x7fdde4101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde4101b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdddc031440 tx=0x7fdddc032ce0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 shutdown_connections 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fddcc0a2b90 0x7fddcc0a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde4107460 0x7fdde41006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde4104b30 0x7fdde41001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 --2- 172.21.15.33:0/2684325485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde4102200 0x7fdde4101b70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 >> 172.21.15.33:0/2684325485 conn(0x7fdde40fbc30 msgr2=0x7fdde4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 shutdown_connections 2024-04-24T23:09:42.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:42.503+0000 7fdde8c7d640 1 -- 172.21.15.33:0/2684325485 wait complete. 2024-04-24T23:09:42.797 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 podman[70256]: 2024-04-24 23:09:42.460480925 +0000 UTC m=+0.153160270 container create b1510d0314f41aede758349e95e6768ee54906f1d61025df5fc0e1d94397403c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T23:09:43.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:42 smithi117 ceph-mon[32529]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:43.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:42 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2684325485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:43.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:42 smithi033 ceph-mon[29201]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:43.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:42 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2684325485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:43.205 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 podman[70256]: 2024-04-24 23:09:42.797344702 +0000 UTC m=+0.490024049 container init b1510d0314f41aede758349e95e6768ee54906f1d61025df5fc0e1d94397403c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T23:09:43.205 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 podman[70256]: 2024-04-24 23:09:42.800111473 +0000 UTC m=+0.492790820 container start b1510d0314f41aede758349e95e6768ee54906f1d61025df5fc0e1d94397403c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:09:43.205 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 ceph-mds[70278]: -- 172.21.15.33:0/3286696617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562de628ba40 con 0x562de625e800 2024-04-24T23:09:43.206 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d[70274]: starting mds.d at 2024-04-24T23:09:43.206 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 bash[70256]: b1510d0314f41aede758349e95e6768ee54906f1d61025df5fc0e1d94397403c 2024-04-24T23:09:43.206 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:09:42 smithi033 systemd[1]: Started Ceph mds.d for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:42 smithi038 ceph-mon[32450]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:42 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2684325485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.33:6836/1577112159,v1:172.21.15.33:6837/1577112159] up:boot 2024-04-24T23:09:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: fsmap 4 up:standby 2024-04-24T23:09:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:09:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:44.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:44.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:43 smithi038 ceph-mon[32450]: Deploying daemon mds.e on smithi038 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.33:6836/1577112159,v1:172.21.15.33:6837/1577112159] up:boot 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: fsmap 4 up:standby 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:43 smithi033 ceph-mon[29201]: Deploying daemon mds.e on smithi038 2024-04-24T23:09:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.33:6836/1577112159,v1:172.21.15.33:6837/1577112159] up:boot 2024-04-24T23:09:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: fsmap 4 up:standby 2024-04-24T23:09:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:09:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:43 smithi117 ceph-mon[32529]: Deploying daemon mds.e on smithi038 2024-04-24T23:09:44.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 -- 172.21.15.33:0/1703574069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434102250 msgr2=0x7ff434108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 --2- 172.21.15.33:0/1703574069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434102250 0x7ff434108ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009ca0 tx=0x7ff42c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 -- 172.21.15.33:0/1703574069 shutdown_connections 2024-04-24T23:09:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 --2- 172.21.15.33:0/1703574069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434102250 0x7ff434108ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 --2- 172.21.15.33:0/1703574069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434101890 0x7ff434101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 --2- 172.21.15.33:0/1703574069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4341006a0 0x7ff434100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 -- 172.21.15.33:0/1703574069 >> 172.21.15.33:0/1703574069 conn(0x7ff4340fbc30 msgr2=0x7ff4340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 -- 172.21.15.33:0/1703574069 shutdown_connections 2024-04-24T23:09:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 -- 172.21.15.33:0/1703574069 wait complete. 2024-04-24T23:09:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.633+0000 7ff43b75a640 1 Processor -- start 2024-04-24T23:09:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 -- start start 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4341006a0 0x7ff434197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 0x7ff434197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff434102250 0x7ff43419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d440 con 0x7ff434101890 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d2c0 con 0x7ff434102250 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff43b75a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d5c0 con 0x7ff4341006a0 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 0x7ff434197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 0x7ff434197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56632/0 (socket says 172.21.15.33:56632) 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 -- 172.21.15.33:0/3723297583 learned_addr learned my addr 172.21.15.33:0/3723297583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff4394f6640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4341006a0 0x7ff434197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff439cf7640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff434102250 0x7ff43419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 -- 172.21.15.33:0/3723297583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4341006a0 msgr2=0x7ff434197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4341006a0 0x7ff434197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 -- 172.21.15.33:0/3723297583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff434102250 msgr2=0x7ff43419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff434102250 0x7ff43419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 -- 172.21.15.33:0/3723297583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43419f6d0 con 0x7ff434101890 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff439cf7640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff434102250 0x7ff43419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.634+0000 7ff438cf5640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 0x7ff434197ff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff43000c890 tx=0x7ff43000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.635+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430004220 con 0x7ff434101890 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.635+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4300043c0 con 0x7ff434101890 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.635+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300049e0 con 0x7ff434101890 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.635+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43419f9c0 con 0x7ff434101890 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.635+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43419fec0 con 0x7ff434101890 2024-04-24T23:09:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.636+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff434107960 con 0x7ff434101890 2024-04-24T23:09:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.637+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4300079b0 con 0x7ff434101890 2024-04-24T23:09:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.637+0000 7ff42a7fc640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4200a2c10 0x7ff4200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.637+0000 7ff4394f6640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4200a2c10 0x7ff4200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.637+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff4300ceb80 con 0x7ff434101890 2024-04-24T23:09:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.638+0000 7ff4394f6640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4200a2c10 0x7ff4200a50d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff4341016f0 tx=0x7ff42400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:44.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.642+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff430014030 con 0x7ff434101890 2024-04-24T23:09:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.865+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff43402d170 con 0x7ff434101890 2024-04-24T23:09:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.865+0000 7ff42a7fc640 1 -- 172.21.15.33:0/3723297583 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7ff43009bf80 con 0x7ff434101890 2024-04-24T23:09:44.867 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:44.868 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:09:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-04-24T23:09:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.867+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4200a2c10 msgr2=0x7ff4200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.867+0000 7ff43b75a640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4200a2c10 0x7ff4200a50d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff4341016f0 tx=0x7ff42400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.867+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 msgr2=0x7ff434197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.867+0000 7ff43b75a640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 0x7ff434197ff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff43000c890 tx=0x7ff43000cd50 comp rx=0 tx=0).stop 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 shutdown_connections 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4200a2c10 0x7ff4200a50d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff434102250 0x7ff43419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff434101890 0x7ff434197ff0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 --2- 172.21.15.33:0/3723297583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4341006a0 0x7ff434197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 >> 172.21.15.33:0/3723297583 conn(0x7ff4340fbc30 msgr2=0x7ff434104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 shutdown_connections 2024-04-24T23:09:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:44.868+0000 7ff43b75a640 1 -- 172.21.15.33:0/3723297583 wait complete. 2024-04-24T23:09:45.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:45 smithi033 ceph-mon[29201]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:45.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:45 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3723297583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:45.309 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:45 smithi038 ceph-mon[32450]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:45.309 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:45 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3723297583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:45.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:45 smithi117 ceph-mon[32529]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:45 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3723297583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:45.671 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 podman[62211]: 2024-04-24 23:09:45.557156982 +0000 UTC m=+0.143005729 container create e93cd3d3bad9bcaec577a81253ebed4716ff8617e2f1fd80b7228d84de2da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T23:09:46.171 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 podman[62211]: 2024-04-24 23:09:45.801631958 +0000 UTC m=+0.387480676 container init e93cd3d3bad9bcaec577a81253ebed4716ff8617e2f1fd80b7228d84de2da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-24T23:09:46.171 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 podman[62211]: 2024-04-24 23:09:45.80438767 +0000 UTC m=+0.390236375 container start e93cd3d3bad9bcaec577a81253ebed4716ff8617e2f1fd80b7228d84de2da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T23:09:46.171 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 ceph-mds[62230]: -- 172.21.15.38:0/2120992047 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613dbe13a40 con 0x5613dbde6c00 2024-04-24T23:09:46.171 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e[62226]: starting mds.e at 2024-04-24T23:09:46.171 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 bash[62211]: e93cd3d3bad9bcaec577a81253ebed4716ff8617e2f1fd80b7228d84de2da197 2024-04-24T23:09:46.171 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:09:45 smithi038 systemd[1]: Started Ceph mds.e for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:46.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/870284793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 msgr2=0x7f8bbc100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/870284793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc100dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009900 tx=0x7f8bb002a410 comp rx=0 tx=0).stop 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/870284793 shutdown_connections 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/870284793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 0x7f8bbc101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/870284793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc100dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/870284793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bbc100010 0x7f8bbc100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/870284793 >> 172.21.15.33:0/870284793 conn(0x7f8bbc0fbc30 msgr2=0x7f8bbc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/870284793 shutdown_connections 2024-04-24T23:09:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.997+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/870284793 wait complete. 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 Processor -- start 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 -- start start 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bbc100010 0x7f8bbc199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 0x7f8bbc1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10f5e0 con 0x7f8bbc100970 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10f460 con 0x7f8bbc100010 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc3a9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10f760 con 0x7f8bbc101310 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38266/0 (socket says 172.21.15.33:38266) 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc203b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 0x7f8bbc1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 -- 172.21.15.33:0/3566105927 learned_addr learned my addr 172.21.15.33:0/3566105927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc183a640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bbc100010 0x7f8bbc199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 -- 172.21.15.33:0/3566105927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 msgr2=0x7f8bbc1a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 0x7f8bbc1a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 -- 172.21.15.33:0/3566105927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bbc100010 msgr2=0x7f8bbc199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bbc100010 0x7f8bbc199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc1039640 1 -- 172.21.15.33:0/3566105927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc1a18b0 con 0x7f8bbc100970 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.998+0000 7f8bc203b640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 0x7f8bbc1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.999+0000 7f8bc1039640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc19a220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009230 tx=0x7f8bb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.999+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb003f030 con 0x7f8bbc100970 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.999+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb0003ce0 con 0x7f8bbc100970 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.999+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc1a1b40 con 0x7f8bbc100970 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.999+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0035070 con 0x7f8bbc100970 2024-04-24T23:09:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:46.999+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc1a2030 con 0x7f8bbc100970 2024-04-24T23:09:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.000+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bbc10ad60 con 0x7f8bbc100970 2024-04-24T23:09:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.001+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8bb0042030 con 0x7f8bbc100970 2024-04-24T23:09:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.002+0000 7f8baeffd640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ba80a2b90 0x7f8ba80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.002+0000 7f8bc183a640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ba80a2b90 0x7f8ba80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.002+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8bb00ef0c0 con 0x7f8bbc100970 2024-04-24T23:09:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.002+0000 7f8bc183a640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ba80a2b90 0x7f8ba80a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc19abb0 tx=0x7f8bb8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:47.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.006+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8bb00bc4c0 con 0x7f8bbc100970 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:47.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: Deploying daemon mds.f on smithi117 2024-04-24T23:09:47.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6834/1005158826,v1:172.21.15.38:6835/1005158826] up:boot 2024-04-24T23:09:47.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: fsmap 5 up:standby 2024-04-24T23:09:47.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:46 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:09:47.229 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:47.229 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: Deploying daemon mds.f on smithi117 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6834/1005158826,v1:172.21.15.38:6835/1005158826] up:boot 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: fsmap 5 up:standby 2024-04-24T23:09:47.230 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:46 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:09:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.233+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bbc02d370 con 0x7f8bbc100970 2024-04-24T23:09:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.234+0000 7f8baeffd640 1 -- 172.21.15.33:0/3566105927 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f8bb00bbdc0 con 0x7f8bbc100970 2024-04-24T23:09:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:47.236 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:09:47.236 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 6 2024-04-24T23:09:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ba80a2b90 msgr2=0x7f8ba80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ba80a2b90 0x7f8ba80a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc19abb0 tx=0x7f8bb8005f90 comp rx=0 tx=0).stop 2024-04-24T23:09:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 msgr2=0x7f8bbc19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc19a220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009230 tx=0x7f8bb00042f0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 shutdown_connections 2024-04-24T23:09:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ba80a2b90 0x7f8ba80a5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc101310 0x7f8bbc1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bbc100970 0x7f8bbc19a220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 --2- 172.21.15.33:0/3566105927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bbc100010 0x7f8bbc199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:47.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 >> 172.21.15.33:0/3566105927 conn(0x7f8bbc0fbc30 msgr2=0x7f8bbc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:47.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 shutdown_connections 2024-04-24T23:09:47.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:47.236+0000 7f8bc3a9e640 1 -- 172.21.15.33:0/3566105927 wait complete. 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: Deploying daemon mds.f on smithi117 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6834/1005158826,v1:172.21.15.38:6835/1005158826] up:boot 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: fsmap 5 up:standby 2024-04-24T23:09:47.422 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:46 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:09:48.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:47 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3566105927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:48.176 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 systemd[1]: Starting Ceph mds.f for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:09:48.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:47 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3566105927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:48.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:47 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3566105927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:48.603 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 podman[62758]: 2024-04-24 23:09:48.206248532 +0000 UTC m=+0.017497944 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:09:48.603 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 podman[62758]: 2024-04-24 23:09:48.316119908 +0000 UTC m=+0.127369310 container create 0e304fca42488cc42e5c32bee56e60dd1f2c82b56f7839832407db47c7106b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:09:48.604 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 podman[62758]: 2024-04-24 23:09:48.60258153 +0000 UTC m=+0.413830929 container init 0e304fca42488cc42e5c32bee56e60dd1f2c82b56f7839832407db47c7106b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T23:09:48.926 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 podman[62758]: 2024-04-24 23:09:48.605213694 +0000 UTC m=+0.416463096 container start 0e304fca42488cc42e5c32bee56e60dd1f2c82b56f7839832407db47c7106b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T23:09:48.926 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mds[62778]: -- 172.21.15.117:0/3243479591 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f77cb51a40 con 0x55f77cb24c00 2024-04-24T23:09:48.926 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: starting mds.f at 2024-04-24T23:09:48.926 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 bash[62758]: 0e304fca42488cc42e5c32bee56e60dd1f2c82b56f7839832407db47c7106b9d 2024-04-24T23:09:48.927 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:09:48 smithi117 systemd[1]: Started Ceph mds.f for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:49.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:49.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:49.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:49.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:49.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:48 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 -- 172.21.15.33:0/1192715371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c102230 msgr2=0x7fed2c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 --2- 172.21.15.33:0/1192715371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c102230 0x7fed2c108ac0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed24009ca0 tx=0x7fed2402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 -- 172.21.15.33:0/1192715371 shutdown_connections 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 --2- 172.21.15.33:0/1192715371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c102230 0x7fed2c108ac0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 --2- 172.21.15.33:0/1192715371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101870 0x7fed2c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 --2- 172.21.15.33:0/1192715371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c100680 0x7fed2c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 -- 172.21.15.33:0/1192715371 >> 172.21.15.33:0/1192715371 conn(0x7fed2c0fbbf0 msgr2=0x7fed2c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 -- 172.21.15.33:0/1192715371 shutdown_connections 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.365+0000 7fed31272640 1 -- 172.21.15.33:0/1192715371 wait complete. 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 Processor -- start 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 -- start start 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c100680 0x7fed2c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 0x7fed2c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c102230 0x7fed2c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10c330 con 0x7fed2c101870 2024-04-24T23:09:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10c1b0 con 0x7fed2c102230 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.366+0000 7fed31272640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10c4b0 con 0x7fed2c100680 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 0x7fed2c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c100680 0x7fed2c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 0x7fed2c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38280/0 (socket says 172.21.15.33:38280) 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 -- 172.21.15.33:0/887763259 learned_addr learned my addr 172.21.15.33:0/887763259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2b7fe640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c102230 0x7fed2c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 -- 172.21.15.33:0/887763259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c100680 msgr2=0x7fed2c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c100680 0x7fed2c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 -- 172.21.15.33:0/887763259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c102230 msgr2=0x7fed2c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c102230 0x7fed2c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 -- 172.21.15.33:0/887763259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a1870 con 0x7fed2c101870 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2affd640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c100680 0x7fed2c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed2a7fc640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 0x7fed2c19a1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fed2000d950 tx=0x7fed2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20017070 con 0x7fed2c101870 2024-04-24T23:09:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed20009c30 con 0x7fed2c101870 2024-04-24T23:09:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a1b60 con 0x7fed2c101870 2024-04-24T23:09:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2000f6b0 con 0x7fed2c101870 2024-04-24T23:09:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.367+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a2100 con 0x7fed2c101870 2024-04-24T23:09:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.368+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c1a22d0 con 0x7fed2c101870 2024-04-24T23:09:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.370+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed200109d0 con 0x7fed2c101870 2024-04-24T23:09:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.370+0000 7fed13fff640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed140a2c10 0x7fed140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.370+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fed200ce7f0 con 0x7fed2c101870 2024-04-24T23:09:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.370+0000 7fed2affd640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed140a2c10 0x7fed140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.371+0000 7fed2affd640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed140a2c10 0x7fed140a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed2c19b130 tx=0x7fed1c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.374+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed20009de0 con 0x7fed2c101870 2024-04-24T23:09:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:48 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:48 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.599+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed2c1a2660 con 0x7fed2c101870 2024-04-24T23:09:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.600+0000 7fed13fff640 1 -- 172.21.15.33:0/887763259 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fed2009bc70 con 0x7fed2c101870 2024-04-24T23:09:49.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:49.602 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T23:09:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed140a2c10 msgr2=0x7fed140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed140a2c10 0x7fed140a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed2c19b130 tx=0x7fed1c00b390 comp rx=0 tx=0).stop 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 msgr2=0x7fed2c19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 0x7fed2c19a1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fed2000d950 tx=0x7fed2000de10 comp rx=0 tx=0).stop 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 shutdown_connections 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed140a2c10 0x7fed140a50d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c102230 0x7fed2c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed2c101870 0x7fed2c19a1a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 --2- 172.21.15.33:0/887763259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c100680 0x7fed2c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 >> 172.21.15.33:0/887763259 conn(0x7fed2c0fbbf0 msgr2=0x7fed2c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 shutdown_connections 2024-04-24T23:09:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:49.602+0000 7fed31272640 1 -- 172.21.15.33:0/887763259 wait complete. 2024-04-24T23:09:50.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:49 smithi038 ceph-mon[32450]: Deploying daemon mds.c on smithi117 2024-04-24T23:09:50.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:49 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6834/221661334,v1:172.21.15.117:6835/221661334] up:boot 2024-04-24T23:09:50.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:49 smithi038 ceph-mon[32450]: fsmap 6 up:standby 2024-04-24T23:09:50.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:49 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:09:50.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:49 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/887763259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:50.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:49 smithi117 ceph-mon[32529]: Deploying daemon mds.c on smithi117 2024-04-24T23:09:50.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:49 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6834/221661334,v1:172.21.15.117:6835/221661334] up:boot 2024-04-24T23:09:50.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:49 smithi117 ceph-mon[32529]: fsmap 6 up:standby 2024-04-24T23:09:50.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:49 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:09:50.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:49 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/887763259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:50.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:49 smithi033 ceph-mon[29201]: Deploying daemon mds.c on smithi117 2024-04-24T23:09:50.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:49 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6834/221661334,v1:172.21.15.117:6835/221661334] up:boot 2024-04-24T23:09:50.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:49 smithi033 ceph-mon[29201]: fsmap 6 up:standby 2024-04-24T23:09:50.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:49 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:09:50.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:49 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/887763259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:51.130 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:50 smithi117 ceph-mon[32529]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:50 smithi038 ceph-mon[32450]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:51.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:50 smithi033 ceph-mon[29201]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:51.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:51.676 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:09:51 smithi117 systemd[1]: Started Ceph mds.c for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 -- 172.21.15.33:0/3623843362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 msgr2=0x7fcf68101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 --2- 172.21.15.33:0/3623843362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68101cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009900 tx=0x7fcf5002a410 comp rx=0 tx=0).stop 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 -- 172.21.15.33:0/3623843362 shutdown_connections 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 --2- 172.21.15.33:0/3623843362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf68102230 0x7fcf68108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 --2- 172.21.15.33:0/3623843362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68101cf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 --2- 172.21.15.33:0/3623843362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 0x7fcf68100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 -- 172.21.15.33:0/3623843362 >> 172.21.15.33:0/3623843362 conn(0x7fcf680fbbf0 msgr2=0x7fcf680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 -- 172.21.15.33:0/3623843362 shutdown_connections 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 -- 172.21.15.33:0/3623843362 wait complete. 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.728+0000 7fcf6d109640 1 Processor -- start 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 -- start start 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 0x7fcf681969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf68102230 0x7fcf6819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810c300 con 0x7fcf68101870 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810c180 con 0x7fcf68102230 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6d109640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810c480 con 0x7fcf68100680 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38296/0 (socket says 172.21.15.33:38296) 2024-04-24T23:09:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 -- 172.21.15.33:0/179314577 learned_addr learned my addr 172.21.15.33:0/179314577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf6759e640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf68102230 0x7fcf6819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf66d9d640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 0x7fcf681969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 -- 172.21.15.33:0/179314577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 msgr2=0x7fcf681969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 0x7fcf681969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 -- 172.21.15.33:0/179314577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf68102230 msgr2=0x7fcf6819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf68102230 0x7fcf6819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 -- 172.21.15.33:0/179314577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6819e5f0 con 0x7fcf68101870 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf66d9d640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 0x7fcf681969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf5ffff640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68196f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf50004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.729+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5003f030 con 0x7fcf68101870 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.730+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6819e880 con 0x7fcf68101870 2024-04-24T23:09:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.730+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50003ce0 con 0x7fcf68101870 2024-04-24T23:09:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.730+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50035070 con 0x7fcf68101870 2024-04-24T23:09:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.730+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6819ed70 con 0x7fcf68101870 2024-04-24T23:09:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.731+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68107a20 con 0x7fcf68101870 2024-04-24T23:09:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.732+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf50042030 con 0x7fcf68101870 2024-04-24T23:09:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.732+0000 7fcf64d99640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf540a2b90 0x7fcf540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.733+0000 7fcf66d9d640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf540a2b90 0x7fcf540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.733+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcf500ef5c0 con 0x7fcf68101870 2024-04-24T23:09:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.733+0000 7fcf66d9d640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf540a2b90 0x7fcf540a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcf580058c0 tx=0x7fcf58005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.737+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcf500bc9c0 con 0x7fcf68101870 2024-04-24T23:09:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.963+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf6802d170 con 0x7fcf68101870 2024-04-24T23:09:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.964+0000 7fcf64d99640 1 -- 172.21.15.33:0/179314577 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fcf500bc2c0 con 0x7fcf68101870 2024-04-24T23:09:51.964 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:51.965 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T23:09:51.965 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-24T23:09:51.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.965+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf540a2b90 msgr2=0x7fcf540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.965+0000 7fcf6d109640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf540a2b90 0x7fcf540a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcf580058c0 tx=0x7fcf58005850 comp rx=0 tx=0).stop 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 msgr2=0x7fcf68196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68196f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf50004360 comp rx=0 tx=0).stop 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 shutdown_connections 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf540a2b90 0x7fcf540a5050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf68102230 0x7fcf6819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf68101870 0x7fcf68196f10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 --2- 172.21.15.33:0/179314577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf68100680 0x7fcf681969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 >> 172.21.15.33:0/179314577 conn(0x7fcf680fbbf0 msgr2=0x7fcf68104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 shutdown_connections 2024-04-24T23:09:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:51.966+0000 7fcf6d109640 1 -- 172.21.15.33:0/179314577 wait complete. 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: Deploying daemon mds.h on smithi038 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/179314577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] up:boot 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: fsmap 7 up:standby 2024-04-24T23:09:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:09:52.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:52.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:52.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:52.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: Deploying daemon mds.h on smithi038 2024-04-24T23:09:52.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/179314577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:52.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] up:boot 2024-04-24T23:09:52.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: fsmap 7 up:standby 2024-04-24T23:09:52.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:09:52.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: Deploying daemon mds.h on smithi038 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/179314577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] up:boot 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: fsmap 7 up:standby 2024-04-24T23:09:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:09:53.893 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:53 smithi038 ceph-mon[32450]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:53 smithi117 ceph-mon[32529]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:53.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:53 smithi033 ceph-mon[29201]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:53.978 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:54.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- 172.21.15.33:0/831840313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 msgr2=0x7f39381045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- 172.21.15.33:0/831840313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f39381045f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402a410 comp rx=0 tx=0).stop 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- 172.21.15.33:0/831840313 shutdown_connections 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- 172.21.15.33:0/831840313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3938107460 0x7f39381098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- 172.21.15.33:0/831840313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938104b30 0x7f3938106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- 172.21.15.33:0/831840313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f39381045f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- 172.21.15.33:0/831840313 >> 172.21.15.33:0/831840313 conn(0x7f39380fbbf0 msgr2=0x7f39380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- 172.21.15.33:0/831840313 shutdown_connections 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- 172.21.15.33:0/831840313 wait complete. 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 Processor -- start 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- start start 2024-04-24T23:09:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f3938199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3938104b30 0x7f393819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938107460 0x7f39381a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f393810d420 con 0x7f3938102200 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f393810d2a0 con 0x7f3938107460 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.092+0000 7f393fc29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f393810d5a0 con 0x7f3938104b30 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f3938199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f3938199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38314/0 (socket says 172.21.15.33:38314) 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 -- 172.21.15.33:0/3293328519 learned_addr learned my addr 172.21.15.33:0/3293328519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393e1c6640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938107460 0x7f39381a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d1c4640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3938104b30 0x7f393819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 -- 172.21.15.33:0/3293328519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3938104b30 msgr2=0x7f393819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3938104b30 0x7f393819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 -- 172.21.15.33:0/3293328519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938107460 msgr2=0x7f39381a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938107460 0x7f39381a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 -- 172.21.15.33:0/3293328519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39381a18f0 con 0x7f3938102200 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393d9c5640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f3938199cb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3934009230 tx=0x7f3934004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393403f030 con 0x7f3938102200 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3934003ce0 con 0x7f3938102200 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39381a1b80 con 0x7f3938102200 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934035070 con 0x7f3938102200 2024-04-24T23:09:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.093+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381a2070 con 0x7f3938102200 2024-04-24T23:09:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.094+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3938108fb0 con 0x7f3938102200 2024-04-24T23:09:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.095+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3934042030 con 0x7f3938102200 2024-04-24T23:09:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.096+0000 7f392effd640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39240a2b90 0x7f39240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.096+0000 7f393d1c4640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39240a2b90 0x7f39240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.096+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f39340ef590 con 0x7f3938102200 2024-04-24T23:09:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.097+0000 7f393d1c4640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39240a2b90 0x7f39240a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3928005d00 tx=0x7f3928005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.101+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39340bc990 con 0x7f3938102200 2024-04-24T23:09:54.148 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 podman[62519]: 2024-04-24 23:09:54.035820302 +0000 UTC m=+0.018542620 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:09:54.148 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 podman[62519]: 2024-04-24 23:09:54.147749595 +0000 UTC m=+0.130471912 container create 3b9ff9eac407c69857d6c45d6227af847631e3707b586ae994d574bdecb3b5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:09:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.325+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f393802d170 con 0x7f3938102200 2024-04-24T23:09:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.326+0000 7f392effd640 1 -- 172.21.15.33:0/3293328519 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f39340bc290 con 0x7f3938102200 2024-04-24T23:09:54.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:54.327 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:09:54.327 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39240a2b90 msgr2=0x7f39240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39240a2b90 0x7f39240a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3928005d00 tx=0x7f3928005c50 comp rx=0 tx=0).stop 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 msgr2=0x7f3938199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f3938199cb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3934009230 tx=0x7f3934004360 comp rx=0 tx=0).stop 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 shutdown_connections 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39240a2b90 0x7f39240a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3938107460 0x7f39381a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3938104b30 0x7f393819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 --2- 172.21.15.33:0/3293328519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3938102200 0x7f3938199cb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 >> 172.21.15.33:0/3293328519 conn(0x7f39380fbbf0 msgr2=0x7f39381018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 shutdown_connections 2024-04-24T23:09:54.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:54.328+0000 7f393fc29640 1 -- 172.21.15.33:0/3293328519 wait complete. 2024-04-24T23:09:54.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:54 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3293328519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:54.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 podman[62519]: 2024-04-24 23:09:54.409230372 +0000 UTC m=+0.391952690 container init 3b9ff9eac407c69857d6c45d6227af847631e3707b586ae994d574bdecb3b5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 podman[62519]: 2024-04-24 23:09:54.412130864 +0000 UTC m=+0.394853184 container start 3b9ff9eac407c69857d6c45d6227af847631e3707b586ae994d574bdecb3b5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 ceph-mds[62538]: -- 172.21.15.38:0/3741691520 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dae3d45a40 con 0x55dae3d19000 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h[62534]: starting mds.h at 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 bash[62519]: 3b9ff9eac407c69857d6c45d6227af847631e3707b586ae994d574bdecb3b5ae 2024-04-24T23:09:54.671 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:09:54 smithi038 systemd[1]: Started Ceph mds.h for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:09:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:54 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3293328519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:54.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:54 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3293328519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:54.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:54.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:55.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6836/287276222,v1:172.21.15.38:6837/287276222] up:boot 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: fsmap 8 up:standby 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:55 smithi038 ceph-mon[32450]: Deploying daemon mds.i on smithi117 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6836/287276222,v1:172.21.15.38:6837/287276222] up:boot 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: fsmap 8 up:standby 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:55.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:55 smithi117 ceph-mon[32529]: Deploying daemon mds.i on smithi117 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6836/287276222,v1:172.21.15.38:6837/287276222] up:boot 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: fsmap 8 up:standby 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:55 smithi033 ceph-mon[29201]: Deploying daemon mds.i on smithi117 2024-04-24T23:09:56.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 -- 172.21.15.33:0/2499636634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4101890 msgr2=0x7fa3c4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2499636634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4101890 0x7fa3c4101d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009900 tx=0x7fa3ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 -- 172.21.15.33:0/2499636634 shutdown_connections 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2499636634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4102250 0x7fa3c4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2499636634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4101890 0x7fa3c4101d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2499636634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 -- 172.21.15.33:0/2499636634 >> 172.21.15.33:0/2499636634 conn(0x7fa3c40fbc30 msgr2=0x7fa3c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 -- 172.21.15.33:0/2499636634 shutdown_connections 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.456+0000 7fa3caf14640 1 -- 172.21.15.33:0/2499636634 wait complete. 2024-04-24T23:09:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 Processor -- start 2024-04-24T23:09:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 -- start start 2024-04-24T23:09:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4101890 0x7fa3c4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 0x7fa3c419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410c320 con 0x7fa3c4102250 2024-04-24T23:09:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410c1a0 con 0x7fa3c4101890 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3caf14640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410c4a0 con 0x7fa3c41006a0 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 0x7fa3c419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c8cb0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 0x7fa3c419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40968/0 (socket says 172.21.15.33:40968) 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 -- 172.21.15.33:0/2794011264 learned_addr learned my addr 172.21.15.33:0/2794011264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c8cb0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46122/0 (socket says 172.21.15.33:46122) 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c3fff640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4101890 0x7fa3c4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 -- 172.21.15.33:0/2794011264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 msgr2=0x7fa3c4197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 -- 172.21.15.33:0/2794011264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4101890 msgr2=0x7fa3c4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4101890 0x7fa3c4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c94b1640 1 -- 172.21.15.33:0/2794011264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419f670 con 0x7fa3c4102250 2024-04-24T23:09:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c3fff640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4101890 0x7fa3c4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.457+0000 7fa3c8cb0640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:09:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.458+0000 7fa3c94b1640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 0x7fa3c419f000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00cce0 tx=0x7fa3bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.458+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00f450 con 0x7fa3c4102250 2024-04-24T23:09:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.458+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc00fad0 con 0x7fa3c4102250 2024-04-24T23:09:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.458+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc018c10 con 0x7fa3c4102250 2024-04-24T23:09:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.458+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419f960 con 0x7fa3c4102250 2024-04-24T23:09:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.458+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c419fe30 con 0x7fa3c4102250 2024-04-24T23:09:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.459+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c41a0060 con 0x7fa3c4102250 2024-04-24T23:09:56.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.460+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa3bc00f5f0 con 0x7fa3c4102250 2024-04-24T23:09:56.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.461+0000 7fa3c1ffb640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2b90 0x7fa3b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.461+0000 7fa3c8cb0640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2b90 0x7fa3b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.461+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa3bc0cf330 con 0x7fa3c4102250 2024-04-24T23:09:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.461+0000 7fa3c8cb0640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2b90 0x7fa3b00a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3c41016f0 tx=0x7fa3b4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.465+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3bc014030 con 0x7fa3c4102250 2024-04-24T23:09:56.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.688+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3c41a02d0 con 0x7fa3c4102250 2024-04-24T23:09:56.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.689+0000 7fa3c1ffb640 1 -- 172.21.15.33:0/2794011264 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7fa3bc09c690 con 0x7fa3c4102250 2024-04-24T23:09:56.690 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:56.690 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:09:56.690 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2b90 msgr2=0x7fa3b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2b90 0x7fa3b00a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3c41016f0 tx=0x7fa3b4005f90 comp rx=0 tx=0).stop 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 msgr2=0x7fa3c419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 0x7fa3c419f000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00cce0 tx=0x7fa3bc00a620 comp rx=0 tx=0).stop 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 shutdown_connections 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2b90 0x7fa3b00a5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102250 0x7fa3c419f000 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4101890 0x7fa3c4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 --2- 172.21.15.33:0/2794011264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41006a0 0x7fa3c4197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 >> 172.21.15.33:0/2794011264 conn(0x7fa3c40fbc30 msgr2=0x7fa3c40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 shutdown_connections 2024-04-24T23:09:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:56.691+0000 7fa3caf14640 1 -- 172.21.15.33:0/2794011264 wait complete. 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2794011264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2794011264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:57.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:57.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2794011264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:09:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1584779444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a010dd60 msgr2=0x7f38a0110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1584779444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a010dd60 0x7f38a0110150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3898009cd0 tx=0x7f389802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1584779444 shutdown_connections 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1584779444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a010dd60 0x7f38a0110150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1584779444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 0x7f38a0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1584779444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a0075780 0x7f38a0073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1584779444 >> 172.21.15.33:0/1584779444 conn(0x7f38a00fcd20 msgr2=0x7f38a00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1584779444 shutdown_connections 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.811+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1584779444 wait complete. 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 Processor -- start 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 -- start start 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 0x7f38a019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 0x7f38a019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a010dd60 0x7f38a01a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0113a00 con 0x7f38a0075780 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0113880 con 0x7f38a00741a0 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a7b7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0113b80 con 0x7f38a010dd60 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 0x7f38a019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 0x7f38a019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40982/0 (socket says 172.21.15.33:40982) 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 -- 172.21.15.33:0/1746357567 learned_addr learned my addr 172.21.15.33:0/1746357567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a611b640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a010dd60 0x7f38a01a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a591a640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 0x7f38a019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 -- 172.21.15.33:0/1746357567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a010dd60 msgr2=0x7f38a01a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a010dd60 0x7f38a01a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 -- 172.21.15.33:0/1746357567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 msgr2=0x7f38a019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 0x7f38a019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a5119640 1 -- 172.21.15.33:0/1746357567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a01a5c70 con 0x7f38a0075780 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.812+0000 7f38a591a640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 0x7f38a019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.813+0000 7f38a5119640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 0x7f38a019e590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f389000c890 tx=0x7f389000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.813+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890004220 con 0x7f38a0075780 2024-04-24T23:09:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.813+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38900043c0 con 0x7f38a0075780 2024-04-24T23:09:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.813+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a01a5f60 con 0x7f38a0075780 2024-04-24T23:09:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.813+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900049e0 con 0x7f38a0075780 2024-04-24T23:09:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.813+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01a6470 con 0x7f38a0075780 2024-04-24T23:09:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.814+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a0103e20 con 0x7f38a0075780 2024-04-24T23:09:58.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.815+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38900079b0 con 0x7f38a0075780 2024-04-24T23:09:58.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.816+0000 7f3896ffd640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f388c0a2c10 0x7f388c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.816+0000 7f38a591a640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f388c0a2c10 0x7f388c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.816+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f38900ceb70 con 0x7f38a0075780 2024-04-24T23:09:58.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.816+0000 7f38a591a640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f388c0a2c10 0x7f388c0a50d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f389c009b40 tx=0x7f389c005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:58.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:58.820+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3890014030 con 0x7f38a0075780 2024-04-24T23:09:58.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:58 smithi038 ceph-mon[32450]: Deploying daemon mds.g on smithi033 2024-04-24T23:09:58.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:58 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] up:boot 2024-04-24T23:09:58.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:58 smithi038 ceph-mon[32450]: fsmap 9 up:standby 2024-04-24T23:09:58.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:09:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:58 smithi117 ceph-mon[32529]: Deploying daemon mds.g on smithi033 2024-04-24T23:09:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:58 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] up:boot 2024-04-24T23:09:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:58 smithi117 ceph-mon[32529]: fsmap 9 up:standby 2024-04-24T23:09:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:09:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:58 smithi033 ceph-mon[29201]: Deploying daemon mds.g on smithi033 2024-04-24T23:09:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:58 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] up:boot 2024-04-24T23:09:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:58 smithi033 ceph-mon[29201]: fsmap 9 up:standby 2024-04-24T23:09:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:09:59.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.047+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38a002d170 con 0x7f38a0075780 2024-04-24T23:09:59.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.047+0000 7f3896ffd640 1 -- 172.21.15.33:0/1746357567 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7995 (secure 0 0 0) 0x7f389009bf70 con 0x7f38a0075780 2024-04-24T23:09:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:09:59.048 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:09:59.049 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 10 2024-04-24T23:09:59.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f388c0a2c10 msgr2=0x7f388c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:59.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f388c0a2c10 0x7f388c0a50d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f389c009b40 tx=0x7f389c005e50 comp rx=0 tx=0).stop 2024-04-24T23:09:59.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 msgr2=0x7f38a019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 0x7f38a019e590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f389000c890 tx=0x7f389000cd50 comp rx=0 tx=0).stop 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 shutdown_connections 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f388c0a2c10 0x7f388c0a50d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a010dd60 0x7f38a01a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0075780 0x7f38a019e590 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 --2- 172.21.15.33:0/1746357567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a00741a0 0x7f38a019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 >> 172.21.15.33:0/1746357567 conn(0x7f38a00fcd20 msgr2=0x7f38a0100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 shutdown_connections 2024-04-24T23:09:59.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:09:59.049+0000 7f38a7b7e640 1 -- 172.21.15.33:0/1746357567 wait complete. 2024-04-24T23:09:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:59 smithi033 ceph-mon[29201]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:09:59 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1746357567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:59.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:59 smithi038 ceph-mon[32450]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:59.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:09:59 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1746357567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:09:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:59 smithi117 ceph-mon[32529]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:09:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:09:59 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1746357567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:00.455 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:10:00 smithi033 systemd[1]: Started Ceph mds.g for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: overall HEALTH_OK 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: overall HEALTH_OK 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: overall HEALTH_OK 2024-04-24T23:10:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:00.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:00.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:01.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.168+0000 7f09fcb34640 1 -- 172.21.15.33:0/1715535545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8107460 msgr2=0x7f09f81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.168+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1715535545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8107460 0x7f09f81098b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009ca0 tx=0x7f09f002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.168+0000 7f09fcb34640 1 -- 172.21.15.33:0/1715535545 shutdown_connections 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.168+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1715535545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8107460 0x7f09f81098b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.168+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1715535545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09f8104b30 0x7f09f8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1715535545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f8102200 0x7f09f81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- 172.21.15.33:0/1715535545 >> 172.21.15.33:0/1715535545 conn(0x7f09f80fbbf0 msgr2=0x7f09f80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- 172.21.15.33:0/1715535545 shutdown_connections 2024-04-24T23:10:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- 172.21.15.33:0/1715535545 wait complete. 2024-04-24T23:10:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 Processor -- start 2024-04-24T23:10:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- start start 2024-04-24T23:10:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 0x7f09f8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09f8104b30 0x7f09f8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f8107460 0x7f09f819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d420 con 0x7f09f8102200 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d2a0 con 0x7f09f8104b30 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.169+0000 7f09fcb34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d5a0 con 0x7f09f8107460 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 0x7f09f8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 0x7f09f8197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41000/0 (socket says 172.21.15.33:41000) 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 -- 172.21.15.33:0/1515149103 learned_addr learned my addr 172.21.15.33:0/1515149103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f6d9d640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f8107460 0x7f09f819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f5d9b640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09f8104b30 0x7f09f8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 -- 172.21.15.33:0/1515149103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f8107460 msgr2=0x7f09f819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f8107460 0x7f09f819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 -- 172.21.15.33:0/1515149103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09f8104b30 msgr2=0x7f09f8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09f8104b30 0x7f09f8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 -- 172.21.15.33:0/1515149103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f819f690 con 0x7f09f8102200 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09f659c640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 0x7f09f8197a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f09e800d770 tx=0x7f09e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8017070 con 0x7f09f8102200 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f819f980 con 0x7f09f8102200 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09e8005500 con 0x7f09f8102200 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8004e00 con 0x7f09f8102200 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.170+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f819fed0 con 0x7f09f8102200 2024-04-24T23:10:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.171+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f8068560 con 0x7f09f8102200 2024-04-24T23:10:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.173+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f09e8004fa0 con 0x7f09f8102200 2024-04-24T23:10:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.174+0000 7f09e77fe640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f09e00a2b90 0x7f09e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.174+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09e80ce620 con 0x7f09f8102200 2024-04-24T23:10:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.174+0000 7f09f5d9b640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f09e00a2b90 0x7f09e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.174+0000 7f09f5d9b640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f09e00a2b90 0x7f09e00a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005fd0 tx=0x7f09ec005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.179+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f09e809ba20 con 0x7f09f8102200 2024-04-24T23:10:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.407+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09f80fd6e0 con 0x7f09f8102200 2024-04-24T23:10:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.408+0000 7f09e77fe640 1 -- 172.21.15.33:0/1515149103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f09e809b320 con 0x7f09f8102200 2024-04-24T23:10:01.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:01.411 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24455,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:10:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-04-24T23:10:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.410+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f09e00a2b90 msgr2=0x7f09e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f09e00a2b90 0x7f09e00a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005fd0 tx=0x7f09ec005850 comp rx=0 tx=0).stop 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 msgr2=0x7f09f8197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 0x7f09f8197a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f09e800d770 tx=0x7f09e800dc30 comp rx=0 tx=0).stop 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 shutdown_connections 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f09e00a2b90 0x7f09e00a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f8107460 0x7f09f819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09f8104b30 0x7f09f8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 --2- 172.21.15.33:0/1515149103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09f8102200 0x7f09f8197a70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 >> 172.21.15.33:0/1515149103 conn(0x7f09f80fbbf0 msgr2=0x7f09f8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 shutdown_connections 2024-04-24T23:10:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:01.411+0000 7f09fcb34640 1 -- 172.21.15.33:0/1515149103 wait complete. 2024-04-24T23:10:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:01 smithi038 ceph-mon[32450]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:01 smithi038 ceph-mon[32450]: Deploying daemon mds.a on smithi033 2024-04-24T23:10:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:01 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.33:6838/2446242312,v1:172.21.15.33:6839/2446242312] up:boot 2024-04-24T23:10:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:01 smithi038 ceph-mon[32450]: fsmap 10 up:standby 2024-04-24T23:10:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:10:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1515149103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:01 smithi117 ceph-mon[32529]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:01 smithi117 ceph-mon[32529]: Deploying daemon mds.a on smithi033 2024-04-24T23:10:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:01 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.33:6838/2446242312,v1:172.21.15.33:6839/2446242312] up:boot 2024-04-24T23:10:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:01 smithi117 ceph-mon[32529]: fsmap 10 up:standby 2024-04-24T23:10:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:10:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1515149103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:01 smithi033 ceph-mon[29201]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:01 smithi033 ceph-mon[29201]: Deploying daemon mds.a on smithi033 2024-04-24T23:10:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:01 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.33:6838/2446242312,v1:172.21.15.33:6839/2446242312] up:boot 2024-04-24T23:10:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:01 smithi033 ceph-mon[29201]: fsmap 10 up:standby 2024-04-24T23:10:01.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:10:01.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1515149103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:02.688 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 systemd[1]: Starting Ceph mds.a for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:02.943 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 podman[71222]: 2024-04-24 23:10:02.591004324 +0000 UTC m=+0.018745432 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:02.943 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 podman[71222]: 2024-04-24 23:10:02.722577241 +0000 UTC m=+0.150318331 container create 5c917eb96288d642ab3969f6eb9ad9413b2268e0f11f8b047a3b36ea51520268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:10:02.943 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 podman[71222]: 2024-04-24 23:10:02.941945894 +0000 UTC m=+0.369686984 container init 5c917eb96288d642ab3969f6eb9ad9413b2268e0f11f8b047a3b36ea51520268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:10:03.205 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 podman[71222]: 2024-04-24 23:10:02.944736128 +0000 UTC m=+0.372477217 container start 5c917eb96288d642ab3969f6eb9ad9413b2268e0f11f8b047a3b36ea51520268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:10:03.205 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 ceph-mds[71241]: -- 172.21.15.33:0/1713384631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56198c77fa40 con 0x56198c752800 2024-04-24T23:10:03.205 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:02 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a[71237]: starting mds.a at 2024-04-24T23:10:03.205 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:03 smithi033 bash[71222]: 5c917eb96288d642ab3969f6eb9ad9413b2268e0f11f8b047a3b36ea51520268 2024-04-24T23:10:03.205 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:10:03 smithi033 systemd[1]: Started Ceph mds.a for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:03.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.530+0000 7f1886493640 1 -- 172.21.15.33:0/3929499541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1880101890 msgr2=0x7f1880101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.530+0000 7f1886493640 1 --2- 172.21.15.33:0/3929499541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1880101890 0x7f1880101d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 -- 172.21.15.33:0/3929499541 shutdown_connections 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 --2- 172.21.15.33:0/3929499541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102250 0x7f1880108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 --2- 172.21.15.33:0/3929499541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1880101890 0x7f1880101d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 --2- 172.21.15.33:0/3929499541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18801006a0 0x7f1880100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 -- 172.21.15.33:0/3929499541 >> 172.21.15.33:0/3929499541 conn(0x7f18800fbc30 msgr2=0x7f18800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 -- 172.21.15.33:0/3929499541 shutdown_connections 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 -- 172.21.15.33:0/3929499541 wait complete. 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 Processor -- start 2024-04-24T23:10:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.531+0000 7f1886493640 1 -- start start 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1886493640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 0x7f188006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1886493640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101890 0x7f188019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1886493640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880102250 0x7f188019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1886493640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f188010c3e0 con 0x7f18801006a0 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1886493640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f188010c260 con 0x7f1880101890 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1886493640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f188010c560 con 0x7f1880102250 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 0x7f188006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 0x7f188006a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41042/0 (socket says 172.21.15.33:41042) 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 -- 172.21.15.33:0/2891975818 learned_addr learned my addr 172.21.15.33:0/2891975818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187f7fe640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101890 0x7f188019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f1884a30640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880102250 0x7f188019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 -- 172.21.15.33:0/2891975818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880102250 msgr2=0x7f188019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880102250 0x7f188019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 -- 172.21.15.33:0/2891975818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101890 msgr2=0x7f188019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101890 0x7f188019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 -- 172.21.15.33:0/2891975818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188019f7b0 con 0x7f18801006a0 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.532+0000 7f187ffff640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 0x7f188006a910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f187400b450 tx=0x7f187400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.533+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874015020 con 0x7f18801006a0 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.533+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188019fa40 con 0x7f18801006a0 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.533+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874011d70 con 0x7f18801006a0 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.533+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187400f340 con 0x7f18801006a0 2024-04-24T23:10:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.533+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188019ffb0 con 0x7f18801006a0 2024-04-24T23:10:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.534+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18801079c0 con 0x7f18801006a0 2024-04-24T23:10:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.535+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1874018020 con 0x7f18801006a0 2024-04-24T23:10:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.535+0000 7f187d7fa640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18680a2b90 0x7f18680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.536+0000 7f187f7fe640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18680a2b90 0x7f18680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.536+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f18740ceb30 con 0x7f18801006a0 2024-04-24T23:10:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.536+0000 7f187f7fe640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18680a2b90 0x7f18680a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1870031440 tx=0x7f18700315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:03.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.539+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f187401f020 con 0x7f18801006a0 2024-04-24T23:10:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.769+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f188002d320 con 0x7f18801006a0 2024-04-24T23:10:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.769+0000 7f187d7fa640 1 -- 172.21.15.33:0/2891975818 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7f187400d040 con 0x7f18801006a0 2024-04-24T23:10:03.771 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:03.771 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24455,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:10:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-04-24T23:10:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18680a2b90 msgr2=0x7f18680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18680a2b90 0x7f18680a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1870031440 tx=0x7f18700315b0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 msgr2=0x7f188006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 0x7f188006a910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f187400b450 tx=0x7f187400b910 comp rx=0 tx=0).stop 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 shutdown_connections 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f18680a2b90 0x7f18680a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1880102250 0x7f188019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101890 0x7f188019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 --2- 172.21.15.33:0/2891975818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801006a0 0x7f188006a910 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 >> 172.21.15.33:0/2891975818 conn(0x7f18800fbc30 msgr2=0x7f1880104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.771+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 shutdown_connections 2024-04-24T23:10:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:03.772+0000 7f1886493640 1 -- 172.21.15.33:0/2891975818 wait complete. 2024-04-24T23:10:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:04.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:04 smithi038 ceph-mon[32450]: Deploying daemon mds.b on smithi038 2024-04-24T23:10:04.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:04 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.33:6840/1277151249,v1:172.21.15.33:6841/1277151249] up:boot 2024-04-24T23:10:04.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:04 smithi038 ceph-mon[32450]: fsmap 11 up:standby 2024-04-24T23:10:04.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:10:04.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:04 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2891975818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:04.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:04 smithi117 ceph-mon[32529]: Deploying daemon mds.b on smithi038 2024-04-24T23:10:04.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:04 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.33:6840/1277151249,v1:172.21.15.33:6841/1277151249] up:boot 2024-04-24T23:10:04.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:04 smithi117 ceph-mon[32529]: fsmap 11 up:standby 2024-04-24T23:10:04.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:10:04.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:04 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2891975818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:04.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:04 smithi033 ceph-mon[29201]: Deploying daemon mds.b on smithi038 2024-04-24T23:10:04.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:04 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.33:6840/1277151249,v1:172.21.15.33:6841/1277151249] up:boot 2024-04-24T23:10:04.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:04 smithi033 ceph-mon[29201]: fsmap 11 up:standby 2024-04-24T23:10:04.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:10:04.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:04 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2891975818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:05.653 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:05 smithi038 systemd[1]: Starting Ceph mds.b for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:05.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 -- 172.21.15.33:0/1529621536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 msgr2=0x7f4e101098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 --2- 172.21.15.33:0/1529621536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e101098b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009ca0 tx=0x7f4e0002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 -- 172.21.15.33:0/1529621536 shutdown_connections 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 --2- 172.21.15.33:0/1529621536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e101098b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 --2- 172.21.15.33:0/1529621536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e10104b30 0x7f4e10106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 --2- 172.21.15.33:0/1529621536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e10102200 0x7f4e101045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 -- 172.21.15.33:0/1529621536 >> 172.21.15.33:0/1529621536 conn(0x7f4e100fbbf0 msgr2=0x7f4e100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 -- 172.21.15.33:0/1529621536 shutdown_connections 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 -- 172.21.15.33:0/1529621536 wait complete. 2024-04-24T23:10:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 Processor -- start 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.899+0000 7f4e14815640 1 -- start start 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e14815640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e10102200 0x7f4e10197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e14815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e10104b30 0x7f4e10198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e14815640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e1019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e14815640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1010d4c0 con 0x7f4e10107460 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e14815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1010d340 con 0x7f4e10104b30 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e14815640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1010d640 con 0x7f4e10102200 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e1019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e1019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38702/0 (socket says 172.21.15.33:38702) 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 -- 172.21.15.33:0/2221638352 learned_addr learned my addr 172.21.15.33:0/2221638352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0dd9b640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e10104b30 0x7f4e10198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0e59c640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e10102200 0x7f4e10197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 -- 172.21.15.33:0/2221638352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e10102200 msgr2=0x7f4e10197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e10102200 0x7f4e10197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 -- 172.21.15.33:0/2221638352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e10104b30 msgr2=0x7f4e10198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e10104b30 0x7f4e10198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 -- 172.21.15.33:0/2221638352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1019f700 con 0x7f4e10107460 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4e0ed9d640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e1019f090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009dc0 tx=0x7f4e00003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004240 con 0x7f4e10107460 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.900+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e00003e40 con 0x7f4e10107460 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.901+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1019f990 con 0x7f4e10107460 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.901+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0003b400 con 0x7f4e10107460 2024-04-24T23:10:05.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.901+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1019ffb0 con 0x7f4e10107460 2024-04-24T23:10:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.901+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e100fd6e0 con 0x7f4e10107460 2024-04-24T23:10:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.903+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4e00030700 con 0x7f4e10107460 2024-04-24T23:10:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.903+0000 7f4dff7fe640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4df80a2c10 0x7f4df80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.903+0000 7f4e0e59c640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4df80a2c10 0x7f4df80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.903+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4e000f0270 con 0x7f4e10107460 2024-04-24T23:10:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.904+0000 7f4e0e59c640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4df80a2c10 0x7f4df80a50d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800a7d0 tx=0x7f4e08008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:05.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:05.907+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4e000bd670 con 0x7f4e10107460 2024-04-24T23:10:05.909 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:05 smithi038 ceph-mon[32450]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:05.909 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:05 smithi038 podman[62833]: 2024-04-24 23:10:05.65480229 +0000 UTC m=+0.018568471 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:05.909 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:05 smithi038 podman[62833]: 2024-04-24 23:10:05.763331594 +0000 UTC m=+0.127097762 container create 94a82dbe1869f24bf46909907ea7e2346f9ca3f16e7215b67a3b751ed2dbb8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:10:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:05 smithi117 ceph-mon[32529]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:05.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:05 smithi033 ceph-mon[29201]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:06.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.133+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e1002d170 con 0x7f4e10107460 2024-04-24T23:10:06.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.134+0000 7f4dff7fe640 1 -- 172.21.15.33:0/2221638352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7f4e000098b0 con 0x7f4e10107460 2024-04-24T23:10:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:06.136 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24455,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:10:06.136 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-04-24T23:10:06.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4df80a2c10 msgr2=0x7f4df80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4df80a2c10 0x7f4df80a50d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800a7d0 tx=0x7f4e08008040 comp rx=0 tx=0).stop 2024-04-24T23:10:06.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 msgr2=0x7f4e1019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e1019f090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009dc0 tx=0x7f4e00003c20 comp rx=0 tx=0).stop 2024-04-24T23:10:06.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 shutdown_connections 2024-04-24T23:10:06.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4df80a2c10 0x7f4df80a50d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10107460 0x7f4e1019f090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e10104b30 0x7f4e10198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 --2- 172.21.15.33:0/2221638352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e10102200 0x7f4e10197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 >> 172.21.15.33:0/2221638352 conn(0x7f4e100fbbf0 msgr2=0x7f4e10102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 shutdown_connections 2024-04-24T23:10:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:06.136+0000 7f4e14815640 1 -- 172.21.15.33:0/2221638352 wait complete. 2024-04-24T23:10:06.420 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:05 smithi038 podman[62833]: 2024-04-24 23:10:05.999842519 +0000 UTC m=+0.363608688 container init 94a82dbe1869f24bf46909907ea7e2346f9ca3f16e7215b67a3b751ed2dbb8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T23:10:06.421 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:06 smithi038 podman[62833]: 2024-04-24 23:10:06.002574636 +0000 UTC m=+0.366340805 container start 94a82dbe1869f24bf46909907ea7e2346f9ca3f16e7215b67a3b751ed2dbb8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:10:06.421 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mds[62852]: -- 172.21.15.38:0/2291954949 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579870fba40 con 0x5579870cf000 2024-04-24T23:10:06.421 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b[62848]: starting mds.b at 2024-04-24T23:10:06.421 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:06 smithi038 bash[62833]: 94a82dbe1869f24bf46909907ea7e2346f9ca3f16e7215b67a3b751ed2dbb8e5 2024-04-24T23:10:06.421 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:10:06 smithi038 systemd[1]: Started Ceph mds.b for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2221638352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2221638352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2221638352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:07 smithi038 ceph-mon[32450]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:07 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] up:boot 2024-04-24T23:10:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:07 smithi038 ceph-mon[32450]: fsmap 12 up:standby 2024-04-24T23:10:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:10:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:07 smithi117 ceph-mon[32529]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:07 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] up:boot 2024-04-24T23:10:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:07 smithi117 ceph-mon[32529]: fsmap 12 up:standby 2024-04-24T23:10:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:10:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:07 smithi033 ceph-mon[29201]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:07 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] up:boot 2024-04-24T23:10:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:07 smithi033 ceph-mon[29201]: fsmap 12 up:standby 2024-04-24T23:10:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:10:08.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 -- 172.21.15.33:0/2622489385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c0102a30 msgr2=0x7ff8c0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/2622489385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c0102a30 0x7ff8c0102eb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009930 tx=0x7ff8bc02a420 comp rx=0 tx=0).stop 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 -- 172.21.15.33:0/2622489385 shutdown_connections 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/2622489385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c01033f0 0x7ff8c0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/2622489385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c0102a30 0x7ff8c0102eb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/2622489385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8c00ffa60 0x7ff8c00ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 -- 172.21.15.33:0/2622489385 >> 172.21.15.33:0/2622489385 conn(0x7ff8c00fb3b0 msgr2=0x7ff8c00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 -- 172.21.15.33:0/2622489385 shutdown_connections 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 -- 172.21.15.33:0/2622489385 wait complete. 2024-04-24T23:10:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 Processor -- start 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.268+0000 7ff8c7bce640 1 -- start start 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 0x7ff8c0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8c0102a30 0x7ff8c0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c01033f0 0x7ff8c019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c010d420 con 0x7ff8c00ffa60 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c010d2a0 con 0x7ff8c01033f0 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c010d5a0 con 0x7ff8c0102a30 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 0x7ff8c0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 0x7ff8c0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38724/0 (socket says 172.21.15.33:38724) 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 -- 172.21.15.33:0/798648542 learned_addr learned my addr 172.21.15.33:0/798648542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c616b640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c01033f0 0x7ff8c019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c5169640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8c0102a30 0x7ff8c0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 -- 172.21.15.33:0/798648542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8c0102a30 msgr2=0x7ff8c0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8c0102a30 0x7ff8c0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 -- 172.21.15.33:0/798648542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c01033f0 msgr2=0x7ff8c019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c01033f0 0x7ff8c019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 -- 172.21.15.33:0/798648542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c019f6e0 con 0x7ff8c00ffa60 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c596a640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 0x7ff8c0197ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d950 tx=0x7ff8b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0017020 con 0x7ff8c00ffa60 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c019f9d0 con 0x7ff8c00ffa60 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b0009c30 con 0x7ff8c00ffa60 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000f740 con 0x7ff8c00ffa60 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.269+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c019fef0 con 0x7ff8c00ffa60 2024-04-24T23:10:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.270+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c0108c30 con 0x7ff8c00ffa60 2024-04-24T23:10:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.271+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8b001a020 con 0x7ff8c00ffa60 2024-04-24T23:10:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.272+0000 7ff8b6ffd640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8ac0a2b90 0x7ff8ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.272+0000 7ff8c5169640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8ac0a2b90 0x7ff8ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.272+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8b00ce5b0 con 0x7ff8c00ffa60 2024-04-24T23:10:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.272+0000 7ff8c5169640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8ac0a2b90 0x7ff8ac0a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc005ab0 tx=0x7ff8bc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.277+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8b0021070 con 0x7ff8c00ffa60 2024-04-24T23:10:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.500+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8c002d170 con 0x7ff8c00ffa60 2024-04-24T23:10:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.500+0000 7ff8b6ffd640 1 -- 172.21.15.33:0/798648542 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7ff8b0009de0 con 0x7ff8c00ffa60 2024-04-24T23:10:08.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:08.503 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24455,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:10:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-04-24T23:10:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.502+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8ac0a2b90 msgr2=0x7ff8ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.502+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8ac0a2b90 0x7ff8ac0a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc005ab0 tx=0x7ff8bc000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.502+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 msgr2=0x7ff8c0197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.502+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 0x7ff8c0197ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d950 tx=0x7ff8b000de10 comp rx=0 tx=0).stop 2024-04-24T23:10:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.502+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 shutdown_connections 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.502+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8ac0a2b90 0x7ff8ac0a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.503+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8c01033f0 0x7ff8c019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.503+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8c0102a30 0x7ff8c0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.503+0000 7ff8c7bce640 1 --2- 172.21.15.33:0/798648542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c00ffa60 0x7ff8c0197ac0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.503+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 >> 172.21.15.33:0/798648542 conn(0x7ff8c00fb3b0 msgr2=0x7ff8c0105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.503+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 shutdown_connections 2024-04-24T23:10:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.503+0000 7ff8c7bce640 1 -- 172.21.15.33:0/798648542 wait complete. 2024-04-24T23:10:08.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 -- 172.21.15.33:0/3415049136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc102250 msgr2=0x7fecdc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 --2- 172.21.15.33:0/3415049136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc102250 0x7fecdc108ae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feccc009900 tx=0x7feccc02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 -- 172.21.15.33:0/3415049136 shutdown_connections 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 --2- 172.21.15.33:0/3415049136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc102250 0x7fecdc108ae0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 --2- 172.21.15.33:0/3415049136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecdc101890 0x7fecdc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 --2- 172.21.15.33:0/3415049136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc1006a0 0x7fecdc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 -- 172.21.15.33:0/3415049136 >> 172.21.15.33:0/3415049136 conn(0x7fecdc0fbc30 msgr2=0x7fecdc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 -- 172.21.15.33:0/3415049136 shutdown_connections 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.616+0000 7fece4476640 1 -- 172.21.15.33:0/3415049136 wait complete. 2024-04-24T23:10:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 Processor -- start 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 -- start start 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 0x7fecdc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc101890 0x7fecdc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecdc102250 0x7fecdc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc10d4a0 con 0x7fecdc1006a0 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc10d320 con 0x7fecdc102250 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece4476640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc10d620 con 0x7fecdc101890 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 0x7fecdc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 0x7fecdc197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38740/0 (socket says 172.21.15.33:38740) 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 -- 172.21.15.33:0/2545880335 learned_addr learned my addr 172.21.15.33:0/2545880335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2a13640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecdc102250 0x7fecdc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece1a11640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc101890 0x7fecdc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 -- 172.21.15.33:0/2545880335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc101890 msgr2=0x7fecdc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc101890 0x7fecdc197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 -- 172.21.15.33:0/2545880335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecdc102250 msgr2=0x7fecdc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecdc102250 0x7fecdc19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 -- 172.21.15.33:0/2545880335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc19f640 con 0x7fecdc1006a0 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.617+0000 7fece2212640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 0x7fecdc197a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fecd400eb70 tx=0x7fecd400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.618+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003b70 con 0x7fecdc1006a0 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.618+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc19f930 con 0x7fecdc1006a0 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.618+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd400ee70 con 0x7fecdc1006a0 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.618+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4010830 con 0x7fecdc1006a0 2024-04-24T23:10:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.618+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc19fe80 con 0x7fecdc1006a0 2024-04-24T23:10:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.619+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc107960 con 0x7fecdc1006a0 2024-04-24T23:10:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.620+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fecd4004020 con 0x7fecdc1006a0 2024-04-24T23:10:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.620+0000 7fecdb7fe640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fecc40a2b90 0x7fecc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.620+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fecd40cf230 con 0x7fecdc1006a0 2024-04-24T23:10:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.620+0000 7fece1a11640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fecc40a2b90 0x7fecc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.621+0000 7fece1a11640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fecc40a2b90 0x7fecc40a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecdc198f30 tx=0x7fecd0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.625+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fecd409c630 con 0x7fecdc1006a0 2024-04-24T23:10:08.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.818+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecdc02d420 con 0x7fecdc1006a0 2024-04-24T23:10:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.819+0000 7fecdb7fe640 1 -- 172.21.15.33:0/2545880335 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18420 (secure 0 0 0) 0x7fecd409bf30 con 0x7fecdc1006a0 2024-04-24T23:10:08.821 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:08.821 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:09:22.310342+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fecc40a2b90 msgr2=0x7fecc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fecc40a2b90 0x7fecc40a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecdc198f30 tx=0x7fecd0005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 msgr2=0x7fecdc197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 0x7fecdc197a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fecd400eb70 tx=0x7fecd400c6a0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 shutdown_connections 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fecc40a2b90 0x7fecc40a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecdc102250 0x7fecdc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc101890 0x7fecdc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 --2- 172.21.15.33:0/2545880335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecdc1006a0 0x7fecdc197a20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 >> 172.21.15.33:0/2545880335 conn(0x7fecdc0fbc30 msgr2=0x7fecdc104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 shutdown_connections 2024-04-24T23:10:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.821+0000 7fece4476640 1 -- 172.21.15.33:0/2545880335 wait complete. 2024-04-24T23:10:08.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:08.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 -- 172.21.15.33:0/2686064216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 msgr2=0x7fa34c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 --2- 172.21.15.33:0/2686064216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c101d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009900 tx=0x7fa33c02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 -- 172.21.15.33:0/2686064216 shutdown_connections 2024-04-24T23:10:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 --2- 172.21.15.33:0/2686064216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c102250 0x7fa34c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 --2- 172.21.15.33:0/2686064216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c101d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 --2- 172.21.15.33:0/2686064216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa34c1006a0 0x7fa34c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 -- 172.21.15.33:0/2686064216 >> 172.21.15.33:0/2686064216 conn(0x7fa34c0fbc10 msgr2=0x7fa34c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 -- 172.21.15.33:0/2686064216 shutdown_connections 2024-04-24T23:10:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.938+0000 7fa354058640 1 -- 172.21.15.33:0/2686064216 wait complete. 2024-04-24T23:10:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 Processor -- start 2024-04-24T23:10:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 -- start start 2024-04-24T23:10:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c1006a0 0x7fa34c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa34c102250 0x7fa34c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10c350 con 0x7fa34c101890 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10c1d0 con 0x7fa34c102250 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa354058640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10c4d0 con 0x7fa34c1006a0 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa351df4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c1006a0 0x7fa34c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38754/0 (socket says 172.21.15.33:38754) 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 -- 172.21.15.33:0/1446117126 learned_addr learned my addr 172.21.15.33:0/1446117126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa351df4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c1006a0 0x7fa34c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:60676/0 (socket says 172.21.15.33:60676) 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3525f5640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa34c102250 0x7fa34c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 -- 172.21.15.33:0/1446117126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c1006a0 msgr2=0x7fa34c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c1006a0 0x7fa34c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 -- 172.21.15.33:0/1446117126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa34c102250 msgr2=0x7fa34c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa34c102250 0x7fa34c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.939+0000 7fa3515f3640 1 -- 172.21.15.33:0/1446117126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c19f710 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.940+0000 7fa3515f3640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c198030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa33c031440 tx=0x7fa33c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.940+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c03f030 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.940+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c19f9a0 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.940+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33c003ce0 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.940+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c035070 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.940+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c19fee0 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.941+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c1a0110 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.942+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa33c042030 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.943+0000 7fa342ffd640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3380a2b90 0x7fa3380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.943+0000 7fa351df4640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3380a2b90 0x7fa3380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.943+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa33c0ef1d0 con 0x7fa34c101890 2024-04-24T23:10:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.943+0000 7fa351df4640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3380a2b90 0x7fa3380a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa34c1016f0 tx=0x7fa34800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:08.947+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa33c0bbee0 con 0x7fa34c101890 2024-04-24T23:10:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.155+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa34c1a02e0 con 0x7fa34c101890 2024-04-24T23:10:09.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.155+0000 7fa342ffd640 1 -- 172.21.15.33:0/1446117126 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa33c0c01e0 con 0x7fa34c101890 2024-04-24T23:10:09.157 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:10:09.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3380a2b90 msgr2=0x7fa3380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3380a2b90 0x7fa3380a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa34c1016f0 tx=0x7fa34800b390 comp rx=0 tx=0).stop 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 msgr2=0x7fa34c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c198030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa33c031440 tx=0x7fa33c032e20 comp rx=0 tx=0).stop 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 shutdown_connections 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3380a2b90 0x7fa3380a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa34c102250 0x7fa34c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34c101890 0x7fa34c198030 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 --2- 172.21.15.33:0/1446117126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34c1006a0 0x7fa34c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 >> 172.21.15.33:0/1446117126 conn(0x7fa34c0fbc10 msgr2=0x7fa34c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 shutdown_connections 2024-04-24T23:10:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.158+0000 7fa354058640 1 -- 172.21.15.33:0/1446117126 wait complete. 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/798648542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:09.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:09.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2545880335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 -- 172.21.15.33:0/4081390650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4104bb0 msgr2=0x7ff4b4106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/4081390650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4104bb0 0x7ff4b4106fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009900 tx=0x7ff4a802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 -- 172.21.15.33:0/4081390650 shutdown_connections 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/4081390650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b41074e0 0x7ff4b41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/4081390650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4104bb0 0x7ff4b4106fa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/4081390650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b4102280 0x7ff4b4104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 -- 172.21.15.33:0/4081390650 >> 172.21.15.33:0/4081390650 conn(0x7ff4b40fbbf0 msgr2=0x7ff4b40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 -- 172.21.15.33:0/4081390650 shutdown_connections 2024-04-24T23:10:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.272+0000 7ff4ba14f640 1 -- 172.21.15.33:0/4081390650 wait complete. 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 Processor -- start 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 -- start start 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 0x7ff4b4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b4104bb0 0x7ff4b4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b41074e0 0x7ff4b419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410d420 con 0x7ff4b4102280 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410d2a0 con 0x7ff4b41074e0 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4ba14f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410d5a0 con 0x7ff4b4104bb0 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 0x7ff4b4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 0x7ff4b4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38784/0 (socket says 172.21.15.33:38784) 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 -- 172.21.15.33:0/2211440492 learned_addr learned my addr 172.21.15.33:0/2211440492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b3fff640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b41074e0 0x7ff4b419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b2ffd640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b4104bb0 0x7ff4b4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 -- 172.21.15.33:0/2211440492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b4104bb0 msgr2=0x7ff4b4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b4104bb0 0x7ff4b4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 -- 172.21.15.33:0/2211440492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b41074e0 msgr2=0x7ff4b419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b41074e0 0x7ff4b419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.273+0000 7ff4b37fe640 1 -- 172.21.15.33:0/2211440492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b419f710 con 0x7ff4b4102280 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.274+0000 7ff4b37fe640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 0x7ff4b4197af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d950 tx=0x7ff4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.274+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4017030 con 0x7ff4b4102280 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.274+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4004020 con 0x7ff4b4102280 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.274+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b419fa00 con 0x7ff4b4102280 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.274+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40047a0 con 0x7ff4b4102280 2024-04-24T23:10:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.274+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b419ff20 con 0x7ff4b4102280 2024-04-24T23:10:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.275+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b4068560 con 0x7ff4b4102280 2024-04-24T23:10:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.276+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4a40041d0 con 0x7ff4b4102280 2024-04-24T23:10:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.277+0000 7ff4b0ff9640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff49c0a2b90 0x7ff49c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.277+0000 7ff4b2ffd640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff49c0a2b90 0x7ff49c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.277+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff4a40ce530 con 0x7ff4b4102280 2024-04-24T23:10:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.277+0000 7ff4b2ffd640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff49c0a2b90 0x7ff49c0a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4a802aa00 tx=0x7ff4a80315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.281+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff4a409b930 con 0x7ff4b4102280 2024-04-24T23:10:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/798648542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2545880335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/798648542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:09.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:09.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:09.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2545880335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.471+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff4b402d170 con 0x7ff4b4102280 2024-04-24T23:10:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.472+0000 7ff4b0ff9640 1 -- 172.21.15.33:0/2211440492 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7ff4a409b230 con 0x7ff4b4102280 2024-04-24T23:10:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:09.473 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-04-24T23:10:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:10:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-04-24T23:10:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-04-24T23:10:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:10:09.474 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff49c0a2b90 msgr2=0x7ff49c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff49c0a2b90 0x7ff49c0a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4a802aa00 tx=0x7ff4a80315b0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 msgr2=0x7ff4b4197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 0x7ff4b4197af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d950 tx=0x7ff4a400de10 comp rx=0 tx=0).stop 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 shutdown_connections 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff49c0a2b90 0x7ff49c0a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b41074e0 0x7ff4b419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b4104bb0 0x7ff4b4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 --2- 172.21.15.33:0/2211440492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b4102280 0x7ff4b4197af0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 >> 172.21.15.33:0/2211440492 conn(0x7ff4b40fbbf0 msgr2=0x7ff4b4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 shutdown_connections 2024-04-24T23:10:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.474+0000 7ff4ba14f640 1 -- 172.21.15.33:0/2211440492 wait complete. 2024-04-24T23:10:09.488 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:10:09.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 -- 172.21.15.33:0/546837175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 msgr2=0x7f59b8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 --2- 172.21.15.33:0/546837175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b8100ac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009860 tx=0x7f59b402a340 comp rx=0 tx=0).stop 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 -- 172.21.15.33:0/546837175 shutdown_connections 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 --2- 172.21.15.33:0/546837175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 0x7f59b8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 --2- 172.21.15.33:0/546837175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 0x7f59b8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 --2- 172.21.15.33:0/546837175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b8100ac0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 -- 172.21.15.33:0/546837175 >> 172.21.15.33:0/546837175 conn(0x7f59b80fbc30 msgr2=0x7f59b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 -- 172.21.15.33:0/546837175 shutdown_connections 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.602+0000 7f59c0316640 1 -- 172.21.15.33:0/546837175 wait complete. 2024-04-24T23:10:09.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 Processor -- start 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 -- start start 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 0x7f59b819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 0x7f59b819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810c320 con 0x7f59b81006a0 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810c1a0 con 0x7f59b8101890 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59c0316640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810c4a0 con 0x7f59b8102250 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b806a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38798/0 (socket says 172.21.15.33:38798) 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 -- 172.21.15.33:0/2680153116 learned_addr learned my addr 172.21.15.33:0/2680153116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59bd8b1640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 0x7f59b819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be8b3640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 0x7f59b819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 -- 172.21.15.33:0/2680153116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 msgr2=0x7f59b819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 0x7f59b819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 -- 172.21.15.33:0/2680153116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 msgr2=0x7f59b819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 0x7f59b819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.603+0000 7f59be0b2640 1 -- 172.21.15.33:0/2680153116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b819f750 con 0x7f59b81006a0 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59be8b3640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 0x7f59b819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59bd8b1640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 0x7f59b819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59be0b2640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b806a8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59b4035040 tx=0x7f59b4033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4037040 con 0x7f59b81006a0 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819f980 con 0x7f59b81006a0 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b4032770 con 0x7f59b81006a0 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40046c0 con 0x7f59b81006a0 2024-04-24T23:10:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.604+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b819fe70 con 0x7f59b81006a0 2024-04-24T23:10:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.605+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b8107a30 con 0x7f59b81006a0 2024-04-24T23:10:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.607+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f59b4004050 con 0x7f59b81006a0 2024-04-24T23:10:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.608+0000 7f59af7fe640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59a40a2b90 0x7f59a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.608+0000 7f59bd8b1640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59a40a2b90 0x7f59a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.608+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f59b40eee40 con 0x7f59b81006a0 2024-04-24T23:10:09.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.608+0000 7f59bd8b1640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59a40a2b90 0x7f59a40a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59b806a620 tx=0x7f59a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.611+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59b40bc240 con 0x7f59b81006a0 2024-04-24T23:10:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:09.802+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f59b802d170 con 0x7f59b81006a0 2024-04-24T23:10:10.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.297+0000 7f59af7fe640 1 -- 172.21.15.33:0/2680153116 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f59b40bbb40 con 0x7f59b81006a0 2024-04-24T23:10:10.299 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59a40a2b90 msgr2=0x7f59a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59a40a2b90 0x7f59a40a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59b806a620 tx=0x7f59a8009260 comp rx=0 tx=0).stop 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 msgr2=0x7f59b806a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b806a8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59b4035040 tx=0x7f59b4033d90 comp rx=0 tx=0).stop 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 shutdown_connections 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59a40a2b90 0x7f59a40a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59b8102250 0x7f59b819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59b8101890 0x7f59b819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 --2- 172.21.15.33:0/2680153116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b81006a0 0x7f59b806a8b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 >> 172.21.15.33:0/2680153116 conn(0x7f59b80fbc30 msgr2=0x7f59b8104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:10.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 shutdown_connections 2024-04-24T23:10:10.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.300+0000 7f59c0316640 1 -- 172.21.15.33:0/2680153116 wait complete. 2024-04-24T23:10:10.315 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T23:10:10.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:10 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1446117126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:10.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:10 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2211440492' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:10:10.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:10 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2680153116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 -- 172.21.15.33:0/3708543055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 msgr2=0x7f7c5c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3708543055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1019e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009ca0 tx=0x7f7c5402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 -- 172.21.15.33:0/3708543055 shutdown_connections 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3708543055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1019e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3708543055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c1007a0 0x7f7c5c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3708543055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c5c0ffa40 0x7f7c5c0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 -- 172.21.15.33:0/3708543055 >> 172.21.15.33:0/3708543055 conn(0x7f7c5c0fb370 msgr2=0x7f7c5c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 -- 172.21.15.33:0/3708543055 shutdown_connections 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 -- 172.21.15.33:0/3708543055 wait complete. 2024-04-24T23:10:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.433+0000 7f7c641dc640 1 Processor -- start 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 -- start start 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c5c0ffa40 0x7f7c5c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c1007a0 0x7f7c5c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c10f600 con 0x7f7c5c101160 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c10f480 con 0x7f7c5c0ffa40 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c641dc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c10f780 con 0x7f7c5c1007a0 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c61f78640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c5c0ffa40 0x7f7c5c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38806/0 (socket says 172.21.15.33:38806) 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 -- 172.21.15.33:0/3989472027 learned_addr learned my addr 172.21.15.33:0/3989472027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c61777640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c1007a0 0x7f7c5c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 -- 172.21.15.33:0/3989472027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c1007a0 msgr2=0x7f7c5c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c1007a0 0x7f7c5c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 -- 172.21.15.33:0/3989472027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c5c0ffa40 msgr2=0x7f7c5c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c5c0ffa40 0x7f7c5c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 -- 172.21.15.33:0/3989472027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c1a18b0 con 0x7f7c5c101160 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.434+0000 7f7c62779640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1a1210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c54031ae0 tx=0x7f7c54031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.435+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54003b70 con 0x7f7c5c101160 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.435+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c1a1b40 con 0x7f7c5c101160 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.435+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c54003d10 con 0x7f7c5c101160 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.435+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5402fc40 con 0x7f7c5c101160 2024-04-24T23:10:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.435+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c1a2090 con 0x7f7c5c101160 2024-04-24T23:10:10.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.436+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5c10b690 con 0x7f7c5c101160 2024-04-24T23:10:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.437+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7c54004050 con 0x7f7c5c101160 2024-04-24T23:10:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.438+0000 7f7c52ffd640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7c480a2b90 0x7f7c480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.438+0000 7f7c61f78640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7c480a2b90 0x7f7c480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.438+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f7c540f0090 con 0x7f7c5c101160 2024-04-24T23:10:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.439+0000 7f7c61f78640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7c480a2b90 0x7f7c480a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c19ab40 tx=0x7f7c58005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:10.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.442+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c540bd280 con 0x7f7c5c101160 2024-04-24T23:10:10.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:10 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1446117126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:10.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:10 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2211440492' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:10:10.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:10 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2680153116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:10:10.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:10.655+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7c5c02d170 con 0x7f7c5c101160 2024-04-24T23:10:10.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:10 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1446117126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:10.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:10 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2211440492' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:10:10.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:10 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2680153116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:10:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.299+0000 7f7c52ffd640 1 -- 172.21.15.33:0/3989472027 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f7c540bcb80 con 0x7f7c5c101160 2024-04-24T23:10:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7c480a2b90 msgr2=0x7f7c480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7c480a2b90 0x7f7c480a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c19ab40 tx=0x7f7c58005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 msgr2=0x7f7c5c1a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1a1210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c54031ae0 tx=0x7f7c54031ee0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 shutdown_connections 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7c480a2b90 0x7f7c480a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5c101160 0x7f7c5c1a1210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c1007a0 0x7f7c5c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 --2- 172.21.15.33:0/3989472027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c5c0ffa40 0x7f7c5c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 >> 172.21.15.33:0/3989472027 conn(0x7f7c5c0fb370 msgr2=0x7f7c5c0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 shutdown_connections 2024-04-24T23:10:11.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.302+0000 7f7c641dc640 1 -- 172.21.15.33:0/3989472027 wait complete. 2024-04-24T23:10:11.323 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:10:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 -- 172.21.15.33:0/2889238000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 msgr2=0x7f3140101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 --2- 172.21.15.33:0/2889238000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140101bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f313c009900 tx=0x7f313c02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 -- 172.21.15.33:0/2889238000 shutdown_connections 2024-04-24T23:10:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 --2- 172.21.15.33:0/2889238000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140103340 0x7f3140109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 --2- 172.21.15.33:0/2889238000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140102980 0x7f3140102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 --2- 172.21.15.33:0/2889238000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140101bb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 -- 172.21.15.33:0/2889238000 >> 172.21.15.33:0/2889238000 conn(0x7f31400fcd20 msgr2=0x7f31400ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 -- 172.21.15.33:0/2889238000 shutdown_connections 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.428+0000 7f314805d640 1 -- 172.21.15.33:0/2889238000 wait complete. 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 Processor -- start 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 -- start start 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140102980 0x7f31401980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140103340 0x7f314019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f314010d480 con 0x7f3140101790 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f314010d300 con 0x7f3140102980 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f314805d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f314010d600 con 0x7f3140103340 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38826/0 (socket says 172.21.15.33:38826) 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 -- 172.21.15.33:0/3337304653 learned_addr learned my addr 172.21.15.33:0/3337304653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f31465fa640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140103340 0x7f314019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f31455f8640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140102980 0x7f31401980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 -- 172.21.15.33:0/3337304653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140103340 msgr2=0x7f314019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140103340 0x7f314019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 -- 172.21.15.33:0/3337304653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140102980 msgr2=0x7f31401980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140102980 0x7f31401980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f3145df9640 1 -- 172.21.15.33:0/3337304653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314019f7b0 con 0x7f3140101790 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f31465fa640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140103340 0x7f314019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.429+0000 7f31455f8640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140102980 0x7f31401980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.430+0000 7f3145df9640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140197b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f313c031610 tx=0x7f313c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.430+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c03f030 con 0x7f3140101790 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.430+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f313c003ce0 con 0x7f3140101790 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.430+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314019fa40 con 0x7f3140101790 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.430+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c035070 con 0x7f3140101790 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.430+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314019ffb0 con 0x7f3140101790 2024-04-24T23:10:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.431+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3140108b00 con 0x7f3140101790 2024-04-24T23:10:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.432+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f313c042030 con 0x7f3140101790 2024-04-24T23:10:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.433+0000 7f3136ffd640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f312c0a2b90 0x7f312c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.433+0000 7f31455f8640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f312c0a2b90 0x7f312c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.433+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f313c0ef7f0 con 0x7f3140101790 2024-04-24T23:10:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.433+0000 7f31455f8640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f312c0a2b90 0x7f312c0a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31401990a0 tx=0x7f3130005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.437+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f313c0bc860 con 0x7f3140101790 2024-04-24T23:10:11.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:11.662+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f314002d170 con 0x7f3140101790 2024-04-24T23:10:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2680153116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3989472027' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:11.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2680153116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3989472027' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:11.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:11.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2680153116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3989472027' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:10:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.325+0000 7f3136ffd640 1 -- 172.21.15.33:0/3337304653 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f313c0bc160 con 0x7f3140101790 2024-04-24T23:10:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f312c0a2b90 msgr2=0x7f312c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f312c0a2b90 0x7f312c0a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31401990a0 tx=0x7f3130005e50 comp rx=0 tx=0).stop 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 msgr2=0x7f3140197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140197b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f313c031610 tx=0x7f313c0043b0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 shutdown_connections 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f312c0a2b90 0x7f312c0a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3140103340 0x7f314019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3140102980 0x7f31401980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 --2- 172.21.15.33:0/3337304653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140101790 0x7f3140197b90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 >> 172.21.15.33:0/3337304653 conn(0x7f31400fcd20 msgr2=0x7f3140105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 shutdown_connections 2024-04-24T23:10:12.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.329+0000 7f314805d640 1 -- 172.21.15.33:0/3337304653 wait complete. 2024-04-24T23:10:12.352 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T23:10:12.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T23:10:12.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 -- 172.21.15.33:0/1802361624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770101790 msgr2=0x7fa770101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 --2- 172.21.15.33:0/1802361624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770101790 0x7fa770101bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa760009900 tx=0x7fa76002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 -- 172.21.15.33:0/1802361624 shutdown_connections 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 --2- 172.21.15.33:0/1802361624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 0x7fa770109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 --2- 172.21.15.33:0/1802361624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa770102980 0x7fa770102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 --2- 172.21.15.33:0/1802361624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770101790 0x7fa770101bb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 -- 172.21.15.33:0/1802361624 >> 172.21.15.33:0/1802361624 conn(0x7fa7700fcd20 msgr2=0x7fa7700ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 -- 172.21.15.33:0/1802361624 shutdown_connections 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 -- 172.21.15.33:0/1802361624 wait complete. 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 Processor -- start 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.466+0000 7fa7765a2640 1 -- start start 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa770101790 0x7fa770197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 0x7fa770198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 0x7fa77019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa77010d3e0 con 0x7fa770102980 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa77010d260 con 0x7fa770101790 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa77010d560 con 0x7fa770103340 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 0x7fa770198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 0x7fa770198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38844/0 (socket says 172.21.15.33:38844) 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 -- 172.21.15.33:0/2321110718 learned_addr learned my addr 172.21.15.33:0/2321110718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76ffff640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa770101790 0x7fa770197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa774b3f640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 0x7fa77019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 -- 172.21.15.33:0/2321110718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 msgr2=0x7fa77019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 0x7fa77019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 -- 172.21.15.33:0/2321110718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa770101790 msgr2=0x7fa770197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa770101790 0x7fa770197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 -- 172.21.15.33:0/2321110718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77019f730 con 0x7fa770102980 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa774b3f640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 0x7fa77019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76f7fe640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 0x7fa770198050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa76400d950 tx=0x7fa76400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764017070 con 0x7fa770102980 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77019fa20 con 0x7fa770102980 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa764009c30 con 0x7fa770102980 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76400f6b0 con 0x7fa770102980 2024-04-24T23:10:12.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.467+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77019ffc0 con 0x7fa770102980 2024-04-24T23:10:12.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.468+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa770108c20 con 0x7fa770102980 2024-04-24T23:10:12.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.470+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa7640109d0 con 0x7fa770102980 2024-04-24T23:10:12.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.470+0000 7fa76d7fa640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2b90 0x7fa7580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.470+0000 7fa76ffff640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2b90 0x7fa7580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.470+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa7640ceb10 con 0x7fa770102980 2024-04-24T23:10:12.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.471+0000 7fa76ffff640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2b90 0x7fa7580a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa760005ab0 tx=0x7fa760000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.474+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa764009de0 con 0x7fa770102980 2024-04-24T23:10:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:12.666+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa77002d170 con 0x7fa770102980 2024-04-24T23:10:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:12 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3989472027' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:10:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:12 smithi038 ceph-mon[32450]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:10:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:12 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3337304653' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:10:12.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:12 smithi038 ceph-mon[32450]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-24T23:10:12.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:12 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3989472027' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:10:12.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:12 smithi117 ceph-mon[32529]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:10:12.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:12 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3337304653' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:10:12.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:12 smithi117 ceph-mon[32529]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-24T23:10:12.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:12 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3989472027' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:10:12.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:12 smithi033 ceph-mon[29201]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:10:12.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:12 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3337304653' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:10:12.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:12 smithi033 ceph-mon[29201]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-24T23:10:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.333+0000 7fa76d7fa640 1 -- 172.21.15.33:0/2321110718 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fa76409bbc0 con 0x7fa770102980 2024-04-24T23:10:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.336+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2b90 msgr2=0x7fa7580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.336+0000 7fa7765a2640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2b90 0x7fa7580a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa760005ab0 tx=0x7fa760000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.336+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 msgr2=0x7fa770198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.336+0000 7fa7765a2640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 0x7fa770198050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa76400d950 tx=0x7fa76400de10 comp rx=0 tx=0).stop 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 shutdown_connections 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2b90 0x7fa7580a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa770103340 0x7fa77019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa770102980 0x7fa770198050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 --2- 172.21.15.33:0/2321110718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa770101790 0x7fa770197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 >> 172.21.15.33:0/2321110718 conn(0x7fa7700fcd20 msgr2=0x7fa770105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 shutdown_connections 2024-04-24T23:10:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.337+0000 7fa7765a2640 1 -- 172.21.15.33:0/2321110718 wait complete. 2024-04-24T23:10:13.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:10:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 -- 172.21.15.33:0/1791592281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 msgr2=0x7fcfa81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 --2- 172.21.15.33:0/1791592281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa81098b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009ca0 tx=0x7fcfa002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 -- 172.21.15.33:0/1791592281 shutdown_connections 2024-04-24T23:10:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 --2- 172.21.15.33:0/1791592281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa81098b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 --2- 172.21.15.33:0/1791592281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfa8104b90 0x7fcfa8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 --2- 172.21.15.33:0/1791592281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa8102260 0x7fcfa8104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 -- 172.21.15.33:0/1791592281 >> 172.21.15.33:0/1791592281 conn(0x7fcfa80fbbf0 msgr2=0x7fcfa80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 -- 172.21.15.33:0/1791592281 shutdown_connections 2024-04-24T23:10:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 -- 172.21.15.33:0/1791592281 wait complete. 2024-04-24T23:10:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.468+0000 7fcfae36a640 1 Processor -- start 2024-04-24T23:10:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 -- start start 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa8102260 0x7fcfa8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfa8104b90 0x7fcfa8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa810d470 con 0x7fcfa81074c0 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa810d2f0 con 0x7fcfa8102260 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfae36a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa810d5f0 con 0x7fcfa8104b90 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38864/0 (socket says 172.21.15.33:38864) 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 -- 172.21.15.33:0/844089231 learned_addr learned my addr 172.21.15.33:0/844089231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfa7fff640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa8102260 0x7fcfa8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfa77fe640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfa8104b90 0x7fcfa8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 -- 172.21.15.33:0/844089231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfa8104b90 msgr2=0x7fcfa8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfa8104b90 0x7fcfa8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 -- 172.21.15.33:0/844089231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa8102260 msgr2=0x7fcfa8197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa8102260 0x7fcfa8197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 -- 172.21.15.33:0/844089231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa819f700 con 0x7fcfa81074c0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.469+0000 7fcfac907640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa819f090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00098d0 tx=0x7fcfa002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.470+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa002fc30 con 0x7fcfa81074c0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.470+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa0003e50 con 0x7fcfa81074c0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.470+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa819f990 con 0x7fcfa81074c0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.470+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00305c0 con 0x7fcfa81074c0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.470+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa819ffb0 con 0x7fcfa81074c0 2024-04-24T23:10:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.471+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa80fd6e0 con 0x7fcfa81074c0 2024-04-24T23:10:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.472+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcfa0003970 con 0x7fcfa81074c0 2024-04-24T23:10:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.473+0000 7fcfa57fa640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf900a2c10 0x7fcf900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.473+0000 7fcfa7fff640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf900a2c10 0x7fcf900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.473+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fcfa00f03e0 con 0x7fcfa81074c0 2024-04-24T23:10:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.473+0000 7fcfa7fff640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf900a2c10 0x7fcf900a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf980058c0 tx=0x7fcf9800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.477+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcfa00bd430 con 0x7fcfa81074c0 2024-04-24T23:10:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:13.664+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcfa8062340 con 0x7fcfa81074c0 2024-04-24T23:10:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: pgmap v287: 97 pgs: 7 active+clean, 23 creating+peering, 67 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:10:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:10:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3337304653' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:10:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-24T23:10:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-24T23:10:13.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:13 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2321110718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: pgmap v287: 97 pgs: 7 active+clean, 23 creating+peering, 67 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3337304653' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-24T23:10:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:13 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2321110718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:10:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: pgmap v287: 97 pgs: 7 active+clean, 23 creating+peering, 67 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:10:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:10:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3337304653' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:10:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-24T23:10:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-24T23:10:13.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:13 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2321110718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:10:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:14 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] up:active 2024-04-24T23:10:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:14 smithi038 ceph-mon[32450]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:14 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2321110718' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:10:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:14 smithi038 ceph-mon[32450]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:10:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:14 smithi038 ceph-mon[32450]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:14 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/844089231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:10:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:14 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] up:active 2024-04-24T23:10:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:14 smithi117 ceph-mon[32529]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:14 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2321110718' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:10:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:14 smithi117 ceph-mon[32529]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:10:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:14 smithi117 ceph-mon[32529]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:14.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:14 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/844089231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:10:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:14 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] up:active 2024-04-24T23:10:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:14 smithi033 ceph-mon[29201]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:14 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2321110718' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:10:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:14 smithi033 ceph-mon[29201]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:10:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:14 smithi033 ceph-mon[29201]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:14 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/844089231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:10:15.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.352+0000 7fcfa57fa640 1 -- 172.21.15.33:0/844089231 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fcfa00bcd30 con 0x7fcfa81074c0 2024-04-24T23:10:15.354 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' created 2024-04-24T23:10:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf900a2c10 msgr2=0x7fcf900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf900a2c10 0x7fcf900a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf980058c0 tx=0x7fcf9800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 msgr2=0x7fcfa819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa819f090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00098d0 tx=0x7fcfa002f840 comp rx=0 tx=0).stop 2024-04-24T23:10:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 shutdown_connections 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf900a2c10 0x7fcf900a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfa81074c0 0x7fcfa819f090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfa8104b90 0x7fcfa8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 --2- 172.21.15.33:0/844089231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfa8102260 0x7fcfa8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.355+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 >> 172.21.15.33:0/844089231 conn(0x7fcfa80fbbf0 msgr2=0x7fcfa8102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.356+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 shutdown_connections 2024-04-24T23:10:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.356+0000 7fcfae36a640 1 -- 172.21.15.33:0/844089231 wait complete. 2024-04-24T23:10:15.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 -- 172.21.15.33:0/1374522275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404101790 msgr2=0x7fa404101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 --2- 172.21.15.33:0/1374522275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404101790 0x7fa404101bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa40002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 -- 172.21.15.33:0/1374522275 shutdown_connections 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 --2- 172.21.15.33:0/1374522275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa404103340 0x7fa404109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 --2- 172.21.15.33:0/1374522275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa404102980 0x7fa404102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 --2- 172.21.15.33:0/1374522275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404101790 0x7fa404101bb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 -- 172.21.15.33:0/1374522275 >> 172.21.15.33:0/1374522275 conn(0x7fa4040fcd20 msgr2=0x7fa4040ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 -- 172.21.15.33:0/1374522275 shutdown_connections 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.487+0000 7fa40c410640 1 -- 172.21.15.33:0/1374522275 wait complete. 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 Processor -- start 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 -- start start 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa404101790 0x7fa4040754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 0x7fa404073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa404103340 0x7fa404074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410d460 con 0x7fa404102980 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410d2e0 con 0x7fa404103340 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40c410640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410d5e0 con 0x7fa404101790 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 0x7fa404073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 0x7fa404073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38878/0 (socket says 172.21.15.33:38878) 2024-04-24T23:10:15.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 -- 172.21.15.33:0/1179835611 learned_addr learned my addr 172.21.15.33:0/1179835611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40a1ac640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa404101790 0x7fa4040754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa40a9ad640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa404103340 0x7fa404074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 -- 172.21.15.33:0/1179835611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa404101790 msgr2=0x7fa4040754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa404101790 0x7fa4040754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 -- 172.21.15.33:0/1179835611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa404103340 msgr2=0x7fa404074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa404103340 0x7fa404074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.488+0000 7fa4099ab640 1 -- 172.21.15.33:0/1179835611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa404074900 con 0x7fa404102980 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.489+0000 7fa4099ab640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 0x7fa404073b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400d950 tx=0x7fa3f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.489+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4017070 con 0x7fa404102980 2024-04-24T23:10:15.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.489+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f4009c30 con 0x7fa404102980 2024-04-24T23:10:15.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.489+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4041a1c60 con 0x7fa404102980 2024-04-24T23:10:15.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.489+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400f6b0 con 0x7fa404102980 2024-04-24T23:10:15.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.489+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4041a21b0 con 0x7fa404102980 2024-04-24T23:10:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.490+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa404108b30 con 0x7fa404102980 2024-04-24T23:10:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.491+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa3f40109d0 con 0x7fa404102980 2024-04-24T23:10:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.492+0000 7fa3fb7fe640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3f00a2b90 0x7fa3f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.492+0000 7fa40a1ac640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3f00a2b90 0x7fa3f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.492+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fa3f40cee00 con 0x7fa404102980 2024-04-24T23:10:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.492+0000 7fa40a1ac640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3f00a2b90 0x7fa3f00a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa400005ab0 tx=0x7fa400000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.497+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3f4009de0 con 0x7fa404102980 2024-04-24T23:10:15.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:15 smithi038 ceph-mon[32450]: pgmap v290: 97 pgs: 26 active+clean, 36 creating+peering, 35 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-24T23:10:15.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:15 smithi038 ceph-mon[32450]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-24T23:10:15.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:15 smithi038 ceph-mon[32450]: Cluster is now healthy 2024-04-24T23:10:15.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:15 smithi038 ceph-mon[32450]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:10:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:15 smithi117 ceph-mon[32529]: pgmap v290: 97 pgs: 26 active+clean, 36 creating+peering, 35 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-24T23:10:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:15 smithi117 ceph-mon[32529]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-24T23:10:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:15 smithi117 ceph-mon[32529]: Cluster is now healthy 2024-04-24T23:10:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:15 smithi117 ceph-mon[32529]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:10:15.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:15.683+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa4040623e0 con 0x7fa404102980 2024-04-24T23:10:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:15 smithi033 ceph-mon[29201]: pgmap v290: 97 pgs: 26 active+clean, 36 creating+peering, 35 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-24T23:10:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:15 smithi033 ceph-mon[29201]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-24T23:10:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:15 smithi033 ceph-mon[29201]: Cluster is now healthy 2024-04-24T23:10:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:15 smithi033 ceph-mon[29201]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:10:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.358+0000 7fa3fb7fe640 1 -- 172.21.15.33:0/1179835611 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fa3f409b4f0 con 0x7fa404102980 2024-04-24T23:10:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T23:10:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3f00a2b90 msgr2=0x7fa3f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3f00a2b90 0x7fa3f00a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa400005ab0 tx=0x7fa400000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 msgr2=0x7fa404073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 0x7fa404073b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400d950 tx=0x7fa3f400de10 comp rx=0 tx=0).stop 2024-04-24T23:10:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 shutdown_connections 2024-04-24T23:10:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3f00a2b90 0x7fa3f00a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa404103340 0x7fa404074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa404102980 0x7fa404073b30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 --2- 172.21.15.33:0/1179835611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa404101790 0x7fa4040754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 >> 172.21.15.33:0/1179835611 conn(0x7fa4040fcd20 msgr2=0x7fa404105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 shutdown_connections 2024-04-24T23:10:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.362+0000 7fa40c410640 1 -- 172.21.15.33:0/1179835611 wait complete. 2024-04-24T23:10:16.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T23:10:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 -- 172.21.15.33:0/138534876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba81006a0 msgr2=0x7f4ba8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 --2- 172.21.15.33:0/138534876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba81006a0 0x7f4ba8100ac0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009900 tx=0x7f4ba402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 -- 172.21.15.33:0/138534876 shutdown_connections 2024-04-24T23:10:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 --2- 172.21.15.33:0/138534876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba8102250 0x7f4ba8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 --2- 172.21.15.33:0/138534876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8101890 0x7f4ba8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 --2- 172.21.15.33:0/138534876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba81006a0 0x7f4ba8100ac0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 -- 172.21.15.33:0/138534876 >> 172.21.15.33:0/138534876 conn(0x7f4ba80fbc30 msgr2=0x7f4ba80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 -- 172.21.15.33:0/138534876 shutdown_connections 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 -- 172.21.15.33:0/138534876 wait complete. 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.496+0000 7f4bafec3640 1 Processor -- start 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 -- start start 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba81006a0 0x7f4ba8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 0x7f4ba8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8102250 0x7f4ba819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba810c320 con 0x7f4ba8101890 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba810c1a0 con 0x7f4ba81006a0 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba810c4a0 con 0x7f4ba8102250 2024-04-24T23:10:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 0x7f4ba8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 0x7f4ba8198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45890/0 (socket says 172.21.15.33:45890) 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 -- 172.21.15.33:0/1797878391 learned_addr learned my addr 172.21.15.33:0/1797878391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bae460640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8102250 0x7f4ba819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4badc5f640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba81006a0 0x7f4ba8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 -- 172.21.15.33:0/1797878391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8102250 msgr2=0x7f4ba819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8102250 0x7f4ba819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 -- 172.21.15.33:0/1797878391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba81006a0 msgr2=0x7f4ba8197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba81006a0 0x7f4ba8197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 -- 172.21.15.33:0/1797878391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba819f700 con 0x7f4ba8101890 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4badc5f640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba81006a0 0x7f4ba8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bad45e640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 0x7f4ba8198020 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e9f0 tx=0x7f4b9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800cd80 con 0x7f4ba8101890 2024-04-24T23:10:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b98005500 con 0x7f4ba8101890 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba819f9f0 con 0x7f4ba8101890 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.497+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98010690 con 0x7f4ba8101890 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.498+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba819ff40 con 0x7f4ba8101890 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.499+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba81a01d0 con 0x7f4ba8101890 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.501+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b980046c0 con 0x7f4ba8101890 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.501+0000 7f4b9effd640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b940a2c10 0x7f4b940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.501+0000 7f4badc5f640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b940a2c10 0x7f4b940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.501+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f4b980cf8a0 con 0x7f4ba8101890 2024-04-24T23:10:16.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.502+0000 7f4badc5f640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b940a2c10 0x7f4b940a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006010 tx=0x7f4ba4032dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.505+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b98053740 con 0x7f4ba8101890 2024-04-24T23:10:16.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:16 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/844089231' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:10:16.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:16 smithi038 ceph-mon[32450]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:10:16.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:16 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1179835611' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:10:16.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:16 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/844089231' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:10:16.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:16 smithi117 ceph-mon[32529]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:10:16.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:16 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1179835611' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:10:16.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:16 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/844089231' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:10:16.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:16 smithi033 ceph-mon[29201]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:10:16.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:16 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1179835611' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:10:16.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:16.731+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4ba81a0520 con 0x7f4ba8101890 2024-04-24T23:10:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.359+0000 7f4b9effd640 1 -- 172.21.15.33:0/1797878391 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f4b9809c5e0 con 0x7f4ba8101890 2024-04-24T23:10:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:added data pool 4 to fsmap 2024-04-24T23:10:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b940a2c10 msgr2=0x7f4b940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b940a2c10 0x7f4b940a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006010 tx=0x7f4ba4032dd0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 msgr2=0x7f4ba8198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 0x7f4ba8198020 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e9f0 tx=0x7f4b9800edf0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 shutdown_connections 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b940a2c10 0x7f4b940a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba8102250 0x7f4ba819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba8101890 0x7f4ba8198020 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 --2- 172.21.15.33:0/1797878391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba81006a0 0x7f4ba8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.363+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 >> 172.21.15.33:0/1797878391 conn(0x7f4ba80fbc30 msgr2=0x7f4ba8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.364+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 shutdown_connections 2024-04-24T23:10:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.364+0000 7f4bafec3640 1 -- 172.21.15.33:0/1797878391 wait complete. 2024-04-24T23:10:17.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.485+0000 7fe653bf9640 1 -- 172.21.15.33:0/2781197173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c101870 msgr2=0x7fe64c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.485+0000 7fe653bf9640 1 --2- 172.21.15.33:0/2781197173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c101870 0x7fe64c101cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe648009900 tx=0x7fe64802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- 172.21.15.33:0/2781197173 shutdown_connections 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 --2- 172.21.15.33:0/2781197173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c102230 0x7fe64c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 --2- 172.21.15.33:0/2781197173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c101870 0x7fe64c101cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 --2- 172.21.15.33:0/2781197173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c100680 0x7fe64c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- 172.21.15.33:0/2781197173 >> 172.21.15.33:0/2781197173 conn(0x7fe64c0fbbf0 msgr2=0x7fe64c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- 172.21.15.33:0/2781197173 shutdown_connections 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- 172.21.15.33:0/2781197173 wait complete. 2024-04-24T23:10:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 Processor -- start 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- start start 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 0x7fe64c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c101870 0x7fe64c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c102230 0x7fe64c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c10c350 con 0x7fe64c100680 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c10c1d0 con 0x7fe64c101870 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe653bf9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c10c4d0 con 0x7fe64c102230 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.486+0000 7fe651995640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 0x7fe64c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 0x7fe64c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45902/0 (socket says 172.21.15.33:45902) 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 -- 172.21.15.33:0/1510418261 learned_addr learned my addr 172.21.15.33:0/1510418261 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651194640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c101870 0x7fe64c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe652196640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c102230 0x7fe64c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 -- 172.21.15.33:0/1510418261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c102230 msgr2=0x7fe64c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c102230 0x7fe64c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 -- 172.21.15.33:0/1510418261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c101870 msgr2=0x7fe64c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c101870 0x7fe64c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 -- 172.21.15.33:0/1510418261 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c19f710 con 0x7fe64c100680 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651194640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c101870 0x7fe64c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe652196640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c102230 0x7fe64c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe651995640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 0x7fe64c197af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c970 tx=0x7fe63c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004220 con 0x7fe64c100680 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe63c0043c0 con 0x7fe64c100680 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c19fa00 con 0x7fe64c100680 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004a00 con 0x7fe64c100680 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.487+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c19fed0 con 0x7fe64c100680 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.488+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c1a0100 con 0x7fe64c100680 2024-04-24T23:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.489+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe63c0079b0 con 0x7fe64c100680 2024-04-24T23:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.490+0000 7fe642ffd640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6380a2b90 0x7fe6380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.490+0000 7fe651194640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6380a2b90 0x7fe6380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.490+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe63c0cf0d0 con 0x7fe64c100680 2024-04-24T23:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.491+0000 7fe651194640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6380a2b90 0x7fe6380a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.495+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe63c014030 con 0x7fe64c100680 2024-04-24T23:10:17.631 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:17 smithi117 ceph-mon[32529]: pgmap v293: 129 pgs: 63 active+clean, 41 creating+peering, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-04-24T23:10:17.631 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:17 smithi117 ceph-mon[32529]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:10:17.631 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:17 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1179835611' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:10:17.631 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:17 smithi117 ceph-mon[32529]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:10:17.631 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:17 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1797878391' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:10:17.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:17 smithi038 ceph-mon[32450]: pgmap v293: 129 pgs: 63 active+clean, 41 creating+peering, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-04-24T23:10:17.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:17 smithi038 ceph-mon[32450]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:10:17.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:17 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1179835611' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:10:17.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:17 smithi038 ceph-mon[32450]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:10:17.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:17 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1797878391' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:10:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:17 smithi033 ceph-mon[29201]: pgmap v293: 129 pgs: 63 active+clean, 41 creating+peering, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-04-24T23:10:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:17 smithi033 ceph-mon[29201]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:10:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:17 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1179835611' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:10:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:17 smithi033 ceph-mon[29201]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:10:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:17 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1797878391' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:10:17.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.719+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe64c1a02d0 con 0x7fe64c100680 2024-04-24T23:10:17.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.719+0000 7fe642ffd640 1 -- 172.21.15.33:0/1510418261 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7fe63c09bdf0 con 0x7fe64c100680 2024-04-24T23:10:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:17.722 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24455,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:16.733281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24508,"qdb_cluster":[24508]},"id":1}]} 2024-04-24T23:10:17.722 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6380a2b90 msgr2=0x7fe6380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6380a2b90 0x7fe6380a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 msgr2=0x7fe64c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 0x7fe64c197af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c970 tx=0x7fe63c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 shutdown_connections 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6380a2b90 0x7fe6380a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c102230 0x7fe64c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe64c101870 0x7fe64c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 --2- 172.21.15.33:0/1510418261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe64c100680 0x7fe64c197af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 >> 172.21.15.33:0/1510418261 conn(0x7fe64c0fbbf0 msgr2=0x7fe64c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 shutdown_connections 2024-04-24T23:10:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.722+0000 7fe653bf9640 1 -- 172.21.15.33:0/1510418261 wait complete. 2024-04-24T23:10:17.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- 172.21.15.33:0/3198203626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 msgr2=0x7f3480106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- 172.21.15.33:0/3198203626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480106fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3470009900 tx=0x7f347002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- 172.21.15.33:0/3198203626 shutdown_connections 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- 172.21.15.33:0/3198203626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34801074e0 0x7f34801098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- 172.21.15.33:0/3198203626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480106fa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- 172.21.15.33:0/3198203626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- 172.21.15.33:0/3198203626 >> 172.21.15.33:0/3198203626 conn(0x7f34800fbc30 msgr2=0x7f34800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- 172.21.15.33:0/3198203626 shutdown_connections 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- 172.21.15.33:0/3198203626 wait complete. 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 Processor -- start 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- start start 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34801074e0 0x7f348019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f348010d420 con 0x7f3480104bb0 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f348010d2a0 con 0x7f3480102280 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.835+0000 7f3488734640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f348010d5a0 con 0x7f34801074e0 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f3485ccf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38342/0 (socket says 172.21.15.33:38342) 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f3485ccf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45918/0 (socket says 172.21.15.33:45918) 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 -- 172.21.15.33:0/2677122398 learned_addr learned my addr 172.21.15.33:0/2677122398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f3486cd1640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34801074e0 0x7f348019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 -- 172.21.15.33:0/2677122398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34801074e0 msgr2=0x7f348019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34801074e0 0x7f348019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 -- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 msgr2=0x7f3480198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 -- 172.21.15.33:0/2677122398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348019f710 con 0x7f3480102280 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34864d0640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480197af0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f347c00e9f0 tx=0x7f347c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c00cd80 con 0x7f3480102280 2024-04-24T23:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348019fa00 con 0x7f3480102280 2024-04-24T23:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f347c010910 con 0x7f3480102280 2024-04-24T23:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0188f0 con 0x7f3480102280 2024-04-24T23:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.836+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348019ff70 con 0x7f3480102280 2024-04-24T23:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.837+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34800fd720 con 0x7f3480102280 2024-04-24T23:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.838+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f347c010ac0 con 0x7f3480102280 2024-04-24T23:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.839+0000 7f34777fe640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f346c0a2b90 0x7f346c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.839+0000 7f3485ccf640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f346c0a2b90 0x7f346c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.839+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f347c0cfc20 con 0x7f3480102280 2024-04-24T23:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.839+0000 7f3485ccf640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f346c0a2b90 0x7f346c0a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3470005ab0 tx=0x7f3470000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:17.843+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f347c09c940 con 0x7f3480102280 2024-04-24T23:10:17.926 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:10:17 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: 2024-04-24T23:10:17.628+0000 7fbfa7362700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:10:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.029+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f348002d170 con 0x7f3480102280 2024-04-24T23:10:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.031+0000 7f34777fe640 1 -- 172.21.15.33:0/2677122398 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7f347c09c240 con 0x7f3480102280 2024-04-24T23:10:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:18.034 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f346c0a2b90 msgr2=0x7f346c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f346c0a2b90 0x7f346c0a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3470005ab0 tx=0x7f3470000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 msgr2=0x7f3480197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480197af0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f347c00e9f0 tx=0x7f347c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 shutdown_connections 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f346c0a2b90 0x7f346c0a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34801074e0 0x7f348019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3480104bb0 0x7f3480198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 --2- 172.21.15.33:0/2677122398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3480102280 0x7f3480197af0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 >> 172.21.15.33:0/2677122398 conn(0x7f34800fbc30 msgr2=0x7f3480102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 shutdown_connections 2024-04-24T23:10:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.033+0000 7f3488734640 1 -- 172.21.15.33:0/2677122398 wait complete. 2024-04-24T23:10:18.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:18.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3989601733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c41033f0 msgr2=0x7f06c4109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3989601733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c41033f0 0x7f06c4109c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009cd0 tx=0x7f06bc02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3989601733 shutdown_connections 2024-04-24T23:10:18.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3989601733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c41033f0 0x7f06c4109c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3989601733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4102a30 0x7f06c4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3989601733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c40ffa60 0x7f06c40ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3989601733 >> 172.21.15.33:0/3989601733 conn(0x7f06c40fb3b0 msgr2=0x7f06c40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3989601733 shutdown_connections 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.147+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3989601733 wait complete. 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 Processor -- start 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 -- start start 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c40ffa60 0x7f06c4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 0x7f06c4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c41033f0 0x7f06c419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06c410d430 con 0x7f06c4102a30 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06c410d2b0 con 0x7f06c40ffa60 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c9b5a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06c410d5b0 con 0x7f06c41033f0 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 0x7f06c4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 0x7f06c4197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45946/0 (socket says 172.21.15.33:45946) 2024-04-24T23:10:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c40ffa60 0x7f06c4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c2ffd640 1 -- 172.21.15.33:0/3873293362 learned_addr learned my addr 172.21.15.33:0/3873293362 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c3fff640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c41033f0 0x7f06c419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c2ffd640 1 -- 172.21.15.33:0/3873293362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c41033f0 msgr2=0x7f06c419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.148+0000 7f06c2ffd640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c41033f0 0x7f06c419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c2ffd640 1 -- 172.21.15.33:0/3873293362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c40ffa60 msgr2=0x7f06c4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c2ffd640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c40ffa60 0x7f06c4197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c2ffd640 1 -- 172.21.15.33:0/3873293362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c419f690 con 0x7f06c4102a30 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c3fff640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c41033f0 0x7f06c419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c2ffd640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 0x7f06c4197fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e9f0 tx=0x7f06b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800cd80 con 0x7f06c4102a30 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8010910 con 0x7f06c4102a30 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c419f980 con 0x7f06c4102a30 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8018990 con 0x7f06c4102a30 2024-04-24T23:10:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.149+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c419ff20 con 0x7f06c4102a30 2024-04-24T23:10:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.150+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c4108b60 con 0x7f06c4102a30 2024-04-24T23:10:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.151+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f06b8010ac0 con 0x7f06c4102a30 2024-04-24T23:10:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.152+0000 7f06c0ff9640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.152+0000 7f06c37fe640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.152+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f06b80cf9f0 con 0x7f06c4102a30 2024-04-24T23:10:18.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.152+0000 7f06c37fe640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.155+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f06b809c6b0 con 0x7f06c4102a30 2024-04-24T23:10:18.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.340+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06c402d3c0 con 0x7f06c4102a30 2024-04-24T23:10:18.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.342+0000 7f06c0ff9640 1 -- 172.21.15.33:0/3873293362 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7f06b809bfb0 con 0x7f06c4102a30 2024-04-24T23:10:18.344 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:18.344 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06ac0a2c10 msgr2=0x7f06ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b400b390 comp rx=0 tx=0).stop 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 msgr2=0x7f06c4197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 0x7f06c4197fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e9f0 tx=0x7f06b800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 shutdown_connections 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c41033f0 0x7f06c419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06c4102a30 0x7f06c4197fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 --2- 172.21.15.33:0/3873293362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c40ffa60 0x7f06c4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 >> 172.21.15.33:0/3873293362 conn(0x7f06c40fb3b0 msgr2=0x7f06c4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 shutdown_connections 2024-04-24T23:10:18.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.344+0000 7f06c9b5a640 1 -- 172.21.15.33:0/3873293362 wait complete. 2024-04-24T23:10:18.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 -- 172.21.15.33:0/1457647549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 msgr2=0x7fe02c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 --2- 172.21.15.33:0/1457647549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c102eb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009930 tx=0x7fe01c02a420 comp rx=0 tx=0).stop 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 -- 172.21.15.33:0/1457647549 shutdown_connections 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 --2- 172.21.15.33:0/1457647549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 0x7fe02c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 --2- 172.21.15.33:0/1457647549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c102eb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 --2- 172.21.15.33:0/1457647549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe02c0ffa60 0x7fe02c0ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 -- 172.21.15.33:0/1457647549 >> 172.21.15.33:0/1457647549 conn(0x7fe02c0fb3b0 msgr2=0x7fe02c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 -- 172.21.15.33:0/1457647549 shutdown_connections 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.472+0000 7fe03383f640 1 -- 172.21.15.33:0/1457647549 wait complete. 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 Processor -- start 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 -- start start 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe02c0ffa60 0x7fe02c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 0x7fe02c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c10d440 con 0x7fe02c102a30 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c10d2c0 con 0x7fe02c0ffa60 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe03383f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c10d5c0 con 0x7fe02c1033f0 2024-04-24T23:10:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45956/0 (socket says 172.21.15.33:45956) 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 -- 172.21.15.33:0/2442791397 learned_addr learned my addr 172.21.15.33:0/2442791397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe0315db640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe02c0ffa60 0x7fe02c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe031ddc640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 0x7fe02c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 -- 172.21.15.33:0/2442791397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 msgr2=0x7fe02c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 0x7fe02c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 -- 172.21.15.33:0/2442791397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe02c0ffa60 msgr2=0x7fe02c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe02c0ffa60 0x7fe02c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe030dda640 1 -- 172.21.15.33:0/2442791397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c19f700 con 0x7fe02c102a30 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.473+0000 7fe031ddc640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 0x7fe02c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.474+0000 7fe030dda640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c198020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005ab0 tx=0x7fe01c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.474+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c03f030 con 0x7fe02c102a30 2024-04-24T23:10:18.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.474+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01c0044d0 con 0x7fe02c102a30 2024-04-24T23:10:18.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.474+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02c19f990 con 0x7fe02c102a30 2024-04-24T23:10:18.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.474+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c035070 con 0x7fe02c102a30 2024-04-24T23:10:18.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.474+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02c19fe80 con 0x7fe02c102a30 2024-04-24T23:10:18.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.475+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe02c108b60 con 0x7fe02c102a30 2024-04-24T23:10:18.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.476+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe01c042030 con 0x7fe02c102a30 2024-04-24T23:10:18.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.477+0000 7fe0227fc640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe0180a2b90 0x7fe0180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.477+0000 7fe0315db640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe0180a2b90 0x7fe0180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.477+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe01c0ef7b0 con 0x7fe02c102a30 2024-04-24T23:10:18.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.477+0000 7fe0315db640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe0180a2b90 0x7fe0180a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe02c069ba0 tx=0x7fe028005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.481+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe01c0bc470 con 0x7fe02c102a30 2024-04-24T23:10:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:18 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1797878391' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:10:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:18 smithi038 ceph-mon[32450]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:10:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:18 smithi038 ceph-mon[32450]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:18 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1510418261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:18 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2677122398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:18 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3873293362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:18 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1797878391' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:10:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:18 smithi117 ceph-mon[32529]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:10:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:18 smithi117 ceph-mon[32529]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:18 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1510418261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:18 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2677122398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:18 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3873293362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1797878391' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:10:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:18 smithi033 ceph-mon[29201]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:10:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:18 smithi033 ceph-mon[29201]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1510418261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2677122398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3873293362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.706+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe02c062370 con 0x7fe02c102a30 2024-04-24T23:10:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.707+0000 7fe0227fc640 1 -- 172.21.15.33:0/2442791397 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7fe01c0bbd70 con 0x7fe02c102a30 2024-04-24T23:10:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:18.709 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24455,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:16.733281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24508,"qdb_cluster":[24508]},"id":1}]} 2024-04-24T23:10:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe0180a2b90 msgr2=0x7fe0180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe0180a2b90 0x7fe0180a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe02c069ba0 tx=0x7fe028005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 msgr2=0x7fe02c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c198020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005ab0 tx=0x7fe01c0042e0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 shutdown_connections 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe0180a2b90 0x7fe0180a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe02c1033f0 0x7fe02c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe02c102a30 0x7fe02c198020 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 --2- 172.21.15.33:0/2442791397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe02c0ffa60 0x7fe02c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 >> 172.21.15.33:0/2442791397 conn(0x7fe02c0fb3b0 msgr2=0x7fe02c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 shutdown_connections 2024-04-24T23:10:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.709+0000 7fe03383f640 1 -- 172.21.15.33:0/2442791397 wait complete. 2024-04-24T23:10:18.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:18.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 -- 172.21.15.33:0/3895018917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994101890 msgr2=0x7fc994101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 --2- 172.21.15.33:0/3895018917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994101890 0x7fc994101d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc988009900 tx=0x7fc98802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 -- 172.21.15.33:0/3895018917 shutdown_connections 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 --2- 172.21.15.33:0/3895018917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc994102250 0x7fc994108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 --2- 172.21.15.33:0/3895018917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994101890 0x7fc994101d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 --2- 172.21.15.33:0/3895018917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9941006a0 0x7fc994100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 -- 172.21.15.33:0/3895018917 >> 172.21.15.33:0/3895018917 conn(0x7fc9940fbc30 msgr2=0x7fc9940fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 -- 172.21.15.33:0/3895018917 shutdown_connections 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.822+0000 7fc999f90640 1 -- 172.21.15.33:0/3895018917 wait complete. 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 Processor -- start 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 -- start start 2024-04-24T23:10:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9941006a0 0x7fc994197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc994101890 0x7fc994198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 0x7fc99419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc99410c320 con 0x7fc994102250 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc99410c1a0 con 0x7fc9941006a0 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc999f90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc99410c4a0 con 0x7fc994101890 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc993fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 0x7fc99419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc993fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 0x7fc99419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45964/0 (socket says 172.21.15.33:45964) 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc993fff640 1 -- 172.21.15.33:0/1006565140 learned_addr learned my addr 172.21.15.33:0/1006565140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc9937fe640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9941006a0 0x7fc994197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc992ffd640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc994101890 0x7fc994198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc993fff640 1 -- 172.21.15.33:0/1006565140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc994101890 msgr2=0x7fc994198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc993fff640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc994101890 0x7fc994198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.823+0000 7fc993fff640 1 -- 172.21.15.33:0/1006565140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9941006a0 msgr2=0x7fc994197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc993fff640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9941006a0 0x7fc994197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc993fff640 1 -- 172.21.15.33:0/1006565140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99419f700 con 0x7fc994102250 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc992ffd640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc994101890 0x7fc994198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc993fff640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 0x7fc99419f090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00eb10 tx=0x7fc98c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c004b20 con 0x7fc994102250 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c004cc0 con 0x7fc994102250 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c010760 con 0x7fc994102250 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99419f9f0 con 0x7fc994102250 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.824+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99419fec0 con 0x7fc994102250 2024-04-24T23:10:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.825+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9941a0150 con 0x7fc994102250 2024-04-24T23:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.826+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc98c005020 con 0x7fc994102250 2024-04-24T23:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.827+0000 7fc990ff9640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc97c0a2b90 0x7fc97c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.827+0000 7fc9937fe640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc97c0a2b90 0x7fc97c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.827+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc98c0cf940 con 0x7fc994102250 2024-04-24T23:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.827+0000 7fc9937fe640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc97c0a2b90 0x7fc97c0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc984005fd0 tx=0x7fc98400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:18.831+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc98c09c600 con 0x7fc994102250 2024-04-24T23:10:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.016+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9941a03c0 con 0x7fc994102250 2024-04-24T23:10:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.018+0000 7fc990ff9640 1 -- 172.21.15.33:0/1006565140 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fc98c09bf00 con 0x7fc994102250 2024-04-24T23:10:19.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:19.020 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc97c0a2b90 msgr2=0x7fc97c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc97c0a2b90 0x7fc97c0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc984005fd0 tx=0x7fc98400b390 comp rx=0 tx=0).stop 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 msgr2=0x7fc99419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 0x7fc99419f090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00eb10 tx=0x7fc98c00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 shutdown_connections 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc97c0a2b90 0x7fc97c0a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc994102250 0x7fc99419f090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc994101890 0x7fc994198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 --2- 172.21.15.33:0/1006565140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9941006a0 0x7fc994197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 >> 172.21.15.33:0/1006565140 conn(0x7fc9940fbc30 msgr2=0x7fc994069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 shutdown_connections 2024-04-24T23:10:19.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.020+0000 7fc999f90640 1 -- 172.21.15.33:0/1006565140 wait complete. 2024-04-24T23:10:19.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:10:11.664486+0000', 'modified': '2024-04-24T23:10:16.733281+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/851112851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 851112851}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 851112851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24508, 'qdb_cluster': [24508]} 2024-04-24T23:10:19.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:10:19.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:10:19.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 -- 172.21.15.33:0/542977151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec103350 msgr2=0x7fe2ec109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 --2- 172.21.15.33:0/542977151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec103350 0x7fe2ec109be0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009ca0 tx=0x7fe2e402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 -- 172.21.15.33:0/542977151 shutdown_connections 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 --2- 172.21.15.33:0/542977151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec103350 0x7fe2ec109be0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 --2- 172.21.15.33:0/542977151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2ec102990 0x7fe2ec102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 --2- 172.21.15.33:0/542977151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 -- 172.21.15.33:0/542977151 >> 172.21.15.33:0/542977151 conn(0x7fe2ec0fcd10 msgr2=0x7fe2ec0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.135+0000 7fe2f2934640 1 -- 172.21.15.33:0/542977151 shutdown_connections 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 -- 172.21.15.33:0/542977151 wait complete. 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 Processor -- start 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 -- start start 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec102990 0x7fe2ec073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2ec103350 0x7fe2ec074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec10d400 con 0x7fe2ec1017a0 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec10d280 con 0x7fe2ec103350 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2f2934640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec10d580 con 0x7fe2ec102990 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2ebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.136+0000 7fe2ebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45974/0 (socket says 172.21.15.33:45974) 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 -- 172.21.15.33:0/2119200427 learned_addr learned my addr 172.21.15.33:0/2119200427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2ec103350 0x7fe2ec074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec102990 0x7fe2ec073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 -- 172.21.15.33:0/2119200427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec102990 msgr2=0x7fe2ec073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec102990 0x7fe2ec073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 -- 172.21.15.33:0/2119200427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2ec103350 msgr2=0x7fe2ec074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2ec103350 0x7fe2ec074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 -- 172.21.15.33:0/2119200427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec074900 con 0x7fe2ec1017a0 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec0754f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c970 tx=0x7fe2d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2e97fa640 1 -- 172.21.15.33:0/2119200427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4004220 con 0x7fe2ec1017a0 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2e97fa640 1 -- 172.21.15.33:0/2119200427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d40043c0 con 0x7fe2ec1017a0 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2e97fa640 1 -- 172.21.15.33:0/2119200427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400f450 con 0x7fe2ec1017a0 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec1a1b40 con 0x7fe2ec1017a0 2024-04-24T23:10:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.137+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec1a20e0 con 0x7fe2ec1017a0 2024-04-24T23:10:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.139+0000 7fe2e97fa640 1 -- 172.21.15.33:0/2119200427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe2d40079b0 con 0x7fe2ec1017a0 2024-04-24T23:10:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 Processor -- start 2024-04-24T23:10:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 -- start start 2024-04-24T23:10:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x560718376860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x56071837b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x56071837e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607182d9f00 con 0x56071837bfa0 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560718282750 con 0x560718376e30 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f5642740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607182d5ec0 con 0x560718376440 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2e97fa640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe2d80a2c10 0x7fe2d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x56071837e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x56071837e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45984/0 (socket says 172.21.15.33:45984) 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 -- 172.21.15.33:0/2387379069 learned_addr learned my addr 172.21.15.33:0/2387379069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x560718376860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x56071837b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe2d80a2c10 0x7fe2d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2e97fa640 1 -- 172.21.15.33:0/2119200427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe2d40cef70 con 0x7fe2ec1017a0 2024-04-24T23:10:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 -- 172.21.15.33:0/2387379069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 msgr2=0x560718376860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x560718376860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 -- 172.21.15.33:0/2387379069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 msgr2=0x56071837b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x56071837b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.140+0000 7fe2eb7fe640 1 -- 172.21.15.33:0/2387379069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560717fee2d0 con 0x56071837bfa0 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x56071837e430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009c90 tx=0x7fe2dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3212785e6cefdd7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe2d80a2c10 0x7fe2d80a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009430 tx=0x7fe2e4005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2e27fc640 1 -- 172.21.15.33:0/2387379069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc01e070 con 0x56071837bfa0 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2e27fc640 1 -- 172.21.15.33:0/2387379069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc019b10 con 0x56071837bfa0 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2e27fc640 1 -- 172.21.15.33:0/2387379069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00ea50 con 0x56071837bfa0 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 -- 172.21.15.33:0/2387379069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 msgr2=0x56071837e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x56071837e430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009c90 tx=0x7fe2dc011810 comp rx=0 tx=0).stop 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 -- 172.21.15.33:0/2387379069 shutdown_connections 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x56071837e430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x56071837b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2387379069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x560718376860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.141+0000 7fe2f5642740 1 -- 172.21.15.33:0/2387379069 >> 172.21.15.33:0/2387379069 conn(0x560718371ab0 msgr2=0x560718371eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 -- 172.21.15.33:0/2387379069 shutdown_connections 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 -- 172.21.15.33:0/2387379069 wait complete. 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 -- start start 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 Processor -- start 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56071837ef50 con 0x56071837bfa0 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560718295cf0 con 0x560718376e30 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56071837f0d0 con 0x560718376440 2024-04-24T23:10:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f5642740 10 client.0 _ll_register_callbacks cb 0x5607182956e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f0ed1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f0ed1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45986/0 (socket says 172.21.15.33:45986) 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/262450029 learned_addr learned my addr 172.21.15.33:0/262450029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.142+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 msgr2=0x5607183807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/262450029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 msgr2=0x560718380d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/262450029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607180b86b0 con 0x56071837bfa0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2e402f540 tx=0x7fe2e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cee49396c8456e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/262450029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40309d0 con 0x56071837bfa0 2024-04-24T23:10:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/262450029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe2e4030b70 con 0x56071837bfa0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2dc019b10 con 0x56071837bfa0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/262450029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe2e4030d70 con 0x56071837bfa0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f5642740 10 client.14754 fetch_fsmap learned FSMap version 17 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56071838ad30 con 0x56071837bfa0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.143+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/262450029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc019b10 con 0x56071837bfa0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/262450029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fe2e4048040 con 0x56071837bfa0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2f5642740 10 client.14754 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2f5642740 10 client.14754 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2f5642740 20 client.14754 populate_metadata read hostname 'smithi033' 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2f5642740 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56071838d4f0 0x56071838e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2f5642740 4 client.14754 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607182e7440 con 0x56071838d4f0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56071838d4f0 0x56071838e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2cb7fe640 10 client.14754 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.144+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56071838d4f0 0x56071838e960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3044e51f356651d server_cookie=efb7fcf971a694a6 in_seq=0 out_seq=0 2024-04-24T23:10:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.145+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/262450029 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607182e7440 con 0x56071838d4f0 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.145+0000 7fe2cb7fe640 10 client.14754 handle_command_reply: tid=0 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.145+0000 7fe2f5642740 1 client.14754 shutdown 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 msgr2=0x560718387d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2e402f540 tx=0x7fe2e40095c0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 shutdown_connections 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56071838d4f0 0x56071838e960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 --2- 172.21.15.33:0/262450029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2cb7fe640 0 client.14754 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 >> 172.21.15.33:0/262450029 conn(0x560718371ab0 msgr2=0x560718370ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 shutdown_connections 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 1 -- 172.21.15.33:0/262450029 wait complete. 2024-04-24T23:10:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.146+0000 7fe2f5642740 20 client.14754 trim_cache size 0 max 16384 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 Processor -- start 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 -- start start 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56071838cca0 con 0x56071837bfa0 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56071838d370 con 0x560718376e30 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f5642740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56071838da40 con 0x560718376440 2024-04-24T23:10:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45990/0 (socket says 172.21.15.33:45990) 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/2076692907 learned_addr learned my addr 172.21.15.33:0/2076692907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/2076692907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 msgr2=0x5607183807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/2076692907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 msgr2=0x560718380d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 -- 172.21.15.33:0/2076692907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071836ffc0 con 0x56071837bfa0 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.164+0000 7fe2f0ed1640 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4031ea0 tx=0x7fe2e40319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25eca2650db8ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2e1ffb640 1 -- 172.21.15.33:0/2076692907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4004d40 con 0x56071837bfa0 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2e1ffb640 1 -- 172.21.15.33:0/2076692907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e4036070 con 0x56071837bfa0 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2e1ffb640 1 -- 172.21.15.33:0/2076692907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e403bb90 con 0x56071837bfa0 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 -- 172.21.15.33:0/2076692907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 msgr2=0x560718387d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4031ea0 tx=0x7fe2e40319d0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 -- 172.21.15.33:0/2076692907 shutdown_connections 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56071837bfa0 0x560718387d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560718376e30 0x560718380d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 --2- 172.21.15.33:0/2076692907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376440 0x5607183807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 -- 172.21.15.33:0/2076692907 >> 172.21.15.33:0/2076692907 conn(0x560718371ab0 msgr2=0x560718377640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 -- 172.21.15.33:0/2076692907 shutdown_connections 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 -- 172.21.15.33:0/2076692907 wait complete. 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.165+0000 7fe2f5642740 1 -- start start 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 Processor -- start 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 0x560718385d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376e30 0x5607183ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56071837bfa0 0x5607183f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56071838e7e0 con 0x560718376440 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56071838e110 con 0x56071837bfa0 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56071838eeb0 con 0x560718376e30 2024-04-24T23:10:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2f5642740 10 client.0 _ll_register_callbacks cb 0x7fe2e402f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 0x560718385d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 0x560718385d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46000/0 (socket says 172.21.15.33:46000) 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 -- 172.21.15.33:0/1251262550 learned_addr learned my addr 172.21.15.33:0/1251262550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 -- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376e30 msgr2=0x5607183ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376e30 0x5607183ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376e30 0x5607183ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 -- 172.21.15.33:0/1251262550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56071837bfa0 msgr2=0x5607183f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56071837bfa0 0x5607183f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 -- 172.21.15.33:0/1251262550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607183f1f60 con 0x560718376440 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376e30 0x5607183ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2ebfff640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 0x560718385d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4097a40 tx=0x7fe2d4008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63460b9e8a332564 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:10:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.166+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/1251262550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40bb040 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/1251262550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe2d4003610 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e4036070 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/1251262550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe2d409b5e0 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 10 client.14760 fetch_fsmap learned FSMap version 17 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5607183f3330 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/1251262550 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e4036070 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/1251262550 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fe2d4007ca0 con 0x560718376440 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 10 client.14760 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 10 client.14760 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 20 client.14760 populate_metadata read hostname 'smithi033' 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607183f7ef0 0x5607183f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 4 client.14760 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "status"}) -- 0x5607182e1790 con 0x5607183f7ef0 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.167+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607183f7ef0 0x5607183f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2cb7fe640 10 client.14760 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2eb7fe640 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607183f7ef0 0x5607183f6c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a823165f0e99ce1e server_cookie=c05bc2a28f22c046 in_seq=0 out_seq=0 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2cb7fe640 1 -- 172.21.15.33:0/1251262550 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5607182e1790 con 0x5607183f7ef0 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2cb7fe640 10 client.14760 handle_command_reply: tid=0 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 client.14760 shutdown 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 msgr2=0x560718385d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 0x560718385d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4097a40 tx=0x7fe2d4008ec0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 shutdown_connections 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607183f7ef0 0x5607183f6c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56071837bfa0 0x5607183f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2cb7fe640 0 client.14760 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560718376e30 0x5607183ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 --2- 172.21.15.33:0/1251262550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560718376440 0x560718385d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 >> 172.21.15.33:0/1251262550 conn(0x560718371ab0 msgr2=0x5607183807d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 shutdown_connections 2024-04-24T23:10:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 1 -- 172.21.15.33:0/1251262550 wait complete. 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.168+0000 7fe2f5642740 20 client.14760 trim_cache size 0 max 16384 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24508, 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.8387868980000004, 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 17, 2024-04-24T23:10:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 90, 2024-04-24T23:10:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T23:10:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 21.977193850999999 2024-04-24T23:10:19.178 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:10:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe2d80a2c10 msgr2=0x7fe2d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe2d80a2c10 0x7fe2d80a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009430 tx=0x7fe2e4005d80 comp rx=0 tx=0).stop 2024-04-24T23:10:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 msgr2=0x7fe2ec0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec0754f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c970 tx=0x7fe2d400ce30 comp rx=0 tx=0).stop 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 shutdown_connections 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe2d80a2c10 0x7fe2d80a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2ec103350 0x7fe2ec074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec102990 0x7fe2ec073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 --2- 172.21.15.33:0/2119200427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2ec1017a0 0x7fe2ec0754f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 >> 172.21.15.33:0/2119200427 conn(0x7fe2ec0fcd10 msgr2=0x7fe2ec105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 shutdown_connections 2024-04-24T23:10:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:19.169+0000 7fe2f2934640 1 -- 172.21.15.33:0/2119200427 wait complete. 2024-04-24T23:10:19.182 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:10:19.182 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:19.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:10:19.427 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:19 smithi038 ceph-mon[32450]: pgmap v296: 129 pgs: 102 active+clean, 16 creating+peering, 11 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T23:10:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:19 smithi038 ceph-mon[32450]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:10:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:19 smithi038 ceph-mon[32450]: Cluster is now healthy 2024-04-24T23:10:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:19 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2442791397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:19.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:19 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1006565140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:19 smithi117 ceph-mon[32529]: pgmap v296: 129 pgs: 102 active+clean, 16 creating+peering, 11 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T23:10:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:19 smithi117 ceph-mon[32529]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:10:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:19 smithi117 ceph-mon[32529]: Cluster is now healthy 2024-04-24T23:10:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:19 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2442791397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:19 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1006565140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:19.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:19 smithi033 ceph-mon[29201]: pgmap v296: 129 pgs: 102 active+clean, 16 creating+peering, 11 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T23:10:19.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:19 smithi033 ceph-mon[29201]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:10:19.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:19 smithi033 ceph-mon[29201]: Cluster is now healthy 2024-04-24T23:10:19.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:19 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2442791397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:19.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:19 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1006565140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/2599350137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 msgr2=0x7f8fe010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/2599350137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe010f0e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b0a0 tx=0x7f8fd800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/2599350137 shutdown_connections 2024-04-24T23:10:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/2599350137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe010f0e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/2599350137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe01082a0 0x7f8fe0108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/2599350137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 0x7f8fe006ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/2599350137 >> 172.21.15.33:0/2599350137 conn(0x7f8fe00780a0 msgr2=0x7f8fe00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/2599350137 shutdown_connections 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.266+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/2599350137 wait complete. 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 Processor -- start 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 -- start start 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 0x7f8fe01011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe0101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe01082a0 0x7f8fe0101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0112520 con 0x7f8fe006b170 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01123a0 con 0x7f8fe006a7d0 2024-04-24T23:10:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe6a3e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01126a0 con 0x7f8fe01082a0 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe0101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe0101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46024/0 (socket says 172.21.15.33:46024) 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 -- 172.21.15.33:0/3743423649 learned_addr learned my addr 172.21.15.33:0/3743423649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fe4fdb700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe01082a0 0x7f8fe0101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdffff700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 0x7f8fe01011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 -- 172.21.15.33:0/3743423649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe01082a0 msgr2=0x7f8fe0101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe01082a0 0x7f8fe0101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 -- 172.21.15.33:0/3743423649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 msgr2=0x7f8fe01011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 0x7f8fe01011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdf7fe700 1 -- 172.21.15.33:0/3743423649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe01a6f20 con 0x7f8fe006b170 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.267+0000 7f8fdffff700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 0x7f8fe01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.268+0000 7f8fdf7fe700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe0101730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000d980 tx=0x7f8fd000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.268+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0016070 con 0x7f8fe006b170 2024-04-24T23:10:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.268+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe01a7210 con 0x7f8fe006b170 2024-04-24T23:10:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.268+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd0004500 con 0x7f8fe006b170 2024-04-24T23:10:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.268+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0004d10 con 0x7f8fe006b170 2024-04-24T23:10:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.268+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe01a77a0 con 0x7f8fe006b170 2024-04-24T23:10:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.269+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe0068f20 con 0x7f8fe006b170 2024-04-24T23:10:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.270+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8fd000bbf0 con 0x7f8fe006b170 2024-04-24T23:10:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.271+0000 7f8fdd7fa700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8fcc0a2c20 0x7f8fcc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.271+0000 7f8fdffff700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8fcc0a2c20 0x7f8fcc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.271+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8fd00cf5e0 con 0x7f8fe006b170 2024-04-24T23:10:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.271+0000 7f8fdffff700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8fcc0a2c20 0x7f8fcc0a50e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005fd0 tx=0x7f8fc8005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.274+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8fd001f020 con 0x7f8fe006b170 2024-04-24T23:10:21.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:21 smithi033 ceph-mon[29201]: pgmap v297: 129 pgs: 105 active+clean, 16 creating+peering, 8 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-24T23:10:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.536+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8fe0102720 con 0x7f8fe006b170 2024-04-24T23:10:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.536+0000 7f8fdd7fa700 1 -- 172.21.15.33:0/3743423649 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f8fd009c300 con 0x7f8fe006b170 2024-04-24T23:10:21.538 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:10:21.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.538+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8fcc0a2c20 msgr2=0x7f8fcc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.538+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8fcc0a2c20 0x7f8fcc0a50e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005fd0 tx=0x7f8fc8005d40 comp rx=0 tx=0).stop 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.538+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 msgr2=0x7f8fe0101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.538+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe0101730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000d980 tx=0x7f8fd000dd40 comp rx=0 tx=0).stop 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 shutdown_connections 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8fcc0a2c20 0x7f8fcc0a50e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe01082a0 0x7f8fe0101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe006a7d0 0x7f8fe01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 --2- 172.21.15.33:0/3743423649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe006b170 0x7f8fe0101730 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 >> 172.21.15.33:0/3743423649 conn(0x7f8fe00780a0 msgr2=0x7f8fe00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 shutdown_connections 2024-04-24T23:10:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:21.539+0000 7f8fe6a3e700 1 -- 172.21.15.33:0/3743423649 wait complete. 2024-04-24T23:10:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:21 smithi038 ceph-mon[32450]: pgmap v297: 129 pgs: 105 active+clean, 16 creating+peering, 8 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-24T23:10:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:21 smithi117 ceph-mon[32529]: pgmap v297: 129 pgs: 105 active+clean, 16 creating+peering, 8 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:10:22.183 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:10:22.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:22.184 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-24T23:10:22.201 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:10:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:10:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:10:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-24T23:10:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42193sec preferred_lft 42193sec 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:22.202 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-24T23:10:22.203 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:10:22.204 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:10:22.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:22.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:22.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:22.398 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:10:22.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:22 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3743423649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:10:22.424 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-24T23:10:22.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:22.425 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-24T23:10:22.494 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-24T23:10:22.494 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:10:22.494 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:10:22.494 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:10:22.494 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:10:22.495 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:22.495 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:10:22.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:10:22.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:10:22.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:10:22.495 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:10:22.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:22.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:22 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3743423649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:10:22.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:22 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3743423649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:10:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:22.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:22.827 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-24T23:10:22.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:22.843 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-24T23:10:22.898 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:22.898 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:10:22.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:10:22.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T23:10:22.898 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:10:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:23.016 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T23:10:23.016 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:10:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:23.143 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:23.143 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:10:23.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-24T23:10:23.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:10:23.143 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:10:23.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:23.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:23.220 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T23:10:23.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:10:23.220 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:23.236 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:10:23.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:10:23.236 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:23.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-24T23:10:23.357 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:10:23.413 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-24T23:10:23.413 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-24T23:10:23.413 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-24T23:10:23.414 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/e71b59a641a0573ff6cabb90b8208c7ba2eb258b8143ca24b20979b546f65024/merged 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/09aaa6386921e4aa0ecb96948f13aceaee0d7eae0cd1c355451a89339464c937/merged 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-24T23:10:23.415 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/0d29673eb424a3b6e537ea3ab41e8eddf02cedba787c7d8ed242c5f7a21bd5ad/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/17530573a21bfce105c28ef968b0208b335ea4b75361965cf85132965a250afc/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/44c72524b7158b13e49cf1175725e959c31adab260ebe796305e0f979a8e98ff/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/152859d89afd9bc93cec717e1c24873d573e33287fd9cf4df93f68e44479c957/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/f9dcff2bf0e58ef2b6fadcc13afce333981d64bda17f6b3fed2b64b7b72e4562/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/40a11e2a23289876391f66fb5ecf014643fb91ca39d46f3bbdde628e9a2548a8/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a76baa72cbeb6f76b530c07be9c05d00915da86892407d4ef46f2b790e2ea960/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b410df17997e05a872876e91852d90b61231f9c6ec84f23f6937171913f81970/merged 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:10:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:10:23.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:23.417 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-24T23:10:23.468 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:10:23.469 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T23:10:23.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-24T23:10:23.535 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:10:23.591 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-24T23:10:23.591 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-24T23:10:23.591 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-24T23:10:23.591 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-24T23:10:23.592 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/e71b59a641a0573ff6cabb90b8208c7ba2eb258b8143ca24b20979b546f65024/merged 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/09aaa6386921e4aa0ecb96948f13aceaee0d7eae0cd1c355451a89339464c937/merged 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-24T23:10:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/0d29673eb424a3b6e537ea3ab41e8eddf02cedba787c7d8ed242c5f7a21bd5ad/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/17530573a21bfce105c28ef968b0208b335ea4b75361965cf85132965a250afc/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/44c72524b7158b13e49cf1175725e959c31adab260ebe796305e0f979a8e98ff/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/152859d89afd9bc93cec717e1c24873d573e33287fd9cf4df93f68e44479c957/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/f9dcff2bf0e58ef2b6fadcc13afce333981d64bda17f6b3fed2b64b7b72e4562/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/40a11e2a23289876391f66fb5ecf014643fb91ca39d46f3bbdde628e9a2548a8/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a76baa72cbeb6f76b530c07be9c05d00915da86892407d4ef46f2b790e2ea960/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b410df17997e05a872876e91852d90b61231f9c6ec84f23f6937171913f81970/merged 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:10:23.594 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:10:23.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:23.595 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-24T23:10:23.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:2024-04-24T23:10:23.628+0000 7fb3b4e3c480 -1 init, newargv = 0x562a37d1c240 newargc=15 2024-04-24T23:10:23.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[73410]: starting ceph client 2024-04-24T23:10:23.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[73410]: starting fuse 2024-04-24T23:10:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:115 2024-04-24T23:10:23.645 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-24T23:10:23.646 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-24T23:10:23.647 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.admin")) 2024-04-24T23:10:23.648 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-24T23:10:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:23 smithi038 ceph-mon[32450]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-24T23:10:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:23 smithi117 ceph-mon[32529]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-24T23:10:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:23 smithi033 ceph-mon[29201]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-24T23:10:23.764 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.admin.73410.asok 2024-04-24T23:10:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:23.775 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73410.asok 2024-04-24T23:10:23.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:23.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73410.asok status 2024-04-24T23:10:23.911 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:10:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-24T23:10:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "admin", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "73410", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 14772, 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-24T23:10:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 14772 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 175307585 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 175307585 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:10:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.14772 192.168.0.1:0/175307585", 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/175307585", 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 17, 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 90, 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-24T23:10:23.914 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:10:23.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:23.920 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-24T23:10:23.920 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:23.935 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-24T23:10:23.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:23.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:23.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:24.001 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T23:10:24.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:10:24.114 INFO:teuthology.orchestra.run.smithi033.stdout:# file: . 2024-04-24T23:10:24.114 INFO:teuthology.orchestra.run.smithi033.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T23:10:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:24.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:24.121 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:10:24.121 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:24.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:24.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 -- 172.21.15.33:0/3804209069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 msgr2=0x7fce80108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 --2- 172.21.15.33:0/3804209069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce80108ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce78009ca0 tx=0x7fce7802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 -- 172.21.15.33:0/3804209069 shutdown_connections 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 --2- 172.21.15.33:0/3804209069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce80108ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 --2- 172.21.15.33:0/3804209069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce80101890 0x7fce80101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 --2- 172.21.15.33:0/3804209069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce801006a0 0x7fce80100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 -- 172.21.15.33:0/3804209069 >> 172.21.15.33:0/3804209069 conn(0x7fce800fbc30 msgr2=0x7fce800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 -- 172.21.15.33:0/3804209069 shutdown_connections 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 -- 172.21.15.33:0/3804209069 wait complete. 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 Processor -- start 2024-04-24T23:10:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.278+0000 7fce87d63640 1 -- start start 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce801006a0 0x7fce801968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce80101890 0x7fce80196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce8019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c300 con 0x7fce80102250 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c180 con 0x7fce80101890 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c480 con 0x7fce801006a0 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce8019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce8019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46036/0 (socket says 172.21.15.33:46036) 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 -- 172.21.15.33:0/2077376578 learned_addr learned my addr 172.21.15.33:0/2077376578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce852fe640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce80101890 0x7fce80196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce85aff640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce801006a0 0x7fce801968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 -- 172.21.15.33:0/2077376578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce801006a0 msgr2=0x7fce801968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce801006a0 0x7fce801968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 -- 172.21.15.33:0/2077376578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce80101890 msgr2=0x7fce80196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce80101890 0x7fce80196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 -- 172.21.15.33:0/2077376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8019e500 con 0x7fce80102250 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce85aff640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce801006a0 0x7fce801968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce86300640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce8019de90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce78005e30 tx=0x7fce78031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78003c30 con 0x7fce80102250 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce78003dd0 con 0x7fce80102250 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7802fde0 con 0x7fce80102250 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8019e790 con 0x7fce80102250 2024-04-24T23:10:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.279+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8019ece0 con 0x7fce80102250 2024-04-24T23:10:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.280+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80107960 con 0x7fce80102250 2024-04-24T23:10:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.281+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce78004050 con 0x7fce80102250 2024-04-24T23:10:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.282+0000 7fce76ffd640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce6c0a2b90 0x7fce6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.282+0000 7fce85aff640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce6c0a2b90 0x7fce6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.282+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fce780f0760 con 0x7fce80102250 2024-04-24T23:10:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.283+0000 7fce85aff640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce6c0a2b90 0x7fce6c0a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005d00 tx=0x7fce7c005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.286+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce780bd480 con 0x7fce80102250 2024-04-24T23:10:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.477+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce8002d340 con 0x7fce80102250 2024-04-24T23:10:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.479+0000 7fce76ffd640 1 -- 172.21.15.33:0/2077376578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fce780bcd80 con 0x7fce80102250 2024-04-24T23:10:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:24.481 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce6c0a2b90 msgr2=0x7fce6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce6c0a2b90 0x7fce6c0a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005d00 tx=0x7fce7c005ef0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 msgr2=0x7fce8019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce8019de90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce78005e30 tx=0x7fce78031ee0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 shutdown_connections 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce6c0a2b90 0x7fce6c0a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce80102250 0x7fce8019de90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce80101890 0x7fce80196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 --2- 172.21.15.33:0/2077376578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce801006a0 0x7fce801968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 >> 172.21.15.33:0/2077376578 conn(0x7fce800fbc30 msgr2=0x7fce80104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 shutdown_connections 2024-04-24T23:10:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.481+0000 7fce87d63640 1 -- 172.21.15.33:0/2077376578 wait complete. 2024-04-24T23:10:24.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:10:24.494 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:24.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 -- 172.21.15.33:0/878825902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 msgr2=0x7f0e3c104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 --2- 172.21.15.33:0/878825902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c104610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009900 tx=0x7f0e2c02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 -- 172.21.15.33:0/878825902 shutdown_connections 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 --2- 172.21.15.33:0/878825902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c107480 0x7f0e3c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 --2- 172.21.15.33:0/878825902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 0x7f0e3c106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 --2- 172.21.15.33:0/878825902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c104610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.597+0000 7f0e40b14640 1 -- 172.21.15.33:0/878825902 >> 172.21.15.33:0/878825902 conn(0x7f0e3c0fbc30 msgr2=0x7f0e3c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.598+0000 7f0e40b14640 1 -- 172.21.15.33:0/878825902 shutdown_connections 2024-04-24T23:10:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.598+0000 7f0e40b14640 1 -- 172.21.15.33:0/878825902 wait complete. 2024-04-24T23:10:24.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.598+0000 7f0e40b14640 1 Processor -- start 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.598+0000 7f0e40b14640 1 -- start start 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.598+0000 7f0e40b14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 0x7f0e3c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c107480 0x7f0e3c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c10d420 con 0x7f0e3c102220 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c10d2a0 con 0x7f0e3c104b50 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c10d5a0 con 0x7f0e3c107480 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46060/0 (socket says 172.21.15.33:46060) 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 -- 172.21.15.33:0/2219645116 learned_addr learned my addr 172.21.15.33:0/2219645116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e39d9b640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 0x7f0e3c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3ad9d640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c107480 0x7f0e3c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 -- 172.21.15.33:0/2219645116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c107480 msgr2=0x7f0e3c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c107480 0x7f0e3c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 -- 172.21.15.33:0/2219645116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 msgr2=0x7f0e3c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 0x7f0e3c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 -- 172.21.15.33:0/2219645116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c19f660 con 0x7f0e3c102220 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e39d9b640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 0x7f0e3c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e3a59c640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c197a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009230 tx=0x7f0e2c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c03f030 con 0x7f0e3c102220 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c003ce0 con 0x7f0e3c102220 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c19f8f0 con 0x7f0e3c102220 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c035070 con 0x7f0e3c102220 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.599+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c19fe30 con 0x7f0e3c102220 2024-04-24T23:10:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.600+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c0fd720 con 0x7f0e3c102220 2024-04-24T23:10:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.601+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0e2c042030 con 0x7f0e3c102220 2024-04-24T23:10:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.602+0000 7f0e2b7fe640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e240a2c10 0x7f0e240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.602+0000 7f0e39d9b640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e240a2c10 0x7f0e240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.602+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0e2c0efcd0 con 0x7f0e3c102220 2024-04-24T23:10:24.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.603+0000 7f0e39d9b640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e240a2c10 0x7f0e240a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c198f50 tx=0x7f0e3000a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.607+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e2c0bc9f0 con 0x7f0e3c102220 2024-04-24T23:10:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.791+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e3c0623e0 con 0x7f0e3c102220 2024-04-24T23:10:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.792+0000 7f0e2b7fe640 1 -- 172.21.15.33:0/2219645116 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f0e2c0bc2f0 con 0x7f0e3c102220 2024-04-24T23:10:24.794 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e240a2c10 msgr2=0x7f0e240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e240a2c10 0x7f0e240a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c198f50 tx=0x7f0e3000a5c0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 msgr2=0x7f0e3c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c197a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009230 tx=0x7f0e2c004360 comp rx=0 tx=0).stop 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 shutdown_connections 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e240a2c10 0x7f0e240a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c107480 0x7f0e3c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e3c104b50 0x7f0e3c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 --2- 172.21.15.33:0/2219645116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e3c102220 0x7f0e3c197a40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 >> 172.21.15.33:0/2219645116 conn(0x7f0e3c0fbc30 msgr2=0x7f0e3c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 shutdown_connections 2024-04-24T23:10:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.794+0000 7f0e40b14640 1 -- 172.21.15.33:0/2219645116 wait complete. 2024-04-24T23:10:24.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:10:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 -- 172.21.15.33:0/1587125645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96181006a0 msgr2=0x7f9618100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 --2- 172.21.15.33:0/1587125645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96181006a0 0x7f9618100ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9608009900 tx=0x7f960802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 -- 172.21.15.33:0/1587125645 shutdown_connections 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 --2- 172.21.15.33:0/1587125645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9618102250 0x7f9618108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 --2- 172.21.15.33:0/1587125645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9618101890 0x7f9618101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 --2- 172.21.15.33:0/1587125645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96181006a0 0x7f9618100ac0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 -- 172.21.15.33:0/1587125645 >> 172.21.15.33:0/1587125645 conn(0x7f96180fbc30 msgr2=0x7f96180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 -- 172.21.15.33:0/1587125645 shutdown_connections 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 -- 172.21.15.33:0/1587125645 wait complete. 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.911+0000 7f961dba9640 1 Processor -- start 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 -- start start 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96181006a0 0x7f9618197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9618101890 0x7f9618198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 0x7f961819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f961810d440 con 0x7f9618102250 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f961810d2c0 con 0x7f9618101890 2024-04-24T23:10:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f961dba9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f961810d5c0 con 0x7f96181006a0 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 0x7f961819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 0x7f961819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46080/0 (socket says 172.21.15.33:46080) 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 -- 172.21.15.33:0/1665911338 learned_addr learned my addr 172.21.15.33:0/1665911338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f96177fe640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96181006a0 0x7f9618197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9616ffd640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9618101890 0x7f9618198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 -- 172.21.15.33:0/1665911338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96181006a0 msgr2=0x7f9618197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96181006a0 0x7f9618197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 -- 172.21.15.33:0/1665911338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9618101890 msgr2=0x7f9618198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9618101890 0x7f9618198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.912+0000 7f9617fff640 1 -- 172.21.15.33:0/1665911338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961819f740 con 0x7f9618102250 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.913+0000 7f9617fff640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 0x7f961819f0d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f961000eaa0 tx=0x7f961000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.913+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961000cdb0 con 0x7f9618102250 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.913+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9610005020 con 0x7f9618102250 2024-04-24T23:10:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.913+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961819fa30 con 0x7f9618102250 2024-04-24T23:10:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.913+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100116b0 con 0x7f9618102250 2024-04-24T23:10:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.913+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961819ff30 con 0x7f9618102250 2024-04-24T23:10:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.914+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96181079e0 con 0x7f9618102250 2024-04-24T23:10:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.915+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9610005560 con 0x7f9618102250 2024-04-24T23:10:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.916+0000 7f9614ff9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96000a2b90 0x7f96000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.916+0000 7f96177fe640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96000a2b90 0x7f96000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.916+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f96100cf9d0 con 0x7f9618102250 2024-04-24T23:10:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.916+0000 7f96177fe640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96000a2b90 0x7f96000a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9608006010 tx=0x7f9608000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:24.919+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9610016030 con 0x7f9618102250 2024-04-24T23:10:25.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.103+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f961802d300 con 0x7f9618102250 2024-04-24T23:10:25.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.104+0000 7f9614ff9640 1 -- 172.21.15.33:0/1665911338 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f961009c690 con 0x7f9618102250 2024-04-24T23:10:25.106 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96000a2b90 msgr2=0x7f96000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96000a2b90 0x7f96000a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9608006010 tx=0x7f9608000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 msgr2=0x7f961819f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 0x7f961819f0d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f961000eaa0 tx=0x7f961000ef60 comp rx=0 tx=0).stop 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 shutdown_connections 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96000a2b90 0x7f96000a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9618102250 0x7f961819f0d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9618101890 0x7f9618198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 --2- 172.21.15.33:0/1665911338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96181006a0 0x7f9618197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 >> 172.21.15.33:0/1665911338 conn(0x7f96180fbc30 msgr2=0x7f9618104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 shutdown_connections 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.106+0000 7f961dba9640 1 -- 172.21.15.33:0/1665911338 wait complete. 2024-04-24T23:10:25.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 -- 172.21.15.33:0/839522282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 msgr2=0x7ff564102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 --2- 172.21.15.33:0/839522282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564102e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff558009900 tx=0x7ff55802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 -- 172.21.15.33:0/839522282 shutdown_connections 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 --2- 172.21.15.33:0/839522282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564103350 0x7ff564109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 --2- 172.21.15.33:0/839522282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564102e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 --2- 172.21.15.33:0/839522282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5641017a0 0x7ff564101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 -- 172.21.15.33:0/839522282 >> 172.21.15.33:0/839522282 conn(0x7ff5640fcd10 msgr2=0x7ff5640ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 -- 172.21.15.33:0/839522282 shutdown_connections 2024-04-24T23:10:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.222+0000 7ff568e46640 1 -- 172.21.15.33:0/839522282 wait complete. 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 Processor -- start 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 -- start start 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5641017a0 0x7ff564197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564103350 0x7ff56419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff56410d420 con 0x7ff564102990 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff56410d2a0 con 0x7ff5641017a0 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff568e46640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff56410d5a0 con 0x7ff564103350 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46106/0 (socket says 172.21.15.33:46106) 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 -- 172.21.15.33:0/3918011598 learned_addr learned my addr 172.21.15.33:0/3918011598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff562d9d640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564103350 0x7ff56419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff56259c640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5641017a0 0x7ff564197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 -- 172.21.15.33:0/3918011598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564103350 msgr2=0x7ff56419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564103350 0x7ff56419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 -- 172.21.15.33:0/3918011598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5641017a0 msgr2=0x7ff564197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5641017a0 0x7ff564197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.223+0000 7ff561d9b640 1 -- 172.21.15.33:0/3918011598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56419f710 con 0x7ff564102990 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.224+0000 7ff561d9b640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564198030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff558009230 tx=0x7ff558004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.224+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55803f030 con 0x7ff564102990 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.224+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff558003ce0 con 0x7ff564102990 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.224+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56419f9a0 con 0x7ff564102990 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.224+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558035070 con 0x7ff564102990 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.224+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56419fee0 con 0x7ff564102990 2024-04-24T23:10:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.225+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5640689b0 con 0x7ff564102990 2024-04-24T23:10:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.226+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff558042030 con 0x7ff564102990 2024-04-24T23:10:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.226+0000 7ff5537fe640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff54c0a2c10 0x7ff54c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.227+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff5580efcd0 con 0x7ff564102990 2024-04-24T23:10:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.227+0000 7ff56259c640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff54c0a2c10 0x7ff54c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.227+0000 7ff56259c640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff54c0a2c10 0x7ff54c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff554005fd0 tx=0x7ff554005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.231+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff5580bc9f0 con 0x7ff564102990 2024-04-24T23:10:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.414+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff564062400 con 0x7ff564102990 2024-04-24T23:10:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.414+0000 7ff5537fe640 1 -- 172.21.15.33:0/3918011598 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff5580bc2f0 con 0x7ff564102990 2024-04-24T23:10:25.416 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:10:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff54c0a2c10 msgr2=0x7ff54c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff54c0a2c10 0x7ff54c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff554005fd0 tx=0x7ff554005850 comp rx=0 tx=0).stop 2024-04-24T23:10:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 msgr2=0x7ff564198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564198030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff558009230 tx=0x7ff558004360 comp rx=0 tx=0).stop 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 shutdown_connections 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff54c0a2c10 0x7ff54c0a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564103350 0x7ff56419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564102990 0x7ff564198030 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 --2- 172.21.15.33:0/3918011598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5641017a0 0x7ff564197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 >> 172.21.15.33:0/3918011598 conn(0x7ff5640fcd10 msgr2=0x7ff564105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.416+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 shutdown_connections 2024-04-24T23:10:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.417+0000 7ff568e46640 1 -- 172.21.15.33:0/3918011598 wait complete. 2024-04-24T23:10:25.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 -- 172.21.15.33:0/137969571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0102250 msgr2=0x7f29e0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 --2- 172.21.15.33:0/137969571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0102250 0x7f29e0108ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009ca0 tx=0x7f29d802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 -- 172.21.15.33:0/137969571 shutdown_connections 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 --2- 172.21.15.33:0/137969571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0102250 0x7f29e0108ae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 --2- 172.21.15.33:0/137969571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e0101890 0x7f29e0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 --2- 172.21.15.33:0/137969571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e01006a0 0x7f29e0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 -- 172.21.15.33:0/137969571 >> 172.21.15.33:0/137969571 conn(0x7f29e00fbc10 msgr2=0x7f29e00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 -- 172.21.15.33:0/137969571 shutdown_connections 2024-04-24T23:10:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 -- 172.21.15.33:0/137969571 wait complete. 2024-04-24T23:10:25.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 Processor -- start 2024-04-24T23:10:25.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.532+0000 7f29e63dc640 1 -- start start 2024-04-24T23:10:25.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e01006a0 0x7f29e01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 0x7f29e0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e0102250 0x7f29e019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29e010d400 con 0x7f29e0101890 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29e010d280 con 0x7f29e01006a0 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29e010d580 con 0x7f29e0102250 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 0x7f29e0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 0x7f29e0197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46136/0 (socket says 172.21.15.33:46136) 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 -- 172.21.15.33:0/795726245 learned_addr learned my addr 172.21.15.33:0/795726245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29dffff640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e01006a0 0x7f29e01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e4979640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e0102250 0x7f29e019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 -- 172.21.15.33:0/795726245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e0102250 msgr2=0x7f29e019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e0102250 0x7f29e019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 -- 172.21.15.33:0/795726245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e01006a0 msgr2=0x7f29e01979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e01006a0 0x7f29e01979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 -- 172.21.15.33:0/795726245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e019f5d0 con 0x7f29e0101890 2024-04-24T23:10:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29df7fe640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 0x7f29e0197ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d950 tx=0x7f29d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017070 con 0x7f29e0101890 2024-04-24T23:10:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e019f8c0 con 0x7f29e0101890 2024-04-24T23:10:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d4009c30 con 0x7f29e0101890 2024-04-24T23:10:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400f6b0 con 0x7f29e0101890 2024-04-24T23:10:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.533+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e019fe10 con 0x7f29e0101890 2024-04-24T23:10:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.534+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e0107960 con 0x7f29e0101890 2024-04-24T23:10:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.535+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29d40109d0 con 0x7f29e0101890 2024-04-24T23:10:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.536+0000 7f29dd7fa640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c80a2b90 0x7f29c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.536+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f29d40cefc0 con 0x7f29e0101890 2024-04-24T23:10:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.536+0000 7f29dffff640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c80a2b90 0x7f29c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.536+0000 7f29dffff640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c80a2b90 0x7f29c80a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29d00058c0 tx=0x7f29d0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.541+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f29d4009de0 con 0x7f29e0101890 2024-04-24T23:10:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:25 smithi038 ceph-mon[32450]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2024-04-24T23:10:25.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2077376578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:25.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2219645116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:25.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1665911338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:10:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:25 smithi117 ceph-mon[32529]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2024-04-24T23:10:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2077376578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2219645116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1665911338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:10:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:25 smithi033 ceph-mon[29201]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2024-04-24T23:10:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2077376578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2219645116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1665911338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:10:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.724+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f29e00623e0 con 0x7f29e0101890 2024-04-24T23:10:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.725+0000 7f29dd7fa640 1 -- 172.21.15.33:0/795726245 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f29d409bce0 con 0x7f29e0101890 2024-04-24T23:10:25.726 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:10:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c80a2b90 msgr2=0x7f29c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c80a2b90 0x7f29c80a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29d00058c0 tx=0x7f29d0005850 comp rx=0 tx=0).stop 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 msgr2=0x7f29e0197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 0x7f29e0197ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d950 tx=0x7f29d400de10 comp rx=0 tx=0).stop 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 shutdown_connections 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c80a2b90 0x7f29c80a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e0102250 0x7f29e019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29e0101890 0x7f29e0197ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 --2- 172.21.15.33:0/795726245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e01006a0 0x7f29e01979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 >> 172.21.15.33:0/795726245 conn(0x7f29e00fbc10 msgr2=0x7f29e0104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 shutdown_connections 2024-04-24T23:10:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.727+0000 7f29e63dc640 1 -- 172.21.15.33:0/795726245 wait complete. 2024-04-24T23:10:25.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:10:25.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.840+0000 7ff254ddc640 1 -- 172.21.15.33:0/1561715596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 msgr2=0x7ff250101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.840+0000 7ff254ddc640 1 --2- 172.21.15.33:0/1561715596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250101cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff244009900 tx=0x7ff24402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 -- 172.21.15.33:0/1561715596 shutdown_connections 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 --2- 172.21.15.33:0/1561715596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff250102230 0x7ff250108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 --2- 172.21.15.33:0/1561715596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250101cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 --2- 172.21.15.33:0/1561715596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff250100680 0x7ff250100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 -- 172.21.15.33:0/1561715596 >> 172.21.15.33:0/1561715596 conn(0x7ff2500fbbf0 msgr2=0x7ff2500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 -- 172.21.15.33:0/1561715596 shutdown_connections 2024-04-24T23:10:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 -- 172.21.15.33:0/1561715596 wait complete. 2024-04-24T23:10:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 Processor -- start 2024-04-24T23:10:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 -- start start 2024-04-24T23:10:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff250100680 0x7ff250197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.841+0000 7ff254ddc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff254ddc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff250102230 0x7ff25019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff254ddc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff25010c2e0 con 0x7ff250101870 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff254ddc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff25010c160 con 0x7ff250102230 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff254ddc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff25010c460 con 0x7ff250100680 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50718/0 (socket says 172.21.15.33:50718) 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 -- 172.21.15.33:0/3347833411 learned_addr learned my addr 172.21.15.33:0/3347833411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24e59c640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff250100680 0x7ff250197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24ed9d640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff250102230 0x7ff25019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 -- 172.21.15.33:0/3347833411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff250100680 msgr2=0x7ff250197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff250100680 0x7ff250197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 -- 172.21.15.33:0/3347833411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff250102230 msgr2=0x7ff25019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff250102230 0x7ff25019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 -- 172.21.15.33:0/3347833411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25019f6b0 con 0x7ff250101870 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff24dd9b640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250197fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff244009230 tx=0x7ff244004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24403f030 con 0x7ff250101870 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff244003ce0 con 0x7ff250101870 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25019f940 con 0x7ff250101870 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244035070 con 0x7ff250101870 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.842+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25019fe30 con 0x7ff250101870 2024-04-24T23:10:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.844+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2501a0060 con 0x7ff250101870 2024-04-24T23:10:25.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.845+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff244042030 con 0x7ff250101870 2024-04-24T23:10:25.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.845+0000 7ff23f7fe640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2380a2b90 0x7ff2380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.845+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff2440efbc0 con 0x7ff250101870 2024-04-24T23:10:25.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.845+0000 7ff24e59c640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2380a2b90 0x7ff2380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.846+0000 7ff24e59c640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2380a2b90 0x7ff2380a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2501016d0 tx=0x7ff24000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:25.850+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2440bc290 con 0x7ff250101870 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.047+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2501a0230 con 0x7ff250101870 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.047+0000 7ff23f7fe640 1 -- 172.21.15.33:0/3347833411 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7ff2440dae20 con 0x7ff250101870 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/946624938 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2813273186 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2813273186 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1214692934 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1172519445 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1299073362 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1299073362 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1574498973 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/288675277 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2398589700 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2380a2b90 msgr2=0x7ff2380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2380a2b90 0x7ff2380a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2501016d0 tx=0x7ff24000b390 comp rx=0 tx=0).stop 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 msgr2=0x7ff250197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250197fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff244009230 tx=0x7ff244004360 comp rx=0 tx=0).stop 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 shutdown_connections 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2380a2b90 0x7ff2380a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff250102230 0x7ff25019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff250101870 0x7ff250197fd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 --2- 172.21.15.33:0/3347833411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff250100680 0x7ff250197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 >> 172.21.15.33:0/3347833411 conn(0x7ff2500fbbf0 msgr2=0x7ff2500688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 shutdown_connections 2024-04-24T23:10:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.049+0000 7ff254ddc640 1 -- 172.21.15.33:0/3347833411 wait complete. 2024-04-24T23:10:26.062 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:10:26.062 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:26.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 -- 172.21.15.33:0/120620878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41018f0 msgr2=0x7efed4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 --2- 172.21.15.33:0/120620878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41018f0 0x7efed4101d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efec4009900 tx=0x7efec402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 -- 172.21.15.33:0/120620878 shutdown_connections 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 --2- 172.21.15.33:0/120620878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efed41022b0 0x7efed4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 --2- 172.21.15.33:0/120620878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41018f0 0x7efed4101d70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 --2- 172.21.15.33:0/120620878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efed4100700 0x7efed4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 -- 172.21.15.33:0/120620878 >> 172.21.15.33:0/120620878 conn(0x7efed40fbc90 msgr2=0x7efed40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 -- 172.21.15.33:0/120620878 shutdown_connections 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.218+0000 7efed88bd640 1 -- 172.21.15.33:0/120620878 wait complete. 2024-04-24T23:10:26.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 Processor -- start 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 -- start start 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efed4100700 0x7efed4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efed41018f0 0x7efed4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 0x7efed419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efed410c360 con 0x7efed41022b0 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efed410c1e0 con 0x7efed41018f0 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed88bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efed410c4e0 con 0x7efed4100700 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 0x7efed419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 0x7efed419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50742/0 (socket says 172.21.15.33:50742) 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 -- 172.21.15.33:0/3450171247 learned_addr learned my addr 172.21.15.33:0/3450171247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed1d9b640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efed41018f0 0x7efed4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed259c640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efed4100700 0x7efed4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 -- 172.21.15.33:0/3450171247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efed4100700 msgr2=0x7efed4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efed4100700 0x7efed4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 -- 172.21.15.33:0/3450171247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efed41018f0 msgr2=0x7efed4197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efed41018f0 0x7efed4197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 -- 172.21.15.33:0/3450171247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed419f6a0 con 0x7efed41022b0 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.219+0000 7efed2d9d640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 0x7efed419f030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efecc00eaa0 tx=0x7efecc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.220+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00cdb0 con 0x7efed41022b0 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.220+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed419f990 con 0x7efed41022b0 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.220+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc005020 con 0x7efed41022b0 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.220+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0116d0 con 0x7efed41022b0 2024-04-24T23:10:26.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.220+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed419feb0 con 0x7efed41022b0 2024-04-24T23:10:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.221+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed41a01a0 con 0x7efed41022b0 2024-04-24T23:10:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.222+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efecc0054d0 con 0x7efed41022b0 2024-04-24T23:10:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.222+0000 7efecb7fe640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efec00a2b90 0x7efec00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.223+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efecc0cf7a0 con 0x7efed41022b0 2024-04-24T23:10:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.223+0000 7efed259c640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efec00a2b90 0x7efec00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.223+0000 7efed259c640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efec00a2b90 0x7efec00a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efed4101750 tx=0x7efebc00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.226+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efecc016030 con 0x7efed41022b0 2024-04-24T23:10:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.412+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efed41a0370 con 0x7efed41022b0 2024-04-24T23:10:26.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.414+0000 7efecb7fe640 1 -- 172.21.15.33:0/3450171247 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7efecc09be70 con 0x7efed41022b0 2024-04-24T23:10:26.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:26.416 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efec00a2b90 msgr2=0x7efec00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efec00a2b90 0x7efec00a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efed4101750 tx=0x7efebc00a2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 msgr2=0x7efed419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 0x7efed419f030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efecc00eaa0 tx=0x7efecc00ef60 comp rx=0 tx=0).stop 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 shutdown_connections 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efec00a2b90 0x7efec00a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed41022b0 0x7efed419f030 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efed41018f0 0x7efed4197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 --2- 172.21.15.33:0/3450171247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efed4100700 0x7efed4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 >> 172.21.15.33:0/3450171247 conn(0x7efed40fbc90 msgr2=0x7efed4069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 shutdown_connections 2024-04-24T23:10:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.416+0000 7efed88bd640 1 -- 172.21.15.33:0/3450171247 wait complete. 2024-04-24T23:10:26.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:10:26.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 -- 172.21.15.33:0/1235766894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 msgr2=0x7f5d48100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1235766894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d48100ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009900 tx=0x7f5d4402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 -- 172.21.15.33:0/1235766894 shutdown_connections 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1235766894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d48102250 0x7f5d48108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1235766894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48101890 0x7f5d48101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1235766894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d48100ac0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 -- 172.21.15.33:0/1235766894 >> 172.21.15.33:0/1235766894 conn(0x7f5d480fbc30 msgr2=0x7f5d480fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 -- 172.21.15.33:0/1235766894 shutdown_connections 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.532+0000 7f5d5042a640 1 -- 172.21.15.33:0/1235766894 wait complete. 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 Processor -- start 2024-04-24T23:10:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 -- start start 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d4810aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d48101890 0x7f5d481094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48102250 0x7f5d48109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4810d440 con 0x7f5d481006a0 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4810d2c0 con 0x7f5d48102250 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d5042a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4810d5c0 con 0x7f5d48101890 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d4810aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d4810aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50756/0 (socket says 172.21.15.33:50756) 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 -- 172.21.15.33:0/1117445185 learned_addr learned my addr 172.21.15.33:0/1117445185 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4d9c5640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d48101890 0x7f5d481094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e9c7640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48102250 0x7f5d48109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 -- 172.21.15.33:0/1117445185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d48101890 msgr2=0x7f5d481094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d48101890 0x7f5d481094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 -- 172.21.15.33:0/1117445185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48102250 msgr2=0x7f5d48109a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48102250 0x7f5d48109a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.533+0000 7f5d4e1c6640 1 -- 172.21.15.33:0/1117445185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4810a2b0 con 0x7f5d481006a0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d4e9c7640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48102250 0x7f5d48109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d4e1c6640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d4810aea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5d44031610 tx=0x7f5d440043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4403f030 con 0x7f5d481006a0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d481a1b70 con 0x7f5d481006a0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d44003ce0 con 0x7f5d481006a0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44035070 con 0x7f5d481006a0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.534+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d481a2110 con 0x7f5d481006a0 2024-04-24T23:10:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.535+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d48107a30 con 0x7f5d481006a0 2024-04-24T23:10:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.536+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5d44042030 con 0x7f5d481006a0 2024-04-24T23:10:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.536+0000 7f5d3f7fe640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.537+0000 7f5d4d9c5640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.537+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5d440efb20 con 0x7f5d481006a0 2024-04-24T23:10:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.537+0000 7f5d4d9c5640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d4810ac10 tx=0x7f5d3800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.540+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d440bc840 con 0x7f5d481006a0 2024-04-24T23:10:26.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3918011598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:10:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/795726245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:10:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3347833411' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:10:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3918011598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:10:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/795726245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:10:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3347833411' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:10:26.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3918011598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:10:26.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/795726245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:10:26.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3347833411' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:10:26.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.726+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d4802d300 con 0x7f5d481006a0 2024-04-24T23:10:26.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.727+0000 7f5d3f7fe640 1 -- 172.21.15.33:0/1117445185 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f5d440bc140 con 0x7f5d481006a0 2024-04-24T23:10:26.729 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 msgr2=0x7f5d340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d4810ac10 tx=0x7f5d3800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 msgr2=0x7f5d4810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d4810aea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5d44031610 tx=0x7f5d440043b0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 shutdown_connections 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d48102250 0x7f5d48109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d48101890 0x7f5d481094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 --2- 172.21.15.33:0/1117445185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481006a0 0x7f5d4810aea0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 >> 172.21.15.33:0/1117445185 conn(0x7f5d480fbc30 msgr2=0x7f5d48104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 shutdown_connections 2024-04-24T23:10:26.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.729+0000 7f5d5042a640 1 -- 172.21.15.33:0/1117445185 wait complete. 2024-04-24T23:10:26.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:10:26.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 -- 172.21.15.33:0/3134681301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad41022c0 msgr2=0x7f7ad41046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 --2- 172.21.15.33:0/3134681301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad41022c0 0x7f7ad41046b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009900 tx=0x7f7ac402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:26.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 -- 172.21.15.33:0/3134681301 shutdown_connections 2024-04-24T23:10:26.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 --2- 172.21.15.33:0/3134681301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad4107520 0x7f7ad4109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 --2- 172.21.15.33:0/3134681301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 --2- 172.21.15.33:0/3134681301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad41022c0 0x7f7ad41046b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 -- 172.21.15.33:0/3134681301 >> 172.21.15.33:0/3134681301 conn(0x7f7ad40fbc30 msgr2=0x7f7ad40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 -- 172.21.15.33:0/3134681301 shutdown_connections 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.844+0000 7f7ad955a640 1 -- 172.21.15.33:0/3134681301 wait complete. 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 Processor -- start 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 -- start start 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ad41022c0 0x7f7ad4101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad4107520 0x7f7ad41007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad410d410 con 0x7f7ad4104bf0 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad410d290 con 0x7f7ad4107520 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad955a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad410d590 con 0x7f7ad41022c0 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7acbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7acbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50778/0 (socket says 172.21.15.33:50778) 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7acbfff640 1 -- 172.21.15.33:0/2906806100 learned_addr learned my addr 172.21.15.33:0/2906806100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad37fe640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad4107520 0x7f7ad41007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.845+0000 7f7ad2ffd640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ad41022c0 0x7f7ad4101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7acbfff640 1 -- 172.21.15.33:0/2906806100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ad41022c0 msgr2=0x7f7ad4101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7acbfff640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ad41022c0 0x7f7ad4101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7acbfff640 1 -- 172.21.15.33:0/2906806100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad4107520 msgr2=0x7f7ad41007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7acbfff640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad4107520 0x7f7ad41007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7acbfff640 1 -- 172.21.15.33:0/2906806100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad4101040 con 0x7f7ad4104bf0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7acbfff640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4100270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00d770 tx=0x7f7abc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc017070 con 0x7f7ad4104bf0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7abc009c30 con 0x7f7ad4104bf0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc00f620 con 0x7f7ad4104bf0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad41a1c30 con 0x7f7ad4104bf0 2024-04-24T23:10:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.846+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad41a2180 con 0x7f7ad4104bf0 2024-04-24T23:10:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.847+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad40fd720 con 0x7f7ad4104bf0 2024-04-24T23:10:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.848+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7abc01a070 con 0x7f7ad4104bf0 2024-04-24T23:10:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.849+0000 7f7ad0ff9640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ac00a2b90 0x7f7ac00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.849+0000 7f7ad2ffd640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ac00a2b90 0x7f7ac00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.849+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7abc0cece0 con 0x7f7ad4104bf0 2024-04-24T23:10:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.849+0000 7f7ad2ffd640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ac00a2b90 0x7f7ac00a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40314e0 tx=0x7f7ac4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:26.853+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7abc009de0 con 0x7f7ad4104bf0 2024-04-24T23:10:27.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.037+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ad4062340 con 0x7f7ad4104bf0 2024-04-24T23:10:27.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.037+0000 7f7ad0ff9640 1 -- 172.21.15.33:0/2906806100 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f7abc09b3b0 con 0x7f7ad4104bf0 2024-04-24T23:10:27.039 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:10:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ac00a2b90 msgr2=0x7f7ac00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ac00a2b90 0x7f7ac00a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40314e0 tx=0x7f7ac4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 msgr2=0x7f7ad4100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4100270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00d770 tx=0x7f7abc00dc30 comp rx=0 tx=0).stop 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 shutdown_connections 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ac00a2b90 0x7f7ac00a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad4107520 0x7f7ad41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ad4104bf0 0x7f7ad4100270 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 --2- 172.21.15.33:0/2906806100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ad41022c0 0x7f7ad4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.039+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 >> 172.21.15.33:0/2906806100 conn(0x7f7ad40fbc30 msgr2=0x7f7ad4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.040+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 shutdown_connections 2024-04-24T23:10:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.040+0000 7f7ad955a640 1 -- 172.21.15.33:0/2906806100 wait complete. 2024-04-24T23:10:27.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:10:27.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 -- 172.21.15.33:0/3045243725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 msgr2=0x7f45dc101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 --2- 172.21.15.33:0/3045243725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc101bc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009900 tx=0x7f45d802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:27.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 -- 172.21.15.33:0/3045243725 shutdown_connections 2024-04-24T23:10:27.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 --2- 172.21.15.33:0/3045243725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc103350 0x7f45dc109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 --2- 172.21.15.33:0/3045243725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45dc102990 0x7f45dc102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 --2- 172.21.15.33:0/3045243725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc101bc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 -- 172.21.15.33:0/3045243725 >> 172.21.15.33:0/3045243725 conn(0x7f45dc0fcd10 msgr2=0x7f45dc0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 -- 172.21.15.33:0/3045243725 shutdown_connections 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.155+0000 7f45e3542640 1 -- 172.21.15.33:0/3045243725 wait complete. 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 Processor -- start 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 -- start start 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc102990 0x7f45dc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45dc103350 0x7f45dc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10d3f0 con 0x7f45dc1017a0 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10d270 con 0x7f45dc102990 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e3542640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10d570 con 0x7f45dc103350 2024-04-24T23:10:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e12de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e12de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50804/0 (socket says 172.21.15.33:50804) 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e12de640 1 -- 172.21.15.33:0/1179954402 learned_addr learned my addr 172.21.15.33:0/1179954402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e1adf640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45dc103350 0x7f45dc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e0add640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc102990 0x7f45dc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e12de640 1 -- 172.21.15.33:0/1179954402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45dc103350 msgr2=0x7f45dc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e12de640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45dc103350 0x7f45dc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.156+0000 7f45e12de640 1 -- 172.21.15.33:0/1179954402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc102990 msgr2=0x7f45dc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45e12de640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc102990 0x7f45dc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45e12de640 1 -- 172.21.15.33:0/1179954402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc19f6c0 con 0x7f45dc1017a0 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45e0add640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc102990 0x7f45dc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45e12de640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc197aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f45d8031610 tx=0x7f45d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d803f030 con 0x7f45dc1017a0 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc19f950 con 0x7f45dc1017a0 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8003ad0 con 0x7f45dc1017a0 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8035070 con 0x7f45dc1017a0 2024-04-24T23:10:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.157+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc19fe40 con 0x7f45dc1017a0 2024-04-24T23:10:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.158+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc108b30 con 0x7f45dc1017a0 2024-04-24T23:10:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.159+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45d8042030 con 0x7f45dc1017a0 2024-04-24T23:10:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.159+0000 7f45d27fc640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45c80a2b90 0x7f45c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.160+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45d80efa60 con 0x7f45dc1017a0 2024-04-24T23:10:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.160+0000 7f45e0add640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45c80a2b90 0x7f45c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.160+0000 7f45e0add640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45c80a2b90 0x7f45c80a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005d00 tx=0x7f45cc005ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.164+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45d80bc780 con 0x7f45dc1017a0 2024-04-24T23:10:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.351+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45dc062340 con 0x7f45dc1017a0 2024-04-24T23:10:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.351+0000 7f45d27fc640 1 -- 172.21.15.33:0/1179954402 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f45d80bc080 con 0x7f45dc1017a0 2024-04-24T23:10:27.353 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:10:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45c80a2b90 msgr2=0x7f45c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45c80a2b90 0x7f45c80a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005d00 tx=0x7f45cc005ed0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 msgr2=0x7f45dc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc197aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f45d8031610 tx=0x7f45d8004270 comp rx=0 tx=0).stop 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 shutdown_connections 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45c80a2b90 0x7f45c80a5050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45dc103350 0x7f45dc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45dc102990 0x7f45dc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 --2- 172.21.15.33:0/1179954402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc1017a0 0x7f45dc197aa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 >> 172.21.15.33:0/1179954402 conn(0x7f45dc0fcd10 msgr2=0x7f45dc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 shutdown_connections 2024-04-24T23:10:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.353+0000 7f45e3542640 1 -- 172.21.15.33:0/1179954402 wait complete. 2024-04-24T23:10:27.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:10:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 -- 172.21.15.33:0/265075253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101870 msgr2=0x7f3d10101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 --2- 172.21.15.33:0/265075253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101870 0x7f3d10101cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009900 tx=0x7f3d0402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 -- 172.21.15.33:0/265075253 shutdown_connections 2024-04-24T23:10:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 --2- 172.21.15.33:0/265075253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10102230 0x7f3d10108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 --2- 172.21.15.33:0/265075253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101870 0x7f3d10101cf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 --2- 172.21.15.33:0/265075253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10100680 0x7f3d10100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 -- 172.21.15.33:0/265075253 >> 172.21.15.33:0/265075253 conn(0x7f3d100fbbf0 msgr2=0x7f3d100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 -- 172.21.15.33:0/265075253 shutdown_connections 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.469+0000 7f3d16072640 1 -- 172.21.15.33:0/265075253 wait complete. 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 Processor -- start 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 -- start start 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10100680 0x7f3d101969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10101870 0x7f3d10196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 0x7f3d1019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010c2e0 con 0x7f3d10102230 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010c160 con 0x7f3d10101870 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d16072640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010c460 con 0x7f3d10100680 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.470+0000 7f3d0ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 0x7f3d1019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 0x7f3d1019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50822/0 (socket says 172.21.15.33:50822) 2024-04-24T23:10:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 -- 172.21.15.33:0/108530395 learned_addr learned my addr 172.21.15.33:0/108530395 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0f7fe640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10100680 0x7f3d101969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0effd640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10101870 0x7f3d10196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 -- 172.21.15.33:0/108530395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10100680 msgr2=0x7f3d101969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10100680 0x7f3d101969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 -- 172.21.15.33:0/108530395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10101870 msgr2=0x7f3d10196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10101870 0x7f3d10196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 -- 172.21.15.33:0/108530395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1019e600 con 0x7f3d10102230 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0ffff640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 0x7f3d1019df90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c970 tx=0x7f3d0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08007bb0 con 0x7f3d10102230 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d08007d50 con 0x7f3d10102230 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1019e8f0 con 0x7f3d10102230 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0800f450 con 0x7f3d10102230 2024-04-24T23:10:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.471+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1019ee40 con 0x7f3d10102230 2024-04-24T23:10:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.472+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d10107a70 con 0x7f3d10102230 2024-04-24T23:10:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.473+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3d08004f60 con 0x7f3d10102230 2024-04-24T23:10:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.474+0000 7f3d0cff9640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.474+0000 7f3d0f7fe640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.474+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3d080ceeb0 con 0x7f3d10102230 2024-04-24T23:10:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.474+0000 7f3d0f7fe640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d101016d0 tx=0x7f3d00009720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.478+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3d08014030 con 0x7f3d10102230 2024-04-24T23:10:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.664+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3d1002d170 con 0x7f3d10102230 2024-04-24T23:10:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.665+0000 7f3d0cff9640 1 -- 172.21.15.33:0/108530395 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f3d0809bbd0 con 0x7f3d10102230 2024-04-24T23:10:27.667 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:10:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cf80a2c10 msgr2=0x7f3cf80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d101016d0 tx=0x7f3d00009720 comp rx=0 tx=0).stop 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 msgr2=0x7f3d1019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 0x7f3d1019df90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c970 tx=0x7f3d0800ce30 comp rx=0 tx=0).stop 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 shutdown_connections 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cf80a2c10 0x7f3cf80a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10102230 0x7f3d1019df90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10101870 0x7f3d10196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 --2- 172.21.15.33:0/108530395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10100680 0x7f3d101969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 >> 172.21.15.33:0/108530395 conn(0x7f3d100fbbf0 msgr2=0x7f3d10104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 shutdown_connections 2024-04-24T23:10:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.667+0000 7f3d16072640 1 -- 172.21.15.33:0/108530395 wait complete. 2024-04-24T23:10:27.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:27 smithi038 ceph-mon[32450]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 326 B/s wr, 0 op/s 2024-04-24T23:10:27.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3450171247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:27.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1117445185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2906806100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1179954402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:27 smithi117 ceph-mon[32529]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 326 B/s wr, 0 op/s 2024-04-24T23:10:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3450171247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1117445185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2906806100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1179954402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:27 smithi033 ceph-mon[29201]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 326 B/s wr, 0 op/s 2024-04-24T23:10:27.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3450171247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:27.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1117445185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2906806100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1179954402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:10:27.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:10:27.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.780+0000 7f37dd864640 1 -- 172.21.15.33:0/1327737326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 msgr2=0x7f37d81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.780+0000 7f37dd864640 1 --2- 172.21.15.33:0/1327737326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d81045f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009900 tx=0x7f37d002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- 172.21.15.33:0/1327737326 shutdown_connections 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 --2- 172.21.15.33:0/1327737326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 0x7f37d81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 --2- 172.21.15.33:0/1327737326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d8104b30 0x7f37d8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 --2- 172.21.15.33:0/1327737326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d81045f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- 172.21.15.33:0/1327737326 >> 172.21.15.33:0/1327737326 conn(0x7f37d80fbbf0 msgr2=0x7f37d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- 172.21.15.33:0/1327737326 shutdown_connections 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- 172.21.15.33:0/1327737326 wait complete. 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 Processor -- start 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- start start 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d8104b30 0x7f37d8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 0x7f37d819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d400 con 0x7f37d8102200 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d280 con 0x7f37d8104b30 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.781+0000 7f37dd864640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d580 con 0x7f37d8107460 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d8197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50840/0 (socket says 172.21.15.33:50840) 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 -- 172.21.15.33:0/1055254617 learned_addr learned my addr 172.21.15.33:0/1055254617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d67fc640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d8104b30 0x7f37d8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d77fe640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 0x7f37d819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 -- 172.21.15.33:0/1055254617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 msgr2=0x7f37d819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 0x7f37d819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 -- 172.21.15.33:0/1055254617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d8104b30 msgr2=0x7f37d8198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d8104b30 0x7f37d8198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 -- 172.21.15.33:0/1055254617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819f730 con 0x7f37d8102200 2024-04-24T23:10:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d77fe640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 0x7f37d819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37d6ffd640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d8197b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009230 tx=0x7f37d0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d003f030 con 0x7f37d8102200 2024-04-24T23:10:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819f9c0 con 0x7f37d8102200 2024-04-24T23:10:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d0003ad0 con 0x7f37d8102200 2024-04-24T23:10:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0035070 con 0x7f37d8102200 2024-04-24T23:10:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.782+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d819ff00 con 0x7f37d8102200 2024-04-24T23:10:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.783+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d80fd6e0 con 0x7f37d8102200 2024-04-24T23:10:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.785+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37d0042030 con 0x7f37d8102200 2024-04-24T23:10:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.786+0000 7f37dc862640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f37c00a2b90 0x7f37c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.786+0000 7f37d67fc640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f37c00a2b90 0x7f37c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.786+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f37d00efd10 con 0x7f37d8102200 2024-04-24T23:10:27.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.786+0000 7f37d67fc640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f37c00a2b90 0x7f37c00a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37d8199020 tx=0x7f37cc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.789+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37d00bca30 con 0x7f37d8102200 2024-04-24T23:10:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.986+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37d802d170 con 0x7f37d8102200 2024-04-24T23:10:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.987+0000 7f37dc862640 1 -- 172.21.15.33:0/1055254617 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7f37d00bc330 con 0x7f37d8102200 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/946624938 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2813273186 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2813273186 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1214692934 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1172519445 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1299073362 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1299073362 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:27.989 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1574498973 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:27.990 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/288675277 2024-04-25T23:02:38.216643+0000 2024-04-24T23:10:27.990 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2398589700 2024-04-25T23:03:05.692528+0000 2024-04-24T23:10:27.990 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f37c00a2b90 msgr2=0x7f37c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f37c00a2b90 0x7f37c00a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37d8199020 tx=0x7f37cc005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 msgr2=0x7f37d8197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d8197b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009230 tx=0x7f37d0031b60 comp rx=0 tx=0).stop 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 shutdown_connections 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f37c00a2b90 0x7f37c00a5050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8107460 0x7f37d819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d8104b30 0x7f37d8198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 --2- 172.21.15.33:0/1055254617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d8102200 0x7f37d8197b10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 >> 172.21.15.33:0/1055254617 conn(0x7f37d80fbbf0 msgr2=0x7f37d8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 shutdown_connections 2024-04-24T23:10:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:27.989+0000 7f37dd864640 1 -- 172.21.15.33:0/1055254617 wait complete. 2024-04-24T23:10:28.001 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-24T23:10:28.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:28.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:10:28.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[73410]: fuse finished with error 0 and tester_r 0 2024-04-24T23:10:28.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:10:28.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/108530395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:10:28.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1055254617' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:10:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/108530395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:10:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1055254617' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:10:28.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/108530395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:10:28.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1055254617' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:10:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:29 smithi038 ceph-mon[32450]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 294 B/s wr, 0 op/s 2024-04-24T23:10:29.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:29 smithi117 ceph-mon[32529]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 294 B/s wr, 0 op/s 2024-04-24T23:10:29.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:29 smithi033 ceph-mon[29201]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 294 B/s wr, 0 op/s 2024-04-24T23:10:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:31 smithi038 ceph-mon[32450]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:10:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:31 smithi117 ceph-mon[32529]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:10:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:31 smithi033 ceph-mon[29201]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:10:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:33 smithi038 ceph-mon[32450]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:33 smithi117 ceph-mon[32529]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:33 smithi033 ceph-mon[29201]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:34.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-24T23:10:34.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:34.129 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T23:10:34.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 -- 172.21.15.33:0/524981597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8101890 msgr2=0x7fbdf8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/524981597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8101890 0x7fbdf8101d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009900 tx=0x7fbdec02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 -- 172.21.15.33:0/524981597 shutdown_connections 2024-04-24T23:10:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/524981597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8102250 0x7fbdf8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/524981597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8101890 0x7fbdf8101d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/524981597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf81006a0 0x7fbdf8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 -- 172.21.15.33:0/524981597 >> 172.21.15.33:0/524981597 conn(0x7fbdf80fbc30 msgr2=0x7fbdf80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 -- 172.21.15.33:0/524981597 shutdown_connections 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.314+0000 7fbdfe6da640 1 -- 172.21.15.33:0/524981597 wait complete. 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 Processor -- start 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 -- start start 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf81006a0 0x7fbdf806a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8101890 0x7fbdf819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 0x7fbdf819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf810c320 con 0x7fbdf8102250 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf810c1a0 con 0x7fbdf8101890 2024-04-24T23:10:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfe6da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf810c4a0 con 0x7fbdf81006a0 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfcc77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 0x7fbdf819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfcc77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 0x7fbdf819ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50850/0 (socket says 172.21.15.33:50850) 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdfcc77640 1 -- 172.21.15.33:0/2655229204 learned_addr learned my addr 172.21.15.33:0/2655229204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.315+0000 7fbdf7fff640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf81006a0 0x7fbdf806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdf77fe640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8101890 0x7fbdf819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfcc77640 1 -- 172.21.15.33:0/2655229204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf81006a0 msgr2=0x7fbdf806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfcc77640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf81006a0 0x7fbdf806a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfcc77640 1 -- 172.21.15.33:0/2655229204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8101890 msgr2=0x7fbdf819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfcc77640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8101890 0x7fbdf819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfcc77640 1 -- 172.21.15.33:0/2655229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf819f6f0 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdf77fe640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8101890 0x7fbdf819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfcc77640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 0x7fbdf819ee30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009fd0 tx=0x7fbdf000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0017070 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf819f9e0 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf0003e50 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00106c0 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.316+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf819ff00 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.317+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf80689b0 con 0x7fbdf8102250 2024-04-24T23:10:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.318+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbdf0003970 con 0x7fbdf8102250 2024-04-24T23:10:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.319+0000 7fbdf57fa640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbde00a2b90 0x7fbde00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.319+0000 7fbdf7fff640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbde00a2b90 0x7fbde00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.319+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbdf00ceff0 con 0x7fbdf8102250 2024-04-24T23:10:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.320+0000 7fbdf7fff640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbde00a2b90 0x7fbde00a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbde80058c0 tx=0x7fbde800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.323+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbdf009bd10 con 0x7fbdf8102250 2024-04-24T23:10:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.512+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdf802d170 con 0x7fbdf8102250 2024-04-24T23:10:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.514+0000 7fbdf57fa640 1 -- 172.21.15.33:0/2655229204 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fbdf009b610 con 0x7fbdf8102250 2024-04-24T23:10:34.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:34.516 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbde00a2b90 msgr2=0x7fbde00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbde00a2b90 0x7fbde00a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbde80058c0 tx=0x7fbde800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 msgr2=0x7fbdf819ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 0x7fbdf819ee30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009fd0 tx=0x7fbdf000b260 comp rx=0 tx=0).stop 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 shutdown_connections 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbde00a2b90 0x7fbde00a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdf8102250 0x7fbdf819ee30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf8101890 0x7fbdf819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 --2- 172.21.15.33:0/2655229204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf81006a0 0x7fbdf806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 >> 172.21.15.33:0/2655229204 conn(0x7fbdf80fbc30 msgr2=0x7fbdf8104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 shutdown_connections 2024-04-24T23:10:34.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.516+0000 7fbdfe6da640 1 -- 172.21.15.33:0/2655229204 wait complete. 2024-04-24T23:10:34.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 -- 172.21.15.33:0/2585090488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb01017b0 msgr2=0x7fbfb0101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 --2- 172.21.15.33:0/2585090488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb01017b0 0x7fbfb0101bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ca0 tx=0x7fbfa802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 -- 172.21.15.33:0/2585090488 shutdown_connections 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 --2- 172.21.15.33:0/2585090488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb0103360 0x7fbfb0109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 --2- 172.21.15.33:0/2585090488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb01029a0 0x7fbfb0102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 --2- 172.21.15.33:0/2585090488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb01017b0 0x7fbfb0101bd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 -- 172.21.15.33:0/2585090488 >> 172.21.15.33:0/2585090488 conn(0x7fbfb00fcd40 msgr2=0x7fbfb00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 -- 172.21.15.33:0/2585090488 shutdown_connections 2024-04-24T23:10:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 -- 172.21.15.33:0/2585090488 wait complete. 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 Processor -- start 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.629+0000 7fbfb764d640 1 -- start start 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb01017b0 0x7fbfb0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb01029a0 0x7fbfb0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 0x7fbfb019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010d480 con 0x7fbfb0103360 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010d300 con 0x7fbfb01017b0 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010d600 con 0x7fbfb01029a0 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 0x7fbfb019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 0x7fbfb019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50876/0 (socket says 172.21.15.33:50876) 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 -- 172.21.15.33:0/1764672595 learned_addr learned my addr 172.21.15.33:0/1764672595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb53e9640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb01017b0 0x7fbfb0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb4be8640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb01029a0 0x7fbfb0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 -- 172.21.15.33:0/1764672595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb01029a0 msgr2=0x7fbfb0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb01029a0 0x7fbfb0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 -- 172.21.15.33:0/1764672595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb01017b0 msgr2=0x7fbfb0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb01017b0 0x7fbfb0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 -- 172.21.15.33:0/1764672595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb019f6f0 con 0x7fbfb0103360 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb4be8640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb01029a0 0x7fbfb0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb5bea640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 0x7fbfb019f080 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c970 tx=0x7fbfa000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0007bb0 con 0x7fbfb0103360 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa0007d50 con 0x7fbfb0103360 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb019f9e0 con 0x7fbfb0103360 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.630+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000f450 con 0x7fbfb0103360 2024-04-24T23:10:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.631+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb019ff30 con 0x7fbfb0103360 2024-04-24T23:10:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.632+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb0108a70 con 0x7fbfb0103360 2024-04-24T23:10:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.633+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbfa0004f00 con 0x7fbfb0103360 2024-04-24T23:10:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.633+0000 7fbfae7fc640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf980a2b90 0x7fbf980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.633+0000 7fbfb53e9640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf980a2b90 0x7fbf980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.633+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbfa00cf270 con 0x7fbfb0103360 2024-04-24T23:10:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.634+0000 7fbfb53e9640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf980a2b90 0x7fbf980a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8000f80 tx=0x7fbfa8005f30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.637+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfa0014030 con 0x7fbfb0103360 2024-04-24T23:10:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.821+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfb0062340 con 0x7fbfb0103360 2024-04-24T23:10:34.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.823+0000 7fbfae7fc640 1 -- 172.21.15.33:0/1764672595 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fbfa009bfb0 con 0x7fbfb0103360 2024-04-24T23:10:34.825 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:34.825 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf980a2b90 msgr2=0x7fbf980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf980a2b90 0x7fbf980a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8000f80 tx=0x7fbfa8005f30 comp rx=0 tx=0).stop 2024-04-24T23:10:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 msgr2=0x7fbfb019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 0x7fbfb019f080 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c970 tx=0x7fbfa000ce30 comp rx=0 tx=0).stop 2024-04-24T23:10:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 shutdown_connections 2024-04-24T23:10:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf980a2b90 0x7fbf980a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb0103360 0x7fbfb019f080 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb01029a0 0x7fbfb0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 --2- 172.21.15.33:0/1764672595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfb01017b0 0x7fbfb0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 >> 172.21.15.33:0/1764672595 conn(0x7fbfb00fcd40 msgr2=0x7fbfb0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 shutdown_connections 2024-04-24T23:10:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.825+0000 7fbfb764d640 1 -- 172.21.15.33:0/1764672595 wait complete. 2024-04-24T23:10:34.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:10:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 -- 172.21.15.33:0/2645614568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e0104bb0 msgr2=0x7f51e0106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/2645614568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e0104bb0 0x7f51e0106fa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009900 tx=0x7f51d402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 -- 172.21.15.33:0/2645614568 shutdown_connections 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/2645614568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e01074e0 0x7f51e01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/2645614568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e0104bb0 0x7f51e0106fa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/2645614568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0102280 0x7f51e0104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 -- 172.21.15.33:0/2645614568 >> 172.21.15.33:0/2645614568 conn(0x7f51e00fbc30 msgr2=0x7f51e00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 -- 172.21.15.33:0/2645614568 shutdown_connections 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.941+0000 7f51e5a7b640 1 -- 172.21.15.33:0/2645614568 wait complete. 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 Processor -- start 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 -- start start 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102280 0x7f51e0101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0104bb0 0x7f51e0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 0x7f51e0100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010d420 con 0x7f51e01074e0 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010d2a0 con 0x7f51e0104bb0 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51e5a7b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010d5a0 con 0x7f51e0102280 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 0x7f51e0100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 0x7f51e0100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50902/0 (socket says 172.21.15.33:50902) 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 -- 172.21.15.33:0/27210916 learned_addr learned my addr 172.21.15.33:0/27210916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51deffd640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0104bb0 0x7f51e0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51df7fe640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102280 0x7f51e0101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 -- 172.21.15.33:0/27210916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102280 msgr2=0x7f51e0101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102280 0x7f51e0101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 -- 172.21.15.33:0/27210916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0104bb0 msgr2=0x7f51e0100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0104bb0 0x7f51e0100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.942+0000 7f51dffff640 1 -- 172.21.15.33:0/27210916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e0101000 con 0x7f51e01074e0 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51deffd640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0104bb0 0x7f51e0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51dffff640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 0x7f51e0100770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c970 tx=0x7f51d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8007bb0 con 0x7f51e01074e0 2024-04-24T23:10:34.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d8007d50 con 0x7f51e01074e0 2024-04-24T23:10:34.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e01a1c10 con 0x7f51e01074e0 2024-04-24T23:10:34.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d800f450 con 0x7f51e01074e0 2024-04-24T23:10:34.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.943+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e01a2160 con 0x7f51e01074e0 2024-04-24T23:10:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.944+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e00fd720 con 0x7f51e01074e0 2024-04-24T23:10:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.945+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f51d8004f20 con 0x7f51e01074e0 2024-04-24T23:10:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.946+0000 7f51dcff9640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51c80a2b90 0x7f51c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.946+0000 7f51df7fe640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51c80a2b90 0x7f51c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.946+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f51d80cf030 con 0x7f51e01074e0 2024-04-24T23:10:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.946+0000 7f51df7fe640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51c80a2b90 0x7f51c80a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005c40 tx=0x7f51d0005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:34.951+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51d8014030 con 0x7f51e01074e0 2024-04-24T23:10:35.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.178+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f51e00623e0 con 0x7f51e01074e0 2024-04-24T23:10:35.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.427+0000 7f51dcff9640 1 -- 172.21.15.33:0/27210916 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f51d809bd50 con 0x7f51e01074e0 2024-04-24T23:10:35.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51c80a2b90 msgr2=0x7f51c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51c80a2b90 0x7f51c80a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005c40 tx=0x7f51d0005bd0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 msgr2=0x7f51e0100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 0x7f51e0100770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c970 tx=0x7f51d800ce30 comp rx=0 tx=0).stop 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 shutdown_connections 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51c80a2b90 0x7f51c80a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e01074e0 0x7f51e0100770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e0104bb0 0x7f51e0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.429+0000 7f51e5a7b640 1 --2- 172.21.15.33:0/27210916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102280 0x7f51e0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.430+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 >> 172.21.15.33:0/27210916 conn(0x7f51e00fbc30 msgr2=0x7f51e0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.430+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 shutdown_connections 2024-04-24T23:10:35.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.430+0000 7f51e5a7b640 1 -- 172.21.15.33:0/27210916 wait complete. 2024-04-24T23:10:35.442 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-24T23:10:35.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:10:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/2863570715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58101890 msgr2=0x7ffa58101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/2863570715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58101890 0x7ffa58101d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/2863570715 shutdown_connections 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/2863570715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58102250 0x7ffa58108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/2863570715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58101890 0x7ffa58101d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/2863570715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 0x7ffa58100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/2863570715 >> 172.21.15.33:0/2863570715 conn(0x7ffa580fbc30 msgr2=0x7ffa580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/2863570715 shutdown_connections 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/2863570715 wait complete. 2024-04-24T23:10:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 Processor -- start 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- start start 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 0x7ffa5806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58101890 0x7ffa5819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 0x7ffa5819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5810c300 con 0x7ffa58102250 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5810c180 con 0x7ffa58101890 2024-04-24T23:10:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5f0cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5810c480 con 0x7ffa581006a0 2024-04-24T23:10:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.547+0000 7ffa5d66a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 0x7ffa5819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 0x7ffa5819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50930/0 (socket says 172.21.15.33:50930) 2024-04-24T23:10:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 -- 172.21.15.33:0/482380713 learned_addr learned my addr 172.21.15.33:0/482380713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5ce69640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 0x7ffa5806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa57fff640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58101890 0x7ffa5819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 -- 172.21.15.33:0/482380713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 msgr2=0x7ffa5806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 0x7ffa5806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 -- 172.21.15.33:0/482380713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58101890 msgr2=0x7ffa5819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58101890 0x7ffa5819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 -- 172.21.15.33:0/482380713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5819f6e0 con 0x7ffa58102250 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5ce69640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 0x7ffa5806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa57fff640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58101890 0x7ffa5819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5d66a640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 0x7ffa5819ee20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000eb10 tx=0x7ffa5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50004b20 con 0x7ffa58102250 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa50004cc0 con 0x7ffa58102250 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5819f9d0 con 0x7ffa58102250 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50010760 con 0x7ffa58102250 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.548+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5819ff20 con 0x7ffa58102250 2024-04-24T23:10:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.549+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa58107a30 con 0x7ffa58102250 2024-04-24T23:10:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.550+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa50005020 con 0x7ffa58102250 2024-04-24T23:10:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.551+0000 7ffa55ffb640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa480a2c10 0x7ffa480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.551+0000 7ffa5ce69640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa480a2c10 0x7ffa480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.551+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffa500cf8e0 con 0x7ffa58102250 2024-04-24T23:10:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.552+0000 7ffa5ce69640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa480a2c10 0x7ffa480a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0058c0 tx=0x7ffa4c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.556+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa5009c600 con 0x7ffa58102250 2024-04-24T23:10:35.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:35 smithi038 ceph-mon[32450]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2655229204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1764672595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/27210916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:10:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:35 smithi117 ceph-mon[32529]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2655229204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1764672595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/27210916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:10:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:35 smithi033 ceph-mon[29201]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2655229204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1764672595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/27210916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:10:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:35.780+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ffa58062340 con 0x7ffa58102250 2024-04-24T23:10:36.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.432+0000 7ffa55ffb640 1 -- 172.21.15.33:0/482380713 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ffa5009bf00 con 0x7ffa58102250 2024-04-24T23:10:36.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa480a2c10 msgr2=0x7ffa480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa480a2c10 0x7ffa480a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0058c0 tx=0x7ffa4c005850 comp rx=0 tx=0).stop 2024-04-24T23:10:36.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 msgr2=0x7ffa5819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 0x7ffa5819ee20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000eb10 tx=0x7ffa5000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 shutdown_connections 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa480a2c10 0x7ffa480a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa58102250 0x7ffa5819ee20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa58101890 0x7ffa5819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 --2- 172.21.15.33:0/482380713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa581006a0 0x7ffa5806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 >> 172.21.15.33:0/482380713 conn(0x7ffa580fbc30 msgr2=0x7ffa58104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.436+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 shutdown_connections 2024-04-24T23:10:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.437+0000 7ffa5f0cd640 1 -- 172.21.15.33:0/482380713 wait complete. 2024-04-24T23:10:36.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 -- 172.21.15.33:0/2791140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c104b90 msgr2=0x7f549c106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 --2- 172.21.15.33:0/2791140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c104b90 0x7f549c106f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5490009900 tx=0x7f549002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 -- 172.21.15.33:0/2791140250 shutdown_connections 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 --2- 172.21.15.33:0/2791140250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f549c1074c0 0x7f549c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 --2- 172.21.15.33:0/2791140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c104b90 0x7f549c106f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 --2- 172.21.15.33:0/2791140250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c102260 0x7f549c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 -- 172.21.15.33:0/2791140250 >> 172.21.15.33:0/2791140250 conn(0x7f549c0fbc10 msgr2=0x7f549c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 -- 172.21.15.33:0/2791140250 shutdown_connections 2024-04-24T23:10:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.564+0000 7f54a1c58640 1 -- 172.21.15.33:0/2791140250 wait complete. 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 Processor -- start 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 -- start start 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c102260 0x7f549c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f549c104b90 0x7f549c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 0x7f549c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10d470 con 0x7f549c1074c0 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10d2f0 con 0x7f549c104b90 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f54a1c58640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10d5f0 con 0x7f549c102260 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f549bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 0x7f549c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f549bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 0x7f549c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40362/0 (socket says 172.21.15.33:40362) 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f549bfff640 1 -- 172.21.15.33:0/723105783 learned_addr learned my addr 172.21.15.33:0/723105783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f549b7fe640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c102260 0x7f549c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.565+0000 7f549affd640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f549c104b90 0x7f549c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f549bfff640 1 -- 172.21.15.33:0/723105783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c102260 msgr2=0x7f549c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f549bfff640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c102260 0x7f549c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f549bfff640 1 -- 172.21.15.33:0/723105783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f549c104b90 msgr2=0x7f549c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f549bfff640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f549c104b90 0x7f549c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f549bfff640 1 -- 172.21.15.33:0/723105783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c19f730 con 0x7f549c1074c0 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f549bfff640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 0x7f549c19f0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f549400eaa0 tx=0x7f549400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549400cdb0 con 0x7f549c1074c0 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494005020 con 0x7f549c1074c0 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c19fa20 con 0x7f549c1074c0 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940116b0 con 0x7f549c1074c0 2024-04-24T23:10:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.566+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c19ff20 con 0x7f549c1074c0 2024-04-24T23:10:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.567+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c0fd700 con 0x7f549c1074c0 2024-04-24T23:10:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.568+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5494005560 con 0x7f549c1074c0 2024-04-24T23:10:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.569+0000 7f5498ff9640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54840a2b90 0x7f54840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.569+0000 7f549b7fe640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54840a2b90 0x7f54840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.569+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f54940cf530 con 0x7f549c1074c0 2024-04-24T23:10:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.569+0000 7f549b7fe640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54840a2b90 0x7f54840a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f548c005fd0 tx=0x7f548c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:36.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.573+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5494016030 con 0x7f549c1074c0 2024-04-24T23:10:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/27210916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:10:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:36 smithi033 ceph-mon[29201]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:36.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/482380713' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:10:36.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:36.799+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f549c062340 con 0x7f549c1074c0 2024-04-24T23:10:36.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/27210916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:10:36.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:36 smithi038 ceph-mon[32450]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:36.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/482380713' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:10:36.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/27210916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:10:36.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:36 smithi117 ceph-mon[32529]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:36.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/482380713' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:10:37.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.443+0000 7f5498ff9640 1 -- 172.21.15.33:0/723105783 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f549409c1f0 con 0x7f549c1074c0 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54840a2b90 msgr2=0x7f54840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54840a2b90 0x7f54840a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f548c005fd0 tx=0x7f548c00b390 comp rx=0 tx=0).stop 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 msgr2=0x7f549c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 0x7f549c19f0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f549400eaa0 tx=0x7f549400ef60 comp rx=0 tx=0).stop 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 shutdown_connections 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54840a2b90 0x7f54840a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549c1074c0 0x7f549c19f0c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f549c104b90 0x7f549c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 --2- 172.21.15.33:0/723105783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c102260 0x7f549c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.445+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 >> 172.21.15.33:0/723105783 conn(0x7f549c0fbc10 msgr2=0x7f549c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.446+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 shutdown_connections 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.446+0000 7f54a1c58640 1 -- 172.21.15.33:0/723105783 wait complete. 2024-04-24T23:10:37.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 -- 172.21.15.33:0/3477901922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc103340 msgr2=0x7faffc109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 --2- 172.21.15.33:0/3477901922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc103340 0x7faffc109bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faff4009900 tx=0x7faff402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 -- 172.21.15.33:0/3477901922 shutdown_connections 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 --2- 172.21.15.33:0/3477901922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc103340 0x7faffc109bd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 --2- 172.21.15.33:0/3477901922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faffc102980 0x7faffc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 --2- 172.21.15.33:0/3477901922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 0x7faffc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 -- 172.21.15.33:0/3477901922 >> 172.21.15.33:0/3477901922 conn(0x7faffc0fcd20 msgr2=0x7faffc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 -- 172.21.15.33:0/3477901922 shutdown_connections 2024-04-24T23:10:37.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 -- 172.21.15.33:0/3477901922 wait complete. 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.571+0000 7fb003e5d640 1 Processor -- start 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 -- start start 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 0x7faffc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 0x7faffc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faffc103340 0x7faffc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faffc10d410 con 0x7faffc102980 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faffc10d290 con 0x7faffc103340 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb003e5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faffc10d590 con 0x7faffc101790 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb0013f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 0x7faffc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 0x7faffc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb0013f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 0x7faffc197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40378/0 (socket says 172.21.15.33:40378) 2024-04-24T23:10:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb0013f8640 1 -- 172.21.15.33:0/51274995 learned_addr learned my addr 172.21.15.33:0/51274995 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb0023fa640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faffc103340 0x7faffc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 -- 172.21.15.33:0/51274995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faffc103340 msgr2=0x7faffc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faffc103340 0x7faffc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 -- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 msgr2=0x7faffc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 0x7faffc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 -- 172.21.15.33:0/51274995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc19f5b0 con 0x7faffc101790 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb0013f8640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 0x7faffc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.572+0000 7fb001bf9640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 0x7faffc197990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faff8005fd0 tx=0x7faff800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.573+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8003970 con 0x7faffc101790 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.573+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc19f8a0 con 0x7faffc101790 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.573+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8004020 con 0x7faffc101790 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.573+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80107b0 con 0x7faffc101790 2024-04-24T23:10:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.573+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc19fda0 con 0x7faffc101790 2024-04-24T23:10:37.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.574+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc108a50 con 0x7faffc101790 2024-04-24T23:10:37.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.575+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faff80041d0 con 0x7faffc101790 2024-04-24T23:10:37.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.575+0000 7faff2ffd640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafe80a2b90 0x7fafe80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.575+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7faff80cfb00 con 0x7faffc101790 2024-04-24T23:10:37.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.575+0000 7fb0013f8640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafe80a2b90 0x7fafe80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.576+0000 7fb0013f8640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafe80a2b90 0x7fafe80a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faffc198ea0 tx=0x7fafec00a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.579+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faff809c7c0 con 0x7faffc101790 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/482380713' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-24T23:10:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/723105783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:10:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:37.806+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7faffc02d170 con 0x7faffc101790 2024-04-24T23:10:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/482380713' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-24T23:10:37.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/723105783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/482380713' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-24T23:10:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/723105783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:10:37.927 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:10:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: 2024-04-24T23:10:37.630+0000 7fbfa7362700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T23:10:38.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.463+0000 7faff2ffd640 1 -- 172.21.15.33:0/51274995 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7faff809c0c0 con 0x7faffc101790 2024-04-24T23:10:38.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafe80a2b90 msgr2=0x7fafe80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafe80a2b90 0x7fafe80a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faffc198ea0 tx=0x7fafec00a380 comp rx=0 tx=0).stop 2024-04-24T23:10:38.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 msgr2=0x7faffc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 0x7faffc197990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faff8005fd0 tx=0x7faff800ef90 comp rx=0 tx=0).stop 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 shutdown_connections 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafe80a2b90 0x7fafe80a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faffc103340 0x7faffc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faffc102980 0x7faffc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 --2- 172.21.15.33:0/51274995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc101790 0x7faffc197990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 >> 172.21.15.33:0/51274995 conn(0x7faffc0fcd20 msgr2=0x7faffc105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 shutdown_connections 2024-04-24T23:10:38.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.466+0000 7fb003e5d640 1 -- 172.21.15.33:0/51274995 wait complete. 2024-04-24T23:10:38.484 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-24T23:10:38.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T23:10:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.594+0000 7f755f7a2640 1 -- 172.21.15.33:0/3031872978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558107460 msgr2=0x7f75581098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.594+0000 7f755f7a2640 1 --2- 172.21.15.33:0/3031872978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558107460 0x7f75581098b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7554009ca0 tx=0x7f755402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- 172.21.15.33:0/3031872978 shutdown_connections 2024-04-24T23:10:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 --2- 172.21.15.33:0/3031872978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558107460 0x7f75581098b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 --2- 172.21.15.33:0/3031872978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7558104b30 0x7f7558106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 --2- 172.21.15.33:0/3031872978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7558102200 0x7f75581045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- 172.21.15.33:0/3031872978 >> 172.21.15.33:0/3031872978 conn(0x7f75580fbbf0 msgr2=0x7f75580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- 172.21.15.33:0/3031872978 shutdown_connections 2024-04-24T23:10:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- 172.21.15.33:0/3031872978 wait complete. 2024-04-24T23:10:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 Processor -- start 2024-04-24T23:10:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- start start 2024-04-24T23:10:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7558102200 0x7f7558199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 0x7f755819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7558107460 0x7f75581a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d420 con 0x7f7558104b30 2024-04-24T23:10:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d2a0 con 0x7f7558102200 2024-04-24T23:10:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.595+0000 7f755f7a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d5a0 con 0x7f7558107460 2024-04-24T23:10:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 0x7f755819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 0x7f755819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40404/0 (socket says 172.21.15.33:40404) 2024-04-24T23:10:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 -- 172.21.15.33:0/207330414 learned_addr learned my addr 172.21.15.33:0/207330414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755dd3f640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7558107460 0x7f75581a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755d53e640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7558102200 0x7f7558199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 -- 172.21.15.33:0/207330414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7558107460 msgr2=0x7f75581a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7558107460 0x7f75581a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 -- 172.21.15.33:0/207330414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7558102200 msgr2=0x7f7558199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7558102200 0x7f7558199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 -- 172.21.15.33:0/207330414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75581a18f0 con 0x7f7558104b30 2024-04-24T23:10:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755cd3d640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 0x7f755819a1f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9f0 tx=0x7f754800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800ccc0 con 0x7f7558104b30 2024-04-24T23:10:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7548005500 con 0x7f7558104b30 2024-04-24T23:10:38.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548010690 con 0x7f7558104b30 2024-04-24T23:10:38.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75581a1be0 con 0x7f7558104b30 2024-04-24T23:10:38.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.596+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75581a2130 con 0x7f7558104b30 2024-04-24T23:10:38.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.597+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7558108f50 con 0x7f7558104b30 2024-04-24T23:10:38.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.598+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75480046c0 con 0x7f7558104b30 2024-04-24T23:10:38.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.599+0000 7f754e7fc640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f75440a2c10 0x7f75440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.599+0000 7f755d53e640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f75440a2c10 0x7f75440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.599+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f75480cf5f0 con 0x7f7558104b30 2024-04-24T23:10:38.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.599+0000 7f755d53e640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f75440a2c10 0x7f75440a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f755819ab80 tx=0x7f7550005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.603+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f754809c310 con 0x7f7558104b30 2024-04-24T23:10:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/723105783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:10:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] up:active 2024-04-24T23:10:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: Cluster is now healthy 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/51274995' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:10:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:10:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.815+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7558061b80 con 0x7f75440a2c10 2024-04-24T23:10:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.839+0000 7f754e7fc640 1 -- 172.21.15.33:0/207330414 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7558061b80 con 0x7f75440a2c10 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f75440a2c10 msgr2=0x7f75440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f75440a2c10 0x7f75440a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f755819ab80 tx=0x7f7550005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 msgr2=0x7f755819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 0x7f755819a1f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9f0 tx=0x7f754800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 shutdown_connections 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f75440a2c10 0x7f75440a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7558107460 0x7f75581a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558104b30 0x7f755819a1f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 --2- 172.21.15.33:0/207330414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7558102200 0x7f7558199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 >> 172.21.15.33:0/207330414 conn(0x7f75580fbbf0 msgr2=0x7f75581018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 shutdown_connections 2024-04-24T23:10:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.841+0000 7f755f7a2640 1 -- 172.21.15.33:0/207330414 wait complete. 2024-04-24T23:10:38.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/723105783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] up:active 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: Cluster is now healthy 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/51274995' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:10:38.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:38 smithi038 ceph-mon[32450]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/723105783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] up:active 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: Cluster is now healthy 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/51274995' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:10:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:38 smithi117 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:10:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 -- 172.21.15.33:0/688392108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00100680 msgr2=0x7f3c00100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 --2- 172.21.15.33:0/688392108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00100680 0x7f3c00100aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009900 tx=0x7f3bfc02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 -- 172.21.15.33:0/688392108 shutdown_connections 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 --2- 172.21.15.33:0/688392108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c00102230 0x7f3c00108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 --2- 172.21.15.33:0/688392108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c00101870 0x7f3c00101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 --2- 172.21.15.33:0/688392108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00100680 0x7f3c00100aa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 -- 172.21.15.33:0/688392108 >> 172.21.15.33:0/688392108 conn(0x7f3c000fbbf0 msgr2=0x7f3c000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 -- 172.21.15.33:0/688392108 shutdown_connections 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 -- 172.21.15.33:0/688392108 wait complete. 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.958+0000 7f3c0869a640 1 Processor -- start 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 -- start start 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c00100680 0x7f3c00069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 0x7f3c0019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c00102230 0x7f3c0019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0010c300 con 0x7f3c00101870 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0010c180 con 0x7f3c00102230 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c0869a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0010c480 con 0x7f3c00100680 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 0x7f3c0019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c06436640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c00100680 0x7f3c00069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 0x7f3c0019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40432/0 (socket says 172.21.15.33:40432) 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 -- 172.21.15.33:0/2781081828 learned_addr learned my addr 172.21.15.33:0/2781081828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c06c37640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c00102230 0x7f3c0019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 -- 172.21.15.33:0/2781081828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c00100680 msgr2=0x7f3c00069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c00100680 0x7f3c00069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 -- 172.21.15.33:0/2781081828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c00102230 msgr2=0x7f3c0019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c00102230 0x7f3c0019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 -- 172.21.15.33:0/2781081828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0019f780 con 0x7f3c00101870 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3c05c35640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 0x7f3c0019c850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000e9f0 tx=0x7f3bf000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.959+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000cd80 con 0x7f3c00101870 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.960+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf0005500 con 0x7f3c00101870 2024-04-24T23:10:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.960+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0019fa10 con 0x7f3c00101870 2024-04-24T23:10:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.960+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0010690 con 0x7f3c00101870 2024-04-24T23:10:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.960+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0019ff10 con 0x7f3c00101870 2024-04-24T23:10:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.961+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c00107a70 con 0x7f3c00101870 2024-04-24T23:10:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.962+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3bf00046c0 con 0x7f3c00101870 2024-04-24T23:10:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.962+0000 7f3bf77fe640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bec0a2b90 0x7f3bec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.962+0000 7f3c06436640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bec0a2b90 0x7f3bec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.963+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3bf00cf6a0 con 0x7f3c00101870 2024-04-24T23:10:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.963+0000 7f3c06436640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bec0a2b90 0x7f3bec0a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005ab0 tx=0x7f3bfc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:38.967+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3bf009c3c0 con 0x7f3c00101870 2024-04-24T23:10:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.184+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3c00061c10 con 0x7f3bec0a2b90 2024-04-24T23:10:39.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:10:38.819+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:10:39.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:10:38.819+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:10:39.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:10:38.819+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:10:39.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:10:38.819+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:10:39.205 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:10:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:10:38.819+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:10:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.211+0000 7f3bf77fe640 1 -- 172.21.15.33:0/2781081828 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c00061c10 con 0x7f3bec0a2b90 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bec0a2b90 msgr2=0x7f3bec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bec0a2b90 0x7f3bec0a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005ab0 tx=0x7f3bfc000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 msgr2=0x7f3c0019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 0x7f3c0019c850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000e9f0 tx=0x7f3bf000edf0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 shutdown_connections 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bec0a2b90 0x7f3bec0a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c00102230 0x7f3c0019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c00101870 0x7f3c0019c850 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 --2- 172.21.15.33:0/2781081828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c00100680 0x7f3c00069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 >> 172.21.15.33:0/2781081828 conn(0x7f3c000fbbf0 msgr2=0x7f3c00104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 shutdown_connections 2024-04-24T23:10:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.213+0000 7f3c0869a640 1 -- 172.21.15.33:0/2781081828 wait complete. 2024-04-24T23:10:39.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T23:10:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.331+0000 7f3f8622c640 1 -- 172.21.15.33:0/2938923476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f801006a0 msgr2=0x7f3f80100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.331+0000 7f3f8622c640 1 --2- 172.21.15.33:0/2938923476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f801006a0 0x7f3f80100ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009900 tx=0x7f3f6802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 -- 172.21.15.33:0/2938923476 shutdown_connections 2024-04-24T23:10:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 --2- 172.21.15.33:0/2938923476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f80102250 0x7f3f80108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 --2- 172.21.15.33:0/2938923476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80101890 0x7f3f80101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 --2- 172.21.15.33:0/2938923476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f801006a0 0x7f3f80100ac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 -- 172.21.15.33:0/2938923476 >> 172.21.15.33:0/2938923476 conn(0x7f3f800fbc30 msgr2=0x7f3f800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 -- 172.21.15.33:0/2938923476 shutdown_connections 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 -- 172.21.15.33:0/2938923476 wait complete. 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 Processor -- start 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 -- start start 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f801006a0 0x7f3f8006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 0x7f3f8019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80102250 0x7f3f8019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.332+0000 7f3f8622c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010c320 con 0x7f3f80101890 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f8622c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010c1a0 con 0x7f3f80102250 2024-04-24T23:10:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f8622c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010c4a0 con 0x7f3f801006a0 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 0x7f3f8019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 0x7f3f8019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40440/0 (socket says 172.21.15.33:40440) 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 -- 172.21.15.33:0/1503860785 learned_addr learned my addr 172.21.15.33:0/1503860785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7ffff640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80102250 0x7f3f8019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7f7fe640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f801006a0 0x7f3f8006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 -- 172.21.15.33:0/1503860785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f801006a0 msgr2=0x7f3f8006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f801006a0 0x7f3f8006a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 -- 172.21.15.33:0/1503860785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80102250 msgr2=0x7f3f8019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80102250 0x7f3f8019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 -- 172.21.15.33:0/1503860785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8019f6f0 con 0x7f3f80101890 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7ffff640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80102250 0x7f3f8019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7effd640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 0x7f3f8019c7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c890 tx=0x7f3f7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70004220 con 0x7f3f80101890 2024-04-24T23:10:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f700043c0 con 0x7f3f80101890 2024-04-24T23:10:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8019f9e0 con 0x7f3f80101890 2024-04-24T23:10:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700049e0 con 0x7f3f80101890 2024-04-24T23:10:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.333+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8019ff30 con 0x7f3f80101890 2024-04-24T23:10:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.334+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f801079c0 con 0x7f3f80101890 2024-04-24T23:10:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.335+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3f700079b0 con 0x7f3f80101890 2024-04-24T23:10:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.336+0000 7f3f7cff9640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f6c0a2b90 0x7f3f6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.336+0000 7f3f7f7fe640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f6c0a2b90 0x7f3f6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.336+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3f700cf2e0 con 0x7f3f80101890 2024-04-24T23:10:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.336+0000 7f3f7f7fe640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f6c0a2b90 0x7f3f6c0a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3f68005ab0 tx=0x7f3f68000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.340+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f70014030 con 0x7f3f80101890 2024-04-24T23:10:39.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.557+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3f80061b80 con 0x7f3f6c0a2b90 2024-04-24T23:10:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.562+0000 7f3f7cff9640 1 -- 172.21.15.33:0/1503860785 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3f80061b80 con 0x7f3f6c0a2b90 2024-04-24T23:10:39.564 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/d6e74965-e912-4c8d-80bb-e10a085d5eba 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f6c0a2b90 msgr2=0x7f3f6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f6c0a2b90 0x7f3f6c0a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3f68005ab0 tx=0x7f3f68000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 msgr2=0x7f3f8019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 0x7f3f8019c7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c890 tx=0x7f3f7000cd50 comp rx=0 tx=0).stop 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 shutdown_connections 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f6c0a2b90 0x7f3f6c0a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f80102250 0x7f3f8019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f80101890 0x7f3f8019c7f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 --2- 172.21.15.33:0/1503860785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f801006a0 0x7f3f8006a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 >> 172.21.15.33:0/1503860785 conn(0x7f3f800fbc30 msgr2=0x7f3f80104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 shutdown_connections 2024-04-24T23:10:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.565+0000 7f3f8622c640 1 -- 172.21.15.33:0/1503860785 wait complete. 2024-04-24T23:10:39.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:39 smithi033 ceph-mon[29201]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T23:10:39.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:39 smithi033 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:10:39.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:39 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] up:active 2024-04-24T23:10:39.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:39 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-04-24T23:10:39.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:39 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:10:39.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T23:10:39.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 -- 172.21.15.33:0/1561894107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 msgr2=0x7f6e2010baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 --2- 172.21.15.33:0/1561894107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e2010baf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009ca0 tx=0x7f6e1802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 -- 172.21.15.33:0/1561894107 shutdown_connections 2024-04-24T23:10:39.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 --2- 172.21.15.33:0/1561894107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e2010baf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 --2- 172.21.15.33:0/1561894107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200741a0 0x7f6e20074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 --2- 172.21.15.33:0/1561894107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e20075780 0x7f6e20073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 -- 172.21.15.33:0/1561894107 >> 172.21.15.33:0/1561894107 conn(0x7f6e200fcd10 msgr2=0x7f6e200ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 -- 172.21.15.33:0/1561894107 shutdown_connections 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.682+0000 7f6e25815640 1 -- 172.21.15.33:0/1561894107 wait complete. 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 Processor -- start 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 -- start start 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e200741a0 0x7f6e20199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20075780 0x7f6e2019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e201a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010f600 con 0x7f6e20109660 2024-04-24T23:10:39.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010f480 con 0x7f6e200741a0 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e25815640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010f780 con 0x7f6e20075780 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e201a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e17fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20075780 0x7f6e2019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e201a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40456/0 (socket says 172.21.15.33:40456) 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 -- 172.21.15.33:0/3527456992 learned_addr learned my addr 172.21.15.33:0/3527456992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1effd640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e200741a0 0x7f6e20199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 -- 172.21.15.33:0/3527456992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20075780 msgr2=0x7f6e2019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20075780 0x7f6e2019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 -- 172.21.15.33:0/3527456992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e200741a0 msgr2=0x7f6e20199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e200741a0 0x7f6e20199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.683+0000 7f6e1f7fe640 1 -- 172.21.15.33:0/3527456992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e201a1880 con 0x7f6e20109660 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.684+0000 7f6e1f7fe640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e201a11e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e18005e30 tx=0x7f6e18031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.684+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18003c30 con 0x7f6e20109660 2024-04-24T23:10:39.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.684+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201a1b10 con 0x7f6e20109660 2024-04-24T23:10:39.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.684+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e18003dd0 con 0x7f6e20109660 2024-04-24T23:10:39.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.684+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1802fde0 con 0x7f6e20109660 2024-04-24T23:10:39.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.684+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a2060 con 0x7f6e20109660 2024-04-24T23:10:39.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.685+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e20108390 con 0x7f6e20109660 2024-04-24T23:10:39.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.686+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6e18004050 con 0x7f6e20109660 2024-04-24T23:10:39.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.687+0000 7f6e24813640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6e0c0a2b90 0x7f6e0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.687+0000 7f6e1effd640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6e0c0a2b90 0x7f6e0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.687+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6e180f06e0 con 0x7f6e20109660 2024-04-24T23:10:39.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.687+0000 7f6e1effd640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6e0c0a2b90 0x7f6e0c0a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e2019b140 tx=0x7f6e1000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.691+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6e180bd400 con 0x7f6e20109660 2024-04-24T23:10:39.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.903+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6e20061b80 con 0x7f6e0c0a2b90 2024-04-24T23:10:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:39 smithi038 ceph-mon[32450]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T23:10:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:39 smithi038 ceph-mon[32450]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:10:39.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:39 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] up:active 2024-04-24T23:10:39.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:39 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-04-24T23:10:39.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:39 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:10:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:39 smithi117 ceph-mon[32529]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T23:10:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:39 smithi117 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:10:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:39 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] up:active 2024-04-24T23:10:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:39 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-04-24T23:10:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:39 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:10:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.928+0000 7f6e24813640 1 -- 172.21.15.33:0/3527456992 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6e20061b80 con 0x7f6e0c0a2b90 2024-04-24T23:10:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6e0c0a2b90 msgr2=0x7f6e0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6e0c0a2b90 0x7f6e0c0a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e2019b140 tx=0x7f6e1000b390 comp rx=0 tx=0).stop 2024-04-24T23:10:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 msgr2=0x7f6e201a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e201a11e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e18005e30 tx=0x7f6e18031ee0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 shutdown_connections 2024-04-24T23:10:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6e0c0a2b90 0x7f6e0c0a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e20109660 0x7f6e201a11e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20075780 0x7f6e2019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 --2- 172.21.15.33:0/3527456992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e200741a0 0x7f6e20199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 >> 172.21.15.33:0/3527456992 conn(0x7f6e200fcd10 msgr2=0x7f6e20105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 shutdown_connections 2024-04-24T23:10:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:39.930+0000 7f6e25815640 1 -- 172.21.15.33:0/3527456992 wait complete. 2024-04-24T23:10:39.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 -- 172.21.15.33:0/3180937206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 msgr2=0x7f0cdc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 --2- 172.21.15.33:0/3180937206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1098b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009ca0 tx=0x7f0cd402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 -- 172.21.15.33:0/3180937206 shutdown_connections 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 --2- 172.21.15.33:0/3180937206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1098b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 --2- 172.21.15.33:0/3180937206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc104b30 0x7f0cdc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 --2- 172.21.15.33:0/3180937206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc102200 0x7f0cdc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.047+0000 7f0ce4742640 1 -- 172.21.15.33:0/3180937206 >> 172.21.15.33:0/3180937206 conn(0x7f0cdc0fbbf0 msgr2=0x7f0cdc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 -- 172.21.15.33:0/3180937206 shutdown_connections 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 -- 172.21.15.33:0/3180937206 wait complete. 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 Processor -- start 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 -- start start 2024-04-24T23:10:40.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc102200 0x7f0cdc101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc104b30 0x7f0cdc1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10d470 con 0x7f0cdc107460 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10d2f0 con 0x7f0cdc104b30 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce4742640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10d5f0 con 0x7f0cdc102200 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce2cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce2cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40480/0 (socket says 172.21.15.33:40480) 2024-04-24T23:10:40.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.048+0000 7f0ce2cdf640 1 -- 172.21.15.33:0/857656639 learned_addr learned my addr 172.21.15.33:0/857656639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce24de640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc102200 0x7f0cdc101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce1cdd640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc104b30 0x7f0cdc1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce2cdf640 1 -- 172.21.15.33:0/857656639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc102200 msgr2=0x7f0cdc101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce2cdf640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc102200 0x7f0cdc101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce2cdf640 1 -- 172.21.15.33:0/857656639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc104b30 msgr2=0x7f0cdc1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce2cdf640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc104b30 0x7f0cdc1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce2cdf640 1 -- 172.21.15.33:0/857656639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc100f80 con 0x7f0cdc107460 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce2cdf640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1006f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40098d0 tx=0x7f0cd402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd402fc30 con 0x7f0cdc107460 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc1a1bf0 con 0x7f0cdc107460 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4003e50 con 0x7f0cdc107460 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40305c0 con 0x7f0cdc107460 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.049+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc1a2190 con 0x7f0cdc107460 2024-04-24T23:10:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.050+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc0fd6e0 con 0x7f0cdc107460 2024-04-24T23:10:40.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.051+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0cd4004b40 con 0x7f0cdc107460 2024-04-24T23:10:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.052+0000 7f0cd37fe640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0cc80a2b90 0x7f0cc80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.052+0000 7f0ce24de640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0cc80a2b90 0x7f0cc80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.052+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0cd40f08f0 con 0x7f0cdc107460 2024-04-24T23:10:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.052+0000 7f0ce24de640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0cc80a2b90 0x7f0cc80a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009890 tx=0x7f0cd8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.055+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0cd40bd5b0 con 0x7f0cdc107460 2024-04-24T23:10:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.268+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0cdc061b80 con 0x7f0cc80a2b90 2024-04-24T23:10:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.273+0000 7f0cd37fe640 1 -- 172.21.15.33:0/857656639 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0cdc061b80 con 0x7f0cc80a2b90 2024-04-24T23:10:40.274 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/63b4b8a1-1dbf-4f1c-a1f5-6d216109848e 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0cc80a2b90 msgr2=0x7f0cc80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0cc80a2b90 0x7f0cc80a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009890 tx=0x7f0cd8008040 comp rx=0 tx=0).stop 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 msgr2=0x7f0cdc1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1006f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40098d0 tx=0x7f0cd402f840 comp rx=0 tx=0).stop 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 shutdown_connections 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0cc80a2b90 0x7f0cc80a5050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cdc107460 0x7f0cdc1006f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc104b30 0x7f0cdc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 --2- 172.21.15.33:0/857656639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc102200 0x7f0cdc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 >> 172.21.15.33:0/857656639 conn(0x7f0cdc0fbbf0 msgr2=0x7f0cdc102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 shutdown_connections 2024-04-24T23:10:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.275+0000 7f0ce4742640 1 -- 172.21.15.33:0/857656639 wait complete. 2024-04-24T23:10:40.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.389+0000 7f61ad012640 1 -- 172.21.15.33:0/3022344002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8102230 msgr2=0x7f61a8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.389+0000 7f61ad012640 1 --2- 172.21.15.33:0/3022344002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8102230 0x7f61a8108ac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009ca0 tx=0x7f61a002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.389+0000 7f61ad012640 1 -- 172.21.15.33:0/3022344002 shutdown_connections 2024-04-24T23:10:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 --2- 172.21.15.33:0/3022344002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8102230 0x7f61a8108ac0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 --2- 172.21.15.33:0/3022344002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61a8101870 0x7f61a8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 --2- 172.21.15.33:0/3022344002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61a8100680 0x7f61a8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 -- 172.21.15.33:0/3022344002 >> 172.21.15.33:0/3022344002 conn(0x7f61a80fbbf0 msgr2=0x7f61a80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 -- 172.21.15.33:0/3022344002 shutdown_connections 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 -- 172.21.15.33:0/3022344002 wait complete. 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 Processor -- start 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.390+0000 7f61ad012640 1 -- start start 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 0x7f61a8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61a8101870 0x7f61a8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61a8102230 0x7f61a819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810c300 con 0x7f61a8100680 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810c180 con 0x7f61a8102230 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810c480 con 0x7f61a8101870 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 0x7f61a8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 0x7f61a8197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40500/0 (socket says 172.21.15.33:40500) 2024-04-24T23:10:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 -- 172.21.15.33:0/843992989 learned_addr learned my addr 172.21.15.33:0/843992989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a759e640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61a8102230 0x7f61a819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a659c640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61a8101870 0x7f61a8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 -- 172.21.15.33:0/843992989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61a8101870 msgr2=0x7f61a8197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61a8101870 0x7f61a8197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 -- 172.21.15.33:0/843992989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61a8102230 msgr2=0x7f61a819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61a8102230 0x7f61a819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 -- 172.21.15.33:0/843992989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a819f660 con 0x7f61a8100680 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61a6d9d640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 0x7f61a8197a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f619800e9c0 tx=0x7f619800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619800cd80 con 0x7f61a8100680 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61980035f0 con 0x7f61a8100680 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a819f950 con 0x7f61a8100680 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198010690 con 0x7f61a8100680 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.391+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a819fea0 con 0x7f61a8100680 2024-04-24T23:10:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.392+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a81a00d0 con 0x7f61a8100680 2024-04-24T23:10:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.393+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f61980037a0 con 0x7f61a8100680 2024-04-24T23:10:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.394+0000 7f618ffff640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61900a2b90 0x7f61900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.394+0000 7f61a659c640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61900a2b90 0x7f61900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.394+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f61980cf7f0 con 0x7f61a8100680 2024-04-24T23:10:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.395+0000 7f61a659c640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61900a2b90 0x7f61900a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f619c005fd0 tx=0x7f619c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.398+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f619809c510 con 0x7f61a8100680 2024-04-24T23:10:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.480+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6198098480 con 0x7f61a8100680 2024-04-24T23:10:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.628+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61a81a0340 con 0x7f61a8100680 2024-04-24T23:10:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.629+0000 7f618ffff640 1 -- 172.21.15.33:0/843992989 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f61980cafe0 con 0x7f61a8100680 2024-04-24T23:10:40.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:40.631 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:38.464642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:10:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:10:40.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61900a2b90 msgr2=0x7f61900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61900a2b90 0x7f61900a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f619c005fd0 tx=0x7f619c005850 comp rx=0 tx=0).stop 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 msgr2=0x7f61a8197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 0x7f61a8197a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f619800e9c0 tx=0x7f619800ee80 comp rx=0 tx=0).stop 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 shutdown_connections 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61900a2b90 0x7f61900a5050 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61a8102230 0x7f61a819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61a8101870 0x7f61a8197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 --2- 172.21.15.33:0/843992989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a8100680 0x7f61a8197a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.631+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 >> 172.21.15.33:0/843992989 conn(0x7f61a80fbbf0 msgr2=0x7f61a80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.632+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 shutdown_connections 2024-04-24T23:10:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.632+0000 7f61ad012640 1 -- 172.21.15.33:0/843992989 wait complete. 2024-04-24T23:10:40.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:40 smithi033 ceph-mon[29201]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:40 smithi033 ceph-mon[29201]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:40 smithi033 ceph-mon[29201]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 -- 172.21.15.33:0/4286346600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180101890 msgr2=0x7fa180101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 --2- 172.21.15.33:0/4286346600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180101890 0x7fa180101d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009900 tx=0x7fa16c02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 -- 172.21.15.33:0/4286346600 shutdown_connections 2024-04-24T23:10:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 --2- 172.21.15.33:0/4286346600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180102250 0x7fa180108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 --2- 172.21.15.33:0/4286346600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180101890 0x7fa180101d10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 --2- 172.21.15.33:0/4286346600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1801006a0 0x7fa180100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 -- 172.21.15.33:0/4286346600 >> 172.21.15.33:0/4286346600 conn(0x7fa1800fbc30 msgr2=0x7fa1800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 -- 172.21.15.33:0/4286346600 shutdown_connections 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.744+0000 7fa186af5640 1 -- 172.21.15.33:0/4286346600 wait complete. 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 Processor -- start 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 -- start start 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1801006a0 0x7fa18006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180101890 0x7fa18019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 0x7fa18019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa18010c300 con 0x7fa180102250 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa18010c180 con 0x7fa180101890 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa186af5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa18010c480 con 0x7fa1801006a0 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 0x7fa18019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 0x7fa18019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40522/0 (socket says 172.21.15.33:40522) 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 -- 172.21.15.33:0/3624399719 learned_addr learned my addr 172.21.15.33:0/3624399719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa17ffff640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180101890 0x7fa18019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa184891640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1801006a0 0x7fa18006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 -- 172.21.15.33:0/3624399719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1801006a0 msgr2=0x7fa18006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1801006a0 0x7fa18006a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 -- 172.21.15.33:0/3624399719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180101890 msgr2=0x7fa18019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180101890 0x7fa18019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa185092640 1 -- 172.21.15.33:0/3624399719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18019f640 con 0x7fa180102250 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.745+0000 7fa17ffff640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180101890 0x7fa18019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.746+0000 7fa185092640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 0x7fa18019ed80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa17800eaa0 tx=0x7fa17800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.746+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17800cdb0 con 0x7fa180102250 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.746+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa178005020 con 0x7fa180102250 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.746+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1780116b0 con 0x7fa180102250 2024-04-24T23:10:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.746+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18019f930 con 0x7fa180102250 2024-04-24T23:10:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.746+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18019fe30 con 0x7fa180102250 2024-04-24T23:10:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.747+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1801079c0 con 0x7fa180102250 2024-04-24T23:10:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.748+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa178005560 con 0x7fa180102250 2024-04-24T23:10:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.748+0000 7fa17dffb640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1700a2c20 0x7fa1700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.749+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa1780cf740 con 0x7fa180102250 2024-04-24T23:10:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.749+0000 7fa184891640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1700a2c20 0x7fa1700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.749+0000 7fa184891640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1700a2c20 0x7fa1700a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa174005fd0 tx=0x7fa174005ce0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.753+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa178016030 con 0x7fa180102250 2024-04-24T23:10:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:40 smithi038 ceph-mon[32450]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:40 smithi038 ceph-mon[32450]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:40 smithi038 ceph-mon[32450]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:40 smithi117 ceph-mon[32529]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:40 smithi117 ceph-mon[32529]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:40 smithi117 ceph-mon[32529]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.938+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa18002d170 con 0x7fa180102250 2024-04-24T23:10:40.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.940+0000 7fa17dffb640 1 -- 172.21.15.33:0/3624399719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fa17809c300 con 0x7fa180102250 2024-04-24T23:10:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:40.942 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1700a2c20 msgr2=0x7fa1700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1700a2c20 0x7fa1700a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa174005fd0 tx=0x7fa174005ce0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 msgr2=0x7fa18019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 0x7fa18019ed80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa17800eaa0 tx=0x7fa17800ef60 comp rx=0 tx=0).stop 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 shutdown_connections 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1700a2c20 0x7fa1700a50e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa180102250 0x7fa18019ed80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa180101890 0x7fa18019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 --2- 172.21.15.33:0/3624399719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1801006a0 0x7fa18006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 >> 172.21.15.33:0/3624399719 conn(0x7fa1800fbc30 msgr2=0x7fa180104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 shutdown_connections 2024-04-24T23:10:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:40.942+0000 7fa186af5640 1 -- 172.21.15.33:0/3624399719 wait complete. 2024-04-24T23:10:40.955 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:10:40.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 -- 172.21.15.33:0/2325637386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac101890 msgr2=0x7fdeac101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/2325637386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac101890 0x7fdeac101d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009900 tx=0x7fdea002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 -- 172.21.15.33:0/2325637386 shutdown_connections 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/2325637386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac102250 0x7fdeac108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/2325637386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac101890 0x7fdeac101d10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/2325637386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 0x7fdeac100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 -- 172.21.15.33:0/2325637386 >> 172.21.15.33:0/2325637386 conn(0x7fdeac0fbc30 msgr2=0x7fdeac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 -- 172.21.15.33:0/2325637386 shutdown_connections 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.055+0000 7fdeb0f90640 1 -- 172.21.15.33:0/2325637386 wait complete. 2024-04-24T23:10:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 Processor -- start 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 -- start start 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 0x7fdeac104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac101890 0x7fdeac102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 0x7fdeac1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10c380 con 0x7fdeac102250 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10c200 con 0x7fdeac1006a0 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeb0f90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10c500 con 0x7fdeac101890 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeaad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 0x7fdeac1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeaad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 0x7fdeac1031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40536/0 (socket says 172.21.15.33:40536) 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeaad9d640 1 -- 172.21.15.33:0/3868244336 learned_addr learned my addr 172.21.15.33:0/3868244336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdeaa59c640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 0x7fdeac104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.056+0000 7fdea9d9b640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac101890 0x7fdeac102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaad9d640 1 -- 172.21.15.33:0/3868244336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac101890 msgr2=0x7fdeac102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaad9d640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac101890 0x7fdeac102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaad9d640 1 -- 172.21.15.33:0/3868244336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 msgr2=0x7fdeac104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaad9d640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 0x7fdeac104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaad9d640 1 -- 172.21.15.33:0/3868244336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac103a40 con 0x7fdeac102250 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdea9d9b640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac101890 0x7fdeac102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaa59c640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 0x7fdeac104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeaad9d640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 0x7fdeac1031b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdea400b970 tx=0x7fdea400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4019070 con 0x7fdeac102250 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac1a1ba0 con 0x7fdeac102250 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea400cb40 con 0x7fdeac102250 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4007610 con 0x7fdeac102250 2024-04-24T23:10:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.057+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac1a2110 con 0x7fdeac102250 2024-04-24T23:10:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.058+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac0689b0 con 0x7fdeac102250 2024-04-24T23:10:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.059+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdea401c070 con 0x7fdeac102250 2024-04-24T23:10:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.059+0000 7fde9b7fe640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde940a2c20 0x7fde940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.060+0000 7fdeaa59c640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde940a2c20 0x7fde940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.060+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdea40ce9e0 con 0x7fdeac102250 2024-04-24T23:10:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.060+0000 7fdeaa59c640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde940a2c20 0x7fde940a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdeac1016f0 tx=0x7fde9c00a5e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.064+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdea409b650 con 0x7fdeac102250 2024-04-24T23:10:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.244+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdeac0fdda0 con 0x7fde940a2c20 2024-04-24T23:10:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.254+0000 7fde9b7fe640 1 -- 172.21.15.33:0/3868244336 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340522 (secure 0 0 0) 0x7fdeac0fdda0 con 0x7fde940a2c20 2024-04-24T23:10:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:41.264 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde940a2c20 msgr2=0x7fde940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde940a2c20 0x7fde940a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdeac1016f0 tx=0x7fde9c00a5e0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 msgr2=0x7fdeac1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 0x7fdeac1031b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdea400b970 tx=0x7fdea400be30 comp rx=0 tx=0).stop 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 shutdown_connections 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde940a2c20 0x7fde940a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac102250 0x7fdeac1031b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeac101890 0x7fdeac102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 --2- 172.21.15.33:0/3868244336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdeac1006a0 0x7fdeac104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 >> 172.21.15.33:0/3868244336 conn(0x7fdeac0fbc30 msgr2=0x7fdeac0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 shutdown_connections 2024-04-24T23:10:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.258+0000 7fdeb0f90640 1 -- 172.21.15.33:0/3868244336 wait complete. 2024-04-24T23:10:41.271 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-24T23:10:39.784385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595232,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":254,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356684,"kb_used_data":11644,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585140,"statfs":{"total":1151940427776,"available":1151575183360,"internally_reserved":0,"allocated":11923456,"data_stored":7549626,"data_compressed":286125,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":365,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003289"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642383+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:10:18.642383+0000","last_peered":"2024-04-24T23:10:18.642383+0000","last_clean":"2024-04-24T23:10:18.642383+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:10:18.642383+0000","last_undegraded":"2024-04-24T23:10:18.642383+0000","last_fullsized":"2024-04-24T23:10:18.642383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641005+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:10:18.641005+0000","last_peered":"2024-04-24T23:10:18.641005+0000","last_clean":"2024-04-24T23:10:18.641005+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:10:18.641005+0000","last_undegraded":"2024-04-24T23:10:18.641005+0000","last_fullsized":"2024-04-24T23:10:18.641005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641194+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:10:18.641194+0000","last_peered":"2024-04-24T23:10:18.641194+0000","last_clean":"2024-04-24T23:10:18.641194+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:10:18.641194+0000","last_undegraded":"2024-04-24T23:10:18.641194+0000","last_fullsized":"2024-04-24T23:10:18.641194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929590+0000","last_change":"2024-04-24T23:10:12.342437+0000","last_active":"2024-04-24T23:10:17.929590+0000","last_peered":"2024-04-24T23:10:17.929590+0000","last_clean":"2024-04-24T23:10:17.929590+0000","last_became_active":"2024-04-24T23:10:12.342064+0000","last_became_peered":"2024-04-24T23:10:12.342064+0000","last_unstale":"2024-04-24T23:10:17.929590+0000","last_undegraded":"2024-04-24T23:10:17.929590+0000","last_fullsized":"2024-04-24T23:10:17.929590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:54.628493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641339+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:10:18.641339+0000","last_peered":"2024-04-24T23:10:18.641339+0000","last_clean":"2024-04-24T23:10:18.641339+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:10:18.641339+0000","last_undegraded":"2024-04-24T23:10:18.641339+0000","last_fullsized":"2024-04-24T23:10:18.641339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641022+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:10:18.641022+0000","last_peered":"2024-04-24T23:10:18.641022+0000","last_clean":"2024-04-24T23:10:18.641022+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:10:18.641022+0000","last_undegraded":"2024-04-24T23:10:18.641022+0000","last_fullsized":"2024-04-24T23:10:18.641022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641926+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:10:18.641926+0000","last_peered":"2024-04-24T23:10:18.641926+0000","last_clean":"2024-04-24T23:10:18.641926+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:10:18.641926+0000","last_undegraded":"2024-04-24T23:10:18.641926+0000","last_fullsized":"2024-04-24T23:10:18.641926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641057+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:10:18.641057+0000","last_peered":"2024-04-24T23:10:18.641057+0000","last_clean":"2024-04-24T23:10:18.641057+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:10:18.641057+0000","last_undegraded":"2024-04-24T23:10:18.641057+0000","last_fullsized":"2024-04-24T23:10:18.641057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364328+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:10:17.364328+0000","last_peered":"2024-04-24T23:10:17.364328+0000","last_clean":"2024-04-24T23:10:17.364328+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:10:17.364328+0000","last_undegraded":"2024-04-24T23:10:17.364328+0000","last_fullsized":"2024-04-24T23:10:17.364328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641939+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:10:18.641939+0000","last_peered":"2024-04-24T23:10:18.641939+0000","last_clean":"2024-04-24T23:10:18.641939+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:10:18.641939+0000","last_undegraded":"2024-04-24T23:10:18.641939+0000","last_fullsized":"2024-04-24T23:10:18.641939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641814+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:10:18.641814+0000","last_peered":"2024-04-24T23:10:18.641814+0000","last_clean":"2024-04-24T23:10:18.641814+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:10:18.641814+0000","last_undegraded":"2024-04-24T23:10:18.641814+0000","last_fullsized":"2024-04-24T23:10:18.641814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929484+0000","last_change":"2024-04-24T23:10:12.340314+0000","last_active":"2024-04-24T23:10:17.929484+0000","last_peered":"2024-04-24T23:10:17.929484+0000","last_clean":"2024-04-24T23:10:17.929484+0000","last_became_active":"2024-04-24T23:10:12.339907+0000","last_became_peered":"2024-04-24T23:10:12.339907+0000","last_unstale":"2024-04-24T23:10:17.929484+0000","last_undegraded":"2024-04-24T23:10:17.929484+0000","last_fullsized":"2024-04-24T23:10:17.929484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:23.960331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364096+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:10:17.364096+0000","last_peered":"2024-04-24T23:10:17.364096+0000","last_clean":"2024-04-24T23:10:17.364096+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:10:17.364096+0000","last_undegraded":"2024-04-24T23:10:17.364096+0000","last_fullsized":"2024-04-24T23:10:17.364096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641499+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:10:18.641499+0000","last_peered":"2024-04-24T23:10:18.641499+0000","last_clean":"2024-04-24T23:10:18.641499+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:10:18.641499+0000","last_undegraded":"2024-04-24T23:10:18.641499+0000","last_fullsized":"2024-04-24T23:10:18.641499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641850+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:10:18.641850+0000","last_peered":"2024-04-24T23:10:18.641850+0000","last_clean":"2024-04-24T23:10:18.641850+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:10:18.641850+0000","last_undegraded":"2024-04-24T23:10:18.641850+0000","last_fullsized":"2024-04-24T23:10:18.641850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363631+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:10:17.363631+0000","last_peered":"2024-04-24T23:10:17.363631+0000","last_clean":"2024-04-24T23:10:17.363631+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:10:17.363631+0000","last_undegraded":"2024-04-24T23:10:17.363631+0000","last_fullsized":"2024-04-24T23:10:17.363631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642275+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:10:18.642275+0000","last_peered":"2024-04-24T23:10:18.642275+0000","last_clean":"2024-04-24T23:10:18.642275+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:10:18.642275+0000","last_undegraded":"2024-04-24T23:10:18.642275+0000","last_fullsized":"2024-04-24T23:10:18.642275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642948+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:10:18.642948+0000","last_peered":"2024-04-24T23:10:18.642948+0000","last_clean":"2024-04-24T23:10:18.642948+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:10:18.642948+0000","last_undegraded":"2024-04-24T23:10:18.642948+0000","last_fullsized":"2024-04-24T23:10:18.642948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642640+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:10:18.642640+0000","last_peered":"2024-04-24T23:10:18.642640+0000","last_clean":"2024-04-24T23:10:18.642640+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:10:18.642640+0000","last_undegraded":"2024-04-24T23:10:18.642640+0000","last_fullsized":"2024-04-24T23:10:18.642640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642317+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:10:18.642317+0000","last_peered":"2024-04-24T23:10:18.642317+0000","last_clean":"2024-04-24T23:10:18.642317+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:10:18.642317+0000","last_undegraded":"2024-04-24T23:10:18.642317+0000","last_fullsized":"2024-04-24T23:10:18.642317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.643019+0000","last_change":"2024-04-24T23:10:11.321831+0000","last_active":"2024-04-24T23:10:18.643019+0000","last_peered":"2024-04-24T23:10:18.643019+0000","last_clean":"2024-04-24T23:10:18.643019+0000","last_became_active":"2024-04-24T23:10:11.321307+0000","last_became_peered":"2024-04-24T23:10:11.321307+0000","last_unstale":"2024-04-24T23:10:18.643019+0000","last_undegraded":"2024-04-24T23:10:18.643019+0000","last_fullsized":"2024-04-24T23:10:18.643019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:56.345320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641403+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:10:18.641403+0000","last_peered":"2024-04-24T23:10:18.641403+0000","last_clean":"2024-04-24T23:10:18.641403+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:10:18.641403+0000","last_undegraded":"2024-04-24T23:10:18.641403+0000","last_fullsized":"2024-04-24T23:10:18.641403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929202+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:10:17.929202+0000","last_peered":"2024-04-24T23:10:17.929202+0000","last_clean":"2024-04-24T23:10:17.929202+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:10:17.929202+0000","last_undegraded":"2024-04-24T23:10:17.929202+0000","last_fullsized":"2024-04-24T23:10:17.929202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641730+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:10:18.641730+0000","last_peered":"2024-04-24T23:10:18.641730+0000","last_clean":"2024-04-24T23:10:18.641730+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:10:18.641730+0000","last_undegraded":"2024-04-24T23:10:18.641730+0000","last_fullsized":"2024-04-24T23:10:18.641730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641013+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:10:18.641013+0000","last_peered":"2024-04-24T23:10:18.641013+0000","last_clean":"2024-04-24T23:10:18.641013+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:10:18.641013+0000","last_undegraded":"2024-04-24T23:10:18.641013+0000","last_fullsized":"2024-04-24T23:10:18.641013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929221+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:10:17.929221+0000","last_peered":"2024-04-24T23:10:17.929221+0000","last_clean":"2024-04-24T23:10:17.929221+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:10:17.929221+0000","last_undegraded":"2024-04-24T23:10:17.929221+0000","last_fullsized":"2024-04-24T23:10:17.929221+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.460162+0000","last_change":"2024-04-24T23:10:11.327335+0000","last_active":"2024-04-24T23:10:37.460162+0000","last_peered":"2024-04-24T23:10:37.460162+0000","last_clean":"2024-04-24T23:10:37.460162+0000","last_became_active":"2024-04-24T23:10:11.326835+0000","last_became_peered":"2024-04-24T23:10:11.326835+0000","last_unstale":"2024-04-24T23:10:37.460162+0000","last_undegraded":"2024-04-24T23:10:37.460162+0000","last_fullsized":"2024-04-24T23:10:37.460162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:21.267805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363824+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:10:17.363824+0000","last_peered":"2024-04-24T23:10:17.363824+0000","last_clean":"2024-04-24T23:10:17.363824+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:10:17.363824+0000","last_undegraded":"2024-04-24T23:10:17.363824+0000","last_fullsized":"2024-04-24T23:10:17.363824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929177+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:10:17.929177+0000","last_peered":"2024-04-24T23:10:17.929177+0000","last_clean":"2024-04-24T23:10:17.929177+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:10:17.929177+0000","last_undegraded":"2024-04-24T23:10:17.929177+0000","last_fullsized":"2024-04-24T23:10:17.929177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.643022+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:10:18.643022+0000","last_peered":"2024-04-24T23:10:18.643022+0000","last_clean":"2024-04-24T23:10:18.643022+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:10:18.643022+0000","last_undegraded":"2024-04-24T23:10:18.643022+0000","last_fullsized":"2024-04-24T23:10:18.643022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366372+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:10:17.366372+0000","last_peered":"2024-04-24T23:10:17.366372+0000","last_clean":"2024-04-24T23:10:17.366372+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:10:17.366372+0000","last_undegraded":"2024-04-24T23:10:17.366372+0000","last_fullsized":"2024-04-24T23:10:17.366372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642066+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:10:18.642066+0000","last_peered":"2024-04-24T23:10:18.642066+0000","last_clean":"2024-04-24T23:10:18.642066+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:10:18.642066+0000","last_undegraded":"2024-04-24T23:10:18.642066+0000","last_fullsized":"2024-04-24T23:10:18.642066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365926+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:10:17.365926+0000","last_peered":"2024-04-24T23:10:17.365926+0000","last_clean":"2024-04-24T23:10:17.365926+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:10:17.365926+0000","last_undegraded":"2024-04-24T23:10:17.365926+0000","last_fullsized":"2024-04-24T23:10:17.365926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642322+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:10:18.642322+0000","last_peered":"2024-04-24T23:10:18.642322+0000","last_clean":"2024-04-24T23:10:18.642322+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:10:18.642322+0000","last_undegraded":"2024-04-24T23:10:18.642322+0000","last_fullsized":"2024-04-24T23:10:18.642322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642372+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:10:18.642372+0000","last_peered":"2024-04-24T23:10:18.642372+0000","last_clean":"2024-04-24T23:10:18.642372+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:10:18.642372+0000","last_undegraded":"2024-04-24T23:10:18.642372+0000","last_fullsized":"2024-04-24T23:10:18.642372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640996+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:10:18.640996+0000","last_peered":"2024-04-24T23:10:18.640996+0000","last_clean":"2024-04-24T23:10:18.640996+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:10:18.640996+0000","last_undegraded":"2024-04-24T23:10:18.640996+0000","last_fullsized":"2024-04-24T23:10:18.640996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366102+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:10:17.366102+0000","last_peered":"2024-04-24T23:10:17.366102+0000","last_clean":"2024-04-24T23:10:17.366102+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:10:17.366102+0000","last_undegraded":"2024-04-24T23:10:17.366102+0000","last_fullsized":"2024-04-24T23:10:17.366102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642247+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:10:18.642247+0000","last_peered":"2024-04-24T23:10:18.642247+0000","last_clean":"2024-04-24T23:10:18.642247+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:10:18.642247+0000","last_undegraded":"2024-04-24T23:10:18.642247+0000","last_fullsized":"2024-04-24T23:10:18.642247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363770+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:10:17.363770+0000","last_peered":"2024-04-24T23:10:17.363770+0000","last_clean":"2024-04-24T23:10:17.363770+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:10:17.363770+0000","last_undegraded":"2024-04-24T23:10:17.363770+0000","last_fullsized":"2024-04-24T23:10:17.363770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640964+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:10:18.640964+0000","last_peered":"2024-04-24T23:10:18.640964+0000","last_clean":"2024-04-24T23:10:18.640964+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:10:18.640964+0000","last_undegraded":"2024-04-24T23:10:18.640964+0000","last_fullsized":"2024-04-24T23:10:18.640964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640891+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:10:18.640891+0000","last_peered":"2024-04-24T23:10:18.640891+0000","last_clean":"2024-04-24T23:10:18.640891+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:10:18.640891+0000","last_undegraded":"2024-04-24T23:10:18.640891+0000","last_fullsized":"2024-04-24T23:10:18.640891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365713+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:10:17.365713+0000","last_peered":"2024-04-24T23:10:17.365713+0000","last_clean":"2024-04-24T23:10:17.365713+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:10:17.365713+0000","last_undegraded":"2024-04-24T23:10:17.365713+0000","last_fullsized":"2024-04-24T23:10:17.365713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642464+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:10:18.642464+0000","last_peered":"2024-04-24T23:10:18.642464+0000","last_clean":"2024-04-24T23:10:18.642464+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:10:18.642464+0000","last_undegraded":"2024-04-24T23:10:18.642464+0000","last_fullsized":"2024-04-24T23:10:18.642464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640827+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:10:18.640827+0000","last_peered":"2024-04-24T23:10:18.640827+0000","last_clean":"2024-04-24T23:10:18.640827+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:10:18.640827+0000","last_undegraded":"2024-04-24T23:10:18.640827+0000","last_fullsized":"2024-04-24T23:10:18.640827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.449443+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:10:36.449443+0000","last_peered":"2024-04-24T23:10:36.449443+0000","last_clean":"2024-04-24T23:10:36.449443+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:10:36.449443+0000","last_undegraded":"2024-04-24T23:10:36.449443+0000","last_fullsized":"2024-04-24T23:10:36.449443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364210+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:10:17.364210+0000","last_peered":"2024-04-24T23:10:17.364210+0000","last_clean":"2024-04-24T23:10:17.364210+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:10:17.364210+0000","last_undegraded":"2024-04-24T23:10:17.364210+0000","last_fullsized":"2024-04-24T23:10:17.364210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642480+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:10:18.642480+0000","last_peered":"2024-04-24T23:10:18.642480+0000","last_clean":"2024-04-24T23:10:18.642480+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:10:18.642480+0000","last_undegraded":"2024-04-24T23:10:18.642480+0000","last_fullsized":"2024-04-24T23:10:18.642480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641473+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:10:18.641473+0000","last_peered":"2024-04-24T23:10:18.641473+0000","last_clean":"2024-04-24T23:10:18.641473+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:10:18.641473+0000","last_undegraded":"2024-04-24T23:10:18.641473+0000","last_fullsized":"2024-04-24T23:10:18.641473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641120+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:10:18.641120+0000","last_peered":"2024-04-24T23:10:18.641120+0000","last_clean":"2024-04-24T23:10:18.641120+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:10:18.641120+0000","last_undegraded":"2024-04-24T23:10:18.641120+0000","last_fullsized":"2024-04-24T23:10:18.641120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929399+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:10:17.929399+0000","last_peered":"2024-04-24T23:10:17.929399+0000","last_clean":"2024-04-24T23:10:17.929399+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:10:17.929399+0000","last_undegraded":"2024-04-24T23:10:17.929399+0000","last_fullsized":"2024-04-24T23:10:17.929399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366115+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:10:17.366115+0000","last_peered":"2024-04-24T23:10:17.366115+0000","last_clean":"2024-04-24T23:10:17.366115+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:10:17.366115+0000","last_undegraded":"2024-04-24T23:10:17.366115+0000","last_fullsized":"2024-04-24T23:10:17.366115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364112+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:10:17.364112+0000","last_peered":"2024-04-24T23:10:17.364112+0000","last_clean":"2024-04-24T23:10:17.364112+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:10:17.364112+0000","last_undegraded":"2024-04-24T23:10:17.364112+0000","last_fullsized":"2024-04-24T23:10:17.364112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642797+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:10:18.642797+0000","last_peered":"2024-04-24T23:10:18.642797+0000","last_clean":"2024-04-24T23:10:18.642797+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:10:18.642797+0000","last_undegraded":"2024-04-24T23:10:18.642797+0000","last_fullsized":"2024-04-24T23:10:18.642797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.449957+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:10:36.449957+0000","last_peered":"2024-04-24T23:10:36.449957+0000","last_clean":"2024-04-24T23:10:36.449957+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:10:36.449957+0000","last_undegraded":"2024-04-24T23:10:36.449957+0000","last_fullsized":"2024-04-24T23:10:36.449957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642749+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:10:18.642749+0000","last_peered":"2024-04-24T23:10:18.642749+0000","last_clean":"2024-04-24T23:10:18.642749+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:10:18.642749+0000","last_undegraded":"2024-04-24T23:10:18.642749+0000","last_fullsized":"2024-04-24T23:10:18.642749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366693+0000","last_change":"2024-04-24T23:10:16.378770+0000","last_active":"2024-04-24T23:10:17.366693+0000","last_peered":"2024-04-24T23:10:17.366693+0000","last_clean":"2024-04-24T23:10:17.366693+0000","last_became_active":"2024-04-24T23:10:16.377971+0000","last_became_peered":"2024-04-24T23:10:16.377971+0000","last_unstale":"2024-04-24T23:10:17.366693+0000","last_undegraded":"2024-04-24T23:10:17.366693+0000","last_fullsized":"2024-04-24T23:10:17.366693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:18.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642752+0000","last_change":"2024-04-24T23:10:11.320845+0000","last_active":"2024-04-24T23:10:18.642752+0000","last_peered":"2024-04-24T23:10:18.642752+0000","last_clean":"2024-04-24T23:10:18.642752+0000","last_became_active":"2024-04-24T23:10:11.320527+0000","last_became_peered":"2024-04-24T23:10:11.320527+0000","last_unstale":"2024-04-24T23:10:18.642752+0000","last_undegraded":"2024-04-24T23:10:18.642752+0000","last_fullsized":"2024-04-24T23:10:18.642752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:40.423345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364390+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:10:17.364390+0000","last_peered":"2024-04-24T23:10:17.364390+0000","last_clean":"2024-04-24T23:10:17.364390+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:10:17.364390+0000","last_undegraded":"2024-04-24T23:10:17.364390+0000","last_fullsized":"2024-04-24T23:10:17.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642065+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:10:18.642065+0000","last_peered":"2024-04-24T23:10:18.642065+0000","last_clean":"2024-04-24T23:10:18.642065+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:10:18.642065+0000","last_undegraded":"2024-04-24T23:10:18.642065+0000","last_fullsized":"2024-04-24T23:10:18.642065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366058+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:10:17.366058+0000","last_peered":"2024-04-24T23:10:17.366058+0000","last_clean":"2024-04-24T23:10:17.366058+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:10:17.366058+0000","last_undegraded":"2024-04-24T23:10:17.366058+0000","last_fullsized":"2024-04-24T23:10:17.366058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641572+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:10:18.641572+0000","last_peered":"2024-04-24T23:10:18.641572+0000","last_clean":"2024-04-24T23:10:18.641572+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:10:18.641572+0000","last_undegraded":"2024-04-24T23:10:18.641572+0000","last_fullsized":"2024-04-24T23:10:18.641572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642963+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:10:18.642963+0000","last_peered":"2024-04-24T23:10:18.642963+0000","last_clean":"2024-04-24T23:10:18.642963+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:10:18.642963+0000","last_undegraded":"2024-04-24T23:10:18.642963+0000","last_fullsized":"2024-04-24T23:10:18.642963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929212+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:10:17.929212+0000","last_peered":"2024-04-24T23:10:17.929212+0000","last_clean":"2024-04-24T23:10:17.929212+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:10:17.929212+0000","last_undegraded":"2024-04-24T23:10:17.929212+0000","last_fullsized":"2024-04-24T23:10:17.929212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641147+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:10:18.641147+0000","last_peered":"2024-04-24T23:10:18.641147+0000","last_clean":"2024-04-24T23:10:18.641147+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:10:18.641147+0000","last_undegraded":"2024-04-24T23:10:18.641147+0000","last_fullsized":"2024-04-24T23:10:18.641147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364453+0000","last_change":"2024-04-24T23:10:16.384232+0000","last_active":"2024-04-24T23:10:17.364453+0000","last_peered":"2024-04-24T23:10:17.364453+0000","last_clean":"2024-04-24T23:10:17.364453+0000","last_became_active":"2024-04-24T23:10:16.383807+0000","last_became_peered":"2024-04-24T23:10:16.383807+0000","last_unstale":"2024-04-24T23:10:17.364453+0000","last_undegraded":"2024-04-24T23:10:17.364453+0000","last_fullsized":"2024-04-24T23:10:17.364453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:45.366140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.460839+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:10:37.460839+0000","last_peered":"2024-04-24T23:10:37.460839+0000","last_clean":"2024-04-24T23:10:37.460839+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:10:37.460839+0000","last_undegraded":"2024-04-24T23:10:37.460839+0000","last_fullsized":"2024-04-24T23:10:37.460839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365819+0000","last_change":"2024-04-24T23:10:12.343904+0000","last_active":"2024-04-24T23:10:17.365819+0000","last_peered":"2024-04-24T23:10:17.365819+0000","last_clean":"2024-04-24T23:10:17.365819+0000","last_became_active":"2024-04-24T23:10:12.343388+0000","last_became_peered":"2024-04-24T23:10:12.343388+0000","last_unstale":"2024-04-24T23:10:17.365819+0000","last_undegraded":"2024-04-24T23:10:17.365819+0000","last_fullsized":"2024-04-24T23:10:17.365819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:39.150159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366371+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:10:17.366371+0000","last_peered":"2024-04-24T23:10:17.366371+0000","last_clean":"2024-04-24T23:10:17.366371+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:10:17.366371+0000","last_undegraded":"2024-04-24T23:10:17.366371+0000","last_fullsized":"2024-04-24T23:10:17.366371+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641828+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:10:18.641828+0000","last_peered":"2024-04-24T23:10:18.641828+0000","last_clean":"2024-04-24T23:10:18.641828+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:10:18.641828+0000","last_undegraded":"2024-04-24T23:10:18.641828+0000","last_fullsized":"2024-04-24T23:10:18.641828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641616+0000","last_change":"2024-04-24T23:10:12.340136+0000","last_active":"2024-04-24T23:10:18.641616+0000","last_peered":"2024-04-24T23:10:18.641616+0000","last_clean":"2024-04-24T23:10:18.641616+0000","last_became_active":"2024-04-24T23:10:12.339785+0000","last_became_peered":"2024-04-24T23:10:12.339785+0000","last_unstale":"2024-04-24T23:10:18.641616+0000","last_undegraded":"2024-04-24T23:10:18.641616+0000","last_fullsized":"2024-04-24T23:10:18.641616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:28.124443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642186+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:10:18.642186+0000","last_peered":"2024-04-24T23:10:18.642186+0000","last_clean":"2024-04-24T23:10:18.642186+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:10:18.642186+0000","last_undegraded":"2024-04-24T23:10:18.642186+0000","last_fullsized":"2024-04-24T23:10:18.642186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642756+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:10:18.642756+0000","last_peered":"2024-04-24T23:10:18.642756+0000","last_clean":"2024-04-24T23:10:18.642756+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:10:18.642756+0000","last_undegraded":"2024-04-24T23:10:18.642756+0000","last_fullsized":"2024-04-24T23:10:18.642756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641559+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:10:18.641559+0000","last_peered":"2024-04-24T23:10:18.641559+0000","last_clean":"2024-04-24T23:10:18.641559+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:10:18.641559+0000","last_undegraded":"2024-04-24T23:10:18.641559+0000","last_fullsized":"2024-04-24T23:10:18.641559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363941+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:10:17.363941+0000","last_peered":"2024-04-24T23:10:17.363941+0000","last_clean":"2024-04-24T23:10:17.363941+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:10:17.363941+0000","last_undegraded":"2024-04-24T23:10:17.363941+0000","last_fullsized":"2024-04-24T23:10:17.363941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641613+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:10:18.641613+0000","last_peered":"2024-04-24T23:10:18.641613+0000","last_clean":"2024-04-24T23:10:18.641613+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:10:18.641613+0000","last_undegraded":"2024-04-24T23:10:18.641613+0000","last_fullsized":"2024-04-24T23:10:18.641613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641230+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:10:18.641230+0000","last_peered":"2024-04-24T23:10:18.641230+0000","last_clean":"2024-04-24T23:10:18.641230+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:10:18.641230+0000","last_undegraded":"2024-04-24T23:10:18.641230+0000","last_fullsized":"2024-04-24T23:10:18.641230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364244+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:10:17.364244+0000","last_peered":"2024-04-24T23:10:17.364244+0000","last_clean":"2024-04-24T23:10:17.364244+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:10:17.364244+0000","last_undegraded":"2024-04-24T23:10:17.364244+0000","last_fullsized":"2024-04-24T23:10:17.364244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366250+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:10:17.366250+0000","last_peered":"2024-04-24T23:10:17.366250+0000","last_clean":"2024-04-24T23:10:17.366250+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:10:17.366250+0000","last_undegraded":"2024-04-24T23:10:17.366250+0000","last_fullsized":"2024-04-24T23:10:17.366250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363581+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:10:17.363581+0000","last_peered":"2024-04-24T23:10:17.363581+0000","last_clean":"2024-04-24T23:10:17.363581+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:10:17.363581+0000","last_undegraded":"2024-04-24T23:10:17.363581+0000","last_fullsized":"2024-04-24T23:10:17.363581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641268+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:10:18.641268+0000","last_peered":"2024-04-24T23:10:18.641268+0000","last_clean":"2024-04-24T23:10:18.641268+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:10:18.641268+0000","last_undegraded":"2024-04-24T23:10:18.641268+0000","last_fullsized":"2024-04-24T23:10:18.641268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640582+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:10:18.640582+0000","last_peered":"2024-04-24T23:10:18.640582+0000","last_clean":"2024-04-24T23:10:18.640582+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:10:18.640582+0000","last_undegraded":"2024-04-24T23:10:18.640582+0000","last_fullsized":"2024-04-24T23:10:18.640582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929239+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:10:17.929239+0000","last_peered":"2024-04-24T23:10:17.929239+0000","last_clean":"2024-04-24T23:10:17.929239+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:10:17.929239+0000","last_undegraded":"2024-04-24T23:10:17.929239+0000","last_fullsized":"2024-04-24T23:10:17.929239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640898+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:10:18.640898+0000","last_peered":"2024-04-24T23:10:18.640898+0000","last_clean":"2024-04-24T23:10:18.640898+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:10:18.640898+0000","last_undegraded":"2024-04-24T23:10:18.640898+0000","last_fullsized":"2024-04-24T23:10:18.640898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366239+0000","last_change":"2024-04-24T23:10:16.447738+0000","last_active":"2024-04-24T23:10:17.366239+0000","last_peered":"2024-04-24T23:10:17.366239+0000","last_clean":"2024-04-24T23:10:17.366239+0000","last_became_active":"2024-04-24T23:10:16.447345+0000","last_became_peered":"2024-04-24T23:10:16.447345+0000","last_unstale":"2024-04-24T23:10:17.366239+0000","last_undegraded":"2024-04-24T23:10:17.366239+0000","last_fullsized":"2024-04-24T23:10:17.366239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:07.830860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363580+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:10:17.363580+0000","last_peered":"2024-04-24T23:10:17.363580+0000","last_clean":"2024-04-24T23:10:17.363580+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:10:17.363580+0000","last_undegraded":"2024-04-24T23:10:17.363580+0000","last_fullsized":"2024-04-24T23:10:17.363580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641076+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:10:18.641076+0000","last_peered":"2024-04-24T23:10:18.641076+0000","last_clean":"2024-04-24T23:10:18.641076+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:10:18.641076+0000","last_undegraded":"2024-04-24T23:10:18.641076+0000","last_fullsized":"2024-04-24T23:10:18.641076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641298+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:10:18.641298+0000","last_peered":"2024-04-24T23:10:18.641298+0000","last_clean":"2024-04-24T23:10:18.641298+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:10:18.641298+0000","last_undegraded":"2024-04-24T23:10:18.641298+0000","last_fullsized":"2024-04-24T23:10:18.641298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642838+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:10:18.642838+0000","last_peered":"2024-04-24T23:10:18.642838+0000","last_clean":"2024-04-24T23:10:18.642838+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:10:18.642838+0000","last_undegraded":"2024-04-24T23:10:18.642838+0000","last_fullsized":"2024-04-24T23:10:18.642838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364149+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:10:17.364149+0000","last_peered":"2024-04-24T23:10:17.364149+0000","last_clean":"2024-04-24T23:10:17.364149+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:10:17.364149+0000","last_undegraded":"2024-04-24T23:10:17.364149+0000","last_fullsized":"2024-04-24T23:10:17.364149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640640+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:10:18.640640+0000","last_peered":"2024-04-24T23:10:18.640640+0000","last_clean":"2024-04-24T23:10:18.640640+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:10:18.640640+0000","last_undegraded":"2024-04-24T23:10:18.640640+0000","last_fullsized":"2024-04-24T23:10:18.640640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642923+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:10:18.642923+0000","last_peered":"2024-04-24T23:10:18.642923+0000","last_clean":"2024-04-24T23:10:18.642923+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:10:18.642923+0000","last_undegraded":"2024-04-24T23:10:18.642923+0000","last_fullsized":"2024-04-24T23:10:18.642923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641697+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:10:18.641697+0000","last_peered":"2024-04-24T23:10:18.641697+0000","last_clean":"2024-04-24T23:10:18.641697+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:10:18.641697+0000","last_undegraded":"2024-04-24T23:10:18.641697+0000","last_fullsized":"2024-04-24T23:10:18.641697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642102+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:10:18.642102+0000","last_peered":"2024-04-24T23:10:18.642102+0000","last_clean":"2024-04-24T23:10:18.642102+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:10:18.642102+0000","last_undegraded":"2024-04-24T23:10:18.642102+0000","last_fullsized":"2024-04-24T23:10:18.642102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929639+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:10:17.929639+0000","last_peered":"2024-04-24T23:10:17.929639+0000","last_clean":"2024-04-24T23:10:17.929639+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:10:17.929639+0000","last_undegraded":"2024-04-24T23:10:17.929639+0000","last_fullsized":"2024-04-24T23:10:17.929639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641106+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:10:18.641106+0000","last_peered":"2024-04-24T23:10:18.641106+0000","last_clean":"2024-04-24T23:10:18.641106+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:10:18.641106+0000","last_undegraded":"2024-04-24T23:10:18.641106+0000","last_fullsized":"2024-04-24T23:10:18.641106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929792+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:10:17.929792+0000","last_peered":"2024-04-24T23:10:17.929792+0000","last_clean":"2024-04-24T23:10:17.929792+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:10:17.929792+0000","last_undegraded":"2024-04-24T23:10:17.929792+0000","last_fullsized":"2024-04-24T23:10:17.929792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.450302+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:10:36.450302+0000","last_peered":"2024-04-24T23:10:36.450302+0000","last_clean":"2024-04-24T23:10:36.450302+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:10:36.450302+0000","last_undegraded":"2024-04-24T23:10:36.450302+0000","last_fullsized":"2024-04-24T23:10:36.450302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641216+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:10:18.641216+0000","last_peered":"2024-04-24T23:10:18.641216+0000","last_clean":"2024-04-24T23:10:18.641216+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:10:18.641216+0000","last_undegraded":"2024-04-24T23:10:18.641216+0000","last_fullsized":"2024-04-24T23:10:18.641216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.643080+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:10:18.643080+0000","last_peered":"2024-04-24T23:10:18.643080+0000","last_clean":"2024-04-24T23:10:18.643080+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:10:18.643080+0000","last_undegraded":"2024-04-24T23:10:18.643080+0000","last_fullsized":"2024-04-24T23:10:18.643080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640865+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:10:18.640865+0000","last_peered":"2024-04-24T23:10:18.640865+0000","last_clean":"2024-04-24T23:10:18.640865+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:10:18.640865+0000","last_undegraded":"2024-04-24T23:10:18.640865+0000","last_fullsized":"2024-04-24T23:10:18.640865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641432+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:10:18.641432+0000","last_peered":"2024-04-24T23:10:18.641432+0000","last_clean":"2024-04-24T23:10:18.641432+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:10:18.641432+0000","last_undegraded":"2024-04-24T23:10:18.641432+0000","last_fullsized":"2024-04-24T23:10:18.641432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929471+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:10:17.929471+0000","last_peered":"2024-04-24T23:10:17.929471+0000","last_clean":"2024-04-24T23:10:17.929471+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:10:17.929471+0000","last_undegraded":"2024-04-24T23:10:17.929471+0000","last_fullsized":"2024-04-24T23:10:17.929471+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642884+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:10:18.642884+0000","last_peered":"2024-04-24T23:10:18.642884+0000","last_clean":"2024-04-24T23:10:18.642884+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:10:18.642884+0000","last_undegraded":"2024-04-24T23:10:18.642884+0000","last_fullsized":"2024-04-24T23:10:18.642884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641720+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:10:18.641720+0000","last_peered":"2024-04-24T23:10:18.641720+0000","last_clean":"2024-04-24T23:10:18.641720+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:10:18.641720+0000","last_undegraded":"2024-04-24T23:10:18.641720+0000","last_fullsized":"2024-04-24T23:10:18.641720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929724+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:10:17.929724+0000","last_peered":"2024-04-24T23:10:17.929724+0000","last_clean":"2024-04-24T23:10:17.929724+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:10:17.929724+0000","last_undegraded":"2024-04-24T23:10:17.929724+0000","last_fullsized":"2024-04-24T23:10:17.929724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.449598+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:10:36.449598+0000","last_peered":"2024-04-24T23:10:36.449598+0000","last_clean":"2024-04-24T23:10:36.449598+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:10:36.449598+0000","last_undegraded":"2024-04-24T23:10:36.449598+0000","last_fullsized":"2024-04-24T23:10:36.449598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641810+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:10:18.641810+0000","last_peered":"2024-04-24T23:10:18.641810+0000","last_clean":"2024-04-24T23:10:18.641810+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:10:18.641810+0000","last_undegraded":"2024-04-24T23:10:18.641810+0000","last_fullsized":"2024-04-24T23:10:18.641810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363890+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:10:17.363890+0000","last_peered":"2024-04-24T23:10:17.363890+0000","last_clean":"2024-04-24T23:10:17.363890+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:10:17.363890+0000","last_undegraded":"2024-04-24T23:10:17.363890+0000","last_fullsized":"2024-04-24T23:10:17.363890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642519+0000","last_change":"2024-04-24T23:10:11.326142+0000","last_active":"2024-04-24T23:10:18.642519+0000","last_peered":"2024-04-24T23:10:18.642519+0000","last_clean":"2024-04-24T23:10:18.642519+0000","last_became_active":"2024-04-24T23:10:11.325728+0000","last_became_peered":"2024-04-24T23:10:11.325728+0000","last_unstale":"2024-04-24T23:10:18.642519+0000","last_undegraded":"2024-04-24T23:10:18.642519+0000","last_fullsized":"2024-04-24T23:10:18.642519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:42.647315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641820+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:10:18.641820+0000","last_peered":"2024-04-24T23:10:18.641820+0000","last_clean":"2024-04-24T23:10:18.641820+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:10:18.641820+0000","last_undegraded":"2024-04-24T23:10:18.641820+0000","last_fullsized":"2024-04-24T23:10:18.641820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641405+0000","last_change":"2024-04-24T23:10:16.387682+0000","last_active":"2024-04-24T23:10:18.641405+0000","last_peered":"2024-04-24T23:10:18.641405+0000","last_clean":"2024-04-24T23:10:18.641405+0000","last_became_active":"2024-04-24T23:10:16.387248+0000","last_became_peered":"2024-04-24T23:10:16.387248+0000","last_unstale":"2024-04-24T23:10:18.641405+0000","last_undegraded":"2024-04-24T23:10:18.641405+0000","last_fullsized":"2024-04-24T23:10:18.641405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:54.837632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366513+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:10:17.366513+0000","last_peered":"2024-04-24T23:10:17.366513+0000","last_clean":"2024-04-24T23:10:17.366513+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:10:17.366513+0000","last_undegraded":"2024-04-24T23:10:17.366513+0000","last_fullsized":"2024-04-24T23:10:17.366513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641267+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:10:18.641267+0000","last_peered":"2024-04-24T23:10:18.641267+0000","last_clean":"2024-04-24T23:10:18.641267+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:10:18.641267+0000","last_undegraded":"2024-04-24T23:10:18.641267+0000","last_fullsized":"2024-04-24T23:10:18.641267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366201+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:10:17.366201+0000","last_peered":"2024-04-24T23:10:17.366201+0000","last_clean":"2024-04-24T23:10:17.366201+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:10:17.366201+0000","last_undegraded":"2024-04-24T23:10:17.366201+0000","last_fullsized":"2024-04-24T23:10:17.366201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364058+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:10:17.364058+0000","last_peered":"2024-04-24T23:10:17.364058+0000","last_clean":"2024-04-24T23:10:17.364058+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:10:17.364058+0000","last_undegraded":"2024-04-24T23:10:17.364058+0000","last_fullsized":"2024-04-24T23:10:17.364058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641734+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:10:18.641734+0000","last_peered":"2024-04-24T23:10:18.641734+0000","last_clean":"2024-04-24T23:10:18.641734+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:10:18.641734+0000","last_undegraded":"2024-04-24T23:10:18.641734+0000","last_fullsized":"2024-04-24T23:10:18.641734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641560+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:10:18.641560+0000","last_peered":"2024-04-24T23:10:18.641560+0000","last_clean":"2024-04-24T23:10:18.641560+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:10:18.641560+0000","last_undegraded":"2024-04-24T23:10:18.641560+0000","last_fullsized":"2024-04-24T23:10:18.641560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642431+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:10:18.642431+0000","last_peered":"2024-04-24T23:10:18.642431+0000","last_clean":"2024-04-24T23:10:18.642431+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:10:18.642431+0000","last_undegraded":"2024-04-24T23:10:18.642431+0000","last_fullsized":"2024-04-24T23:10:18.642431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642824+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:10:18.642824+0000","last_peered":"2024-04-24T23:10:18.642824+0000","last_clean":"2024-04-24T23:10:18.642824+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:10:18.642824+0000","last_undegraded":"2024-04-24T23:10:18.642824+0000","last_fullsized":"2024-04-24T23:10:18.642824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641637+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:10:18.641637+0000","last_peered":"2024-04-24T23:10:18.641637+0000","last_clean":"2024-04-24T23:10:18.641637+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:10:18.641637+0000","last_undegraded":"2024-04-24T23:10:18.641637+0000","last_fullsized":"2024-04-24T23:10:18.641637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364390+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:10:17.364390+0000","last_peered":"2024-04-24T23:10:17.364390+0000","last_clean":"2024-04-24T23:10:17.364390+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:10:17.364390+0000","last_undegraded":"2024-04-24T23:10:17.364390+0000","last_fullsized":"2024-04-24T23:10:17.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641637+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:10:18.641637+0000","last_peered":"2024-04-24T23:10:18.641637+0000","last_clean":"2024-04-24T23:10:18.641637+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:10:18.641637+0000","last_undegraded":"2024-04-24T23:10:18.641637+0000","last_fullsized":"2024-04-24T23:10:18.641637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642012+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:10:18.642012+0000","last_peered":"2024-04-24T23:10:18.642012+0000","last_clean":"2024-04-24T23:10:18.642012+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:10:18.642012+0000","last_undegraded":"2024-04-24T23:10:18.642012+0000","last_fullsized":"2024-04-24T23:10:18.642012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641808+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:10:18.641808+0000","last_peered":"2024-04-24T23:10:18.641808+0000","last_clean":"2024-04-24T23:10:18.641808+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:10:18.641808+0000","last_undegraded":"2024-04-24T23:10:18.641808+0000","last_fullsized":"2024-04-24T23:10:18.641808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365821+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:10:17.365821+0000","last_peered":"2024-04-24T23:10:17.365821+0000","last_clean":"2024-04-24T23:10:17.365821+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:10:17.365821+0000","last_undegraded":"2024-04-24T23:10:17.365821+0000","last_fullsized":"2024-04-24T23:10:17.365821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640927+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:10:18.640927+0000","last_peered":"2024-04-24T23:10:18.640927+0000","last_clean":"2024-04-24T23:10:18.640927+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:10:18.640927+0000","last_undegraded":"2024-04-24T23:10:18.640927+0000","last_fullsized":"2024-04-24T23:10:18.640927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363909+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:10:17.363909+0000","last_peered":"2024-04-24T23:10:17.363909+0000","last_clean":"2024-04-24T23:10:17.363909+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:10:17.363909+0000","last_undegraded":"2024-04-24T23:10:17.363909+0000","last_fullsized":"2024-04-24T23:10:17.363909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363989+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:10:17.363989+0000","last_peered":"2024-04-24T23:10:17.363989+0000","last_clean":"2024-04-24T23:10:17.363989+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:10:17.363989+0000","last_undegraded":"2024-04-24T23:10:17.363989+0000","last_fullsized":"2024-04-24T23:10:17.363989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642193+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:10:18.642193+0000","last_peered":"2024-04-24T23:10:18.642193+0000","last_clean":"2024-04-24T23:10:18.642193+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:10:18.642193+0000","last_undegraded":"2024-04-24T23:10:18.642193+0000","last_fullsized":"2024-04-24T23:10:18.642193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4864,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":16204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1074670,"data_compressed":51245,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1071216,"data_compressed":51245,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":484142,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":480370,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1070816,"data_compressed":51245,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:10:41.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3121290713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0102230 msgr2=0x7f6ab0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3121290713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0102230 0x7f6ab0108ac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ca0 tx=0x7f6aa802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3121290713 shutdown_connections 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3121290713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0102230 0x7f6ab0108ac0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3121290713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0101870 0x7f6ab0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3121290713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100680 0x7f6ab0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3121290713 >> 172.21.15.33:0/3121290713 conn(0x7f6ab00fbbf0 msgr2=0x7f6ab00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3121290713 shutdown_connections 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3121290713 wait complete. 2024-04-24T23:10:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 Processor -- start 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- start start 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100680 0x7f6ab0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 0x7f6ab0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0102230 0x7f6ab019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.411+0000 7f6ab6ca5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab010c300 con 0x7f6ab0101870 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab6ca5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab010c180 con 0x7f6ab0102230 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab6ca5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab010c480 con 0x7f6ab0100680 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 0x7f6ab0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 0x7f6ab0197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40552/0 (socket says 172.21.15.33:40552) 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 -- 172.21.15.33:0/3803289830 learned_addr learned my addr 172.21.15.33:0/3803289830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab5242640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0102230 0x7f6ab019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab4a41640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100680 0x7f6ab0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 -- 172.21.15.33:0/3803289830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100680 msgr2=0x7f6ab0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100680 0x7f6ab0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 -- 172.21.15.33:0/3803289830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0102230 msgr2=0x7f6ab019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0102230 0x7f6ab019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 -- 172.21.15.33:0/3803289830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab019f680 con 0x7f6ab0101870 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab5242640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0102230 0x7f6ab019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aaffff640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 0x7f6ab0197fa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d950 tx=0x7f6a9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c017030 con 0x7f6ab0101870 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9c004020 con 0x7f6ab0101870 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab019f970 con 0x7f6ab0101870 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0047a0 con 0x7f6ab0101870 2024-04-24T23:10:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.412+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab019fec0 con 0x7f6ab0101870 2024-04-24T23:10:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.413+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab01a0150 con 0x7f6ab0101870 2024-04-24T23:10:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.414+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a9c0041d0 con 0x7f6ab0101870 2024-04-24T23:10:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.415+0000 7f6aadffb640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6aa00a2c20 0x7f6aa00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.415+0000 7f6ab4a41640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6aa00a2c20 0x7f6aa00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.415+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6a9c0ceeb0 con 0x7f6ab0101870 2024-04-24T23:10:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.416+0000 7f6ab4a41640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6aa00a2c20 0x7f6aa00a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.420+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a9c09ba70 con 0x7f6ab0101870 2024-04-24T23:10:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.602+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ab00fdd60 con 0x7f6aa00a2c20 2024-04-24T23:10:41.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.612+0000 7f6aadffb640 1 -- 172.21.15.33:0/3803289830 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340522 (secure 0 0 0) 0x7f6ab00fdd60 con 0x7f6aa00a2c20 2024-04-24T23:10:41.617 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:41.619 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6aa00a2c20 msgr2=0x7f6aa00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6aa00a2c20 0x7f6aa00a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa4005850 comp rx=0 tx=0).stop 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 msgr2=0x7f6ab0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 0x7f6ab0197fa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d950 tx=0x7f6a9c00de10 comp rx=0 tx=0).stop 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 shutdown_connections 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6aa00a2c20 0x7f6aa00a50e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab0102230 0x7f6ab019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab0101870 0x7f6ab0197fa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 --2- 172.21.15.33:0/3803289830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100680 0x7f6ab0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.615+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 >> 172.21.15.33:0/3803289830 conn(0x7f6ab00fbbf0 msgr2=0x7f6ab00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.616+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 shutdown_connections 2024-04-24T23:10:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:41.616+0000 7f6ab6ca5640 1 -- 172.21.15.33:0/3803289830 wait complete. 2024-04-24T23:10:41.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:41 smithi033 ceph-mon[29201]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-24T23:10:41.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:41 smithi033 ceph-mon[29201]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:41.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:41 smithi033 ceph-mon[29201]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:41.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:41 smithi033 ceph-mon[29201]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:10:41.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:41 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/843992989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:41 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3624399719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.628 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-24T23:10:39.784385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595232,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":254,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356684,"kb_used_data":11644,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585140,"statfs":{"total":1151940427776,"available":1151575183360,"internally_reserved":0,"allocated":11923456,"data_stored":7549626,"data_compressed":286125,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":365,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003289"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642383+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:10:18.642383+0000","last_peered":"2024-04-24T23:10:18.642383+0000","last_clean":"2024-04-24T23:10:18.642383+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:10:18.642383+0000","last_undegraded":"2024-04-24T23:10:18.642383+0000","last_fullsized":"2024-04-24T23:10:18.642383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641005+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:10:18.641005+0000","last_peered":"2024-04-24T23:10:18.641005+0000","last_clean":"2024-04-24T23:10:18.641005+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:10:18.641005+0000","last_undegraded":"2024-04-24T23:10:18.641005+0000","last_fullsized":"2024-04-24T23:10:18.641005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641194+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:10:18.641194+0000","last_peered":"2024-04-24T23:10:18.641194+0000","last_clean":"2024-04-24T23:10:18.641194+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:10:18.641194+0000","last_undegraded":"2024-04-24T23:10:18.641194+0000","last_fullsized":"2024-04-24T23:10:18.641194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929590+0000","last_change":"2024-04-24T23:10:12.342437+0000","last_active":"2024-04-24T23:10:17.929590+0000","last_peered":"2024-04-24T23:10:17.929590+0000","last_clean":"2024-04-24T23:10:17.929590+0000","last_became_active":"2024-04-24T23:10:12.342064+0000","last_became_peered":"2024-04-24T23:10:12.342064+0000","last_unstale":"2024-04-24T23:10:17.929590+0000","last_undegraded":"2024-04-24T23:10:17.929590+0000","last_fullsized":"2024-04-24T23:10:17.929590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:54.628493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641339+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:10:18.641339+0000","last_peered":"2024-04-24T23:10:18.641339+0000","last_clean":"2024-04-24T23:10:18.641339+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:10:18.641339+0000","last_undegraded":"2024-04-24T23:10:18.641339+0000","last_fullsized":"2024-04-24T23:10:18.641339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641022+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:10:18.641022+0000","last_peered":"2024-04-24T23:10:18.641022+0000","last_clean":"2024-04-24T23:10:18.641022+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:10:18.641022+0000","last_undegraded":"2024-04-24T23:10:18.641022+0000","last_fullsized":"2024-04-24T23:10:18.641022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641926+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:10:18.641926+0000","last_peered":"2024-04-24T23:10:18.641926+0000","last_clean":"2024-04-24T23:10:18.641926+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:10:18.641926+0000","last_undegraded":"2024-04-24T23:10:18.641926+0000","last_fullsized":"2024-04-24T23:10:18.641926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641057+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:10:18.641057+0000","last_peered":"2024-04-24T23:10:18.641057+0000","last_clean":"2024-04-24T23:10:18.641057+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:10:18.641057+0000","last_undegraded":"2024-04-24T23:10:18.641057+0000","last_fullsized":"2024-04-24T23:10:18.641057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364328+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:10:17.364328+0000","last_peered":"2024-04-24T23:10:17.364328+0000","last_clean":"2024-04-24T23:10:17.364328+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:10:17.364328+0000","last_undegraded":"2024-04-24T23:10:17.364328+0000","last_fullsized":"2024-04-24T23:10:17.364328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641939+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:10:18.641939+0000","last_peered":"2024-04-24T23:10:18.641939+0000","last_clean":"2024-04-24T23:10:18.641939+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:10:18.641939+0000","last_undegraded":"2024-04-24T23:10:18.641939+0000","last_fullsized":"2024-04-24T23:10:18.641939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641814+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:10:18.641814+0000","last_peered":"2024-04-24T23:10:18.641814+0000","last_clean":"2024-04-24T23:10:18.641814+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:10:18.641814+0000","last_undegraded":"2024-04-24T23:10:18.641814+0000","last_fullsized":"2024-04-24T23:10:18.641814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929484+0000","last_change":"2024-04-24T23:10:12.340314+0000","last_active":"2024-04-24T23:10:17.929484+0000","last_peered":"2024-04-24T23:10:17.929484+0000","last_clean":"2024-04-24T23:10:17.929484+0000","last_became_active":"2024-04-24T23:10:12.339907+0000","last_became_peered":"2024-04-24T23:10:12.339907+0000","last_unstale":"2024-04-24T23:10:17.929484+0000","last_undegraded":"2024-04-24T23:10:17.929484+0000","last_fullsized":"2024-04-24T23:10:17.929484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:23.960331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364096+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:10:17.364096+0000","last_peered":"2024-04-24T23:10:17.364096+0000","last_clean":"2024-04-24T23:10:17.364096+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:10:17.364096+0000","last_undegraded":"2024-04-24T23:10:17.364096+0000","last_fullsized":"2024-04-24T23:10:17.364096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641499+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:10:18.641499+0000","last_peered":"2024-04-24T23:10:18.641499+0000","last_clean":"2024-04-24T23:10:18.641499+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:10:18.641499+0000","last_undegraded":"2024-04-24T23:10:18.641499+0000","last_fullsized":"2024-04-24T23:10:18.641499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641850+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:10:18.641850+0000","last_peered":"2024-04-24T23:10:18.641850+0000","last_clean":"2024-04-24T23:10:18.641850+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:10:18.641850+0000","last_undegraded":"2024-04-24T23:10:18.641850+0000","last_fullsized":"2024-04-24T23:10:18.641850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363631+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:10:17.363631+0000","last_peered":"2024-04-24T23:10:17.363631+0000","last_clean":"2024-04-24T23:10:17.363631+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:10:17.363631+0000","last_undegraded":"2024-04-24T23:10:17.363631+0000","last_fullsized":"2024-04-24T23:10:17.363631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642275+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:10:18.642275+0000","last_peered":"2024-04-24T23:10:18.642275+0000","last_clean":"2024-04-24T23:10:18.642275+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:10:18.642275+0000","last_undegraded":"2024-04-24T23:10:18.642275+0000","last_fullsized":"2024-04-24T23:10:18.642275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642948+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:10:18.642948+0000","last_peered":"2024-04-24T23:10:18.642948+0000","last_clean":"2024-04-24T23:10:18.642948+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:10:18.642948+0000","last_undegraded":"2024-04-24T23:10:18.642948+0000","last_fullsized":"2024-04-24T23:10:18.642948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642640+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:10:18.642640+0000","last_peered":"2024-04-24T23:10:18.642640+0000","last_clean":"2024-04-24T23:10:18.642640+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:10:18.642640+0000","last_undegraded":"2024-04-24T23:10:18.642640+0000","last_fullsized":"2024-04-24T23:10:18.642640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642317+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:10:18.642317+0000","last_peered":"2024-04-24T23:10:18.642317+0000","last_clean":"2024-04-24T23:10:18.642317+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:10:18.642317+0000","last_undegraded":"2024-04-24T23:10:18.642317+0000","last_fullsized":"2024-04-24T23:10:18.642317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.643019+0000","last_change":"2024-04-24T23:10:11.321831+0000","last_active":"2024-04-24T23:10:18.643019+0000","last_peered":"2024-04-24T23:10:18.643019+0000","last_clean":"2024-04-24T23:10:18.643019+0000","last_became_active":"2024-04-24T23:10:11.321307+0000","last_became_peered":"2024-04-24T23:10:11.321307+0000","last_unstale":"2024-04-24T23:10:18.643019+0000","last_undegraded":"2024-04-24T23:10:18.643019+0000","last_fullsized":"2024-04-24T23:10:18.643019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:56.345320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641403+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:10:18.641403+0000","last_peered":"2024-04-24T23:10:18.641403+0000","last_clean":"2024-04-24T23:10:18.641403+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:10:18.641403+0000","last_undegraded":"2024-04-24T23:10:18.641403+0000","last_fullsized":"2024-04-24T23:10:18.641403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929202+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:10:17.929202+0000","last_peered":"2024-04-24T23:10:17.929202+0000","last_clean":"2024-04-24T23:10:17.929202+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:10:17.929202+0000","last_undegraded":"2024-04-24T23:10:17.929202+0000","last_fullsized":"2024-04-24T23:10:17.929202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641730+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:10:18.641730+0000","last_peered":"2024-04-24T23:10:18.641730+0000","last_clean":"2024-04-24T23:10:18.641730+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:10:18.641730+0000","last_undegraded":"2024-04-24T23:10:18.641730+0000","last_fullsized":"2024-04-24T23:10:18.641730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641013+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:10:18.641013+0000","last_peered":"2024-04-24T23:10:18.641013+0000","last_clean":"2024-04-24T23:10:18.641013+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:10:18.641013+0000","last_undegraded":"2024-04-24T23:10:18.641013+0000","last_fullsized":"2024-04-24T23:10:18.641013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929221+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:10:17.929221+0000","last_peered":"2024-04-24T23:10:17.929221+0000","last_clean":"2024-04-24T23:10:17.929221+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:10:17.929221+0000","last_undegraded":"2024-04-24T23:10:17.929221+0000","last_fullsized":"2024-04-24T23:10:17.929221+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.460162+0000","last_change":"2024-04-24T23:10:11.327335+0000","last_active":"2024-04-24T23:10:37.460162+0000","last_peered":"2024-04-24T23:10:37.460162+0000","last_clean":"2024-04-24T23:10:37.460162+0000","last_became_active":"2024-04-24T23:10:11.326835+0000","last_became_peered":"2024-04-24T23:10:11.326835+0000","last_unstale":"2024-04-24T23:10:37.460162+0000","last_undegraded":"2024-04-24T23:10:37.460162+0000","last_fullsized":"2024-04-24T23:10:37.460162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:21.267805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363824+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:10:17.363824+0000","last_peered":"2024-04-24T23:10:17.363824+0000","last_clean":"2024-04-24T23:10:17.363824+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:10:17.363824+0000","last_undegraded":"2024-04-24T23:10:17.363824+0000","last_fullsized":"2024-04-24T23:10:17.363824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929177+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:10:17.929177+0000","last_peered":"2024-04-24T23:10:17.929177+0000","last_clean":"2024-04-24T23:10:17.929177+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:10:17.929177+0000","last_undegraded":"2024-04-24T23:10:17.929177+0000","last_fullsized":"2024-04-24T23:10:17.929177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.643022+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:10:18.643022+0000","last_peered":"2024-04-24T23:10:18.643022+0000","last_clean":"2024-04-24T23:10:18.643022+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:10:18.643022+0000","last_undegraded":"2024-04-24T23:10:18.643022+0000","last_fullsized":"2024-04-24T23:10:18.643022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366372+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:10:17.366372+0000","last_peered":"2024-04-24T23:10:17.366372+0000","last_clean":"2024-04-24T23:10:17.366372+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:10:17.366372+0000","last_undegraded":"2024-04-24T23:10:17.366372+0000","last_fullsized":"2024-04-24T23:10:17.366372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642066+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:10:18.642066+0000","last_peered":"2024-04-24T23:10:18.642066+0000","last_clean":"2024-04-24T23:10:18.642066+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:10:18.642066+0000","last_undegraded":"2024-04-24T23:10:18.642066+0000","last_fullsized":"2024-04-24T23:10:18.642066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365926+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:10:17.365926+0000","last_peered":"2024-04-24T23:10:17.365926+0000","last_clean":"2024-04-24T23:10:17.365926+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:10:17.365926+0000","last_undegraded":"2024-04-24T23:10:17.365926+0000","last_fullsized":"2024-04-24T23:10:17.365926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642322+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:10:18.642322+0000","last_peered":"2024-04-24T23:10:18.642322+0000","last_clean":"2024-04-24T23:10:18.642322+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:10:18.642322+0000","last_undegraded":"2024-04-24T23:10:18.642322+0000","last_fullsized":"2024-04-24T23:10:18.642322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642372+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:10:18.642372+0000","last_peered":"2024-04-24T23:10:18.642372+0000","last_clean":"2024-04-24T23:10:18.642372+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:10:18.642372+0000","last_undegraded":"2024-04-24T23:10:18.642372+0000","last_fullsized":"2024-04-24T23:10:18.642372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640996+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:10:18.640996+0000","last_peered":"2024-04-24T23:10:18.640996+0000","last_clean":"2024-04-24T23:10:18.640996+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:10:18.640996+0000","last_undegraded":"2024-04-24T23:10:18.640996+0000","last_fullsized":"2024-04-24T23:10:18.640996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366102+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:10:17.366102+0000","last_peered":"2024-04-24T23:10:17.366102+0000","last_clean":"2024-04-24T23:10:17.366102+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:10:17.366102+0000","last_undegraded":"2024-04-24T23:10:17.366102+0000","last_fullsized":"2024-04-24T23:10:17.366102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642247+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:10:18.642247+0000","last_peered":"2024-04-24T23:10:18.642247+0000","last_clean":"2024-04-24T23:10:18.642247+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:10:18.642247+0000","last_undegraded":"2024-04-24T23:10:18.642247+0000","last_fullsized":"2024-04-24T23:10:18.642247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363770+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:10:17.363770+0000","last_peered":"2024-04-24T23:10:17.363770+0000","last_clean":"2024-04-24T23:10:17.363770+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:10:17.363770+0000","last_undegraded":"2024-04-24T23:10:17.363770+0000","last_fullsized":"2024-04-24T23:10:17.363770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640964+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:10:18.640964+0000","last_peered":"2024-04-24T23:10:18.640964+0000","last_clean":"2024-04-24T23:10:18.640964+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:10:18.640964+0000","last_undegraded":"2024-04-24T23:10:18.640964+0000","last_fullsized":"2024-04-24T23:10:18.640964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640891+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:10:18.640891+0000","last_peered":"2024-04-24T23:10:18.640891+0000","last_clean":"2024-04-24T23:10:18.640891+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:10:18.640891+0000","last_undegraded":"2024-04-24T23:10:18.640891+0000","last_fullsized":"2024-04-24T23:10:18.640891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365713+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:10:17.365713+0000","last_peered":"2024-04-24T23:10:17.365713+0000","last_clean":"2024-04-24T23:10:17.365713+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:10:17.365713+0000","last_undegraded":"2024-04-24T23:10:17.365713+0000","last_fullsized":"2024-04-24T23:10:17.365713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642464+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:10:18.642464+0000","last_peered":"2024-04-24T23:10:18.642464+0000","last_clean":"2024-04-24T23:10:18.642464+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:10:18.642464+0000","last_undegraded":"2024-04-24T23:10:18.642464+0000","last_fullsized":"2024-04-24T23:10:18.642464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640827+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:10:18.640827+0000","last_peered":"2024-04-24T23:10:18.640827+0000","last_clean":"2024-04-24T23:10:18.640827+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:10:18.640827+0000","last_undegraded":"2024-04-24T23:10:18.640827+0000","last_fullsized":"2024-04-24T23:10:18.640827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.449443+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:10:36.449443+0000","last_peered":"2024-04-24T23:10:36.449443+0000","last_clean":"2024-04-24T23:10:36.449443+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:10:36.449443+0000","last_undegraded":"2024-04-24T23:10:36.449443+0000","last_fullsized":"2024-04-24T23:10:36.449443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364210+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:10:17.364210+0000","last_peered":"2024-04-24T23:10:17.364210+0000","last_clean":"2024-04-24T23:10:17.364210+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:10:17.364210+0000","last_undegraded":"2024-04-24T23:10:17.364210+0000","last_fullsized":"2024-04-24T23:10:17.364210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642480+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:10:18.642480+0000","last_peered":"2024-04-24T23:10:18.642480+0000","last_clean":"2024-04-24T23:10:18.642480+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:10:18.642480+0000","last_undegraded":"2024-04-24T23:10:18.642480+0000","last_fullsized":"2024-04-24T23:10:18.642480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641473+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:10:18.641473+0000","last_peered":"2024-04-24T23:10:18.641473+0000","last_clean":"2024-04-24T23:10:18.641473+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:10:18.641473+0000","last_undegraded":"2024-04-24T23:10:18.641473+0000","last_fullsized":"2024-04-24T23:10:18.641473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641120+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:10:18.641120+0000","last_peered":"2024-04-24T23:10:18.641120+0000","last_clean":"2024-04-24T23:10:18.641120+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:10:18.641120+0000","last_undegraded":"2024-04-24T23:10:18.641120+0000","last_fullsized":"2024-04-24T23:10:18.641120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929399+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:10:17.929399+0000","last_peered":"2024-04-24T23:10:17.929399+0000","last_clean":"2024-04-24T23:10:17.929399+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:10:17.929399+0000","last_undegraded":"2024-04-24T23:10:17.929399+0000","last_fullsized":"2024-04-24T23:10:17.929399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366115+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:10:17.366115+0000","last_peered":"2024-04-24T23:10:17.366115+0000","last_clean":"2024-04-24T23:10:17.366115+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:10:17.366115+0000","last_undegraded":"2024-04-24T23:10:17.366115+0000","last_fullsized":"2024-04-24T23:10:17.366115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364112+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:10:17.364112+0000","last_peered":"2024-04-24T23:10:17.364112+0000","last_clean":"2024-04-24T23:10:17.364112+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:10:17.364112+0000","last_undegraded":"2024-04-24T23:10:17.364112+0000","last_fullsized":"2024-04-24T23:10:17.364112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642797+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:10:18.642797+0000","last_peered":"2024-04-24T23:10:18.642797+0000","last_clean":"2024-04-24T23:10:18.642797+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:10:18.642797+0000","last_undegraded":"2024-04-24T23:10:18.642797+0000","last_fullsized":"2024-04-24T23:10:18.642797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.449957+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:10:36.449957+0000","last_peered":"2024-04-24T23:10:36.449957+0000","last_clean":"2024-04-24T23:10:36.449957+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:10:36.449957+0000","last_undegraded":"2024-04-24T23:10:36.449957+0000","last_fullsized":"2024-04-24T23:10:36.449957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642749+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:10:18.642749+0000","last_peered":"2024-04-24T23:10:18.642749+0000","last_clean":"2024-04-24T23:10:18.642749+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:10:18.642749+0000","last_undegraded":"2024-04-24T23:10:18.642749+0000","last_fullsized":"2024-04-24T23:10:18.642749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366693+0000","last_change":"2024-04-24T23:10:16.378770+0000","last_active":"2024-04-24T23:10:17.366693+0000","last_peered":"2024-04-24T23:10:17.366693+0000","last_clean":"2024-04-24T23:10:17.366693+0000","last_became_active":"2024-04-24T23:10:16.377971+0000","last_became_peered":"2024-04-24T23:10:16.377971+0000","last_unstale":"2024-04-24T23:10:17.366693+0000","last_undegraded":"2024-04-24T23:10:17.366693+0000","last_fullsized":"2024-04-24T23:10:17.366693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:18.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642752+0000","last_change":"2024-04-24T23:10:11.320845+0000","last_active":"2024-04-24T23:10:18.642752+0000","last_peered":"2024-04-24T23:10:18.642752+0000","last_clean":"2024-04-24T23:10:18.642752+0000","last_became_active":"2024-04-24T23:10:11.320527+0000","last_became_peered":"2024-04-24T23:10:11.320527+0000","last_unstale":"2024-04-24T23:10:18.642752+0000","last_undegraded":"2024-04-24T23:10:18.642752+0000","last_fullsized":"2024-04-24T23:10:18.642752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:40.423345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364390+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:10:17.364390+0000","last_peered":"2024-04-24T23:10:17.364390+0000","last_clean":"2024-04-24T23:10:17.364390+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:10:17.364390+0000","last_undegraded":"2024-04-24T23:10:17.364390+0000","last_fullsized":"2024-04-24T23:10:17.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642065+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:10:18.642065+0000","last_peered":"2024-04-24T23:10:18.642065+0000","last_clean":"2024-04-24T23:10:18.642065+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:10:18.642065+0000","last_undegraded":"2024-04-24T23:10:18.642065+0000","last_fullsized":"2024-04-24T23:10:18.642065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366058+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:10:17.366058+0000","last_peered":"2024-04-24T23:10:17.366058+0000","last_clean":"2024-04-24T23:10:17.366058+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:10:17.366058+0000","last_undegraded":"2024-04-24T23:10:17.366058+0000","last_fullsized":"2024-04-24T23:10:17.366058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641572+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:10:18.641572+0000","last_peered":"2024-04-24T23:10:18.641572+0000","last_clean":"2024-04-24T23:10:18.641572+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:10:18.641572+0000","last_undegraded":"2024-04-24T23:10:18.641572+0000","last_fullsized":"2024-04-24T23:10:18.641572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642963+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:10:18.642963+0000","last_peered":"2024-04-24T23:10:18.642963+0000","last_clean":"2024-04-24T23:10:18.642963+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:10:18.642963+0000","last_undegraded":"2024-04-24T23:10:18.642963+0000","last_fullsized":"2024-04-24T23:10:18.642963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929212+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:10:17.929212+0000","last_peered":"2024-04-24T23:10:17.929212+0000","last_clean":"2024-04-24T23:10:17.929212+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:10:17.929212+0000","last_undegraded":"2024-04-24T23:10:17.929212+0000","last_fullsized":"2024-04-24T23:10:17.929212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641147+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:10:18.641147+0000","last_peered":"2024-04-24T23:10:18.641147+0000","last_clean":"2024-04-24T23:10:18.641147+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:10:18.641147+0000","last_undegraded":"2024-04-24T23:10:18.641147+0000","last_fullsized":"2024-04-24T23:10:18.641147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364453+0000","last_change":"2024-04-24T23:10:16.384232+0000","last_active":"2024-04-24T23:10:17.364453+0000","last_peered":"2024-04-24T23:10:17.364453+0000","last_clean":"2024-04-24T23:10:17.364453+0000","last_became_active":"2024-04-24T23:10:16.383807+0000","last_became_peered":"2024-04-24T23:10:16.383807+0000","last_unstale":"2024-04-24T23:10:17.364453+0000","last_undegraded":"2024-04-24T23:10:17.364453+0000","last_fullsized":"2024-04-24T23:10:17.364453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:45.366140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.460839+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:10:37.460839+0000","last_peered":"2024-04-24T23:10:37.460839+0000","last_clean":"2024-04-24T23:10:37.460839+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:10:37.460839+0000","last_undegraded":"2024-04-24T23:10:37.460839+0000","last_fullsized":"2024-04-24T23:10:37.460839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365819+0000","last_change":"2024-04-24T23:10:12.343904+0000","last_active":"2024-04-24T23:10:17.365819+0000","last_peered":"2024-04-24T23:10:17.365819+0000","last_clean":"2024-04-24T23:10:17.365819+0000","last_became_active":"2024-04-24T23:10:12.343388+0000","last_became_peered":"2024-04-24T23:10:12.343388+0000","last_unstale":"2024-04-24T23:10:17.365819+0000","last_undegraded":"2024-04-24T23:10:17.365819+0000","last_fullsized":"2024-04-24T23:10:17.365819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:39.150159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366371+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:10:17.366371+0000","last_peered":"2024-04-24T23:10:17.366371+0000","last_clean":"2024-04-24T23:10:17.366371+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:10:17.366371+0000","last_undegraded":"2024-04-24T23:10:17.366371+0000","last_fullsized":"2024-04-24T23:10:17.366371+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641828+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:10:18.641828+0000","last_peered":"2024-04-24T23:10:18.641828+0000","last_clean":"2024-04-24T23:10:18.641828+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:10:18.641828+0000","last_undegraded":"2024-04-24T23:10:18.641828+0000","last_fullsized":"2024-04-24T23:10:18.641828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641616+0000","last_change":"2024-04-24T23:10:12.340136+0000","last_active":"2024-04-24T23:10:18.641616+0000","last_peered":"2024-04-24T23:10:18.641616+0000","last_clean":"2024-04-24T23:10:18.641616+0000","last_became_active":"2024-04-24T23:10:12.339785+0000","last_became_peered":"2024-04-24T23:10:12.339785+0000","last_unstale":"2024-04-24T23:10:18.641616+0000","last_undegraded":"2024-04-24T23:10:18.641616+0000","last_fullsized":"2024-04-24T23:10:18.641616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:28.124443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642186+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:10:18.642186+0000","last_peered":"2024-04-24T23:10:18.642186+0000","last_clean":"2024-04-24T23:10:18.642186+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:10:18.642186+0000","last_undegraded":"2024-04-24T23:10:18.642186+0000","last_fullsized":"2024-04-24T23:10:18.642186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642756+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:10:18.642756+0000","last_peered":"2024-04-24T23:10:18.642756+0000","last_clean":"2024-04-24T23:10:18.642756+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:10:18.642756+0000","last_undegraded":"2024-04-24T23:10:18.642756+0000","last_fullsized":"2024-04-24T23:10:18.642756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641559+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:10:18.641559+0000","last_peered":"2024-04-24T23:10:18.641559+0000","last_clean":"2024-04-24T23:10:18.641559+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:10:18.641559+0000","last_undegraded":"2024-04-24T23:10:18.641559+0000","last_fullsized":"2024-04-24T23:10:18.641559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363941+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:10:17.363941+0000","last_peered":"2024-04-24T23:10:17.363941+0000","last_clean":"2024-04-24T23:10:17.363941+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:10:17.363941+0000","last_undegraded":"2024-04-24T23:10:17.363941+0000","last_fullsized":"2024-04-24T23:10:17.363941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641613+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:10:18.641613+0000","last_peered":"2024-04-24T23:10:18.641613+0000","last_clean":"2024-04-24T23:10:18.641613+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:10:18.641613+0000","last_undegraded":"2024-04-24T23:10:18.641613+0000","last_fullsized":"2024-04-24T23:10:18.641613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641230+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:10:18.641230+0000","last_peered":"2024-04-24T23:10:18.641230+0000","last_clean":"2024-04-24T23:10:18.641230+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:10:18.641230+0000","last_undegraded":"2024-04-24T23:10:18.641230+0000","last_fullsized":"2024-04-24T23:10:18.641230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364244+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:10:17.364244+0000","last_peered":"2024-04-24T23:10:17.364244+0000","last_clean":"2024-04-24T23:10:17.364244+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:10:17.364244+0000","last_undegraded":"2024-04-24T23:10:17.364244+0000","last_fullsized":"2024-04-24T23:10:17.364244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366250+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:10:17.366250+0000","last_peered":"2024-04-24T23:10:17.366250+0000","last_clean":"2024-04-24T23:10:17.366250+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:10:17.366250+0000","last_undegraded":"2024-04-24T23:10:17.366250+0000","last_fullsized":"2024-04-24T23:10:17.366250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363581+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:10:17.363581+0000","last_peered":"2024-04-24T23:10:17.363581+0000","last_clean":"2024-04-24T23:10:17.363581+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:10:17.363581+0000","last_undegraded":"2024-04-24T23:10:17.363581+0000","last_fullsized":"2024-04-24T23:10:17.363581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641268+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:10:18.641268+0000","last_peered":"2024-04-24T23:10:18.641268+0000","last_clean":"2024-04-24T23:10:18.641268+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:10:18.641268+0000","last_undegraded":"2024-04-24T23:10:18.641268+0000","last_fullsized":"2024-04-24T23:10:18.641268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640582+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:10:18.640582+0000","last_peered":"2024-04-24T23:10:18.640582+0000","last_clean":"2024-04-24T23:10:18.640582+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:10:18.640582+0000","last_undegraded":"2024-04-24T23:10:18.640582+0000","last_fullsized":"2024-04-24T23:10:18.640582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929239+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:10:17.929239+0000","last_peered":"2024-04-24T23:10:17.929239+0000","last_clean":"2024-04-24T23:10:17.929239+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:10:17.929239+0000","last_undegraded":"2024-04-24T23:10:17.929239+0000","last_fullsized":"2024-04-24T23:10:17.929239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640898+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:10:18.640898+0000","last_peered":"2024-04-24T23:10:18.640898+0000","last_clean":"2024-04-24T23:10:18.640898+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:10:18.640898+0000","last_undegraded":"2024-04-24T23:10:18.640898+0000","last_fullsized":"2024-04-24T23:10:18.640898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366239+0000","last_change":"2024-04-24T23:10:16.447738+0000","last_active":"2024-04-24T23:10:17.366239+0000","last_peered":"2024-04-24T23:10:17.366239+0000","last_clean":"2024-04-24T23:10:17.366239+0000","last_became_active":"2024-04-24T23:10:16.447345+0000","last_became_peered":"2024-04-24T23:10:16.447345+0000","last_unstale":"2024-04-24T23:10:17.366239+0000","last_undegraded":"2024-04-24T23:10:17.366239+0000","last_fullsized":"2024-04-24T23:10:17.366239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:07.830860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363580+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:10:17.363580+0000","last_peered":"2024-04-24T23:10:17.363580+0000","last_clean":"2024-04-24T23:10:17.363580+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:10:17.363580+0000","last_undegraded":"2024-04-24T23:10:17.363580+0000","last_fullsized":"2024-04-24T23:10:17.363580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641076+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:10:18.641076+0000","last_peered":"2024-04-24T23:10:18.641076+0000","last_clean":"2024-04-24T23:10:18.641076+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:10:18.641076+0000","last_undegraded":"2024-04-24T23:10:18.641076+0000","last_fullsized":"2024-04-24T23:10:18.641076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641298+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:10:18.641298+0000","last_peered":"2024-04-24T23:10:18.641298+0000","last_clean":"2024-04-24T23:10:18.641298+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:10:18.641298+0000","last_undegraded":"2024-04-24T23:10:18.641298+0000","last_fullsized":"2024-04-24T23:10:18.641298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642838+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:10:18.642838+0000","last_peered":"2024-04-24T23:10:18.642838+0000","last_clean":"2024-04-24T23:10:18.642838+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:10:18.642838+0000","last_undegraded":"2024-04-24T23:10:18.642838+0000","last_fullsized":"2024-04-24T23:10:18.642838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364149+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:10:17.364149+0000","last_peered":"2024-04-24T23:10:17.364149+0000","last_clean":"2024-04-24T23:10:17.364149+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:10:17.364149+0000","last_undegraded":"2024-04-24T23:10:17.364149+0000","last_fullsized":"2024-04-24T23:10:17.364149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640640+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:10:18.640640+0000","last_peered":"2024-04-24T23:10:18.640640+0000","last_clean":"2024-04-24T23:10:18.640640+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:10:18.640640+0000","last_undegraded":"2024-04-24T23:10:18.640640+0000","last_fullsized":"2024-04-24T23:10:18.640640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642923+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:10:18.642923+0000","last_peered":"2024-04-24T23:10:18.642923+0000","last_clean":"2024-04-24T23:10:18.642923+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:10:18.642923+0000","last_undegraded":"2024-04-24T23:10:18.642923+0000","last_fullsized":"2024-04-24T23:10:18.642923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641697+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:10:18.641697+0000","last_peered":"2024-04-24T23:10:18.641697+0000","last_clean":"2024-04-24T23:10:18.641697+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:10:18.641697+0000","last_undegraded":"2024-04-24T23:10:18.641697+0000","last_fullsized":"2024-04-24T23:10:18.641697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642102+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:10:18.642102+0000","last_peered":"2024-04-24T23:10:18.642102+0000","last_clean":"2024-04-24T23:10:18.642102+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:10:18.642102+0000","last_undegraded":"2024-04-24T23:10:18.642102+0000","last_fullsized":"2024-04-24T23:10:18.642102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929639+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:10:17.929639+0000","last_peered":"2024-04-24T23:10:17.929639+0000","last_clean":"2024-04-24T23:10:17.929639+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:10:17.929639+0000","last_undegraded":"2024-04-24T23:10:17.929639+0000","last_fullsized":"2024-04-24T23:10:17.929639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641106+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:10:18.641106+0000","last_peered":"2024-04-24T23:10:18.641106+0000","last_clean":"2024-04-24T23:10:18.641106+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:10:18.641106+0000","last_undegraded":"2024-04-24T23:10:18.641106+0000","last_fullsized":"2024-04-24T23:10:18.641106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929792+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:10:17.929792+0000","last_peered":"2024-04-24T23:10:17.929792+0000","last_clean":"2024-04-24T23:10:17.929792+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:10:17.929792+0000","last_undegraded":"2024-04-24T23:10:17.929792+0000","last_fullsized":"2024-04-24T23:10:17.929792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.450302+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:10:36.450302+0000","last_peered":"2024-04-24T23:10:36.450302+0000","last_clean":"2024-04-24T23:10:36.450302+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:10:36.450302+0000","last_undegraded":"2024-04-24T23:10:36.450302+0000","last_fullsized":"2024-04-24T23:10:36.450302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641216+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:10:18.641216+0000","last_peered":"2024-04-24T23:10:18.641216+0000","last_clean":"2024-04-24T23:10:18.641216+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:10:18.641216+0000","last_undegraded":"2024-04-24T23:10:18.641216+0000","last_fullsized":"2024-04-24T23:10:18.641216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.643080+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:10:18.643080+0000","last_peered":"2024-04-24T23:10:18.643080+0000","last_clean":"2024-04-24T23:10:18.643080+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:10:18.643080+0000","last_undegraded":"2024-04-24T23:10:18.643080+0000","last_fullsized":"2024-04-24T23:10:18.643080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640865+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:10:18.640865+0000","last_peered":"2024-04-24T23:10:18.640865+0000","last_clean":"2024-04-24T23:10:18.640865+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:10:18.640865+0000","last_undegraded":"2024-04-24T23:10:18.640865+0000","last_fullsized":"2024-04-24T23:10:18.640865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641432+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:10:18.641432+0000","last_peered":"2024-04-24T23:10:18.641432+0000","last_clean":"2024-04-24T23:10:18.641432+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:10:18.641432+0000","last_undegraded":"2024-04-24T23:10:18.641432+0000","last_fullsized":"2024-04-24T23:10:18.641432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929471+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:10:17.929471+0000","last_peered":"2024-04-24T23:10:17.929471+0000","last_clean":"2024-04-24T23:10:17.929471+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:10:17.929471+0000","last_undegraded":"2024-04-24T23:10:17.929471+0000","last_fullsized":"2024-04-24T23:10:17.929471+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642884+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:10:18.642884+0000","last_peered":"2024-04-24T23:10:18.642884+0000","last_clean":"2024-04-24T23:10:18.642884+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:10:18.642884+0000","last_undegraded":"2024-04-24T23:10:18.642884+0000","last_fullsized":"2024-04-24T23:10:18.642884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641720+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:10:18.641720+0000","last_peered":"2024-04-24T23:10:18.641720+0000","last_clean":"2024-04-24T23:10:18.641720+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:10:18.641720+0000","last_undegraded":"2024-04-24T23:10:18.641720+0000","last_fullsized":"2024-04-24T23:10:18.641720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.929724+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:10:17.929724+0000","last_peered":"2024-04-24T23:10:17.929724+0000","last_clean":"2024-04-24T23:10:17.929724+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:10:17.929724+0000","last_undegraded":"2024-04-24T23:10:17.929724+0000","last_fullsized":"2024-04-24T23:10:17.929724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.449598+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:10:36.449598+0000","last_peered":"2024-04-24T23:10:36.449598+0000","last_clean":"2024-04-24T23:10:36.449598+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:10:36.449598+0000","last_undegraded":"2024-04-24T23:10:36.449598+0000","last_fullsized":"2024-04-24T23:10:36.449598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641810+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:10:18.641810+0000","last_peered":"2024-04-24T23:10:18.641810+0000","last_clean":"2024-04-24T23:10:18.641810+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:10:18.641810+0000","last_undegraded":"2024-04-24T23:10:18.641810+0000","last_fullsized":"2024-04-24T23:10:18.641810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363890+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:10:17.363890+0000","last_peered":"2024-04-24T23:10:17.363890+0000","last_clean":"2024-04-24T23:10:17.363890+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:10:17.363890+0000","last_undegraded":"2024-04-24T23:10:17.363890+0000","last_fullsized":"2024-04-24T23:10:17.363890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642519+0000","last_change":"2024-04-24T23:10:11.326142+0000","last_active":"2024-04-24T23:10:18.642519+0000","last_peered":"2024-04-24T23:10:18.642519+0000","last_clean":"2024-04-24T23:10:18.642519+0000","last_became_active":"2024-04-24T23:10:11.325728+0000","last_became_peered":"2024-04-24T23:10:11.325728+0000","last_unstale":"2024-04-24T23:10:18.642519+0000","last_undegraded":"2024-04-24T23:10:18.642519+0000","last_fullsized":"2024-04-24T23:10:18.642519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:42.647315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641820+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:10:18.641820+0000","last_peered":"2024-04-24T23:10:18.641820+0000","last_clean":"2024-04-24T23:10:18.641820+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:10:18.641820+0000","last_undegraded":"2024-04-24T23:10:18.641820+0000","last_fullsized":"2024-04-24T23:10:18.641820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641405+0000","last_change":"2024-04-24T23:10:16.387682+0000","last_active":"2024-04-24T23:10:18.641405+0000","last_peered":"2024-04-24T23:10:18.641405+0000","last_clean":"2024-04-24T23:10:18.641405+0000","last_became_active":"2024-04-24T23:10:16.387248+0000","last_became_peered":"2024-04-24T23:10:16.387248+0000","last_unstale":"2024-04-24T23:10:18.641405+0000","last_undegraded":"2024-04-24T23:10:18.641405+0000","last_fullsized":"2024-04-24T23:10:18.641405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:54.837632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366513+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:10:17.366513+0000","last_peered":"2024-04-24T23:10:17.366513+0000","last_clean":"2024-04-24T23:10:17.366513+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:10:17.366513+0000","last_undegraded":"2024-04-24T23:10:17.366513+0000","last_fullsized":"2024-04-24T23:10:17.366513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641267+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:10:18.641267+0000","last_peered":"2024-04-24T23:10:18.641267+0000","last_clean":"2024-04-24T23:10:18.641267+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:10:18.641267+0000","last_undegraded":"2024-04-24T23:10:18.641267+0000","last_fullsized":"2024-04-24T23:10:18.641267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.366201+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:10:17.366201+0000","last_peered":"2024-04-24T23:10:17.366201+0000","last_clean":"2024-04-24T23:10:17.366201+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:10:17.366201+0000","last_undegraded":"2024-04-24T23:10:17.366201+0000","last_fullsized":"2024-04-24T23:10:17.366201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364058+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:10:17.364058+0000","last_peered":"2024-04-24T23:10:17.364058+0000","last_clean":"2024-04-24T23:10:17.364058+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:10:17.364058+0000","last_undegraded":"2024-04-24T23:10:17.364058+0000","last_fullsized":"2024-04-24T23:10:17.364058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641734+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:10:18.641734+0000","last_peered":"2024-04-24T23:10:18.641734+0000","last_clean":"2024-04-24T23:10:18.641734+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:10:18.641734+0000","last_undegraded":"2024-04-24T23:10:18.641734+0000","last_fullsized":"2024-04-24T23:10:18.641734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641560+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:10:18.641560+0000","last_peered":"2024-04-24T23:10:18.641560+0000","last_clean":"2024-04-24T23:10:18.641560+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:10:18.641560+0000","last_undegraded":"2024-04-24T23:10:18.641560+0000","last_fullsized":"2024-04-24T23:10:18.641560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642431+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:10:18.642431+0000","last_peered":"2024-04-24T23:10:18.642431+0000","last_clean":"2024-04-24T23:10:18.642431+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:10:18.642431+0000","last_undegraded":"2024-04-24T23:10:18.642431+0000","last_fullsized":"2024-04-24T23:10:18.642431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642824+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:10:18.642824+0000","last_peered":"2024-04-24T23:10:18.642824+0000","last_clean":"2024-04-24T23:10:18.642824+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:10:18.642824+0000","last_undegraded":"2024-04-24T23:10:18.642824+0000","last_fullsized":"2024-04-24T23:10:18.642824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641637+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:10:18.641637+0000","last_peered":"2024-04-24T23:10:18.641637+0000","last_clean":"2024-04-24T23:10:18.641637+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:10:18.641637+0000","last_undegraded":"2024-04-24T23:10:18.641637+0000","last_fullsized":"2024-04-24T23:10:18.641637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.364390+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:10:17.364390+0000","last_peered":"2024-04-24T23:10:17.364390+0000","last_clean":"2024-04-24T23:10:17.364390+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:10:17.364390+0000","last_undegraded":"2024-04-24T23:10:17.364390+0000","last_fullsized":"2024-04-24T23:10:17.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641637+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:10:18.641637+0000","last_peered":"2024-04-24T23:10:18.641637+0000","last_clean":"2024-04-24T23:10:18.641637+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:10:18.641637+0000","last_undegraded":"2024-04-24T23:10:18.641637+0000","last_fullsized":"2024-04-24T23:10:18.641637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642012+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:10:18.642012+0000","last_peered":"2024-04-24T23:10:18.642012+0000","last_clean":"2024-04-24T23:10:18.642012+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:10:18.642012+0000","last_undegraded":"2024-04-24T23:10:18.642012+0000","last_fullsized":"2024-04-24T23:10:18.642012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.641808+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:10:18.641808+0000","last_peered":"2024-04-24T23:10:18.641808+0000","last_clean":"2024-04-24T23:10:18.641808+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:10:18.641808+0000","last_undegraded":"2024-04-24T23:10:18.641808+0000","last_fullsized":"2024-04-24T23:10:18.641808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.365821+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:10:17.365821+0000","last_peered":"2024-04-24T23:10:17.365821+0000","last_clean":"2024-04-24T23:10:17.365821+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:10:17.365821+0000","last_undegraded":"2024-04-24T23:10:17.365821+0000","last_fullsized":"2024-04-24T23:10:17.365821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.640927+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:10:18.640927+0000","last_peered":"2024-04-24T23:10:18.640927+0000","last_clean":"2024-04-24T23:10:18.640927+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:10:18.640927+0000","last_undegraded":"2024-04-24T23:10:18.640927+0000","last_fullsized":"2024-04-24T23:10:18.640927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363909+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:10:17.363909+0000","last_peered":"2024-04-24T23:10:17.363909+0000","last_clean":"2024-04-24T23:10:17.363909+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:10:17.363909+0000","last_undegraded":"2024-04-24T23:10:17.363909+0000","last_fullsized":"2024-04-24T23:10:17.363909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:17.363989+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:10:17.363989+0000","last_peered":"2024-04-24T23:10:17.363989+0000","last_clean":"2024-04-24T23:10:17.363989+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:10:17.363989+0000","last_undegraded":"2024-04-24T23:10:17.363989+0000","last_fullsized":"2024-04-24T23:10:17.363989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:10:18.642193+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:10:18.642193+0000","last_peered":"2024-04-24T23:10:18.642193+0000","last_clean":"2024-04-24T23:10:18.642193+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:10:18.642193+0000","last_undegraded":"2024-04-24T23:10:18.642193+0000","last_fullsized":"2024-04-24T23:10:18.642193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4864,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":16204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1074670,"data_compressed":51245,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1071216,"data_compressed":51245,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":484142,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":480370,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14710,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1070816,"data_compressed":51245,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:10:41.633 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:10:41.634 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T23:10:41.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T23:10:41.916 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:41 smithi038 ceph-mon[32450]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-24T23:10:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:41 smithi038 ceph-mon[32450]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:41 smithi038 ceph-mon[32450]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:41 smithi038 ceph-mon[32450]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:10:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:41 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/843992989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:41 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3624399719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:41 smithi117 ceph-mon[32529]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-24T23:10:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:41 smithi117 ceph-mon[32529]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:41 smithi117 ceph-mon[32529]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:41 smithi117 ceph-mon[32529]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:10:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:41 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/843992989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:41 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3624399719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:42.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:42 smithi033 ceph-mon[29201]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:42.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:42 smithi033 ceph-mon[29201]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:42.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:42 smithi038 ceph-mon[32450]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:42.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:42 smithi038 ceph-mon[32450]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:42.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:42 smithi117 ceph-mon[32529]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:42.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:42 smithi117 ceph-mon[32529]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:42.926 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:10:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: 2024-04-24T23:10:42.631+0000 7fbfa7362700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-24T23:10:43.639 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:43 smithi033 ceph-mon[29201]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 -- 172.21.15.33:0/2372005523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160107fa0 msgr2=0x7f816010a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 --2- 172.21.15.33:0/2372005523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160107fa0 0x7f816010a410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8158009990 tx=0x7f8158009ca0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 -- 172.21.15.33:0/2372005523 shutdown_connections 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 --2- 172.21.15.33:0/2372005523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160107fa0 0x7f816010a410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 --2- 172.21.15.33:0/2372005523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160102d40 0x7f8160105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 --2- 172.21.15.33:0/2372005523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160105670 0x7f8160107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 -- 172.21.15.33:0/2372005523 >> 172.21.15.33:0/2372005523 conn(0x7f81600fc750 msgr2=0x7f81600feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 -- 172.21.15.33:0/2372005523 shutdown_connections 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.890+0000 7f8165aea700 1 -- 172.21.15.33:0/2372005523 wait complete. 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 Processor -- start 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 -- start start 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 0x7f81601026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160105670 0x7f8160100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160107fa0 0x7f8160101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f816010dee0 con 0x7f8160102d40 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f816010dd60 con 0x7f8160105670 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f8165aea700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f816010e060 con 0x7f8160107fa0 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f815f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 0x7f81601026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f815f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 0x7f81601026a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40570/0 (socket says 172.21.15.33:40570) 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f815f7fe700 1 -- 172.21.15.33:0/2026371279 learned_addr learned my addr 172.21.15.33:0/2026371279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.891+0000 7f815ffff700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160107fa0 0x7f8160101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815effd700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160105670 0x7f8160100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815ffff700 1 -- 172.21.15.33:0/2026371279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160105670 msgr2=0x7f8160100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815ffff700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160105670 0x7f8160100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815ffff700 1 -- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 msgr2=0x7f81601026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815ffff700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 0x7f81601026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815ffff700 1 -- 172.21.15.33:0/2026371279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8160101ae0 con 0x7f8160107fa0 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815effd700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160105670 0x7f8160100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815f7fe700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 0x7f81601026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815ffff700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160107fa0 0x7f8160101230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8158003890 tx=0x7f81580047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158003d30 con 0x7f8160107fa0 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81601a24b0 con 0x7f8160107fa0 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8158004830 con 0x7f8160107fa0 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580054c0 con 0x7f8160107fa0 2024-04-24T23:10:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.892+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81601a2920 con 0x7f8160107fa0 2024-04-24T23:10:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.893+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81600507a0 con 0x7f8160107fa0 2024-04-24T23:10:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.895+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8158029060 con 0x7f8160107fa0 2024-04-24T23:10:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.895+0000 7f815cff9700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f814c0a2d60 0x7f814c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.895+0000 7f815f7fe700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f814c0a2d60 0x7f814c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.895+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f81580d01b0 con 0x7f8160107fa0 2024-04-24T23:10:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.896+0000 7f815f7fe700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f814c0a2d60 0x7f814c0a5220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8150007fd0 tx=0x7f815000c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:43.899+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8158013070 con 0x7f8160107fa0 2024-04-24T23:10:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:43 smithi038 ceph-mon[32450]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-24T23:10:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:43 smithi117 ceph-mon[32529]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-24T23:10:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.191+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f816002dc00 con 0x7f8160107fa0 2024-04-24T23:10:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.197+0000 7f815cff9700 1 -- 172.21.15.33:0/2026371279 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f816002dc00 con 0x7f8160107fa0 2024-04-24T23:10:44.199 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-04-24T23:10:44.199 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQB0kSlmDwOICxAAhbKPczYAPbirGavod0xvwg== 2024-04-24T23:10:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f814c0a2d60 msgr2=0x7f814c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f814c0a2d60 0x7f814c0a5220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8150007fd0 tx=0x7f815000c040 comp rx=0 tx=0).stop 2024-04-24T23:10:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160107fa0 msgr2=0x7f8160101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160107fa0 0x7f8160101230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8158003890 tx=0x7f81580047b0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 shutdown_connections 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f814c0a2d60 0x7f814c0a5220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8160107fa0 0x7f8160101230 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8160102d40 0x7f81601026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 --2- 172.21.15.33:0/2026371279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8160105670 0x7f8160100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 >> 172.21.15.33:0/2026371279 conn(0x7f81600fc750 msgr2=0x7f8160103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 shutdown_connections 2024-04-24T23:10:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:44.199+0000 7f8165aea700 1 -- 172.21.15.33:0/2026371279 wait complete. 2024-04-24T23:10:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:44 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2026371279' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:10:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:44 smithi038 ceph-mon[32450]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:10:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:44 smithi038 ceph-mon[32450]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:10:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:44 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2026371279' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:10:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:44 smithi117 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:10:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:44 smithi117 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:10:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:44 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2026371279' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:10:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:44 smithi033 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:10:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:44 smithi033 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:10:45.001 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:10:45.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T23:10:45.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:10:45.039 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:10:45.039 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:10:45.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T23:10:45.327 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:45.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:45 smithi033 ceph-mon[29201]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:10:45.826 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:45 smithi117 ceph-mon[32529]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:10:45.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:45 smithi038 ceph-mon[32450]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:10:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 -- 172.21.15.33:0/351629275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 msgr2=0x7fc86c108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 --2- 172.21.15.33:0/351629275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c108690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009990 tx=0x7fc85c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 -- 172.21.15.33:0/351629275 shutdown_connections 2024-04-24T23:10:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 --2- 172.21.15.33:0/351629275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc86c06a8e0 0x7fc86c10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 --2- 172.21.15.33:0/351629275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c108690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 --2- 172.21.15.33:0/351629275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069f40 0x7fc86c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 -- 172.21.15.33:0/351629275 >> 172.21.15.33:0/351629275 conn(0x7fc86c0780e0 msgr2=0x7fc86c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 -- 172.21.15.33:0/351629275 shutdown_connections 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.183+0000 7fc8717a4700 1 -- 172.21.15.33:0/351629275 wait complete. 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 Processor -- start 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 -- start start 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069f40 0x7fc86c101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc86c06a8e0 0x7fc86c101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112510 con 0x7fc86c1082b0 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112390 con 0x7fc86c06a8e0 2024-04-24T23:10:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc8717a4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112690 con 0x7fc86c069f40 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51648/0 (socket says 172.21.15.33:51648) 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc86b7fe700 1 -- 172.21.15.33:0/206048942 learned_addr learned my addr 172.21.15.33:0/206048942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.184+0000 7fc86affd700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069f40 0x7fc86c101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86a7fc700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc86c06a8e0 0x7fc86c101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86b7fe700 1 -- 172.21.15.33:0/206048942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069f40 msgr2=0x7fc86c101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86b7fe700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069f40 0x7fc86c101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86b7fe700 1 -- 172.21.15.33:0/206048942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc86c06a8e0 msgr2=0x7fc86c101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86b7fe700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc86c06a8e0 0x7fc86c101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86b7fe700 1 -- 172.21.15.33:0/206048942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c1a6fb0 con 0x7fc86c1082b0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc86b7fe700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c101c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc864007ee0 tx=0x7fc86400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86400dbd0 con 0x7fc86c1082b0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c1a72a0 con 0x7fc86c1082b0 2024-04-24T23:10:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86400dd70 con 0x7fc86c1082b0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864005a60 con 0x7fc86c1082b0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.185+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a77e0 con 0x7fc86c1082b0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.186+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c068f20 con 0x7fc86c1082b0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.187+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc86401a020 con 0x7fc86c1082b0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.188+0000 7fc853fff700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8540a2cb0 0x7fc8540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.188+0000 7fc86affd700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8540a2cb0 0x7fc8540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.188+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc8640ceac0 con 0x7fc86c1082b0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.188+0000 7fc86affd700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8540a2cb0 0x7fc8540a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00b5c0 tx=0x7fc85c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.192+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc864010070 con 0x7fc86c1082b0 2024-04-24T23:10:47.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.459+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc86c02db30 con 0x7fc86c1082b0 2024-04-24T23:10:47.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.464+0000 7fc853fff700 1 -- 172.21.15.33:0/206048942 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7fc86409b680 con 0x7fc86c1082b0 2024-04-24T23:10:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8540a2cb0 msgr2=0x7fc8540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8540a2cb0 0x7fc8540a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00b5c0 tx=0x7fc85c005960 comp rx=0 tx=0).stop 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 msgr2=0x7fc86c101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c101c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc864007ee0 tx=0x7fc86400d3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 shutdown_connections 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8540a2cb0 0x7fc8540a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc86c1082b0 0x7fc86c101c80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069f40 0x7fc86c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 --2- 172.21.15.33:0/206048942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc86c06a8e0 0x7fc86c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 >> 172.21.15.33:0/206048942 conn(0x7fc86c0780e0 msgr2=0x7fc86c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 shutdown_connections 2024-04-24T23:10:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:47.467+0000 7fc8717a4700 1 -- 172.21.15.33:0/206048942 wait complete. 2024-04-24T23:10:47.791 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:47 smithi033 ceph-mon[29201]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-04-24T23:10:47.791 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:47 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/206048942' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:10:47.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:47 smithi038 ceph-mon[32450]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-04-24T23:10:47.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:47 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/206048942' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:10:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:47 smithi117 ceph-mon[32529]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-04-24T23:10:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:47 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/206048942' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:10:48.234 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3494091302},{"type":"v1","addr":"172.21.15.33:6801","nonce":3494091302}]},"active_addr":"172.21.15.33:6801/3494091302","active_change":"2024-04-24T23:03:05.692638+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14214,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":4249192916}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1685989953}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2578669762}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3132001874}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1202512859}]}]} 2024-04-24T23:10:48.239 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T23:10:48.239 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T23:10:48.239 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:10:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:49.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:49 smithi033 ceph-mon[29201]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-04-24T23:10:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:49 smithi038 ceph-mon[32450]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-04-24T23:10:49.998 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:49 smithi117 ceph-mon[32529]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-04-24T23:10:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 -- 172.21.15.33:0/3350371325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd138107570 msgr2=0x7fd13810f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 --2- 172.21.15.33:0/3350371325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd138107570 0x7fd13810f140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd13000b0a0 tx=0x7fd13000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 -- 172.21.15.33:0/3350371325 shutdown_connections 2024-04-24T23:10:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 --2- 172.21.15.33:0/3350371325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd138107570 0x7fd13810f140 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 --2- 172.21.15.33:0/3350371325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1381024e0 0x7fd1381028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 --2- 172.21.15.33:0/3350371325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 0x7fd138106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 -- 172.21.15.33:0/3350371325 >> 172.21.15.33:0/3350371325 conn(0x7fd138079390 msgr2=0x7fd1380797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 -- 172.21.15.33:0/3350371325 shutdown_connections 2024-04-24T23:10:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.468+0000 7fd13cc37700 1 -- 172.21.15.33:0/3350371325 wait complete. 2024-04-24T23:10:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 Processor -- start 2024-04-24T23:10:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 -- start start 2024-04-24T23:10:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 0x7fd13819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 0x7fd13819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd138107570 0x7fd138198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd138112550 con 0x7fd1381024e0 2024-04-24T23:10:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1381123d0 con 0x7fd138102e90 2024-04-24T23:10:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13cc37700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1381126d0 con 0x7fd138107570 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 0x7fd13819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 0x7fd13819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51660/0 (socket says 172.21.15.33:51660) 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 -- 172.21.15.33:0/355506369 learned_addr learned my addr 172.21.15.33:0/355506369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd135d9b700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 0x7fd13819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd136d9d700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd138107570 0x7fd138198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 -- 172.21.15.33:0/355506369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd138107570 msgr2=0x7fd138198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd138107570 0x7fd138198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 -- 172.21.15.33:0/355506369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 msgr2=0x7fd13819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 0x7fd13819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd13659c700 1 -- 172.21.15.33:0/355506369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd138198ff0 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.469+0000 7fd135d9b700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 0x7fd13819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.470+0000 7fd13659c700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 0x7fd13819e690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd12800d980 tx=0x7fd12800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.470+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128017070 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.470+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1381992e0 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.470+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd128005020 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.470+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128009bc0 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.470+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1381a54f0 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.471+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd138103970 con 0x7fd1381024e0 2024-04-24T23:10:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.472+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd1280051d0 con 0x7fd1381024e0 2024-04-24T23:10:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.472+0000 7fd1277fe700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1200a2cb0 0x7fd1200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.473+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd1280cf340 con 0x7fd1381024e0 2024-04-24T23:10:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.473+0000 7fd135d9b700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1200a2cb0 0x7fd1200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.473+0000 7fd135d9b700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1200a2cb0 0x7fd1200a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd138108aa0 tx=0x7fd12c005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.477+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd128020020 con 0x7fd1381024e0 2024-04-24T23:10:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.684+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd13802db30 con 0x7fd1381024e0 2024-04-24T23:10:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.685+0000 7fd1277fe700 1 -- 172.21.15.33:0/355506369 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fd12809bf00 con 0x7fd1381024e0 2024-04-24T23:10:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:50.687 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1200a2cb0 msgr2=0x7fd1200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1200a2cb0 0x7fd1200a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd138108aa0 tx=0x7fd12c005d80 comp rx=0 tx=0).stop 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 msgr2=0x7fd13819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 0x7fd13819e690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd12800d980 tx=0x7fd12800dc90 comp rx=0 tx=0).stop 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 shutdown_connections 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1200a2cb0 0x7fd1200a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd138107570 0x7fd138198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1381024e0 0x7fd13819e690 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 --2- 172.21.15.33:0/355506369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd138102e90 0x7fd13819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.687+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 >> 172.21.15.33:0/355506369 conn(0x7fd138079390 msgr2=0x7fd138100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.688+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 shutdown_connections 2024-04-24T23:10:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:50.688+0000 7fd13cc37700 1 -- 172.21.15.33:0/355506369 wait complete. 2024-04-24T23:10:51.603 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T23:10:51.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:10:51.858 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:51.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:51 smithi038 ceph-mon[32450]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-24T23:10:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:51 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/355506369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:51 smithi117 ceph-mon[32529]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-24T23:10:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/355506369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:51 smithi033 ceph-mon[29201]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-24T23:10:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:51 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/355506369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:53.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:53 smithi033 ceph-mon[29201]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-04-24T23:10:53.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 -- 172.21.15.33:0/1654954343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb781071a0 msgr2=0x7fcb78107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:53.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 --2- 172.21.15.33:0/1654954343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb781071a0 0x7fcb78107580 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009990 tx=0x7fcb60009ca0 comp rx=0 tx=0).stop 2024-04-24T23:10:53.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 -- 172.21.15.33:0/1654954343 shutdown_connections 2024-04-24T23:10:53.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 --2- 172.21.15.33:0/1654954343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb78101af0 0x7fcb7810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 --2- 172.21.15.33:0/1654954343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb781071a0 0x7fcb78107580 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 --2- 172.21.15.33:0/1654954343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb781015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 -- 172.21.15.33:0/1654954343 >> 172.21.15.33:0/1654954343 conn(0x7fcb780780e0 msgr2=0x7fcb780784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 -- 172.21.15.33:0/1654954343 shutdown_connections 2024-04-24T23:10:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.813+0000 7fcb7e637700 1 -- 172.21.15.33:0/1654954343 wait complete. 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.814+0000 7fcb7e637700 1 Processor -- start 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.814+0000 7fcb7e637700 1 -- start start 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7e637700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb78199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7e637700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 0x7fcb78199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7e637700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb781071a0 0x7fcb7819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7e637700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78111440 con 0x7fcb78101af0 2024-04-24T23:10:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7e637700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781112c0 con 0x7fcb78101130 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7e637700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781115c0 con 0x7fcb781071a0 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb77fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb78199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 0x7fcb78199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb77fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb78199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:52682/0 (socket says 172.21.15.33:52682) 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 0x7fcb78199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51684/0 (socket says 172.21.15.33:51684) 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb77fff700 1 -- 172.21.15.33:0/9730929 learned_addr learned my addr 172.21.15.33:0/9730929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb7cbd4700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb781071a0 0x7fcb7819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 -- 172.21.15.33:0/9730929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb781071a0 msgr2=0x7fcb7819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb781071a0 0x7fcb7819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 -- 172.21.15.33:0/9730929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 msgr2=0x7fcb78199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb78199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 -- 172.21.15.33:0/9730929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb781a4d30 con 0x7fcb78101af0 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb77fff700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb78199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.815+0000 7fcb777fe700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 0x7fcb78199e80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800d950 tx=0x7fcb6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.816+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68016070 con 0x7fcb78101af0 2024-04-24T23:10:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.816+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6800fd70 con 0x7fcb78101af0 2024-04-24T23:10:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.816+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb781a4fc0 con 0x7fcb78101af0 2024-04-24T23:10:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.816+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800e3b0 con 0x7fcb78101af0 2024-04-24T23:10:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.816+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb781a54d0 con 0x7fcb78101af0 2024-04-24T23:10:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.817+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb78068f20 con 0x7fcb78101af0 2024-04-24T23:10:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.818+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcb68019070 con 0x7fcb78101af0 2024-04-24T23:10:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.819+0000 7fcb757fa700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb640a2cb0 0x7fcb640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.819+0000 7fcb77fff700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb640a2cb0 0x7fcb640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.819+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcb680cedb0 con 0x7fcb78101af0 2024-04-24T23:10:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.819+0000 7fcb77fff700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb640a2cb0 0x7fcb640a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b5c0 tx=0x7fcb60005c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:53.823+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcb68010460 con 0x7fcb78101af0 2024-04-24T23:10:53.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:53 smithi038 ceph-mon[32450]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-04-24T23:10:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:53 smithi117 ceph-mon[32529]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-04-24T23:10:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.032+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb7802db30 con 0x7fcb78101af0 2024-04-24T23:10:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.033+0000 7fcb757fa700 1 -- 172.21.15.33:0/9730929 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24092 (secure 0 0 0) 0x7fcb6809b970 con 0x7fcb78101af0 2024-04-24T23:10:54.036 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:10:54.036 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:10:16.733107+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb640a2cb0 msgr2=0x7fcb640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb640a2cb0 0x7fcb640a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b5c0 tx=0x7fcb60005c00 comp rx=0 tx=0).stop 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 msgr2=0x7fcb78199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 0x7fcb78199e80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800d950 tx=0x7fcb6800dc60 comp rx=0 tx=0).stop 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 shutdown_connections 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb640a2cb0 0x7fcb640a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb781071a0 0x7fcb7819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb78101130 0x7fcb78199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 --2- 172.21.15.33:0/9730929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb78101af0 0x7fcb78199e80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 >> 172.21.15.33:0/9730929 conn(0x7fcb780780e0 msgr2=0x7fcb780ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 shutdown_connections 2024-04-24T23:10:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:10:54.036+0000 7fcb7e637700 1 -- 172.21.15.33:0/9730929 wait complete. 2024-04-24T23:10:54.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:54 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/9730929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:54.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:54 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/9730929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:54.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:54 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/9730929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:55.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T23:10:55.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T23:10:55.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T23:10:55.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T23:10:55.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T23:10:55.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T23:10:55.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T23:10:55.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T23:10:55.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T23:10:55.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T23:10:55.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T23:10:55.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T23:10:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:55 smithi038 ceph-mon[32450]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-24T23:10:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:55 smithi117 ceph-mon[32529]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-24T23:10:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:55.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:55 smithi033 ceph-mon[29201]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-24T23:10:57.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:57 smithi033 ceph-mon[29201]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-24T23:10:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:57.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:57 smithi038 ceph-mon[32450]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-24T23:10:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:57 smithi117 ceph-mon[32529]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-24T23:10:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:10:59 smithi033 ceph-mon[29201]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-24T23:10:59.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:10:59 smithi038 ceph-mon[32450]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-24T23:10:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:10:59 smithi117 ceph-mon[32529]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-24T23:11:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 -- 172.21.15.33:0/2068389591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 msgr2=0x7fc8fc10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 --2- 172.21.15.33:0/2068389591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc10dfe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00b0a0 tx=0x7fc8ec00b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 -- 172.21.15.33:0/2068389591 shutdown_connections 2024-04-24T23:11:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 --2- 172.21.15.33:0/2068389591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc10dfe0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 --2- 172.21.15.33:0/2068389591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8fc1071a0 0x7fc8fc107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 --2- 172.21.15.33:0/2068389591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8fc101130 0x7fc8fc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 -- 172.21.15.33:0/2068389591 >> 172.21.15.33:0/2068389591 conn(0x7fc8fc0780e0 msgr2=0x7fc8fc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 -- 172.21.15.33:0/2068389591 shutdown_connections 2024-04-24T23:11:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.279+0000 7fc9008fc700 1 -- 172.21.15.33:0/2068389591 wait complete. 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 Processor -- start 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 -- start start 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8fc101130 0x7fc8fc197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8fc1071a0 0x7fc8fc197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc111410 con 0x7fc8fc101af0 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc111290 con 0x7fc8fc1071a0 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc9008fc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc111590 con 0x7fc8fc101130 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc8f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc8f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42474/0 (socket says 172.21.15.33:42474) 2024-04-24T23:11:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.280+0000 7fc8f9d9b700 1 -- 172.21.15.33:0/3814692851 learned_addr learned my addr 172.21.15.33:0/3814692851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8fa59c700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8fc101130 0x7fc8fc197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8fad9d700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8fc1071a0 0x7fc8fc197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f9d9b700 1 -- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8fc101130 msgr2=0x7fc8fc197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f9d9b700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8fc101130 0x7fc8fc197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f9d9b700 1 -- 172.21.15.33:0/3814692851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8fc1071a0 msgr2=0x7fc8fc197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f9d9b700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8fc1071a0 0x7fc8fc197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f9d9b700 1 -- 172.21.15.33:0/3814692851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1a6cf0 con 0x7fc8fc101af0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f9d9b700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc1979c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c9b0 tx=0x7fc8e400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007b00 con 0x7fc8fc101af0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e400ce50 con 0x7fc8fc101af0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc1a6fe0 con 0x7fc8fc101af0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4018660 con 0x7fc8fc101af0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.281+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1a7440 con 0x7fc8fc101af0 2024-04-24T23:11:00.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.282+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8fc100360 con 0x7fc8fc101af0 2024-04-24T23:11:00.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.284+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc8e400f450 con 0x7fc8fc101af0 2024-04-24T23:11:00.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.284+0000 7fc8f37fe700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8e80a2cb0 0x7fc8e80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.284+0000 7fc8fa59c700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8e80a2cb0 0x7fc8e80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.284+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc8e40cf6e0 con 0x7fc8fc101af0 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.284+0000 7fc8f37fe700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] conn(0x7fc8e80ad970 0x7fc8e80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.285+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 --> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8e80b0480 con 0x7fc8e80ad970 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.285+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc8e40cfb30 con 0x7fc8fc101af0 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.285+0000 7fc8fa59c700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8e80a2cb0 0x7fc8e80a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400abd0 tx=0x7fc8f400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.285+0000 7fc8fad9d700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] conn(0x7fc8e80ad970 0x7fc8e80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.285+0000 7fc8fad9d700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] conn(0x7fc8e80ad970 0x7fc8e80afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.286+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== osd.8 v2:172.21.15.117:6816/1247270534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8e80b0480 con 0x7fc8e80ad970 2024-04-24T23:11:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.302+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 --> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8fc102850 con 0x7fc8e80ad970 2024-04-24T23:11:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc8f37fe700 1 -- 172.21.15.33:0/3814692851 <== osd.8 v2:172.21.15.117:6816/1247270534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc8fc102850 con 0x7fc8e80ad970 2024-04-24T23:11:00.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] conn(0x7fc8e80ad970 msgr2=0x7fc8e80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] conn(0x7fc8e80ad970 0x7fc8e80afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8e80a2cb0 msgr2=0x7fc8e80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc8e80a2cb0 0x7fc8e80a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400abd0 tx=0x7fc8f400a380 comp rx=0 tx=0).stop 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 msgr2=0x7fc8fc1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 --2- 172.21.15.33:0/3814692851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc101af0 0x7fc8fc1979c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c9b0 tx=0x7fc8e400ccc0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc8fad9d700 1 -- 172.21.15.33:0/3814692851 reap_dead start 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 shutdown_connections 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 >> 172.21.15.33:0/3814692851 conn(0x7fc8fc0780e0 msgr2=0x7fc8fc0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 shutdown_connections 2024-04-24T23:11:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:00.303+0000 7fc9008fc700 1 -- 172.21.15.33:0/3814692851 wait complete. 2024-04-24T23:11:01.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:01 smithi033 ceph-mon[29201]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-24T23:11:01.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:01 smithi038 ceph-mon[32450]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-24T23:11:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:01 smithi117 ceph-mon[32529]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-24T23:11:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:03 smithi033 ceph-mon[29201]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-24T23:11:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:03 smithi038 ceph-mon[32450]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-24T23:11:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:03 smithi117 ceph-mon[32529]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 -- 172.21.15.33:0/1721332104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 msgr2=0x7f8b9c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/1721332104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c101790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0099c0 tx=0x7f8b8c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 -- 172.21.15.33:0/1721332104 shutdown_connections 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/1721332104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b9c106490 0x7f8b9c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/1721332104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c101790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/1721332104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c101d60 0x7f8b9c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 -- 172.21.15.33:0/1721332104 >> 172.21.15.33:0/1721332104 conn(0x7f8b9c0781d0 msgr2=0x7f8b9c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 -- 172.21.15.33:0/1721332104 shutdown_connections 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.157+0000 7f8ba14ed700 1 -- 172.21.15.33:0/1721332104 wait complete. 2024-04-24T23:11:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.158+0000 7f8ba14ed700 1 Processor -- start 2024-04-24T23:11:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.158+0000 7f8ba14ed700 1 -- start start 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.158+0000 7f8ba14ed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.158+0000 7f8ba14ed700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c101d60 0x7f8b9c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.158+0000 7f8ba14ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b9c106490 0x7f8b9c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.158+0000 7f8ba14ed700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111470 con 0x7f8b9c1013b0 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8ba14ed700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1112f0 con 0x7f8b9c106490 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8ba14ed700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1115f0 con 0x7f8b9c101d60 2024-04-24T23:11:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42494/0 (socket says 172.21.15.33:42494) 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 -- 172.21.15.33:0/2630034517 learned_addr learned my addr 172.21.15.33:0/2630034517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9b7fe700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b9c106490 0x7f8b9c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9a7fc700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c101d60 0x7f8b9c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 -- 172.21.15.33:0/2630034517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c101d60 msgr2=0x7f8b9c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c101d60 0x7f8b9c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 -- 172.21.15.33:0/2630034517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b9c106490 msgr2=0x7f8b9c197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b9c106490 0x7f8b9c197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 -- 172.21.15.33:0/2630034517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c198050 con 0x7f8b9c1013b0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b9affd700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c19d6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006010 tx=0x7f8b8c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c026070 con 0x7f8b9c1013b0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c1982e0 con 0x7f8b9c1013b0 2024-04-24T23:11:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c004cd0 con 0x7f8b9c1013b0 2024-04-24T23:11:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.159+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c003ba0 con 0x7f8b9c1013b0 2024-04-24T23:11:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.160+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c06a960 con 0x7f8b9c1013b0 2024-04-24T23:11:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.160+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b9c06af10 con 0x7f8b9c1013b0 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b8c029070 con 0x7f8b9c1013b0 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b83fff700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b840a4ec0 0x7f8b840a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b9a7fc700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b840a4ec0 0x7f8b840a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8b8c0d04b0 con 0x7f8b9c1013b0 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b83fff700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] conn(0x7f8b840afb80 0x7f8b840b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 --> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b8c001d30 con 0x7f8b840afb80 2024-04-24T23:11:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.162+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c0d0900 con 0x7f8b9c1013b0 2024-04-24T23:11:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.163+0000 7f8b9b7fe700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] conn(0x7f8b840afb80 0x7f8b840b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.163+0000 7f8b9a7fc700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b840a4ec0 0x7f8b840a7380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c198f30 tx=0x7f8b9000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.163+0000 7f8b9b7fe700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] conn(0x7f8b840afb80 0x7f8b840b1fc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.164+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== osd.0 v2:172.21.15.33:6802/2906488193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b8c001d30 con 0x7f8b840afb80 2024-04-24T23:11:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 --> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b9c06b390 con 0x7f8b840afb80 2024-04-24T23:11:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8b83fff700 1 -- 172.21.15.33:0/2630034517 <== osd.0 v2:172.21.15.33:6802/2906488193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b9c06b390 con 0x7f8b840afb80 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] conn(0x7f8b840afb80 msgr2=0x7f8b840b1fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] conn(0x7f8b840afb80 0x7f8b840b1fc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b840a4ec0 msgr2=0x7f8b840a7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b840a4ec0 0x7f8b840a7380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c198f30 tx=0x7f8b9000b410 comp rx=0 tx=0).stop 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 msgr2=0x7f8b9c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 --2- 172.21.15.33:0/2630034517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b9c1013b0 0x7f8b9c19d6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006010 tx=0x7f8b8c009f80 comp rx=0 tx=0).stop 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8b9b7fe700 1 -- 172.21.15.33:0/2630034517 reap_dead start 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 shutdown_connections 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.182+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 >> 172.21.15.33:0/2630034517 conn(0x7f8b9c0781d0 msgr2=0x7f8b9c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.183+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 shutdown_connections 2024-04-24T23:11:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:05.183+0000 7f8ba14ed700 1 -- 172.21.15.33:0/2630034517 wait complete. 2024-04-24T23:11:05.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:05 smithi033 ceph-mon[29201]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:11:05.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:05 smithi038 ceph-mon[32450]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:11:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:05 smithi117 ceph-mon[32529]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:11:06.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-24T23:11:06.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-24T23:11:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-24T23:11:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T23:11:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-24T23:11:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-24T23:11:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-24T23:11:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-24T23:11:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T23:11:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-24T23:11:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-24T23:11:06.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-24T23:11:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T23:11:06.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-24T23:11:06.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-24T23:11:06.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-24T23:11:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-24T23:11:07.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-24T23:11:07.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:07 smithi033 ceph-mon[29201]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-24T23:11:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:07 smithi038 ceph-mon[32450]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-24T23:11:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-24T23:11:07.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T23:11:07.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-24T23:11:07.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-24T23:11:07.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-24T23:11:07.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:07 smithi117 ceph-mon[32529]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:11:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:08 smithi033 ceph-mon[29201]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:11:08.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:08 smithi038 ceph-mon[32450]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:11:08.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:08 smithi117 ceph-mon[32529]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:11:09.161 INFO:teuthology.orchestra.run.smithi033.stdout:257698037799 2024-04-24T23:11:09.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T23:11:09.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 -- 172.21.15.33:0/2386148434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 msgr2=0x7f80b81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 --2- 172.21.15.33:0/2386148434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b81015b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009990 tx=0x7f80ac009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 -- 172.21.15.33:0/2386148434 shutdown_connections 2024-04-24T23:11:09.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 --2- 172.21.15.33:0/2386148434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b8101af0 0x7f80b810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 --2- 172.21.15.33:0/2386148434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81071a0 0x7f80b8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 --2- 172.21.15.33:0/2386148434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b81015b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 -- 172.21.15.33:0/2386148434 >> 172.21.15.33:0/2386148434 conn(0x7f80b80780e0 msgr2=0x7f80b80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 -- 172.21.15.33:0/2386148434 shutdown_connections 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.524+0000 7f80bd744700 1 -- 172.21.15.33:0/2386148434 wait complete. 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 Processor -- start 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 -- start start 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8101af0 0x7f80b8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b81071a0 0x7f80b81a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8111410 con 0x7f80b8101130 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8111290 con 0x7f80b81071a0 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80bd744700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8111590 con 0x7f80b8101af0 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40734/0 (socket says 172.21.15.33:40734) 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80b6ffd700 1 -- 172.21.15.33:0/23467735 learned_addr learned my addr 172.21.15.33:0/23467735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.525+0000 7f80b77fe700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b81071a0 0x7f80b81a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b67fc700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8101af0 0x7f80b8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b6ffd700 1 -- 172.21.15.33:0/23467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8101af0 msgr2=0x7f80b8199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b6ffd700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8101af0 0x7f80b8199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b6ffd700 1 -- 172.21.15.33:0/23467735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b81071a0 msgr2=0x7f80b81a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b6ffd700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80b81071a0 0x7f80b81a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b6ffd700 1 -- 172.21.15.33:0/23467735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b81a4d30 con 0x7f80b8101130 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b67fc700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8101af0 0x7f80b8199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80b6ffd700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b8199770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c9b0 tx=0x7f80a800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8007b00 con 0x7f80b8101130 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a8007ca0 con 0x7f80b8101130 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b81a4f60 con 0x7f80b8101130 2024-04-24T23:11:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8018730 con 0x7f80b8101130 2024-04-24T23:11:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.526+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a5410 con 0x7f80b8101130 2024-04-24T23:11:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.527+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80b81003e0 con 0x7f80b8101130 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.528+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f80a80188d0 con 0x7f80b8101130 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f809ffff700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80a00a2cb0 0x7f80a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f80b67fc700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80a00a2cb0 0x7f80a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f80a80cf8a0 con 0x7f80b8101130 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f809ffff700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] conn(0x7f80a00ade10 0x7f80a00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 --> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80a8010e70 con 0x7f80a00ade10 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f80a80cfcf0 con 0x7f80b8101130 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.529+0000 7f80b77fe700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] conn(0x7f80a00ade10 0x7f80a00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.530+0000 7f80b67fc700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80a00a2cb0 0x7f80a00a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006010 tx=0x7f80ac00b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.530+0000 7f80b77fe700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] conn(0x7f80a00ade10 0x7f80a00b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.531+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== osd.9 v2:172.21.15.33:6826/498632081 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80a8010e70 con 0x7f80a00ade10 2024-04-24T23:11:09.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.547+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 --> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80b8102850 con 0x7f80a00ade10 2024-04-24T23:11:09.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f809ffff700 1 -- 172.21.15.33:0/23467735 <== osd.9 v2:172.21.15.33:6826/498632081 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f80b8102850 con 0x7f80a00ade10 2024-04-24T23:11:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] conn(0x7f80a00ade10 msgr2=0x7f80a00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] conn(0x7f80a00ade10 0x7f80a00b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80a00a2cb0 msgr2=0x7f80a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80a00a2cb0 0x7f80a00a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006010 tx=0x7f80ac00b540 comp rx=0 tx=0).stop 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 msgr2=0x7f80b8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 --2- 172.21.15.33:0/23467735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b8101130 0x7f80b8199770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c9b0 tx=0x7f80a800ccc0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80b77fe700 1 -- 172.21.15.33:0/23467735 reap_dead start 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 shutdown_connections 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 >> 172.21.15.33:0/23467735 conn(0x7f80b80780e0 msgr2=0x7f80b80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 shutdown_connections 2024-04-24T23:11:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:09.548+0000 7f80bd744700 1 -- 172.21.15.33:0/23467735 wait complete. 2024-04-24T23:11:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:09 smithi033 ceph-mon[29201]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:11:09.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:09 smithi038 ceph-mon[32450]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:11:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:09 smithi117 ceph-mon[32529]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:11:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 -- 172.21.15.33:0/1510008492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074460 msgr2=0x7f5900074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 --2- 172.21.15.33:0/1510008492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074460 0x7f5900074840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009990 tx=0x7f58fc009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 -- 172.21.15.33:0/1510008492 shutdown_connections 2024-04-24T23:11:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 --2- 172.21.15.33:0/1510008492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f590010ec30 0x7f5900111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 --2- 172.21.15.33:0/1510008492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074460 0x7f5900074840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 --2- 172.21.15.33:0/1510008492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5900074d80 0x7f590010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 -- 172.21.15.33:0/1510008492 >> 172.21.15.33:0/1510008492 conn(0x7f5900079390 msgr2=0x7f59000797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 -- 172.21.15.33:0/1510008492 shutdown_connections 2024-04-24T23:11:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.273+0000 7f5907854700 1 -- 172.21.15.33:0/1510008492 wait complete. 2024-04-24T23:11:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 Processor -- start 2024-04-24T23:11:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 -- start start 2024-04-24T23:11:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5900074460 0x7f590019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074d80 0x7f590019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f590010ec30 0x7f590019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5900114800 con 0x7f5900074d80 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5900114680 con 0x7f5900074460 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5907854700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5900114980 con 0x7f590010ec30 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5904def700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074d80 0x7f590019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5904def700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074d80 0x7f590019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40754/0 (socket says 172.21.15.33:40754) 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.274+0000 7f5904def700 1 -- 172.21.15.33:0/3347589930 learned_addr learned my addr 172.21.15.33:0/3347589930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5905df1700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f590010ec30 0x7f590019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f59055f0700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5900074460 0x7f590019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5904def700 1 -- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f590010ec30 msgr2=0x7f590019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5904def700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f590010ec30 0x7f590019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5904def700 1 -- 172.21.15.33:0/3347589930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5900074460 msgr2=0x7f590019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5904def700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5900074460 0x7f590019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5904def700 1 -- 172.21.15.33:0/3347589930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590019fd50 con 0x7f5900074d80 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5905df1700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f590010ec30 0x7f590019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5904def700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074d80 0x7f590019b2d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b530 tx=0x7f58f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0015020 con 0x7f5900074d80 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f0011d10 con 0x7f5900074d80 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f000f2e0 con 0x7f5900074d80 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59001a0040 con 0x7f5900074d80 2024-04-24T23:11:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.275+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59001a78c0 con 0x7f5900074d80 2024-04-24T23:11:11.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.276+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59000629c0 con 0x7f5900074d80 2024-04-24T23:11:11.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.278+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f58f0018020 con 0x7f5900074d80 2024-04-24T23:11:11.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.278+0000 7f58f67fc700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58ec0a2cb0 0x7f58ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f59055f0700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58ec0a2cb0 0x7f58ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f58f00cf3e0 con 0x7f5900074d80 2024-04-24T23:11:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f58f67fc700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] conn(0x7f58ec0ade10 0x7f58ec0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 --> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58f0013ec0 con 0x7f58ec0ade10 2024-04-24T23:11:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f58f00cf790 con 0x7f5900074d80 2024-04-24T23:11:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f59055f0700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58ec0a2cb0 0x7f58ec0a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0098e0 tx=0x7f58fc005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.279+0000 7f5905df1700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] conn(0x7f58ec0ade10 0x7f58ec0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.280+0000 7f5905df1700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] conn(0x7f58ec0ade10 0x7f58ec0b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.280+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== osd.5 v2:172.21.15.117:6808/2567468541 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58f0013ec0 con 0x7f58ec0ade10 2024-04-24T23:11:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.297+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 --> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5900062dc0 con 0x7f58ec0ade10 2024-04-24T23:11:11.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.297+0000 7f58f67fc700 1 -- 172.21.15.33:0/3347589930 <== osd.5 v2:172.21.15.117:6808/2567468541 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5900062dc0 con 0x7f58ec0ade10 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] conn(0x7f58ec0ade10 msgr2=0x7f58ec0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] conn(0x7f58ec0ade10 0x7f58ec0b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58ec0a2cb0 msgr2=0x7f58ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58ec0a2cb0 0x7f58ec0a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0098e0 tx=0x7f58fc005960 comp rx=0 tx=0).stop 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074d80 msgr2=0x7f590019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 --2- 172.21.15.33:0/3347589930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900074d80 0x7f590019b2d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b530 tx=0x7f58f000b840 comp rx=0 tx=0).stop 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5905df1700 1 -- 172.21.15.33:0/3347589930 reap_dead start 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 shutdown_connections 2024-04-24T23:11:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 >> 172.21.15.33:0/3347589930 conn(0x7f5900079390 msgr2=0x7f59000ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 shutdown_connections 2024-04-24T23:11:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.298+0000 7f5907854700 1 -- 172.21.15.33:0/3347589930 wait complete. 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 -- 172.21.15.33:0/2682151071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c1071a0 msgr2=0x7f363c107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 --2- 172.21.15.33:0/2682151071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c1071a0 0x7f363c107580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f363400b0a0 tx=0x7f363400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 -- 172.21.15.33:0/2682151071 shutdown_connections 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 --2- 172.21.15.33:0/2682151071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c101af0 0x7f363c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 --2- 172.21.15.33:0/2682151071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c1071a0 0x7f363c107580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 --2- 172.21.15.33:0/2682151071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 0x7f363c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 -- 172.21.15.33:0/2682151071 >> 172.21.15.33:0/2682151071 conn(0x7f363c0780e0 msgr2=0x7f363c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 -- 172.21.15.33:0/2682151071 shutdown_connections 2024-04-24T23:11:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.622+0000 7f3640f98700 1 -- 172.21.15.33:0/2682151071 wait complete. 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 Processor -- start 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 -- start start 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 0x7f363c102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c101af0 0x7f363c103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c1071a0 0x7f363c103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f363c111460 con 0x7f363c101af0 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1112e0 con 0x7f363c1071a0 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3640f98700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1115e0 con 0x7f363c101130 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f3639d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c101af0 0x7f363c103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.623+0000 7f363a59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 0x7f363c102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f3639d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c101af0 0x7f363c103350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40798/0 (socket says 172.21.15.33:40798) 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f3639d9b700 1 -- 172.21.15.33:0/3436476204 learned_addr learned my addr 172.21.15.33:0/3436476204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 0x7f363c102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34972/0 (socket says 172.21.15.33:34972) 2024-04-24T23:11:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363ad9d700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c1071a0 0x7f363c103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 -- 172.21.15.33:0/3436476204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c1071a0 msgr2=0x7f363c103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c1071a0 0x7f363c103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 -- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c101af0 msgr2=0x7f363c103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f363c101af0 0x7f363c103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 -- 172.21.15.33:0/3436476204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1a6f70 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f363a59c700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 0x7f363c102e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f363400b780 tx=0x7f3634004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634021070 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1a7200 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634007b80 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36340052c0 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.624+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a76b0 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.625+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f363c105220 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3634024070 con 0x7f363c101130 2024-04-24T23:11:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f36337fe700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36280a2cb0 0x7f36280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f3639d9b700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36280a2cb0 0x7f36280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f363400f020 con 0x7f363c101130 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f36337fe700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] conn(0x7f36280ade10 0x7f36280b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 --> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3634026e50 con 0x7f36280ade10 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.627+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f36340d1130 con 0x7f363c101130 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.628+0000 7f363ad9d700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] conn(0x7f36280ade10 0x7f36280b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.628+0000 7f3639d9b700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36280a2cb0 0x7f36280a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f363c078650 tx=0x7f3624009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.628+0000 7f363ad9d700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] conn(0x7f36280ade10 0x7f36280b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.629+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== osd.2 v2:172.21.15.117:6800/1483584233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3634026e50 con 0x7f36280ade10 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.645+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 --> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f363c102850 con 0x7f36280ade10 2024-04-24T23:11:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f36337fe700 1 -- 172.21.15.33:0/3436476204 <== osd.2 v2:172.21.15.117:6800/1483584233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f363c102850 con 0x7f36280ade10 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] conn(0x7f36280ade10 msgr2=0x7f36280b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] conn(0x7f36280ade10 0x7f36280b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36280a2cb0 msgr2=0x7f36280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f36280a2cb0 0x7f36280a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f363c078650 tx=0x7f3624009260 comp rx=0 tx=0).stop 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 msgr2=0x7f363c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 --2- 172.21.15.33:0/3436476204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f363c101130 0x7f363c102e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f363400b780 tx=0x7f3634004d00 comp rx=0 tx=0).stop 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f363ad9d700 1 -- 172.21.15.33:0/3436476204 reap_dead start 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 shutdown_connections 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.646+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 >> 172.21.15.33:0/3436476204 conn(0x7f363c0780e0 msgr2=0x7f363c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.647+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 shutdown_connections 2024-04-24T23:11:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.647+0000 7f3640f98700 1 -- 172.21.15.33:0/3436476204 wait complete. 2024-04-24T23:11:11.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:11 smithi038 ceph-mon[32450]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 72 B/s, 0 objects/s recovering 2024-04-24T23:11:11.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:11.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:11 smithi117 ceph-mon[32529]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 72 B/s, 0 objects/s recovering 2024-04-24T23:11:11.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:11.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:11 smithi033 ceph-mon[29201]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 72 B/s, 0 objects/s recovering 2024-04-24T23:11:11.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:12.439 INFO:teuthology.orchestra.run.smithi033.stdout:77309411400 2024-04-24T23:11:12.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 -- 172.21.15.33:0/4169924177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b481064c0 msgr2=0x7f7b4810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/4169924177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b481064c0 0x7f7b4810e060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000b0d0 tx=0x7f7b4000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 -- 172.21.15.33:0/4169924177 shutdown_connections 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/4169924177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b481064c0 0x7f7b4810e060 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/4169924177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b48101770 0x7f7b48101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/4169924177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b48102120 0x7f7b48105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 -- 172.21.15.33:0/4169924177 >> 172.21.15.33:0/4169924177 conn(0x7f7b480780e0 msgr2=0x7f7b480784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 -- 172.21.15.33:0/4169924177 shutdown_connections 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 -- 172.21.15.33:0/4169924177 wait complete. 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.485+0000 7f7b4e07c700 1 Processor -- start 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 -- start start 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b48101770 0x7f7b4819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b48102120 0x7f7b4819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b481064c0 0x7f7b48198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481114a0 con 0x7f7b48101770 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48111320 con 0x7f7b481064c0 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b4e07c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48111620 con 0x7f7b48102120 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b48101770 0x7f7b4819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b48101770 0x7f7b4819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40776/0 (socket says 172.21.15.33:40776) 2024-04-24T23:11:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 -- 172.21.15.33:0/576647633 learned_addr learned my addr 172.21.15.33:0/576647633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b46ffd700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b48102120 0x7f7b4819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b47fff700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b481064c0 0x7f7b48198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 -- 172.21.15.33:0/576647633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b48102120 msgr2=0x7f7b4819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b48102120 0x7f7b4819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 -- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b481064c0 msgr2=0x7f7b48198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b481064c0 0x7f7b48198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.486+0000 7f7b477fe700 1 -- 172.21.15.33:0/576647633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b48198ff0 con 0x7f7b48101770 2024-04-24T23:11:12.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.487+0000 7f7b477fe700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b48101770 0x7f7b4819e640 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800e9c0 tx=0x7f7b3800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.487+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800cbd0 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.487+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481992e0 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.487+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3800ee60 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.487+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b48102d60 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.487+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38005630 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.488+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b34001060 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.489+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b38004750 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b44ff9700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b300a2d30 0x7f7b300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b46ffd700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b300a2d30 0x7f7b300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b380cfd00 con 0x7f7b48101770 2024-04-24T23:11:12.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b44ff9700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] conn(0x7f7b300ade90 0x7f7b300b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 --> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b38022e50 con 0x7f7b300ade90 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7b380d00b0 con 0x7f7b48101770 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b47fff700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] conn(0x7f7b300ade90 0x7f7b300b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.490+0000 7f7b46ffd700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b300a2d30 0x7f7b300a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b481013e0 tx=0x7f7b3c0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.491+0000 7f7b47fff700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] conn(0x7f7b300ade90 0x7f7b300b0330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.492+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== osd.4 v2:172.21.15.38:6808/1943632582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b38022e50 con 0x7f7b300ade90 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.508+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 --> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b34002e90 con 0x7f7b300ade90 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b44ff9700 1 -- 172.21.15.33:0/576647633 <== osd.4 v2:172.21.15.38:6808/1943632582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b34002e90 con 0x7f7b300ade90 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] conn(0x7f7b300ade90 msgr2=0x7f7b300b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] conn(0x7f7b300ade90 0x7f7b300b0330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b300a2d30 msgr2=0x7f7b300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b300a2d30 0x7f7b300a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b481013e0 tx=0x7f7b3c0073d0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b48101770 msgr2=0x7f7b4819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 --2- 172.21.15.33:0/576647633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b48101770 0x7f7b4819e640 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800e9c0 tx=0x7f7b3800ecd0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b47fff700 1 -- 172.21.15.33:0/576647633 reap_dead start 2024-04-24T23:11:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 shutdown_connections 2024-04-24T23:11:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 >> 172.21.15.33:0/576647633 conn(0x7f7b480780e0 msgr2=0x7f7b4806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 shutdown_connections 2024-04-24T23:11:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:11.509+0000 7f7b4e07c700 1 -- 172.21.15.33:0/576647633 wait complete. 2024-04-24T23:11:13.144 INFO:teuthology.orchestra.run.smithi033.stdout:146028888124 2024-04-24T23:11:13.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T23:11:13.837 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:13 smithi033 ceph-mon[29201]: pgmap v325: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 72 B/s, 0 objects/s recovering 2024-04-24T23:11:13.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:13 smithi038 ceph-mon[32450]: pgmap v325: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 72 B/s, 0 objects/s recovering 2024-04-24T23:11:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:13 smithi117 ceph-mon[32529]: pgmap v325: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 72 B/s, 0 objects/s recovering 2024-04-24T23:11:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:14.196 INFO:teuthology.orchestra.run.smithi033.stdout:167503724600 2024-04-24T23:11:14.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T23:11:15.146 INFO:teuthology.orchestra.run.smithi033.stdout:292057776162 2024-04-24T23:11:15.146 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 -- 172.21.15.33:0/1087223646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 msgr2=0x7fad70105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 --2- 172.21.15.33:0/1087223646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad70105d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fad600099c0 tx=0x7fad60009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 -- 172.21.15.33:0/1087223646 shutdown_connections 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 --2- 172.21.15.33:0/1087223646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad70106450 0x7fad7010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 --2- 172.21.15.33:0/1087223646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad701013c0 0x7fad701017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 --2- 172.21.15.33:0/1087223646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad70105d70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 -- 172.21.15.33:0/1087223646 >> 172.21.15.33:0/1087223646 conn(0x7fad700780e0 msgr2=0x7fad700784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 -- 172.21.15.33:0/1087223646 shutdown_connections 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.383+0000 7fad756dd700 1 -- 172.21.15.33:0/1087223646 wait complete. 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 Processor -- start 2024-04-24T23:11:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 -- start start 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad701013c0 0x7fad7019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad7019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad70106450 0x7fad70197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad70111430 con 0x7fad70101d70 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad701112b0 con 0x7fad701013c0 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad756dd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad701115b0 con 0x7fad70106450 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad6e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad7019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad6e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad7019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40820/0 (socket says 172.21.15.33:40820) 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad6e7fc700 1 -- 172.21.15.33:0/2005442 learned_addr learned my addr 172.21.15.33:0/2005442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad6f7fe700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad70106450 0x7fad70197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.384+0000 7fad6effd700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad701013c0 0x7fad7019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.385+0000 7fad6e7fc700 1 -- 172.21.15.33:0/2005442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad70106450 msgr2=0x7fad70197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.385+0000 7fad6e7fc700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad70106450 0x7fad70197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.385+0000 7fad6e7fc700 1 -- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad701013c0 msgr2=0x7fad7019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.385+0000 7fad6e7fc700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad701013c0 0x7fad7019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.385+0000 7fad6e7fc700 1 -- 172.21.15.33:0/2005442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad70198010 con 0x7fad70101d70 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.385+0000 7fad6e7fc700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad7019db50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c010 tx=0x7fad600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.386+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60003960 con 0x7fad70101d70 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.386+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad60003fe0 con 0x7fad70101d70 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.386+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6000f460 con 0x7fad70101d70 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.386+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701982a0 con 0x7fad70101d70 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.386+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7006a960 con 0x7fad70101d70 2024-04-24T23:11:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.387+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad7006afd0 con 0x7fad70101d70 2024-04-24T23:11:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.389+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad6001f4d0 con 0x7fad70101d70 2024-04-24T23:11:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad57fff700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad580a2cb0 0x7fad580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fad600d4950 con 0x7fad70101d70 2024-04-24T23:11:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad6effd700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad580a2cb0 0x7fad580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad57fff700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] conn(0x7fad580ade10 0x7fad580b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 --> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad60001d30 con 0x7fad580ade10 2024-04-24T23:11:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fad600d4d00 con 0x7fad70101d70 2024-04-24T23:11:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad6f7fe700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] conn(0x7fad580ade10 0x7fad580b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.390+0000 7fad6effd700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad580a2cb0 0x7fad580a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad640093a0 tx=0x7fad64009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.392+0000 7fad6f7fe700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] conn(0x7fad580ade10 0x7fad580b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.392+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== osd.7 v2:172.21.15.38:6816/3279589891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad60001d30 con 0x7fad580ade10 2024-04-24T23:11:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.409+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 --> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad7006b450 con 0x7fad580ade10 2024-04-24T23:11:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.410+0000 7fad57fff700 1 -- 172.21.15.33:0/2005442 <== osd.7 v2:172.21.15.38:6816/3279589891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fad7006b450 con 0x7fad580ade10 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.410+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] conn(0x7fad580ade10 msgr2=0x7fad580b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.410+0000 7fad756dd700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] conn(0x7fad580ade10 0x7fad580b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad580a2cb0 msgr2=0x7fad580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad580a2cb0 0x7fad580a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad640093a0 tx=0x7fad64009530 comp rx=0 tx=0).stop 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 msgr2=0x7fad7019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 --2- 172.21.15.33:0/2005442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad70101d70 0x7fad7019db50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c010 tx=0x7fad600047b0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad6f7fe700 1 -- 172.21.15.33:0/2005442 reap_dead start 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 shutdown_connections 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 >> 172.21.15.33:0/2005442 conn(0x7fad700780e0 msgr2=0x7fad700ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 shutdown_connections 2024-04-24T23:11:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:15.411+0000 7fad756dd700 1 -- 172.21.15.33:0/2005442 wait complete. 2024-04-24T23:11:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:15 smithi033 ceph-mon[29201]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 149 B/s, 2 keys/s, 1 objects/s recovering 2024-04-24T23:11:15.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:15 smithi038 ceph-mon[32450]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 149 B/s, 2 keys/s, 1 objects/s recovering 2024-04-24T23:11:15.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:15 smithi117 ceph-mon[32529]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 149 B/s, 2 keys/s, 1 objects/s recovering 2024-04-24T23:11:16.104 INFO:teuthology.orchestra.run.smithi033.stdout:223338299438 2024-04-24T23:11:16.104 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T23:11:17.103 INFO:teuthology.orchestra.run.smithi033.stdout:34359738448 2024-04-24T23:11:17.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 -- 172.21.15.33:0/3018238866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 msgr2=0x7f8a08108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 --2- 172.21.15.33:0/3018238866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a08108690 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b0a0 tx=0x7f89f800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 -- 172.21.15.33:0/3018238866 shutdown_connections 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 --2- 172.21.15.33:0/3018238866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0806a8e0 0x7f8a0810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 --2- 172.21.15.33:0/3018238866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a08108690 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 --2- 172.21.15.33:0/3018238866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08069f40 0x7f8a0806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 -- 172.21.15.33:0/3018238866 >> 172.21.15.33:0/3018238866 conn(0x7f8a080780e0 msgr2=0x7f8a080784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 -- 172.21.15.33:0/3018238866 shutdown_connections 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.332+0000 7f8a10166700 1 -- 172.21.15.33:0/3018238866 wait complete. 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 Processor -- start 2024-04-24T23:11:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 -- start start 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08069f40 0x7f8a0819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0806a8e0 0x7f8a0819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a081988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08112560 con 0x7f8a081082b0 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081123e0 con 0x7f8a08069f40 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a10166700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081126e0 con 0x7f8a0806a8e0 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a081988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a081988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37700/0 (socket says 172.21.15.33:37700) 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 -- 172.21.15.33:0/1060520150 learned_addr learned my addr 172.21.15.33:0/1060520150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0d701700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0806a8e0 0x7f8a0819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0df02700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08069f40 0x7f8a0819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 -- 172.21.15.33:0/1060520150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0806a8e0 msgr2=0x7f8a0819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0806a8e0 0x7f8a0819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 -- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08069f40 msgr2=0x7f8a0819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08069f40 0x7f8a0819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0e703700 1 -- 172.21.15.33:0/1060520150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a08199000 con 0x7f8a081082b0 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.333+0000 7f8a0df02700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08069f40 0x7f8a0819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.334+0000 7f8a0e703700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a081988a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000d980 tx=0x7f8a0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.334+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00016070 con 0x7f8a081082b0 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.334+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a00004d10 con 0x7f8a081082b0 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.334+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081992f0 con 0x7f8a081082b0 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.334+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000055e0 con 0x7f8a081082b0 2024-04-24T23:11:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.334+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a55b0 con 0x7f8a081082b0 2024-04-24T23:11:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.335+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a08062980 con 0x7f8a081082b0 2024-04-24T23:11:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.336+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a00004750 con 0x7f8a081082b0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a06ffd700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f89f00a2cb0 0x7f89f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a0df02700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f89f00a2cb0 0x7f89f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8a000cf9f0 con 0x7f8a081082b0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a06ffd700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] conn(0x7f89f00ade10 0x7f89f00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 --> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a00015ec0 con 0x7f89f00ade10 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f8a000cfda0 con 0x7f8a081082b0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a0df02700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f89f00a2cb0 0x7f89f00a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89f80062a0 tx=0x7f89f8006210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.337+0000 7f8a0d701700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] conn(0x7f89f00ade10 0x7f89f00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.338+0000 7f8a0d701700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] conn(0x7f89f00ade10 0x7f89f00b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.339+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== osd.10 v2:172.21.15.38:6824/1322822430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a00015ec0 con 0x7f89f00ade10 2024-04-24T23:11:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.357+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 --> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a08062e00 con 0x7f89f00ade10 2024-04-24T23:11:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.358+0000 7f8a06ffd700 1 -- 172.21.15.33:0/1060520150 <== osd.10 v2:172.21.15.38:6824/1322822430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a08062e00 con 0x7f89f00ade10 2024-04-24T23:11:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.358+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] conn(0x7f89f00ade10 msgr2=0x7f89f00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.358+0000 7f8a10166700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] conn(0x7f89f00ade10 0x7f89f00b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f89f00a2cb0 msgr2=0x7f89f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f89f00a2cb0 0x7f89f00a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89f80062a0 tx=0x7f89f8006210 comp rx=0 tx=0).stop 2024-04-24T23:11:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 msgr2=0x7f8a081988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 --2- 172.21.15.33:0/1060520150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a081082b0 0x7f8a081988a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000d980 tx=0x7f8a0000dc90 comp rx=0 tx=0).stop 2024-04-24T23:11:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a0e703700 1 -- 172.21.15.33:0/1060520150 reap_dead start 2024-04-24T23:11:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 shutdown_connections 2024-04-24T23:11:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 >> 172.21.15.33:0/1060520150 conn(0x7f8a080780e0 msgr2=0x7f8a080fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 shutdown_connections 2024-04-24T23:11:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.359+0000 7f8a10166700 1 -- 172.21.15.33:0/1060520150 wait complete. 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 -- 172.21.15.33:0/2425995591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c1071a0 msgr2=0x7ff61c107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 --2- 172.21.15.33:0/2425995591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c1071a0 0x7ff61c107580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff614009990 tx=0x7ff614009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 -- 172.21.15.33:0/2425995591 shutdown_connections 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 --2- 172.21.15.33:0/2425995591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c101af0 0x7ff61c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 --2- 172.21.15.33:0/2425995591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c1071a0 0x7ff61c107580 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 --2- 172.21.15.33:0/2425995591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff61c101130 0x7ff61c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 -- 172.21.15.33:0/2425995591 >> 172.21.15.33:0/2425995591 conn(0x7ff61c0780e0 msgr2=0x7ff61c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 -- 172.21.15.33:0/2425995591 shutdown_connections 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.572+0000 7ff622173700 1 -- 172.21.15.33:0/2425995591 wait complete. 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 Processor -- start 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 -- start start 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c101130 0x7ff61c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff61c101af0 0x7ff61c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c1071a0 0x7ff61c1a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111410 con 0x7ff61c101130 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111290 con 0x7ff61c101af0 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff622173700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111590 con 0x7ff61c1071a0 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.573+0000 7ff61b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c101130 0x7ff61c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c101130 0x7ff61c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37716/0 (socket says 172.21.15.33:37716) 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 -- 172.21.15.33:0/1424856279 learned_addr learned my addr 172.21.15.33:0/1424856279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61bfff700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c1071a0 0x7ff61c1a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61affd700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff61c101af0 0x7ff61c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 -- 172.21.15.33:0/1424856279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c1071a0 msgr2=0x7ff61c1a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c1071a0 0x7ff61c1a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 -- 172.21.15.33:0/1424856279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff61c101af0 msgr2=0x7ff61c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff61c101af0 0x7ff61c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 -- 172.21.15.33:0/1424856279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c1a4d70 con 0x7ff61c101130 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff61b7fe700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c101130 0x7ff61c1997b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6140038b0 tx=0x7ff614009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614027070 con 0x7ff61c101130 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff614004050 con 0x7ff61c101130 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614022c00 con 0x7ff61c101130 2024-04-24T23:11:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c1a4fa0 con 0x7ff61c101130 2024-04-24T23:11:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.574+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a53a0 con 0x7ff61c101130 2024-04-24T23:11:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.575+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff61c0628c0 con 0x7ff61c101130 2024-04-24T23:11:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.578+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff61402a070 con 0x7ff61c101130 2024-04-24T23:11:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.578+0000 7ff618ff9700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6080a2cb0 0x7ff6080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.579+0000 7ff61affd700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6080a2cb0 0x7ff6080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.579+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff6140d0e80 con 0x7ff61c101130 2024-04-24T23:11:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.579+0000 7ff618ff9700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] conn(0x7ff6080ade10 0x7ff6080b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.579+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 --> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff614009030 con 0x7ff6080ade10 2024-04-24T23:11:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.579+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff6140d12d0 con 0x7ff61c101130 2024-04-24T23:11:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.579+0000 7ff61affd700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6080a2cb0 0x7ff6080a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0787c0 tx=0x7ff60400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.580+0000 7ff61bfff700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] conn(0x7ff6080ade10 0x7ff6080b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.580+0000 7ff61bfff700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] conn(0x7ff6080ade10 0x7ff6080b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.582+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== osd.6 v2:172.21.15.33:6818/713285361 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff614009030 con 0x7ff6080ade10 2024-04-24T23:11:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.601+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 --> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff61c062d40 con 0x7ff6080ade10 2024-04-24T23:11:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.601+0000 7ff618ff9700 1 -- 172.21.15.33:0/1424856279 <== osd.6 v2:172.21.15.33:6818/713285361 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff61c062d40 con 0x7ff6080ade10 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] conn(0x7ff6080ade10 msgr2=0x7ff6080b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] conn(0x7ff6080ade10 0x7ff6080b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6080a2cb0 msgr2=0x7ff6080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6080a2cb0 0x7ff6080a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0787c0 tx=0x7ff60400b410 comp rx=0 tx=0).stop 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c101130 msgr2=0x7ff61c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 --2- 172.21.15.33:0/1424856279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff61c101130 0x7ff61c1997b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6140038b0 tx=0x7ff614009f50 comp rx=0 tx=0).stop 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff61bfff700 1 -- 172.21.15.33:0/1424856279 reap_dead start 2024-04-24T23:11:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 shutdown_connections 2024-04-24T23:11:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 >> 172.21.15.33:0/1424856279 conn(0x7ff61c0780e0 msgr2=0x7ff61c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 shutdown_connections 2024-04-24T23:11:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:17.602+0000 7ff622173700 1 -- 172.21.15.33:0/1424856279 wait complete. 2024-04-24T23:11:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:17 smithi038 ceph-mon[32450]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 129 B/s, 2 keys/s, 1 objects/s recovering 2024-04-24T23:11:17.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:17 smithi117 ceph-mon[32529]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 129 B/s, 2 keys/s, 1 objects/s recovering 2024-04-24T23:11:17.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:17 smithi033 ceph-mon[29201]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 129 B/s, 2 keys/s, 1 objects/s recovering 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2578432518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80100e50 msgr2=0x7f7d80101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2578432518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80100e50 0x7f7d80101230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d740099c0 tx=0x7f7d74009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2578432518 shutdown_connections 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2578432518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d80107670 0x7f7d8010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2578432518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80100e50 0x7f7d80101230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2578432518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d80101800 0x7f7d80106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2578432518 >> 172.21.15.33:0/2578432518 conn(0x7f7d80076450 msgr2=0x7f7d80076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2578432518 shutdown_connections 2024-04-24T23:11:18.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2578432518 wait complete. 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.579+0000 7f7d86d3a700 1 Processor -- start 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 -- start start 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d80100e50 0x7f7d801a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80101800 0x7f7d801a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d80107670 0x7f7d8019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80112570 con 0x7f7d80101800 2024-04-24T23:11:18.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801123f0 con 0x7f7d80107670 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d86d3a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801126f0 con 0x7f7d80100e50 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80101800 0x7f7d801a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80101800 0x7f7d801a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37734/0 (socket says 172.21.15.33:37734) 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 -- 172.21.15.33:0/2073368819 learned_addr learned my addr 172.21.15.33:0/2073368819 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d84ad6700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d80100e50 0x7f7d801a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d852d7700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d80107670 0x7f7d8019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 -- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d80100e50 msgr2=0x7f7d801a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d80100e50 0x7f7d801a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 -- 172.21.15.33:0/2073368819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d80107670 msgr2=0x7f7d8019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d80107670 0x7f7d8019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 -- 172.21.15.33:0/2073368819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8019b0d0 con 0x7f7d80101800 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d852d7700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d80107670 0x7f7d8019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.580+0000 7f7d7ffff700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80101800 0x7f7d801a0cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00d980 tx=0x7f7d6c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.581+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c016070 con 0x7f7d80101800 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.581+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6c004500 con 0x7f7d80101800 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.581+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8019b3c0 con 0x7f7d80101800 2024-04-24T23:11:18.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.581+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c004d00 con 0x7f7d80101800 2024-04-24T23:11:18.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.581+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801a7680 con 0x7f7d80101800 2024-04-24T23:11:18.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.582+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d800629c0 con 0x7f7d80101800 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.583+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7d6c00bbf0 con 0x7f7d80101800 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.583+0000 7f7d7dffb700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d700a2cb0 0x7f7d700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.583+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7d6c0cf400 con 0x7f7d80101800 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d84ad6700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d700a2cb0 0x7f7d700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d7dffb700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] conn(0x7f7d700ade10 0x7f7d700b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 --> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d700b0980 con 0x7f7d700ade10 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7d6c0d6050 con 0x7f7d80101800 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d84ad6700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d700a2cb0 0x7f7d700a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d74000c00 tx=0x7f7d74005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d852d7700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] conn(0x7f7d700ade10 0x7f7d700b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.584+0000 7f7d852d7700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] conn(0x7f7d700ade10 0x7f7d700b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.585+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== osd.11 v2:172.21.15.117:6824/2336105808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d700b0980 con 0x7f7d700ade10 2024-04-24T23:11:18.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.603+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 --> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d80062e40 con 0x7f7d700ade10 2024-04-24T23:11:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.603+0000 7f7d7dffb700 1 -- 172.21.15.33:0/2073368819 <== osd.11 v2:172.21.15.117:6824/2336105808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7d80062e40 con 0x7f7d700ade10 2024-04-24T23:11:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] conn(0x7f7d700ade10 msgr2=0x7f7d700b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] conn(0x7f7d700ade10 0x7f7d700b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d700a2cb0 msgr2=0x7f7d700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d700a2cb0 0x7f7d700a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d74000c00 tx=0x7f7d74005fd0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80101800 msgr2=0x7f7d801a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d86d3a700 1 --2- 172.21.15.33:0/2073368819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d80101800 0x7f7d801a0cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00d980 tx=0x7f7d6c00dd40 comp rx=0 tx=0).stop 2024-04-24T23:11:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.604+0000 7f7d852d7700 1 -- 172.21.15.33:0/2073368819 reap_dead start 2024-04-24T23:11:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.605+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 shutdown_connections 2024-04-24T23:11:18.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.605+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 >> 172.21.15.33:0/2073368819 conn(0x7f7d80076450 msgr2=0x7f7d800fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:18.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.605+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 shutdown_connections 2024-04-24T23:11:18.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:18.606+0000 7f7d86d3a700 1 -- 172.21.15.33:0/2073368819 wait complete. 2024-04-24T23:11:19.513 INFO:teuthology.orchestra.run.smithi033.stdout:313532612640 2024-04-24T23:11:19.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 -- 172.21.15.33:0/3488640993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc107570 msgr2=0x7f88cc10f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3488640993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc107570 0x7f88cc10f140 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f88c400b0a0 tx=0x7f88c400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 -- 172.21.15.33:0/3488640993 shutdown_connections 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3488640993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc107570 0x7f88cc10f140 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3488640993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc1024e0 0x7f88cc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3488640993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88cc102e90 0x7f88cc106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 -- 172.21.15.33:0/3488640993 >> 172.21.15.33:0/3488640993 conn(0x7f88cc079390 msgr2=0x7f88cc0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 -- 172.21.15.33:0/3488640993 shutdown_connections 2024-04-24T23:11:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.708+0000 7f88d36b5700 1 -- 172.21.15.33:0/3488640993 wait complete. 2024-04-24T23:11:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 Processor -- start 2024-04-24T23:11:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 -- start start 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc1024e0 0x7f88cc19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc102e90 0x7f88cc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88cc107570 0x7f88cc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1125a0 con 0x7f88cc1024e0 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc112420 con 0x7f88cc107570 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d36b5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc112720 con 0x7f88cc102e90 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc1024e0 0x7f88cc19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc1024e0 0x7f88cc19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37744/0 (socket says 172.21.15.33:37744) 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 -- 172.21.15.33:0/3111447189 learned_addr learned my addr 172.21.15.33:0/3111447189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d0c50700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc102e90 0x7f88cc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1c52700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88cc107570 0x7f88cc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 -- 172.21.15.33:0/3111447189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc102e90 msgr2=0x7f88cc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc102e90 0x7f88cc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 -- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88cc107570 msgr2=0x7f88cc1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88cc107570 0x7f88cc1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.709+0000 7f88d1451700 1 -- 172.21.15.33:0/3111447189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc199020 con 0x7f88cc1024e0 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88d1c52700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88cc107570 0x7f88cc1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88d1451700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc1024e0 0x7f88cc19e6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88c800d980 tx=0x7f88c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8017070 con 0x7f88cc1024e0 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc199310 con 0x7f88cc1024e0 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c8005020 con 0x7f88cc1024e0 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8009bc0 con 0x7f88cc1024e0 2024-04-24T23:11:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.710+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1a5580 con 0x7f88cc1024e0 2024-04-24T23:11:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.711+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88cc062940 con 0x7f88cc1024e0 2024-04-24T23:11:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.712+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f88c80051d0 con 0x7f88cc1024e0 2024-04-24T23:11:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.713+0000 7f88c27fc700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88b80a2cb0 0x7f88b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.713+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f88c80cf460 con 0x7f88cc1024e0 2024-04-24T23:11:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.713+0000 7f88d0c50700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88b80a2cb0 0x7f88b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.713+0000 7f88c27fc700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] conn(0x7f88b80ade10 0x7f88b80b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.713+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 --> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88c8022e50 con 0x7f88b80ade10 2024-04-24T23:11:19.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.713+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f88c80d6050 con 0x7f88cc1024e0 2024-04-24T23:11:19.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.714+0000 7f88d0c50700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88b80a2cb0 0x7f88b80a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88cc108aa0 tx=0x7f88bc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.714+0000 7f88d1c52700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] conn(0x7f88b80ade10 0x7f88b80b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.714+0000 7f88d1c52700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] conn(0x7f88b80ade10 0x7f88b80b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.715+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== osd.1 v2:172.21.15.38:6800/2106722793 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88c8022e50 con 0x7f88b80ade10 2024-04-24T23:11:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.734+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 --> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88cc062d70 con 0x7f88b80ade10 2024-04-24T23:11:19.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:19 smithi033 ceph-mon[29201]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 117 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:11:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88c27fc700 1 -- 172.21.15.33:0/3111447189 <== osd.1 v2:172.21.15.38:6800/2106722793 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f88cc062d70 con 0x7f88b80ade10 2024-04-24T23:11:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] conn(0x7f88b80ade10 msgr2=0x7f88b80b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] conn(0x7f88b80ade10 0x7f88b80b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88b80a2cb0 msgr2=0x7f88b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88b80a2cb0 0x7f88b80a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88cc108aa0 tx=0x7f88bc00b410 comp rx=0 tx=0).stop 2024-04-24T23:11:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc1024e0 msgr2=0x7f88cc19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 --2- 172.21.15.33:0/3111447189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88cc1024e0 0x7f88cc19e6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88c800d980 tx=0x7f88c800dc90 comp rx=0 tx=0).stop 2024-04-24T23:11:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d1c52700 1 -- 172.21.15.33:0/3111447189 reap_dead start 2024-04-24T23:11:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 shutdown_connections 2024-04-24T23:11:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 >> 172.21.15.33:0/3111447189 conn(0x7f88cc079390 msgr2=0x7f88cc100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.735+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 shutdown_connections 2024-04-24T23:11:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:19.736+0000 7f88d36b5700 1 -- 172.21.15.33:0/3111447189 wait complete. 2024-04-24T23:11:19.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:19 smithi038 ceph-mon[32450]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 117 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:11:19.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:19 smithi117 ceph-mon[32529]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 117 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:11:20.688 INFO:teuthology.orchestra.run.smithi033.stdout:55834574925 2024-04-24T23:11:20.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T23:11:21.326 INFO:teuthology.orchestra.run.smithi033.stdout:347892351003 2024-04-24T23:11:21.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T23:11:21.767 INFO:teuthology.orchestra.run.smithi033.stdout:201863462964 2024-04-24T23:11:21.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T23:11:21.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:21 smithi033 ceph-mon[29201]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 99 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:21 smithi038 ceph-mon[32450]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 99 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:21.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:21 smithi117 ceph-mon[32529]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 99 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 -- 172.21.15.33:0/2873647822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 msgr2=0x7f95b8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 --2- 172.21.15.33:0/2873647822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b8107580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009990 tx=0x7f95a8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 -- 172.21.15.33:0/2873647822 shutdown_connections 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 --2- 172.21.15.33:0/2873647822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b8101af0 0x7f95b810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 --2- 172.21.15.33:0/2873647822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b8107580 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 --2- 172.21.15.33:0/2873647822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95b8101130 0x7f95b81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 -- 172.21.15.33:0/2873647822 >> 172.21.15.33:0/2873647822 conn(0x7f95b80780e0 msgr2=0x7f95b80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 -- 172.21.15.33:0/2873647822 shutdown_connections 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 -- 172.21.15.33:0/2873647822 wait complete. 2024-04-24T23:11:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.968+0000 7f95be931700 1 Processor -- start 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 -- start start 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95b8101130 0x7f95b819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b8101af0 0x7f95b819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8111410 con 0x7f95b81071a0 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8111290 con 0x7f95b8101130 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95be931700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8111590 con 0x7f95b8101af0 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b81987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37752/0 (socket says 172.21.15.33:37752) 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 -- 172.21.15.33:0/323302659 learned_addr learned my addr 172.21.15.33:0/323302659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95b7fff700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95b8101130 0x7f95b819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95b77fe700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b8101af0 0x7f95b819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 -- 172.21.15.33:0/323302659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b8101af0 msgr2=0x7f95b819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b8101af0 0x7f95b819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 -- 172.21.15.33:0/323302659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95b8101130 msgr2=0x7f95b819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95b8101130 0x7f95b819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95bcece700 1 -- 172.21.15.33:0/323302659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b8198f90 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.969+0000 7f95b77fe700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b8101af0 0x7f95b819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95bcece700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b81987b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f95b0007ee0 tx=0x7f95b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b000dbd0 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b8199280 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b000dd70 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b00133f0 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b81a5490 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.970+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f95b81062b0 con 0x7f95b81071a0 2024-04-24T23:11:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.972+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f95b001a020 con 0x7f95b81071a0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.972+0000 7f95b57fa700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95a00a2cb0 0x7f95a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f95b00cf300 con 0x7f95b81071a0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b7fff700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95a00a2cb0 0x7f95a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b57fa700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] conn(0x7f95a00ade10 0x7f95a00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 --> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95b001de50 con 0x7f95a00ade10 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f95b00cf7e0 con 0x7f95b81071a0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b77fe700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] conn(0x7f95a00ade10 0x7f95a00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b7fff700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95a00a2cb0 0x7f95a00a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95a8006010 tx=0x7f95a800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.973+0000 7f95b77fe700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] conn(0x7f95a00ade10 0x7f95a00b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.974+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== osd.3 v2:172.21.15.33:6810/1011423676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95b001de50 con 0x7f95a00ade10 2024-04-24T23:11:21.983 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.994+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 --> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95b8102850 con 0x7f95a00ade10 2024-04-24T23:11:21.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.994+0000 7f95b57fa700 1 -- 172.21.15.33:0/323302659 <== osd.3 v2:172.21.15.33:6810/1011423676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f95b8102850 con 0x7f95a00ade10 2024-04-24T23:11:21.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.995+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] conn(0x7f95a00ade10 msgr2=0x7f95a00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.995+0000 7f95be931700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] conn(0x7f95a00ade10 0x7f95a00b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.995+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95a00a2cb0 msgr2=0x7f95a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.995+0000 7f95be931700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95a00a2cb0 0x7f95a00a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95a8006010 tx=0x7f95a800b540 comp rx=0 tx=0).stop 2024-04-24T23:11:21.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.995+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 msgr2=0x7f95b81987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.995+0000 7f95be931700 1 --2- 172.21.15.33:0/323302659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95b81071a0 0x7f95b81987b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f95b0007ee0 tx=0x7f95b000d3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.996+0000 7f95bcece700 1 -- 172.21.15.33:0/323302659 reap_dead start 2024-04-24T23:11:21.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.996+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 shutdown_connections 2024-04-24T23:11:21.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.996+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 >> 172.21.15.33:0/323302659 conn(0x7f95b80780e0 msgr2=0x7f95b806a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:21.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.996+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 shutdown_connections 2024-04-24T23:11:21.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:21.996+0000 7f95be931700 1 -- 172.21.15.33:0/323302659 wait complete. 2024-04-24T23:11:22.680 INFO:teuthology.orchestra.run.smithi033.stdout:111669149764 2024-04-24T23:11:22.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T23:11:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.402 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:23 smithi033 ceph-mon[29201]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 51 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:23 smithi038 ceph-mon[32450]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 51 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:23 smithi117 ceph-mon[32529]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 51 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:25.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:25 smithi033 ceph-mon[29201]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 51 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:25 smithi038 ceph-mon[32450]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 51 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:25 smithi117 ceph-mon[32529]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 51 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T23:11:27.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:27 smithi033 ceph-mon[29201]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:27.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:27 smithi038 ceph-mon[32450]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:27 smithi117 ceph-mon[32529]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:29.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:29 smithi033 ceph-mon[29201]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 -- 172.21.15.33:0/3552989429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160102ec0 msgr2=0x7f9160106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 --2- 172.21.15.33:0/3552989429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160102ec0 0x7f9160106f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91480099c0 tx=0x7f9148009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 -- 172.21.15.33:0/3552989429 shutdown_connections 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 --2- 172.21.15.33:0/3552989429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91601075e0 0x7f916010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 --2- 172.21.15.33:0/3552989429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 0x7f91601028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 --2- 172.21.15.33:0/3552989429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160102ec0 0x7f9160106f00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 -- 172.21.15.33:0/3552989429 >> 172.21.15.33:0/3552989429 conn(0x7f91600793e0 msgr2=0x7f91600797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 -- 172.21.15.33:0/3552989429 shutdown_connections 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.856+0000 7f9166205700 1 -- 172.21.15.33:0/3552989429 wait complete. 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.857+0000 7f9166205700 1 Processor -- start 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.857+0000 7f9166205700 1 -- start start 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f9166205700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 0x7f916019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f9166205700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9160102ec0 0x7f916019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f9166205700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 0x7f9160198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f9166205700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91601125f0 con 0x7f91601075e0 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f9166205700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112470 con 0x7f9160102510 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f9166205700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112770 con 0x7f9160102ec0 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 0x7f9160198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 0x7f9160198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42094/0 (socket says 172.21.15.33:42094) 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 -- 172.21.15.33:0/355208792 learned_addr learned my addr 172.21.15.33:0/355208792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915f7fe700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 0x7f916019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915effd700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9160102ec0 0x7f916019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 -- 172.21.15.33:0/355208792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9160102ec0 msgr2=0x7f916019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9160102ec0 0x7f916019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 -- 172.21.15.33:0/355208792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 msgr2=0x7f916019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 0x7f916019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915ffff700 1 -- 172.21.15.33:0/355208792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9160199070 con 0x7f91601075e0 2024-04-24T23:11:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.858+0000 7f915f7fe700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 0x7f916019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:29.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.859+0000 7f915ffff700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 0x7f9160198910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f915800d980 tx=0x7f915800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:29.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.859+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158017070 con 0x7f91601075e0 2024-04-24T23:11:29.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.859+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9160199360 con 0x7f91601075e0 2024-04-24T23:11:29.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.859+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9158005020 con 0x7f91601075e0 2024-04-24T23:11:29.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.859+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158009b50 con 0x7f91601075e0 2024-04-24T23:11:29.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.859+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91601a55d0 con 0x7f91601075e0 2024-04-24T23:11:29.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.860+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9160068f20 con 0x7f91601075e0 2024-04-24T23:11:29.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.861+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9158009d10 con 0x7f91601075e0 2024-04-24T23:11:29.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.862+0000 7f915cff9700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f914c0a2cb0 0x7f914c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:29.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.862+0000 7f915f7fe700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f914c0a2cb0 0x7f914c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:29.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.862+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f91580cf390 con 0x7f91601075e0 2024-04-24T23:11:29.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.862+0000 7f915f7fe700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f914c0a2cb0 0x7f914c0a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9150007fd0 tx=0x7f9150007830 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:29.867+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9158020020 con 0x7f91601075e0 2024-04-24T23:11:29.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:29 smithi038 ceph-mon[32450]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:29.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:29 smithi117 ceph-mon[32529]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.087+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f916002db30 con 0x7f91601075e0 2024-04-24T23:11:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.087+0000 7f915cff9700 1 -- 172.21.15.33:0/355208792 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f915809bdd0 con 0x7f91601075e0 2024-04-24T23:11:30.090 INFO:teuthology.orchestra.run.smithi033.stdout:34359738452 2024-04-24T23:11:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.089+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f914c0a2cb0 msgr2=0x7f914c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.089+0000 7f9166205700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f914c0a2cb0 0x7f914c0a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9150007fd0 tx=0x7f9150007830 comp rx=0 tx=0).stop 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 msgr2=0x7f9160198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 0x7f9160198910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f915800d980 tx=0x7f915800dc90 comp rx=0 tx=0).stop 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 shutdown_connections 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f914c0a2cb0 0x7f914c0a5170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91601075e0 0x7f9160198910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9160102510 0x7f916019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 --2- 172.21.15.33:0/355208792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9160102ec0 0x7f916019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 >> 172.21.15.33:0/355208792 conn(0x7f91600793e0 msgr2=0x7f9160100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 shutdown_connections 2024-04-24T23:11:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:30.090+0000 7f9166205700 1 -- 172.21.15.33:0/355208792 wait complete. 2024-04-24T23:11:30.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:30 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/355208792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:11:30.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:30 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/355208792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:11:30.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:30 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/355208792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:11:31.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:31 smithi038 ceph-mon[32450]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-24T23:11:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:31 smithi117 ceph-mon[32529]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-24T23:11:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:31 smithi033 ceph-mon[29201]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.987+0000 7f2af7d60700 1 -- 172.21.15.33:0/4220573445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af0102e90 msgr2=0x7f2af0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.987+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4220573445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af0102e90 0x7f2af0106ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009990 tx=0x7f2ae4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 -- 172.21.15.33:0/4220573445 shutdown_connections 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4220573445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2af01075a0 0x7f2af010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4220573445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af01024e0 0x7f2af01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4220573445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af0102e90 0x7f2af0106ec0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 -- 172.21.15.33:0/4220573445 >> 172.21.15.33:0/4220573445 conn(0x7f2af00793e0 msgr2=0x7f2af00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 -- 172.21.15.33:0/4220573445 shutdown_connections 2024-04-24T23:11:32.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 -- 172.21.15.33:0/4220573445 wait complete. 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 Processor -- start 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.988+0000 7f2af7d60700 1 -- start start 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af7d60700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af01024e0 0x7f2af019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af7d60700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2af0102e90 0x7f2af019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af7d60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 0x7f2af0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af7d60700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0112540 con 0x7f2af01075a0 2024-04-24T23:11:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af7d60700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2af01123c0 con 0x7f2af0102e90 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af7d60700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2af01126c0 con 0x7f2af01024e0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 0x7f2af0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 0x7f2af0198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42122/0 (socket says 172.21.15.33:42122) 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 -- 172.21.15.33:0/4272440675 learned_addr learned my addr 172.21.15.33:0/4272440675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af52fb700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2af0102e90 0x7f2af019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af5afc700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af01024e0 0x7f2af019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 -- 172.21.15.33:0/4272440675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af01024e0 msgr2=0x7f2af019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af01024e0 0x7f2af019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 -- 172.21.15.33:0/4272440675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2af0102e90 msgr2=0x7f2af019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2af0102e90 0x7f2af019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 -- 172.21.15.33:0/4272440675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af0198fe0 con 0x7f2af01075a0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.989+0000 7f2af62fd700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 0x7f2af0198800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8007f10 tx=0x7f2ae800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.990+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800dd20 con 0x7f2af01075a0 2024-04-24T23:11:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.990+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8004020 con 0x7f2af01075a0 2024-04-24T23:11:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.990+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8015640 con 0x7f2af01075a0 2024-04-24T23:11:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.990+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af01992d0 con 0x7f2af01075a0 2024-04-24T23:11:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.990+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af01a5490 con 0x7f2af01075a0 2024-04-24T23:11:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.991+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af01039a0 con 0x7f2af01075a0 2024-04-24T23:11:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.992+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2ae80041d0 con 0x7f2af01075a0 2024-04-24T23:11:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.993+0000 7f2ae2ffd700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2adc0a2d30 0x7f2adc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.993+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2ae80cf610 con 0x7f2af01075a0 2024-04-24T23:11:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.993+0000 7f2af5afc700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2adc0a2d30 0x7f2adc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.993+0000 7f2af5afc700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2adc0a2d30 0x7f2adc0a51f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00ac50 tx=0x7f2aec00a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:32.996+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ae8010070 con 0x7f2af01075a0 2024-04-24T23:11:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.208+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2af002db30 con 0x7f2af01075a0 2024-04-24T23:11:33.210 INFO:teuthology.orchestra.run.smithi033.stdout:77309411404 2024-04-24T23:11:33.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.208+0000 7f2ae2ffd700 1 -- 172.21.15.33:0/4272440675 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ae809c0b0 con 0x7f2af01075a0 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2adc0a2d30 msgr2=0x7f2adc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2adc0a2d30 0x7f2adc0a51f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00ac50 tx=0x7f2aec00a380 comp rx=0 tx=0).stop 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 msgr2=0x7f2af0198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 0x7f2af0198800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8007f10 tx=0x7f2ae800d3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 shutdown_connections 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2adc0a2d30 0x7f2adc0a51f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2af01075a0 0x7f2af0198800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af01024e0 0x7f2af019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 --2- 172.21.15.33:0/4272440675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2af0102e90 0x7f2af019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.210+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 >> 172.21.15.33:0/4272440675 conn(0x7f2af00793e0 msgr2=0x7f2af01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.211+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 shutdown_connections 2024-04-24T23:11:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:33.211+0000 7f2af7d60700 1 -- 172.21.15.33:0/4272440675 wait complete. 2024-04-24T23:11:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:33 smithi033 ceph-mon[29201]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-24T23:11:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:33 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4272440675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:11:33.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:33 smithi038 ceph-mon[32450]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-24T23:11:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:33 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4272440675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:11:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:33 smithi117 ceph-mon[32529]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-24T23:11:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:33 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4272440675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:11:34.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-04-24T23:11:34.742 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:35 smithi038 ceph-mon[32450]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:35 smithi117 ceph-mon[32529]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:35 smithi033 ceph-mon[29201]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 -- 172.21.15.33:0/1463360974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 msgr2=0x7f9e841015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 --2- 172.21.15.33:0/1463360974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e841015b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0098f0 tx=0x7f9e6c009c00 comp rx=0 tx=0).stop 2024-04-24T23:11:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 -- 172.21.15.33:0/1463360974 shutdown_connections 2024-04-24T23:11:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 --2- 172.21.15.33:0/1463360974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e84101af0 0x7f9e8410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 --2- 172.21.15.33:0/1463360974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e841071a0 0x7f9e84107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 --2- 172.21.15.33:0/1463360974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e841015b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 -- 172.21.15.33:0/1463360974 >> 172.21.15.33:0/1463360974 conn(0x7f9e840780e0 msgr2=0x7f9e840784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:36.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 -- 172.21.15.33:0/1463360974 shutdown_connections 2024-04-24T23:11:36.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.390+0000 7f9e88b16700 1 -- 172.21.15.33:0/1463360974 wait complete. 2024-04-24T23:11:36.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 Processor -- start 2024-04-24T23:11:36.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 -- start start 2024-04-24T23:11:36.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e84102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e84101af0 0x7f9e84103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 0x7f9e84103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84111410 con 0x7f9e841071a0 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84111290 con 0x7f9e84101130 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e88b16700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84111590 con 0x7f9e84101af0 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e82d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 0x7f9e84103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e82d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 0x7f9e84103890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59986/0 (socket says 172.21.15.33:59986) 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e82d9d700 1 -- 172.21.15.33:0/2056581495 learned_addr learned my addr 172.21.15.33:0/2056581495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.391+0000 7f9e8259c700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e84102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e81d9b700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e84101af0 0x7f9e84103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e8259c700 1 -- 172.21.15.33:0/2056581495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e84101af0 msgr2=0x7f9e84103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e8259c700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e84101af0 0x7f9e84103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e8259c700 1 -- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 msgr2=0x7f9e84103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e8259c700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 0x7f9e84103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e8259c700 1 -- 172.21.15.33:0/2056581495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e841a6f20 con 0x7f9e84101130 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e82d9d700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 0x7f9e84103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e81d9b700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e84101af0 0x7f9e84103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e8259c700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e84102e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009fd0 tx=0x7f9e7400c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400ce40 con 0x7f9e84101130 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e841a71b0 con 0x7f9e84101130 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e74003a00 con 0x7f9e84101130 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e841a7710 con 0x7f9e84101130 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.392+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74007d30 con 0x7f9e84101130 2024-04-24T23:11:36.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.393+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e84068f20 con 0x7f9e84101130 2024-04-24T23:11:36.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.395+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e7401a020 con 0x7f9e84101130 2024-04-24T23:11:36.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.395+0000 7f9e7b7fe700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e700a2d30 0x7f9e700a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.395+0000 7f9e81d9b700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e700a2d30 0x7f9e700a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.395+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9e740cf4f0 con 0x7f9e84101130 2024-04-24T23:11:36.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.396+0000 7f9e81d9b700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e700a2d30 0x7f9e700a51f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009fd0 tx=0x7f9e6c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.399+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e74010040 con 0x7f9e84101130 2024-04-24T23:11:36.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.612+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9e841a7990 con 0x7f9e84101130 2024-04-24T23:11:36.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.613+0000 7f9e7b7fe700 1 -- 172.21.15.33:0/2056581495 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e74009cb0 con 0x7f9e84101130 2024-04-24T23:11:36.615 INFO:teuthology.orchestra.run.smithi033.stdout:167503724604 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e700a2d30 msgr2=0x7f9e700a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e700a2d30 0x7f9e700a51f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009fd0 tx=0x7f9e6c019040 comp rx=0 tx=0).stop 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 msgr2=0x7f9e84102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e84102e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009fd0 tx=0x7f9e7400c620 comp rx=0 tx=0).stop 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 shutdown_connections 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e700a2d30 0x7f9e700a51f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e841071a0 0x7f9e84103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e84101130 0x7f9e84102e10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 --2- 172.21.15.33:0/2056581495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e84101af0 0x7f9e84103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 >> 172.21.15.33:0/2056581495 conn(0x7f9e840780e0 msgr2=0x7f9e84100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 shutdown_connections 2024-04-24T23:11:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:36.615+0000 7f9e88b16700 1 -- 172.21.15.33:0/2056581495 wait complete. 2024-04-24T23:11:36.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2056581495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:11:36.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2056581495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:11:36.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2056581495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 -- 172.21.15.33:0/323397481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 msgr2=0x7f404c108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 --2- 172.21.15.33:0/323397481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c108680 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f403c009990 tx=0x7f403c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 -- 172.21.15.33:0/323397481 shutdown_connections 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 --2- 172.21.15.33:0/323397481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 0x7f404c10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 --2- 172.21.15.33:0/323397481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c108680 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 --2- 172.21.15.33:0/323397481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f404c06a7d0 0x7f404c06ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 -- 172.21.15.33:0/323397481 >> 172.21.15.33:0/323397481 conn(0x7f404c0780a0 msgr2=0x7f404c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 -- 172.21.15.33:0/323397481 shutdown_connections 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.592+0000 7f405458a700 1 -- 172.21.15.33:0/323397481 wait complete. 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 Processor -- start 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 -- start start 2024-04-24T23:11:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f404c06a7d0 0x7f404c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 0x7f404c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1125b0 con 0x7f404c1082a0 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f404c112430 con 0x7f404c06a7d0 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f405458a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f404c112730 con 0x7f404c06b170 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60002/0 (socket says 172.21.15.33:60002) 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 -- 172.21.15.33:0/2254944871 learned_addr learned my addr 172.21.15.33:0/2254944871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052326700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f404c06a7d0 0x7f404c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4051b25700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 0x7f404c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 -- 172.21.15.33:0/2254944871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 msgr2=0x7f404c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 0x7f404c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 -- 172.21.15.33:0/2254944871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f404c06a7d0 msgr2=0x7f404c1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f404c06a7d0 0x7f404c1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4052b27700 1 -- 172.21.15.33:0/2254944871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c1a7070 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.593+0000 7f4051b25700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 0x7f404c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.594+0000 7f4052b27700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c101c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f404400b970 tx=0x7f404400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.594+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044019070 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.594+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4044014440 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.594+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40440075d0 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.594+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1a7360 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.594+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a7850 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.595+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c068f20 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.596+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f404401c070 con 0x7f404c1082a0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.597+0000 7f404b7fe700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40340a2cb0 0x7f40340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.597+0000 7f4052326700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40340a2cb0 0x7f40340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.597+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f40440ce910 con 0x7f404c1082a0 2024-04-24T23:11:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.597+0000 7f4052326700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40340a2cb0 0x7f40340a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f403c009750 tx=0x7f403c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.601+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f404409b460 con 0x7f404c1082a0 2024-04-24T23:11:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.812+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f404c102750 con 0x7f404c1082a0 2024-04-24T23:11:37.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:37 smithi033 ceph-mon[29201]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.812+0000 7f404b7fe700 1 -- 172.21.15.33:0/2254944871 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f404c102750 con 0x7f404c1082a0 2024-04-24T23:11:37.815 INFO:teuthology.orchestra.run.smithi033.stdout:201863462967 2024-04-24T23:11:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40340a2cb0 msgr2=0x7f40340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40340a2cb0 0x7f40340a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f403c009750 tx=0x7f403c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 msgr2=0x7f404c101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c101c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f404400b970 tx=0x7f404400bd30 comp rx=0 tx=0).stop 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 shutdown_connections 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40340a2cb0 0x7f40340a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404c1082a0 0x7f404c101c70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f404c06a7d0 0x7f404c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 --2- 172.21.15.33:0/2254944871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404c06b170 0x7f404c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 >> 172.21.15.33:0/2254944871 conn(0x7f404c0780a0 msgr2=0x7f404c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 shutdown_connections 2024-04-24T23:11:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:37.815+0000 7f405458a700 1 -- 172.21.15.33:0/2254944871 wait complete. 2024-04-24T23:11:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:37 smithi038 ceph-mon[32450]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:37 smithi117 ceph-mon[32529]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2254944871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2254944871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:11:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2254944871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 -- 172.21.15.33:0/2847903340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 msgr2=0x7f13181028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 --2- 172.21.15.33:0/2847903340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f13181028c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f130c009990 tx=0x7f130c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 -- 172.21.15.33:0/2847903340 shutdown_connections 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 --2- 172.21.15.33:0/2847903340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1318107570 0x7f131810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 --2- 172.21.15.33:0/2847903340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f13181028c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 --2- 172.21.15.33:0/2847903340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102e90 0x7f1318106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 -- 172.21.15.33:0/2847903340 >> 172.21.15.33:0/2847903340 conn(0x7f1318079390 msgr2=0x7f13180797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 -- 172.21.15.33:0/2847903340 shutdown_connections 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.387+0000 7f131ef28700 1 -- 172.21.15.33:0/2847903340 wait complete. 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.388+0000 7f131ef28700 1 Processor -- start 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.388+0000 7f131ef28700 1 -- start start 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f131819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102e90 0x7f131819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1318107570 0x7f13181987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112530 con 0x7f13181024e0 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13181123b0 con 0x7f1318107570 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13181126b0 con 0x7f1318102e90 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f131819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f131819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60016/0 (socket says 172.21.15.33:60016) 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 -- 172.21.15.33:0/1050876956 learned_addr learned my addr 172.21.15.33:0/1050876956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131d4c5700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1318107570 0x7f13181987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f1317fff700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102e90 0x7f131819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 -- 172.21.15.33:0/1050876956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102e90 msgr2=0x7f131819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102e90 0x7f131819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 -- 172.21.15.33:0/1050876956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1318107570 msgr2=0x7f13181987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1318107570 0x7f13181987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 -- 172.21.15.33:0/1050876956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1318198fd0 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ccc4700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f131819e670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f130c000c00 tx=0x7f130c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c027070 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c003780 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1318199260 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.389+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c022dd0 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.390+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181a54d0 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.390+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131810a790 con 0x7f13181024e0 2024-04-24T23:11:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.392+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f130c02a070 con 0x7f13181024e0 2024-04-24T23:11:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.392+0000 7f1315ffb700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13080a2cb0 0x7f13080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.392+0000 7f1317fff700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13080a2cb0 0x7f13080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.392+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f130c0d05b0 con 0x7f13181024e0 2024-04-24T23:11:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.393+0000 7f1317fff700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13080a2cb0 0x7f13080a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1304005fd0 tx=0x7f1304005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.396+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f130c013070 con 0x7f13181024e0 2024-04-24T23:11:39.560 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724600 got 167503724604 for osd.5 2024-04-24T23:11:39.560 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:39.615 INFO:teuthology.orchestra.run.smithi033.stdout:111669149767 2024-04-24T23:11:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.613+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1318199d20 con 0x7f13181024e0 2024-04-24T23:11:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.613+0000 7f1315ffb700 1 -- 172.21.15.33:0/1050876956 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f130c09d170 con 0x7f13181024e0 2024-04-24T23:11:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13080a2cb0 msgr2=0x7f13080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13080a2cb0 0x7f13080a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1304005fd0 tx=0x7f1304005f60 comp rx=0 tx=0).stop 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 msgr2=0x7f131819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f131819e670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f130c000c00 tx=0x7f130c0047b0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 shutdown_connections 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13080a2cb0 0x7f13080a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1318107570 0x7f13181987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13181024e0 0x7f131819e670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 --2- 172.21.15.33:0/1050876956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102e90 0x7f131819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 >> 172.21.15.33:0/1050876956 conn(0x7f1318079390 msgr2=0x7f1318101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 shutdown_connections 2024-04-24T23:11:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:39.615+0000 7f131ef28700 1 -- 172.21.15.33:0/1050876956 wait complete. 2024-04-24T23:11:39.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:39 smithi033 ceph-mon[29201]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:39.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:39 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1050876956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:11:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:39 smithi038 ceph-mon[32450]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:39 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1050876956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:11:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:39 smithi117 ceph-mon[32529]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:39 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1050876956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:11:40.847 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462964 got 201863462967 for osd.6 2024-04-24T23:11:40.847 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 -- 172.21.15.33:0/3603446581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 msgr2=0x7f531410d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 --2- 172.21.15.33:0/3603446581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531410d730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c240 tx=0x7f530c00c550 comp rx=0 tx=0).stop 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 -- 172.21.15.33:0/3603446581 shutdown_connections 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 --2- 172.21.15.33:0/3603446581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531410d730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 --2- 172.21.15.33:0/3603446581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53141068d0 0x7f5314106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 --2- 172.21.15.33:0/3603446581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5314100c40 0x7f53141010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.336+0000 7f5318e69700 1 -- 172.21.15.33:0/3603446581 >> 172.21.15.33:0/3603446581 conn(0x7f5314077330 msgr2=0x7f53140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.337+0000 7f5318e69700 1 -- 172.21.15.33:0/3603446581 shutdown_connections 2024-04-24T23:11:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.337+0000 7f5318e69700 1 -- 172.21.15.33:0/3603446581 wait complete. 2024-04-24T23:11:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.337+0000 7f5318e69700 1 Processor -- start 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 -- start start 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5314100c40 0x7f531419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53141068d0 0x7f53141988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53141103d0 con 0x7f5314101600 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5314110250 con 0x7f53141068d0 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5318e69700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5314110550 con 0x7f5314100c40 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60036/0 (socket says 172.21.15.33:60036) 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 -- 172.21.15.33:0/842513250 learned_addr learned my addr 172.21.15.33:0/842513250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f531259c700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5314100c40 0x7f531419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5312d9d700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53141068d0 0x7f53141988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 -- 172.21.15.33:0/842513250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5314100c40 msgr2=0x7f531419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5314100c40 0x7f531419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 -- 172.21.15.33:0/842513250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53141068d0 msgr2=0x7f53141988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53141068d0 0x7f53141988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.338+0000 7f5311d9b700 1 -- 172.21.15.33:0/842513250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5314199000 con 0x7f5314101600 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.339+0000 7f5311d9b700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531419eb80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f530400b530 tx=0x7f530400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.339+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304015020 con 0x7f5314101600 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.339+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53141992f0 con 0x7f5314101600 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.339+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5304011d10 con 0x7f5314101600 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.339+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530400f2e0 con 0x7f5314101600 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.339+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141a5510 con 0x7f5314101600 2024-04-24T23:11:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.340+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5314101fa0 con 0x7f5314101600 2024-04-24T23:11:41.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.341+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5304018020 con 0x7f5314101600 2024-04-24T23:11:41.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.342+0000 7f530b7fe700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f53000a2cb0 0x7f53000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.342+0000 7f531259c700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f53000a2cb0 0x7f53000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.342+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f53040cf090 con 0x7f5314101600 2024-04-24T23:11:41.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.342+0000 7f531259c700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f53000a2cb0 0x7f53000a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005950 tx=0x7f52fc00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.346+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f530401f020 con 0x7f5314101600 2024-04-24T23:11:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.553+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5314199dd0 con 0x7f5314101600 2024-04-24T23:11:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.553+0000 7f530b7fe700 1 -- 172.21.15.33:0/842513250 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f530409bc40 con 0x7f5314101600 2024-04-24T23:11:41.555 INFO:teuthology.orchestra.run.smithi033.stdout:347892351007 2024-04-24T23:11:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.555+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f53000a2cb0 msgr2=0x7f53000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.555+0000 7f5318e69700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f53000a2cb0 0x7f53000a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005950 tx=0x7f52fc00a300 comp rx=0 tx=0).stop 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.555+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 msgr2=0x7f531419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.555+0000 7f5318e69700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531419eb80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f530400b530 tx=0x7f530400b840 comp rx=0 tx=0).stop 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 shutdown_connections 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f53000a2cb0 0x7f53000a5170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53141068d0 0x7f53141988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5314100c40 0x7f531419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 --2- 172.21.15.33:0/842513250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5314101600 0x7f531419eb80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 >> 172.21.15.33:0/842513250 conn(0x7f5314077330 msgr2=0x7f53141036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 shutdown_connections 2024-04-24T23:11:41.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:41.556+0000 7f5318e69700 1 -- 172.21.15.33:0/842513250 wait complete. 2024-04-24T23:11:41.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411404 for osd.2 2024-04-24T23:11:41.749 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:41 smithi038 ceph-mon[32450]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:41 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/842513250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:11:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:41 smithi117 ceph-mon[32529]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:41 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/842513250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:11:41.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:41 smithi033 ceph-mon[29201]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:41.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:41 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/842513250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:11:43.498 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149764 got 111669149767 for osd.3 2024-04-24T23:11:43.498 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 -- 172.21.15.33:0/568467446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 msgr2=0x7f1b54106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 --2- 172.21.15.33:0/568467446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b54106ec0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009990 tx=0x7f1b3c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 -- 172.21.15.33:0/568467446 shutdown_connections 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 --2- 172.21.15.33:0/568467446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b541075a0 0x7f1b5410f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 --2- 172.21.15.33:0/568467446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 0x7f1b54102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 --2- 172.21.15.33:0/568467446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b54106ec0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.711+0000 7f1b58c44700 1 -- 172.21.15.33:0/568467446 >> 172.21.15.33:0/568467446 conn(0x7f1b540792c0 msgr2=0x7f1b540796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.712+0000 7f1b58c44700 1 -- 172.21.15.33:0/568467446 shutdown_connections 2024-04-24T23:11:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.712+0000 7f1b58c44700 1 -- 172.21.15.33:0/568467446 wait complete. 2024-04-24T23:11:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.712+0000 7f1b58c44700 1 Processor -- start 2024-04-24T23:11:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 -- start start 2024-04-24T23:11:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 0x7f1b5419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b5419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b541075a0 0x7f1b54198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54112590 con 0x7f1b54102ef0 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54112410 con 0x7f1b54102540 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b58c44700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54112710 con 0x7f1b541075a0 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b5419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b5419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60044/0 (socket says 172.21.15.33:60044) 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 -- 172.21.15.33:0/1500311559 learned_addr learned my addr 172.21.15.33:0/1500311559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b5259c700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 0x7f1b5419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b52d9d700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b541075a0 0x7f1b54198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 -- 172.21.15.33:0/1500311559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b541075a0 msgr2=0x7f1b54198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b541075a0 0x7f1b54198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 -- 172.21.15.33:0/1500311559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 msgr2=0x7f1b5419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 0x7f1b5419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b51d9b700 1 -- 172.21.15.33:0/1500311559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b54198ff0 con 0x7f1b54102ef0 2024-04-24T23:11:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.713+0000 7f1b5259c700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 0x7f1b5419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.714+0000 7f1b51d9b700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b5419ebd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.714+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c027070 con 0x7f1b54102ef0 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.714+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c003d70 con 0x7f1b54102ef0 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.714+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b54199280 con 0x7f1b54102ef0 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.714+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00f460 con 0x7f1b54102ef0 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.714+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b541a54f0 con 0x7f1b54102ef0 2024-04-24T23:11:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.715+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b541039a0 con 0x7f1b54102ef0 2024-04-24T23:11:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.716+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1b3c02a070 con 0x7f1b54102ef0 2024-04-24T23:11:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.717+0000 7f1b4b7fe700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1b400a2d30 0x7f1b400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.717+0000 7f1b5259c700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1b400a2d30 0x7f1b400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.718+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1b3c0d0c20 con 0x7f1b54102ef0 2024-04-24T23:11:43.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:43 smithi033 ceph-mon[29201]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:11:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.718+0000 7f1b5259c700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1b400a2d30 0x7f1b400a51f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b44011490 tx=0x7f1b44009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.721+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b3c031020 con 0x7f1b54102ef0 2024-04-24T23:11:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:43 smithi038 ceph-mon[32450]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:11:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:43 smithi117 ceph-mon[32529]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:11:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.934+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1b5402db30 con 0x7f1b54102ef0 2024-04-24T23:11:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.935+0000 7f1b4b7fe700 1 -- 172.21.15.33:0/1500311559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b3c013070 con 0x7f1b54102ef0 2024-04-24T23:11:43.936 INFO:teuthology.orchestra.run.smithi033.stdout:223338299444 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1b400a2d30 msgr2=0x7f1b400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1b400a2d30 0x7f1b400a51f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b44011490 tx=0x7f1b44009260 comp rx=0 tx=0).stop 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 msgr2=0x7f1b5419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b5419ebd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c00ba90 comp rx=0 tx=0).stop 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 shutdown_connections 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1b400a2d30 0x7f1b400a51f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b541075a0 0x7f1b54198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54102540 0x7f1b5419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 --2- 172.21.15.33:0/1500311559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b54102ef0 0x7f1b5419ebd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 >> 172.21.15.33:0/1500311559 conn(0x7f1b540792c0 msgr2=0x7f1b541002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 shutdown_connections 2024-04-24T23:11:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:43.937+0000 7f1b58c44700 1 -- 172.21.15.33:0/1500311559 wait complete. 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 -- 172.21.15.33:0/4139618076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 msgr2=0x7f5574108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 --2- 172.21.15.33:0/4139618076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574108690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5564009990 tx=0x7f5564009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 -- 172.21.15.33:0/4139618076 shutdown_connections 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 --2- 172.21.15.33:0/4139618076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557406a8e0 0x7f557410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 --2- 172.21.15.33:0/4139618076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574108690 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 --2- 172.21.15.33:0/4139618076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5574069f40 0x7f557406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 -- 172.21.15.33:0/4139618076 >> 172.21.15.33:0/4139618076 conn(0x7f55740780e0 msgr2=0x7f55740784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 -- 172.21.15.33:0/4139618076 shutdown_connections 2024-04-24T23:11:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 -- 172.21.15.33:0/4139618076 wait complete. 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.372+0000 7f557928f700 1 Processor -- start 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 -- start start 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5574069f40 0x7f5574101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557406a8e0 0x7f5574101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5574112570 con 0x7f55741082b0 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f55741123f0 con 0x7f5574069f40 2024-04-24T23:11:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f557928f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55741126f0 con 0x7f557406a8e0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60060/0 (socket says 172.21.15.33:60060) 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 -- 172.21.15.33:0/3750115722 learned_addr learned my addr 172.21.15.33:0/3750115722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f5572ffd700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5574069f40 0x7f5574101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55727fc700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557406a8e0 0x7f5574101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 -- 172.21.15.33:0/3750115722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557406a8e0 msgr2=0x7f5574101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557406a8e0 0x7f5574101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 -- 172.21.15.33:0/3750115722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5574069f40 msgr2=0x7f5574101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.373+0000 7f55737fe700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5574069f40 0x7f5574101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f55737fe700 1 -- 172.21.15.33:0/3750115722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741a6f90 con 0x7f55741082b0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f55737fe700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574101c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f556c00b550 tx=0x7f556c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c015020 con 0x7f55741082b0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f556c011d10 con 0x7f55741082b0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741a7280 con 0x7f55741082b0 2024-04-24T23:11:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00f2e0 con 0x7f55741082b0 2024-04-24T23:11:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.374+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55741a77c0 con 0x7f55741082b0 2024-04-24T23:11:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.375+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5574068f20 con 0x7f55741082b0 2024-04-24T23:11:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.376+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f556c018020 con 0x7f55741082b0 2024-04-24T23:11:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.377+0000 7f555bfff700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f555c0a2d30 0x7f555c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.377+0000 7f5572ffd700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f555c0a2d30 0x7f555c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.377+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f556c0ceb00 con 0x7f55741082b0 2024-04-24T23:11:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.377+0000 7f5572ffd700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f555c0a2d30 0x7f555c0a51f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f556400b5c0 tx=0x7f5564005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.382+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f556c01f020 con 0x7f55741082b0 2024-04-24T23:11:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.592+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f557402db30 con 0x7f55741082b0 2024-04-24T23:11:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.592+0000 7f555bfff700 1 -- 172.21.15.33:0/3750115722 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f556c09b6b0 con 0x7f55741082b0 2024-04-24T23:11:44.594 INFO:teuthology.orchestra.run.smithi033.stdout:146028888130 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f555c0a2d30 msgr2=0x7f555c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f555c0a2d30 0x7f555c0a51f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f556400b5c0 tx=0x7f5564005960 comp rx=0 tx=0).stop 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 msgr2=0x7f5574101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574101c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f556c00b550 tx=0x7f556c00b860 comp rx=0 tx=0).stop 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 shutdown_connections 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f555c0a2d30 0x7f555c0a51f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55741082b0 0x7f5574101c80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5574069f40 0x7f5574101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 --2- 172.21.15.33:0/3750115722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557406a8e0 0x7f5574101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 >> 172.21.15.33:0/3750115722 conn(0x7f55740780e0 msgr2=0x7f55740ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 shutdown_connections 2024-04-24T23:11:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:44.594+0000 7f557928f700 1 -- 172.21.15.33:0/3750115722 wait complete. 2024-04-24T23:11:44.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:44 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1500311559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:11:44.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:44 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3750115722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:11:44.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:44 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1500311559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:11:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:44 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3750115722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:11:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:44 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1500311559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:11:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:44 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3750115722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:11:45.354 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888124 got 146028888130 for osd.4 2024-04-24T23:11:45.354 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:45.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:45 smithi038 ceph-mon[32450]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:45.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:45 smithi117 ceph-mon[32529]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:45 smithi033 ceph-mon[29201]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:46.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351003 got 347892351007 for osd.11 2024-04-24T23:11:46.042 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:46.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299444 for osd.7 2024-04-24T23:11:46.600 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 -- 172.21.15.33:0/1199108002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 msgr2=0x7f4294112ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 --2- 172.21.15.33:0/1199108002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f4294112ec0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4284009990 tx=0x7f4284009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 -- 172.21.15.33:0/1199108002 shutdown_connections 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 --2- 172.21.15.33:0/1199108002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 0x7f42941157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 --2- 172.21.15.33:0/1199108002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4294074460 0x7f4294074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 --2- 172.21.15.33:0/1199108002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f4294112ec0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.167+0000 7f429b531700 1 -- 172.21.15.33:0/1199108002 >> 172.21.15.33:0/1199108002 conn(0x7f42940793e0 msgr2=0x7f42940797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.168+0000 7f429b531700 1 -- 172.21.15.33:0/1199108002 shutdown_connections 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.168+0000 7f429b531700 1 -- 172.21.15.33:0/1199108002 wait complete. 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.168+0000 7f429b531700 1 Processor -- start 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.168+0000 7f429b531700 1 -- start start 2024-04-24T23:11:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f429b531700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4294074460 0x7f429419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f429b531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f429419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f429b531700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 0x7f42941a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f429b531700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4294118b90 con 0x7f4294074d80 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f429b531700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4294118a10 con 0x7f4294113400 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f429b531700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4294118d10 con 0x7f4294074460 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f429419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f429419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43478/0 (socket says 172.21.15.33:43478) 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 -- 172.21.15.33:0/1951236486 learned_addr learned my addr 172.21.15.33:0/1951236486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4299ace700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 0x7f42941a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f42992cd700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4294074460 0x7f429419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 -- 172.21.15.33:0/1951236486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4294074460 msgr2=0x7f429419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4294074460 0x7f429419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 -- 172.21.15.33:0/1951236486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 msgr2=0x7f42941a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 0x7f42941a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4298acc700 1 -- 172.21.15.33:0/1951236486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42941a3ff0 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.169+0000 7f4299ace700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 0x7f42941a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.170+0000 7f4298acc700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f429419f520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f428400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.170+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284027070 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.170+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4284003d70 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.170+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428400f460 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.170+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42941a4280 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.170+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941abb60 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.171+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4294075b80 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.172+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f428402a070 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.173+0000 7f428a7fc700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42800a2cb0 0x7f42800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.173+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f42840d0690 con 0x7f4294074d80 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.173+0000 7f42992cd700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42800a2cb0 0x7f42800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.174+0000 7f42992cd700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42800a2cb0 0x7f42800a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4290009c60 tx=0x7f4290009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:47.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.177+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4284013070 con 0x7f4294074d80 2024-04-24T23:11:47.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.073+0000 7fd09059b700 1 -- 172.21.15.33:0/582228366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 msgr2=0x7fd08810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.073+0000 7fd09059b700 1 --2- 172.21.15.33:0/582228366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd08810e060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b0d0 tx=0x7fd08000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.073+0000 7fd09059b700 1 -- 172.21.15.33:0/582228366 shutdown_connections 2024-04-24T23:11:47.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 --2- 172.21.15.33:0/582228366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd08810e060 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 --2- 172.21.15.33:0/582228366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd088101770 0x7fd088101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 --2- 172.21.15.33:0/582228366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd088102120 0x7fd088105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 -- 172.21.15.33:0/582228366 >> 172.21.15.33:0/582228366 conn(0x7fd0880780e0 msgr2=0x7fd0880784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 -- 172.21.15.33:0/582228366 shutdown_connections 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 -- 172.21.15.33:0/582228366 wait complete. 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 Processor -- start 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 -- start start 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd088101770 0x7fd088102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd088102120 0x7fd088103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.074+0000 7fd09059b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd0881038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd09059b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd088111480 con 0x7fd0881064c0 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd09059b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd088111300 con 0x7fd088101770 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd09059b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd088111600 con 0x7fd088102120 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd0881038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08e337700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd088101770 0x7fd088102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd0881038b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43464/0 (socket says 172.21.15.33:43464) 2024-04-24T23:11:47.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 -- 172.21.15.33:0/1607258148 learned_addr learned my addr 172.21.15.33:0/1607258148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:47.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08db36700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd088102120 0x7fd088103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.314 INFO:teuthology.orchestra.run.smithi033.stdout:292057776170 2024-04-24T23:11:47.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 -- 172.21.15.33:0/1607258148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd088102120 msgr2=0x7fd088103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd088102120 0x7fd088103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 -- 172.21.15.33:0/1607258148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd088101770 msgr2=0x7fd088102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd088101770 0x7fd088102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 -- 172.21.15.33:0/1607258148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0881a6ef0 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd08eb38700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd0881038b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd080006070 tx=0x7fd080007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080003c00 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0881a7180 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd080003da0 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08001b960 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.075+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0881a7630 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.077+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd088102870 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.078+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd080005670 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.079+0000 7fd07f7fe700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0740a2cb0 0x7fd0740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.079+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd08000f020 con 0x7fd0881064c0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.079+0000 7fd08e337700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0740a2cb0 0x7fd0740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.080+0000 7fd08e337700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0740a2cb0 0x7fd0740a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd084009cb0 tx=0x7fd084009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:47.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.083+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd08002b020 con 0x7fd0881064c0 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.298+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd08802db30 con 0x7fd0881064c0 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.298+0000 7fd07f7fe700 1 -- 172.21.15.33:0/1607258148 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd08009d340 con 0x7fd0881064c0 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0740a2cb0 msgr2=0x7fd0740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0740a2cb0 0x7fd0740a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd084009cb0 tx=0x7fd084009260 comp rx=0 tx=0).stop 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 msgr2=0x7fd0881038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd0881038b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd080006070 tx=0x7fd080007a20 comp rx=0 tx=0).stop 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 shutdown_connections 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0740a2cb0 0x7fd0740a5170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0881064c0 0x7fd0881038b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd088101770 0x7fd088102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 --2- 172.21.15.33:0/1607258148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd088102120 0x7fd088103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 >> 172.21.15.33:0/1607258148 conn(0x7fd0880780e0 msgr2=0x7fd08806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.300+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 shutdown_connections 2024-04-24T23:11:47.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.301+0000 7fd09059b700 1 -- 172.21.15.33:0/1607258148 wait complete. 2024-04-24T23:11:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.396+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4294068f20 con 0x7f4294074d80 2024-04-24T23:11:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.396+0000 7f428a7fc700 1 -- 172.21.15.33:0/1951236486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4294068f20 con 0x7f4294074d80 2024-04-24T23:11:47.398 INFO:teuthology.orchestra.run.smithi033.stdout:55834574930 2024-04-24T23:11:47.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42800a2cb0 msgr2=0x7f42800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42800a2cb0 0x7f42800a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4290009c60 tx=0x7f4290009260 comp rx=0 tx=0).stop 2024-04-24T23:11:47.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 msgr2=0x7f429419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f429419f520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f428400bab0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 shutdown_connections 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42800a2cb0 0x7f42800a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4294113400 0x7f42941a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4294074460 0x7f429419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 --2- 172.21.15.33:0/1951236486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4294074d80 0x7f429419f520 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 >> 172.21.15.33:0/1951236486 conn(0x7f42940793e0 msgr2=0x7f42941121c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 shutdown_connections 2024-04-24T23:11:47.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:47.398+0000 7f429b531700 1 -- 172.21.15.33:0/1951236486 wait complete. 2024-04-24T23:11:47.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:47 smithi033 ceph-mon[29201]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:47.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:47 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1607258148' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:11:47.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:47 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1951236486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:11:47.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:47 smithi038 ceph-mon[32450]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:47 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1607258148' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:11:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:47 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1951236486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:11:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:47 smithi117 ceph-mon[32529]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:47 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1607258148' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:11:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:47 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1951236486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:11:48.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574930 for osd.1 2024-04-24T23:11:48.203 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:48.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 -- 172.21.15.33:0/3149807219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac06a8e0 msgr2=0x7f3cac10f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/3149807219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac06a8e0 0x7f3cac10f0f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400b0a0 tx=0x7f3ca400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 -- 172.21.15.33:0/3149807219 shutdown_connections 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/3149807219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac06a8e0 0x7f3cac10f0f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/3149807219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac1082b0 0x7f3cac108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/3149807219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cac069f40 0x7f3cac06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 -- 172.21.15.33:0/3149807219 >> 172.21.15.33:0/3149807219 conn(0x7f3cac0780e0 msgr2=0x7f3cac0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 -- 172.21.15.33:0/3149807219 shutdown_connections 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.502+0000 7f3cb21d9700 1 -- 172.21.15.33:0/3149807219 wait complete. 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 Processor -- start 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 -- start start 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac069f40 0x7f3cac101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cac06a8e0 0x7f3cac101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 0x7f3cac101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac112500 con 0x7f3cac1082b0 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac112380 con 0x7f3cac069f40 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cb21d9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac112680 con 0x7f3cac06a8e0 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.503+0000 7f3cabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 0x7f3cac101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 0x7f3cac101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43504/0 (socket says 172.21.15.33:43504) 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 -- 172.21.15.33:0/1164613244 learned_addr learned my addr 172.21.15.33:0/1164613244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3caaffd700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cac06a8e0 0x7f3cac101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cab7fe700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac069f40 0x7f3cac101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 -- 172.21.15.33:0/1164613244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cac06a8e0 msgr2=0x7f3cac101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cac06a8e0 0x7f3cac101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 -- 172.21.15.33:0/1164613244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac069f40 msgr2=0x7f3cac101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac069f40 0x7f3cac101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 -- 172.21.15.33:0/1164613244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac1a6f80 con 0x7f3cac1082b0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cab7fe700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac069f40 0x7f3cac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.504+0000 7f3cabfff700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 0x7f3cac101c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400b780 tx=0x7f3ca4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.505+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4021070 con 0x7f3cac1082b0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.505+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca4007b80 con 0x7f3cac1082b0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.505+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4005370 con 0x7f3cac1082b0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.505+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac1a7210 con 0x7f3cac1082b0 2024-04-24T23:11:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.505+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1a7710 con 0x7f3cac1082b0 2024-04-24T23:11:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.506+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac068f20 con 0x7f3cac1082b0 2024-04-24T23:11:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.508+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ca4024070 con 0x7f3cac1082b0 2024-04-24T23:11:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.509+0000 7f3ca8ff9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3c940a2d30 0x7f3c940a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.509+0000 7f3cab7fe700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3c940a2d30 0x7f3c940a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.509+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3ca400f020 con 0x7f3cac1082b0 2024-04-24T23:11:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.510+0000 7f3cab7fe700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3c940a2d30 0x7f3c940a51f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009c90 tx=0x7f3c9c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.513+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ca402b020 con 0x7f3cac1082b0 2024-04-24T23:11:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.720+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3cac02db30 con 0x7f3cac1082b0 2024-04-24T23:11:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.720+0000 7f3ca8ff9700 1 -- 172.21.15.33:0/1164613244 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3ca409d910 con 0x7f3cac1082b0 2024-04-24T23:11:48.722 INFO:teuthology.orchestra.run.smithi033.stdout:313532612646 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3c940a2d30 msgr2=0x7f3c940a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3c940a2d30 0x7f3c940a51f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009c90 tx=0x7f3c9c009260 comp rx=0 tx=0).stop 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 msgr2=0x7f3cac101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 0x7f3cac101c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400b780 tx=0x7f3ca4004d00 comp rx=0 tx=0).stop 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 shutdown_connections 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3c940a2d30 0x7f3c940a51f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cac1082b0 0x7f3cac101c80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac069f40 0x7f3cac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 --2- 172.21.15.33:0/1164613244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cac06a8e0 0x7f3cac101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.722+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 >> 172.21.15.33:0/1164613244 conn(0x7f3cac0780e0 msgr2=0x7f3cac0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.723+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 shutdown_connections 2024-04-24T23:11:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:48.723+0000 7f3cb21d9700 1 -- 172.21.15.33:0/1164613244 wait complete. 2024-04-24T23:11:49.056 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776162 got 292057776170 for osd.9 2024-04-24T23:11:49.056 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 -- 172.21.15.33:0/3260235562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102510 msgr2=0x7f19801028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 --2- 172.21.15.33:0/3260235562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102510 0x7f19801028f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f197c009990 tx=0x7f197c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 -- 172.21.15.33:0/3260235562 shutdown_connections 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 --2- 172.21.15.33:0/3260235562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1980107590 0x7f198010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 --2- 172.21.15.33:0/3260235562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102510 0x7f19801028f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 --2- 172.21.15.33:0/3260235562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1980102ec0 0x7f1980106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 -- 172.21.15.33:0/3260235562 >> 172.21.15.33:0/3260235562 conn(0x7f19800793a0 msgr2=0x7f19800797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 -- 172.21.15.33:0/3260235562 shutdown_connections 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.747+0000 7f1988691700 1 -- 172.21.15.33:0/3260235562 wait complete. 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 Processor -- start 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 -- start start 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1980102510 0x7f1980074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 0x7f1980074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1980107590 0x7f1980074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1980112550 con 0x7f1980102ec0 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19801123d0 con 0x7f1980102510 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1988691700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19801126d0 con 0x7f1980107590 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1985c2c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 0x7f1980074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1985c2c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 0x7f1980074a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43518/0 (socket says 172.21.15.33:43518) 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1985c2c700 1 -- 172.21.15.33:0/1456510848 learned_addr learned my addr 172.21.15.33:0/1456510848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f198642d700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1980102510 0x7f1980074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.748+0000 7f1986c2e700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1980107590 0x7f1980074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1985c2c700 1 -- 172.21.15.33:0/1456510848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1980107590 msgr2=0x7f1980074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1985c2c700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1980107590 0x7f1980074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1985c2c700 1 -- 172.21.15.33:0/1456510848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1980102510 msgr2=0x7f1980074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1985c2c700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1980102510 0x7f1980074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1985c2c700 1 -- 172.21.15.33:0/1456510848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801a6e90 con 0x7f1980102ec0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1985c2c700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 0x7f1980074a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f197000c9e0 tx=0x7f197000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970007aa0 con 0x7f1980102ec0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1970007c40 con 0x7f1980102ec0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801a7180 con 0x7f1980102ec0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970018660 con 0x7f1980102ec0 2024-04-24T23:11:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.749+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a76c0 con 0x7f1980102ec0 2024-04-24T23:11:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.750+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198010a7b0 con 0x7f1980102ec0 2024-04-24T23:11:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.751+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f197000f450 con 0x7f1980102ec0 2024-04-24T23:11:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.752+0000 7f19777fe700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f196c0a2cb0 0x7f196c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.752+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f19700cf530 con 0x7f1980102ec0 2024-04-24T23:11:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.753+0000 7f198642d700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f196c0a2cb0 0x7f196c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.753+0000 7f198642d700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f196c0a2cb0 0x7f196c0a5170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f197c0098e0 tx=0x7f197c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.757+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1970014030 con 0x7f1980102ec0 2024-04-24T23:11:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:49 smithi038 ceph-mon[32450]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:49 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1164613244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:11:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:49 smithi117 ceph-mon[32529]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:49 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1164613244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:11:49.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:49 smithi033 ceph-mon[29201]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:49.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:49 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1164613244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:11:49.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.976+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f198010a190 con 0x7f1980102ec0 2024-04-24T23:11:49.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.977+0000 7f19777fe700 1 -- 172.21.15.33:0/1456510848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f197009bf70 con 0x7f1980102ec0 2024-04-24T23:11:49.979 INFO:teuthology.orchestra.run.smithi033.stdout:257698037808 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f196c0a2cb0 msgr2=0x7f196c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f196c0a2cb0 0x7f196c0a5170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f197c0098e0 tx=0x7f197c005960 comp rx=0 tx=0).stop 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 msgr2=0x7f1980074a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 0x7f1980074a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f197000c9e0 tx=0x7f197000ccf0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 shutdown_connections 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f196c0a2cb0 0x7f196c0a5170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1980107590 0x7f1980074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1980102510 0x7f1980074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 --2- 172.21.15.33:0/1456510848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980102ec0 0x7f1980074a70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 >> 172.21.15.33:0/1456510848 conn(0x7f19800793a0 msgr2=0x7f1980100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 shutdown_connections 2024-04-24T23:11:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:49.979+0000 7f1988691700 1 -- 172.21.15.33:0/1456510848 wait complete. 2024-04-24T23:11:50.240 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612640 got 313532612646 for osd.10 2024-04-24T23:11:50.241 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:50.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:50 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1456510848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:11:50.924 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037799 got 257698037808 for osd.8 2024-04-24T23:11:50.924 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:50.924 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T23:11:50.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:11:50.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:50 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1456510848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:11:51.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:50 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1456510848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:11:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:51 smithi117 ceph-mon[32529]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:51 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:51 smithi033 ceph-mon[29201]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:51 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:51 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:52.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:51 smithi038 ceph-mon[32450]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:52.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:51 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:52.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:51 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:52.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:52.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:52.849 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 -- 172.21.15.33:0/2431569976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b84100820 msgr2=0x7f8b84100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 --2- 172.21.15.33:0/2431569976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b84100820 0x7f8b84100c80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009990 tx=0x7f8b6c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 -- 172.21.15.33:0/2431569976 shutdown_connections 2024-04-24T23:11:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 --2- 172.21.15.33:0/2431569976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b841011c0 0x7f8b8410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 --2- 172.21.15.33:0/2431569976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b841082c0 0x7f8b841086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 --2- 172.21.15.33:0/2431569976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b84100820 0x7f8b84100c80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 -- 172.21.15.33:0/2431569976 >> 172.21.15.33:0/2431569976 conn(0x7f8b840777a0 msgr2=0x7f8b840feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.851+0000 7f8b89312700 1 -- 172.21.15.33:0/2431569976 shutdown_connections 2024-04-24T23:11:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 -- 172.21.15.33:0/2431569976 wait complete. 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 Processor -- start 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 -- start start 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b84100820 0x7f8b8419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b841011c0 0x7f8b8419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 0x7f8b84198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84112520 con 0x7f8b841082c0 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841123a0 con 0x7f8b84100820 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b89312700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841126a0 con 0x7f8b841011c0 2024-04-24T23:11:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.852+0000 7f8b837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 0x7f8b84198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 0x7f8b84198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43524/0 (socket says 172.21.15.33:43524) 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b7bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b841011c0 0x7f8b8419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 -- 172.21.15.33:0/1907895872 learned_addr learned my addr 172.21.15.33:0/1907895872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b82ffd700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b84100820 0x7f8b8419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 -- 172.21.15.33:0/1907895872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b841011c0 msgr2=0x7f8b8419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b841011c0 0x7f8b8419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 -- 172.21.15.33:0/1907895872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b84100820 msgr2=0x7f8b8419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b84100820 0x7f8b8419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 -- 172.21.15.33:0/1907895872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b84199020 con 0x7f8b841082c0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b837fe700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 0x7f8b84198740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c007d70 tx=0x7f8b7c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c016020 con 0x7f8b841082c0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b7c00ac90 con 0x7f8b841082c0 2024-04-24T23:11:52.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b84199310 con 0x7f8b841082c0 2024-04-24T23:11:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0115c0 con 0x7f8b841082c0 2024-04-24T23:11:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.853+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b841a55f0 con 0x7f8b841082c0 2024-04-24T23:11:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.854+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b84103b40 con 0x7f8b841082c0 2024-04-24T23:11:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.855+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b7c019020 con 0x7f8b841082c0 2024-04-24T23:11:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.856+0000 7f8b80ff9700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b700a2d30 0x7f8b700a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.856+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8b7c0cf310 con 0x7f8b841082c0 2024-04-24T23:11:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.856+0000 7f8b82ffd700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b700a2d30 0x7f8b700a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.856+0000 7f8b82ffd700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b700a2d30 0x7f8b700a51f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8b74007b60 tx=0x7f8b740073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:52.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:52.861+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b7c020020 con 0x7f8b841082c0 2024-04-24T23:11:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.064+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b84062670 con 0x7f8b700a2d30 2024-04-24T23:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.074+0000 7f8b80ff9700 1 -- 172.21.15.33:0/1907895872 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340686 (secure 0 0 0) 0x7f8b84062670 con 0x7f8b700a2d30 2024-04-24T23:11:53.081 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b700a2d30 msgr2=0x7f8b700a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b700a2d30 0x7f8b700a51f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8b74007b60 tx=0x7f8b740073d0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 msgr2=0x7f8b84198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 0x7f8b84198740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c007d70 tx=0x7f8b7c00a340 comp rx=0 tx=0).stop 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 shutdown_connections 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8b700a2d30 0x7f8b700a51f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b841082c0 0x7f8b84198740 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b84100820 0x7f8b8419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 --2- 172.21.15.33:0/1907895872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b841011c0 0x7f8b8419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 >> 172.21.15.33:0/1907895872 conn(0x7f8b840777a0 msgr2=0x7f8b840fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 shutdown_connections 2024-04-24T23:11:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:53.077+0000 7f8b89312700 1 -- 172.21.15.33:0/1907895872 wait complete. 2024-04-24T23:11:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:11:53.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:53.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:53.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:53.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:53.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:53.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:11:53.802 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-24T23:11:51.805419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656378,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":591,"num_read_kb":412,"num_write":342,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181934,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357304,"kb_used_data":12236,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584520,"statfs":{"total":1151940427776,"available":1151574548480,"internally_reserved":0,"allocated":12529664,"data_stored":7991360,"data_compressed":362838,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567156+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:11:07.567156+0000","last_peered":"2024-04-24T23:11:07.567156+0000","last_clean":"2024-04-24T23:11:07.567156+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:11:07.567156+0000","last_undegraded":"2024-04-24T23:11:07.567156+0000","last_fullsized":"2024-04-24T23:11:07.567156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572649+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:11:07.572649+0000","last_peered":"2024-04-24T23:11:07.572649+0000","last_clean":"2024-04-24T23:11:07.572649+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:11:07.572649+0000","last_undegraded":"2024-04-24T23:11:07.572649+0000","last_fullsized":"2024-04-24T23:11:07.572649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572946+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:11:07.572946+0000","last_peered":"2024-04-24T23:11:07.572946+0000","last_clean":"2024-04-24T23:11:07.572946+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:11:07.572946+0000","last_undegraded":"2024-04-24T23:11:07.572946+0000","last_fullsized":"2024-04-24T23:11:07.572946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573213+0000","last_change":"2024-04-24T23:11:07.572557+0000","last_active":"2024-04-24T23:11:07.573213+0000","last_peered":"2024-04-24T23:11:07.573213+0000","last_clean":"2024-04-24T23:11:07.573213+0000","last_became_active":"2024-04-24T23:11:07.572144+0000","last_became_peered":"2024-04-24T23:11:07.572144+0000","last_unstale":"2024-04-24T23:11:07.573213+0000","last_undegraded":"2024-04-24T23:11:07.573213+0000","last_fullsized":"2024-04-24T23:11:07.573213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:42.681992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562275+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:11:07.562275+0000","last_peered":"2024-04-24T23:11:07.562275+0000","last_clean":"2024-04-24T23:11:07.562275+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:11:07.562275+0000","last_undegraded":"2024-04-24T23:11:07.562275+0000","last_fullsized":"2024-04-24T23:11:07.562275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567631+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:11:07.567631+0000","last_peered":"2024-04-24T23:11:07.567631+0000","last_clean":"2024-04-24T23:11:07.567631+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:11:07.567631+0000","last_undegraded":"2024-04-24T23:11:07.567631+0000","last_fullsized":"2024-04-24T23:11:07.567631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565934+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:11:07.565934+0000","last_peered":"2024-04-24T23:11:07.565934+0000","last_clean":"2024-04-24T23:11:07.565934+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:11:07.565934+0000","last_undegraded":"2024-04-24T23:11:07.565934+0000","last_fullsized":"2024-04-24T23:11:07.565934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568885+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:11:07.568885+0000","last_peered":"2024-04-24T23:11:07.568885+0000","last_clean":"2024-04-24T23:11:07.568885+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:11:07.568885+0000","last_undegraded":"2024-04-24T23:11:07.568885+0000","last_fullsized":"2024-04-24T23:11:07.568885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570909+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:11:07.570909+0000","last_peered":"2024-04-24T23:11:07.570909+0000","last_clean":"2024-04-24T23:11:07.570909+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:11:07.570909+0000","last_undegraded":"2024-04-24T23:11:07.570909+0000","last_fullsized":"2024-04-24T23:11:07.570909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567846+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:11:07.567846+0000","last_peered":"2024-04-24T23:11:07.567846+0000","last_clean":"2024-04-24T23:11:07.567846+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:11:07.567846+0000","last_undegraded":"2024-04-24T23:11:07.567846+0000","last_fullsized":"2024-04-24T23:11:07.567846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566596+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:11:07.566596+0000","last_peered":"2024-04-24T23:11:07.566596+0000","last_clean":"2024-04-24T23:11:07.566596+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:11:07.566596+0000","last_undegraded":"2024-04-24T23:11:07.566596+0000","last_fullsized":"2024-04-24T23:11:07.566596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569777+0000","last_change":"2024-04-24T23:11:07.568947+0000","last_active":"2024-04-24T23:11:07.569777+0000","last_peered":"2024-04-24T23:11:07.569777+0000","last_clean":"2024-04-24T23:11:07.569777+0000","last_became_active":"2024-04-24T23:11:07.568528+0000","last_became_peered":"2024-04-24T23:11:07.568528+0000","last_unstale":"2024-04-24T23:11:07.569777+0000","last_undegraded":"2024-04-24T23:11:07.569777+0000","last_fullsized":"2024-04-24T23:11:07.569777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:23.452961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564088+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:11:07.564088+0000","last_peered":"2024-04-24T23:11:07.564088+0000","last_clean":"2024-04-24T23:11:07.564088+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:11:07.564088+0000","last_undegraded":"2024-04-24T23:11:07.564088+0000","last_fullsized":"2024-04-24T23:11:07.564088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573126+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:11:07.573126+0000","last_peered":"2024-04-24T23:11:07.573126+0000","last_clean":"2024-04-24T23:11:07.573126+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:11:07.573126+0000","last_undegraded":"2024-04-24T23:11:07.573126+0000","last_fullsized":"2024-04-24T23:11:07.573126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572678+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:11:07.572678+0000","last_peered":"2024-04-24T23:11:07.572678+0000","last_clean":"2024-04-24T23:11:07.572678+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:11:07.572678+0000","last_undegraded":"2024-04-24T23:11:07.572678+0000","last_fullsized":"2024-04-24T23:11:07.572678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569658+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:11:07.569658+0000","last_peered":"2024-04-24T23:11:07.569658+0000","last_clean":"2024-04-24T23:11:07.569658+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:11:07.569658+0000","last_undegraded":"2024-04-24T23:11:07.569658+0000","last_fullsized":"2024-04-24T23:11:07.569658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567031+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:11:07.567031+0000","last_peered":"2024-04-24T23:11:07.567031+0000","last_clean":"2024-04-24T23:11:07.567031+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:11:07.567031+0000","last_undegraded":"2024-04-24T23:11:07.567031+0000","last_fullsized":"2024-04-24T23:11:07.567031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561761+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:11:07.561761+0000","last_peered":"2024-04-24T23:11:07.561761+0000","last_clean":"2024-04-24T23:11:07.561761+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:11:07.561761+0000","last_undegraded":"2024-04-24T23:11:07.561761+0000","last_fullsized":"2024-04-24T23:11:07.561761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570090+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:11:07.570090+0000","last_peered":"2024-04-24T23:11:07.570090+0000","last_clean":"2024-04-24T23:11:07.570090+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:11:07.570090+0000","last_undegraded":"2024-04-24T23:11:07.570090+0000","last_fullsized":"2024-04-24T23:11:07.570090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568876+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:11:07.568876+0000","last_peered":"2024-04-24T23:11:07.568876+0000","last_clean":"2024-04-24T23:11:07.568876+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:11:07.568876+0000","last_undegraded":"2024-04-24T23:11:07.568876+0000","last_fullsized":"2024-04-24T23:11:07.568876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.769518+0000","last_change":"2024-04-24T23:11:07.769518+0000","last_active":"2024-04-24T23:11:07.769518+0000","last_peered":"2024-04-24T23:11:07.769518+0000","last_clean":"2024-04-24T23:11:07.769518+0000","last_became_active":"2024-04-24T23:11:07.763040+0000","last_became_peered":"2024-04-24T23:11:07.763040+0000","last_unstale":"2024-04-24T23:11:07.769518+0000","last_undegraded":"2024-04-24T23:11:07.769518+0000","last_fullsized":"2024-04-24T23:11:07.769518+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.056568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569338+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:11:07.569338+0000","last_peered":"2024-04-24T23:11:07.569338+0000","last_clean":"2024-04-24T23:11:07.569338+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:11:07.569338+0000","last_undegraded":"2024-04-24T23:11:07.569338+0000","last_fullsized":"2024-04-24T23:11:07.569338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761071+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:11:07.761071+0000","last_peered":"2024-04-24T23:11:07.761071+0000","last_clean":"2024-04-24T23:11:07.761071+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:11:07.761071+0000","last_undegraded":"2024-04-24T23:11:07.761071+0000","last_fullsized":"2024-04-24T23:11:07.761071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569611+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:11:07.569611+0000","last_peered":"2024-04-24T23:11:07.569611+0000","last_clean":"2024-04-24T23:11:07.569611+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:11:07.569611+0000","last_undegraded":"2024-04-24T23:11:07.569611+0000","last_fullsized":"2024-04-24T23:11:07.569611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573786+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:11:07.573786+0000","last_peered":"2024-04-24T23:11:07.573786+0000","last_clean":"2024-04-24T23:11:07.573786+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:11:07.573786+0000","last_undegraded":"2024-04-24T23:11:07.573786+0000","last_fullsized":"2024-04-24T23:11:07.573786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761141+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:11:07.761141+0000","last_peered":"2024-04-24T23:11:07.761141+0000","last_clean":"2024-04-24T23:11:07.761141+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:11:07.761141+0000","last_undegraded":"2024-04-24T23:11:07.761141+0000","last_fullsized":"2024-04-24T23:11:07.761141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.583051+0000","last_change":"2024-04-24T23:11:07.583051+0000","last_active":"2024-04-24T23:11:07.583051+0000","last_peered":"2024-04-24T23:11:07.583051+0000","last_clean":"2024-04-24T23:11:07.583051+0000","last_became_active":"2024-04-24T23:11:07.572502+0000","last_became_peered":"2024-04-24T23:11:07.572502+0000","last_unstale":"2024-04-24T23:11:07.583051+0000","last_undegraded":"2024-04-24T23:11:07.583051+0000","last_fullsized":"2024-04-24T23:11:07.583051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:21.581198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570706+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:11:07.570706+0000","last_peered":"2024-04-24T23:11:07.570706+0000","last_clean":"2024-04-24T23:11:07.570706+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:11:07.570706+0000","last_undegraded":"2024-04-24T23:11:07.570706+0000","last_fullsized":"2024-04-24T23:11:07.570706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761055+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:11:07.761055+0000","last_peered":"2024-04-24T23:11:07.761055+0000","last_clean":"2024-04-24T23:11:07.761055+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:11:07.761055+0000","last_undegraded":"2024-04-24T23:11:07.761055+0000","last_fullsized":"2024-04-24T23:11:07.761055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562795+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:11:07.562795+0000","last_peered":"2024-04-24T23:11:07.562795+0000","last_clean":"2024-04-24T23:11:07.562795+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:11:07.562795+0000","last_undegraded":"2024-04-24T23:11:07.562795+0000","last_fullsized":"2024-04-24T23:11:07.562795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565967+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:11:07.565967+0000","last_peered":"2024-04-24T23:11:07.565967+0000","last_clean":"2024-04-24T23:11:07.565967+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:11:07.565967+0000","last_undegraded":"2024-04-24T23:11:07.565967+0000","last_fullsized":"2024-04-24T23:11:07.565967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569772+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:11:07.569772+0000","last_peered":"2024-04-24T23:11:07.569772+0000","last_clean":"2024-04-24T23:11:07.569772+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:11:07.569772+0000","last_undegraded":"2024-04-24T23:11:07.569772+0000","last_fullsized":"2024-04-24T23:11:07.569772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:48.606666+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:11:48.606666+0000","last_peered":"2024-04-24T23:11:48.606666+0000","last_clean":"2024-04-24T23:11:48.606666+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:11:48.606666+0000","last_undegraded":"2024-04-24T23:11:48.606666+0000","last_fullsized":"2024-04-24T23:11:48.606666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562252+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:11:07.562252+0000","last_peered":"2024-04-24T23:11:07.562252+0000","last_clean":"2024-04-24T23:11:07.562252+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:11:07.562252+0000","last_undegraded":"2024-04-24T23:11:07.562252+0000","last_fullsized":"2024-04-24T23:11:07.562252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569464+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:11:07.569464+0000","last_peered":"2024-04-24T23:11:07.569464+0000","last_clean":"2024-04-24T23:11:07.569464+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:11:07.569464+0000","last_undegraded":"2024-04-24T23:11:07.569464+0000","last_fullsized":"2024-04-24T23:11:07.569464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567502+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:11:07.567502+0000","last_peered":"2024-04-24T23:11:07.567502+0000","last_clean":"2024-04-24T23:11:07.567502+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:11:07.567502+0000","last_undegraded":"2024-04-24T23:11:07.567502+0000","last_fullsized":"2024-04-24T23:11:07.567502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565397+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:11:07.565397+0000","last_peered":"2024-04-24T23:11:07.565397+0000","last_clean":"2024-04-24T23:11:07.565397+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:11:07.565397+0000","last_undegraded":"2024-04-24T23:11:07.565397+0000","last_fullsized":"2024-04-24T23:11:07.565397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566861+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:11:07.566861+0000","last_peered":"2024-04-24T23:11:07.566861+0000","last_clean":"2024-04-24T23:11:07.566861+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:11:07.566861+0000","last_undegraded":"2024-04-24T23:11:07.566861+0000","last_fullsized":"2024-04-24T23:11:07.566861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563169+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:11:07.563169+0000","last_peered":"2024-04-24T23:11:07.563169+0000","last_clean":"2024-04-24T23:11:07.563169+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:11:07.563169+0000","last_undegraded":"2024-04-24T23:11:07.563169+0000","last_fullsized":"2024-04-24T23:11:07.563169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568782+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:11:07.568782+0000","last_peered":"2024-04-24T23:11:07.568782+0000","last_clean":"2024-04-24T23:11:07.568782+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:11:07.568782+0000","last_undegraded":"2024-04-24T23:11:07.568782+0000","last_fullsized":"2024-04-24T23:11:07.568782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572537+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:11:07.572537+0000","last_peered":"2024-04-24T23:11:07.572537+0000","last_clean":"2024-04-24T23:11:07.572537+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:11:07.572537+0000","last_undegraded":"2024-04-24T23:11:07.572537+0000","last_fullsized":"2024-04-24T23:11:07.572537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564005+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:11:07.564005+0000","last_peered":"2024-04-24T23:11:07.564005+0000","last_clean":"2024-04-24T23:11:07.564005+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:11:07.564005+0000","last_undegraded":"2024-04-24T23:11:07.564005+0000","last_fullsized":"2024-04-24T23:11:07.564005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.560882+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:11:07.560882+0000","last_peered":"2024-04-24T23:11:07.560882+0000","last_clean":"2024-04-24T23:11:07.560882+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:11:07.560882+0000","last_undegraded":"2024-04-24T23:11:07.560882+0000","last_fullsized":"2024-04-24T23:11:07.560882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568846+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:11:07.568846+0000","last_peered":"2024-04-24T23:11:07.568846+0000","last_clean":"2024-04-24T23:11:07.568846+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:11:07.568846+0000","last_undegraded":"2024-04-24T23:11:07.568846+0000","last_fullsized":"2024-04-24T23:11:07.568846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:50.609220+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:11:50.609220+0000","last_peered":"2024-04-24T23:11:50.609220+0000","last_clean":"2024-04-24T23:11:50.609220+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:11:50.609220+0000","last_undegraded":"2024-04-24T23:11:50.609220+0000","last_fullsized":"2024-04-24T23:11:50.609220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570643+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:11:07.570643+0000","last_peered":"2024-04-24T23:11:07.570643+0000","last_clean":"2024-04-24T23:11:07.570643+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:11:07.570643+0000","last_undegraded":"2024-04-24T23:11:07.570643+0000","last_fullsized":"2024-04-24T23:11:07.570643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.560796+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:11:07.560796+0000","last_peered":"2024-04-24T23:11:07.560796+0000","last_clean":"2024-04-24T23:11:07.560796+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:11:07.560796+0000","last_undegraded":"2024-04-24T23:11:07.560796+0000","last_fullsized":"2024-04-24T23:11:07.560796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566074+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:11:07.566074+0000","last_peered":"2024-04-24T23:11:07.566074+0000","last_clean":"2024-04-24T23:11:07.566074+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:11:07.566074+0000","last_undegraded":"2024-04-24T23:11:07.566074+0000","last_fullsized":"2024-04-24T23:11:07.566074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573966+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:11:07.573966+0000","last_peered":"2024-04-24T23:11:07.573966+0000","last_clean":"2024-04-24T23:11:07.573966+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:11:07.573966+0000","last_undegraded":"2024-04-24T23:11:07.573966+0000","last_fullsized":"2024-04-24T23:11:07.573966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761375+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:11:07.761375+0000","last_peered":"2024-04-24T23:11:07.761375+0000","last_clean":"2024-04-24T23:11:07.761375+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:11:07.761375+0000","last_undegraded":"2024-04-24T23:11:07.761375+0000","last_fullsized":"2024-04-24T23:11:07.761375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565827+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:11:07.565827+0000","last_peered":"2024-04-24T23:11:07.565827+0000","last_clean":"2024-04-24T23:11:07.565827+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:11:07.565827+0000","last_undegraded":"2024-04-24T23:11:07.565827+0000","last_fullsized":"2024-04-24T23:11:07.565827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570770+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:11:07.570770+0000","last_peered":"2024-04-24T23:11:07.570770+0000","last_clean":"2024-04-24T23:11:07.570770+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:11:07.570770+0000","last_undegraded":"2024-04-24T23:11:07.570770+0000","last_fullsized":"2024-04-24T23:11:07.570770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.560932+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:11:07.560932+0000","last_peered":"2024-04-24T23:11:07.560932+0000","last_clean":"2024-04-24T23:11:07.560932+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:11:07.560932+0000","last_undegraded":"2024-04-24T23:11:07.560932+0000","last_fullsized":"2024-04-24T23:11:07.560932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":213,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:51.593992+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:11:51.593992+0000","last_peered":"2024-04-24T23:11:51.593992+0000","last_clean":"2024-04-24T23:11:51.593992+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:11:51.593992+0000","last_undegraded":"2024-04-24T23:11:51.593992+0000","last_fullsized":"2024-04-24T23:11:51.593992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562482+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:11:07.562482+0000","last_peered":"2024-04-24T23:11:07.562482+0000","last_clean":"2024-04-24T23:11:07.562482+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:11:07.562482+0000","last_undegraded":"2024-04-24T23:11:07.562482+0000","last_fullsized":"2024-04-24T23:11:07.562482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566374+0000","last_change":"2024-04-24T23:10:16.378770+0000","last_active":"2024-04-24T23:11:07.566374+0000","last_peered":"2024-04-24T23:11:07.566374+0000","last_clean":"2024-04-24T23:11:07.566374+0000","last_became_active":"2024-04-24T23:10:16.377971+0000","last_became_peered":"2024-04-24T23:10:16.377971+0000","last_unstale":"2024-04-24T23:11:07.566374+0000","last_undegraded":"2024-04-24T23:11:07.566374+0000","last_fullsized":"2024-04-24T23:11:07.566374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:18.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.579273+0000","last_change":"2024-04-24T23:11:07.579273+0000","last_active":"2024-04-24T23:11:07.579273+0000","last_peered":"2024-04-24T23:11:07.579273+0000","last_clean":"2024-04-24T23:11:07.579273+0000","last_became_active":"2024-04-24T23:11:07.570480+0000","last_became_peered":"2024-04-24T23:11:07.570480+0000","last_unstale":"2024-04-24T23:11:07.579273+0000","last_undegraded":"2024-04-24T23:11:07.579273+0000","last_fullsized":"2024-04-24T23:11:07.579273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:21.291453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563748+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:11:07.563748+0000","last_peered":"2024-04-24T23:11:07.563748+0000","last_clean":"2024-04-24T23:11:07.563748+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:11:07.563748+0000","last_undegraded":"2024-04-24T23:11:07.563748+0000","last_fullsized":"2024-04-24T23:11:07.563748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566285+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:11:07.566285+0000","last_peered":"2024-04-24T23:11:07.566285+0000","last_clean":"2024-04-24T23:11:07.566285+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:11:07.566285+0000","last_undegraded":"2024-04-24T23:11:07.566285+0000","last_fullsized":"2024-04-24T23:11:07.566285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564922+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:11:07.564922+0000","last_peered":"2024-04-24T23:11:07.564922+0000","last_clean":"2024-04-24T23:11:07.564922+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:11:07.564922+0000","last_undegraded":"2024-04-24T23:11:07.564922+0000","last_fullsized":"2024-04-24T23:11:07.564922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566639+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:11:07.566639+0000","last_peered":"2024-04-24T23:11:07.566639+0000","last_clean":"2024-04-24T23:11:07.566639+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:11:07.566639+0000","last_undegraded":"2024-04-24T23:11:07.566639+0000","last_fullsized":"2024-04-24T23:11:07.566639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561502+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:11:07.561502+0000","last_peered":"2024-04-24T23:11:07.561502+0000","last_clean":"2024-04-24T23:11:07.561502+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:11:07.561502+0000","last_undegraded":"2024-04-24T23:11:07.561502+0000","last_fullsized":"2024-04-24T23:11:07.561502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.579195+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:11:49.579195+0000","last_peered":"2024-04-24T23:11:49.579195+0000","last_clean":"2024-04-24T23:11:49.579195+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:11:49.579195+0000","last_undegraded":"2024-04-24T23:11:49.579195+0000","last_fullsized":"2024-04-24T23:11:49.579195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566969+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:11:07.566969+0000","last_peered":"2024-04-24T23:11:07.566969+0000","last_clean":"2024-04-24T23:11:07.566969+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:11:07.566969+0000","last_undegraded":"2024-04-24T23:11:07.566969+0000","last_fullsized":"2024-04-24T23:11:07.566969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570768+0000","last_change":"2024-04-24T23:11:07.570768+0000","last_active":"2024-04-24T23:11:07.570768+0000","last_peered":"2024-04-24T23:11:07.570768+0000","last_clean":"2024-04-24T23:11:07.570768+0000","last_became_active":"2024-04-24T23:11:07.570306+0000","last_became_peered":"2024-04-24T23:11:07.570306+0000","last_unstale":"2024-04-24T23:11:07.570768+0000","last_undegraded":"2024-04-24T23:11:07.570768+0000","last_fullsized":"2024-04-24T23:11:07.570768+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:10.450660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561940+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:11:07.561940+0000","last_peered":"2024-04-24T23:11:07.561940+0000","last_clean":"2024-04-24T23:11:07.561940+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:11:07.561940+0000","last_undegraded":"2024-04-24T23:11:07.561940+0000","last_fullsized":"2024-04-24T23:11:07.561940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.762999+0000","last_change":"2024-04-24T23:11:07.762601+0000","last_active":"2024-04-24T23:11:07.762999+0000","last_peered":"2024-04-24T23:11:07.762999+0000","last_clean":"2024-04-24T23:11:07.762999+0000","last_became_active":"2024-04-24T23:11:07.762305+0000","last_became_peered":"2024-04-24T23:11:07.762305+0000","last_unstale":"2024-04-24T23:11:07.762999+0000","last_undegraded":"2024-04-24T23:11:07.762999+0000","last_fullsized":"2024-04-24T23:11:07.762999+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:36.209046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564598+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:11:07.564598+0000","last_peered":"2024-04-24T23:11:07.564598+0000","last_clean":"2024-04-24T23:11:07.564598+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:11:07.564598+0000","last_undegraded":"2024-04-24T23:11:07.564598+0000","last_fullsized":"2024-04-24T23:11:07.564598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562595+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:11:07.562595+0000","last_peered":"2024-04-24T23:11:07.562595+0000","last_clean":"2024-04-24T23:11:07.562595+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:11:07.562595+0000","last_undegraded":"2024-04-24T23:11:07.562595+0000","last_fullsized":"2024-04-24T23:11:07.562595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569318+0000","last_change":"2024-04-24T23:11:07.569318+0000","last_active":"2024-04-24T23:11:07.569318+0000","last_peered":"2024-04-24T23:11:07.569318+0000","last_clean":"2024-04-24T23:11:07.569318+0000","last_became_active":"2024-04-24T23:11:07.568799+0000","last_became_peered":"2024-04-24T23:11:07.568799+0000","last_unstale":"2024-04-24T23:11:07.569318+0000","last_undegraded":"2024-04-24T23:11:07.569318+0000","last_fullsized":"2024-04-24T23:11:07.569318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:40.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562085+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:11:07.562085+0000","last_peered":"2024-04-24T23:11:07.562085+0000","last_clean":"2024-04-24T23:11:07.562085+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:11:07.562085+0000","last_undegraded":"2024-04-24T23:11:07.562085+0000","last_fullsized":"2024-04-24T23:11:07.562085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561443+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:11:07.561443+0000","last_peered":"2024-04-24T23:11:07.561443+0000","last_clean":"2024-04-24T23:11:07.561443+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:11:07.561443+0000","last_undegraded":"2024-04-24T23:11:07.561443+0000","last_fullsized":"2024-04-24T23:11:07.561443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567387+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:11:07.567387+0000","last_peered":"2024-04-24T23:11:07.567387+0000","last_clean":"2024-04-24T23:11:07.567387+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:11:07.567387+0000","last_undegraded":"2024-04-24T23:11:07.567387+0000","last_fullsized":"2024-04-24T23:11:07.567387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563534+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:11:07.563534+0000","last_peered":"2024-04-24T23:11:07.563534+0000","last_clean":"2024-04-24T23:11:07.563534+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:11:07.563534+0000","last_undegraded":"2024-04-24T23:11:07.563534+0000","last_fullsized":"2024-04-24T23:11:07.563534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565986+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:11:07.565986+0000","last_peered":"2024-04-24T23:11:07.565986+0000","last_clean":"2024-04-24T23:11:07.565986+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:11:07.565986+0000","last_undegraded":"2024-04-24T23:11:07.565986+0000","last_fullsized":"2024-04-24T23:11:07.565986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567244+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:11:07.567244+0000","last_peered":"2024-04-24T23:11:07.567244+0000","last_clean":"2024-04-24T23:11:07.567244+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:11:07.567244+0000","last_undegraded":"2024-04-24T23:11:07.567244+0000","last_fullsized":"2024-04-24T23:11:07.567244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563609+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:11:07.563609+0000","last_peered":"2024-04-24T23:11:07.563609+0000","last_clean":"2024-04-24T23:11:07.563609+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:11:07.563609+0000","last_undegraded":"2024-04-24T23:11:07.563609+0000","last_fullsized":"2024-04-24T23:11:07.563609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564129+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:11:07.564129+0000","last_peered":"2024-04-24T23:11:07.564129+0000","last_clean":"2024-04-24T23:11:07.564129+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:11:07.564129+0000","last_undegraded":"2024-04-24T23:11:07.564129+0000","last_fullsized":"2024-04-24T23:11:07.564129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.578584+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:11:49.578584+0000","last_peered":"2024-04-24T23:11:49.578584+0000","last_clean":"2024-04-24T23:11:49.578584+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:11:49.578584+0000","last_undegraded":"2024-04-24T23:11:49.578584+0000","last_fullsized":"2024-04-24T23:11:49.578584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566234+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:11:07.566234+0000","last_peered":"2024-04-24T23:11:07.566234+0000","last_clean":"2024-04-24T23:11:07.566234+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:11:07.566234+0000","last_undegraded":"2024-04-24T23:11:07.566234+0000","last_fullsized":"2024-04-24T23:11:07.566234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568966+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:11:07.568966+0000","last_peered":"2024-04-24T23:11:07.568966+0000","last_clean":"2024-04-24T23:11:07.568966+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:11:07.568966+0000","last_undegraded":"2024-04-24T23:11:07.568966+0000","last_fullsized":"2024-04-24T23:11:07.568966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761310+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:11:07.761310+0000","last_peered":"2024-04-24T23:11:07.761310+0000","last_clean":"2024-04-24T23:11:07.761310+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:11:07.761310+0000","last_undegraded":"2024-04-24T23:11:07.761310+0000","last_fullsized":"2024-04-24T23:11:07.761310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":350,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565786+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:11:07.565786+0000","last_peered":"2024-04-24T23:11:07.565786+0000","last_clean":"2024-04-24T23:11:07.565786+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:11:07.565786+0000","last_undegraded":"2024-04-24T23:11:07.565786+0000","last_fullsized":"2024-04-24T23:11:07.565786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.571671+0000","last_change":"2024-04-24T23:11:07.571671+0000","last_active":"2024-04-24T23:11:07.571671+0000","last_peered":"2024-04-24T23:11:07.571671+0000","last_clean":"2024-04-24T23:11:07.571671+0000","last_became_active":"2024-04-24T23:11:07.571082+0000","last_became_peered":"2024-04-24T23:11:07.571082+0000","last_unstale":"2024-04-24T23:11:07.571671+0000","last_undegraded":"2024-04-24T23:11:07.571671+0000","last_fullsized":"2024-04-24T23:11:07.571671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:11.106931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563033+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:11:07.563033+0000","last_peered":"2024-04-24T23:11:07.563033+0000","last_clean":"2024-04-24T23:11:07.563033+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:11:07.563033+0000","last_undegraded":"2024-04-24T23:11:07.563033+0000","last_fullsized":"2024-04-24T23:11:07.563033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567114+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:11:07.567114+0000","last_peered":"2024-04-24T23:11:07.567114+0000","last_clean":"2024-04-24T23:11:07.567114+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:11:07.567114+0000","last_undegraded":"2024-04-24T23:11:07.567114+0000","last_fullsized":"2024-04-24T23:11:07.567114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566285+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:11:07.566285+0000","last_peered":"2024-04-24T23:11:07.566285+0000","last_clean":"2024-04-24T23:11:07.566285+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:11:07.566285+0000","last_undegraded":"2024-04-24T23:11:07.566285+0000","last_fullsized":"2024-04-24T23:11:07.566285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562846+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:11:07.562846+0000","last_peered":"2024-04-24T23:11:07.562846+0000","last_clean":"2024-04-24T23:11:07.562846+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:11:07.562846+0000","last_undegraded":"2024-04-24T23:11:07.562846+0000","last_fullsized":"2024-04-24T23:11:07.562846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563789+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:11:07.563789+0000","last_peered":"2024-04-24T23:11:07.563789+0000","last_clean":"2024-04-24T23:11:07.563789+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:11:07.563789+0000","last_undegraded":"2024-04-24T23:11:07.563789+0000","last_fullsized":"2024-04-24T23:11:07.563789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568770+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:11:07.568770+0000","last_peered":"2024-04-24T23:11:07.568770+0000","last_clean":"2024-04-24T23:11:07.568770+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:11:07.568770+0000","last_undegraded":"2024-04-24T23:11:07.568770+0000","last_fullsized":"2024-04-24T23:11:07.568770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561646+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:11:07.561646+0000","last_peered":"2024-04-24T23:11:07.561646+0000","last_clean":"2024-04-24T23:11:07.561646+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:11:07.561646+0000","last_undegraded":"2024-04-24T23:11:07.561646+0000","last_fullsized":"2024-04-24T23:11:07.561646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569324+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:11:07.569324+0000","last_peered":"2024-04-24T23:11:07.569324+0000","last_clean":"2024-04-24T23:11:07.569324+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:11:07.569324+0000","last_undegraded":"2024-04-24T23:11:07.569324+0000","last_fullsized":"2024-04-24T23:11:07.569324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568462+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:11:07.568462+0000","last_peered":"2024-04-24T23:11:07.568462+0000","last_clean":"2024-04-24T23:11:07.568462+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:11:07.568462+0000","last_undegraded":"2024-04-24T23:11:07.568462+0000","last_fullsized":"2024-04-24T23:11:07.568462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761859+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:11:07.761859+0000","last_peered":"2024-04-24T23:11:07.761859+0000","last_clean":"2024-04-24T23:11:07.761859+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:11:07.761859+0000","last_undegraded":"2024-04-24T23:11:07.761859+0000","last_fullsized":"2024-04-24T23:11:07.761859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573753+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:11:07.573753+0000","last_peered":"2024-04-24T23:11:07.573753+0000","last_clean":"2024-04-24T23:11:07.573753+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:11:07.573753+0000","last_undegraded":"2024-04-24T23:11:07.573753+0000","last_fullsized":"2024-04-24T23:11:07.573753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761378+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:11:07.761378+0000","last_peered":"2024-04-24T23:11:07.761378+0000","last_clean":"2024-04-24T23:11:07.761378+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:11:07.761378+0000","last_undegraded":"2024-04-24T23:11:07.761378+0000","last_fullsized":"2024-04-24T23:11:07.761378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569647+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:11:07.569647+0000","last_peered":"2024-04-24T23:11:07.569647+0000","last_clean":"2024-04-24T23:11:07.569647+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:11:07.569647+0000","last_undegraded":"2024-04-24T23:11:07.569647+0000","last_fullsized":"2024-04-24T23:11:07.569647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561956+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:11:07.561956+0000","last_peered":"2024-04-24T23:11:07.561956+0000","last_clean":"2024-04-24T23:11:07.561956+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:11:07.561956+0000","last_undegraded":"2024-04-24T23:11:07.561956+0000","last_fullsized":"2024-04-24T23:11:07.561956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561616+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:11:07.561616+0000","last_peered":"2024-04-24T23:11:07.561616+0000","last_clean":"2024-04-24T23:11:07.561616+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:11:07.561616+0000","last_undegraded":"2024-04-24T23:11:07.561616+0000","last_fullsized":"2024-04-24T23:11:07.561616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568968+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:11:07.568968+0000","last_peered":"2024-04-24T23:11:07.568968+0000","last_clean":"2024-04-24T23:11:07.568968+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:11:07.568968+0000","last_undegraded":"2024-04-24T23:11:07.568968+0000","last_fullsized":"2024-04-24T23:11:07.568968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567063+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:11:07.567063+0000","last_peered":"2024-04-24T23:11:07.567063+0000","last_clean":"2024-04-24T23:11:07.567063+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:11:07.567063+0000","last_undegraded":"2024-04-24T23:11:07.567063+0000","last_fullsized":"2024-04-24T23:11:07.567063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761430+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:11:07.761430+0000","last_peered":"2024-04-24T23:11:07.761430+0000","last_clean":"2024-04-24T23:11:07.761430+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:11:07.761430+0000","last_undegraded":"2024-04-24T23:11:07.761430+0000","last_fullsized":"2024-04-24T23:11:07.761430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569406+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:11:07.569406+0000","last_peered":"2024-04-24T23:11:07.569406+0000","last_clean":"2024-04-24T23:11:07.569406+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:11:07.569406+0000","last_undegraded":"2024-04-24T23:11:07.569406+0000","last_fullsized":"2024-04-24T23:11:07.569406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569402+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:11:07.569402+0000","last_peered":"2024-04-24T23:11:07.569402+0000","last_clean":"2024-04-24T23:11:07.569402+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:11:07.569402+0000","last_undegraded":"2024-04-24T23:11:07.569402+0000","last_fullsized":"2024-04-24T23:11:07.569402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761368+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:11:07.761368+0000","last_peered":"2024-04-24T23:11:07.761368+0000","last_clean":"2024-04-24T23:11:07.761368+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:11:07.761368+0000","last_undegraded":"2024-04-24T23:11:07.761368+0000","last_fullsized":"2024-04-24T23:11:07.761368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562381+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:11:07.562381+0000","last_peered":"2024-04-24T23:11:07.562381+0000","last_clean":"2024-04-24T23:11:07.562381+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:11:07.562381+0000","last_undegraded":"2024-04-24T23:11:07.562381+0000","last_fullsized":"2024-04-24T23:11:07.562381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566130+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:11:07.566130+0000","last_peered":"2024-04-24T23:11:07.566130+0000","last_clean":"2024-04-24T23:11:07.566130+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:11:07.566130+0000","last_undegraded":"2024-04-24T23:11:07.566130+0000","last_fullsized":"2024-04-24T23:11:07.566130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569856+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:11:07.569856+0000","last_peered":"2024-04-24T23:11:07.569856+0000","last_clean":"2024-04-24T23:11:07.569856+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:11:07.569856+0000","last_undegraded":"2024-04-24T23:11:07.569856+0000","last_fullsized":"2024-04-24T23:11:07.569856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.579626+0000","last_change":"2024-04-24T23:11:07.579626+0000","last_active":"2024-04-24T23:11:07.579626+0000","last_peered":"2024-04-24T23:11:07.579626+0000","last_clean":"2024-04-24T23:11:07.579626+0000","last_became_active":"2024-04-24T23:11:07.574907+0000","last_became_peered":"2024-04-24T23:11:07.574907+0000","last_unstale":"2024-04-24T23:11:07.579626+0000","last_undegraded":"2024-04-24T23:11:07.579626+0000","last_fullsized":"2024-04-24T23:11:07.579626+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:56.428190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566713+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:11:07.566713+0000","last_peered":"2024-04-24T23:11:07.566713+0000","last_clean":"2024-04-24T23:11:07.566713+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:11:07.566713+0000","last_undegraded":"2024-04-24T23:11:07.566713+0000","last_fullsized":"2024-04-24T23:11:07.566713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573251+0000","last_change":"2024-04-24T23:10:16.387682+0000","last_active":"2024-04-24T23:11:07.573251+0000","last_peered":"2024-04-24T23:11:07.573251+0000","last_clean":"2024-04-24T23:11:07.573251+0000","last_became_active":"2024-04-24T23:10:16.387248+0000","last_became_peered":"2024-04-24T23:10:16.387248+0000","last_unstale":"2024-04-24T23:11:07.573251+0000","last_undegraded":"2024-04-24T23:11:07.573251+0000","last_fullsized":"2024-04-24T23:11:07.573251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:54.837632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566107+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:11:07.566107+0000","last_peered":"2024-04-24T23:11:07.566107+0000","last_clean":"2024-04-24T23:11:07.566107+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:11:07.566107+0000","last_undegraded":"2024-04-24T23:11:07.566107+0000","last_fullsized":"2024-04-24T23:11:07.566107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566083+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:11:07.566083+0000","last_peered":"2024-04-24T23:11:07.566083+0000","last_clean":"2024-04-24T23:11:07.566083+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:11:07.566083+0000","last_undegraded":"2024-04-24T23:11:07.566083+0000","last_fullsized":"2024-04-24T23:11:07.566083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563867+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:11:07.563867+0000","last_peered":"2024-04-24T23:11:07.563867+0000","last_clean":"2024-04-24T23:11:07.563867+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:11:07.563867+0000","last_undegraded":"2024-04-24T23:11:07.563867+0000","last_fullsized":"2024-04-24T23:11:07.563867+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563781+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:11:07.563781+0000","last_peered":"2024-04-24T23:11:07.563781+0000","last_clean":"2024-04-24T23:11:07.563781+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:11:07.563781+0000","last_undegraded":"2024-04-24T23:11:07.563781+0000","last_fullsized":"2024-04-24T23:11:07.563781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566491+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:11:07.566491+0000","last_peered":"2024-04-24T23:11:07.566491+0000","last_clean":"2024-04-24T23:11:07.566491+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:11:07.566491+0000","last_undegraded":"2024-04-24T23:11:07.566491+0000","last_fullsized":"2024-04-24T23:11:07.566491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567186+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:11:07.567186+0000","last_peered":"2024-04-24T23:11:07.567186+0000","last_clean":"2024-04-24T23:11:07.567186+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:11:07.567186+0000","last_undegraded":"2024-04-24T23:11:07.567186+0000","last_fullsized":"2024-04-24T23:11:07.567186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569014+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:11:07.569014+0000","last_peered":"2024-04-24T23:11:07.569014+0000","last_clean":"2024-04-24T23:11:07.569014+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:11:07.569014+0000","last_undegraded":"2024-04-24T23:11:07.569014+0000","last_fullsized":"2024-04-24T23:11:07.569014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569275+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:11:07.569275+0000","last_peered":"2024-04-24T23:11:07.569275+0000","last_clean":"2024-04-24T23:11:07.569275+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:11:07.569275+0000","last_undegraded":"2024-04-24T23:11:07.569275+0000","last_fullsized":"2024-04-24T23:11:07.569275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573422+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:11:07.573422+0000","last_peered":"2024-04-24T23:11:07.573422+0000","last_clean":"2024-04-24T23:11:07.573422+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:11:07.573422+0000","last_undegraded":"2024-04-24T23:11:07.573422+0000","last_fullsized":"2024-04-24T23:11:07.573422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563925+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:11:07.563925+0000","last_peered":"2024-04-24T23:11:07.563925+0000","last_clean":"2024-04-24T23:11:07.563925+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:11:07.563925+0000","last_undegraded":"2024-04-24T23:11:07.563925+0000","last_fullsized":"2024-04-24T23:11:07.563925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566415+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:11:07.566415+0000","last_peered":"2024-04-24T23:11:07.566415+0000","last_clean":"2024-04-24T23:11:07.566415+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:11:07.566415+0000","last_undegraded":"2024-04-24T23:11:07.566415+0000","last_fullsized":"2024-04-24T23:11:07.566415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569421+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:11:07.569421+0000","last_peered":"2024-04-24T23:11:07.569421+0000","last_clean":"2024-04-24T23:11:07.569421+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:11:07.569421+0000","last_undegraded":"2024-04-24T23:11:07.569421+0000","last_fullsized":"2024-04-24T23:11:07.569421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569664+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:11:07.569664+0000","last_peered":"2024-04-24T23:11:07.569664+0000","last_clean":"2024-04-24T23:11:07.569664+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:11:07.569664+0000","last_undegraded":"2024-04-24T23:11:07.569664+0000","last_fullsized":"2024-04-24T23:11:07.569664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564019+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:11:07.564019+0000","last_peered":"2024-04-24T23:11:07.564019+0000","last_clean":"2024-04-24T23:11:07.564019+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:11:07.564019+0000","last_undegraded":"2024-04-24T23:11:07.564019+0000","last_fullsized":"2024-04-24T23:11:07.564019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569084+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:11:07.569084+0000","last_peered":"2024-04-24T23:11:07.569084+0000","last_clean":"2024-04-24T23:11:07.569084+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:11:07.569084+0000","last_undegraded":"2024-04-24T23:11:07.569084+0000","last_fullsized":"2024-04-24T23:11:07.569084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570190+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:11:07.570190+0000","last_peered":"2024-04-24T23:11:07.570190+0000","last_clean":"2024-04-24T23:11:07.570190+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:11:07.570190+0000","last_undegraded":"2024-04-24T23:11:07.570190+0000","last_fullsized":"2024-04-24T23:11:07.570190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563999+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:11:07.563999+0000","last_peered":"2024-04-24T23:11:07.563999+0000","last_clean":"2024-04-24T23:11:07.563999+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:11:07.563999+0000","last_undegraded":"2024-04-24T23:11:07.563999+0000","last_fullsized":"2024-04-24T23:11:07.563999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566686+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:11:07.566686+0000","last_peered":"2024-04-24T23:11:07.566686+0000","last_clean":"2024-04-24T23:11:07.566686+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:11:07.566686+0000","last_undegraded":"2024-04-24T23:11:07.566686+0000","last_fullsized":"2024-04-24T23:11:07.566686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65776,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":465,"num_read_kb":303,"num_write":159,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1198,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":197328,"data_compressed":4821,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":503892,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1153292,"data_compressed":58843,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":508190,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":503432,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":1099923,"data_compressed":57236,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499270,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":561537,"data_compressed":22308,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500658,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":561689,"data_compressed":22308,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":499464,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888132,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":504893,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1095120,"data_compressed":57236,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62357,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62509,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63744,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:11:53.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:11:53.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:53 smithi033 ceph-mon[29201]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:11:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:54.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:53 smithi038 ceph-mon[32450]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:11:54.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:53 smithi117 ceph-mon[32529]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:11:54.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:54 smithi033 ceph-mon[29201]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:55.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:54 smithi038 ceph-mon[32450]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:54 smithi117 ceph-mon[32529]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:55.893 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:55 smithi033 ceph-mon[29201]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 -- 172.21.15.33:0/4168397977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 msgr2=0x7fc52c108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 --2- 172.21.15.33:0/4168397977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c108690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc524009990 tx=0x7fc524009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 -- 172.21.15.33:0/4168397977 shutdown_connections 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 --2- 172.21.15.33:0/4168397977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52c06b390 0x7fc52c10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 --2- 172.21.15.33:0/4168397977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c108690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 --2- 172.21.15.33:0/4168397977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc52c06a9f0 0x7fc52c06ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 -- 172.21.15.33:0/4168397977 >> 172.21.15.33:0/4168397977 conn(0x7fc52c0780e0 msgr2=0x7fc52c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 -- 172.21.15.33:0/4168397977 shutdown_connections 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.907+0000 7fc533886700 1 -- 172.21.15.33:0/4168397977 wait complete. 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 Processor -- start 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 -- start start 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52c06a9f0 0x7fc52c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc52c06b390 0x7fc52c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c112510 con 0x7fc52c1082b0 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c112390 con 0x7fc52c06b390 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc533886700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c112690 con 0x7fc52c06a9f0 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49074/0 (socket says 172.21.15.33:49074) 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 -- 172.21.15.33:0/2917409266 learned_addr learned my addr 172.21.15.33:0/2917409266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531622700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52c06a9f0 0x7fc52c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc530e21700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc52c06b390 0x7fc52c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 -- 172.21.15.33:0/2917409266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52c06a9f0 msgr2=0x7fc52c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52c06a9f0 0x7fc52c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 -- 172.21.15.33:0/2917409266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc52c06b390 msgr2=0x7fc52c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc52c06b390 0x7fc52c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 -- 172.21.15.33:0/2917409266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c199030 con 0x7fc52c1082b0 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.908+0000 7fc531e23700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c198850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc51c007ee0 tx=0x7fc51c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.909+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c004bb0 con 0x7fc52c1082b0 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.909+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c199320 con 0x7fc52c1082b0 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.909+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51c004d50 con 0x7fc52c1082b0 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.909+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c005960 con 0x7fc52c1082b0 2024-04-24T23:11:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.909+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c1a5530 con 0x7fc52c1082b0 2024-04-24T23:11:55.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.910+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c103960 con 0x7fc52c1082b0 2024-04-24T23:11:55.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.911+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc51c01a020 con 0x7fc52c1082b0 2024-04-24T23:11:55.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.911+0000 7fc52a7fc700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5140a2cb0 0x7fc5140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.912+0000 7fc531622700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5140a2cb0 0x7fc5140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.912+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc51c0cec00 con 0x7fc52c1082b0 2024-04-24T23:11:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.912+0000 7fc531622700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5140a2cb0 0x7fc5140a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b5c0 tx=0x7fc524005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:55.916+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc51c010070 con 0x7fc52c1082b0 2024-04-24T23:11:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.120+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc52c062670 con 0x7fc5140a2cb0 2024-04-24T23:11:56.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.130+0000 7fc52a7fc700 1 -- 172.21.15.33:0/2917409266 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340686 (secure 0 0 0) 0x7fc52c062670 con 0x7fc5140a2cb0 2024-04-24T23:11:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:11:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5140a2cb0 msgr2=0x7fc5140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5140a2cb0 0x7fc5140a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b5c0 tx=0x7fc524005960 comp rx=0 tx=0).stop 2024-04-24T23:11:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 msgr2=0x7fc52c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c198850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc51c007ee0 tx=0x7fc51c00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 shutdown_connections 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5140a2cb0 0x7fc5140a5170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc52c1082b0 0x7fc52c198850 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc52c06a9f0 0x7fc52c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 --2- 172.21.15.33:0/2917409266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc52c06b390 0x7fc52c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 >> 172.21.15.33:0/2917409266 conn(0x7fc52c0780e0 msgr2=0x7fc52c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 shutdown_connections 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:56.133+0000 7fc533886700 1 -- 172.21.15.33:0/2917409266 wait complete. 2024-04-24T23:11:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:11:56.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:55 smithi038 ceph-mon[32450]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:56.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:55 smithi117 ceph-mon[32529]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:56.855 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-24T23:11:55.806536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656378,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":601,"num_read_kb":412,"num_write":342,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181934,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357304,"kb_used_data":12236,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584520,"statfs":{"total":1151940427776,"available":1151574548480,"internally_reserved":0,"allocated":12529664,"data_stored":7991360,"data_compressed":362838,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003216"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567156+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:11:07.567156+0000","last_peered":"2024-04-24T23:11:07.567156+0000","last_clean":"2024-04-24T23:11:07.567156+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:11:07.567156+0000","last_undegraded":"2024-04-24T23:11:07.567156+0000","last_fullsized":"2024-04-24T23:11:07.567156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572649+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:11:07.572649+0000","last_peered":"2024-04-24T23:11:07.572649+0000","last_clean":"2024-04-24T23:11:07.572649+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:11:07.572649+0000","last_undegraded":"2024-04-24T23:11:07.572649+0000","last_fullsized":"2024-04-24T23:11:07.572649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572946+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:11:07.572946+0000","last_peered":"2024-04-24T23:11:07.572946+0000","last_clean":"2024-04-24T23:11:07.572946+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:11:07.572946+0000","last_undegraded":"2024-04-24T23:11:07.572946+0000","last_fullsized":"2024-04-24T23:11:07.572946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573213+0000","last_change":"2024-04-24T23:11:07.572557+0000","last_active":"2024-04-24T23:11:07.573213+0000","last_peered":"2024-04-24T23:11:07.573213+0000","last_clean":"2024-04-24T23:11:07.573213+0000","last_became_active":"2024-04-24T23:11:07.572144+0000","last_became_peered":"2024-04-24T23:11:07.572144+0000","last_unstale":"2024-04-24T23:11:07.573213+0000","last_undegraded":"2024-04-24T23:11:07.573213+0000","last_fullsized":"2024-04-24T23:11:07.573213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:42.681992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562275+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:11:07.562275+0000","last_peered":"2024-04-24T23:11:07.562275+0000","last_clean":"2024-04-24T23:11:07.562275+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:11:07.562275+0000","last_undegraded":"2024-04-24T23:11:07.562275+0000","last_fullsized":"2024-04-24T23:11:07.562275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567631+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:11:07.567631+0000","last_peered":"2024-04-24T23:11:07.567631+0000","last_clean":"2024-04-24T23:11:07.567631+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:11:07.567631+0000","last_undegraded":"2024-04-24T23:11:07.567631+0000","last_fullsized":"2024-04-24T23:11:07.567631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565934+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:11:07.565934+0000","last_peered":"2024-04-24T23:11:07.565934+0000","last_clean":"2024-04-24T23:11:07.565934+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:11:07.565934+0000","last_undegraded":"2024-04-24T23:11:07.565934+0000","last_fullsized":"2024-04-24T23:11:07.565934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568885+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:11:07.568885+0000","last_peered":"2024-04-24T23:11:07.568885+0000","last_clean":"2024-04-24T23:11:07.568885+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:11:07.568885+0000","last_undegraded":"2024-04-24T23:11:07.568885+0000","last_fullsized":"2024-04-24T23:11:07.568885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570909+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:11:07.570909+0000","last_peered":"2024-04-24T23:11:07.570909+0000","last_clean":"2024-04-24T23:11:07.570909+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:11:07.570909+0000","last_undegraded":"2024-04-24T23:11:07.570909+0000","last_fullsized":"2024-04-24T23:11:07.570909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567846+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:11:07.567846+0000","last_peered":"2024-04-24T23:11:07.567846+0000","last_clean":"2024-04-24T23:11:07.567846+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:11:07.567846+0000","last_undegraded":"2024-04-24T23:11:07.567846+0000","last_fullsized":"2024-04-24T23:11:07.567846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566596+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:11:07.566596+0000","last_peered":"2024-04-24T23:11:07.566596+0000","last_clean":"2024-04-24T23:11:07.566596+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:11:07.566596+0000","last_undegraded":"2024-04-24T23:11:07.566596+0000","last_fullsized":"2024-04-24T23:11:07.566596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569777+0000","last_change":"2024-04-24T23:11:07.568947+0000","last_active":"2024-04-24T23:11:07.569777+0000","last_peered":"2024-04-24T23:11:07.569777+0000","last_clean":"2024-04-24T23:11:07.569777+0000","last_became_active":"2024-04-24T23:11:07.568528+0000","last_became_peered":"2024-04-24T23:11:07.568528+0000","last_unstale":"2024-04-24T23:11:07.569777+0000","last_undegraded":"2024-04-24T23:11:07.569777+0000","last_fullsized":"2024-04-24T23:11:07.569777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:23.452961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564088+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:11:07.564088+0000","last_peered":"2024-04-24T23:11:07.564088+0000","last_clean":"2024-04-24T23:11:07.564088+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:11:07.564088+0000","last_undegraded":"2024-04-24T23:11:07.564088+0000","last_fullsized":"2024-04-24T23:11:07.564088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573126+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:11:07.573126+0000","last_peered":"2024-04-24T23:11:07.573126+0000","last_clean":"2024-04-24T23:11:07.573126+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:11:07.573126+0000","last_undegraded":"2024-04-24T23:11:07.573126+0000","last_fullsized":"2024-04-24T23:11:07.573126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572678+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:11:07.572678+0000","last_peered":"2024-04-24T23:11:07.572678+0000","last_clean":"2024-04-24T23:11:07.572678+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:11:07.572678+0000","last_undegraded":"2024-04-24T23:11:07.572678+0000","last_fullsized":"2024-04-24T23:11:07.572678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569658+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:11:07.569658+0000","last_peered":"2024-04-24T23:11:07.569658+0000","last_clean":"2024-04-24T23:11:07.569658+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:11:07.569658+0000","last_undegraded":"2024-04-24T23:11:07.569658+0000","last_fullsized":"2024-04-24T23:11:07.569658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567031+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:11:07.567031+0000","last_peered":"2024-04-24T23:11:07.567031+0000","last_clean":"2024-04-24T23:11:07.567031+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:11:07.567031+0000","last_undegraded":"2024-04-24T23:11:07.567031+0000","last_fullsized":"2024-04-24T23:11:07.567031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561761+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:11:07.561761+0000","last_peered":"2024-04-24T23:11:07.561761+0000","last_clean":"2024-04-24T23:11:07.561761+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:11:07.561761+0000","last_undegraded":"2024-04-24T23:11:07.561761+0000","last_fullsized":"2024-04-24T23:11:07.561761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570090+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:11:07.570090+0000","last_peered":"2024-04-24T23:11:07.570090+0000","last_clean":"2024-04-24T23:11:07.570090+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:11:07.570090+0000","last_undegraded":"2024-04-24T23:11:07.570090+0000","last_fullsized":"2024-04-24T23:11:07.570090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568876+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:11:07.568876+0000","last_peered":"2024-04-24T23:11:07.568876+0000","last_clean":"2024-04-24T23:11:07.568876+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:11:07.568876+0000","last_undegraded":"2024-04-24T23:11:07.568876+0000","last_fullsized":"2024-04-24T23:11:07.568876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.769518+0000","last_change":"2024-04-24T23:11:07.769518+0000","last_active":"2024-04-24T23:11:07.769518+0000","last_peered":"2024-04-24T23:11:07.769518+0000","last_clean":"2024-04-24T23:11:07.769518+0000","last_became_active":"2024-04-24T23:11:07.763040+0000","last_became_peered":"2024-04-24T23:11:07.763040+0000","last_unstale":"2024-04-24T23:11:07.769518+0000","last_undegraded":"2024-04-24T23:11:07.769518+0000","last_fullsized":"2024-04-24T23:11:07.769518+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.056568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569338+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:11:07.569338+0000","last_peered":"2024-04-24T23:11:07.569338+0000","last_clean":"2024-04-24T23:11:07.569338+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:11:07.569338+0000","last_undegraded":"2024-04-24T23:11:07.569338+0000","last_fullsized":"2024-04-24T23:11:07.569338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761071+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:11:07.761071+0000","last_peered":"2024-04-24T23:11:07.761071+0000","last_clean":"2024-04-24T23:11:07.761071+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:11:07.761071+0000","last_undegraded":"2024-04-24T23:11:07.761071+0000","last_fullsized":"2024-04-24T23:11:07.761071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569611+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:11:07.569611+0000","last_peered":"2024-04-24T23:11:07.569611+0000","last_clean":"2024-04-24T23:11:07.569611+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:11:07.569611+0000","last_undegraded":"2024-04-24T23:11:07.569611+0000","last_fullsized":"2024-04-24T23:11:07.569611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573786+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:11:07.573786+0000","last_peered":"2024-04-24T23:11:07.573786+0000","last_clean":"2024-04-24T23:11:07.573786+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:11:07.573786+0000","last_undegraded":"2024-04-24T23:11:07.573786+0000","last_fullsized":"2024-04-24T23:11:07.573786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761141+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:11:07.761141+0000","last_peered":"2024-04-24T23:11:07.761141+0000","last_clean":"2024-04-24T23:11:07.761141+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:11:07.761141+0000","last_undegraded":"2024-04-24T23:11:07.761141+0000","last_fullsized":"2024-04-24T23:11:07.761141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.583051+0000","last_change":"2024-04-24T23:11:07.583051+0000","last_active":"2024-04-24T23:11:07.583051+0000","last_peered":"2024-04-24T23:11:07.583051+0000","last_clean":"2024-04-24T23:11:07.583051+0000","last_became_active":"2024-04-24T23:11:07.572502+0000","last_became_peered":"2024-04-24T23:11:07.572502+0000","last_unstale":"2024-04-24T23:11:07.583051+0000","last_undegraded":"2024-04-24T23:11:07.583051+0000","last_fullsized":"2024-04-24T23:11:07.583051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:21.581198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570706+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:11:07.570706+0000","last_peered":"2024-04-24T23:11:07.570706+0000","last_clean":"2024-04-24T23:11:07.570706+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:11:07.570706+0000","last_undegraded":"2024-04-24T23:11:07.570706+0000","last_fullsized":"2024-04-24T23:11:07.570706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761055+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:11:07.761055+0000","last_peered":"2024-04-24T23:11:07.761055+0000","last_clean":"2024-04-24T23:11:07.761055+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:11:07.761055+0000","last_undegraded":"2024-04-24T23:11:07.761055+0000","last_fullsized":"2024-04-24T23:11:07.761055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562795+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:11:07.562795+0000","last_peered":"2024-04-24T23:11:07.562795+0000","last_clean":"2024-04-24T23:11:07.562795+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:11:07.562795+0000","last_undegraded":"2024-04-24T23:11:07.562795+0000","last_fullsized":"2024-04-24T23:11:07.562795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565967+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:11:07.565967+0000","last_peered":"2024-04-24T23:11:07.565967+0000","last_clean":"2024-04-24T23:11:07.565967+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:11:07.565967+0000","last_undegraded":"2024-04-24T23:11:07.565967+0000","last_fullsized":"2024-04-24T23:11:07.565967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569772+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:11:07.569772+0000","last_peered":"2024-04-24T23:11:07.569772+0000","last_clean":"2024-04-24T23:11:07.569772+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:11:07.569772+0000","last_undegraded":"2024-04-24T23:11:07.569772+0000","last_fullsized":"2024-04-24T23:11:07.569772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:53.614844+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:11:53.614844+0000","last_peered":"2024-04-24T23:11:53.614844+0000","last_clean":"2024-04-24T23:11:53.614844+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:11:53.614844+0000","last_undegraded":"2024-04-24T23:11:53.614844+0000","last_fullsized":"2024-04-24T23:11:53.614844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562252+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:11:07.562252+0000","last_peered":"2024-04-24T23:11:07.562252+0000","last_clean":"2024-04-24T23:11:07.562252+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:11:07.562252+0000","last_undegraded":"2024-04-24T23:11:07.562252+0000","last_fullsized":"2024-04-24T23:11:07.562252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569464+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:11:07.569464+0000","last_peered":"2024-04-24T23:11:07.569464+0000","last_clean":"2024-04-24T23:11:07.569464+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:11:07.569464+0000","last_undegraded":"2024-04-24T23:11:07.569464+0000","last_fullsized":"2024-04-24T23:11:07.569464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567502+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:11:07.567502+0000","last_peered":"2024-04-24T23:11:07.567502+0000","last_clean":"2024-04-24T23:11:07.567502+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:11:07.567502+0000","last_undegraded":"2024-04-24T23:11:07.567502+0000","last_fullsized":"2024-04-24T23:11:07.567502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565397+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:11:07.565397+0000","last_peered":"2024-04-24T23:11:07.565397+0000","last_clean":"2024-04-24T23:11:07.565397+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:11:07.565397+0000","last_undegraded":"2024-04-24T23:11:07.565397+0000","last_fullsized":"2024-04-24T23:11:07.565397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566861+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:11:07.566861+0000","last_peered":"2024-04-24T23:11:07.566861+0000","last_clean":"2024-04-24T23:11:07.566861+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:11:07.566861+0000","last_undegraded":"2024-04-24T23:11:07.566861+0000","last_fullsized":"2024-04-24T23:11:07.566861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563169+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:11:07.563169+0000","last_peered":"2024-04-24T23:11:07.563169+0000","last_clean":"2024-04-24T23:11:07.563169+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:11:07.563169+0000","last_undegraded":"2024-04-24T23:11:07.563169+0000","last_fullsized":"2024-04-24T23:11:07.563169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568782+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:11:07.568782+0000","last_peered":"2024-04-24T23:11:07.568782+0000","last_clean":"2024-04-24T23:11:07.568782+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:11:07.568782+0000","last_undegraded":"2024-04-24T23:11:07.568782+0000","last_fullsized":"2024-04-24T23:11:07.568782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.572537+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:11:07.572537+0000","last_peered":"2024-04-24T23:11:07.572537+0000","last_clean":"2024-04-24T23:11:07.572537+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:11:07.572537+0000","last_undegraded":"2024-04-24T23:11:07.572537+0000","last_fullsized":"2024-04-24T23:11:07.572537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564005+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:11:07.564005+0000","last_peered":"2024-04-24T23:11:07.564005+0000","last_clean":"2024-04-24T23:11:07.564005+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:11:07.564005+0000","last_undegraded":"2024-04-24T23:11:07.564005+0000","last_fullsized":"2024-04-24T23:11:07.564005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.560882+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:11:07.560882+0000","last_peered":"2024-04-24T23:11:07.560882+0000","last_clean":"2024-04-24T23:11:07.560882+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:11:07.560882+0000","last_undegraded":"2024-04-24T23:11:07.560882+0000","last_fullsized":"2024-04-24T23:11:07.560882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568846+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:11:07.568846+0000","last_peered":"2024-04-24T23:11:07.568846+0000","last_clean":"2024-04-24T23:11:07.568846+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:11:07.568846+0000","last_undegraded":"2024-04-24T23:11:07.568846+0000","last_fullsized":"2024-04-24T23:11:07.568846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:50.609220+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:11:50.609220+0000","last_peered":"2024-04-24T23:11:50.609220+0000","last_clean":"2024-04-24T23:11:50.609220+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:11:50.609220+0000","last_undegraded":"2024-04-24T23:11:50.609220+0000","last_fullsized":"2024-04-24T23:11:50.609220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570643+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:11:07.570643+0000","last_peered":"2024-04-24T23:11:07.570643+0000","last_clean":"2024-04-24T23:11:07.570643+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:11:07.570643+0000","last_undegraded":"2024-04-24T23:11:07.570643+0000","last_fullsized":"2024-04-24T23:11:07.570643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.560796+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:11:07.560796+0000","last_peered":"2024-04-24T23:11:07.560796+0000","last_clean":"2024-04-24T23:11:07.560796+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:11:07.560796+0000","last_undegraded":"2024-04-24T23:11:07.560796+0000","last_fullsized":"2024-04-24T23:11:07.560796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566074+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:11:07.566074+0000","last_peered":"2024-04-24T23:11:07.566074+0000","last_clean":"2024-04-24T23:11:07.566074+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:11:07.566074+0000","last_undegraded":"2024-04-24T23:11:07.566074+0000","last_fullsized":"2024-04-24T23:11:07.566074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573966+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:11:07.573966+0000","last_peered":"2024-04-24T23:11:07.573966+0000","last_clean":"2024-04-24T23:11:07.573966+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:11:07.573966+0000","last_undegraded":"2024-04-24T23:11:07.573966+0000","last_fullsized":"2024-04-24T23:11:07.573966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761375+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:11:07.761375+0000","last_peered":"2024-04-24T23:11:07.761375+0000","last_clean":"2024-04-24T23:11:07.761375+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:11:07.761375+0000","last_undegraded":"2024-04-24T23:11:07.761375+0000","last_fullsized":"2024-04-24T23:11:07.761375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565827+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:11:07.565827+0000","last_peered":"2024-04-24T23:11:07.565827+0000","last_clean":"2024-04-24T23:11:07.565827+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:11:07.565827+0000","last_undegraded":"2024-04-24T23:11:07.565827+0000","last_fullsized":"2024-04-24T23:11:07.565827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570770+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:11:07.570770+0000","last_peered":"2024-04-24T23:11:07.570770+0000","last_clean":"2024-04-24T23:11:07.570770+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:11:07.570770+0000","last_undegraded":"2024-04-24T23:11:07.570770+0000","last_fullsized":"2024-04-24T23:11:07.570770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.560932+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:11:07.560932+0000","last_peered":"2024-04-24T23:11:07.560932+0000","last_clean":"2024-04-24T23:11:07.560932+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:11:07.560932+0000","last_undegraded":"2024-04-24T23:11:07.560932+0000","last_fullsized":"2024-04-24T23:11:07.560932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":213,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:51.593992+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:11:51.593992+0000","last_peered":"2024-04-24T23:11:51.593992+0000","last_clean":"2024-04-24T23:11:51.593992+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:11:51.593992+0000","last_undegraded":"2024-04-24T23:11:51.593992+0000","last_fullsized":"2024-04-24T23:11:51.593992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562482+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:11:07.562482+0000","last_peered":"2024-04-24T23:11:07.562482+0000","last_clean":"2024-04-24T23:11:07.562482+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:11:07.562482+0000","last_undegraded":"2024-04-24T23:11:07.562482+0000","last_fullsized":"2024-04-24T23:11:07.562482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566374+0000","last_change":"2024-04-24T23:10:16.378770+0000","last_active":"2024-04-24T23:11:07.566374+0000","last_peered":"2024-04-24T23:11:07.566374+0000","last_clean":"2024-04-24T23:11:07.566374+0000","last_became_active":"2024-04-24T23:10:16.377971+0000","last_became_peered":"2024-04-24T23:10:16.377971+0000","last_unstale":"2024-04-24T23:11:07.566374+0000","last_undegraded":"2024-04-24T23:11:07.566374+0000","last_fullsized":"2024-04-24T23:11:07.566374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:18.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.579273+0000","last_change":"2024-04-24T23:11:07.579273+0000","last_active":"2024-04-24T23:11:07.579273+0000","last_peered":"2024-04-24T23:11:07.579273+0000","last_clean":"2024-04-24T23:11:07.579273+0000","last_became_active":"2024-04-24T23:11:07.570480+0000","last_became_peered":"2024-04-24T23:11:07.570480+0000","last_unstale":"2024-04-24T23:11:07.579273+0000","last_undegraded":"2024-04-24T23:11:07.579273+0000","last_fullsized":"2024-04-24T23:11:07.579273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:21.291453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563748+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:11:07.563748+0000","last_peered":"2024-04-24T23:11:07.563748+0000","last_clean":"2024-04-24T23:11:07.563748+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:11:07.563748+0000","last_undegraded":"2024-04-24T23:11:07.563748+0000","last_fullsized":"2024-04-24T23:11:07.563748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566285+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:11:07.566285+0000","last_peered":"2024-04-24T23:11:07.566285+0000","last_clean":"2024-04-24T23:11:07.566285+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:11:07.566285+0000","last_undegraded":"2024-04-24T23:11:07.566285+0000","last_fullsized":"2024-04-24T23:11:07.566285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564922+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:11:07.564922+0000","last_peered":"2024-04-24T23:11:07.564922+0000","last_clean":"2024-04-24T23:11:07.564922+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:11:07.564922+0000","last_undegraded":"2024-04-24T23:11:07.564922+0000","last_fullsized":"2024-04-24T23:11:07.564922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566639+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:11:07.566639+0000","last_peered":"2024-04-24T23:11:07.566639+0000","last_clean":"2024-04-24T23:11:07.566639+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:11:07.566639+0000","last_undegraded":"2024-04-24T23:11:07.566639+0000","last_fullsized":"2024-04-24T23:11:07.566639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561502+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:11:07.561502+0000","last_peered":"2024-04-24T23:11:07.561502+0000","last_clean":"2024-04-24T23:11:07.561502+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:11:07.561502+0000","last_undegraded":"2024-04-24T23:11:07.561502+0000","last_fullsized":"2024-04-24T23:11:07.561502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:54.587150+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:11:54.587150+0000","last_peered":"2024-04-24T23:11:54.587150+0000","last_clean":"2024-04-24T23:11:54.587150+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:11:54.587150+0000","last_undegraded":"2024-04-24T23:11:54.587150+0000","last_fullsized":"2024-04-24T23:11:54.587150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566969+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:11:07.566969+0000","last_peered":"2024-04-24T23:11:07.566969+0000","last_clean":"2024-04-24T23:11:07.566969+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:11:07.566969+0000","last_undegraded":"2024-04-24T23:11:07.566969+0000","last_fullsized":"2024-04-24T23:11:07.566969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570768+0000","last_change":"2024-04-24T23:11:07.570768+0000","last_active":"2024-04-24T23:11:07.570768+0000","last_peered":"2024-04-24T23:11:07.570768+0000","last_clean":"2024-04-24T23:11:07.570768+0000","last_became_active":"2024-04-24T23:11:07.570306+0000","last_became_peered":"2024-04-24T23:11:07.570306+0000","last_unstale":"2024-04-24T23:11:07.570768+0000","last_undegraded":"2024-04-24T23:11:07.570768+0000","last_fullsized":"2024-04-24T23:11:07.570768+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:10.450660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561940+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:11:07.561940+0000","last_peered":"2024-04-24T23:11:07.561940+0000","last_clean":"2024-04-24T23:11:07.561940+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:11:07.561940+0000","last_undegraded":"2024-04-24T23:11:07.561940+0000","last_fullsized":"2024-04-24T23:11:07.561940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.762999+0000","last_change":"2024-04-24T23:11:07.762601+0000","last_active":"2024-04-24T23:11:07.762999+0000","last_peered":"2024-04-24T23:11:07.762999+0000","last_clean":"2024-04-24T23:11:07.762999+0000","last_became_active":"2024-04-24T23:11:07.762305+0000","last_became_peered":"2024-04-24T23:11:07.762305+0000","last_unstale":"2024-04-24T23:11:07.762999+0000","last_undegraded":"2024-04-24T23:11:07.762999+0000","last_fullsized":"2024-04-24T23:11:07.762999+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:36.209046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564598+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:11:07.564598+0000","last_peered":"2024-04-24T23:11:07.564598+0000","last_clean":"2024-04-24T23:11:07.564598+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:11:07.564598+0000","last_undegraded":"2024-04-24T23:11:07.564598+0000","last_fullsized":"2024-04-24T23:11:07.564598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562595+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:11:07.562595+0000","last_peered":"2024-04-24T23:11:07.562595+0000","last_clean":"2024-04-24T23:11:07.562595+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:11:07.562595+0000","last_undegraded":"2024-04-24T23:11:07.562595+0000","last_fullsized":"2024-04-24T23:11:07.562595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569318+0000","last_change":"2024-04-24T23:11:07.569318+0000","last_active":"2024-04-24T23:11:07.569318+0000","last_peered":"2024-04-24T23:11:07.569318+0000","last_clean":"2024-04-24T23:11:07.569318+0000","last_became_active":"2024-04-24T23:11:07.568799+0000","last_became_peered":"2024-04-24T23:11:07.568799+0000","last_unstale":"2024-04-24T23:11:07.569318+0000","last_undegraded":"2024-04-24T23:11:07.569318+0000","last_fullsized":"2024-04-24T23:11:07.569318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:40.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562085+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:11:07.562085+0000","last_peered":"2024-04-24T23:11:07.562085+0000","last_clean":"2024-04-24T23:11:07.562085+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:11:07.562085+0000","last_undegraded":"2024-04-24T23:11:07.562085+0000","last_fullsized":"2024-04-24T23:11:07.562085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561443+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:11:07.561443+0000","last_peered":"2024-04-24T23:11:07.561443+0000","last_clean":"2024-04-24T23:11:07.561443+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:11:07.561443+0000","last_undegraded":"2024-04-24T23:11:07.561443+0000","last_fullsized":"2024-04-24T23:11:07.561443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567387+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:11:07.567387+0000","last_peered":"2024-04-24T23:11:07.567387+0000","last_clean":"2024-04-24T23:11:07.567387+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:11:07.567387+0000","last_undegraded":"2024-04-24T23:11:07.567387+0000","last_fullsized":"2024-04-24T23:11:07.567387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563534+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:11:07.563534+0000","last_peered":"2024-04-24T23:11:07.563534+0000","last_clean":"2024-04-24T23:11:07.563534+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:11:07.563534+0000","last_undegraded":"2024-04-24T23:11:07.563534+0000","last_fullsized":"2024-04-24T23:11:07.563534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565986+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:11:07.565986+0000","last_peered":"2024-04-24T23:11:07.565986+0000","last_clean":"2024-04-24T23:11:07.565986+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:11:07.565986+0000","last_undegraded":"2024-04-24T23:11:07.565986+0000","last_fullsized":"2024-04-24T23:11:07.565986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567244+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:11:07.567244+0000","last_peered":"2024-04-24T23:11:07.567244+0000","last_clean":"2024-04-24T23:11:07.567244+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:11:07.567244+0000","last_undegraded":"2024-04-24T23:11:07.567244+0000","last_fullsized":"2024-04-24T23:11:07.567244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563609+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:11:07.563609+0000","last_peered":"2024-04-24T23:11:07.563609+0000","last_clean":"2024-04-24T23:11:07.563609+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:11:07.563609+0000","last_undegraded":"2024-04-24T23:11:07.563609+0000","last_fullsized":"2024-04-24T23:11:07.563609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564129+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:11:07.564129+0000","last_peered":"2024-04-24T23:11:07.564129+0000","last_clean":"2024-04-24T23:11:07.564129+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:11:07.564129+0000","last_undegraded":"2024-04-24T23:11:07.564129+0000","last_fullsized":"2024-04-24T23:11:07.564129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.578584+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:11:49.578584+0000","last_peered":"2024-04-24T23:11:49.578584+0000","last_clean":"2024-04-24T23:11:49.578584+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:11:49.578584+0000","last_undegraded":"2024-04-24T23:11:49.578584+0000","last_fullsized":"2024-04-24T23:11:49.578584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566234+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:11:07.566234+0000","last_peered":"2024-04-24T23:11:07.566234+0000","last_clean":"2024-04-24T23:11:07.566234+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:11:07.566234+0000","last_undegraded":"2024-04-24T23:11:07.566234+0000","last_fullsized":"2024-04-24T23:11:07.566234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568966+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:11:07.568966+0000","last_peered":"2024-04-24T23:11:07.568966+0000","last_clean":"2024-04-24T23:11:07.568966+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:11:07.568966+0000","last_undegraded":"2024-04-24T23:11:07.568966+0000","last_fullsized":"2024-04-24T23:11:07.568966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761310+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:11:07.761310+0000","last_peered":"2024-04-24T23:11:07.761310+0000","last_clean":"2024-04-24T23:11:07.761310+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:11:07.761310+0000","last_undegraded":"2024-04-24T23:11:07.761310+0000","last_fullsized":"2024-04-24T23:11:07.761310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":350,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.565786+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:11:07.565786+0000","last_peered":"2024-04-24T23:11:07.565786+0000","last_clean":"2024-04-24T23:11:07.565786+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:11:07.565786+0000","last_undegraded":"2024-04-24T23:11:07.565786+0000","last_fullsized":"2024-04-24T23:11:07.565786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.571671+0000","last_change":"2024-04-24T23:11:07.571671+0000","last_active":"2024-04-24T23:11:07.571671+0000","last_peered":"2024-04-24T23:11:07.571671+0000","last_clean":"2024-04-24T23:11:07.571671+0000","last_became_active":"2024-04-24T23:11:07.571082+0000","last_became_peered":"2024-04-24T23:11:07.571082+0000","last_unstale":"2024-04-24T23:11:07.571671+0000","last_undegraded":"2024-04-24T23:11:07.571671+0000","last_fullsized":"2024-04-24T23:11:07.571671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:11.106931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563033+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:11:07.563033+0000","last_peered":"2024-04-24T23:11:07.563033+0000","last_clean":"2024-04-24T23:11:07.563033+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:11:07.563033+0000","last_undegraded":"2024-04-24T23:11:07.563033+0000","last_fullsized":"2024-04-24T23:11:07.563033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567114+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:11:07.567114+0000","last_peered":"2024-04-24T23:11:07.567114+0000","last_clean":"2024-04-24T23:11:07.567114+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:11:07.567114+0000","last_undegraded":"2024-04-24T23:11:07.567114+0000","last_fullsized":"2024-04-24T23:11:07.567114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566285+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:11:07.566285+0000","last_peered":"2024-04-24T23:11:07.566285+0000","last_clean":"2024-04-24T23:11:07.566285+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:11:07.566285+0000","last_undegraded":"2024-04-24T23:11:07.566285+0000","last_fullsized":"2024-04-24T23:11:07.566285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562846+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:11:07.562846+0000","last_peered":"2024-04-24T23:11:07.562846+0000","last_clean":"2024-04-24T23:11:07.562846+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:11:07.562846+0000","last_undegraded":"2024-04-24T23:11:07.562846+0000","last_fullsized":"2024-04-24T23:11:07.562846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563789+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:11:07.563789+0000","last_peered":"2024-04-24T23:11:07.563789+0000","last_clean":"2024-04-24T23:11:07.563789+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:11:07.563789+0000","last_undegraded":"2024-04-24T23:11:07.563789+0000","last_fullsized":"2024-04-24T23:11:07.563789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568770+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:11:07.568770+0000","last_peered":"2024-04-24T23:11:07.568770+0000","last_clean":"2024-04-24T23:11:07.568770+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:11:07.568770+0000","last_undegraded":"2024-04-24T23:11:07.568770+0000","last_fullsized":"2024-04-24T23:11:07.568770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561646+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:11:07.561646+0000","last_peered":"2024-04-24T23:11:07.561646+0000","last_clean":"2024-04-24T23:11:07.561646+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:11:07.561646+0000","last_undegraded":"2024-04-24T23:11:07.561646+0000","last_fullsized":"2024-04-24T23:11:07.561646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569324+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:11:07.569324+0000","last_peered":"2024-04-24T23:11:07.569324+0000","last_clean":"2024-04-24T23:11:07.569324+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:11:07.569324+0000","last_undegraded":"2024-04-24T23:11:07.569324+0000","last_fullsized":"2024-04-24T23:11:07.569324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568462+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:11:07.568462+0000","last_peered":"2024-04-24T23:11:07.568462+0000","last_clean":"2024-04-24T23:11:07.568462+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:11:07.568462+0000","last_undegraded":"2024-04-24T23:11:07.568462+0000","last_fullsized":"2024-04-24T23:11:07.568462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761859+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:11:07.761859+0000","last_peered":"2024-04-24T23:11:07.761859+0000","last_clean":"2024-04-24T23:11:07.761859+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:11:07.761859+0000","last_undegraded":"2024-04-24T23:11:07.761859+0000","last_fullsized":"2024-04-24T23:11:07.761859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573753+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:11:07.573753+0000","last_peered":"2024-04-24T23:11:07.573753+0000","last_clean":"2024-04-24T23:11:07.573753+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:11:07.573753+0000","last_undegraded":"2024-04-24T23:11:07.573753+0000","last_fullsized":"2024-04-24T23:11:07.573753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761378+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:11:07.761378+0000","last_peered":"2024-04-24T23:11:07.761378+0000","last_clean":"2024-04-24T23:11:07.761378+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:11:07.761378+0000","last_undegraded":"2024-04-24T23:11:07.761378+0000","last_fullsized":"2024-04-24T23:11:07.761378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569647+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:11:07.569647+0000","last_peered":"2024-04-24T23:11:07.569647+0000","last_clean":"2024-04-24T23:11:07.569647+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:11:07.569647+0000","last_undegraded":"2024-04-24T23:11:07.569647+0000","last_fullsized":"2024-04-24T23:11:07.569647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561956+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:11:07.561956+0000","last_peered":"2024-04-24T23:11:07.561956+0000","last_clean":"2024-04-24T23:11:07.561956+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:11:07.561956+0000","last_undegraded":"2024-04-24T23:11:07.561956+0000","last_fullsized":"2024-04-24T23:11:07.561956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.561616+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:11:07.561616+0000","last_peered":"2024-04-24T23:11:07.561616+0000","last_clean":"2024-04-24T23:11:07.561616+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:11:07.561616+0000","last_undegraded":"2024-04-24T23:11:07.561616+0000","last_fullsized":"2024-04-24T23:11:07.561616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.568968+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:11:07.568968+0000","last_peered":"2024-04-24T23:11:07.568968+0000","last_clean":"2024-04-24T23:11:07.568968+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:11:07.568968+0000","last_undegraded":"2024-04-24T23:11:07.568968+0000","last_fullsized":"2024-04-24T23:11:07.568968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567063+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:11:07.567063+0000","last_peered":"2024-04-24T23:11:07.567063+0000","last_clean":"2024-04-24T23:11:07.567063+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:11:07.567063+0000","last_undegraded":"2024-04-24T23:11:07.567063+0000","last_fullsized":"2024-04-24T23:11:07.567063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761430+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:11:07.761430+0000","last_peered":"2024-04-24T23:11:07.761430+0000","last_clean":"2024-04-24T23:11:07.761430+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:11:07.761430+0000","last_undegraded":"2024-04-24T23:11:07.761430+0000","last_fullsized":"2024-04-24T23:11:07.761430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569406+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:11:07.569406+0000","last_peered":"2024-04-24T23:11:07.569406+0000","last_clean":"2024-04-24T23:11:07.569406+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:11:07.569406+0000","last_undegraded":"2024-04-24T23:11:07.569406+0000","last_fullsized":"2024-04-24T23:11:07.569406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569402+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:11:07.569402+0000","last_peered":"2024-04-24T23:11:07.569402+0000","last_clean":"2024-04-24T23:11:07.569402+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:11:07.569402+0000","last_undegraded":"2024-04-24T23:11:07.569402+0000","last_fullsized":"2024-04-24T23:11:07.569402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.761368+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:11:07.761368+0000","last_peered":"2024-04-24T23:11:07.761368+0000","last_clean":"2024-04-24T23:11:07.761368+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:11:07.761368+0000","last_undegraded":"2024-04-24T23:11:07.761368+0000","last_fullsized":"2024-04-24T23:11:07.761368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.562381+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:11:07.562381+0000","last_peered":"2024-04-24T23:11:07.562381+0000","last_clean":"2024-04-24T23:11:07.562381+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:11:07.562381+0000","last_undegraded":"2024-04-24T23:11:07.562381+0000","last_fullsized":"2024-04-24T23:11:07.562381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566130+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:11:07.566130+0000","last_peered":"2024-04-24T23:11:07.566130+0000","last_clean":"2024-04-24T23:11:07.566130+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:11:07.566130+0000","last_undegraded":"2024-04-24T23:11:07.566130+0000","last_fullsized":"2024-04-24T23:11:07.566130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569856+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:11:07.569856+0000","last_peered":"2024-04-24T23:11:07.569856+0000","last_clean":"2024-04-24T23:11:07.569856+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:11:07.569856+0000","last_undegraded":"2024-04-24T23:11:07.569856+0000","last_fullsized":"2024-04-24T23:11:07.569856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.579626+0000","last_change":"2024-04-24T23:11:07.579626+0000","last_active":"2024-04-24T23:11:07.579626+0000","last_peered":"2024-04-24T23:11:07.579626+0000","last_clean":"2024-04-24T23:11:07.579626+0000","last_became_active":"2024-04-24T23:11:07.574907+0000","last_became_peered":"2024-04-24T23:11:07.574907+0000","last_unstale":"2024-04-24T23:11:07.579626+0000","last_undegraded":"2024-04-24T23:11:07.579626+0000","last_fullsized":"2024-04-24T23:11:07.579626+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:56.428190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566713+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:11:07.566713+0000","last_peered":"2024-04-24T23:11:07.566713+0000","last_clean":"2024-04-24T23:11:07.566713+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:11:07.566713+0000","last_undegraded":"2024-04-24T23:11:07.566713+0000","last_fullsized":"2024-04-24T23:11:07.566713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573251+0000","last_change":"2024-04-24T23:10:16.387682+0000","last_active":"2024-04-24T23:11:07.573251+0000","last_peered":"2024-04-24T23:11:07.573251+0000","last_clean":"2024-04-24T23:11:07.573251+0000","last_became_active":"2024-04-24T23:10:16.387248+0000","last_became_peered":"2024-04-24T23:10:16.387248+0000","last_unstale":"2024-04-24T23:11:07.573251+0000","last_undegraded":"2024-04-24T23:11:07.573251+0000","last_fullsized":"2024-04-24T23:11:07.573251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:54.837632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566107+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:11:07.566107+0000","last_peered":"2024-04-24T23:11:07.566107+0000","last_clean":"2024-04-24T23:11:07.566107+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:11:07.566107+0000","last_undegraded":"2024-04-24T23:11:07.566107+0000","last_fullsized":"2024-04-24T23:11:07.566107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566083+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:11:07.566083+0000","last_peered":"2024-04-24T23:11:07.566083+0000","last_clean":"2024-04-24T23:11:07.566083+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:11:07.566083+0000","last_undegraded":"2024-04-24T23:11:07.566083+0000","last_fullsized":"2024-04-24T23:11:07.566083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563867+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:11:07.563867+0000","last_peered":"2024-04-24T23:11:07.563867+0000","last_clean":"2024-04-24T23:11:07.563867+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:11:07.563867+0000","last_undegraded":"2024-04-24T23:11:07.563867+0000","last_fullsized":"2024-04-24T23:11:07.563867+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563781+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:11:07.563781+0000","last_peered":"2024-04-24T23:11:07.563781+0000","last_clean":"2024-04-24T23:11:07.563781+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:11:07.563781+0000","last_undegraded":"2024-04-24T23:11:07.563781+0000","last_fullsized":"2024-04-24T23:11:07.563781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566491+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:11:07.566491+0000","last_peered":"2024-04-24T23:11:07.566491+0000","last_clean":"2024-04-24T23:11:07.566491+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:11:07.566491+0000","last_undegraded":"2024-04-24T23:11:07.566491+0000","last_fullsized":"2024-04-24T23:11:07.566491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.567186+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:11:07.567186+0000","last_peered":"2024-04-24T23:11:07.567186+0000","last_clean":"2024-04-24T23:11:07.567186+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:11:07.567186+0000","last_undegraded":"2024-04-24T23:11:07.567186+0000","last_fullsized":"2024-04-24T23:11:07.567186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569014+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:11:07.569014+0000","last_peered":"2024-04-24T23:11:07.569014+0000","last_clean":"2024-04-24T23:11:07.569014+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:11:07.569014+0000","last_undegraded":"2024-04-24T23:11:07.569014+0000","last_fullsized":"2024-04-24T23:11:07.569014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569275+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:11:07.569275+0000","last_peered":"2024-04-24T23:11:07.569275+0000","last_clean":"2024-04-24T23:11:07.569275+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:11:07.569275+0000","last_undegraded":"2024-04-24T23:11:07.569275+0000","last_fullsized":"2024-04-24T23:11:07.569275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.573422+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:11:07.573422+0000","last_peered":"2024-04-24T23:11:07.573422+0000","last_clean":"2024-04-24T23:11:07.573422+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:11:07.573422+0000","last_undegraded":"2024-04-24T23:11:07.573422+0000","last_fullsized":"2024-04-24T23:11:07.573422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563925+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:11:07.563925+0000","last_peered":"2024-04-24T23:11:07.563925+0000","last_clean":"2024-04-24T23:11:07.563925+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:11:07.563925+0000","last_undegraded":"2024-04-24T23:11:07.563925+0000","last_fullsized":"2024-04-24T23:11:07.563925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566415+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:11:07.566415+0000","last_peered":"2024-04-24T23:11:07.566415+0000","last_clean":"2024-04-24T23:11:07.566415+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:11:07.566415+0000","last_undegraded":"2024-04-24T23:11:07.566415+0000","last_fullsized":"2024-04-24T23:11:07.566415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569421+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:11:07.569421+0000","last_peered":"2024-04-24T23:11:07.569421+0000","last_clean":"2024-04-24T23:11:07.569421+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:11:07.569421+0000","last_undegraded":"2024-04-24T23:11:07.569421+0000","last_fullsized":"2024-04-24T23:11:07.569421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569664+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:11:07.569664+0000","last_peered":"2024-04-24T23:11:07.569664+0000","last_clean":"2024-04-24T23:11:07.569664+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:11:07.569664+0000","last_undegraded":"2024-04-24T23:11:07.569664+0000","last_fullsized":"2024-04-24T23:11:07.569664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.564019+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:11:07.564019+0000","last_peered":"2024-04-24T23:11:07.564019+0000","last_clean":"2024-04-24T23:11:07.564019+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:11:07.564019+0000","last_undegraded":"2024-04-24T23:11:07.564019+0000","last_fullsized":"2024-04-24T23:11:07.564019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.569084+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:11:07.569084+0000","last_peered":"2024-04-24T23:11:07.569084+0000","last_clean":"2024-04-24T23:11:07.569084+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:11:07.569084+0000","last_undegraded":"2024-04-24T23:11:07.569084+0000","last_fullsized":"2024-04-24T23:11:07.569084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.570190+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:11:07.570190+0000","last_peered":"2024-04-24T23:11:07.570190+0000","last_clean":"2024-04-24T23:11:07.570190+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:11:07.570190+0000","last_undegraded":"2024-04-24T23:11:07.570190+0000","last_fullsized":"2024-04-24T23:11:07.570190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.563999+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:11:07.563999+0000","last_peered":"2024-04-24T23:11:07.563999+0000","last_clean":"2024-04-24T23:11:07.563999+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:11:07.563999+0000","last_undegraded":"2024-04-24T23:11:07.563999+0000","last_fullsized":"2024-04-24T23:11:07.563999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:11:07.566686+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:11:07.566686+0000","last_peered":"2024-04-24T23:11:07.566686+0000","last_clean":"2024-04-24T23:11:07.566686+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:11:07.566686+0000","last_undegraded":"2024-04-24T23:11:07.566686+0000","last_fullsized":"2024-04-24T23:11:07.566686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65776,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":475,"num_read_kb":303,"num_write":159,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1198,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":197328,"data_compressed":4821,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":503892,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1153292,"data_compressed":58843,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":508190,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":503432,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":1099923,"data_compressed":57236,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499270,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":561537,"data_compressed":22308,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500658,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":561689,"data_compressed":22308,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":499464,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":504893,"data_compressed":20701,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1095120,"data_compressed":57236,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62357,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62509,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63744,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:11:56.862 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T23:11:56.863 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:11:56.863 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:11:56.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T23:11:57.116 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:57 smithi033 ceph-mon[29201]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:57 smithi033 ceph-mon[29201]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:58.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:57 smithi038 ceph-mon[32450]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:57 smithi038 ceph-mon[32450]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:58.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:57 smithi117 ceph-mon[32529]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:11:58.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:57 smithi117 ceph-mon[32529]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 -- 172.21.15.33:0/3587893170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c102e90 msgr2=0x7f104c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 --2- 172.21.15.33:0/3587893170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c102e90 0x7f104c106e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1034009990 tx=0x7f1034009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 -- 172.21.15.33:0/3587893170 shutdown_connections 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 --2- 172.21.15.33:0/3587893170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104c107570 0x7f104c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 --2- 172.21.15.33:0/3587893170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f104c1024e0 0x7f104c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 --2- 172.21.15.33:0/3587893170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c102e90 0x7f104c106e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 -- 172.21.15.33:0/3587893170 >> 172.21.15.33:0/3587893170 conn(0x7f104c079390 msgr2=0x7f104c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 -- 172.21.15.33:0/3587893170 shutdown_connections 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.937+0000 7f1052132700 1 -- 172.21.15.33:0/3587893170 wait complete. 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.938+0000 7f1052132700 1 Processor -- start 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.938+0000 7f1052132700 1 -- start start 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1052132700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f104c1024e0 0x7f104c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1052132700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104c102e90 0x7f104c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1052132700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 0x7f104c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1052132700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f104c112510 con 0x7f104c107570 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1052132700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f104c112390 con 0x7f104c102e90 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1052132700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f104c112690 con 0x7f104c1024e0 2024-04-24T23:11:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 0x7f104c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 0x7f104c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49098/0 (socket says 172.21.15.33:49098) 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 -- 172.21.15.33:0/827058869 learned_addr learned my addr 172.21.15.33:0/827058869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104b7fe700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f104c1024e0 0x7f104c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104affd700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104c102e90 0x7f104c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 -- 172.21.15.33:0/827058869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f104c1024e0 msgr2=0x7f104c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f104c1024e0 0x7f104c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 -- 172.21.15.33:0/827058869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104c102e90 msgr2=0x7f104c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104c102e90 0x7f104c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 -- 172.21.15.33:0/827058869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c199030 con 0x7f104c107570 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f104bfff700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 0x7f104c198850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f104400ada0 tx=0x7f104400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.939+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044017020 con 0x7f104c107570 2024-04-24T23:11:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.940+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c199320 con 0x7f104c107570 2024-04-24T23:11:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.940+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f104400cdb0 con 0x7f104c107570 2024-04-24T23:11:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.940+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1a54e0 con 0x7f104c107570 2024-04-24T23:11:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.940+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440053e0 con 0x7f104c107570 2024-04-24T23:11:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.941+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c103970 con 0x7f104c107570 2024-04-24T23:11:58.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.942+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f104400c8d0 con 0x7f104c107570 2024-04-24T23:11:58.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.942+0000 7f1048ff9700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f10380a2d30 0x7f10380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:58.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.942+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f10440ceb10 con 0x7f104c107570 2024-04-24T23:11:58.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.942+0000 7f104b7fe700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f10380a2d30 0x7f10380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:58.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.943+0000 7f104b7fe700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f10380a2d30 0x7f10380a51f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:58.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:58.947+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1044021020 con 0x7f104c107570 2024-04-24T23:11:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:58 smithi033 ceph-mon[29201]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:59.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:58 smithi038 ceph-mon[32450]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:59.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:58 smithi117 ceph-mon[32529]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:11:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.204+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f104c02db30 con 0x7f104c107570 2024-04-24T23:11:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.204+0000 7f1048ff9700 1 -- 172.21.15.33:0/827058869 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f104409b640 con 0x7f104c107570 2024-04-24T23:11:59.205 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:11:59.205 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:11:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f10380a2d30 msgr2=0x7f10380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f10380a2d30 0x7f10380a51f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c00b410 comp rx=0 tx=0).stop 2024-04-24T23:11:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 msgr2=0x7f104c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 0x7f104c198850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f104400ada0 tx=0x7f104400c330 comp rx=0 tx=0).stop 2024-04-24T23:11:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 shutdown_connections 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f10380a2d30 0x7f10380a51f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c107570 0x7f104c198850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f104c1024e0 0x7f104c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 --2- 172.21.15.33:0/827058869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f104c102e90 0x7f104c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 >> 172.21.15.33:0/827058869 conn(0x7f104c079390 msgr2=0x7f104c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 shutdown_connections 2024-04-24T23:11:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:11:59.206+0000 7f1052132700 1 -- 172.21.15.33:0/827058869 wait complete. 2024-04-24T23:11:59.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:11:59 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/827058869' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:11:59.930 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:11:59.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:00.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 -- 172.21.15.33:0/3167478853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58101870 msgr2=0x7fbf58101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 --2- 172.21.15.33:0/3167478853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58101870 0x7fbf58101cf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009900 tx=0x7fbf4c02a3f0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 -- 172.21.15.33:0/3167478853 shutdown_connections 2024-04-24T23:12:00.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 --2- 172.21.15.33:0/3167478853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf58102230 0x7fbf58108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 --2- 172.21.15.33:0/3167478853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58101870 0x7fbf58101cf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 --2- 172.21.15.33:0/3167478853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58100680 0x7fbf58100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 -- 172.21.15.33:0/3167478853 >> 172.21.15.33:0/3167478853 conn(0x7fbf580fbbf0 msgr2=0x7fbf580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 -- 172.21.15.33:0/3167478853 shutdown_connections 2024-04-24T23:12:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.042+0000 7fbf5d761640 1 -- 172.21.15.33:0/3167478853 wait complete. 2024-04-24T23:12:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 Processor -- start 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 -- start start 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf58100680 0x7fbf58199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58101870 0x7fbf5819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 0x7fbf581a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810c2e0 con 0x7fbf58102230 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810c160 con 0x7fbf58100680 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf5d761640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810c460 con 0x7fbf58101870 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 0x7fbf581a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 0x7fbf581a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49110/0 (socket says 172.21.15.33:49110) 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf577fe640 1 -- 172.21.15.33:0/2099770934 learned_addr learned my addr 172.21.15.33:0/2099770934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf567fc640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58101870 0x7fbf5819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.043+0000 7fbf56ffd640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf58100680 0x7fbf58199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf577fe640 1 -- 172.21.15.33:0/2099770934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58101870 msgr2=0x7fbf5819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf577fe640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58101870 0x7fbf5819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf577fe640 1 -- 172.21.15.33:0/2099770934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf58100680 msgr2=0x7fbf58199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf577fe640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf58100680 0x7fbf58199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf577fe640 1 -- 172.21.15.33:0/2099770934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf581a18b0 con 0x7fbf58102230 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf577fe640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 0x7fbf581a1210 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000cad0 tx=0x7fbf5000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000ae20 con 0x7fbf58102230 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf50005020 con 0x7fbf58102230 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000f6f0 con 0x7fbf58102230 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581a1ba0 con 0x7fbf58102230 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.044+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a2070 con 0x7fbf58102230 2024-04-24T23:12:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.045+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf581a22a0 con 0x7fbf58102230 2024-04-24T23:12:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.046+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbf500051d0 con 0x7fbf58102230 2024-04-24T23:12:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.047+0000 7fbf3ffff640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf400a2c20 0x7fbf400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.047+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbf500cf6a0 con 0x7fbf58102230 2024-04-24T23:12:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.047+0000 7fbf56ffd640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf400a2c20 0x7fbf400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.047+0000 7fbf56ffd640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf400a2c20 0x7fbf400a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf480058c0 tx=0x7fbf48005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.051+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbf50014030 con 0x7fbf58102230 2024-04-24T23:12:00.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:11:59 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/827058869' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:12:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:11:59 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/827058869' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:12:00.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.238+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf581a2510 con 0x7fbf58102230 2024-04-24T23:12:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.239+0000 7fbf3ffff640 1 -- 172.21.15.33:0/2099770934 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24620 (secure 0 0 0) 0x7fbf5009c140 con 0x7fbf58102230 2024-04-24T23:12:00.240 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:00.241 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:11:07.557817+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf400a2c20 msgr2=0x7fbf400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf400a2c20 0x7fbf400a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf480058c0 tx=0x7fbf48005850 comp rx=0 tx=0).stop 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 msgr2=0x7fbf581a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 0x7fbf581a1210 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000cad0 tx=0x7fbf5000cf90 comp rx=0 tx=0).stop 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 shutdown_connections 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf400a2c20 0x7fbf400a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf58102230 0x7fbf581a1210 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58101870 0x7fbf5819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 --2- 172.21.15.33:0/2099770934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf58100680 0x7fbf58199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.241+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 >> 172.21.15.33:0/2099770934 conn(0x7fbf580fbbf0 msgr2=0x7fbf580fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.242+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 shutdown_connections 2024-04-24T23:12:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.242+0000 7fbf5d761640 1 -- 172.21.15.33:0/2099770934 wait complete. 2024-04-24T23:12:00.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:00.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:00.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 -- 172.21.15.33:0/2685381755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980102980 msgr2=0x7f5980102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 --2- 172.21.15.33:0/2685381755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980102980 0x7f5980102e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5974009900 tx=0x7f597402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 -- 172.21.15.33:0/2685381755 shutdown_connections 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 --2- 172.21.15.33:0/2685381755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5980103340 0x7f5980109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 --2- 172.21.15.33:0/2685381755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980102980 0x7f5980102e00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 --2- 172.21.15.33:0/2685381755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5980101790 0x7f5980101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 -- 172.21.15.33:0/2685381755 >> 172.21.15.33:0/2685381755 conn(0x7f59800fcd20 msgr2=0x7f59800ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 -- 172.21.15.33:0/2685381755 shutdown_connections 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.359+0000 7f5986622640 1 -- 172.21.15.33:0/2685381755 wait complete. 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 Processor -- start 2024-04-24T23:12:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 -- start start 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5980101790 0x7f5980199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5980102980 0x7f598019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 0x7f59801a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f598010d3f0 con 0x7f5980103340 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f598010d270 con 0x7f5980102980 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5986622640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f598010d570 con 0x7f5980101790 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5984bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 0x7f59801a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5984bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 0x7f59801a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49138/0 (socket says 172.21.15.33:49138) 2024-04-24T23:12:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f5984bbf640 1 -- 172.21.15.33:0/3468585077 learned_addr learned my addr 172.21.15.33:0/3468585077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f597ffff640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5980101790 0x7f5980199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.360+0000 7f597f7fe640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5980102980 0x7f598019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5984bbf640 1 -- 172.21.15.33:0/3468585077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5980101790 msgr2=0x7f5980199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5984bbf640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5980101790 0x7f5980199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5984bbf640 1 -- 172.21.15.33:0/3468585077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5980102980 msgr2=0x7f598019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5984bbf640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5980102980 0x7f598019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5984bbf640 1 -- 172.21.15.33:0/3468585077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59801a18f0 con 0x7f5980103340 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5984bbf640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 0x7f59801a1250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f597800eaa0 tx=0x7f597800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597800cdb0 con 0x7f5980103340 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5978005020 con 0x7f5980103340 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59801a1be0 con 0x7f5980103340 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59780116b0 con 0x7f5980103340 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.361+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59801a20b0 con 0x7f5980103340 2024-04-24T23:12:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.362+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59801a22e0 con 0x7f5980103340 2024-04-24T23:12:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.363+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5978005560 con 0x7f5980103340 2024-04-24T23:12:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.364+0000 7f597d7fa640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59680a2c20 0x7f59680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.364+0000 7f597ffff640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59680a2c20 0x7f59680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.364+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f59780cfb80 con 0x7f5980103340 2024-04-24T23:12:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.365+0000 7f597ffff640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59680a2c20 0x7f59680a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f598019ab80 tx=0x7f5970005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.368+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5978016030 con 0x7f5980103340 2024-04-24T23:12:00.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.555+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59801a24b0 con 0x7f5980103340 2024-04-24T23:12:00.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.555+0000 7f597d7fa640 1 -- 172.21.15.33:0/3468585077 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f597809c5c0 con 0x7f5980103340 2024-04-24T23:12:00.556 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:00.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.557+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59680a2c20 msgr2=0x7f59680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.557+0000 7f5986622640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59680a2c20 0x7f59680a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f598019ab80 tx=0x7f5970005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.557+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 msgr2=0x7f59801a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.557+0000 7f5986622640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 0x7f59801a1250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f597800eaa0 tx=0x7f597800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 shutdown_connections 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59680a2c20 0x7f59680a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5980103340 0x7f59801a1250 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5980102980 0x7f598019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 --2- 172.21.15.33:0/3468585077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5980101790 0x7f5980199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 >> 172.21.15.33:0/3468585077 conn(0x7f59800fcd20 msgr2=0x7f5980108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 shutdown_connections 2024-04-24T23:12:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.558+0000 7f5986622640 1 -- 172.21.15.33:0/3468585077 wait complete. 2024-04-24T23:12:00.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 -- 172.21.15.33:0/436145162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f01017a0 msgr2=0x7f14f0101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 --2- 172.21.15.33:0/436145162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f01017a0 0x7f14f0101bc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 -- 172.21.15.33:0/436145162 shutdown_connections 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 --2- 172.21.15.33:0/436145162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f0103350 0x7f14f0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 --2- 172.21.15.33:0/436145162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14f0102990 0x7f14f0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 --2- 172.21.15.33:0/436145162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f01017a0 0x7f14f0101bc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 -- 172.21.15.33:0/436145162 >> 172.21.15.33:0/436145162 conn(0x7f14f00fcd10 msgr2=0x7f14f00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 -- 172.21.15.33:0/436145162 shutdown_connections 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.674+0000 7f14f72fc640 1 -- 172.21.15.33:0/436145162 wait complete. 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 Processor -- start 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 -- start start 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14f01017a0 0x7f14f00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f0102990 0x7f14f0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 0x7f14f0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14f010d470 con 0x7f14f0103350 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14f010d2f0 con 0x7f14f01017a0 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f72fc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14f010d5f0 con 0x7f14f0102990 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 0x7f14f0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 0x7f14f0074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49168/0 (socket says 172.21.15.33:49168) 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 -- 172.21.15.33:0/1839034265 learned_addr learned my addr 172.21.15.33:0/1839034265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f4897640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f0102990 0x7f14f0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5098640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14f01017a0 0x7f14f00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 -- 172.21.15.33:0/1839034265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f0102990 msgr2=0x7f14f0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f0102990 0x7f14f0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 -- 172.21.15.33:0/1839034265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14f01017a0 msgr2=0x7f14f00754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14f01017a0 0x7f14f00754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.675+0000 7f14f5899640 1 -- 172.21.15.33:0/1839034265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f0074900 con 0x7f14f0103350 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.676+0000 7f14f5899640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 0x7f14f0074070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f14e800eaa0 tx=0x7f14e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.676+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800cdb0 con 0x7f14f0103350 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.676+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8005020 con 0x7f14f0103350 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.676+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e80116b0 con 0x7f14f0103350 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.676+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f01a1bd0 con 0x7f14f0103350 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.676+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f01a2120 con 0x7f14f0103350 2024-04-24T23:12:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.677+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f0108ae0 con 0x7f14f0103350 2024-04-24T23:12:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.678+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f14e8005560 con 0x7f14f0103350 2024-04-24T23:12:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.679+0000 7f14e67fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14dc0a2c20 0x7f14dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.679+0000 7f14f5098640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14dc0a2c20 0x7f14dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.679+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f14e80cf7f0 con 0x7f14f0103350 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.679+0000 7f14f5098640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14dc0a2c20 0x7f14dc0a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14ec02aa00 tx=0x7f14ec031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.683+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14e8016030 con 0x7f14f0103350 2024-04-24T23:12:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.869+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14f002d3c0 con 0x7f14f0103350 2024-04-24T23:12:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.870+0000 7f14e67fc640 1 -- 172.21.15.33:0/1839034265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f14e809c230 con 0x7f14f0103350 2024-04-24T23:12:00.871 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14dc0a2c20 msgr2=0x7f14dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14dc0a2c20 0x7f14dc0a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14ec02aa00 tx=0x7f14ec031400 comp rx=0 tx=0).stop 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 msgr2=0x7f14f0074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 0x7f14f0074070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f14e800eaa0 tx=0x7f14e800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 shutdown_connections 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14dc0a2c20 0x7f14dc0a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14f0103350 0x7f14f0074070 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f0102990 0x7f14f0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 --2- 172.21.15.33:0/1839034265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14f01017a0 0x7f14f00754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 >> 172.21.15.33:0/1839034265 conn(0x7f14f00fcd10 msgr2=0x7f14f0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 shutdown_connections 2024-04-24T23:12:00.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.872+0000 7f14f72fc640 1 -- 172.21.15.33:0/1839034265 wait complete. 2024-04-24T23:12:00.883 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:00 smithi033 ceph-mon[29201]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:00.883 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:00 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2099770934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:00.883 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:00 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3468585077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:00.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 -- 172.21.15.33:0/3402448380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 msgr2=0x7fe1ac108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 --2- 172.21.15.33:0/3402448380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac108ae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009ca0 tx=0x7fe1a402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 -- 172.21.15.33:0/3402448380 shutdown_connections 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 --2- 172.21.15.33:0/3402448380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac108ae0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 --2- 172.21.15.33:0/3402448380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac101890 0x7fe1ac101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 --2- 172.21.15.33:0/3402448380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ac1006a0 0x7fe1ac100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 -- 172.21.15.33:0/3402448380 >> 172.21.15.33:0/3402448380 conn(0x7fe1ac0fbc10 msgr2=0x7fe1ac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 -- 172.21.15.33:0/3402448380 shutdown_connections 2024-04-24T23:12:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.988+0000 7fe1b3076640 1 -- 172.21.15.33:0/3402448380 wait complete. 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 Processor -- start 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 -- start start 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac1006a0 0x7fe1ac06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ac101890 0x7fe1ac19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10c410 con 0x7fe1ac102250 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10c290 con 0x7fe1ac101890 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b3076640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10c590 con 0x7fe1ac1006a0 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49186/0 (socket says 172.21.15.33:49186) 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 -- 172.21.15.33:0/332973532 learned_addr learned my addr 172.21.15.33:0/332973532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1abfff640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ac101890 0x7fe1ac19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b0e12640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac1006a0 0x7fe1ac06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 -- 172.21.15.33:0/332973532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac1006a0 msgr2=0x7fe1ac06a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac1006a0 0x7fe1ac06a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 -- 172.21.15.33:0/332973532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ac101890 msgr2=0x7fe1ac19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ac101890 0x7fe1ac19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 -- 172.21.15.33:0/332973532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac19f840 con 0x7fe1ac102250 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b0e12640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac1006a0 0x7fe1ac06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.989+0000 7fe1b1613640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac19ef30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40098d0 tx=0x7fe1a402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.990+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a402fc30 con 0x7fe1ac102250 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.990+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac19fa70 con 0x7fe1ac102250 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.990+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a4003e50 con 0x7fe1ac102250 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.990+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4004810 con 0x7fe1ac102250 2024-04-24T23:12:00.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.990+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1a0040 con 0x7fe1ac102250 2024-04-24T23:12:00.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.991+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac107a30 con 0x7fe1ac102250 2024-04-24T23:12:00.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.992+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe1a4003970 con 0x7fe1ac102250 2024-04-24T23:12:00.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.993+0000 7fe1a9ffb640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe19c0a2c20 0x7fe19c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.993+0000 7fe1b0e12640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe19c0a2c20 0x7fe19c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.993+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe1a40f0930 con 0x7fe1ac102250 2024-04-24T23:12:00.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.993+0000 7fe1b0e12640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe19c0a2c20 0x7fe19c0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006fd0 tx=0x7fe1a0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:00.998+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1a40bd600 con 0x7fe1ac102250 2024-04-24T23:12:01.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:00 smithi038 ceph-mon[32450]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:01.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:00 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2099770934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:01.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:00 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3468585077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:00 smithi117 ceph-mon[32529]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:00 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2099770934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:00 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3468585077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:01.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.184+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1ac02d3c0 con 0x7fe1ac102250 2024-04-24T23:12:01.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.185+0000 7fe1a9ffb640 1 -- 172.21.15.33:0/332973532 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fe1a40bcf00 con 0x7fe1ac102250 2024-04-24T23:12:01.185 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe19c0a2c20 msgr2=0x7fe19c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe19c0a2c20 0x7fe19c0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006fd0 tx=0x7fe1a0008040 comp rx=0 tx=0).stop 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 msgr2=0x7fe1ac19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac19ef30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40098d0 tx=0x7fe1a402f840 comp rx=0 tx=0).stop 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 shutdown_connections 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe19c0a2c20 0x7fe19c0a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ac102250 0x7fe1ac19ef30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ac101890 0x7fe1ac19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 --2- 172.21.15.33:0/332973532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ac1006a0 0x7fe1ac06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 >> 172.21.15.33:0/332973532 conn(0x7fe1ac0fbc10 msgr2=0x7fe1ac104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 shutdown_connections 2024-04-24T23:12:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.187+0000 7fe1b3076640 1 -- 172.21.15.33:0/332973532 wait complete. 2024-04-24T23:12:01.199 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:01.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 -- 172.21.15.33:0/3485111865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2730100050 msgr2=0x7f2730100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 --2- 172.21.15.33:0/3485111865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2730100050 0x7f2730100470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f272c009900 tx=0x7f272c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 -- 172.21.15.33:0/3485111865 shutdown_connections 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 --2- 172.21.15.33:0/3485111865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2730101350 0x7f27301017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 --2- 172.21.15.33:0/3485111865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301009b0 0x7f2730100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 --2- 172.21.15.33:0/3485111865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2730100050 0x7f2730100470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 -- 172.21.15.33:0/3485111865 >> 172.21.15.33:0/3485111865 conn(0x7f27300fbbf0 msgr2=0x7f27300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 -- 172.21.15.33:0/3485111865 shutdown_connections 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 -- 172.21.15.33:0/3485111865 wait complete. 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.305+0000 7f273841c640 1 Processor -- start 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 -- start start 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2730100050 0x7f2730199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27301009b0 0x7f273019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 0x7f27301a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f273010f630 con 0x7f2730101350 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f273010f4b0 con 0x7f27301009b0 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f273010f7b0 con 0x7f2730100050 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 0x7f27301a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 0x7f27301a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49200/0 (socket says 172.21.15.33:49200) 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 -- 172.21.15.33:0/2802419598 learned_addr learned my addr 172.21.15.33:0/2802419598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27359b7640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27301009b0 0x7f273019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27361b8640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2730100050 0x7f2730199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 -- 172.21.15.33:0/2802419598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2730100050 msgr2=0x7f2730199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2730100050 0x7f2730199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 -- 172.21.15.33:0/2802419598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27301009b0 msgr2=0x7f273019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27301009b0 0x7f273019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 -- 172.21.15.33:0/2802419598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301a18e0 con 0x7f2730101350 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27369b9640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 0x7f27301a1240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f272800eaa0 tx=0x7f272800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272800cdb0 con 0x7f2730101350 2024-04-24T23:12:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27301a1bd0 con 0x7f2730101350 2024-04-24T23:12:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2728005020 con 0x7f2730101350 2024-04-24T23:12:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.306+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280116d0 con 0x7f2730101350 2024-04-24T23:12:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.307+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301a2170 con 0x7f2730101350 2024-04-24T23:12:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.308+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273010af40 con 0x7f2730101350 2024-04-24T23:12:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.309+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27280054d0 con 0x7f2730101350 2024-04-24T23:12:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.309+0000 7f27277fe640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f271c0a2c20 0x7f271c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.309+0000 7f27361b8640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f271c0a2c20 0x7f271c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.309+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f27280cf750 con 0x7f2730101350 2024-04-24T23:12:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.310+0000 7f27361b8640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f271c0a2c20 0x7f271c0a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f273019ab70 tx=0x7f272c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.314+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2728016030 con 0x7f2730101350 2024-04-24T23:12:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.500+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f273002d320 con 0x7f2730101350 2024-04-24T23:12:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.500+0000 7f27277fe640 1 -- 172.21.15.33:0/2802419598 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f272809c1f0 con 0x7f2730101350 2024-04-24T23:12:01.501 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f271c0a2c20 msgr2=0x7f271c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f271c0a2c20 0x7f271c0a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f273019ab70 tx=0x7f272c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 msgr2=0x7f27301a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 0x7f27301a1240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f272800eaa0 tx=0x7f272800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 shutdown_connections 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f271c0a2c20 0x7f271c0a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2730101350 0x7f27301a1240 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27301009b0 0x7f273019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 --2- 172.21.15.33:0/2802419598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2730100050 0x7f2730199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.502+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 >> 172.21.15.33:0/2802419598 conn(0x7f27300fbbf0 msgr2=0x7f27300fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.503+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 shutdown_connections 2024-04-24T23:12:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.503+0000 7f273841c640 1 -- 172.21.15.33:0/2802419598 wait complete. 2024-04-24T23:12:01.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 -- 172.21.15.33:0/128200937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008102230 msgr2=0x7ff008108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 --2- 172.21.15.33:0/128200937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008102230 0x7ff008108ac0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff000009ca0 tx=0x7ff00002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 -- 172.21.15.33:0/128200937 shutdown_connections 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 --2- 172.21.15.33:0/128200937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008102230 0x7ff008108ac0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 --2- 172.21.15.33:0/128200937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008101870 0x7ff008101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 --2- 172.21.15.33:0/128200937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008100680 0x7ff008100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.617+0000 7ff00f35a640 1 -- 172.21.15.33:0/128200937 >> 172.21.15.33:0/128200937 conn(0x7ff0080fbbf0 msgr2=0x7ff0080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 -- 172.21.15.33:0/128200937 shutdown_connections 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 -- 172.21.15.33:0/128200937 wait complete. 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 Processor -- start 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 -- start start 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 0x7ff008104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008101870 0x7ff008102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008102230 0x7ff008103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810c2e0 con 0x7ff008100680 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810c160 con 0x7ff008101870 2024-04-24T23:12:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00f35a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810c460 con 0x7ff008102230 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00d0f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 0x7ff008104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00d0f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 0x7ff008104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49212/0 (socket says 172.21.15.33:49212) 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.618+0000 7ff00d0f6640 1 -- 172.21.15.33:0/1089079708 learned_addr learned my addr 172.21.15.33:0/1089079708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d8f7640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008102230 0x7ff008103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00c8f5640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008101870 0x7ff008102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d0f6640 1 -- 172.21.15.33:0/1089079708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008102230 msgr2=0x7ff008103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d0f6640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008102230 0x7ff008103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d0f6640 1 -- 172.21.15.33:0/1089079708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008101870 msgr2=0x7ff008102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d0f6640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008101870 0x7ff008102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d0f6640 1 -- 172.21.15.33:0/1089079708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff008103a20 con 0x7ff008100680 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00d0f6640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 0x7ff008104610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400cd80 con 0x7ff008100680 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0040035f0 con 0x7ff008100680 2024-04-24T23:12:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004010690 con 0x7ff008100680 2024-04-24T23:12:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0081a1b90 con 0x7ff008100680 2024-04-24T23:12:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.619+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a2130 con 0x7ff008100680 2024-04-24T23:12:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.620+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0081a2300 con 0x7ff008100680 2024-04-24T23:12:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.622+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff0040037a0 con 0x7ff008100680 2024-04-24T23:12:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.623+0000 7feffe7fc640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7feff40a2ca0 0x7feff40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.623+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff0040cfae0 con 0x7ff008100680 2024-04-24T23:12:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.623+0000 7ff00c8f5640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7feff40a2ca0 0x7feff40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.623+0000 7ff00c8f5640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7feff40a2ca0 0x7feff40a5160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feff8005fd0 tx=0x7feff8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.627+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff00409c610 con 0x7ff008100680 2024-04-24T23:12:01.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.856+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0081a2640 con 0x7ff008100680 2024-04-24T23:12:01.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.857+0000 7feffe7fc640 1 -- 172.21.15.33:0/1089079708 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7ff00409bf10 con 0x7ff008100680 2024-04-24T23:12:01.858 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:01.858 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:38.464642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:12:01.859 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:12:01.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7feff40a2ca0 msgr2=0x7feff40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7feff40a2ca0 0x7feff40a5160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feff8005fd0 tx=0x7feff8005850 comp rx=0 tx=0).stop 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 msgr2=0x7ff008104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 0x7ff008104610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).stop 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 shutdown_connections 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7feff40a2ca0 0x7feff40a5160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008102230 0x7ff008103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008101870 0x7ff008102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 --2- 172.21.15.33:0/1089079708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff008100680 0x7ff008104610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.859+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 >> 172.21.15.33:0/1089079708 conn(0x7ff0080fbbf0 msgr2=0x7ff0080fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.860+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 shutdown_connections 2024-04-24T23:12:01.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.860+0000 7ff00f35a640 1 -- 172.21.15.33:0/1089079708 wait complete. 2024-04-24T23:12:01.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:01.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 -- 172.21.15.33:0/2022627374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 msgr2=0x7fb1ac101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 --2- 172.21.15.33:0/2022627374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac101cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb194009900 tx=0x7fb19402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:01.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 -- 172.21.15.33:0/2022627374 shutdown_connections 2024-04-24T23:12:01.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 --2- 172.21.15.33:0/2022627374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac102230 0x7fb1ac108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 --2- 172.21.15.33:0/2022627374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac101cf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 --2- 172.21.15.33:0/2022627374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 0x7fb1ac100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 -- 172.21.15.33:0/2022627374 >> 172.21.15.33:0/2022627374 conn(0x7fb1ac0fbbf0 msgr2=0x7fb1ac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 -- 172.21.15.33:0/2022627374 shutdown_connections 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.975+0000 7fb1b237e640 1 -- 172.21.15.33:0/2022627374 wait complete. 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 Processor -- start 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 -- start start 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 0x7fb1ac199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac102230 0x7fb1ac1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10d460 con 0x7fb1ac101870 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10d2e0 con 0x7fb1ac100680 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b237e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10d5e0 con 0x7fb1ac102230 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49228/0 (socket says 172.21.15.33:49228) 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1a3fff640 1 -- 172.21.15.33:0/167787285 learned_addr learned my addr 172.21.15.33:0/167787285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1abfff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 0x7fb1ac199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.976+0000 7fb1b091b640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac102230 0x7fb1ac1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a3fff640 1 -- 172.21.15.33:0/167787285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac102230 msgr2=0x7fb1ac1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a3fff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac102230 0x7fb1ac1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a3fff640 1 -- 172.21.15.33:0/167787285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 msgr2=0x7fb1ac199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a3fff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 0x7fb1ac199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a3fff640 1 -- 172.21.15.33:0/167787285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac1a1940 con 0x7fb1ac101870 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1abfff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 0x7fb1ac199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a3fff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac19a270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb194009230 tx=0x7fb194004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19403f030 con 0x7fb1ac101870 2024-04-24T23:12:01.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb194003ce0 con 0x7fb1ac101870 2024-04-24T23:12:01.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac1a1bd0 con 0x7fb1ac101870 2024-04-24T23:12:01.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194035070 con 0x7fb1ac101870 2024-04-24T23:12:01.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.977+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac1a2110 con 0x7fb1ac101870 2024-04-24T23:12:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.978+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac1a22e0 con 0x7fb1ac101870 2024-04-24T23:12:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.980+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb194042030 con 0x7fb1ac101870 2024-04-24T23:12:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.980+0000 7fb1a9ffb640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1980a2ca0 0x7fb1980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.981+0000 7fb1abfff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1980a2ca0 0x7fb1980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.981+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb1940f0170 con 0x7fb1ac101870 2024-04-24T23:12:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.981+0000 7fb1abfff640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1980a2ca0 0x7fb1980a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac19b200 tx=0x7fb19c005dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:01.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:01.984+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1940bcc10 con 0x7fb1ac101870 2024-04-24T23:12:02.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1839034265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/332973532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2802419598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.170+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1ac1a29e0 con 0x7fb1ac101870 2024-04-24T23:12:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.171+0000 7fb1a9ffb640 1 -- 172.21.15.33:0/167787285 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24620 (secure 0 0 0) 0x7fb1940bc510 con 0x7fb1ac101870 2024-04-24T23:12:02.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:02.173 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:11:07.557817+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.173+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1980a2ca0 msgr2=0x7fb1980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.173+0000 7fb1b237e640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1980a2ca0 0x7fb1980a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac19b200 tx=0x7fb19c005dd0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.173+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 msgr2=0x7fb1ac19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.173+0000 7fb1b237e640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac19a270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb194009230 tx=0x7fb194004430 comp rx=0 tx=0).stop 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 shutdown_connections 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1980a2ca0 0x7fb1980a5160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac102230 0x7fb1ac1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac101870 0x7fb1ac19a270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 --2- 172.21.15.33:0/167787285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1ac100680 0x7fb1ac199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 >> 172.21.15.33:0/167787285 conn(0x7fb1ac0fbbf0 msgr2=0x7fb1ac109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 shutdown_connections 2024-04-24T23:12:02.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.174+0000 7fb1b237e640 1 -- 172.21.15.33:0/167787285 wait complete. 2024-04-24T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1839034265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/332973532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2802419598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1839034265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/332973532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.185 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2802419598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:02.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:02.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3513524011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 msgr2=0x7ff7b0106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3513524011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0106f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009900 tx=0x7ff7a002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3513524011 shutdown_connections 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3513524011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b01074c0 0x7ff7b01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3513524011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0106f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3513524011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b0102260 0x7ff7b0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.290+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3513524011 >> 172.21.15.33:0/3513524011 conn(0x7ff7b00fbbf0 msgr2=0x7ff7b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.291+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3513524011 shutdown_connections 2024-04-24T23:12:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.291+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3513524011 wait complete. 2024-04-24T23:12:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.291+0000 7ff7b50ee640 1 Processor -- start 2024-04-24T23:12:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.291+0000 7ff7b50ee640 1 -- start start 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7b50ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b0102260 0x7ff7b0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7b50ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7b50ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b01074c0 0x7ff7b019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7b50ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010d400 con 0x7ff7b0104b90 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7b50ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010d280 con 0x7ff7b01074c0 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7b50ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010d580 con 0x7ff7b0102260 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49246/0 (socket says 172.21.15.33:49246) 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 -- 172.21.15.33:0/3995242636 learned_addr learned my addr 172.21.15.33:0/3995242636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7af59e640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b01074c0 0x7ff7b019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7aed9d640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b0102260 0x7ff7b0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 -- 172.21.15.33:0/3995242636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b0102260 msgr2=0x7ff7b0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b0102260 0x7ff7b0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 -- 172.21.15.33:0/3995242636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b01074c0 msgr2=0x7ff7b019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b01074c0 0x7ff7b019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 -- 172.21.15.33:0/3995242636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019f730 con 0x7ff7b0104b90 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7aed9d640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b0102260 0x7ff7b0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7ae59c640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0198050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0031610 tx=0x7ff7a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.292+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a003f030 con 0x7ff7b0104b90 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.293+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a0003ad0 con 0x7ff7b0104b90 2024-04-24T23:12:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.293+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019f9c0 con 0x7ff7b0104b90 2024-04-24T23:12:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.293+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0035070 con 0x7ff7b0104b90 2024-04-24T23:12:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.293+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b019feb0 con 0x7ff7b0104b90 2024-04-24T23:12:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.294+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b00fd6e0 con 0x7ff7b0104b90 2024-04-24T23:12:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.295+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff7a0042030 con 0x7ff7b0104b90 2024-04-24T23:12:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.295+0000 7ff7a7fff640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff79c0a2c20 0x7ff79c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.295+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff7a00efd10 con 0x7ff7b0104b90 2024-04-24T23:12:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.295+0000 7ff7aed9d640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff79c0a2c20 0x7ff79c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.296+0000 7ff7aed9d640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff79c0a2c20 0x7ff79c0a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff798009b40 tx=0x7ff798005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.300+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7a00bc7b0 con 0x7ff7b0104b90 2024-04-24T23:12:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.486+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7b002d390 con 0x7ff7b0104b90 2024-04-24T23:12:02.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.486+0000 7ff7a7fff640 1 -- 172.21.15.33:0/3995242636 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7ff7a00bc0b0 con 0x7ff7b0104b90 2024-04-24T23:12:02.487 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff79c0a2c20 msgr2=0x7ff79c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff79c0a2c20 0x7ff79c0a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff798009b40 tx=0x7ff798005e50 comp rx=0 tx=0).stop 2024-04-24T23:12:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 msgr2=0x7ff7b0198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0198050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0031610 tx=0x7ff7a0004270 comp rx=0 tx=0).stop 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 shutdown_connections 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff79c0a2c20 0x7ff79c0a50e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7b01074c0 0x7ff7b019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7b0104b90 0x7ff7b0198050 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 --2- 172.21.15.33:0/3995242636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b0102260 0x7ff7b0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 >> 172.21.15.33:0/3995242636 conn(0x7ff7b00fbbf0 msgr2=0x7ff7b0102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 shutdown_connections 2024-04-24T23:12:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.488+0000 7ff7b50ee640 1 -- 172.21.15.33:0/3995242636 wait complete. 2024-04-24T23:12:02.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 -- 172.21.15.33:0/913098144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102990 msgr2=0x7fe994102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 --2- 172.21.15.33:0/913098144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102990 0x7fe994102e10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe984009900 tx=0x7fe98402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 -- 172.21.15.33:0/913098144 shutdown_connections 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 --2- 172.21.15.33:0/913098144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe994103350 0x7fe994109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 --2- 172.21.15.33:0/913098144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994102990 0x7fe994102e10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 --2- 172.21.15.33:0/913098144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9941017a0 0x7fe994101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 -- 172.21.15.33:0/913098144 >> 172.21.15.33:0/913098144 conn(0x7fe9940fcd10 msgr2=0x7fe9940ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 -- 172.21.15.33:0/913098144 shutdown_connections 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 -- 172.21.15.33:0/913098144 wait complete. 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 Processor -- start 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.605+0000 7fe99b970640 1 -- start start 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941017a0 0x7fe994197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994102990 0x7fe994197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 0x7fe99419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410d420 con 0x7fe994103350 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410d2a0 con 0x7fe994102990 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410d5a0 con 0x7fe9941017a0 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 0x7fe99419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 0x7fe99419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49262/0 (socket says 172.21.15.33:49262) 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 -- 172.21.15.33:0/3729650013 learned_addr learned my addr 172.21.15.33:0/3729650013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99970c640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941017a0 0x7fe994197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe998f0b640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994102990 0x7fe994197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 -- 172.21.15.33:0/3729650013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941017a0 msgr2=0x7fe994197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941017a0 0x7fe994197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 -- 172.21.15.33:0/3729650013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994102990 msgr2=0x7fe994197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994102990 0x7fe994197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 -- 172.21.15.33:0/3729650013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99419f6c0 con 0x7fe994103350 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe998f0b640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994102990 0x7fe994197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe999f0d640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 0x7fe99419f050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00eaa0 tx=0x7fe98c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00cdb0 con 0x7fe994103350 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005020 con 0x7fe994103350 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99419f9b0 con 0x7fe994103350 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.606+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0116b0 con 0x7fe994103350 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.607+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99419fed0 con 0x7fe994103350 2024-04-24T23:12:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.607+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe994108ae0 con 0x7fe994103350 2024-04-24T23:12:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.608+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe98c005560 con 0x7fe994103350 2024-04-24T23:12:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.609+0000 7fe98a7fc640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe9800a2c20 0x7fe9800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.609+0000 7fe99970c640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe9800a2c20 0x7fe9800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.609+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe98c0cf850 con 0x7fe994103350 2024-04-24T23:12:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.610+0000 7fe99970c640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe9800a2c20 0x7fe9800a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe990005db0 tx=0x7fe990005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.614+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe98c016030 con 0x7fe994103350 2024-04-24T23:12:02.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.800+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe99402d170 con 0x7fe994103350 2024-04-24T23:12:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.800+0000 7fe98a7fc640 1 -- 172.21.15.33:0/3729650013 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fe98c09c2f0 con 0x7fe994103350 2024-04-24T23:12:02.801 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe9800a2c20 msgr2=0x7fe9800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe9800a2c20 0x7fe9800a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe990005db0 tx=0x7fe990005d40 comp rx=0 tx=0).stop 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 msgr2=0x7fe99419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 0x7fe99419f050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00eaa0 tx=0x7fe98c00ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 shutdown_connections 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe9800a2c20 0x7fe9800a50e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994103350 0x7fe99419f050 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994102990 0x7fe994197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 --2- 172.21.15.33:0/3729650013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941017a0 0x7fe994197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 >> 172.21.15.33:0/3729650013 conn(0x7fe9940fcd10 msgr2=0x7fe994105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 shutdown_connections 2024-04-24T23:12:02.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.802+0000 7fe99b970640 1 -- 172.21.15.33:0/3729650013 wait complete. 2024-04-24T23:12:02.814 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 -- 172.21.15.33:0/3475797829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 msgr2=0x7f2c2c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 --2- 172.21.15.33:0/3475797829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c105010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009900 tx=0x7f2c2002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 -- 172.21.15.33:0/3475797829 shutdown_connections 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 --2- 172.21.15.33:0/3475797829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c2c105550 0x7f2c2c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 --2- 172.21.15.33:0/3475797829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c105010 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 --2- 172.21.15.33:0/3475797829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c2c1039a0 0x7f2c2c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 -- 172.21.15.33:0/3475797829 >> 172.21.15.33:0/3475797829 conn(0x7f2c2c0feef0 msgr2=0x7f2c2c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 -- 172.21.15.33:0/3475797829 shutdown_connections 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 -- 172.21.15.33:0/3475797829 wait complete. 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 Processor -- start 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.918+0000 7f2c347ce640 1 -- start start 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c2c1039a0 0x7f2c2c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c2c105550 0x7f2c2c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c10f600 con 0x7f2c2c104b90 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c10f480 con 0x7f2c2c105550 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c10f780 con 0x7f2c2c1039a0 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49282/0 (socket says 172.21.15.33:49282) 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 -- 172.21.15.33:0/428478912 learned_addr learned my addr 172.21.15.33:0/428478912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c3256a640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c2c1039a0 0x7f2c2c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c32d6b640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c2c105550 0x7f2c2c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 -- 172.21.15.33:0/428478912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c2c1039a0 msgr2=0x7f2c2c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c2c1039a0 0x7f2c2c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 -- 172.21.15.33:0/428478912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c2c105550 msgr2=0x7f2c2c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c2c105550 0x7f2c2c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 -- 172.21.15.33:0/428478912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1a1900 con 0x7f2c2c104b90 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c31d69640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c19a230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009230 tx=0x7f2c20004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2003f030 con 0x7f2c2c104b90 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1a1b90 con 0x7f2c2c104b90 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c20003ce0 con 0x7f2c2c104b90 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20035070 con 0x7f2c2c104b90 2024-04-24T23:12:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.919+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1a20d0 con 0x7f2c2c104b90 2024-04-24T23:12:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.920+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c10ad10 con 0x7f2c2c104b90 2024-04-24T23:12:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.922+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2c20042030 con 0x7f2c2c104b90 2024-04-24T23:12:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.922+0000 7f2c2b7fe640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c140a2c20 0x7f2c140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.922+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2c200f0200 con 0x7f2c2c104b90 2024-04-24T23:12:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.922+0000 7f2c3256a640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c140a2c20 0x7f2c140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.923+0000 7f2c3256a640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c140a2c20 0x7f2c140a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005db0 tx=0x7f2c1c005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:02.926+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c200bcc40 con 0x7f2c2c104b90 2024-04-24T23:12:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.109+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c2c02d330 con 0x7f2c2c104b90 2024-04-24T23:12:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.110+0000 7f2c2b7fe640 1 -- 172.21.15.33:0/428478912 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f2c200bc540 con 0x7f2c2c104b90 2024-04-24T23:12:03.111 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c140a2c20 msgr2=0x7f2c140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c140a2c20 0x7f2c140a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005db0 tx=0x7f2c1c005d40 comp rx=0 tx=0).stop 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 msgr2=0x7f2c2c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c19a230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009230 tx=0x7f2c20004430 comp rx=0 tx=0).stop 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 shutdown_connections 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c140a2c20 0x7f2c140a50e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c2c105550 0x7f2c2c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c2c104b90 0x7f2c2c19a230 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 --2- 172.21.15.33:0/428478912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c2c1039a0 0x7f2c2c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 >> 172.21.15.33:0/428478912 conn(0x7f2c2c0feef0 msgr2=0x7f2c2c107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 shutdown_connections 2024-04-24T23:12:03.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.112+0000 7f2c347ce640 1 -- 172.21.15.33:0/428478912 wait complete. 2024-04-24T23:12:03.122 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:02 smithi033 ceph-mon[29201]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:12:03.122 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:02 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1089079708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.123 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:02 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/167787285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.123 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:02 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3995242636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:03.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:03.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:02 smithi038 ceph-mon[32450]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:12:03.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:02 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1089079708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:02 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/167787285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:02 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3995242636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:03.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:02 smithi117 ceph-mon[32529]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:12:03.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:02 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1089079708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:02 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/167787285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:02 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3995242636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 -- 172.21.15.33:0/2700845464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 msgr2=0x7fa76c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 --2- 172.21.15.33:0/2700845464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1098b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa764009ca0 tx=0x7fa76402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 -- 172.21.15.33:0/2700845464 shutdown_connections 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 --2- 172.21.15.33:0/2700845464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1098b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 --2- 172.21.15.33:0/2700845464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76c104b30 0x7fa76c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 --2- 172.21.15.33:0/2700845464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa76c102200 0x7fa76c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 -- 172.21.15.33:0/2700845464 >> 172.21.15.33:0/2700845464 conn(0x7fa76c0fbbf0 msgr2=0x7fa76c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 -- 172.21.15.33:0/2700845464 shutdown_connections 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.228+0000 7fa7737e7640 1 -- 172.21.15.33:0/2700845464 wait complete. 2024-04-24T23:12:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 Processor -- start 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 -- start start 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa76c102200 0x7fa76c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76c104b30 0x7fa76c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c10d400 con 0x7fa76c107460 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c10d280 con 0x7fa76c102200 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa7737e7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c10d580 con 0x7fa76c104b30 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49294/0 (socket says 172.21.15.33:49294) 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 -- 172.21.15.33:0/1054590750 learned_addr learned my addr 172.21.15.33:0/1054590750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771583640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa76c102200 0x7fa76c101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa770d82640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76c104b30 0x7fa76c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 -- 172.21.15.33:0/1054590750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76c104b30 msgr2=0x7fa76c1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76c104b30 0x7fa76c1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 -- 172.21.15.33:0/1054590750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa76c102200 msgr2=0x7fa76c101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa76c102200 0x7fa76c101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.229+0000 7fa771d84640 1 -- 172.21.15.33:0/1054590750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c100f80 con 0x7fa76c107460 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.230+0000 7fa771d84640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1006f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa764031ae0 tx=0x7fa764031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.230+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764003c30 con 0x7fa76c107460 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.230+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa764003dd0 con 0x7fa76c107460 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.230+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c1a1bc0 con 0x7fa76c107460 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.230+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76402fcb0 con 0x7fa76c107460 2024-04-24T23:12:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.230+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1a2110 con 0x7fa76c107460 2024-04-24T23:12:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.231+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c0fd6e0 con 0x7fa76c107460 2024-04-24T23:12:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.232+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa764004050 con 0x7fa76c107460 2024-04-24T23:12:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.232+0000 7fa75e7fc640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2c20 0x7fa7580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.232+0000 7fa771583640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2c20 0x7fa7580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.233+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa7640f0690 con 0x7fa76c107460 2024-04-24T23:12:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.233+0000 7fa771583640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2c20 0x7fa7580a50e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa768009bd0 tx=0x7fa768005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.237+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa7640bd240 con 0x7fa76c107460 2024-04-24T23:12:03.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.423+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa76c062340 con 0x7fa76c107460 2024-04-24T23:12:03.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.423+0000 7fa75e7fc640 1 -- 172.21.15.33:0/1054590750 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fa7640bcb40 con 0x7fa76c107460 2024-04-24T23:12:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2c20 msgr2=0x7fa7580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2c20 0x7fa7580a50e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa768009bd0 tx=0x7fa768005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 msgr2=0x7fa76c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1006f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa764031ae0 tx=0x7fa764031ee0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 shutdown_connections 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa7580a2c20 0x7fa7580a50e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa76c107460 0x7fa76c1006f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76c104b30 0x7fa76c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 --2- 172.21.15.33:0/1054590750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa76c102200 0x7fa76c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 >> 172.21.15.33:0/1054590750 conn(0x7fa76c0fbbf0 msgr2=0x7fa76c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 shutdown_connections 2024-04-24T23:12:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.425+0000 7fa7737e7640 1 -- 172.21.15.33:0/1054590750 wait complete. 2024-04-24T23:12:03.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:03.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 -- 172.21.15.33:0/3649311912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c102250 msgr2=0x7f1f1c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 --2- 172.21.15.33:0/3649311912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c102250 0x7f1f1c108ae0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009ca0 tx=0x7f1f1402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 -- 172.21.15.33:0/3649311912 shutdown_connections 2024-04-24T23:12:03.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 --2- 172.21.15.33:0/3649311912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c102250 0x7f1f1c108ae0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 --2- 172.21.15.33:0/3649311912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f1c101890 0x7f1f1c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 --2- 172.21.15.33:0/3649311912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 -- 172.21.15.33:0/3649311912 >> 172.21.15.33:0/3649311912 conn(0x7f1f1c0fbc30 msgr2=0x7f1f1c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 -- 172.21.15.33:0/3649311912 shutdown_connections 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.538+0000 7f1f21d12640 1 -- 172.21.15.33:0/3649311912 wait complete. 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 Processor -- start 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 -- start start 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c101890 0x7f1f1c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f1c102250 0x7f1f1c1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c10c300 con 0x7f1f1c1006a0 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c10c180 con 0x7f1f1c101890 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f21d12640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c10c480 con 0x7f1f1c102250 2024-04-24T23:12:03.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49318/0 (socket says 172.21.15.33:49318) 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f1b7fe640 1 -- 172.21.15.33:0/2221596442 learned_addr learned my addr 172.21.15.33:0/2221596442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.539+0000 7f1f1bfff640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f1c102250 0x7f1f1c1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1affd640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c101890 0x7f1f1c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1b7fe640 1 -- 172.21.15.33:0/2221596442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f1c102250 msgr2=0x7f1f1c1031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1b7fe640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f1c102250 0x7f1f1c1031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1b7fe640 1 -- 172.21.15.33:0/2221596442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c101890 msgr2=0x7f1f1c102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1b7fe640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c101890 0x7f1f1c102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1b7fe640 1 -- 172.21.15.33:0/2221596442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c103a40 con 0x7f1f1c1006a0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f1b7fe640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c104630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d770 tx=0x7f1f0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c017070 con 0x7f1f1c1006a0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f0c009c30 con 0x7f1f1c1006a0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c1a1b50 con 0x7f1f1c1006a0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c003dd0 con 0x7f1f1c1006a0 2024-04-24T23:12:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.540+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c1a20a0 con 0x7f1f1c1006a0 2024-04-24T23:12:03.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.541+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c1a2270 con 0x7f1f1c1006a0 2024-04-24T23:12:03.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.542+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1f0c003fb0 con 0x7f1f1c1006a0 2024-04-24T23:12:03.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.543+0000 7f1f18ff9640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1f040a2c20 0x7f1f040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.543+0000 7f1f1affd640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1f040a2c20 0x7f1f040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.543+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1f0c0cef70 con 0x7f1f1c1006a0 2024-04-24T23:12:03.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.543+0000 7f1f1affd640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1f040a2c20 0x7f1f040a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f10005e10 tx=0x7f1f10005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.547+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f0c09ba10 con 0x7f1f1c1006a0 2024-04-24T23:12:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.773+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f1c1a24e0 con 0x7f1f1c1006a0 2024-04-24T23:12:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.773+0000 7f1f18ff9640 1 -- 172.21.15.33:0/2221596442 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f1f0c009de0 con 0x7f1f1c1006a0 2024-04-24T23:12:03.774 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:03.775 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:38.464642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:12:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1f040a2c20 msgr2=0x7f1f040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1f040a2c20 0x7f1f040a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f10005e10 tx=0x7f1f10005da0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 msgr2=0x7f1f1c104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c104630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d770 tx=0x7f1f0c00dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 shutdown_connections 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1f040a2c20 0x7f1f040a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.775+0000 7f1f21d12640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f1c102250 0x7f1f1c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.776+0000 7f1f21d12640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c101890 0x7f1f1c102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.776+0000 7f1f21d12640 1 --2- 172.21.15.33:0/2221596442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1006a0 0x7f1f1c104630 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.776+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 >> 172.21.15.33:0/2221596442 conn(0x7f1f1c0fbc30 msgr2=0x7f1f1c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.776+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 shutdown_connections 2024-04-24T23:12:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.776+0000 7f1f21d12640 1 -- 172.21.15.33:0/2221596442 wait complete. 2024-04-24T23:12:03.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 -- 172.21.15.33:0/1931571488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 msgr2=0x7f2508102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 --2- 172.21.15.33:0/1931571488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508102e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009900 tx=0x7f24fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 -- 172.21.15.33:0/1931571488 shutdown_connections 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 --2- 172.21.15.33:0/1931571488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2508103360 0x7f2508109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 --2- 172.21.15.33:0/1931571488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508102e20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 --2- 172.21.15.33:0/1931571488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25081017b0 0x7f2508101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 -- 172.21.15.33:0/1931571488 >> 172.21.15.33:0/1931571488 conn(0x7f25080fcd40 msgr2=0x7f25080ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 -- 172.21.15.33:0/1931571488 shutdown_connections 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 -- 172.21.15.33:0/1931571488 wait complete. 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.889+0000 7f250ce38640 1 Processor -- start 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 -- start start 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25081017b0 0x7f25080754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2508103360 0x7f2508074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f250810d430 con 0x7f25081029a0 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f250810d2b0 con 0x7f25081017b0 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250ce38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f250810d5b0 con 0x7f2508103360 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49334/0 (socket says 172.21.15.33:49334) 2024-04-24T23:12:03.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 -- 172.21.15.33:0/4293566760 learned_addr learned my addr 172.21.15.33:0/4293566760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250659c640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25081017b0 0x7f25080754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2506d9d640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2508103360 0x7f2508074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 -- 172.21.15.33:0/4293566760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2508103360 msgr2=0x7f2508074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2508103360 0x7f2508074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 -- 172.21.15.33:0/4293566760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25081017b0 msgr2=0x7f25080754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25081017b0 0x7f25080754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 -- 172.21.15.33:0/4293566760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2508074900 con 0x7f25081029a0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f250659c640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25081017b0 0x7f25080754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.890+0000 7f2505d9b640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508073b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009230 tx=0x7f24fc031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.891+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc03f030 con 0x7f25081029a0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.891+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24fc003ad0 con 0x7f25081029a0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.891+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25081a1b90 con 0x7f25081029a0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.891+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc035070 con 0x7f25081029a0 2024-04-24T23:12:03.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.891+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081a2130 con 0x7f25081029a0 2024-04-24T23:12:03.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.892+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2508108ad0 con 0x7f25081029a0 2024-04-24T23:12:03.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.893+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f24fc042030 con 0x7f25081029a0 2024-04-24T23:12:03.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.893+0000 7f24f77fe640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f24f00a2ca0 0x7f24f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.894+0000 7f250659c640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f24f00a2ca0 0x7f24f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.894+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24fc0efb00 con 0x7f25081029a0 2024-04-24T23:12:03.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.894+0000 7f250659c640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f24f00a2ca0 0x7f24f00a5160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2508102800 tx=0x7f2500005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:03.898+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f24fc0bc5a0 con 0x7f25081029a0 2024-04-24T23:12:04.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.089+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2508062340 con 0x7f25081029a0 2024-04-24T23:12:04.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.090+0000 7f24f77fe640 1 -- 172.21.15.33:0/4293566760 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24620 (secure 0 0 0) 0x7f24fc0bbea0 con 0x7f25081029a0 2024-04-24T23:12:04.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:04.092 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:11:07.557817+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f24f00a2ca0 msgr2=0x7f24f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f24f00a2ca0 0x7f24f00a5160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2508102800 tx=0x7f2500005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 msgr2=0x7f2508073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508073b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009230 tx=0x7f24fc031b60 comp rx=0 tx=0).stop 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 shutdown_connections 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f24f00a2ca0 0x7f24f00a5160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2508103360 0x7f2508074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25081029a0 0x7f2508073b30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 --2- 172.21.15.33:0/4293566760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25081017b0 0x7f25080754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 >> 172.21.15.33:0/4293566760 conn(0x7f25080fcd40 msgr2=0x7f2508105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 shutdown_connections 2024-04-24T23:12:04.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.093+0000 7f250ce38640 1 -- 172.21.15.33:0/4293566760 wait complete. 2024-04-24T23:12:04.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:03 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3729650013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:03 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/428478912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:03 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1054590750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:03 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2221596442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:04.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:04.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:03 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3729650013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:03 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/428478912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:03 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1054590750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:03 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2221596442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:03 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3729650013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:03 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/428478912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:03 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1054590750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:03 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2221596442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 -- 172.21.15.33:0/4235198666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b601034b0 msgr2=0x7f7b60109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 --2- 172.21.15.33:0/4235198666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b601034b0 0x7f7b60109d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009cd0 tx=0x7f7b5802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 -- 172.21.15.33:0/4235198666 shutdown_connections 2024-04-24T23:12:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 --2- 172.21.15.33:0/4235198666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b601034b0 0x7f7b60109d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 --2- 172.21.15.33:0/4235198666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b60102af0 0x7f7b60102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 --2- 172.21.15.33:0/4235198666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b600ffa80 0x7f7b600ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 -- 172.21.15.33:0/4235198666 >> 172.21.15.33:0/4235198666 conn(0x7f7b600fb3b0 msgr2=0x7f7b600fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 -- 172.21.15.33:0/4235198666 shutdown_connections 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.206+0000 7f7b64c27640 1 -- 172.21.15.33:0/4235198666 wait complete. 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 Processor -- start 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 -- start start 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 0x7f7b601979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b60102af0 0x7f7b60197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b601034b0 0x7f7b6019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010d420 con 0x7f7b600ffa80 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010d2a0 con 0x7f7b601034b0 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b64c27640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010d5a0 con 0x7f7b60102af0 2024-04-24T23:12:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 0x7f7b601979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 0x7f7b601979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49362/0 (socket says 172.21.15.33:49362) 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 -- 172.21.15.33:0/2778130152 learned_addr learned my addr 172.21.15.33:0/2778130152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5ed9d640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b601034b0 0x7f7b6019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5dd9b640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b60102af0 0x7f7b60197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 -- 172.21.15.33:0/2778130152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b60102af0 msgr2=0x7f7b60197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b60102af0 0x7f7b60197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 -- 172.21.15.33:0/2778130152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b601034b0 msgr2=0x7f7b6019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b601034b0 0x7f7b6019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5e59c640 1 -- 172.21.15.33:0/2778130152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6019f5f0 con 0x7f7b600ffa80 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.207+0000 7f7b5ed9d640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b601034b0 0x7f7b6019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.208+0000 7f7b5e59c640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 0x7f7b601979d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000d950 tx=0x7f7b5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.208+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50017070 con 0x7f7b600ffa80 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.208+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50009c30 con 0x7f7b600ffa80 2024-04-24T23:12:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.208+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6019f8e0 con 0x7f7b600ffa80 2024-04-24T23:12:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.208+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000f600 con 0x7f7b600ffa80 2024-04-24T23:12:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.208+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6019fde0 con 0x7f7b600ffa80 2024-04-24T23:12:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.209+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b60108b50 con 0x7f7b600ffa80 2024-04-24T23:12:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.210+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b5001a070 con 0x7f7b600ffa80 2024-04-24T23:12:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.211+0000 7f7b4f7fe640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b480a2c20 0x7f7b480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.211+0000 7f7b5dd9b640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b480a2c20 0x7f7b480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.211+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b500cf0c0 con 0x7f7b600ffa80 2024-04-24T23:12:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.211+0000 7f7b5dd9b640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b480a2c20 0x7f7b480a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7b60198ee0 tx=0x7f7b54005cc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.215+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7b50009de0 con 0x7f7b600ffa80 2024-04-24T23:12:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.442+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b6002d170 con 0x7f7b600ffa80 2024-04-24T23:12:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.443+0000 7f7b4f7fe640 1 -- 172.21.15.33:0/2778130152 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f7b5009bb00 con 0x7f7b600ffa80 2024-04-24T23:12:04.444 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:04.444 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:38.464642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:12:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b480a2c20 msgr2=0x7f7b480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b480a2c20 0x7f7b480a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7b60198ee0 tx=0x7f7b54005cc0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 msgr2=0x7f7b601979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 0x7f7b601979d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000d950 tx=0x7f7b5000de10 comp rx=0 tx=0).stop 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 shutdown_connections 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b480a2c20 0x7f7b480a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b601034b0 0x7f7b6019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b60102af0 0x7f7b60197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 --2- 172.21.15.33:0/2778130152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600ffa80 0x7f7b601979d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 >> 172.21.15.33:0/2778130152 conn(0x7f7b600fb3b0 msgr2=0x7f7b60105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 shutdown_connections 2024-04-24T23:12:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.445+0000 7f7b64c27640 1 -- 172.21.15.33:0/2778130152 wait complete. 2024-04-24T23:12:04.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:10:11.664486+0000', 'modified': '2024-04-24T23:10:38.464642+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24508, 'mds_1': 24481, 'mds_2': 24479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/287276222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 287276222}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 287276222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'g', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/2446242312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 2446242312}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 2446242312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/221661334', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 221661334}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 221661334}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.38:6839/920905379', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 920905379}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 920905379}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.117:6837/878889242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 878889242}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 878889242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/851112851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 851112851}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 851112851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [24479, 24481, 24508]} 2024-04-24T23:12:04.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:12:04.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:12:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 -- 172.21.15.33:0/34153469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 msgr2=0x7f22b41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 --2- 172.21.15.33:0/34153469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b41031b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 -- 172.21.15.33:0/34153469 shutdown_connections 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 --2- 172.21.15.33:0/34153469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b41001c0 0x7f22b4109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 --2- 172.21.15.33:0/34153469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b40ff800 0x7f22b40ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 --2- 172.21.15.33:0/34153469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b41031b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 -- 172.21.15.33:0/34153469 >> 172.21.15.33:0/34153469 conn(0x7f22b40fb380 msgr2=0x7f22b40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 -- 172.21.15.33:0/34153469 shutdown_connections 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.559+0000 7f22ba714640 1 -- 172.21.15.33:0/34153469 wait complete. 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 Processor -- start 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 -- start start 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b40ff800 0x7f22b4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b41001c0 0x7f22b4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410d410 con 0x7f22b4102d90 2024-04-24T23:12:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410d290 con 0x7f22b41001c0 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.560+0000 7f22ba714640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410d590 con 0x7f22b40ff800 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49372/0 (socket says 172.21.15.33:49372) 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 -- 172.21.15.33:0/268336443 learned_addr learned my addr 172.21.15.33:0/268336443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b40ff800 0x7f22b4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b37fe640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b41001c0 0x7f22b4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 -- 172.21.15.33:0/268336443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b40ff800 msgr2=0x7f22b4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b40ff800 0x7f22b4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 -- 172.21.15.33:0/268336443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b41001c0 msgr2=0x7f22b4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b41001c0 0x7f22b4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 -- 172.21.15.33:0/268336443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b419f6f0 con 0x7f22b4102d90 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b8cb1640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b419f080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00eb10 tx=0x7f22ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b17fa640 1 -- 172.21.15.33:0/268336443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac004b20 con 0x7f22b4102d90 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b17fa640 1 -- 172.21.15.33:0/268336443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac004cc0 con 0x7f22b4102d90 2024-04-24T23:12:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22b17fa640 1 -- 172.21.15.33:0/268336443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac010760 con 0x7f22b4102d90 2024-04-24T23:12:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b419f9e0 con 0x7f22b4102d90 2024-04-24T23:12:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.561+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b419fee0 con 0x7f22b4102d90 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.563+0000 7f22b17fa640 1 -- 172.21.15.33:0/268336443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f22ac005020 con 0x7f22b4102d90 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 Processor -- start 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 -- start start 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc1d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc22930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc25430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55787bb80f00 con 0x55787bc1de30 2024-04-24T23:12:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55787bb29750 con 0x55787bc22fa0 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22bd429740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55787bb7cec0 con 0x55787bc1d440 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc22930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc22930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49380/0 (socket says 172.21.15.33:49380) 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 -- 172.21.15.33:0/1351268783 learned_addr learned my addr 172.21.15.33:0/1351268783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b17fa640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22a00a2c20 0x7f22a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc25430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc1d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 -- 172.21.15.33:0/1351268783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 msgr2=0x55787bc1d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc1d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 -- 172.21.15.33:0/1351268783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 msgr2=0x55787bc25430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc25430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 -- 172.21.15.33:0/1351268783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787b8952d0 con 0x55787bc1de30 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc1d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b8cb1640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22a00a2c20 0x7f22a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b17fa640 1 -- 172.21.15.33:0/268336443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f22ac0cfb40 con 0x7f22b4102d90 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.564+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc22930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f22a4031610 tx=0x7f22a4032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df19b7c7267dab81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22aa7fc640 1 -- 172.21.15.33:0/1351268783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4040070 con 0x55787bc1de30 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22aa7fc640 1 -- 172.21.15.33:0/1351268783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4032e20 con 0x55787bc1de30 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22aa7fc640 1 -- 172.21.15.33:0/1351268783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035040 con 0x55787bc1de30 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22b8cb1640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22a00a2c20 0x7f22a00a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22ac098850 tx=0x7f22ac06bd30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 -- 172.21.15.33:0/1351268783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 msgr2=0x55787bc22930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc22930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f22a4031610 tx=0x7f22a4032c30 comp rx=0 tx=0).stop 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 -- 172.21.15.33:0/1351268783 shutdown_connections 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc25430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc22930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 --2- 172.21.15.33:0/1351268783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc1d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 -- 172.21.15.33:0/1351268783 >> 172.21.15.33:0/1351268783 conn(0x55787bc18ab0 msgr2=0x55787bc18eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 -- 172.21.15.33:0/1351268783 shutdown_connections 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.565+0000 7f22bd429740 1 -- 172.21.15.33:0/1351268783 wait complete. 2024-04-24T23:12:04.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 -- start start 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 Processor -- start 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc1d440 0x55787bc27800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc27d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc22fa0 0x55787bc2ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55787bc25ef0 con 0x55787bc1de30 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55787bb3ccf0 con 0x55787bc1d440 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55787bc26070 con 0x55787bc22fa0 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22bd429740 10 client.0 _ll_register_callbacks cb 0x55787bb3c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc27d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc27d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49386/0 (socket says 172.21.15.33:49386) 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b3fff640 1 -- 172.21.15.33:0/1277220662 learned_addr learned my addr 172.21.15.33:0/1277220662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc1d440 0x55787bc27800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc22fa0 0x55787bc2ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b3fff640 1 -- 172.21.15.33:0/1277220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc22fa0 msgr2=0x55787bc2ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.566+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc22fa0 0x55787bc2ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22b3fff640 1 -- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc1d440 msgr2=0x55787bc27800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc1d440 0x55787bc27800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22b3fff640 1 -- 172.21.15.33:0/1277220662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787b95f6b0 con 0x55787bc1de30 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc1d440 0x55787bc27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc27d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f22a402a930 tx=0x7f22a4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90404f7d9b469249 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22937fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22937fe640 1 -- 172.21.15.33:0/1277220662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4040070 con 0x55787bc1de30 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22937fe640 1 -- 172.21.15.33:0/1277220662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f22a4004040 con 0x55787bc1de30 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a4032e20 con 0x55787bc1de30 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22937fe640 1 -- 172.21.15.33:0/1277220662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f22a4058050 con 0x55787bc1de30 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22bd429740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55787bc31cd0 con 0x55787bc1de30 2024-04-24T23:12:04.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22937fe640 1 -- 172.21.15.33:0/1277220662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4032e20 con 0x55787bc1de30 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22937fe640 1 -- 172.21.15.33:0/1277220662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f22a4004780 con 0x55787bc1de30 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22bd429740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.567+0000 7f22bd429740 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22bd429740 20 client.15222 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22bd429740 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc33e70 0x55787bc352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22bd429740 4 client.15222 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55787bb8e440 con 0x55787bc33e70 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc33e70 0x55787bc352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc33e70 0x55787bc352e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f31da93c744b57f server_cookie=ba050b94da656412 in_seq=0 out_seq=0 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.568+0000 7f22937fe640 10 client.15222 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22937fe640 1 -- 172.21.15.33:0/1277220662 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55787bb8e440 con 0x55787bc33e70 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22937fe640 10 client.15222 handle_command_reply: tid=0 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 client.15222 shutdown 2024-04-24T23:12:04.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 msgr2=0x55787bc27d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc27d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f22a402a930 tx=0x7f22a4005770 comp rx=0 tx=0).stop 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 shutdown_connections 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc33e70 0x55787bc352e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc22fa0 0x55787bc2ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc27d40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 --2- 172.21.15.33:0/1277220662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc1d440 0x55787bc27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22937fe640 0 client.15222 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 >> 172.21.15.33:0/1277220662 conn(0x55787bc18ab0 msgr2=0x55787bc17ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.569+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 shutdown_connections 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.570+0000 7f22bd429740 1 -- 172.21.15.33:0/1277220662 wait complete. 2024-04-24T23:12:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.570+0000 7f22bd429740 20 client.15222 trim_cache size 0 max 16384 2024-04-24T23:12:04.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 Processor -- start 2024-04-24T23:12:04.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 -- start start 2024-04-24T23:12:04.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc1e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55787bc33620 con 0x55787bc1de30 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55787bc33cf0 con 0x55787bc22fa0 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22bd429740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55787bc343c0 con 0x55787bc1d440 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc2ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49396/0 (socket says 172.21.15.33:49396) 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 -- 172.21.15.33:0/100152253 learned_addr learned my addr 172.21.15.33:0/100152253 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b37fe640 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc1e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 -- 172.21.15.33:0/100152253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 msgr2=0x55787bc13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 -- 172.21.15.33:0/100152253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 msgr2=0x55787bc1e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc1e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 -- 172.21.15.33:0/100152253 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787bc18570 con 0x55787bc1de30 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b37fe640 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc1e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.588+0000 7f22b3fff640 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc2ed40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4765cd1cbb642f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22a9ffb640 1 -- 172.21.15.33:0/100152253 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4061070 con 0x55787bc1de30 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22a9ffb640 1 -- 172.21.15.33:0/100152253 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4005250 con 0x55787bc1de30 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22a9ffb640 1 -- 172.21.15.33:0/100152253 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4040070 con 0x55787bc1de30 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 -- 172.21.15.33:0/100152253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 msgr2=0x55787bc2ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc2ed40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a4005fd0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 -- 172.21.15.33:0/100152253 shutdown_connections 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc1e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc2ed40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 --2- 172.21.15.33:0/100152253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 -- 172.21.15.33:0/100152253 >> 172.21.15.33:0/100152253 conn(0x55787bc18ab0 msgr2=0x55787bc18e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 -- 172.21.15.33:0/100152253 shutdown_connections 2024-04-24T23:12:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 -- 172.21.15.33:0/100152253 wait complete. 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.589+0000 7f22bd429740 1 -- start start 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 Processor -- start 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc2cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc95e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55787bc35160 con 0x55787bc1de30 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55787bc34a90 con 0x55787bc22fa0 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55787bc35830 con 0x55787bc1d440 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22bd429740 10 client.0 _ll_register_callbacks cb 0x7f22a4009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc95e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc95e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49400/0 (socket says 172.21.15.33:49400) 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 -- 172.21.15.33:0/1487504616 learned_addr learned my addr 172.21.15.33:0/1487504616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 -- 172.21.15.33:0/1487504616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 msgr2=0x55787bc2cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc2cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 -- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 msgr2=0x55787bc98350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc98350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 -- 172.21.15.33:0/1487504616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787bc98a60 con 0x55787bc1de30 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22b3fff640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc95e00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f22a40092a0 tx=0x7f22a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8d5080310b75ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.590+0000 7f22937fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22937fe640 1 -- 172.21.15.33:0/1487504616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4040070 con 0x55787bc1de30 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22937fe640 1 -- 172.21.15.33:0/1487504616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f22a404d570 con 0x55787bc1de30 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a4005250 con 0x55787bc1de30 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22937fe640 1 -- 172.21.15.33:0/1487504616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f22a4061040 con 0x55787bc1de30 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 10 client.15228 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55787bc99d60 con 0x55787bc1de30 2024-04-24T23:12:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22937fe640 1 -- 172.21.15.33:0/1487504616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4005250 con 0x55787bc1de30 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22937fe640 1 -- 172.21.15.33:0/1487504616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f22a403c440 con 0x55787bc1de30 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 10 client.15228 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 20 client.15228 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc9e470 0x55787bc9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 4 client.15228 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "status"}) -- 0x55787bb88790 con 0x55787bc9e470 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.591+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc9e470 0x55787bc9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22b37fe640 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc9e470 0x55787bc9d180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f24f97acc5b6656 server_cookie=88ff57c943bd932b in_seq=0 out_seq=0 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22937fe640 10 client.15228 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22937fe640 1 -- 172.21.15.33:0/1487504616 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55787bb88790 con 0x55787bc9e470 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22937fe640 10 client.15228 handle_command_reply: tid=0 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 client.15228 shutdown 2024-04-24T23:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 msgr2=0x55787bc95e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc95e00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f22a40092a0 tx=0x7f22a4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 shutdown_connections 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55787bc9e470 0x55787bc9d180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55787bc22fa0 0x55787bc98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55787bc1de30 0x55787bc95e00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 --2- 172.21.15.33:0/1487504616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787bc1d440 0x55787bc2cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22937fe640 0 client.15228 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 >> 172.21.15.33:0/1487504616 conn(0x55787bc18ab0 msgr2=0x55787bc13690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 shutdown_connections 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 1 -- 172.21.15.33:0/1487504616 wait complete. 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.592+0000 7f22bd429740 20 client.15228 trim_cache size 0 max 16384 2024-04-24T23:12:04.599 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24479, 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 87.142113308999996, 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 92, 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 118.56404904 2024-04-24T23:12:04.600 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22a00a2c20 msgr2=0x7f22a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22a00a2c20 0x7f22a00a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22ac098850 tx=0x7f22ac06bd30 comp rx=0 tx=0).stop 2024-04-24T23:12:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 msgr2=0x7f22b419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b419f080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00eb10 tx=0x7f22ac00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 shutdown_connections 2024-04-24T23:12:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22a00a2c20 0x7f22a00a50e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22b4102d90 0x7f22b419f080 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b41001c0 0x7f22b4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 --2- 172.21.15.33:0/268336443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b40ff800 0x7f22b4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 >> 172.21.15.33:0/268336443 conn(0x7f22b40fb380 msgr2=0x7f22b4105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 shutdown_connections 2024-04-24T23:12:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.593+0000 7f22ba714640 1 -- 172.21.15.33:0/268336443 wait complete. 2024-04-24T23:12:04.605 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 -- 172.21.15.33:0/2092281605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 msgr2=0x7facd8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 --2- 172.21.15.33:0/2092281605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd8108ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7facd0009ca0 tx=0x7facd002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 -- 172.21.15.33:0/2092281605 shutdown_connections 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 --2- 172.21.15.33:0/2092281605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd8108ae0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 --2- 172.21.15.33:0/2092281605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd8101890 0x7facd8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 --2- 172.21.15.33:0/2092281605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd81006a0 0x7facd8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 -- 172.21.15.33:0/2092281605 >> 172.21.15.33:0/2092281605 conn(0x7facd80fbc30 msgr2=0x7facd80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 -- 172.21.15.33:0/2092281605 shutdown_connections 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 -- 172.21.15.33:0/2092281605 wait complete. 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 Processor -- start 2024-04-24T23:12:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 -- start start 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd81006a0 0x7facd806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.707+0000 7facdfd29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8101890 0x7facd819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdfd29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdfd29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facd810c320 con 0x7facd8102250 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdfd29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7facd810c1a0 con 0x7facd81006a0 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdfd29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facd810c4a0 con 0x7facd8101890 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49426/0 (socket says 172.21.15.33:49426) 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 -- 172.21.15.33:0/3678023217 learned_addr learned my addr 172.21.15.33:0/3678023217 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facddac5640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd81006a0 0x7facd806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8101890 0x7facd819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 -- 172.21.15.33:0/3678023217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8101890 msgr2=0x7facd819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8101890 0x7facd819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 -- 172.21.15.33:0/3678023217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd81006a0 msgr2=0x7facd806a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd81006a0 0x7facd806a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 -- 172.21.15.33:0/3678023217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd819f650 con 0x7facd8102250 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8101890 0x7facd819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facde2c6640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd819ed90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7facd0031ae0 tx=0x7facd0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facceffd640 1 -- 172.21.15.33:0/3678023217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0003c30 con 0x7facd8102250 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facceffd640 1 -- 172.21.15.33:0/3678023217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd0003dd0 con 0x7facd8102250 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd819f8e0 con 0x7facd8102250 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facceffd640 1 -- 172.21.15.33:0/3678023217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd002fcb0 con 0x7facd8102250 2024-04-24T23:12:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.708+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd819fe30 con 0x7facd8102250 2024-04-24T23:12:04.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.710+0000 7facceffd640 1 -- 172.21.15.33:0/3678023217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7facd0004050 con 0x7facd8102250 2024-04-24T23:12:04.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 Processor -- start 2024-04-24T23:12:04.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 -- start start 2024-04-24T23:12:04.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc165840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ddc0bcf00 con 0x557ddc15e240 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ddc065750 con 0x557ddc15d850 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7face2a3c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ddc0b8ec0 con 0x557ddc1633b0 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7facceffd640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7facc40a2c20 0x7facc40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.711+0000 7facddac5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc162d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49432/0 (socket says 172.21.15.33:49432) 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 -- 172.21.15.33:0/3080797181 learned_addr learned my addr 172.21.15.33:0/3080797181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facde2c6640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7facc40a2c20 0x7facc40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facceffd640 1 -- 172.21.15.33:0/3678023217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7facd00f07d0 con 0x7facd8102250 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 -- 172.21.15.33:0/3080797181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 msgr2=0x557ddc165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc165840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 -- 172.21.15.33:0/3080797181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 msgr2=0x557ddc15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc15dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 -- 172.21.15.33:0/3080797181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ddbdd12d0 con 0x557ddc15e240 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facddac5640 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc162d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7facd400bc80 tx=0x7facd400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17436e063afd3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facde2c6640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7facc40a2c20 0x7facc40a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7facd0031a40 tx=0x7facd008ec30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facc3fff640 1 -- 172.21.15.33:0/3080797181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4015070 con 0x557ddc15e240 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facc3fff640 1 -- 172.21.15.33:0/3080797181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd401a940 con 0x557ddc15e240 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.712+0000 7facc3fff640 1 -- 172.21.15.33:0/3080797181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40118a0 con 0x557ddc15e240 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 -- 172.21.15.33:0/3080797181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 msgr2=0x557ddc162d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc162d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7facd400bc80 tx=0x7facd400f730 comp rx=0 tx=0).stop 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 -- 172.21.15.33:0/3080797181 shutdown_connections 2024-04-24T23:12:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc162d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 --2- 172.21.15.33:0/3080797181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc15dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 -- 172.21.15.33:0/3080797181 >> 172.21.15.33:0/3080797181 conn(0x557ddc154ab0 msgr2=0x557ddc154eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 -- 172.21.15.33:0/3080797181 shutdown_connections 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 -- 172.21.15.33:0/3080797181 wait complete. 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.713+0000 7face2a3c740 1 -- start start 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 Processor -- start 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15d850 0x557ddc167c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc168190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc1633b0 0x557ddc16f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ddc166360 con 0x557ddc15e240 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ddc078cf0 con 0x557ddc1633b0 2024-04-24T23:12:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ddc1664e0 con 0x557ddc15d850 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7face2a3c740 10 client.0 _ll_register_callbacks cb 0x557ddc0786e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc168190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc168190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49438/0 (socket says 172.21.15.33:49438) 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 -- 172.21.15.33:0/2647794227 learned_addr learned my addr 172.21.15.33:0/2647794227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facdd2c4640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc1633b0 0x557ddc16f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facdd2c4640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15d850 0x557ddc167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 -- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15d850 msgr2=0x557ddc167c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15d850 0x557ddc167c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 -- 172.21.15.33:0/2647794227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc1633b0 msgr2=0x557ddc16f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc1633b0 0x557ddc16f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 -- 172.21.15.33:0/2647794227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ddbe9b6b0 con 0x557ddc15e240 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facdd2c4640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15d850 0x557ddc167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:04.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facddac5640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc168190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7facd40058c0 tx=0x7facd401a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135605511b9651e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.714+0000 7facc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7facc17fa640 1 -- 172.21.15.33:0/2647794227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4015070 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7facc17fa640 1 -- 172.21.15.33:0/2647794227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7facd401aaf0 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd401a940 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7facc17fa640 1 -- 172.21.15.33:0/2647794227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facd402e050 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557ddc1721e0 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7facc17fa640 1 -- 172.21.15.33:0/2647794227 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd401a940 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7facc17fa640 1 -- 172.21.15.33:0/2647794227 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7facd4004ac0 con 0x557ddc15e240 2024-04-24T23:12:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 20 client.15240 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1744b0 0x557ddc175920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 4 client.15240 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.715+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ddc0ca440 con 0x557ddc1744b0 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.716+0000 7facdd2c4640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1744b0 0x557ddc175920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.716+0000 7facdd2c4640 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1744b0 0x557ddc175920 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=651377be2de886d8 server_cookie=1166796377bd54c3 in_seq=0 out_seq=0 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.716+0000 7facc17fa640 10 client.15240 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7facc17fa640 1 -- 172.21.15.33:0/2647794227 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ddc0ca440 con 0x557ddc1744b0 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7facc17fa640 10 client.15240 handle_command_reply: tid=0 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 client.15240 shutdown 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 msgr2=0x557ddc168190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc168190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7facd40058c0 tx=0x7facd401a460 comp rx=0 tx=0).stop 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 shutdown_connections 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1744b0 0x557ddc175920 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc1633b0 0x557ddc16f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7facc17fa640 0 client.15240 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15e240 0x557ddc168190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 --2- 172.21.15.33:0/2647794227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15d850 0x557ddc167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 >> 172.21.15.33:0/2647794227 conn(0x557ddc154ab0 msgr2=0x557ddc153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 shutdown_connections 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 1 -- 172.21.15.33:0/2647794227 wait complete. 2024-04-24T23:12:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.717+0000 7face2a3c740 20 client.15240 trim_cache size 0 max 16384 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.735+0000 7face2a3c740 1 Processor -- start 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.735+0000 7face2a3c740 1 -- start start 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.735+0000 7face2a3c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc153f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7face2a3c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7face2a3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 0x557ddc154e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7face2a3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ddc173c60 con 0x557ddc1633b0 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7face2a3c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ddc174330 con 0x557ddc15d850 2024-04-24T23:12:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7face2a3c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ddc174a00 con 0x557ddc15e240 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 0x557ddc154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 0x557ddc154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49452/0 (socket says 172.21.15.33:49452) 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 -- 172.21.15.33:0/3670579335 learned_addr learned my addr 172.21.15.33:0/3670579335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facddac5640 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 -- 172.21.15.33:0/3670579335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15e240 msgr2=0x557ddc16f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 -- 172.21.15.33:0/3670579335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 msgr2=0x557ddc153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 -- 172.21.15.33:0/3670579335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ddc15e620 con 0x557ddc1633b0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facdd2c4640 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 0x557ddc154e90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x557ddc078d80 tx=0x7facc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22774b9d822e1aae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facc37fe640 1 -- 172.21.15.33:0/3670579335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8025070 con 0x557ddc1633b0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facc37fe640 1 -- 172.21.15.33:0/3670579335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc80128e0 con 0x557ddc1633b0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.736+0000 7facc37fe640 1 -- 172.21.15.33:0/3670579335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8003850 con 0x557ddc1633b0 2024-04-24T23:12:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 -- 172.21.15.33:0/3670579335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 msgr2=0x557ddc154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 0x557ddc154e90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x557ddc078d80 tx=0x7facc8004730 comp rx=0 tx=0).stop 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 -- 172.21.15.33:0/3670579335 shutdown_connections 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc1633b0 0x557ddc154e90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 --2- 172.21.15.33:0/3670579335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15d850 0x557ddc153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 -- 172.21.15.33:0/3670579335 >> 172.21.15.33:0/3670579335 conn(0x557ddc154ab0 msgr2=0x557ddc15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 -- 172.21.15.33:0/3670579335 shutdown_connections 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 -- 172.21.15.33:0/3670579335 wait complete. 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 -- start start 2024-04-24T23:12:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.737+0000 7face2a3c740 1 Processor -- start 2024-04-24T23:12:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 0x557ddc161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc1d8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ddc1757a0 con 0x557ddc15d850 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ddc1750d0 con 0x557ddc15e240 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ddc175e70 con 0x557ddc1633b0 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 10 client.0 _ll_register_callbacks cb 0x7facd400bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 0x557ddc161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 0x557ddc161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49468/0 (socket says 172.21.15.33:49468) 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 -- 172.21.15.33:0/4242183553 learned_addr learned my addr 172.21.15.33:0/4242183553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facddac5640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc1d8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 -- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 msgr2=0x557ddc1d8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc1d8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 -- 172.21.15.33:0/4242183553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15e240 msgr2=0x557ddc16f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 -- 172.21.15.33:0/4242183553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ddc1d8c80 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facdd2c4640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 0x557ddc161fb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7facc8009570 tx=0x7facc801fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dafdd84b36a75f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facc17fa640 1 -- 172.21.15.33:0/4242183553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8025040 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7facc17fa640 1 -- 172.21.15.33:0/4242183553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7facc802b400 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.738+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facc80128e0 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7facc17fa640 1 -- 172.21.15.33:0/4242183553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facc8039050 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 10 client.15246 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557ddc1d9f80 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7facc17fa640 1 -- 172.21.15.33:0/4242183553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc80128e0 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7facc17fa640 1 -- 172.21.15.33:0/4242183553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7facc8011600 con 0x557ddc15d850 2024-04-24T23:12:04.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 10 client.15246 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 10 client.15246 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 20 client.15246 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1de7e0 0x557ddc1dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 4 client.15246 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "status"}) -- 0x557ddc0c4790 con 0x557ddc1de7e0 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.739+0000 7facddac5640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1de7e0 0x557ddc1dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7facddac5640 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1de7e0 0x557ddc1dd4f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45bd9f5031574b42 server_cookie=6d0de8cca427a8b8 in_seq=0 out_seq=0 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7facc17fa640 10 client.15246 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7facc17fa640 1 -- 172.21.15.33:0/4242183553 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557ddc0c4790 con 0x557ddc1de7e0 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7facc17fa640 10 client.15246 handle_command_reply: tid=0 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 client.15246 shutdown 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 msgr2=0x557ddc161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 0x557ddc161fb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7facc8009570 tx=0x7facc801fea0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 shutdown_connections 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557ddc1de7e0 0x557ddc1dd4f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ddc1633b0 0x557ddc1d8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ddc15e240 0x557ddc16f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7facc17fa640 0 client.15246 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 --2- 172.21.15.33:0/4242183553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ddc15d850 0x557ddc161fb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 >> 172.21.15.33:0/4242183553 conn(0x557ddc154ab0 msgr2=0x557ddc154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 shutdown_connections 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 1 -- 172.21.15.33:0/4242183553 wait complete. 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.740+0000 7face2a3c740 20 client.15246 trim_cache size 0 max 16384 2024-04-24T23:12:04.747 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24481, 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 88.298875914999996, 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 92, 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 133.32170290299999 2024-04-24T23:12:04.748 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7facc40a2c20 msgr2=0x7facc40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7facc40a2c20 0x7facc40a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7facd0031a40 tx=0x7facd008ec30 comp rx=0 tx=0).stop 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 msgr2=0x7facd819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd819ed90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7facd0031ae0 tx=0x7facd0031ee0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 shutdown_connections 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7facc40a2c20 0x7facc40a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facd8102250 0x7facd819ed90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8101890 0x7facd819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 --2- 172.21.15.33:0/3678023217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd81006a0 0x7facd806a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 >> 172.21.15.33:0/3678023217 conn(0x7facd80fbc30 msgr2=0x7facd8104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 shutdown_connections 2024-04-24T23:12:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.741+0000 7facdfd29640 1 -- 172.21.15.33:0/3678023217 wait complete. 2024-04-24T23:12:04.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 -- 172.21.15.33:0/1523961152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 msgr2=0x7f0e4c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 --2- 172.21.15.33:0/1523961152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c101d10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009900 tx=0x7f0e4002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 -- 172.21.15.33:0/1523961152 shutdown_connections 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 --2- 172.21.15.33:0/1523961152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e4c102250 0x7f0e4c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 --2- 172.21.15.33:0/1523961152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c101d10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 --2- 172.21.15.33:0/1523961152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4c1006a0 0x7f0e4c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 -- 172.21.15.33:0/1523961152 >> 172.21.15.33:0/1523961152 conn(0x7f0e4c0fbc30 msgr2=0x7f0e4c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 -- 172.21.15.33:0/1523961152 shutdown_connections 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 -- 172.21.15.33:0/1523961152 wait complete. 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 Processor -- start 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.856+0000 7f0e54170640 1 -- start start 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e4c1006a0 0x7f0e4c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4c102250 0x7f0e4c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10c380 con 0x7f0e4c101890 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10c200 con 0x7f0e4c102250 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10c500 con 0x7f0e4c1006a0 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49472/0 (socket says 172.21.15.33:49472) 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 -- 172.21.15.33:0/3679972622 learned_addr learned my addr 172.21.15.33:0/3679972622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e4c1006a0 0x7f0e4c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5270d640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4c102250 0x7f0e4c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 -- 172.21.15.33:0/3679972622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e4c1006a0 msgr2=0x7f0e4c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e4c1006a0 0x7f0e4c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 -- 172.21.15.33:0/3679972622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4c102250 msgr2=0x7f0e4c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4c102250 0x7f0e4c19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 -- 172.21.15.33:0/3679972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c19f7e0 con 0x7f0e4c101890 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e5170b640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c198170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0e40006010 tx=0x7f0e40004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e4affd640 1 -- 172.21.15.33:0/3679972622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4003f030 con 0x7f0e4c101890 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e4affd640 1 -- 172.21.15.33:0/3679972622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40003ce0 con 0x7f0e4c101890 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c19fa70 con 0x7f0e4c101890 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.857+0000 7f0e4affd640 1 -- 172.21.15.33:0/3679972622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40035070 con 0x7f0e4c101890 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.858+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c19ffb0 con 0x7f0e4c101890 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e4affd640 1 -- 172.21.15.33:0/3679972622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0e40042030 con 0x7f0e4c101890 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 Processor -- start 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 -- start start 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1c440 0x562ef5f1c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 0x562ef5f24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ef5e7ff00 con 0x562ef5f21fa0 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ef5e28750 con 0x562ef5f1ce30 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ef5e7bec0 con 0x562ef5f1c440 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e5170b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 0x562ef5f24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e51f0c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e5170b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 0x562ef5f24430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49488/0 (socket says 172.21.15.33:49488) 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e5170b640 1 -- 172.21.15.33:0/1283332323 learned_addr learned my addr 172.21.15.33:0/1283332323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.860+0000 7f0e4affd640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e340a2ca0 0x7f0e340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e5270d640 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1c440 0x562ef5f1c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e5270d640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e340a2ca0 0x7f0e340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1283332323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1c440 msgr2=0x562ef5f1c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1c440 0x562ef5f1c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1283332323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 msgr2=0x562ef5f24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 0x562ef5f24430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1283332323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef5b942d0 con 0x562ef5f1ce30 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e5170b640 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 0x562ef5f24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e4affd640 1 -- 172.21.15.33:0/3679972622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0e400efdb0 con 0x7f0e4c101890 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f21930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0e44005fd0 tx=0x7f0e44003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3d25995feac770 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e5270d640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e340a2ca0 0x7f0e340a5160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x562ef5f1d690 tx=0x7f0e3c0090e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e33fff640 1 -- 172.21.15.33:0/1283332323 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44004b20 con 0x562ef5f1ce30 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e33fff640 1 -- 172.21.15.33:0/1283332323 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e44004cc0 con 0x562ef5f1ce30 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.861+0000 7f0e33fff640 1 -- 172.21.15.33:0/1283332323 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44012780 con 0x562ef5f1ce30 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1283332323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 msgr2=0x562ef5f21930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f21930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0e44005fd0 tx=0x7f0e44003e80 comp rx=0 tx=0).stop 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1283332323 shutdown_connections 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f21fa0 0x562ef5f24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f21930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1283332323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1c440 0x562ef5f1c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1283332323 >> 172.21.15.33:0/1283332323 conn(0x562ef5f17ab0 msgr2=0x562ef5f17eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1283332323 shutdown_connections 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1283332323 wait complete. 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 -- start start 2024-04-24T23:12:04.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.862+0000 7f0e56e7d740 1 Processor -- start 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f26830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1ce30 0x562ef5f26d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f21fa0 0x562ef5f2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f24f50 con 0x562ef5f1c440 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ef5e3bcf0 con 0x562ef5f21fa0 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f250d0 con 0x562ef5f1ce30 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e56e7d740 10 client.0 _ll_register_callbacks cb 0x562ef5e3b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f26830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f26830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49504/0 (socket says 172.21.15.33:49504) 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1868490135 learned_addr learned my addr 172.21.15.33:0/1868490135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e5270d640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f21fa0 0x562ef5f2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e5170b640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1ce30 0x562ef5f26d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1ce30 msgr2=0x562ef5f26d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1ce30 0x562ef5f26d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1868490135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f21fa0 msgr2=0x562ef5f2ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f21fa0 0x562ef5f2ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1868490135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef5c5e6b0 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f26830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000f80 tx=0x7f0e44009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1254c3a0acd1dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.863+0000 7f0e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e317fa640 1 -- 172.21.15.33:0/1868490135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44005280 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e317fa640 1 -- 172.21.15.33:0/1868490135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0e44005420 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e44004cc0 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e317fa640 1 -- 172.21.15.33:0/1868490135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e44005640 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ef5f30e00 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e317fa640 1 -- 172.21.15.33:0/1868490135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e44004cc0 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e317fa640 1 -- 172.21.15.33:0/1868490135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0e4400f040 con 0x562ef5f1c440 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 20 client.15258 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f33210 0x562ef5f34680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 4 client.15258 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.864+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef5e8d440 con 0x562ef5f33210 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.865+0000 7f0e5170b640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f33210 0x562ef5f34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.865+0000 7f0e317fa640 10 client.15258 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.865+0000 7f0e5170b640 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f33210 0x562ef5f34680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=569b81919c4247fe server_cookie=53d7984e8f438eca in_seq=0 out_seq=0 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e317fa640 1 -- 172.21.15.33:0/1868490135 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ef5e8d440 con 0x562ef5f33210 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e317fa640 10 client.15258 handle_command_reply: tid=0 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 client.15258 shutdown 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 msgr2=0x562ef5f26830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f26830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000f80 tx=0x7f0e44009db0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 shutdown_connections 2024-04-24T23:12:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f33210 0x562ef5f34680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f21fa0 0x562ef5f2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f1ce30 0x562ef5f26d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e317fa640 0 client.15258 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1868490135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f26830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 >> 172.21.15.33:0/1868490135 conn(0x562ef5f17ab0 msgr2=0x562ef5f16ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 shutdown_connections 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1868490135 wait complete. 2024-04-24T23:12:04.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.866+0000 7f0e56e7d740 20 client.15258 trim_cache size 0 max 16384 2024-04-24T23:12:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 Processor -- start 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 -- start start 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f24390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f2dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f329c0 con 0x562ef5f1c440 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f33090 con 0x562ef5f1ce30 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f33760 con 0x562ef5f21fa0 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e51f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f24390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e51f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f24390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49512/0 (socket says 172.21.15.33:49512) 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.884+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1973243078 learned_addr learned my addr 172.21.15.33:0/1973243078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1973243078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 msgr2=0x562ef5f1b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1973243078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 msgr2=0x562ef5f2dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f2dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e51f0c640 1 -- 172.21.15.33:0/1973243078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef5f11cb0 con 0x562ef5f1c440 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f24390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009fc0 tx=0x7f0e44003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8356931bc44feab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e337fe640 1 -- 172.21.15.33:0/1973243078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4401a780 con 0x562ef5f1c440 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e337fe640 1 -- 172.21.15.33:0/1973243078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e4401a920 con 0x562ef5f1c440 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e337fe640 1 -- 172.21.15.33:0/1973243078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011b20 con 0x562ef5f1c440 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1973243078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 msgr2=0x562ef5f24390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f24390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009fc0 tx=0x7f0e44003e80 comp rx=0 tx=0).stop 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1973243078 shutdown_connections 2024-04-24T23:12:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f1b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1ce30 0x562ef5f2dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/1973243078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1c440 0x562ef5f24390 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.885+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1973243078 >> 172.21.15.33:0/1973243078 conn(0x562ef5f17ab0 msgr2=0x562ef5f32150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1973243078 shutdown_connections 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 -- 172.21.15.33:0/1973243078 wait complete. 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 -- start start 2024-04-24T23:12:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 Processor -- start 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1c440 0x562ef5f2bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 0x562ef5f951a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f34500 con 0x562ef5f1ce30 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f33e30 con 0x562ef5f1c440 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ef5f34bd0 con 0x562ef5f21fa0 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e56e7d740 10 client.0 _ll_register_callbacks cb 0x7f0e44009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e5170b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 0x562ef5f951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e5170b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 0x562ef5f951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49514/0 (socket says 172.21.15.33:49514) 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.886+0000 7f0e5170b640 1 -- 172.21.15.33:0/3075350830 learned_addr learned my addr 172.21.15.33:0/3075350830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5270d640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1c440 0x562ef5f2bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5170b640 1 -- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 msgr2=0x562ef5f976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5170b640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5170b640 1 -- 172.21.15.33:0/3075350830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1c440 msgr2=0x562ef5f2bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5170b640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1c440 0x562ef5f2bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5170b640 1 -- 172.21.15.33:0/3075350830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef5f97e00 con 0x562ef5f1ce30 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1c440 0x562ef5f2bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e5170b640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 0x562ef5f951a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0e400ed480 tx=0x7f0e4003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54407de3db116ced server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e317fa640 1 -- 172.21.15.33:0/3075350830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40049070 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e317fa640 1 -- 172.21.15.33:0/3075350830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0e400ed650 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e4401a920 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e317fa640 1 -- 172.21.15.33:0/3075350830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e400bbb60 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e56e7d740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ef5f99100 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e317fa640 1 -- 172.21.15.33:0/3075350830 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e4401a920 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e317fa640 1 -- 172.21.15.33:0/3075350830 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0e400b7f50 con 0x562ef5f1ce30 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.887+0000 7f0e56e7d740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e56e7d740 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e56e7d740 20 client.15264 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f9dd20 0x562ef5f9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e56e7d740 4 client.15264 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "status"}) -- 0x562ef5e87790 con 0x562ef5f9dd20 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f9dd20 0x562ef5f9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e51f0c640 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f9dd20 0x562ef5f9a830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8009e05210a71c95 server_cookie=fcfbb3e56dd44679 in_seq=0 out_seq=0 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e317fa640 10 client.15264 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e317fa640 1 -- 172.21.15.33:0/3075350830 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562ef5e87790 con 0x562ef5f9dd20 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e317fa640 10 client.15264 handle_command_reply: tid=0 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.888+0000 7f0e56e7d740 1 client.15264 shutdown 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 msgr2=0x562ef5f951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 0x562ef5f951a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0e400ed480 tx=0x7f0e4003b800 comp rx=0 tx=0).stop 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 shutdown_connections 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ef5f9dd20 0x562ef5f9a830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ef5f21fa0 0x562ef5f976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e317fa640 0 client.15264 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ef5f1ce30 0x562ef5f951a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 --2- 172.21.15.33:0/3075350830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ef5f1c440 0x562ef5f2bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 >> 172.21.15.33:0/3075350830 conn(0x562ef5f17ab0 msgr2=0x562ef5f213c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 shutdown_connections 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 1 -- 172.21.15.33:0/3075350830 wait complete. 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e56e7d740 20 client.15264 trim_cache size 0 max 16384 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24508, 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 112.558200408, 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 92, 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 127.696608639 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e340a2ca0 msgr2=0x7f0e340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e340a2ca0 0x7f0e340a5160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x562ef5f1d690 tx=0x7f0e3c0090e0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 msgr2=0x7f0e4c198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c198170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0e40006010 tx=0x7f0e40004360 comp rx=0 tx=0).stop 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 shutdown_connections 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0e340a2ca0 0x7f0e340a5160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e4c102250 0x7f0e4c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4c101890 0x7f0e4c198170 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 --2- 172.21.15.33:0/3679972622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e4c1006a0 0x7f0e4c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.889+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 >> 172.21.15.33:0/3679972622 conn(0x7f0e4c0fbc30 msgr2=0x7f0e4c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.890+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 shutdown_connections 2024-04-24T23:12:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:04.890+0000 7f0e54170640 1 -- 172.21.15.33:0/3679972622 wait complete. 2024-04-24T23:12:04.901 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T23:12:04.901 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:12:04.911 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-24T23:12:04.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T23:12:05.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:05.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:04 smithi038 ceph-mon[32450]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:05.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:04 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4293566760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:05.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:04 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2778130152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:05.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:04 smithi117 ceph-mon[32529]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:05.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:04 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4293566760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:05.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:04 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2778130152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:04 smithi033 ceph-mon[29201]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:04 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4293566760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:04 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2778130152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:06 smithi033 ceph-mon[29201]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-24T23:12:06.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:06 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 -- 172.21.15.33:0/735733472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 msgr2=0x7f3ab810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 --2- 172.21.15.33:0/735733472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab810a3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009990 tx=0x7f3ab0009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 -- 172.21.15.33:0/735733472 shutdown_connections 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 --2- 172.21.15.33:0/735733472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab810a3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 --2- 172.21.15.33:0/735733472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 0x7f3ab8105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 --2- 172.21.15.33:0/735733472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab8105650 0x7f3ab8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 -- 172.21.15.33:0/735733472 >> 172.21.15.33:0/735733472 conn(0x7f3ab80fc730 msgr2=0x7f3ab80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 -- 172.21.15.33:0/735733472 shutdown_connections 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.013+0000 7f3ac0079700 1 -- 172.21.15.33:0/735733472 wait complete. 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 Processor -- start 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 -- start start 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 0x7f3ab819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab8105650 0x7f3ab819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab81a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810de50 con 0x7f3ab8107f80 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810dcd0 con 0x7f3ab8102d20 2024-04-24T23:12:07.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3ac0079700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810dfd0 con 0x7f3ab8105650 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab81a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab81a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51436/0 (socket says 172.21.15.33:51436) 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 -- 172.21.15.33:0/1249311392 learned_addr learned my addr 172.21.15.33:0/1249311392 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abd614700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab8105650 0x7f3ab819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abde15700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 0x7f3ab819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 -- 172.21.15.33:0/1249311392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab8105650 msgr2=0x7f3ab819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab8105650 0x7f3ab819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 -- 172.21.15.33:0/1249311392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 msgr2=0x7f3ab819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 0x7f3ab819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abe616700 1 -- 172.21.15.33:0/1249311392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81a22d0 con 0x7f3ab8107f80 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.014+0000 7f3abde15700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 0x7f3ab819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.015+0000 7f3abe616700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab81a1c30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0003890 tx=0x7f3ab0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.015+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0026070 con 0x7f3ab8107f80 2024-04-24T23:12:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.015+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a2560 con 0x7f3ab8107f80 2024-04-24T23:12:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.015+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab0004150 con 0x7f3ab8107f80 2024-04-24T23:12:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.015+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0004900 con 0x7f3ab8107f80 2024-04-24T23:12:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.015+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a29c0 con 0x7f3ab8107f80 2024-04-24T23:12:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.016+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab8068f20 con 0x7f3ab8107f80 2024-04-24T23:12:07.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.017+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ab0029070 con 0x7f3ab8107f80 2024-04-24T23:12:07.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.017+0000 7f3aaeffd700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3aa40a2c20 0x7f3aa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.018+0000 7f3abde15700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3aa40a2c20 0x7f3aa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.018+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f3ab00cfd80 con 0x7f3ab8107f80 2024-04-24T23:12:07.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.018+0000 7f3abde15700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3aa40a2c20 0x7f3aa40a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40060b0 tx=0x7f3ab40076a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.022+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ab005d740 con 0x7f3ab8107f80 2024-04-24T23:12:07.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:06 smithi038 ceph-mon[32450]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:07.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-24T23:12:07.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:06 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-04-24T23:12:07.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:06 smithi117 ceph-mon[32529]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:07.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-24T23:12:07.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:06 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-04-24T23:12:07.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.244+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3ab8062670 con 0x7f3aa40a2c20 2024-04-24T23:12:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.245+0000 7f3aaeffd700 1 -- 172.21.15.33:0/1249311392 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3ab8062670 con 0x7f3aa40a2c20 2024-04-24T23:12:07.247 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-04-24T23:12:07.247 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-04-24T23:12:07.247 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-04-24T23:12:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3aa40a2c20 msgr2=0x7f3aa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3aa40a2c20 0x7f3aa40a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40060b0 tx=0x7f3ab40076a0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 msgr2=0x7f3ab81a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab81a1c30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0003890 tx=0x7f3ab0009510 comp rx=0 tx=0).stop 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 shutdown_connections 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3aa40a2c20 0x7f3aa40a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab8107f80 0x7f3ab81a1c30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8102d20 0x7f3ab819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 --2- 172.21.15.33:0/1249311392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab8105650 0x7f3ab819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 >> 172.21.15.33:0/1249311392 conn(0x7f3ab80fc730 msgr2=0x7f3ab8102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 shutdown_connections 2024-04-24T23:12:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:07.247+0000 7f3ac0079700 1 -- 172.21.15.33:0/1249311392 wait complete. 2024-04-24T23:12:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-24T23:12:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-04-24T23:12:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:07 smithi033 ceph-mon[29201]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T23:12:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:07 smithi033 ceph-mon[29201]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:12:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:07 smithi033 ceph-mon[29201]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:08.039 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T23:12:08.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-24T23:12:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-04-24T23:12:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:07 smithi038 ceph-mon[32450]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T23:12:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:07 smithi038 ceph-mon[32450]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:12:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:07 smithi038 ceph-mon[32450]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:08.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-24T23:12:08.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-04-24T23:12:08.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:07 smithi117 ceph-mon[32529]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T23:12:08.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:07 smithi117 ceph-mon[32529]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:12:08.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:07 smithi117 ceph-mon[32529]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:08.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:08 smithi033 ceph-mon[29201]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-24T23:12:08.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:08 smithi033 ceph-mon[29201]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:12:09.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:08 smithi038 ceph-mon[32450]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-24T23:12:09.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:08 smithi038 ceph-mon[32450]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:12:09.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:08 smithi117 ceph-mon[32529]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-24T23:12:09.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:08 smithi117 ceph-mon[32529]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 -- 172.21.15.33:0/169223196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103e30 msgr2=0x7f672010a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 --2- 172.21.15.33:0/169223196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103e30 0x7f672010a6e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f671800b0d0 tx=0x7f671800b3e0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 -- 172.21.15.33:0/169223196 shutdown_connections 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 --2- 172.21.15.33:0/169223196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103e30 0x7f672010a6e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 --2- 172.21.15.33:0/169223196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 0x7f67201026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 --2- 172.21.15.33:0/169223196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103490 0x7f67201038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 -- 172.21.15.33:0/169223196 >> 172.21.15.33:0/169223196 conn(0x7f67200fd830 msgr2=0x7f67200ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 -- 172.21.15.33:0/169223196 shutdown_connections 2024-04-24T23:12:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.087+0000 7f6724ef6700 1 -- 172.21.15.33:0/169223196 wait complete. 2024-04-24T23:12:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.088+0000 7f6724ef6700 1 Processor -- start 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.088+0000 7f6724ef6700 1 -- start start 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.088+0000 7f6724ef6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 0x7f67201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.088+0000 7f6724ef6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 0x7f6720198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.088+0000 7f6724ef6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 0x7f672019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f6724ef6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f672010de50 con 0x7f6720103490 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f6724ef6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f672010dcd0 con 0x7f6720102290 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f6724ef6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f672010dfd0 con 0x7f6720103e30 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 0x7f6720198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 0x7f672019f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 0x7f6720198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51452/0 (socket says 172.21.15.33:51452) 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671dd9b700 1 -- 172.21.15.33:0/4204727344 learned_addr learned my addr 172.21.15.33:0/4204727344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 0x7f672019f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42950/0 (socket says 172.21.15.33:42950) 2024-04-24T23:12:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671e59c700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 0x7f67201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 -- 172.21.15.33:0/4204727344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 msgr2=0x7f67201983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 0x7f67201983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 -- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 msgr2=0x7f6720198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 0x7f6720198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671ed9d700 1 -- 172.21.15.33:0/4204727344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67201a0030 con 0x7f6720103e30 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671e59c700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 0x7f67201983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.089+0000 7f671dd9b700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 0x7f6720198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.090+0000 7f671ed9d700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 0x7f672019f990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f671800b0d0 tx=0x7f6718007d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.090+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718021020 con 0x7f6720103e30 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.090+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f671801d950 con 0x7f6720103e30 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.090+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67201a02c0 con 0x7f6720103e30 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.090+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718005430 con 0x7f6720103e30 2024-04-24T23:12:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.090+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67201a0850 con 0x7f6720103e30 2024-04-24T23:12:10.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.091+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6720068f20 con 0x7f6720103e30 2024-04-24T23:12:10.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.092+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6718024020 con 0x7f6720103e30 2024-04-24T23:12:10.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.093+0000 7f67177fe700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f670c0a2ca0 0x7f670c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.093+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f671800f020 con 0x7f6720103e30 2024-04-24T23:12:10.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.093+0000 7f671e59c700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f670c0a2ca0 0x7f670c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.093+0000 7f671e59c700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f670c0a2ca0 0x7f670c0a5160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6708005950 tx=0x7f67080058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:10.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.098+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f671809d140 con 0x7f6720103e30 2024-04-24T23:12:10.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.312+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6720062670 con 0x7f670c0a2ca0 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.326+0000 7f67177fe700 1 -- 172.21.15.33:0/4204727344 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f6720062670 con 0x7f670c0a2ca0 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (2m) 19s ago 2m 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 5c917eb96288 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi038 running (2m) 57s ago 2m 16.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 94a82dbe1869 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi117 running (2m) 57s ago 2m 17.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 53bf91c80d1e 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (2m) 19s ago 2m 15.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef b1510d0314f4 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi038 running (2m) 57s ago 2m 16.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef e93cd3d3bad9 2024-04-24T23:12:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi117 running (2m) 57s ago 2m 17.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0e304fca4248 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (2m) 19s ago 2m 17.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1428cf86b9b5 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi038 running (2m) 57s ago 2m 16.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3b9ff9eac407 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi117 running (2m) 57s ago 2m 27.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef b1a0c926bc87 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (2m) 19s ago 2m 15.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 930d7a69a0d2 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi038 running (2m) 57s ago 2m 16.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef e0d23058e920 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi117 running (2m) 57s ago 2m 15.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef ba59b5856139 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (9m) 19s ago 9m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef a72910c86a2e 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi038 *:8443,8765 running (7m) 57s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef aec5763f7f14 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi117 *:8443,8765 running (7m) 57s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 56296d78a223 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (9m) 19s ago 10m 52.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f59ab6d57b36 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi038 running (8m) 57s ago 8m 40.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef dfde4ac5135b 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi117 running (8m) 57s ago 8m 37.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 266d12945107 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 19s ago 7m 45.6M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 745e9e8a1f57 2024-04-24T23:12:10.329 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi038 running (7m) 57s ago 7m 57.1M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 5d7c154479e8 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi117 running (6m) 57s ago 6m 56.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 487caf15c0a8 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (6m) 19s ago 6m 44.2M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef ad189a7f55cd 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi038 running (5m) 57s ago 5m 42.4M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 3e599fb22567 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi117 running (5m) 57s ago 5m 44.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 695b6e98e29f 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (5m) 19s ago 5m 41.7M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 9128638536a3 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi038 running (4m) 57s ago 4m 55.9M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 8933c2386c60 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi117 running (4m) 57s ago 4m 43.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 3e5b24f5ac9b 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (3m) 19s ago 3m 52.5M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef b954d274acda 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi038 running (3m) 57s ago 3m 43.4M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 2a55c46f3c43 2024-04-24T23:12:10.330 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi117 running (2m) 57s ago 2m 42.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef d0e42bacd60a 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f670c0a2ca0 msgr2=0x7f670c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f670c0a2ca0 0x7f670c0a5160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6708005950 tx=0x7f67080058e0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 msgr2=0x7f672019f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 0x7f672019f990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f671800b0d0 tx=0x7f6718007d90 comp rx=0 tx=0).stop 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 shutdown_connections 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f670c0a2ca0 0x7f670c0a5160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6720103e30 0x7f672019f990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6720102290 0x7f67201983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 --2- 172.21.15.33:0/4204727344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6720103490 0x7f6720198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.328+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 >> 172.21.15.33:0/4204727344 conn(0x7f67200fd830 msgr2=0x7f6720106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.329+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 shutdown_connections 2024-04-24T23:12:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:10.329+0000 7f6724ef6700 1 -- 172.21.15.33:0/4204727344 wait complete. 2024-04-24T23:12:11.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:10 smithi033 ceph-mon[29201]: pgmap v356: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-24T23:12:11.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:10 smithi033 ceph-mon[29201]: from='client.24997 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:11.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T23:12:11.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:10 smithi038 ceph-mon[32450]: pgmap v356: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-24T23:12:11.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:10 smithi038 ceph-mon[32450]: from='client.24997 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:10 smithi117 ceph-mon[32529]: pgmap v356: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-24T23:12:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:10 smithi117 ceph-mon[32529]: from='client.24997 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:11.294 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:12 smithi033 ceph-mon[29201]: pgmap v357: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-04-24T23:12:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 -- 172.21.15.33:0/4051615630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c102da0 msgr2=0x7fa05c105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 --2- 172.21.15.33:0/4051615630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c102da0 0x7fa05c105190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009990 tx=0x7fa04c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 -- 172.21.15.33:0/4051615630 shutdown_connections 2024-04-24T23:12:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 --2- 172.21.15.33:0/4051615630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa05c108000 0x7fa05c10a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 --2- 172.21.15.33:0/4051615630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c102da0 0x7fa05c105190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 --2- 172.21.15.33:0/4051615630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c1056d0 0x7fa05c107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 -- 172.21.15.33:0/4051615630 >> 172.21.15.33:0/4051615630 conn(0x7fa05c0fc750 msgr2=0x7fa05c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 -- 172.21.15.33:0/4051615630 shutdown_connections 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.040+0000 7fa0640d6700 1 -- 172.21.15.33:0/4051615630 wait complete. 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 Processor -- start 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 -- start start 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c102da0 0x7fa05c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa05c1056d0 0x7fa05c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 0x7fa05c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10dec0 con 0x7fa05c108000 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10dd40 con 0x7fa05c1056d0 2024-04-24T23:12:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa0640d6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10e040 con 0x7fa05c102da0 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 0x7fa05c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 0x7fa05c19fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51476/0 (socket says 172.21.15.33:51476) 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 -- 172.21.15.33:0/3112388137 learned_addr learned my addr 172.21.15.33:0/3112388137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa061671700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa05c1056d0 0x7fa05c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa061e72700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c102da0 0x7fa05c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 -- 172.21.15.33:0/3112388137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c102da0 msgr2=0x7fa05c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c102da0 0x7fa05c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 -- 172.21.15.33:0/3112388137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa05c1056d0 msgr2=0x7fa05c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa05c1056d0 0x7fa05c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa062673700 1 -- 172.21.15.33:0/3112388137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c1a0110 con 0x7fa05c108000 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.041+0000 7fa061e72700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c102da0 0x7fa05c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.042+0000 7fa062673700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 0x7fa05c19fa70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa05400efc0 tx=0x7fa05400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.042+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0540092e0 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.042+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa054004bb0 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.042+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c1a03a0 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.042+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054004390 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.042+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c1a0890 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.043+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c068f20 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.044+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa05401b030 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.044+0000 7fa052ffd700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0480a2c20 0x7fa0480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.045+0000 7fa061e72700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0480a2c20 0x7fa0480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.045+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0540cf970 con 0x7fa05c108000 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.045+0000 7fa061e72700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0480a2c20 0x7fa0480a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000c00 tx=0x7fa04c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.048+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa054022020 con 0x7fa05c108000 2024-04-24T23:12:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:12 smithi038 ceph-mon[32450]: pgmap v357: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-04-24T23:12:13.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:12 smithi117 ceph-mon[32529]: pgmap v357: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-04-24T23:12:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.263+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa05c062670 con 0x7fa0480a2c20 2024-04-24T23:12:13.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.269+0000 7fa052ffd700 1 -- 172.21.15.33:0/3112388137 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa05c062670 con 0x7fa0480a2c20 2024-04-24T23:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T23:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 60s ago 2m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-24T23:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 60s ago 8m smithi033=x;smithi038=y;smithi117=z;count:3 2024-04-24T23:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 60s ago 8m smithi033:172.21.15.33=a;smithi038:172.21.15.38=b;smithi117:172.21.15.117=c;count:3 2024-04-24T23:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 60s ago - 2024-04-24T23:12:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.271+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0480a2c20 msgr2=0x7fa0480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.271+0000 7fa0640d6700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0480a2c20 0x7fa0480a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000c00 tx=0x7fa04c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 msgr2=0x7fa05c19fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 0x7fa05c19fa70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa05400efc0 tx=0x7fa05400c450 comp rx=0 tx=0).stop 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 shutdown_connections 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0480a2c20 0x7fa0480a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa05c108000 0x7fa05c19fa70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c102da0 0x7fa05c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 --2- 172.21.15.33:0/3112388137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa05c1056d0 0x7fa05c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 >> 172.21.15.33:0/3112388137 conn(0x7fa05c0fc750 msgr2=0x7fa05c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 shutdown_connections 2024-04-24T23:12:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:13.272+0000 7fa0640d6700 1 -- 172.21.15.33:0/3112388137 wait complete. 2024-04-24T23:12:14.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T23:12:14.119 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:13 smithi033 ceph-mon[29201]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:13 smithi038 ceph-mon[32450]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:14.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:13 smithi117 ceph-mon[32529]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:15.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:14 smithi038 ceph-mon[32450]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-24T23:12:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:14 smithi117 ceph-mon[32529]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-24T23:12:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:14 smithi033 ceph-mon[29201]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-24T23:12:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 -- 172.21.15.33:0/2200462910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241034b0 msgr2=0x7fc324103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 --2- 172.21.15.33:0/2200462910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241034b0 0x7fc324103910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc3140099c0 tx=0x7fc314009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 -- 172.21.15.33:0/2200462910 shutdown_connections 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 --2- 172.21.15.33:0/2200462910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc324103e50 0x7fc32410a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 --2- 172.21.15.33:0/2200462910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3241022b0 0x7fc3241026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 --2- 172.21.15.33:0/2200462910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241034b0 0x7fc324103910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 -- 172.21.15.33:0/2200462910 >> 172.21.15.33:0/2200462910 conn(0x7fc3240fd850 msgr2=0x7fc3240ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 -- 172.21.15.33:0/2200462910 shutdown_connections 2024-04-24T23:12:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.036+0000 7fc3289db700 1 -- 172.21.15.33:0/2200462910 wait complete. 2024-04-24T23:12:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 Processor -- start 2024-04-24T23:12:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 -- start start 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 0x7fc324075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3241034b0 0x7fc324074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324103e50 0x7fc324074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410df10 con 0x7fc3241022b0 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410dd90 con 0x7fc3241034b0 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.037+0000 7fc3289db700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410e090 con 0x7fc324103e50 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 0x7fc324075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc321d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3241034b0 0x7fc324074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 0x7fc324075fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42236/0 (socket says 172.21.15.33:42236) 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 -- 172.21.15.33:0/2126741204 learned_addr learned my addr 172.21.15.33:0/2126741204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc322d9d700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324103e50 0x7fc324074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 -- 172.21.15.33:0/2126741204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324103e50 msgr2=0x7fc324074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324103e50 0x7fc324074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 -- 172.21.15.33:0/2126741204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3241034b0 msgr2=0x7fc324074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3241034b0 0x7fc324074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 -- 172.21.15.33:0/2126741204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc324075410 con 0x7fc3241022b0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc321d9b700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3241034b0 0x7fc324074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc32259c700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 0x7fc324075fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc31800b500 tx=0x7fc31800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318015020 con 0x7fc3241022b0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3241a2680 con 0x7fc3241022b0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc318011d10 con 0x7fc3241022b0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.038+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318010db0 con 0x7fc3241022b0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.039+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3241a2c40 con 0x7fc3241022b0 2024-04-24T23:12:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.040+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc324068f20 con 0x7fc3241022b0 2024-04-24T23:12:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.041+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc318018020 con 0x7fc3241022b0 2024-04-24T23:12:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.041+0000 7fc3137fe700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc30c0a2c20 0x7fc30c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.041+0000 7fc321d9b700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc30c0a2c20 0x7fc30c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.041+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc3180ceb10 con 0x7fc3241022b0 2024-04-24T23:12:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.042+0000 7fc321d9b700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc30c0a2c20 0x7fc30c0a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc314000c00 tx=0x7fc314005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.046+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc31801f020 con 0x7fc3241022b0 2024-04-24T23:12:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.262+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc324062670 con 0x7fc30c0a2c20 2024-04-24T23:12:16.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.263+0000 7fc3137fe700 1 -- 172.21.15.33:0/2126741204 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc324062670 con 0x7fc30c0a2c20 2024-04-24T23:12:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-04-24T23:12:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-04-24T23:12:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:smithi038 172.21.15.38 2024-04-24T23:12:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:smithi117 172.21.15.117 2024-04-24T23:12:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-04-24T23:12:16.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc30c0a2c20 msgr2=0x7fc30c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc30c0a2c20 0x7fc30c0a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc314000c00 tx=0x7fc314005fb0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 msgr2=0x7fc324075fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 0x7fc324075fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc31800b500 tx=0x7fc31800b810 comp rx=0 tx=0).stop 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 shutdown_connections 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc30c0a2c20 0x7fc30c0a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324103e50 0x7fc324074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3241022b0 0x7fc324075fd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 --2- 172.21.15.33:0/2126741204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3241034b0 0x7fc324074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 >> 172.21.15.33:0/2126741204 conn(0x7fc3240fd850 msgr2=0x7fc3241067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 shutdown_connections 2024-04-24T23:12:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:16.265+0000 7fc3289db700 1 -- 172.21.15.33:0/2126741204 wait complete. 2024-04-24T23:12:16.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T23:12:17.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:16 smithi038 ceph-mon[32450]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 114 B/s wr, 4 op/s 2024-04-24T23:12:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:16 smithi038 ceph-mon[32450]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:16 smithi117 ceph-mon[32529]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 114 B/s wr, 4 op/s 2024-04-24T23:12:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:16 smithi117 ceph-mon[32529]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:16 smithi033 ceph-mon[29201]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 114 B/s wr, 4 op/s 2024-04-24T23:12:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:16 smithi033 ceph-mon[29201]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 -- 172.21.15.33:0/3101680441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff404102d50 msgr2=0x7ff404109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 --2- 172.21.15.33:0/3101680441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff404102d50 0x7ff404109600 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b0d0 tx=0x7ff3fc00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:12:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 -- 172.21.15.33:0/3101680441 shutdown_connections 2024-04-24T23:12:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 --2- 172.21.15.33:0/3101680441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff404102d50 0x7ff404109600 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 --2- 172.21.15.33:0/3101680441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041011b0 0x7ff4041015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 --2- 172.21.15.33:0/3101680441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4041023b0 0x7ff404102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 -- 172.21.15.33:0/3101680441 >> 172.21.15.33:0/3101680441 conn(0x7ff4040fc750 msgr2=0x7ff4040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 -- 172.21.15.33:0/3101680441 shutdown_connections 2024-04-24T23:12:18.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.963+0000 7ff40972b700 1 -- 172.21.15.33:0/3101680441 wait complete. 2024-04-24T23:12:18.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:18 smithi033 ceph-mon[29201]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T23:12:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.964+0000 7ff40972b700 1 Processor -- start 2024-04-24T23:12:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.964+0000 7ff40972b700 1 -- start start 2024-04-24T23:12:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff40972b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 0x7ff40419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff40972b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041023b0 0x7ff40419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff40972b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff404102d50 0x7ff4041a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff40972b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410cd80 con 0x7ff4041011b0 2024-04-24T23:12:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff40972b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410cc00 con 0x7ff4041023b0 2024-04-24T23:12:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff40972b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410cf00 con 0x7ff404102d50 2024-04-24T23:12:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 0x7ff40419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 0x7ff40419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42250/0 (socket says 172.21.15.33:42250) 2024-04-24T23:12:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 -- 172.21.15.33:0/980599241 learned_addr learned my addr 172.21.15.33:0/980599241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff4027fc700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041023b0 0x7ff40419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff4037fe700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff404102d50 0x7ff4041a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 -- 172.21.15.33:0/980599241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff404102d50 msgr2=0x7ff4041a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff404102d50 0x7ff4041a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 -- 172.21.15.33:0/980599241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041023b0 msgr2=0x7ff40419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041023b0 0x7ff40419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 -- 172.21.15.33:0/980599241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041a21d0 con 0x7ff4041011b0 2024-04-24T23:12:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff4027fc700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041023b0 0x7ff40419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff402ffd700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 0x7ff40419a570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e9a0 tx=0x7ff3f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.965+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4004220 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.966+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f40043c0 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.966+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041a24c0 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.966+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a2a00 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.966+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4003df0 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.967+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f0005360 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.968+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3f400c810 con 0x7ff4041011b0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.968+0000 7ff3ebfff700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.969+0000 7ff4027fc700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.969+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff3f40cf660 con 0x7ff4041011b0 2024-04-24T23:12:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.969+0000 7ff4027fc700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800afd0 tx=0x7ff3f800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:18.972+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff3f409c230 con 0x7ff4041011b0 2024-04-24T23:12:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:18 smithi038 ceph-mon[32450]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T23:12:19.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:18 smithi117 ceph-mon[32529]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T23:12:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.186+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff3f0000bf0 con 0x7ff3ec0a2c20 2024-04-24T23:12:19.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.188+0000 7ff3ebfff700 1 -- 172.21.15.33:0/980599241 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff3f0000bf0 con 0x7ff3ec0a2c20 2024-04-24T23:12:19.190 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T23:12:19.190 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:12:19.190 INFO:teuthology.orchestra.run.smithi033.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:12:19.190 INFO:teuthology.orchestra.run.smithi033.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff3ec0a2c20 msgr2=0x7ff3ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800afd0 tx=0x7ff3f800a380 comp rx=0 tx=0).stop 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 msgr2=0x7ff40419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 0x7ff40419a570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e9a0 tx=0x7ff3f400ecb0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 shutdown_connections 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff404102d50 0x7ff4041a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041011b0 0x7ff40419a570 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 --2- 172.21.15.33:0/980599241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4041023b0 0x7ff40419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 >> 172.21.15.33:0/980599241 conn(0x7ff4040fc750 msgr2=0x7ff40406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 shutdown_connections 2024-04-24T23:12:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:19.190+0000 7ff40972b700 1 -- 172.21.15.33:0/980599241 wait complete. 2024-04-24T23:12:19.863 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:12:19.872 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-24T23:12:19.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:12:20.130 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:20.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:19 smithi038 ceph-mon[32450]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:20.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:19 smithi117 ceph-mon[32529]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:20.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:19 smithi033 ceph-mon[29201]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:21.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:20 smithi038 ceph-mon[32450]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:12:21.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:20 smithi117 ceph-mon[32529]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:12:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:20 smithi033 ceph-mon[29201]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:12:21.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 -- 172.21.15.33:0/3185410435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630102390 msgr2=0x7fb6301027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 --2- 172.21.15.33:0/3185410435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630102390 0x7fb6301027f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb6180099c0 tx=0x7fb618009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 -- 172.21.15.33:0/3185410435 shutdown_connections 2024-04-24T23:12:21.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 --2- 172.21.15.33:0/3185410435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb630102d30 0x7fb6301095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 --2- 172.21.15.33:0/3185410435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630101190 0x7fb6301015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 --2- 172.21.15.33:0/3185410435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630102390 0x7fb6301027f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 -- 172.21.15.33:0/3185410435 >> 172.21.15.33:0/3185410435 conn(0x7fb6300fc730 msgr2=0x7fb6300feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:21.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 -- 172.21.15.33:0/3185410435 shutdown_connections 2024-04-24T23:12:21.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.818+0000 7fb635540700 1 -- 172.21.15.33:0/3185410435 wait complete. 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 Processor -- start 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 -- start start 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 0x7fb6301984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630102390 0x7fb6301989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb630102d30 0x7fb63019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010cd60 con 0x7fb630101190 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010cbe0 con 0x7fb630102d30 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb635540700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb63010cee0 con 0x7fb630102390 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 0x7fb6301984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 0x7fb6301984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42264/0 (socket says 172.21.15.33:42264) 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 -- 172.21.15.33:0/35977588 learned_addr learned my addr 172.21.15.33:0/35977588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62e7fc700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630102390 0x7fb6301989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62f7fe700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb630102d30 0x7fb63019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 -- 172.21.15.33:0/35977588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630102390 msgr2=0x7fb6301989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630102390 0x7fb6301989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 -- 172.21.15.33:0/35977588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb630102d30 msgr2=0x7fb63019fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb630102d30 0x7fb63019fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.819+0000 7fb62effd700 1 -- 172.21.15.33:0/35977588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6301a0100 con 0x7fb630101190 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb62e7fc700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630102390 0x7fb6301989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb62effd700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 0x7fb6301984a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb620007e30 tx=0x7fb62000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620016020 con 0x7fb630101190 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6301a03f0 con 0x7fb630101190 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62000adc0 con 0x7fb630101190 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301a0980 con 0x7fb630101190 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.820+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200116e0 con 0x7fb630101190 2024-04-24T23:12:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.821+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb630068f20 con 0x7fb630101190 2024-04-24T23:12:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.822+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb620019020 con 0x7fb630101190 2024-04-24T23:12:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.823+0000 7fb627fff700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb61c0a2c20 0x7fb61c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.823+0000 7fb62e7fc700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb61c0a2c20 0x7fb61c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.823+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb6200cf4b0 con 0x7fb630101190 2024-04-24T23:12:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.823+0000 7fb62e7fc700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb61c0a2c20 0x7fb61c0a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb6180039c0 tx=0x7fb618005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:21.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:21.828+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb620020020 con 0x7fb630101190 2024-04-24T23:12:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.085+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb6301a0ca0 con 0x7fb630101190 2024-04-24T23:12:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.086+0000 7fb627fff700 1 -- 172.21.15.33:0/35977588 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3043 (secure 0 0 0) 0x7fb62009c020 con 0x7fb630101190 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:e24 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 24 2024-04-24T23:12:22.088 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-24T23:10:11.664486+0000 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-24T23:10:38.464642+0000 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 3 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2 2024-04-24T23:12:22.089 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24508,1=24481,2=24479} 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3,4] 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 24508 members: 24479,24481,24508 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{0:24508} state up:active seq 5 addr [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{0:24469} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/221661334,v1:172.21.15.117:6835/221661334] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{1:24481} state up:active seq 13 addr [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.090 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{1:24455} state up:standby-replay seq 1 addr [v2:172.21.15.33:6838/2446242312,v1:172.21.15.33:6839/2446242312] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{2:24479} state up:active seq 9 addr [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.38:6836/287276222,v1:172.21.15.38:6837/287276222] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.33:6834/1264288095,v1:172.21.15.33:6835/1264288095] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.33:6836/1577112159,v1:172.21.15.33:6837/1577112159] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{-1:14628} state up:standby seq 1 addr [v2:172.21.15.33:6840/1277151249,v1:172.21.15.33:6841/1277151249] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{-1:24380} state up:standby seq 1 addr [v2:172.21.15.38:6832/2605742679,v1:172.21.15.38:6833/2605742679] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{-1:24404} state up:standby seq 1 addr [v2:172.21.15.38:6834/1005158826,v1:172.21.15.38:6835/1005158826] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.091 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.117:6832/2542827674,v1:172.21.15.117:6833/2542827674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:22.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb61c0a2c20 msgr2=0x7fb61c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:22.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb61c0a2c20 0x7fb61c0a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb6180039c0 tx=0x7fb618005fd0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 msgr2=0x7fb6301984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:22.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 0x7fb6301984a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb620007e30 tx=0x7fb62000a340 comp rx=0 tx=0).stop 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 shutdown_connections 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb61c0a2c20 0x7fb61c0a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb630102d30 0x7fb63019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb630101190 0x7fb6301984a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 --2- 172.21.15.33:0/35977588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630102390 0x7fb6301989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 >> 172.21.15.33:0/35977588 conn(0x7fb6300fc730 msgr2=0x7fb6300fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 shutdown_connections 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:22.088+0000 7fb635540700 1 -- 172.21.15.33:0/35977588 wait complete. 2024-04-24T23:12:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:12:22.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T23:12:23.056 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:23.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:22 smithi033 ceph-mon[29201]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T23:12:23.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:22 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/35977588' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:12:23.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:22 smithi038 ceph-mon[32450]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T23:12:23.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:22 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/35977588' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:12:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:22 smithi117 ceph-mon[32529]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T23:12:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:22 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/35977588' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:12:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 -- 172.21.15.33:0/1563514222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8102d30 msgr2=0x7fa4b81095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/1563514222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8102d30 0x7fa4b81095e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b0d0 tx=0x7fa4b000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 -- 172.21.15.33:0/1563514222 shutdown_connections 2024-04-24T23:12:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/1563514222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8102d30 0x7fa4b81095e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/1563514222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b8101190 0x7fa4b81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/1563514222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102390 0x7fa4b81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 -- 172.21.15.33:0/1563514222 >> 172.21.15.33:0/1563514222 conn(0x7fa4b80fc730 msgr2=0x7fa4b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 -- 172.21.15.33:0/1563514222 shutdown_connections 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.753+0000 7fa4bdc12700 1 -- 172.21.15.33:0/1563514222 wait complete. 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 Processor -- start 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 -- start start 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 0x7fa4b806c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b8102390 0x7fa4b806c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102d30 0x7fa4b8073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810cd80 con 0x7fa4b8101190 2024-04-24T23:12:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810cc00 con 0x7fa4b8102d30 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4bdc12700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810cf00 con 0x7fa4b8102390 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 0x7fa4b806c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 0x7fa4b806c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42288/0 (socket says 172.21.15.33:42288) 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4b77fe700 1 -- 172.21.15.33:0/2489294679 learned_addr learned my addr 172.21.15.33:0/2489294679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4b7fff700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102d30 0x7fa4b8073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.754+0000 7fa4aeffd700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b8102390 0x7fa4b806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b77fe700 1 -- 172.21.15.33:0/2489294679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b8102390 msgr2=0x7fa4b806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b77fe700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b8102390 0x7fa4b806c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b77fe700 1 -- 172.21.15.33:0/2489294679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102d30 msgr2=0x7fa4b8073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b77fe700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102d30 0x7fa4b8073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b77fe700 1 -- 172.21.15.33:0/2489294679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b8073d00 con 0x7fa4b8101190 2024-04-24T23:12:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b7fff700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102d30 0x7fa4b8073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b77fe700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 0x7fa4b806c040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800d980 tx=0x7fa4a800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8017070 con 0x7fa4b8101190 2024-04-24T23:12:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b8073ff0 con 0x7fa4b8101190 2024-04-24T23:12:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a8009b40 con 0x7fa4b8101190 2024-04-24T23:12:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a800f620 con 0x7fa4b8101190 2024-04-24T23:12:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.755+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a9280 con 0x7fa4b8101190 2024-04-24T23:12:24.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.756+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b8068f20 con 0x7fa4b8101190 2024-04-24T23:12:24.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.757+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4a8010460 con 0x7fa4b8101190 2024-04-24T23:12:24.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.758+0000 7fa4b57fa700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.758+0000 7fa4aeffd700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.758+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4a80cf0f0 con 0x7fa4b8101190 2024-04-24T23:12:24.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.759+0000 7fa4aeffd700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005fd0 tx=0x7fa4a0005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.762+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4a809bcc0 con 0x7fa4b8101190 2024-04-24T23:12:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.973+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa4b81a9b90 con 0x7fa4b8101190 2024-04-24T23:12:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.974+0000 7fa4b57fa700 1 -- 172.21.15.33:0/2489294679 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5643 (secure 0 0 0) 0x7fa4a809b5c0 con 0x7fa4b8101190 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 94 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-24T23:02:09.732561+0000 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-24T23:12:07.840931+0000 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 27 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-04-24T23:12:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-24T23:12:24.977 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/2906488193,v1:172.21.15.33:6803/2906488193] [v2:172.21.15.33:6804/2906488193,v1:172.21.15.33:6805/2906488193] exists,up d93333af-c06d-4be4-91c8-bd832217bd8b 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6800/2106722793,v1:172.21.15.38:6801/2106722793] [v2:172.21.15.38:6802/2106722793,v1:172.21.15.38:6803/2106722793] exists,up 0c497d09-e263-481e-a4dc-5352cb3ae2d6 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/1483584233,v1:172.21.15.117:6801/1483584233] [v2:172.21.15.117:6802/1483584233,v1:172.21.15.117:6803/1483584233] exists,up 956cb2b5-6d5f-4817-b066-74c8227c7960 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/1011423676,v1:172.21.15.33:6811/1011423676] [v2:172.21.15.33:6812/1011423676,v1:172.21.15.33:6813/1011423676] exists,up 8ec098a5-9343-4caf-931d-f4c79d4fbdf3 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6808/1943632582,v1:172.21.15.38:6809/1943632582] [v2:172.21.15.38:6810/1943632582,v1:172.21.15.38:6811/1943632582] exists,up 34a597a4-f838-4edf-9198-7d674b4aa9d1 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/2567468541,v1:172.21.15.117:6809/2567468541] [v2:172.21.15.117:6810/2567468541,v1:172.21.15.117:6811/2567468541] exists,up f4fef747-c797-4d33-b6bd-c1f7d55e052b 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 47 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/713285361,v1:172.21.15.33:6819/713285361] [v2:172.21.15.33:6820/713285361,v1:172.21.15.33:6821/713285361] exists,up aeb1c83d-c2cd-4d7b-95dc-dff2b00400db 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6816/3279589891,v1:172.21.15.38:6817/3279589891] [v2:172.21.15.38:6818/3279589891,v1:172.21.15.38:6819/3279589891] exists,up 1e464d3b-4514-43db-9449-25f51b362bab 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/1247270534,v1:172.21.15.117:6817/1247270534] [v2:172.21.15.117:6818/1247270534,v1:172.21.15.117:6819/1247270534] exists,up 69890093-ead1-4825-8a46-26a2b2c1b401 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/498632081,v1:172.21.15.33:6827/498632081] [v2:172.21.15.33:6828/498632081,v1:172.21.15.33:6829/498632081] exists,up 8293c811-4410-4f09-9ea7-4de698fc76c0 2024-04-24T23:12:24.978 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6824/1322822430,v1:172.21.15.38:6825/1322822430] [v2:172.21.15.38:6826/1322822430,v1:172.21.15.38:6827/1322822430] exists,up 57c8391d-387d-400c-80da-ac841c5d4608 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/2336105808,v1:172.21.15.117:6825/2336105808] [v2:172.21.15.117:6826/2336105808,v1:172.21.15.117:6827/2336105808] exists,up c8842cc2-f8eb-4b78-9439-9aea72e0ae4b 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.c [2,1] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.13 [2,11] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1d [8,11] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1f [8,4] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.e [4,8] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.11 [5,9] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.34 [4,9] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.3c [4,9] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.6 [9,11] 2024-04-24T23:12:24.979 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.b [9,8] 2024-04-24T23:12:24.980 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.15 [5,10] 2024-04-24T23:12:24.980 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.16 [9,8] 2024-04-24T23:12:24.980 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/946624938 expires 2024-04-25T23:03:05.692528+0000 2024-04-24T23:12:24.980 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2813273186 expires 2024-04-25T23:03:05.692528+0000 2024-04-24T23:12:24.980 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2813273186 expires 2024-04-25T23:03:05.692528+0000 2024-04-24T23:12:24.981 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1214692934 expires 2024-04-25T23:02:38.216643+0000 2024-04-24T23:12:24.981 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1172519445 expires 2024-04-25T23:02:38.216643+0000 2024-04-24T23:12:24.981 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/1299073362 expires 2024-04-25T23:02:38.216643+0000 2024-04-24T23:12:24.981 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/1299073362 expires 2024-04-25T23:02:38.216643+0000 2024-04-24T23:12:24.981 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1574498973 expires 2024-04-25T23:03:05.692528+0000 2024-04-24T23:12:24.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/288675277 expires 2024-04-25T23:02:38.216643+0000 2024-04-24T23:12:24.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2398589700 expires 2024-04-25T23:03:05.692528+0000 2024-04-24T23:12:24.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4a40a2c20 msgr2=0x7fa4a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005fd0 tx=0x7fa4a0005f60 comp rx=0 tx=0).stop 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 msgr2=0x7fa4b806c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 0x7fa4b806c040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800d980 tx=0x7fa4a800dd40 comp rx=0 tx=0).stop 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 shutdown_connections 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4a40a2c20 0x7fa4a40a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b8102d30 0x7fa4b8073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b8101190 0x7fa4b806c040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 --2- 172.21.15.33:0/2489294679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b8102390 0x7fa4b806c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 >> 172.21.15.33:0/2489294679 conn(0x7fa4b80fc730 msgr2=0x7fa4b80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 shutdown_connections 2024-04-24T23:12:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:24.976+0000 7fa4bdc12700 1 -- 172.21.15.33:0/2489294679 wait complete. 2024-04-24T23:12:25.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:24 smithi038 ceph-mon[32450]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:25.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:24 smithi117 ceph-mon[32529]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:25.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:24 smithi033 ceph-mon[29201]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:25.675 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T23:12:25.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:25.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.791+0000 7f260c163640 1 -- 172.21.15.33:0/2459452996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604101310 msgr2=0x7f2604101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.791+0000 7f260c163640 1 --2- 172.21.15.33:0/2459452996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604101310 0x7f2604101790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ca0 tx=0x7f25fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.791+0000 7f260c163640 1 -- 172.21.15.33:0/2459452996 shutdown_connections 2024-04-24T23:12:25.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 --2- 172.21.15.33:0/2459452996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604101310 0x7f2604101790 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 --2- 172.21.15.33:0/2459452996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604100970 0x7f2604100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 --2- 172.21.15.33:0/2459452996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2604100010 0x7f2604100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- 172.21.15.33:0/2459452996 >> 172.21.15.33:0/2459452996 conn(0x7f26040fbc30 msgr2=0x7f26040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- 172.21.15.33:0/2459452996 shutdown_connections 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- 172.21.15.33:0/2459452996 wait complete. 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 Processor -- start 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- start start 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 0x7f2604199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2604100970 0x7f260419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604101310 0x7f26041a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f260410f660 con 0x7f2604100010 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f260410f4e0 con 0x7f2604100970 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.792+0000 7f260c163640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f260410f7e0 con 0x7f2604101310 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 0x7f2604199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 0x7f2604199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60988/0 (socket says 172.21.15.33:60988) 2024-04-24T23:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 -- 172.21.15.33:0/2527335955 learned_addr learned my addr 172.21.15.33:0/2527335955 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f26096fe640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2604100970 0x7f260419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f260a700640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604101310 0x7f26041a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 -- 172.21.15.33:0/2527335955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604101310 msgr2=0x7f26041a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604101310 0x7f26041a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 -- 172.21.15.33:0/2527335955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2604100970 msgr2=0x7f260419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2604100970 0x7f260419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 -- 172.21.15.33:0/2527335955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26041a1870 con 0x7f2604100010 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f260a700640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604101310 0x7f26041a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f2609eff640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 0x7f2604199c60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f260000d950 tx=0x7f260000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600017070 con 0x7f2604100010 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260000de80 con 0x7f2604100010 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041a1b60 con 0x7f2604100010 2024-04-24T23:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000f620 con 0x7f2604100010 2024-04-24T23:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.793+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a20b0 con 0x7f2604100010 2024-04-24T23:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.794+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260410ad60 con 0x7f2604100010 2024-04-24T23:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.795+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2600009c30 con 0x7f2604100010 2024-04-24T23:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.796+0000 7f25faffd640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f25f00a2b90 0x7f25f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.796+0000 7f26096fe640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f25f00a2b90 0x7f25f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.796+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f26000cf0d0 con 0x7f2604100010 2024-04-24T23:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.797+0000 7f26096fe640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f25f00a2b90 0x7f25f00a5050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009990 tx=0x7f25f4009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.801+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f260009bca0 con 0x7f2604100010 2024-04-24T23:12:25.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2489294679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:12:25.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.987+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26040623e0 con 0x7f2604100010 2024-04-24T23:12:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.989+0000 7f25faffd640 1 -- 172.21.15.33:0/2527335955 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7f260009b5a0 con 0x7f2604100010 2024-04-24T23:12:25.991 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:25.992 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f25f00a2b90 msgr2=0x7f25f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f25f00a2b90 0x7f25f00a5050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009990 tx=0x7f25f4009110 comp rx=0 tx=0).stop 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 msgr2=0x7f2604199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 0x7f2604199c60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f260000d950 tx=0x7f260000dd50 comp rx=0 tx=0).stop 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 shutdown_connections 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f25f00a2b90 0x7f25f00a5050 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604101310 0x7f26041a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2604100970 0x7f260419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 --2- 172.21.15.33:0/2527335955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2604100010 0x7f2604199c60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 >> 172.21.15.33:0/2527335955 conn(0x7f26040fbc30 msgr2=0x7f26040fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 shutdown_connections 2024-04-24T23:12:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:25.991+0000 7f260c163640 1 -- 172.21.15.33:0/2527335955 wait complete. 2024-04-24T23:12:26.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:26.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- 172.21.15.33:0/762078866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 msgr2=0x7efc30108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- 172.21.15.33:0/762078866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc30108ac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efc28009ca0 tx=0x7efc2802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- 172.21.15.33:0/762078866 shutdown_connections 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- 172.21.15.33:0/762078866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc30108ac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- 172.21.15.33:0/762078866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30101870 0x7efc30101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- 172.21.15.33:0/762078866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc30100680 0x7efc30100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- 172.21.15.33:0/762078866 >> 172.21.15.33:0/762078866 conn(0x7efc300fbbf0 msgr2=0x7efc300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- 172.21.15.33:0/762078866 shutdown_connections 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- 172.21.15.33:0/762078866 wait complete. 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 Processor -- start 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- start start 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30100680 0x7efc30197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc30101870 0x7efc30197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc3019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010d480 con 0x7efc30102230 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010d300 con 0x7efc30101870 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.109+0000 7efc385d5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010d600 con 0x7efc30100680 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc3019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc3019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32784/0 (socket says 172.21.15.33:32784) 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 -- 172.21.15.33:0/604533083 learned_addr learned my addr 172.21.15.33:0/604533083 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc35b70640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc30101870 0x7efc30197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36371640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30100680 0x7efc30197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 -- 172.21.15.33:0/604533083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30100680 msgr2=0x7efc30197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30100680 0x7efc30197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 -- 172.21.15.33:0/604533083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc30101870 msgr2=0x7efc30197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc30101870 0x7efc30197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 -- 172.21.15.33:0/604533083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3019f680 con 0x7efc30102230 2024-04-24T23:12:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc36b72640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc3019f010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc280319f0 tx=0x7efc2802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28003c70 con 0x7efc30102230 2024-04-24T23:12:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3019f910 con 0x7efc30102230 2024-04-24T23:12:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc28003e10 con 0x7efc30102230 2024-04-24T23:12:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2802fd30 con 0x7efc30102230 2024-04-24T23:12:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.110+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3019fe60 con 0x7efc30102230 2024-04-24T23:12:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.111+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30107940 con 0x7efc30102230 2024-04-24T23:12:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.113+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc28004040 con 0x7efc30102230 2024-04-24T23:12:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.113+0000 7efc277fe640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efc1c0a2b90 0x7efc1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.113+0000 7efc36371640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efc1c0a2b90 0x7efc1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.114+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efc280f08c0 con 0x7efc30102230 2024-04-24T23:12:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.114+0000 7efc36371640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efc1c0a2b90 0x7efc1c0a5050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005fd0 tx=0x7efc2c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.118+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc280bd490 con 0x7efc30102230 2024-04-24T23:12:26.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2489294679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:12:26.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2489294679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:12:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.301+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc30062340 con 0x7efc30102230 2024-04-24T23:12:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.301+0000 7efc277fe640 1 -- 172.21.15.33:0/604533083 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7efc280bcd90 con 0x7efc30102230 2024-04-24T23:12:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efc1c0a2b90 msgr2=0x7efc1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efc1c0a2b90 0x7efc1c0a5050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005fd0 tx=0x7efc2c00b410 comp rx=0 tx=0).stop 2024-04-24T23:12:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 msgr2=0x7efc3019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc3019f010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc280319f0 tx=0x7efc2802f450 comp rx=0 tx=0).stop 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 shutdown_connections 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efc1c0a2b90 0x7efc1c0a5050 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc30102230 0x7efc3019f010 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc30101870 0x7efc30197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 --2- 172.21.15.33:0/604533083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30100680 0x7efc30197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 >> 172.21.15.33:0/604533083 conn(0x7efc300fbbf0 msgr2=0x7efc30104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 shutdown_connections 2024-04-24T23:12:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.303+0000 7efc385d5640 1 -- 172.21.15.33:0/604533083 wait complete. 2024-04-24T23:12:26.317 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:26.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 -- 172.21.15.33:0/150857457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b501006a0 msgr2=0x7f4b50100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 --2- 172.21.15.33:0/150857457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b501006a0 0x7f4b50100ac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009900 tx=0x7f4b4002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 -- 172.21.15.33:0/150857457 shutdown_connections 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 --2- 172.21.15.33:0/150857457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b50102250 0x7f4b50108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 --2- 172.21.15.33:0/150857457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b50101890 0x7f4b50101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 --2- 172.21.15.33:0/150857457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b501006a0 0x7f4b50100ac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 -- 172.21.15.33:0/150857457 >> 172.21.15.33:0/150857457 conn(0x7f4b500fbc30 msgr2=0x7f4b500fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 -- 172.21.15.33:0/150857457 shutdown_connections 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.419+0000 7f4b56a37640 1 -- 172.21.15.33:0/150857457 wait complete. 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 Processor -- start 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 -- start start 2024-04-24T23:12:26.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b501006a0 0x7f4b50197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b50101890 0x7f4b50198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 0x7f4b5019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010d440 con 0x7f4b50102250 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010d2c0 con 0x7f4b50101890 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b56a37640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010d5c0 con 0x7f4b501006a0 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b54fd4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 0x7f4b5019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b54fd4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 0x7f4b5019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32798/0 (socket says 172.21.15.33:32798) 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b54fd4640 1 -- 172.21.15.33:0/3916009160 learned_addr learned my addr 172.21.15.33:0/3916009160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.420+0000 7f4b4ffff640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b501006a0 0x7f4b50197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b4f7fe640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b50101890 0x7f4b50198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b54fd4640 1 -- 172.21.15.33:0/3916009160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b501006a0 msgr2=0x7f4b50197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b54fd4640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b501006a0 0x7f4b50197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b54fd4640 1 -- 172.21.15.33:0/3916009160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b50101890 msgr2=0x7f4b50198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b54fd4640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b50101890 0x7f4b50198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b54fd4640 1 -- 172.21.15.33:0/3916009160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5019f6f0 con 0x7f4b50102250 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b54fd4640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 0x7f4b5019f080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800eaa0 tx=0x7f4b4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800cdb0 con 0x7f4b50102250 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b48005020 con 0x7f4b50102250 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5019f9e0 con 0x7f4b50102250 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480116b0 con 0x7f4b50102250 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.421+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5019ff30 con 0x7f4b50102250 2024-04-24T23:12:26.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.422+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50107a60 con 0x7f4b50102250 2024-04-24T23:12:26.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.423+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b48005560 con 0x7f4b50102250 2024-04-24T23:12:26.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.424+0000 7f4b4d7fa640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.424+0000 7f4b4ffff640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.424+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b480cf540 con 0x7f4b50102250 2024-04-24T23:12:26.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.424+0000 7f4b4ffff640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4b40005ab0 tx=0x7f4b40000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.428+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b48016030 con 0x7f4b50102250 2024-04-24T23:12:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.612+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b50062340 con 0x7f4b50102250 2024-04-24T23:12:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.612+0000 7f4b4d7fa640 1 -- 172.21.15.33:0/3916009160 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f4b4809bac0 con 0x7f4b50102250 2024-04-24T23:12:26.614 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.614+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b3c0a2c10 msgr2=0x7f4b3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.614+0000 7f4b56a37640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4b40005ab0 tx=0x7f4b40000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 msgr2=0x7f4b5019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 0x7f4b5019f080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800eaa0 tx=0x7f4b4800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 shutdown_connections 2024-04-24T23:12:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b50102250 0x7f4b5019f080 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b50101890 0x7f4b50198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 --2- 172.21.15.33:0/3916009160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b501006a0 0x7f4b50197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 >> 172.21.15.33:0/3916009160 conn(0x7f4b500fbc30 msgr2=0x7f4b50104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 shutdown_connections 2024-04-24T23:12:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.615+0000 7f4b56a37640 1 -- 172.21.15.33:0/3916009160 wait complete. 2024-04-24T23:12:26.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:26.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 -- 172.21.15.33:0/3930676586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 msgr2=0x7efea01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 --2- 172.21.15.33:0/3930676586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea01098b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efe98009ca0 tx=0x7efe9802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 -- 172.21.15.33:0/3930676586 shutdown_connections 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 --2- 172.21.15.33:0/3930676586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea01098b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 --2- 172.21.15.33:0/3930676586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104b30 0x7efea0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 --2- 172.21.15.33:0/3930676586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 0x7efea01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 -- 172.21.15.33:0/3930676586 >> 172.21.15.33:0/3930676586 conn(0x7efea00fbbf0 msgr2=0x7efea00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 -- 172.21.15.33:0/3930676586 shutdown_connections 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 -- 172.21.15.33:0/3930676586 wait complete. 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 Processor -- start 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.731+0000 7efea664c640 1 -- start start 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 0x7efea01979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104b30 0x7efea0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efea010d400 con 0x7efea0107460 2024-04-24T23:12:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efea010d280 con 0x7efea0102200 2024-04-24T23:12:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efea010d580 con 0x7efea0104b30 2024-04-24T23:12:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32814/0 (socket says 172.21.15.33:32814) 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 -- 172.21.15.33:0/4026367354 learned_addr learned my addr 172.21.15.33:0/4026367354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efe9ffff640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 0x7efea01979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efe9f7fe640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104b30 0x7efea0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 -- 172.21.15.33:0/4026367354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104b30 msgr2=0x7efea0197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104b30 0x7efea0197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 -- 172.21.15.33:0/4026367354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 msgr2=0x7efea01979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 0x7efea01979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 -- 172.21.15.33:0/4026367354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea019f600 con 0x7efea0107460 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efe9ffff640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 0x7efea01979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea4be9640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea019ef90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efe980098d0 tx=0x7efe9802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9802fc30 con 0x7efea0107460 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea019f890 con 0x7efea0107460 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe98003e50 con 0x7efea0107460 2024-04-24T23:12:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980305c0 con 0x7efea0107460 2024-04-24T23:12:26.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.732+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea019fde0 con 0x7efea0107460 2024-04-24T23:12:26.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.733+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea00fd6e0 con 0x7efea0107460 2024-04-24T23:12:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.735+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe98003970 con 0x7efea0107460 2024-04-24T23:12:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.736+0000 7efe9d7fa640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe880a2b90 0x7efe880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.736+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe980f09e0 con 0x7efea0107460 2024-04-24T23:12:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.736+0000 7efe9ffff640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe880a2b90 0x7efe880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.736+0000 7efe9ffff640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe880a2b90 0x7efe880a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efe9000abe0 tx=0x7efe9000a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.739+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe980bd5b0 con 0x7efea0107460 2024-04-24T23:12:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.923+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efea002d300 con 0x7efea0107460 2024-04-24T23:12:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.924+0000 7efe9d7fa640 1 -- 172.21.15.33:0/4026367354 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7efe980bceb0 con 0x7efea0107460 2024-04-24T23:12:26.926 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe880a2b90 msgr2=0x7efe880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe880a2b90 0x7efe880a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efe9000abe0 tx=0x7efe9000a5c0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 msgr2=0x7efea019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea019ef90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efe980098d0 tx=0x7efe9802f840 comp rx=0 tx=0).stop 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 shutdown_connections 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe880a2b90 0x7efe880a5050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efea0107460 0x7efea019ef90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104b30 0x7efea0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 --2- 172.21.15.33:0/4026367354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efea0102200 0x7efea01979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 >> 172.21.15.33:0/4026367354 conn(0x7efea00fbbf0 msgr2=0x7efea0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 shutdown_connections 2024-04-24T23:12:26.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:26.926+0000 7efea664c640 1 -- 172.21.15.33:0/4026367354 wait complete. 2024-04-24T23:12:26.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 -- 172.21.15.33:0/540876418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744102230 msgr2=0x7f8744108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 --2- 172.21.15.33:0/540876418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744102230 0x7f8744108ac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f873c009ca0 tx=0x7f873c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 -- 172.21.15.33:0/540876418 shutdown_connections 2024-04-24T23:12:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 --2- 172.21.15.33:0/540876418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744102230 0x7f8744108ac0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 --2- 172.21.15.33:0/540876418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8744101870 0x7f8744101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 --2- 172.21.15.33:0/540876418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744100680 0x7f8744100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 -- 172.21.15.33:0/540876418 >> 172.21.15.33:0/540876418 conn(0x7f87440fbbf0 msgr2=0x7f87440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 -- 172.21.15.33:0/540876418 shutdown_connections 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.043+0000 7f874a2ba640 1 -- 172.21.15.33:0/540876418 wait complete. 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 Processor -- start 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 -- start start 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8744100680 0x7f8744197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 0x7f8744197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744102230 0x7f874419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f874410c300 con 0x7f8744101870 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f874410c180 con 0x7f8744100680 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f874a2ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f874410c480 con 0x7f8744102230 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 0x7f8744197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 0x7f8744197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32830/0 (socket says 172.21.15.33:32830) 2024-04-24T23:12:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 -- 172.21.15.33:0/623065700 learned_addr learned my addr 172.21.15.33:0/623065700 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f8743fff640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8744100680 0x7f8744197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f8748857640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744102230 0x7f874419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 -- 172.21.15.33:0/623065700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744102230 msgr2=0x7f874419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744102230 0x7f874419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 -- 172.21.15.33:0/623065700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8744100680 msgr2=0x7f8744197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8744100680 0x7f8744197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 -- 172.21.15.33:0/623065700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874419f6b0 con 0x7f8744101870 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.044+0000 7f87437fe640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 0x7f8744197fd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f873800d950 tx=0x7f873800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.045+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738017030 con 0x7f8744101870 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.045+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8738004020 con 0x7f8744101870 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.045+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874419f9a0 con 0x7f8744101870 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.045+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87380047a0 con 0x7f8744101870 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.045+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874419fef0 con 0x7f8744101870 2024-04-24T23:12:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.046+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87441a0180 con 0x7f8744101870 2024-04-24T23:12:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.047+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87380041d0 con 0x7f8744101870 2024-04-24T23:12:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.047+0000 7f87417fa640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f872c0a2b90 0x7f872c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.048+0000 7f8743fff640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f872c0a2b90 0x7f872c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.048+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87380cf510 con 0x7f8744101870 2024-04-24T23:12:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.048+0000 7f8743fff640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f872c0a2b90 0x7f872c0a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8734005fd0 tx=0x7f8734005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.051+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f873809c0e0 con 0x7f8744101870 2024-04-24T23:12:27.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:26 smithi038 ceph-mon[32450]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2527335955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/604533083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3916009160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:26 smithi117 ceph-mon[32529]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2527335955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/604533083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3916009160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:26 smithi033 ceph-mon[29201]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:27.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2527335955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/604533083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3916009160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.235+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f87441a0350 con 0x7f8744101870 2024-04-24T23:12:27.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.235+0000 7f87417fa640 1 -- 172.21.15.33:0/623065700 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f873809b9e0 con 0x7f8744101870 2024-04-24T23:12:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f872c0a2b90 msgr2=0x7f872c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f872c0a2b90 0x7f872c0a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8734005fd0 tx=0x7f8734005850 comp rx=0 tx=0).stop 2024-04-24T23:12:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 msgr2=0x7f8744197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 0x7f8744197fd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f873800d950 tx=0x7f873800de10 comp rx=0 tx=0).stop 2024-04-24T23:12:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 shutdown_connections 2024-04-24T23:12:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f872c0a2b90 0x7f872c0a5050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744102230 0x7f874419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8744101870 0x7f8744197fd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 --2- 172.21.15.33:0/623065700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8744100680 0x7f8744197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 >> 172.21.15.33:0/623065700 conn(0x7f87440fbbf0 msgr2=0x7f87440fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 shutdown_connections 2024-04-24T23:12:27.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.237+0000 7f874a2ba640 1 -- 172.21.15.33:0/623065700 wait complete. 2024-04-24T23:12:27.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- 172.21.15.33:0/134792009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01018b0 msgr2=0x7fa8b0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/134792009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01018b0 0x7fa8b0101d30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- 172.21.15.33:0/134792009 shutdown_connections 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/134792009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b0102270 0x7fa8b0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/134792009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01018b0 0x7fa8b0101d30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/134792009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01006c0 0x7fa8b0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- 172.21.15.33:0/134792009 >> 172.21.15.33:0/134792009 conn(0x7fa8b00fbc30 msgr2=0x7fa8b00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- 172.21.15.33:0/134792009 shutdown_connections 2024-04-24T23:12:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- 172.21.15.33:0/134792009 wait complete. 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 Processor -- start 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- start start 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 0x7fa8b0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01018b0 0x7fa8b0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b0102270 0x7fa8b019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010c390 con 0x7fa8b01006c0 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010c210 con 0x7fa8b01018b0 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b6d61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010c510 con 0x7fa8b0102270 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.353+0000 7fa8b4afd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 0x7fa8b0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 0x7fa8b0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32846/0 (socket says 172.21.15.33:32846) 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 -- 172.21.15.33:0/1265539862 learned_addr learned my addr 172.21.15.33:0/1265539862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b52fe640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b0102270 0x7fa8b019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8affff640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01018b0 0x7fa8b0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 -- 172.21.15.33:0/1265539862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b0102270 msgr2=0x7fa8b019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b0102270 0x7fa8b019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 -- 172.21.15.33:0/1265539862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01018b0 msgr2=0x7fa8b0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01018b0 0x7fa8b0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 -- 172.21.15.33:0/1265539862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b019f710 con 0x7fa8b01006c0 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8affff640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01018b0 0x7fa8b0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b4afd640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 0x7fa8b0197b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000d770 tx=0x7fa8a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0017070 con 0x7fa8b01006c0 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a0009c30 con 0x7fa8b01006c0 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b019fa00 con 0x7fa8b01006c0 2024-04-24T23:12:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0003dd0 con 0x7fa8b01006c0 2024-04-24T23:12:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.354+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b019ff50 con 0x7fa8b01006c0 2024-04-24T23:12:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.355+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b01a0180 con 0x7fa8b01006c0 2024-04-24T23:12:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.356+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8a0003fb0 con 0x7fa8b01006c0 2024-04-24T23:12:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.357+0000 7fa8adffb640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa89c0a2b90 0x7fa89c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.357+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8a00cee10 con 0x7fa8b01006c0 2024-04-24T23:12:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.357+0000 7fa8affff640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa89c0a2b90 0x7fa89c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.357+0000 7fa8affff640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa89c0a2b90 0x7fa89c0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa89802a930 tx=0x7fa898000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.361+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa8a009b9e0 con 0x7fa8b01006c0 2024-04-24T23:12:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.585+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8b01a0470 con 0x7fa8b01006c0 2024-04-24T23:12:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.586+0000 7fa8adffb640 1 -- 172.21.15.33:0/1265539862 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7fa8a0009de0 con 0x7fa8b01006c0 2024-04-24T23:12:27.589 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:27.589 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:38.464642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:12:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:12:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa89c0a2b90 msgr2=0x7fa89c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa89c0a2b90 0x7fa89c0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa89802a930 tx=0x7fa898000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 msgr2=0x7fa8b0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 0x7fa8b0197b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000d770 tx=0x7fa8a000dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 shutdown_connections 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa89c0a2b90 0x7fa89c0a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b0102270 0x7fa8b019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b01018b0 0x7fa8b0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 --2- 172.21.15.33:0/1265539862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b01006c0 0x7fa8b0197b40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 >> 172.21.15.33:0/1265539862 conn(0x7fa8b00fbc30 msgr2=0x7fa8b0069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 shutdown_connections 2024-04-24T23:12:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.588+0000 7fa8b6d61640 1 -- 172.21.15.33:0/1265539862 wait complete. 2024-04-24T23:12:27.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.706+0000 7fe4585a5640 1 -- 172.21.15.33:0/1669389797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 msgr2=0x7fe450100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.706+0000 7fe4585a5640 1 --2- 172.21.15.33:0/1669389797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450100aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- 172.21.15.33:0/1669389797 shutdown_connections 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 --2- 172.21.15.33:0/1669389797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe450102230 0x7fe450108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 --2- 172.21.15.33:0/1669389797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe450101870 0x7fe450101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 --2- 172.21.15.33:0/1669389797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450100aa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- 172.21.15.33:0/1669389797 >> 172.21.15.33:0/1669389797 conn(0x7fe4500fbbf0 msgr2=0x7fe4500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- 172.21.15.33:0/1669389797 shutdown_connections 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- 172.21.15.33:0/1669389797 wait complete. 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 Processor -- start 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- start start 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe450101870 0x7fe45019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe450102230 0x7fe4501a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d480 con 0x7fe450100680 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d300 con 0x7fe450101870 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe4585a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d600 con 0x7fe450102230 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.707+0000 7fe456341640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32862/0 (socket says 172.21.15.33:32862) 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 -- 172.21.15.33:0/256898258 learned_addr learned my addr 172.21.15.33:0/256898258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456b42640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe450102230 0x7fe4501a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe455b40640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe450101870 0x7fe45019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 -- 172.21.15.33:0/256898258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe450102230 msgr2=0x7fe4501a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe450102230 0x7fe4501a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 -- 172.21.15.33:0/256898258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe450101870 msgr2=0x7fe45019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe450101870 0x7fe45019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 -- 172.21.15.33:0/256898258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4501a18f0 con 0x7fe450100680 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe456341640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450199cb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe44c031440 tx=0x7fe44c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c03f030 con 0x7fe450100680 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44c003ce0 con 0x7fe450100680 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501a1b80 con 0x7fe450100680 2024-04-24T23:12:27.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c035070 con 0x7fe450100680 2024-04-24T23:12:27.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.708+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501a2070 con 0x7fe450100680 2024-04-24T23:12:27.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.709+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4501a2240 con 0x7fe450100680 2024-04-24T23:12:27.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.710+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe44c042030 con 0x7fe450100680 2024-04-24T23:12:27.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.711+0000 7fe4477fe640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe43c0a2b90 0x7fe43c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.711+0000 7fe455b40640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe43c0a2b90 0x7fe43c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.711+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe44c0ef920 con 0x7fe450100680 2024-04-24T23:12:27.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.712+0000 7fe455b40640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe43c0a2b90 0x7fe43c0a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe45019b1b0 tx=0x7fe44000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.716+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe44c0bbe40 con 0x7fe450100680 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.904+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4501a25d0 con 0x7fe450100680 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.905+0000 7fe4477fe640 1 -- 172.21.15.33:0/256898258 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7fe44c0c0140 con 0x7fe450100680 2024-04-24T23:12:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:27.908 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:27.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe43c0a2b90 msgr2=0x7fe43c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe43c0a2b90 0x7fe43c0a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe45019b1b0 tx=0x7fe44000b410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 msgr2=0x7fe450199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450199cb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe44c031440 tx=0x7fe44c032ce0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 shutdown_connections 2024-04-24T23:12:27.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe43c0a2b90 0x7fe43c0a5050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe450102230 0x7fe4501a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe450101870 0x7fe45019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 --2- 172.21.15.33:0/256898258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe450100680 0x7fe450199cb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.907+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 >> 172.21.15.33:0/256898258 conn(0x7fe4500fbbf0 msgr2=0x7fe450109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.908+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 shutdown_connections 2024-04-24T23:12:27.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:27.908+0000 7fe4585a5640 1 -- 172.21.15.33:0/256898258 wait complete. 2024-04-24T23:12:27.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:27.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 -- 172.21.15.33:0/3426595532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014102250 msgr2=0x7f3014108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 --2- 172.21.15.33:0/3426595532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014102250 0x7f3014108ae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f300c009ca0 tx=0x7f300c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 -- 172.21.15.33:0/3426595532 shutdown_connections 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 --2- 172.21.15.33:0/3426595532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014102250 0x7f3014108ae0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 --2- 172.21.15.33:0/3426595532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014101890 0x7f3014101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 --2- 172.21.15.33:0/3426595532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30141006a0 0x7f3014100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 -- 172.21.15.33:0/3426595532 >> 172.21.15.33:0/3426595532 conn(0x7f30140fbc10 msgr2=0x7f30140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 -- 172.21.15.33:0/3426595532 shutdown_connections 2024-04-24T23:12:28.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.026+0000 7f301c568640 1 -- 172.21.15.33:0/3426595532 wait complete. 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 Processor -- start 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 -- start start 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30141006a0 0x7f301406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 0x7f301419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014102250 0x7f301419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f301410c320 con 0x7f3014101890 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f301410c1a0 con 0x7f3014102250 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301c568640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f301410c4a0 con 0x7f30141006a0 2024-04-24T23:12:28.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 0x7f301419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 0x7f301419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32874/0 (socket says 172.21.15.33:32874) 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 -- 172.21.15.33:0/881427751 learned_addr learned my addr 172.21.15.33:0/881427751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301ab05640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014102250 0x7f301419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301a304640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30141006a0 0x7f301406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 -- 172.21.15.33:0/881427751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30141006a0 msgr2=0x7f301406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30141006a0 0x7f301406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 -- 172.21.15.33:0/881427751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014102250 msgr2=0x7f301419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014102250 0x7f301419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 -- 172.21.15.33:0/881427751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301419f6e0 con 0x7f3014101890 2024-04-24T23:12:28.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f301ab05640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014102250 0x7f301419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:28.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f3019b03640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 0x7f301419c7b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f300400d950 tx=0x7f300400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.027+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004017030 con 0x7f3014101890 2024-04-24T23:12:28.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.028+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3004004020 con 0x7f3014101890 2024-04-24T23:12:28.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.028+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301419f970 con 0x7f3014101890 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.028+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040047a0 con 0x7f3014101890 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.028+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301419fec0 con 0x7f3014101890 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.028+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30141079c0 con 0x7f3014101890 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.030+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f30040041d0 con 0x7f3014101890 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.030+0000 7f300b7fe640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30000a2b90 0x7f30000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.030+0000 7f301a304640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30000a2b90 0x7f30000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.030+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30040cee70 con 0x7f3014101890 2024-04-24T23:12:28.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.031+0000 7f301a304640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30000a2b90 0x7f30000a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f3010005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.035+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f300409ba40 con 0x7f3014101890 2024-04-24T23:12:28.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4026367354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:28.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/623065700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:28.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1265539862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:28.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4026367354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:28.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/623065700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:28.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1265539862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:28.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4026367354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:28.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/623065700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:28.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1265539862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.217+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3014062370 con 0x7f3014101890 2024-04-24T23:12:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.218+0000 7f300b7fe640 1 -- 172.21.15.33:0/881427751 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f300409b340 con 0x7f3014101890 2024-04-24T23:12:28.219 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30000a2b90 msgr2=0x7f30000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30000a2b90 0x7f30000a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f3010005850 comp rx=0 tx=0).stop 2024-04-24T23:12:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 msgr2=0x7f301419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 0x7f301419c7b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f300400d950 tx=0x7f300400de10 comp rx=0 tx=0).stop 2024-04-24T23:12:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 shutdown_connections 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30000a2b90 0x7f30000a5050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3014102250 0x7f301419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3014101890 0x7f301419c7b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 --2- 172.21.15.33:0/881427751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30141006a0 0x7f301406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 >> 172.21.15.33:0/881427751 conn(0x7f30140fbc10 msgr2=0x7f3014104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 shutdown_connections 2024-04-24T23:12:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.220+0000 7f301c568640 1 -- 172.21.15.33:0/881427751 wait complete. 2024-04-24T23:12:28.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 -- 172.21.15.33:0/2356510418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2968101890 msgr2=0x7f2968101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 --2- 172.21.15.33:0/2356510418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2968101890 0x7f2968101d10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2958009900 tx=0x7f295802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 -- 172.21.15.33:0/2356510418 shutdown_connections 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 --2- 172.21.15.33:0/2356510418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968102250 0x7f2968108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 --2- 172.21.15.33:0/2356510418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2968101890 0x7f2968101d10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 --2- 172.21.15.33:0/2356510418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29681006a0 0x7f2968100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 -- 172.21.15.33:0/2356510418 >> 172.21.15.33:0/2356510418 conn(0x7f29680fbc30 msgr2=0x7f29680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 -- 172.21.15.33:0/2356510418 shutdown_connections 2024-04-24T23:12:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 -- 172.21.15.33:0/2356510418 wait complete. 2024-04-24T23:12:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 Processor -- start 2024-04-24T23:12:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.340+0000 7f29707e3640 1 -- start start 2024-04-24T23:12:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f29707e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 0x7f2968197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f29707e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2968101890 0x7f2968198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f29707e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968102250 0x7f296819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f29707e3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f296810c350 con 0x7f29681006a0 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f29707e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f296810c1d0 con 0x7f2968101890 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f29707e3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f296810c4d0 con 0x7f2968102250 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 0x7f2968197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296dd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2968101890 0x7f2968198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 0x7f2968197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32884/0 (socket says 172.21.15.33:32884) 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296dd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2968101890 0x7f2968198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34348/0 (socket says 172.21.15.33:34348) 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 -- 172.21.15.33:0/2967879239 learned_addr learned my addr 172.21.15.33:0/2967879239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296ed80640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968102250 0x7f296819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 -- 172.21.15.33:0/2967879239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968102250 msgr2=0x7f296819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968102250 0x7f296819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 -- 172.21.15.33:0/2967879239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2968101890 msgr2=0x7f2968198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2968101890 0x7f2968198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 -- 172.21.15.33:0/2967879239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2968102ab0 con 0x7f29681006a0 2024-04-24T23:12:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.341+0000 7f296e57f640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 0x7f2968197ae0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f296400c970 tx=0x7f296400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.342+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964007c10 con 0x7f29681006a0 2024-04-24T23:12:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.342+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2964007db0 con 0x7f29681006a0 2024-04-24T23:12:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.342+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400f940 con 0x7f29681006a0 2024-04-24T23:12:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.342+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2968102da0 con 0x7f29681006a0 2024-04-24T23:12:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.342+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2968103310 con 0x7f29681006a0 2024-04-24T23:12:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.343+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29681035d0 con 0x7f29681006a0 2024-04-24T23:12:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.344+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f296400fae0 con 0x7f29681006a0 2024-04-24T23:12:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.345+0000 7f295f7fe640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29540a2c10 0x7f29540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.345+0000 7f296dd7e640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29540a2c10 0x7f29540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.345+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f29640cf670 con 0x7f29681006a0 2024-04-24T23:12:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.345+0000 7f296dd7e640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29540a2c10 0x7f29540a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2958005ab0 tx=0x7f29580315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.350+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f296409c2d0 con 0x7f29681006a0 2024-04-24T23:12:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.534+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2968103f00 con 0x7f29681006a0 2024-04-24T23:12:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.535+0000 7f295f7fe640 1 -- 172.21.15.33:0/2967879239 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f296409bbd0 con 0x7f29681006a0 2024-04-24T23:12:28.537 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29540a2c10 msgr2=0x7f29540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29540a2c10 0x7f29540a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2958005ab0 tx=0x7f29580315b0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 msgr2=0x7f2968197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 0x7f2968197ae0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f296400c970 tx=0x7f296400ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 shutdown_connections 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29540a2c10 0x7f29540a50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2968102250 0x7f296819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2968101890 0x7f2968198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 --2- 172.21.15.33:0/2967879239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29681006a0 0x7f2968197ae0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 >> 172.21.15.33:0/2967879239 conn(0x7f29680fbc30 msgr2=0x7f29680fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 shutdown_connections 2024-04-24T23:12:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.537+0000 7f29707e3640 1 -- 172.21.15.33:0/2967879239 wait complete. 2024-04-24T23:12:28.550 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 -- 172.21.15.33:0/2784670490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1801017a0 msgr2=0x7fe180101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 --2- 172.21.15.33:0/2784670490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1801017a0 0x7fe180101bc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009900 tx=0x7fe17c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 -- 172.21.15.33:0/2784670490 shutdown_connections 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 --2- 172.21.15.33:0/2784670490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe180103350 0x7fe180109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 --2- 172.21.15.33:0/2784670490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe180102990 0x7fe180102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 --2- 172.21.15.33:0/2784670490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1801017a0 0x7fe180101bc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 -- 172.21.15.33:0/2784670490 >> 172.21.15.33:0/2784670490 conn(0x7fe1800fcd10 msgr2=0x7fe1800ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 -- 172.21.15.33:0/2784670490 shutdown_connections 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 -- 172.21.15.33:0/2784670490 wait complete. 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.653+0000 7fe187415640 1 Processor -- start 2024-04-24T23:12:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 -- start start 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1801017a0 0x7fe180197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 0x7fe180198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe180103350 0x7fe18019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe18010d410 con 0x7fe180102990 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe18010d290 con 0x7fe1801017a0 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe18010d590 con 0x7fe180103350 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 0x7fe180198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 0x7fe180198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32902/0 (socket says 172.21.15.33:32902) 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 -- 172.21.15.33:0/2726915808 learned_addr learned my addr 172.21.15.33:0/2726915808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1859b2640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe180103350 0x7fe18019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1851b1640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1801017a0 0x7fe180197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 -- 172.21.15.33:0/2726915808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe180103350 msgr2=0x7fe18019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe180103350 0x7fe18019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 -- 172.21.15.33:0/2726915808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1801017a0 msgr2=0x7fe180197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1801017a0 0x7fe180197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 -- 172.21.15.33:0/2726915808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18019f700 con 0x7fe180102990 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1849b0640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 0x7fe180198020 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e9f0 tx=0x7fe17000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000cd80 con 0x7fe180102990 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe170005500 con 0x7fe180102990 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170010690 con 0x7fe180102990 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.654+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18019f9f0 con 0x7fe180102990 2024-04-24T23:12:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.655+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18019fef0 con 0x7fe180102990 2024-04-24T23:12:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.655+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe180108b60 con 0x7fe180102990 2024-04-24T23:12:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.657+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe1700046c0 con 0x7fe180102990 2024-04-24T23:12:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.657+0000 7fe1767fc640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe16c0a2b90 0x7fe16c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.657+0000 7fe1851b1640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe16c0a2b90 0x7fe16c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.657+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1700cf860 con 0x7fe180102990 2024-04-24T23:12:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.658+0000 7fe1851b1640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe16c0a2b90 0x7fe16c0a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006010 tx=0x7fe17c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.662+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe17009c430 con 0x7fe180102990 2024-04-24T23:12:28.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.845+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe180062340 con 0x7fe180102990 2024-04-24T23:12:28.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.846+0000 7fe1767fc640 1 -- 172.21.15.33:0/2726915808 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fe17009bd30 con 0x7fe180102990 2024-04-24T23:12:28.848 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:28.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe16c0a2b90 msgr2=0x7fe16c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe16c0a2b90 0x7fe16c0a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006010 tx=0x7fe17c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 msgr2=0x7fe180198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 0x7fe180198020 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e9f0 tx=0x7fe17000edf0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 shutdown_connections 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe16c0a2b90 0x7fe16c0a5050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe180103350 0x7fe18019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180102990 0x7fe180198020 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 --2- 172.21.15.33:0/2726915808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1801017a0 0x7fe180197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 >> 172.21.15.33:0/2726915808 conn(0x7fe1800fcd10 msgr2=0x7fe180105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 shutdown_connections 2024-04-24T23:12:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.848+0000 7fe187415640 1 -- 172.21.15.33:0/2726915808 wait complete. 2024-04-24T23:12:28.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 -- 172.21.15.33:0/2656101245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c101870 msgr2=0x7ff25c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 --2- 172.21.15.33:0/2656101245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c101870 0x7ff25c101cf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff244009900 tx=0x7ff24402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 -- 172.21.15.33:0/2656101245 shutdown_connections 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 --2- 172.21.15.33:0/2656101245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25c102230 0x7ff25c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 --2- 172.21.15.33:0/2656101245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c101870 0x7ff25c101cf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 --2- 172.21.15.33:0/2656101245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25c100680 0x7ff25c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 -- 172.21.15.33:0/2656101245 >> 172.21.15.33:0/2656101245 conn(0x7ff25c0fbbf0 msgr2=0x7ff25c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 -- 172.21.15.33:0/2656101245 shutdown_connections 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.966+0000 7ff262b34640 1 -- 172.21.15.33:0/2656101245 wait complete. 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 Processor -- start 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 -- start start 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25c100680 0x7ff25c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25c101870 0x7ff25c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 0x7ff25c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10c2e0 con 0x7ff25c102230 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10c160 con 0x7ff25c100680 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff262b34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10c460 con 0x7ff25c101870 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff2610d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 0x7ff25c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff2610d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 0x7ff25c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32920/0 (socket says 172.21.15.33:32920) 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff2610d1640 1 -- 172.21.15.33:0/3172309993 learned_addr learned my addr 172.21.15.33:0/3172309993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff2608d0640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25c100680 0x7ff25c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.967+0000 7ff25bfff640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25c101870 0x7ff25c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff2610d1640 1 -- 172.21.15.33:0/3172309993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25c101870 msgr2=0x7ff25c196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff2610d1640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25c101870 0x7ff25c196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff2610d1640 1 -- 172.21.15.33:0/3172309993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25c100680 msgr2=0x7ff25c1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff2610d1640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25c100680 0x7ff25c1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff2610d1640 1 -- 172.21.15.33:0/3172309993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c19e4e0 con 0x7ff25c102230 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff2610d1640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 0x7ff25c19de70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff25400cce0 tx=0x7ff25400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25400f450 con 0x7ff25c102230 2024-04-24T23:12:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff25400fad0 con 0x7ff25c102230 2024-04-24T23:12:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254018c10 con 0x7ff25c102230 2024-04-24T23:12:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c19e7d0 con 0x7ff25c102230 2024-04-24T23:12:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.968+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c19eca0 con 0x7ff25c102230 2024-04-24T23:12:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.969+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c1079a0 con 0x7ff25c102230 2024-04-24T23:12:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.970+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff25400f5f0 con 0x7ff25c102230 2024-04-24T23:12:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.971+0000 7ff259ffb640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2480a2b90 0x7ff2480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.971+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2540cfb60 con 0x7ff25c102230 2024-04-24T23:12:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.971+0000 7ff2608d0640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2480a2b90 0x7ff2480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.971+0000 7ff2608d0640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2480a2b90 0x7ff2480a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0058c0 tx=0x7ff24c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:28.975+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff254014030 con 0x7ff25c102230 2024-04-24T23:12:29.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.160+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff25c02d170 con 0x7ff25c102230 2024-04-24T23:12:29.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.160+0000 7ff259ffb640 1 -- 172.21.15.33:0/3172309993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7ff25409c730 con 0x7ff25c102230 2024-04-24T23:12:29.162 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2480a2b90 msgr2=0x7ff2480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2480a2b90 0x7ff2480a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0058c0 tx=0x7ff24c005850 comp rx=0 tx=0).stop 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 msgr2=0x7ff25c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 0x7ff25c19de70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff25400cce0 tx=0x7ff25400a620 comp rx=0 tx=0).stop 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 shutdown_connections 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff2480a2b90 0x7ff2480a5050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff25c102230 0x7ff25c19de70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25c101870 0x7ff25c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 --2- 172.21.15.33:0/3172309993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25c100680 0x7ff25c1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 >> 172.21.15.33:0/3172309993 conn(0x7ff25c0fbbf0 msgr2=0x7ff25c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 shutdown_connections 2024-04-24T23:12:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.162+0000 7ff262b34640 1 -- 172.21.15.33:0/3172309993 wait complete. 2024-04-24T23:12:29.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:28 smithi038 ceph-mon[32450]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:12:29.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/256898258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:29.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/881427751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2967879239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2726915808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:28 smithi033 ceph-mon[29201]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:12:29.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/256898258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:29.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/881427751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2967879239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2726915808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:28 smithi117 ceph-mon[32529]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:12:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/256898258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/881427751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2967879239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2726915808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 -- 172.21.15.33:0/1222732270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a8103360 msgr2=0x7f64a8109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 --2- 172.21.15.33:0/1222732270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a8103360 0x7f64a8109bf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6498009ca0 tx=0x7f649802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 -- 172.21.15.33:0/1222732270 shutdown_connections 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 --2- 172.21.15.33:0/1222732270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a8103360 0x7f64a8109bf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 --2- 172.21.15.33:0/1222732270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a81029a0 0x7f64a8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 --2- 172.21.15.33:0/1222732270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81017b0 0x7f64a8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 -- 172.21.15.33:0/1222732270 >> 172.21.15.33:0/1222732270 conn(0x7f64a80fcd40 msgr2=0x7f64a80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 -- 172.21.15.33:0/1222732270 shutdown_connections 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 -- 172.21.15.33:0/1222732270 wait complete. 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 Processor -- start 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 -- start start 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.276+0000 7f64ad42c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 0x7f64a81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81029a0 0x7f64a8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a8103360 0x7f64a819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64a810d430 con 0x7f64a81017b0 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64a810d2b0 con 0x7f64a81029a0 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64a810d5b0 con 0x7f64a8103360 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 0x7f64a81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 0x7f64a81979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32944/0 (socket says 172.21.15.33:32944) 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 -- 172.21.15.33:0/3312229622 learned_addr learned my addr 172.21.15.33:0/3312229622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a67fc640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81029a0 0x7f64a8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a77fe640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a8103360 0x7f64a819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 -- 172.21.15.33:0/3312229622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a8103360 msgr2=0x7f64a819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a8103360 0x7f64a819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 -- 172.21.15.33:0/3312229622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81029a0 msgr2=0x7f64a8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81029a0 0x7f64a8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 -- 172.21.15.33:0/3312229622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a819f5d0 con 0x7f64a81017b0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a67fc640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81029a0 0x7f64a8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64a6ffd640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 0x7f64a81979b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c970 tx=0x7f64a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0007c10 con 0x7f64a81017b0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a0007db0 con 0x7f64a81017b0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a819f8c0 con 0x7f64a81017b0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a000f940 con 0x7f64a81017b0 2024-04-24T23:12:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.277+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a819fe10 con 0x7f64a81017b0 2024-04-24T23:12:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.278+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a8108a00 con 0x7f64a81017b0 2024-04-24T23:12:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.279+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64a000fae0 con 0x7f64a81017b0 2024-04-24T23:12:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.280+0000 7f649ffff640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64940a2b90 0x7f64940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.280+0000 7f64a67fc640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64940a2b90 0x7f64940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.280+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64a00cf880 con 0x7f64a81017b0 2024-04-24T23:12:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.281+0000 7f64a67fc640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64940a2b90 0x7f64940a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64900058c0 tx=0x7f6490005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.285+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f64a009c4e0 con 0x7f64a81017b0 2024-04-24T23:12:29.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.521+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64a802d170 con 0x7f64a81017b0 2024-04-24T23:12:29.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.522+0000 7f649ffff640 1 -- 172.21.15.33:0/3312229622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f64a009bde0 con 0x7f64a81017b0 2024-04-24T23:12:29.524 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:29.524 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:10:38.464642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:12:29.524 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.523+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64940a2b90 msgr2=0x7f64940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.523+0000 7f64ad42c640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64940a2b90 0x7f64940a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64900058c0 tx=0x7f6490005850 comp rx=0 tx=0).stop 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.523+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 msgr2=0x7f64a81979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.523+0000 7f64ad42c640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 0x7f64a81979b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c970 tx=0x7f64a000ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.523+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 shutdown_connections 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.523+0000 7f64ad42c640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64940a2b90 0x7f64940a5050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.524+0000 7f64ad42c640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a8103360 0x7f64a819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.524+0000 7f64ad42c640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64a81029a0 0x7f64a8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.524+0000 7f64ad42c640 1 --2- 172.21.15.33:0/3312229622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a81017b0 0x7f64a81979b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.524+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 >> 172.21.15.33:0/3312229622 conn(0x7f64a80fcd40 msgr2=0x7f64a8105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.524+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 shutdown_connections 2024-04-24T23:12:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.524+0000 7f64ad42c640 1 -- 172.21.15.33:0/3312229622 wait complete. 2024-04-24T23:12:29.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 -- 172.21.15.33:0/1037625654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c102250 msgr2=0x7fc98c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 --2- 172.21.15.33:0/1037625654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c102250 0x7fc98c108ae0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc984009ca0 tx=0x7fc98402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 -- 172.21.15.33:0/1037625654 shutdown_connections 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 --2- 172.21.15.33:0/1037625654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c102250 0x7fc98c108ae0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 --2- 172.21.15.33:0/1037625654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 0x7fc98c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 --2- 172.21.15.33:0/1037625654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c1006a0 0x7fc98c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 -- 172.21.15.33:0/1037625654 >> 172.21.15.33:0/1037625654 conn(0x7fc98c0fbc30 msgr2=0x7fc98c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 -- 172.21.15.33:0/1037625654 shutdown_connections 2024-04-24T23:12:29.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.637+0000 7fc99411d640 1 -- 172.21.15.33:0/1037625654 wait complete. 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 Processor -- start 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 -- start start 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 0x7fc98c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 0x7fc98c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c102250 0x7fc98c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c10d440 con 0x7fc98c1006a0 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c10d2c0 con 0x7fc98c102250 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc99411d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c10d5c0 con 0x7fc98c101890 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 0x7fc98c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 0x7fc98c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32960/0 (socket says 172.21.15.33:32960) 2024-04-24T23:12:29.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 -- 172.21.15.33:0/2569011097 learned_addr learned my addr 172.21.15.33:0/2569011097 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc9926ba640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c102250 0x7fc98c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc9916b8640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 0x7fc98c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 -- 172.21.15.33:0/2569011097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 msgr2=0x7fc98c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 0x7fc98c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 -- 172.21.15.33:0/2569011097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c102250 msgr2=0x7fc98c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c102250 0x7fc98c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 -- 172.21.15.33:0/2569011097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c19f650 con 0x7fc98c1006a0 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc9926ba640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c102250 0x7fc98c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc9916b8640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 0x7fc98c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.638+0000 7fc991eb9640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 0x7fc98c197a30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc98800d770 tx=0x7fc98800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.639+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988017070 con 0x7fc98c1006a0 2024-04-24T23:12:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.639+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98c19f940 con 0x7fc98c1006a0 2024-04-24T23:12:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.639+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc988009c30 con 0x7fc98c1006a0 2024-04-24T23:12:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.639+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988003dd0 con 0x7fc98c1006a0 2024-04-24T23:12:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.639+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98c19fee0 con 0x7fc98c1006a0 2024-04-24T23:12:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.640+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98c0689b0 con 0x7fc98c1006a0 2024-04-24T23:12:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.641+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc988004000 con 0x7fc98c1006a0 2024-04-24T23:12:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.641+0000 7fc982ffd640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9780a2b90 0x7fc9780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.642+0000 7fc9916b8640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9780a2b90 0x7fc9780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.642+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc9880cf210 con 0x7fc98c1006a0 2024-04-24T23:12:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.642+0000 7fc9916b8640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9780a2b90 0x7fc9780a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc98c198f40 tx=0x7fc97c005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.645+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc988009de0 con 0x7fc98c1006a0 2024-04-24T23:12:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.831+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc98c062340 con 0x7fc98c1006a0 2024-04-24T23:12:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.833+0000 7fc982ffd640 1 -- 172.21.15.33:0/2569011097 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7fc98809bde0 con 0x7fc98c1006a0 2024-04-24T23:12:29.835 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:29.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9780a2b90 msgr2=0x7fc9780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9780a2b90 0x7fc9780a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc98c198f40 tx=0x7fc97c005e50 comp rx=0 tx=0).stop 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 msgr2=0x7fc98c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 0x7fc98c197a30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc98800d770 tx=0x7fc98800dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 shutdown_connections 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9780a2b90 0x7fc9780a5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc98c102250 0x7fc98c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc98c101890 0x7fc98c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 --2- 172.21.15.33:0/2569011097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc98c1006a0 0x7fc98c197a30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 >> 172.21.15.33:0/2569011097 conn(0x7fc98c0fbc30 msgr2=0x7fc98c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 shutdown_connections 2024-04-24T23:12:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.835+0000 7fc99411d640 1 -- 172.21.15.33:0/2569011097 wait complete. 2024-04-24T23:12:29.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:10:11.664486+0000', 'modified': '2024-04-24T23:10:38.464642+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24508, 'mds_1': 24481, 'mds_2': 24479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/287276222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 287276222}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 287276222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'g', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/2446242312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 2446242312}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 2446242312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/221661334', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 221661334}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 221661334}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.38:6839/920905379', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 920905379}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 920905379}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.117:6837/878889242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 878889242}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 878889242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/851112851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 851112851}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 851112851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [24479, 24481, 24508]} 2024-04-24T23:12:29.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:12:29.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.950+0000 7f812c590640 1 -- 172.21.15.33:0/1885414315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 msgr2=0x7f8124101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.950+0000 7f812c590640 1 --2- 172.21.15.33:0/1885414315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124101cf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8114009900 tx=0x7f811402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- 172.21.15.33:0/1885414315 shutdown_connections 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 --2- 172.21.15.33:0/1885414315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 0x7f8124108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 --2- 172.21.15.33:0/1885414315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124101cf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 --2- 172.21.15.33:0/1885414315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 0x7f8124100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- 172.21.15.33:0/1885414315 >> 172.21.15.33:0/1885414315 conn(0x7f81240fbbf0 msgr2=0x7f81240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- 172.21.15.33:0/1885414315 shutdown_connections 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- 172.21.15.33:0/1885414315 wait complete. 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 Processor -- start 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- start start 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 0x7f8124196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 0x7f812419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f812410c300 con 0x7f8124101870 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f812410c180 con 0x7f8124100680 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f812c590640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f812410c480 con 0x7f8124102230 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f8129b2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f8129b2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32976/0 (socket says 172.21.15.33:32976) 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.951+0000 7f8129b2b640 1 -- 172.21.15.33:0/908537926 learned_addr learned my addr 172.21.15.33:0/908537926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f812ab2d640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 0x7f812419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f812a32c640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 0x7f8124196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f8129b2b640 1 -- 172.21.15.33:0/908537926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 msgr2=0x7f812419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f8129b2b640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 0x7f812419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f8129b2b640 1 -- 172.21.15.33:0/908537926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 msgr2=0x7f8124196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f8129b2b640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 0x7f8124196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f8129b2b640 1 -- 172.21.15.33:0/908537926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812419e560 con 0x7f8124101870 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f812ab2d640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 0x7f812419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f812a32c640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 0x7f8124196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f8129b2b640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124196e80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8114009230 tx=0x7f8114004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f811b7fe640 1 -- 172.21.15.33:0/908537926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811403f030 con 0x7f8124101870 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f811b7fe640 1 -- 172.21.15.33:0/908537926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114003ce0 con 0x7f8124101870 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812419e7f0 con 0x7f8124101870 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f811b7fe640 1 -- 172.21.15.33:0/908537926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114035070 con 0x7f8124101870 2024-04-24T23:12:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.952+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812419ece0 con 0x7f8124101870 2024-04-24T23:12:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 Processor -- start 2024-04-24T23:12:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f811b7fe640 1 -- 172.21.15.33:0/908537926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8114042030 con 0x7f8124101870 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 -- start start 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 0x560e01ad9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01adc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e01a37f00 con 0x560e01ad4e30 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e019e0750 con 0x560e01ad9fa0 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812f37d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e01a33ec0 con 0x560e01ad4440 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 0x560e01ad9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 0x560e01ad9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32990/0 (socket says 172.21.15.33:32990) 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 -- 172.21.15.33:0/1437997761 learned_addr learned my addr 172.21.15.33:0/1437997761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812ab2d640 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f8129b2b640 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01adc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 -- 172.21.15.33:0/1437997761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 msgr2=0x560e01ad4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f811b7fe640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81100a2b90 0x7f81100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 -- 172.21.15.33:0/1437997761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 msgr2=0x560e01adc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01adc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.955+0000 7f812a32c640 1 -- 172.21.15.33:0/1437997761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0174c2d0 con 0x560e01ad4e30 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.956+0000 7f812ab2d640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81100a2b90 0x7f81100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.956+0000 7f811b7fe640 1 -- 172.21.15.33:0/908537926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81140efe70 con 0x7f8124101870 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.956+0000 7f812a32c640 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 0x560e01ad9930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f812000bbf0 tx=0x7f812000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb236b13d83728e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.956+0000 7f8118bfe640 1 -- 172.21.15.33:0/1437997761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120004220 con 0x560e01ad4e30 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.956+0000 7f8118bfe640 1 -- 172.21.15.33:0/1437997761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81200043c0 con 0x560e01ad4e30 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.956+0000 7f8118bfe640 1 -- 172.21.15.33:0/1437997761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812000f860 con 0x560e01ad4e30 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- 172.21.15.33:0/1437997761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 msgr2=0x560e01ad9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812ab2d640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81100a2b90 0x7f81100a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x560e01ad5690 tx=0x7f811c004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 0x560e01ad9930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f812000bbf0 tx=0x7f812000d710 comp rx=0 tx=0).stop 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- 172.21.15.33:0/1437997761 shutdown_connections 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01adc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4e30 0x560e01ad9930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- 172.21.15.33:0/1437997761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- 172.21.15.33:0/1437997761 >> 172.21.15.33:0/1437997761 conn(0x560e01acfab0 msgr2=0x560e01acfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- 172.21.15.33:0/1437997761 shutdown_connections 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- 172.21.15.33:0/1437997761 wait complete. 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- start start 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 Processor -- start 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01ade890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4e30 0x560e01adedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01ae5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e01adcf50 con 0x560e01ad4440 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e019f3cf0 con 0x560e01ad9fa0 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e01add0d0 con 0x560e01ad4e30 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812f37d740 10 client.0 _ll_register_callbacks cb 0x560e019f36e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.957+0000 7f812a32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01ade890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01ade890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32992/0 (socket says 172.21.15.33:32992) 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 -- 172.21.15.33:0/4161107936 learned_addr learned my addr 172.21.15.33:0/4161107936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f8129b2b640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4e30 0x560e01adedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 -- 172.21.15.33:0/4161107936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4e30 msgr2=0x560e01adedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812ab2d640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01ae5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4e30 0x560e01adedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 -- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 msgr2=0x560e01ae5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01ae5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 -- 172.21.15.33:0/4161107936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e018166b0 con 0x560e01ad4440 2024-04-24T23:12:29.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812ab2d640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01ae5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812a32c640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01ade890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8120017000 tx=0x7f8120005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396363a372b835ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f810dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f810dffb640 1 -- 172.21.15.33:0/4161107936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120024020 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f810dffb640 1 -- 172.21.15.33:0/4161107936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81200035f0 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81200043c0 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f810dffb640 1 -- 172.21.15.33:0/4161107936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f812002f050 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812f37d740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560e01aeaf90 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.958+0000 7f810dffb640 1 -- 172.21.15.33:0/4161107936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81200043c0 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f810dffb640 1 -- 172.21.15.33:0/4161107936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8120017430 con 0x560e01ad4440 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f812f37d740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f812f37d740 10 client.15396 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f812f37d740 20 client.15396 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f812f37d740 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01aed280 0x560e01aee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f812f37d740 4 client.15396 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e01a45440 con 0x560e01aed280 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f8129b2b640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01aed280 0x560e01aee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f810dffb640 10 client.15396 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.959+0000 7f8129b2b640 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01aed280 0x560e01aee6f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34fdf6b646a96de9 server_cookie=caf82ad9a9ad100e in_seq=0 out_seq=0 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f810dffb640 1 -- 172.21.15.33:0/4161107936 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e01a45440 con 0x560e01aed280 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f810dffb640 10 client.15396 handle_command_reply: tid=0 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 client.15396 shutdown 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 msgr2=0x560e01ade890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01ade890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8120017000 tx=0x7f8120005f20 comp rx=0 tx=0).stop 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 shutdown_connections 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01aed280 0x560e01aee6f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad9fa0 0x560e01ae5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f810dffb640 0 client.15396 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4e30 0x560e01adedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 --2- 172.21.15.33:0/4161107936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01ade890 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 >> 172.21.15.33:0/4161107936 conn(0x560e01acfab0 msgr2=0x560e01aceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 shutdown_connections 2024-04-24T23:12:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.960+0000 7f812f37d740 1 -- 172.21.15.33:0/4161107936 wait complete. 2024-04-24T23:12:29.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.961+0000 7f812f37d740 20 client.15396 trim_cache size 0 max 16384 2024-04-24T23:12:29.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.978+0000 7f812f37d740 1 Processor -- start 2024-04-24T23:12:29.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 -- start start 2024-04-24T23:12:29.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01acfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01aca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad9fa0 0x560e01ad2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e01aeca30 con 0x560e01ad4440 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e01aed100 con 0x560e01ad4e30 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812f37d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e01aed7d0 con 0x560e01ad9fa0 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01acfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01acfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32996/0 (socket says 172.21.15.33:32996) 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 -- 172.21.15.33:0/1341634239 learned_addr learned my addr 172.21.15.33:0/1341634239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f8129b2b640 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01aca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 -- 172.21.15.33:0/1341634239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad9fa0 msgr2=0x560e01ad2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad9fa0 0x560e01ad2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 -- 172.21.15.33:0/1341634239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 msgr2=0x560e01aca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01aca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 -- 172.21.15.33:0/1341634239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e01ad0780 con 0x560e01ad4440 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f812a32c640 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01acfe90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f812000b980 tx=0x7f812000b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ebbe0cd666f891 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f810ffff640 1 -- 172.21.15.33:0/1341634239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120024070 con 0x560e01ad4440 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f810ffff640 1 -- 172.21.15.33:0/1341634239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812000dc10 con 0x560e01ad4440 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.979+0000 7f810ffff640 1 -- 172.21.15.33:0/1341634239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120003d30 con 0x560e01ad4440 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 -- 172.21.15.33:0/1341634239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 msgr2=0x560e01acfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01acfe90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f812000b980 tx=0x7f812000b520 comp rx=0 tx=0).stop 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 -- 172.21.15.33:0/1341634239 shutdown_connections 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad9fa0 0x560e01ad2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01aca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- 172.21.15.33:0/1341634239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad4440 0x560e01acfe90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 -- 172.21.15.33:0/1341634239 >> 172.21.15.33:0/1341634239 conn(0x560e01acfab0 msgr2=0x560e01ad57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 -- 172.21.15.33:0/1341634239 shutdown_connections 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 -- 172.21.15.33:0/1341634239 wait complete. 2024-04-24T23:12:29.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 -- start start 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 Processor -- start 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01ad8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.980+0000 7f812f37d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 0x560e01ae42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e01aee570 con 0x560e01ad9fa0 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e01aedea0 con 0x560e01ad4e30 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e01aeec40 con 0x560e01ad4440 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 10 client.0 _ll_register_callbacks cb 0x7f81200045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 0x560e01ae42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 0x560e01ae42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32998/0 (socket says 172.21.15.33:32998) 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 -- 172.21.15.33:0/2444364588 learned_addr learned my addr 172.21.15.33:0/2444364588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812a32c640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f8129b2b640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01ad8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 -- 172.21.15.33:0/2444364588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 msgr2=0x560e01ad82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 -- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 msgr2=0x560e01ad8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01ad8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 -- 172.21.15.33:0/2444364588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e01ae48a0 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812ab2d640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 0x560e01ae42b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f811c00f950 tx=0x7f811c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce490bf3823e72f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f810dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f810dffb640 1 -- 172.21.15.33:0/2444364588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c01d940 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f810dffb640 1 -- 172.21.15.33:0/2444364588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f811c01dae0 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f812000dc10 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f810dffb640 1 -- 172.21.15.33:0/2444364588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f811c01dce0 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.981+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560e01b53600 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f810dffb640 1 -- 172.21.15.33:0/2444364588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812000dc10 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f810dffb640 1 -- 172.21.15.33:0/2444364588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f811c01b420 con 0x560e01ad9fa0 2024-04-24T23:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812f37d740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812f37d740 10 client.15402 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812f37d740 20 client.15402 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812f37d740 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01b582f0 0x560e01b57000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812f37d740 4 client.15402 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "status"}) -- 0x560e01a3f790 con 0x560e01b582f0 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.982+0000 7f812a32c640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01b582f0 0x560e01b57000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812a32c640 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01b582f0 0x560e01b57000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78d2034ce4671f6f server_cookie=b794b59fa9f029c0 in_seq=0 out_seq=0 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f810dffb640 10 client.15402 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f810dffb640 1 -- 172.21.15.33:0/2444364588 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560e01a3f790 con 0x560e01b582f0 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f810dffb640 10 client.15402 handle_command_reply: tid=0 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 client.15402 shutdown 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 msgr2=0x560e01ae42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 0x560e01ae42b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f811c00f950 tx=0x7f811c009700 comp rx=0 tx=0).stop 2024-04-24T23:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 shutdown_connections 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560e01b582f0 0x560e01b57000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e01ad9fa0 0x560e01ae42b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e01ad4e30 0x560e01ad8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 --2- 172.21.15.33:0/2444364588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e01ad4440 0x560e01ad82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f810dffb640 0 client.15402 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 >> 172.21.15.33:0/2444364588 conn(0x560e01acfab0 msgr2=0x560e01aca690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 shutdown_connections 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 1 -- 172.21.15.33:0/2444364588 wait complete. 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.983+0000 7f812f37d740 20 client.15402 trim_cache size 0 max 16384 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:29.991 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24479, 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 112.533849312, 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 94, 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 143.95578205499999 2024-04-24T23:12:29.992 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81100a2b90 msgr2=0x7f81100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81100a2b90 0x7f81100a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x560e01ad5690 tx=0x7f811c004a40 comp rx=0 tx=0).stop 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 msgr2=0x7f8124196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124196e80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8114009230 tx=0x7f8114004360 comp rx=0 tx=0).stop 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 shutdown_connections 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81100a2b90 0x7f81100a5050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8124102230 0x7f812419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8124101870 0x7f8124196e80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 --2- 172.21.15.33:0/908537926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8124100680 0x7f8124196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 >> 172.21.15.33:0/908537926 conn(0x7f81240fbbf0 msgr2=0x7f8124104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 shutdown_connections 2024-04-24T23:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:29.984+0000 7f812c590640 1 -- 172.21.15.33:0/908537926 wait complete. 2024-04-24T23:12:29.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 -- 172.21.15.33:0/1726414073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 msgr2=0x7fba68109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 --2- 172.21.15.33:0/1726414073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68109be0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fba58009900 tx=0x7fba5802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 -- 172.21.15.33:0/1726414073 shutdown_connections 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 --2- 172.21.15.33:0/1726414073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68109be0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 --2- 172.21.15.33:0/1726414073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba68102990 0x7fba68102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 --2- 172.21.15.33:0/1726414073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba681017a0 0x7fba68101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 -- 172.21.15.33:0/1726414073 >> 172.21.15.33:0/1726414073 conn(0x7fba680fcd10 msgr2=0x7fba680ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 -- 172.21.15.33:0/1726414073 shutdown_connections 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.098+0000 7fba6f664640 1 -- 172.21.15.33:0/1726414073 wait complete. 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 Processor -- start 2024-04-24T23:12:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 -- start start 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba681017a0 0x7fba680754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba68102990 0x7fba68073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba6810d420 con 0x7fba68103350 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fba6810d2a0 con 0x7fba681017a0 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6f664640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba6810d5a0 con 0x7fba68102990 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33016/0 (socket says 172.21.15.33:33016) 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 -- 172.21.15.33:0/2824266123 learned_addr learned my addr 172.21.15.33:0/2824266123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6d400640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba681017a0 0x7fba680754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6cbff640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba68102990 0x7fba68073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 -- 172.21.15.33:0/2824266123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba68102990 msgr2=0x7fba68073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba68102990 0x7fba68073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 -- 172.21.15.33:0/2824266123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba681017a0 msgr2=0x7fba680754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba681017a0 0x7fba680754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 -- 172.21.15.33:0/2824266123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba68074900 con 0x7fba68103350 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.099+0000 7fba6dc01640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68074070 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fba58009230 tx=0x7fba580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.100+0000 7fba5e7fc640 1 -- 172.21.15.33:0/2824266123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5803f030 con 0x7fba68103350 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.100+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba681a1b80 con 0x7fba68103350 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.100+0000 7fba5e7fc640 1 -- 172.21.15.33:0/2824266123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba58003ce0 con 0x7fba68103350 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.100+0000 7fba5e7fc640 1 -- 172.21.15.33:0/2824266123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58035070 con 0x7fba68103350 2024-04-24T23:12:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.100+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681a2120 con 0x7fba68103350 2024-04-24T23:12:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba5e7fc640 1 -- 172.21.15.33:0/2824266123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba58042030 con 0x7fba68103350 2024-04-24T23:12:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 Processor -- start 2024-04-24T23:12:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 -- start start 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 0x5598780c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3e30 0x5598780c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559878026f00 con 0x5598780c3440 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559877fcf750 con 0x5598780c3e30 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba72372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559878022ec0 con 0x5598780c8fa0 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.102+0000 7fba5e7fc640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba540a2b90 0x7fba540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 0x5598780c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 0x5598780c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33028/0 (socket says 172.21.15.33:33028) 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 -- 172.21.15.33:0/604539516 learned_addr learned my addr 172.21.15.33:0/604539516 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6d400640 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3e30 0x5598780c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6dc01640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba540a2b90 0x7fba540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba5e7fc640 1 -- 172.21.15.33:0/2824266123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fba580ef8c0 con 0x7fba68103350 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 -- 172.21.15.33:0/604539516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 msgr2=0x5598780cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 -- 172.21.15.33:0/604539516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3e30 msgr2=0x5598780c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3e30 0x5598780c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 -- 172.21.15.33:0/604539516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559877d3b2d0 con 0x5598780c3440 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6cbff640 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 0x5598780c3860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba600123c0 tx=0x7fba600096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeafe52d81224e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba6dc01640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba540a2b90 0x7fba540a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba580ed3a0 tx=0x7fba580e6920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba537fe640 1 -- 172.21.15.33:0/604539516 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6001a030 con 0x5598780c3440 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba537fe640 1 -- 172.21.15.33:0/604539516 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60012070 con 0x5598780c3440 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.103+0000 7fba537fe640 1 -- 172.21.15.33:0/604539516 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60010400 con 0x5598780c3440 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 -- 172.21.15.33:0/604539516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 msgr2=0x5598780c3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 0x5598780c3860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba600123c0 tx=0x7fba600096c0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 -- 172.21.15.33:0/604539516 shutdown_connections 2024-04-24T23:12:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3e30 0x5598780c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 --2- 172.21.15.33:0/604539516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3440 0x5598780c3860 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 -- 172.21.15.33:0/604539516 >> 172.21.15.33:0/604539516 conn(0x5598780beab0 msgr2=0x5598780beeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 -- 172.21.15.33:0/604539516 shutdown_connections 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 -- 172.21.15.33:0/604539516 wait complete. 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 -- start start 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.104+0000 7fba72372740 1 Processor -- start 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5598780cbf50 con 0x5598780c3e30 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559877fe2cf0 con 0x5598780c3440 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5598780cc0d0 con 0x5598780c8fa0 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba72372740 10 client.0 _ll_register_callbacks cb 0x559877fe26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33042/0 (socket says 172.21.15.33:33042) 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 -- 172.21.15.33:0/831223389 learned_addr learned my addr 172.21.15.33:0/831223389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6cbff640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6cbff640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 -- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 msgr2=0x5598780d4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780d4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 -- 172.21.15.33:0/831223389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 msgr2=0x5598780cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 -- 172.21.15.33:0/831223389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559877e056b0 con 0x5598780c3e30 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6cbff640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba6d400640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780cddb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fba64011ce0 tx=0x7fba6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f531737330c82c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.105+0000 7fba50ff9640 1 -- 172.21.15.33:0/831223389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64003970 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba50ff9640 1 -- 172.21.15.33:0/831223389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fba64003b10 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba60012070 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba50ff9640 1 -- 172.21.15.33:0/831223389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fba64003d10 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5598780d7ed0 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba50ff9640 1 -- 172.21.15.33:0/831223389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60012070 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba50ff9640 1 -- 172.21.15.33:0/831223389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fba6401b680 con 0x5598780c3e30 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 10 client.15414 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 20 client.15414 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598780da600 0x5598780dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 4 client.15414 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559878034440 con 0x5598780da600 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.106+0000 7fba6cbff640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598780da600 0x5598780dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.107+0000 7fba6cbff640 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598780da600 0x5598780dba70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc3c7ae4fc62172e server_cookie=e79557325ccd8a22 in_seq=0 out_seq=0 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.107+0000 7fba50ff9640 10 client.15414 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba50ff9640 1 -- 172.21.15.33:0/831223389 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559878034440 con 0x5598780da600 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba50ff9640 10 client.15414 handle_command_reply: tid=0 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 client.15414 shutdown 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 msgr2=0x5598780cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780cddb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fba64011ce0 tx=0x7fba6400f790 comp rx=0 tx=0).stop 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 shutdown_connections 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598780da600 0x5598780dba70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c8fa0 0x5598780d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780cddb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba50ff9640 0 client.15414 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 --2- 172.21.15.33:0/831223389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 >> 172.21.15.33:0/831223389 conn(0x5598780beab0 msgr2=0x5598780bdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 shutdown_connections 2024-04-24T23:12:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 1 -- 172.21.15.33:0/831223389 wait complete. 2024-04-24T23:12:30.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.108+0000 7fba72372740 20 client.15414 trim_cache size 0 max 16384 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.126+0000 7fba72372740 1 Processor -- start 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.126+0000 7fba72372740 1 -- start start 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba72372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3440 0x5598780c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba72372740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba72372740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c8fa0 0x5598780d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba72372740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5598780d9db0 con 0x5598780c3e30 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba72372740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598780da480 con 0x5598780c8fa0 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba72372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5598780dab50 con 0x5598780c3440 2024-04-24T23:12:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33044/0 (socket says 172.21.15.33:33044) 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 -- 172.21.15.33:0/4081524526 learned_addr learned my addr 172.21.15.33:0/4081524526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 -- 172.21.15.33:0/4081524526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3440 msgr2=0x5598780c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3440 0x5598780c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 -- 172.21.15.33:0/4081524526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c8fa0 msgr2=0x5598780d4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c8fa0 0x5598780d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 -- 172.21.15.33:0/4081524526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598780c4210 con 0x5598780c3e30 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba6d400640 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780c5730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fba6401bb10 tx=0x7fba64005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a57c766e874abf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba52ffd640 1 -- 172.21.15.33:0/4081524526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6402d030 con 0x5598780c3e30 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba52ffd640 1 -- 172.21.15.33:0/4081524526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6400fe80 con 0x5598780c3e30 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.127+0000 7fba52ffd640 1 -- 172.21.15.33:0/4081524526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba640053a0 con 0x5598780c3e30 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 -- 172.21.15.33:0/4081524526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 msgr2=0x5598780c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780c5730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fba6401bb10 tx=0x7fba64005b50 comp rx=0 tx=0).stop 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 -- 172.21.15.33:0/4081524526 shutdown_connections 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c8fa0 0x5598780d4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c3e30 0x5598780c5730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 --2- 172.21.15.33:0/4081524526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3440 0x5598780c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 -- 172.21.15.33:0/4081524526 >> 172.21.15.33:0/4081524526 conn(0x5598780beab0 msgr2=0x5598780c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 -- 172.21.15.33:0/4081524526 shutdown_connections 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 -- 172.21.15.33:0/4081524526 wait complete. 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 -- start start 2024-04-24T23:12:30.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.128+0000 7fba72372740 1 Processor -- start 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3e30 0x5598780d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 0x5598780d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5598780db8f0 con 0x5598780c8fa0 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598780db220 con 0x5598780c3440 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5598780dbfc0 con 0x5598780c3e30 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 10 client.0 _ll_register_callbacks cb 0x7fba64011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 0x5598780d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 0x5598780d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33046/0 (socket says 172.21.15.33:33046) 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 -- 172.21.15.33:0/3044407646 learned_addr learned my addr 172.21.15.33:0/3044407646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6d400640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3e30 0x5598780d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 -- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3e30 msgr2=0x5598780d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3e30 0x5598780d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 -- 172.21.15.33:0/3044407646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 msgr2=0x5598780d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 -- 172.21.15.33:0/3044407646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598780d4830 con 0x5598780c8fa0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6d400640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3e30 0x5598780d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba6cbff640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 0x5598780d4270 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba60003890 tx=0x7fba60016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4c44e6c94e8081 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba50ff9640 1 -- 172.21.15.33:0/3044407646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba600120d0 con 0x5598780c8fa0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba50ff9640 1 -- 172.21.15.33:0/3044407646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fba6002eae0 con 0x5598780c8fa0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.129+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55987802e790 con 0x5598780c8fa0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba50ff9640 1 -- 172.21.15.33:0/3044407646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fba60037050 con 0x5598780c8fa0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559878140e20 con 0x5598780c8fa0 2024-04-24T23:12:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba50ff9640 1 -- 172.21.15.33:0/3044407646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55987802e790 con 0x5598780c8fa0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba50ff9640 1 -- 172.21.15.33:0/3044407646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fba6002ece0 con 0x5598780c8fa0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 10 client.15420 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 20 client.15420 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598781458b0 0x5598781445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 4 client.15420 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "status"}) -- 0x559877facb80 con 0x5598781458b0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.130+0000 7fba6d400640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598781458b0 0x5598781445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba50ff9640 10 client.15420 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba6d400640 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598781458b0 0x5598781445c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c09bc1ff6150f40 server_cookie=7bd909aedcb320de in_seq=0 out_seq=0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba50ff9640 1 -- 172.21.15.33:0/3044407646 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559877facb80 con 0x5598781458b0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba50ff9640 10 client.15420 handle_command_reply: tid=0 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 client.15420 shutdown 2024-04-24T23:12:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 msgr2=0x5598780d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 0x5598780d4270 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba60003890 tx=0x7fba60016040 comp rx=0 tx=0).stop 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 shutdown_connections 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5598781458b0 0x5598781445c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5598780c8fa0 0x5598780d4270 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598780c3e30 0x5598780d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba50ff9640 0 client.15420 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 --2- 172.21.15.33:0/3044407646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598780c3440 0x5598780d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 >> 172.21.15.33:0/3044407646 conn(0x5598780beab0 msgr2=0x5598780c51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 shutdown_connections 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 1 -- 172.21.15.33:0/3044407646 wait complete. 2024-04-24T23:12:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.131+0000 7fba72372740 20 client.15420 trim_cache size 0 max 16384 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24481, 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 113.69084748100001, 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 94, 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 158.713674217 2024-04-24T23:12:30.140 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba540a2b90 msgr2=0x7fba540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba540a2b90 0x7fba540a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba580ed3a0 tx=0x7fba580e6920 comp rx=0 tx=0).stop 2024-04-24T23:12:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 msgr2=0x7fba68074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68074070 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fba58009230 tx=0x7fba580042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 shutdown_connections 2024-04-24T23:12:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba540a2b90 0x7fba540a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba68103350 0x7fba68074070 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba68102990 0x7fba68073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 --2- 172.21.15.33:0/2824266123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba681017a0 0x7fba680754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 >> 172.21.15.33:0/2824266123 conn(0x7fba680fcd10 msgr2=0x7fba68105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 shutdown_connections 2024-04-24T23:12:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.132+0000 7fba6f664640 1 -- 172.21.15.33:0/2824266123 wait complete. 2024-04-24T23:12:30.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:29 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3172309993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:30.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:29 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3312229622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:30.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:29 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2569011097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:30.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:12:30.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:29 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3172309993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:30.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:29 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3312229622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:30.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:29 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2569011097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:30.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:29 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3172309993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:30.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:29 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3312229622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:30.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:29 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2569011097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.245+0000 7f94b84b3640 1 -- 172.21.15.33:0/1587119173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0103340 msgr2=0x7f94b0109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.245+0000 7f94b84b3640 1 --2- 172.21.15.33:0/1587119173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0103340 0x7f94b0109bd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009ca0 tx=0x7f94a002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- 172.21.15.33:0/1587119173 shutdown_connections 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 --2- 172.21.15.33:0/1587119173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0103340 0x7f94b0109bd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 --2- 172.21.15.33:0/1587119173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0102980 0x7f94b0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 --2- 172.21.15.33:0/1587119173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0101790 0x7f94b0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- 172.21.15.33:0/1587119173 >> 172.21.15.33:0/1587119173 conn(0x7f94b00fcd20 msgr2=0x7f94b00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- 172.21.15.33:0/1587119173 shutdown_connections 2024-04-24T23:12:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- 172.21.15.33:0/1587119173 wait complete. 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 Processor -- start 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- start start 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 0x7f94b0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0102980 0x7f94b0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0103340 0x7f94b019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94b010d410 con 0x7f94b0101790 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94b010d290 con 0x7f94b0103340 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b84b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94b010d590 con 0x7f94b0102980 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.246+0000 7f94b624f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 0x7f94b0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 0x7f94b0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33066/0 (socket says 172.21.15.33:33066) 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 -- 172.21.15.33:0/3790139293 learned_addr learned my addr 172.21.15.33:0/3790139293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0102980 0x7f94b0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b6a50640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0103340 0x7f94b019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 -- 172.21.15.33:0/3790139293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0102980 msgr2=0x7f94b0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0102980 0x7f94b0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 -- 172.21.15.33:0/3790139293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0103340 msgr2=0x7f94b019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0103340 0x7f94b019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 -- 172.21.15.33:0/3790139293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b019f640 con 0x7f94b0101790 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b6a50640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0103340 0x7f94b019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b624f640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 0x7f94b0197a20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f94a800d770 tx=0x7f94a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94af7fe640 1 -- 172.21.15.33:0/3790139293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8017070 con 0x7f94b0101790 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94af7fe640 1 -- 172.21.15.33:0/3790139293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a8009c30 con 0x7f94b0101790 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b019f930 con 0x7f94b0101790 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94af7fe640 1 -- 172.21.15.33:0/3790139293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8003dd0 con 0x7f94b0101790 2024-04-24T23:12:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.247+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b019fed0 con 0x7f94b0101790 2024-04-24T23:12:30.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.249+0000 7f94af7fe640 1 -- 172.21.15.33:0/3790139293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94a8003fb0 con 0x7f94b0101790 2024-04-24T23:12:30.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 Processor -- start 2024-04-24T23:12:30.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 -- start start 2024-04-24T23:12:30.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae700d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700de30 0x562ae7012930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae7012fa0 0x562ae7015430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ae6f70f00 con 0x562ae700d440 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ae6f19750 con 0x562ae7012fa0 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ae6f6cec0 con 0x562ae700de30 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94b6a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae700d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94b6a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae700d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33080/0 (socket says 172.21.15.33:33080) 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.250+0000 7f94b6a50640 1 -- 172.21.15.33:0/1846651583 learned_addr learned my addr 172.21.15.33:0/1846651583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94af7fe640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94980a2c10 0x7f94980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b624f640 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700de30 0x562ae7012930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae7012fa0 0x562ae7015430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94980a2c10 0x7f94980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94af7fe640 1 -- 172.21.15.33:0/3790139293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94a80ced70 con 0x7f94b0101790 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 -- 172.21.15.33:0/1846651583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700de30 msgr2=0x562ae7012930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700de30 0x562ae7012930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 -- 172.21.15.33:0/1846651583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae7012fa0 msgr2=0x562ae7015430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae7012fa0 0x562ae7015430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 -- 172.21.15.33:0/1846651583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ae6c852d0 con 0x562ae700d440 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b624f640 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700de30 0x562ae7012930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae700d860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94a0006010 tx=0x7f94a0031e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cb8381931f9cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94acbfe640 1 -- 172.21.15.33:0/1846651583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0004d40 con 0x562ae700d440 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94acbfe640 1 -- 172.21.15.33:0/1846651583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a004e070 con 0x562ae700d440 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94acbfe640 1 -- 172.21.15.33:0/1846651583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a002f870 con 0x562ae700d440 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.251+0000 7f94b6a50640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94980a2c10 0x7f94980a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94a004d000 tx=0x7f94a004b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1846651583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 msgr2=0x562ae700d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae700d860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94a0006010 tx=0x7f94a0031e90 comp rx=0 tx=0).stop 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1846651583 shutdown_connections 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae7012fa0 0x562ae7015430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700de30 0x562ae7012930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1846651583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae700d860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1846651583 >> 172.21.15.33:0/1846651583 conn(0x562ae7008ab0 msgr2=0x562ae7008eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1846651583 shutdown_connections 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1846651583 wait complete. 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 -- start start 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.252+0000 7f94bb1c2740 1 Processor -- start 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae7017710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae7017c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae7012fa0 0x562ae701ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ae7015dd0 con 0x562ae700d440 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ae6f2ccf0 con 0x562ae700de30 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ae7015f50 con 0x562ae7012fa0 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 10 client.0 _ll_register_callbacks cb 0x562ae6f2c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:30.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae7017710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae7017710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33094/0 (socket says 172.21.15.33:33094) 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 -- 172.21.15.33:0/3814379005 learned_addr learned my addr 172.21.15.33:0/3814379005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae7017c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 -- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae7012fa0 msgr2=0x562ae701ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b6a50640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae7012fa0 0x562ae701ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae7012fa0 0x562ae701ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 -- 172.21.15.33:0/3814379005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 msgr2=0x562ae7017c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae7017c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 -- 172.21.15.33:0/3814379005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ae6d4f6b0 con 0x562ae700d440 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b6a50640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae7012fa0 0x562ae701ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94b624f640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae7017710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f94a80cc480 tx=0x7f94a806a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a340e7296aa820 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f9495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f9495ffb640 1 -- 172.21.15.33:0/3814379005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8020030 con 0x562ae700d440 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f9495ffb640 1 -- 172.21.15.33:0/3814379005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94a80042a0 con 0x562ae700d440 2024-04-24T23:12:30.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.253+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a004e070 con 0x562ae700d440 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f9495ffb640 1 -- 172.21.15.33:0/3814379005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f94a809ab40 con 0x562ae700d440 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 10 client.15432 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ae7021ce0 con 0x562ae700d440 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f9495ffb640 1 -- 172.21.15.33:0/3814379005 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a004e070 con 0x562ae700d440 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f9495ffb640 1 -- 172.21.15.33:0/3814379005 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f94a8097850 con 0x562ae700d440 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 10 client.15432 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 20 client.15432 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae7024380 0x562ae70257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 4 client.15432 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ae6f7e440 con 0x562ae7024380 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.254+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae7024380 0x562ae70257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.255+0000 7f9495ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:30.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.255+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae7024380 0x562ae70257f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1ec0da29949fa4 server_cookie=76045eb6d9bcc733 in_seq=0 out_seq=0 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f9495ffb640 1 -- 172.21.15.33:0/3814379005 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ae6f7e440 con 0x562ae7024380 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f9495ffb640 10 client.15432 handle_command_reply: tid=0 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 client.15432 shutdown 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 msgr2=0x562ae7017710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae7017710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f94a80cc480 tx=0x7f94a806a160 comp rx=0 tx=0).stop 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 shutdown_connections 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae7024380 0x562ae70257f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae7012fa0 0x562ae701ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae7017c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f9495ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/3814379005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae700d440 0x562ae7017710 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 >> 172.21.15.33:0/3814379005 conn(0x562ae7008ab0 msgr2=0x562ae7007ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 shutdown_connections 2024-04-24T23:12:30.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 1 -- 172.21.15.33:0/3814379005 wait complete. 2024-04-24T23:12:30.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.256+0000 7f94bb1c2740 20 client.15432 trim_cache size 0 max 16384 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 Processor -- start 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 -- start start 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae7003690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae7017320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ae7023b30 con 0x562ae7012fa0 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ae7024200 con 0x562ae700de30 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.274+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ae70248d0 con 0x562ae700d440 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae7017320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae7017320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33104/0 (socket says 172.21.15.33:33104) 2024-04-24T23:12:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 -- 172.21.15.33:0/1167419266 learned_addr learned my addr 172.21.15.33:0/1167419266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b624f640 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae7003690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 -- 172.21.15.33:0/1167419266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 msgr2=0x562ae7003690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae7003690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 -- 172.21.15.33:0/1167419266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 msgr2=0x562ae701ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 -- 172.21.15.33:0/1167419266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ae7012cf0 con 0x562ae7012fa0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b624f640 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae7003690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f94b6a50640 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae7017320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f94a0031f80 tx=0x7f94a002f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24f7fe0196f995 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f9497fff640 1 -- 172.21.15.33:0/1167419266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a002f810 con 0x562ae7012fa0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f9497fff640 1 -- 172.21.15.33:0/1167419266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a005b070 con 0x562ae7012fa0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.275+0000 7f9497fff640 1 -- 172.21.15.33:0/1167419266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0004350 con 0x562ae7012fa0 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1167419266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 msgr2=0x562ae7017320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae7017320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f94a0031f80 tx=0x7f94a002f500 comp rx=0 tx=0).stop 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1167419266 shutdown_connections 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae7017320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/1167419266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae7003690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1167419266 >> 172.21.15.33:0/1167419266 conn(0x562ae7008ab0 msgr2=0x562ae7008e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1167419266 shutdown_connections 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- 172.21.15.33:0/1167419266 wait complete. 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- start start 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 Processor -- start 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae701cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae701d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ae7025670 con 0x562ae7012fa0 2024-04-24T23:12:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562ae7024fa0 con 0x562ae700de30 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ae7025d40 con 0x562ae700d440 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94bb1c2740 10 client.0 _ll_register_callbacks cb 0x7f94a800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94b6a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae701d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94b6a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae701d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33106/0 (socket says 172.21.15.33:33106) 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94b6a50640 1 -- 172.21.15.33:0/2386731702 learned_addr learned my addr 172.21.15.33:0/2386731702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94b624f640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae701cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.276+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b6a50640 1 -- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 msgr2=0x562ae701cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b6a50640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae701cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b6a50640 1 -- 172.21.15.33:0/2386731702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 msgr2=0x562ae701d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b6a50640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b6a50640 1 -- 172.21.15.33:0/2386731702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ae701e000 con 0x562ae7012fa0 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b624f640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae701cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b5a4e640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94b6a50640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae701d870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005d50 tx=0x7f94a002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a8a6df062512c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f9495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f9495ffb640 1 -- 172.21.15.33:0/2386731702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00040b0 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f9495ffb640 1 -- 172.21.15.33:0/2386731702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94a0004250 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a005b070 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f9495ffb640 1 -- 172.21.15.33:0/2386731702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f94a006d050 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94bb1c2740 10 client.15438 fetch_fsmap learned FSMap version 24 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562ae708a3f0 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f9495ffb640 1 -- 172.21.15.33:0/2386731702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a005b070 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f9495ffb640 1 -- 172.21.15.33:0/2386731702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f94a0030c90 con 0x562ae7012fa0 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.277+0000 7f94bb1c2740 10 client.15438 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94bb1c2740 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94bb1c2740 20 client.15438 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae708ebd0 0x562ae708d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94bb1c2740 4 client.15438 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "status"}) -- 0x562ae6f78790 con 0x562ae708ebd0 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94b624f640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae708ebd0 0x562ae708d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94b624f640 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae708ebd0 0x562ae708d8e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec1bd19e7c64ee6 server_cookie=d9200ddfaa6111f5 in_seq=0 out_seq=0 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f9495ffb640 10 client.15438 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f9495ffb640 1 -- 172.21.15.33:0/2386731702 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562ae6f78790 con 0x562ae708ebd0 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f9495ffb640 10 client.15438 handle_command_reply: tid=0 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.278+0000 7f94bb1c2740 1 client.15438 shutdown 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 msgr2=0x562ae701d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae701d870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005d50 tx=0x7f94a002f310 comp rx=0 tx=0).stop 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 shutdown_connections 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562ae708ebd0 0x562ae708d8e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ae7012fa0 0x562ae701d870 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ae700de30 0x562ae701d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f9495ffb640 0 client.15438 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 --2- 172.21.15.33:0/2386731702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ae700d440 0x562ae701cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 >> 172.21.15.33:0/2386731702 conn(0x562ae7008ab0 msgr2=0x562ae7003690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 shutdown_connections 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 1 -- 172.21.15.33:0/2386731702 wait complete. 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94bb1c2740 20 client.15438 trim_cache size 0 max 16384 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8917d4f4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-24T23:12:30.288 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24508, 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 137.949162683, 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 94, 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 153.087582489 2024-04-24T23:12:30.289 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94980a2c10 msgr2=0x7f94980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94980a2c10 0x7f94980a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94a004d000 tx=0x7f94a004b000 comp rx=0 tx=0).stop 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 msgr2=0x7f94b0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 0x7f94b0197a20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f94a800d770 tx=0x7f94a800dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 shutdown_connections 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94980a2c10 0x7f94980a50d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94b0103340 0x7f94b019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0102980 0x7f94b0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 --2- 172.21.15.33:0/3790139293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b0101790 0x7f94b0197a20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 >> 172.21.15.33:0/3790139293 conn(0x7f94b00fcd20 msgr2=0x7f94b0105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 shutdown_connections 2024-04-24T23:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:30.279+0000 7f94b84b3640 1 -- 172.21.15.33:0/3790139293 wait complete. 2024-04-24T23:12:30.292 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:12:30.301 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:12:30.302 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:12:30.302 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:12:30.302 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T23:12:30.314 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T23:12:30.314 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T23:12:30.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:30.315 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-24T23:12:30.330 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T23:12:30.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:30.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:12:30.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:30.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-24T23:12:30.491 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-24T23:12:30.491 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:12:30.491 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:30.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:12:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:31.116 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:30 smithi033 ceph-mon[29201]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:31.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:30 smithi038 ceph-mon[32450]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:31.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:30 smithi117 ceph-mon[32529]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 -- 172.21.15.33:0/1052355530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b81082a0 msgr2=0x7f67b8108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 --2- 172.21.15.33:0/1052355530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b81082a0 0x7f67b8108680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009990 tx=0x7f67a8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 -- 172.21.15.33:0/1052355530 shutdown_connections 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 --2- 172.21.15.33:0/1052355530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806b170 0x7f67b810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 --2- 172.21.15.33:0/1052355530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b81082a0 0x7f67b8108680 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 --2- 172.21.15.33:0/1052355530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b806a7d0 0x7f67b806ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 -- 172.21.15.33:0/1052355530 >> 172.21.15.33:0/1052355530 conn(0x7f67b80780a0 msgr2=0x7f67b80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 -- 172.21.15.33:0/1052355530 shutdown_connections 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.483+0000 7f67be75a700 1 -- 172.21.15.33:0/1052355530 wait complete. 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 Processor -- start 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 -- start start 2024-04-24T23:12:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806a7d0 0x7f67b81011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 0x7f67b8101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b81082a0 0x7f67b8101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8112500 con 0x7f67b806b170 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8112380 con 0x7f67b806a7d0 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67be75a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8112680 con 0x7f67b81082a0 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 0x7f67b8101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 0x7f67b8101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33128/0 (socket says 172.21.15.33:33128) 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67b77fe700 1 -- 172.21.15.33:0/1092912531 learned_addr learned my addr 172.21.15.33:0/1092912531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67b7fff700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806a7d0 0x7f67b81011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.484+0000 7f67bccf7700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b81082a0 0x7f67b8101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b77fe700 1 -- 172.21.15.33:0/1092912531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b81082a0 msgr2=0x7f67b8101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b77fe700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b81082a0 0x7f67b8101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b77fe700 1 -- 172.21.15.33:0/1092912531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806a7d0 msgr2=0x7f67b81011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b77fe700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806a7d0 0x7f67b81011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b77fe700 1 -- 172.21.15.33:0/1092912531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b81a6fe0 con 0x7f67b806b170 2024-04-24T23:12:32.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b7fff700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806a7d0 0x7f67b81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:32.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67bccf7700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b81082a0 0x7f67b8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:32.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b77fe700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 0x7f67b8101730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00d980 tx=0x7f67ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac017070 con 0x7f67b806b170 2024-04-24T23:12:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b81a72d0 con 0x7f67b806b170 2024-04-24T23:12:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ac005020 con 0x7f67b806b170 2024-04-24T23:12:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac009bc0 con 0x7f67b806b170 2024-04-24T23:12:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.485+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b81a77c0 con 0x7f67b806b170 2024-04-24T23:12:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.486+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b8103950 con 0x7f67b806b170 2024-04-24T23:12:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.488+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67ac0051d0 con 0x7f67b806b170 2024-04-24T23:12:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.489+0000 7f67b57fa700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f67a00a2c20 0x7f67a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.489+0000 7f67b7fff700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f67a00a2c20 0x7f67a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.489+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67ac0cf320 con 0x7f67b806b170 2024-04-24T23:12:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.490+0000 7f67b7fff700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f67a00a2c20 0x7f67a00a50e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67a80098e0 tx=0x7f67a8005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.498+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67ac020020 con 0x7f67b806b170 2024-04-24T23:12:32.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.758+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67b8102720 con 0x7f67b806b170 2024-04-24T23:12:32.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.758+0000 7f67b57fa700 1 -- 172.21.15.33:0/1092912531 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f67ac09bef0 con 0x7f67b806b170 2024-04-24T23:12:32.760 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:12:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.761+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f67a00a2c20 msgr2=0x7f67a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.761+0000 7f67be75a700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f67a00a2c20 0x7f67a00a50e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67a80098e0 tx=0x7f67a8005960 comp rx=0 tx=0).stop 2024-04-24T23:12:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.761+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 msgr2=0x7f67b8101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.761+0000 7f67be75a700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 0x7f67b8101730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00d980 tx=0x7f67ac00dc90 comp rx=0 tx=0).stop 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 shutdown_connections 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f67a00a2c20 0x7f67a00a50e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b81082a0 0x7f67b8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67b806a7d0 0x7f67b81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 --2- 172.21.15.33:0/1092912531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b806b170 0x7f67b8101730 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 >> 172.21.15.33:0/1092912531 conn(0x7f67b80780a0 msgr2=0x7f67b80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 shutdown_connections 2024-04-24T23:12:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:32.762+0000 7f67be75a700 1 -- 172.21.15.33:0/1092912531 wait complete. 2024-04-24T23:12:33.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:32 smithi033 ceph-mon[29201]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:12:33.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:32 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1092912531' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:33.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:32 smithi038 ceph-mon[32450]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:12:33.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:32 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1092912531' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:33.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:32 smithi117 ceph-mon[32529]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:12:33.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:32 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1092912531' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:12:33.399 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:12:33.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:33.400 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-24T23:12:33.415 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-24T23:12:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:12:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42062sec preferred_lft 42062sec 2024-04-24T23:12:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-24T23:12:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:33.417 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-24T23:12:33.417 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:12:33.417 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:33.417 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:12:33.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:12:33.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-24T23:12:33.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-24T23:12:33.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:12:33.418 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:12:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:33.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:33.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:33.557 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:12:33.581 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-24T23:12:33.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:33.582 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-24T23:12:33.639 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-24T23:12:33.639 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:12:33.639 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:12:33.639 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:12:33.639 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:12:33.640 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:33.640 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:12:33.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:12:33.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:12:33.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:12:33.640 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:12:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:33.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:33.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:33.767 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-24T23:12:33.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:33.783 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-24T23:12:33.838 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:33.838 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:12:33.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:12:33.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:12:33.838 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:12:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:33.956 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:12:33.957 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:12:34.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:34.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:34.070 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:34.070 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-24T23:12:34.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-24T23:12:34.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:12:34.070 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-24T23:12:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:34.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:34.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:12:34.149 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:34.165 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:12:34.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:12:34.165 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:34.221 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-24T23:12:34.221 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-24T23:12:34.221 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/63b4b8a1-1dbf-4f1c-a1f5-6d216109848e 2024-04-24T23:12:34.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:34.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/63b4b8a1-1dbf-4f1c-a1f5-6d216109848e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-24T23:12:34.304 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-24T23:12:34.304 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-24T23:12:34.304 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2024-04-24T23:12:34.421 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-24T23:12:34.422 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-24T23:12:34.422 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2024-04-24T23:12:34.422 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.38:3300,172.21.15.117:3300" 2024-04-24T23:12:34.422 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:3300,172.21.15.38:3300,172.21.15.117:3300:/volumes/qa/sv_1/63b4b8a1-1dbf-4f1c-a1f5-6d216109848e 2024-04-24T23:12:34.422 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=8917d4f4-028e-11ef-bc93-c7b262605968" will pass to kernel 2024-04-24T23:12:34.422 INFO:teuthology.orchestra.run.smithi033.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:12:34.423 INFO:teuthology.orchestra.run.smithi033.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:12:34.423 INFO:teuthology.orchestra.run.smithi033.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:12:34.423 INFO:teuthology.orchestra.run.smithi033.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:12:34.423 INFO:teuthology.orchestra.run.smithi033.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:12:34.425 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:12:34.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:34.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:34.491 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:12:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:12:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:34.591 INFO:teuthology.orchestra.run.smithi033.stdout:client25103 2024-04-24T23:12:34.591 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:12:34.591 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:12:34.608 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:12:34.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:34.705 INFO:teuthology.orchestra.run.smithi033.stdout:client25103 2024-04-24T23:12:34.705 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:12:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:12:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:34.762 INFO:teuthology.orchestra.run.smithi033.stdout:client25103 2024-04-24T23:12:34.763 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8917d4f4-028e-11ef-bc93-c7b262605968.client25103/status') 2024-04-24T23:12:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:34.821 INFO:teuthology.orchestra.run.smithi033.stdout:instance: client.25103 (3)172.21.15.33:0/3885757883 2024-04-24T23:12:34.821 INFO:teuthology.orchestra.run.smithi033.stdout:blocklisted: false 2024-04-24T23:12:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records in 2024-04-24T23:12:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records out 2024-04-24T23:12:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:71 bytes copied, 2.5244e-05 s, 2.8 MB/s 2024-04-24T23:12:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:34.829 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T23:12:34.840 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:12:34.840 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-24T23:12:34.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-24T23:12:34.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 -- 172.21.15.33:0/3995921165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 msgr2=0x7faa48102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 --2- 172.21.15.33:0/3995921165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48102e40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faa30009900 tx=0x7faa3002a3f0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 -- 172.21.15.33:0/3995921165 shutdown_connections 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 --2- 172.21.15.33:0/3995921165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48103380 0x7faa48109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 --2- 172.21.15.33:0/3995921165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48102e40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 --2- 172.21.15.33:0/3995921165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa481017d0 0x7faa48101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 -- 172.21.15.33:0/3995921165 >> 172.21.15.33:0/3995921165 conn(0x7faa480fcd60 msgr2=0x7faa480ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 -- 172.21.15.33:0/3995921165 shutdown_connections 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.942+0000 7faa4e37d640 1 -- 172.21.15.33:0/3995921165 wait complete. 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 Processor -- start 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 -- start start 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481017d0 0x7faa48197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48103380 0x7faa4819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810d480 con 0x7faa481029c0 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810d300 con 0x7faa481017d0 2024-04-24T23:12:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4e37d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810d600 con 0x7faa48103380 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33142/0 (socket says 172.21.15.33:33142) 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 -- 172.21.15.33:0/1289907461 learned_addr learned my addr 172.21.15.33:0/1289907461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa47fff640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481017d0 0x7faa48197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4c91a640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48103380 0x7faa4819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 -- 172.21.15.33:0/1289907461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48103380 msgr2=0x7faa4819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48103380 0x7faa4819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 -- 172.21.15.33:0/1289907461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481017d0 msgr2=0x7faa48197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481017d0 0x7faa48197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa477fe640 1 -- 172.21.15.33:0/1289907461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4819f540 con 0x7faa481029c0 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.943+0000 7faa4c91a640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48103380 0x7faa4819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.944+0000 7faa477fe640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48197e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa3002a9e0 tx=0x7faa30032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.944+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3003f030 con 0x7faa481029c0 2024-04-24T23:12:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.944+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa30003ce0 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.944+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4819f7d0 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.944+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30035070 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.944+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4819fd10 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.945+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa48108a20 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.946+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa30042030 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.947+0000 7faa457fa640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faa340a2cb0 0x7faa340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.947+0000 7faa47fff640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faa340a2cb0 0x7faa340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.947+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa300effa0 con 0x7faa481029c0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.947+0000 7faa47fff640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faa340a2cb0 0x7faa340a5170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faa380058c0 tx=0x7faa38005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:34.950+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa300bcb70 con 0x7faa481029c0 2024-04-24T23:12:35.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:34 smithi038 ceph-mon[32450]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:35.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.174+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7faa4802d170 con 0x7faa481029c0 2024-04-24T23:12:35.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:34 smithi117 ceph-mon[32529]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:35.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:34 smithi033 ceph-mon[29201]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.922+0000 7faa457fa640 1 -- 172.21.15.33:0/1289907461 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7faa300bc470 con 0x7faa481029c0 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.924+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faa340a2cb0 msgr2=0x7faa340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faa340a2cb0 0x7faa340a5170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faa380058c0 tx=0x7faa38005850 comp rx=0 tx=0).stop 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 msgr2=0x7faa48197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48197e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa3002a9e0 tx=0x7faa30032da0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 shutdown_connections 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faa340a2cb0 0x7faa340a5170 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48103380 0x7faa4819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa481029c0 0x7faa48197e60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 --2- 172.21.15.33:0/1289907461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481017d0 0x7faa48197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 >> 172.21.15.33:0/1289907461 conn(0x7faa480fcd60 msgr2=0x7faa48105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 shutdown_connections 2024-04-24T23:12:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:35.925+0000 7faa4e37d640 1 -- 172.21.15.33:0/1289907461 wait complete. 2024-04-24T23:12:35.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-24T23:12:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 -- 172.21.15.33:0/265417066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 msgr2=0x7f1250109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 --2- 172.21.15.33:0/265417066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f1250109c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1248009ca0 tx=0x7f124802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 -- 172.21.15.33:0/265417066 shutdown_connections 2024-04-24T23:12:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 --2- 172.21.15.33:0/265417066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f1250109c10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 --2- 172.21.15.33:0/265417066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12501029c0 0x7f1250102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 --2- 172.21.15.33:0/265417066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12501017d0 0x7f1250101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 -- 172.21.15.33:0/265417066 >> 172.21.15.33:0/265417066 conn(0x7f12500fcd60 msgr2=0x7f12500ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 -- 172.21.15.33:0/265417066 shutdown_connections 2024-04-24T23:12:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.050+0000 7f1254ab3640 1 -- 172.21.15.33:0/265417066 wait complete. 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 Processor -- start 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 -- start start 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12501017d0 0x7f12501978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12501029c0 0x7f1250197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f125019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f125010d430 con 0x7f1250103380 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f125010d2b0 con 0x7f12501029c0 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f1254ab3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f125010d5b0 con 0x7f12501017d0 2024-04-24T23:12:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f124ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f125019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f124ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f125019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47878/0 (socket says 172.21.15.33:47878) 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f124ed9d640 1 -- 172.21.15.33:0/3968810080 learned_addr learned my addr 172.21.15.33:0/3968810080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f124e59c640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12501017d0 0x7f12501978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.051+0000 7f124dd9b640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12501029c0 0x7f1250197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f124ed9d640 1 -- 172.21.15.33:0/3968810080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12501017d0 msgr2=0x7f12501978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f124ed9d640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12501017d0 0x7f12501978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f124ed9d640 1 -- 172.21.15.33:0/3968810080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12501029c0 msgr2=0x7f1250197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f124ed9d640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12501029c0 0x7f1250197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f124ed9d640 1 -- 172.21.15.33:0/3968810080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125019f510 con 0x7f1250103380 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f124ed9d640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f125019eea0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f12480098d0 tx=0x7f124802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124802fc30 con 0x7f1250103380 2024-04-24T23:12:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125019f7a0 con 0x7f1250103380 2024-04-24T23:12:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1248003e50 con 0x7f1250103380 2024-04-24T23:12:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12480305c0 con 0x7f1250103380 2024-04-24T23:12:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.052+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125019fd40 con 0x7f1250103380 2024-04-24T23:12:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.053+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1250108a20 con 0x7f1250103380 2024-04-24T23:12:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.054+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1248003970 con 0x7f1250103380 2024-04-24T23:12:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.055+0000 7f123f7fe640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12380a2c30 0x7f12380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.055+0000 7f124e59c640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12380a2c30 0x7f12380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.055+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f12480f07e0 con 0x7f1250103380 2024-04-24T23:12:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.055+0000 7f124e59c640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12380a2c30 0x7f12380a50f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1250102820 tx=0x7f124000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.059+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12480bd4d0 con 0x7f1250103380 2024-04-24T23:12:36.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:35 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1289907461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T23:12:36.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:35 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1289907461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T23:12:36.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:35 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1289907461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T23:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.272+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1250061c20 con 0x7f12380a2c30 2024-04-24T23:12:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.291+0000 7f123f7fe640 1 -- 172.21.15.33:0/3968810080 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1250061c20 con 0x7f12380a2c30 2024-04-24T23:12:36.293 INFO:teuthology.orchestra.run.smithi033.stdout:{} 2024-04-24T23:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12380a2c30 msgr2=0x7f12380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12380a2c30 0x7f12380a50f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1250102820 tx=0x7f124000a2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 msgr2=0x7f125019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f125019eea0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f12480098d0 tx=0x7f124802f840 comp rx=0 tx=0).stop 2024-04-24T23:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 shutdown_connections 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12380a2c30 0x7f12380a50f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250103380 0x7f125019eea0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12501029c0 0x7f1250197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 --2- 172.21.15.33:0/3968810080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12501017d0 0x7f12501978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 >> 172.21.15.33:0/3968810080 conn(0x7f12500fcd60 msgr2=0x7f1250105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 shutdown_connections 2024-04-24T23:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.293+0000 7f1254ab3640 1 -- 172.21.15.33:0/3968810080 wait complete. 2024-04-24T23:12:36.306 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T23:12:36.317 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-24T23:12:36.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:36.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 -- 172.21.15.33:0/3016257030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1017b0 msgr2=0x7f9bac101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3016257030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1017b0 0x7f9bac101bd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009900 tx=0x7f9ba402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:36.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 -- 172.21.15.33:0/3016257030 shutdown_connections 2024-04-24T23:12:36.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3016257030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac103360 0x7f9bac109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3016257030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bac1029a0 0x7f9bac102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3016257030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1017b0 0x7f9bac101bd0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 -- 172.21.15.33:0/3016257030 >> 172.21.15.33:0/3016257030 conn(0x7f9bac0fcd40 msgr2=0x7f9bac0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 -- 172.21.15.33:0/3016257030 shutdown_connections 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 -- 172.21.15.33:0/3016257030 wait complete. 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.443+0000 7f9bb159e640 1 Processor -- start 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 -- start start 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1017b0 0x7f9bac197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 0x7f9bac198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bac103360 0x7f9bac19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac10d480 con 0x7f9bac1029a0 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac10d300 con 0x7f9bac1017b0 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bb159e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac10d600 con 0x7f9bac103360 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 0x7f9bac198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 0x7f9bac198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47912/0 (socket says 172.21.15.33:47912) 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 -- 172.21.15.33:0/3670839911 learned_addr learned my addr 172.21.15.33:0/3670839911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9bab7fe640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bac103360 0x7f9bac19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baaffd640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1017b0 0x7f9bac197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 -- 172.21.15.33:0/3670839911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bac103360 msgr2=0x7f9bac19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bac103360 0x7f9bac19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 -- 172.21.15.33:0/3670839911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1017b0 msgr2=0x7f9bac197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1017b0 0x7f9bac197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baa7fc640 1 -- 172.21.15.33:0/3670839911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac19f760 con 0x7f9bac1029a0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.444+0000 7f9baaffd640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1017b0 0x7f9bac197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.445+0000 7f9baa7fc640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 0x7f9bac198080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000b4a0 tx=0x7f9ba000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.445+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0014070 con 0x7f9bac1029a0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.445+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba0010af0 con 0x7f9bac1029a0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.445+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac19fa50 con 0x7f9bac1029a0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.445+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0017d10 con 0x7f9bac1029a0 2024-04-24T23:12:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.445+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac19ffd0 con 0x7f9bac1029a0 2024-04-24T23:12:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.446+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac108ba0 con 0x7f9bac1029a0 2024-04-24T23:12:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.447+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ba0017070 con 0x7f9bac1029a0 2024-04-24T23:12:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.448+0000 7f9b93fff640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b940a2b90 0x7f9b940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.448+0000 7f9baaffd640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b940a2b90 0x7f9b940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.448+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ba00ce8c0 con 0x7f9bac1029a0 2024-04-24T23:12:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.448+0000 7f9baaffd640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b940a2b90 0x7f9b940a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4000c00 tx=0x7f9ba4031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.453+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ba009b490 con 0x7f9bac1029a0 2024-04-24T23:12:36.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.641+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bac062340 con 0x7f9bac1029a0 2024-04-24T23:12:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.643+0000 7f9b93fff640 1 -- 172.21.15.33:0/3670839911 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7f9ba009ad90 con 0x7f9bac1029a0 2024-04-24T23:12:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:36.645 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b940a2b90 msgr2=0x7f9b940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b940a2b90 0x7f9b940a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4000c00 tx=0x7f9ba4031400 comp rx=0 tx=0).stop 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 msgr2=0x7f9bac198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 0x7f9bac198080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000b4a0 tx=0x7f9ba000b960 comp rx=0 tx=0).stop 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 shutdown_connections 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b940a2b90 0x7f9b940a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bac103360 0x7f9bac19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bac1029a0 0x7f9bac198080 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 --2- 172.21.15.33:0/3670839911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bac1017b0 0x7f9bac197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 >> 172.21.15.33:0/3670839911 conn(0x7f9bac0fcd40 msgr2=0x7f9bac105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 shutdown_connections 2024-04-24T23:12:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.645+0000 7f9bb159e640 1 -- 172.21.15.33:0/3670839911 wait complete. 2024-04-24T23:12:36.657 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:12:36.276+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:36.657 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:12:36.276+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:36.657 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:12:36.276+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:36.658 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:12:36.276+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:36.658 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:12:36.276+0000 7f7b140fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:36.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:36.659 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 -- 172.21.15.33:0/74418369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30100950 msgr2=0x7f9f30100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/74418369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30100950 0x7f9f30100dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009900 tx=0x7f9f1802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 -- 172.21.15.33:0/74418369 shutdown_connections 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/74418369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30101310 0x7f9f30108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/74418369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30100950 0x7f9f30100dd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/74418369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101c20 0x7f9f30102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 -- 172.21.15.33:0/74418369 >> 172.21.15.33:0/74418369 conn(0x7f9f300fb380 msgr2=0x7f9f300fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 -- 172.21.15.33:0/74418369 shutdown_connections 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 -- 172.21.15.33:0/74418369 wait complete. 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 Processor -- start 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.763+0000 7f9f35f5a640 1 -- start start 2024-04-24T23:12:36.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30100950 0x7f9f30197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101310 0x7f9f30198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 0x7f9f3019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3010c300 con 0x7f9f30101c20 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3010c180 con 0x7f9f30100950 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3010c480 con 0x7f9f30101310 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 0x7f9f3019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 0x7f9f3019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47932/0 (socket says 172.21.15.33:47932) 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 -- 172.21.15.33:0/374227442 learned_addr learned my addr 172.21.15.33:0/374227442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2f7fe640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30100950 0x7f9f30197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f26ffd640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101310 0x7f9f30198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 -- 172.21.15.33:0/374227442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101310 msgr2=0x7f9f30198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101310 0x7f9f30198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 -- 172.21.15.33:0/374227442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30100950 msgr2=0x7f9f30197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30100950 0x7f9f30197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 -- 172.21.15.33:0/374227442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3019f740 con 0x7f9f30101c20 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f26ffd640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101310 0x7f9f30198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2ffff640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 0x7f9f3019f0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800eb10 tx=0x7f9f2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28004b20 con 0x7f9f30101c20 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3019fa30 con 0x7f9f30101c20 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f28004cc0 con 0x7f9f30101c20 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28010760 con 0x7f9f30101c20 2024-04-24T23:12:36.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.764+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3019ff50 con 0x7f9f30101c20 2024-04-24T23:12:36.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.765+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f301a0210 con 0x7f9f30101c20 2024-04-24T23:12:36.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.767+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f28005000 con 0x7f9f30101c20 2024-04-24T23:12:36.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.767+0000 7f9f2d7fa640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f1c0a2b90 0x7f9f1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.767+0000 7f9f2f7fe640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f1c0a2b90 0x7f9f1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.767+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9f280cfc20 con 0x7f9f30101c20 2024-04-24T23:12:36.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.768+0000 7f9f2f7fe640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f1c0a2b90 0x7f9f1c0a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f30069b20 tx=0x7f9f2000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.771+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f28053740 con 0x7f9f30101c20 2024-04-24T23:12:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.953+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f301a03e0 con 0x7f9f30101c20 2024-04-24T23:12:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.954+0000 7f9f2d7fa640 1 -- 172.21.15.33:0/374227442 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f9f2809c7f0 con 0x7f9f30101c20 2024-04-24T23:12:36.956 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.955+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f1c0a2b90 msgr2=0x7f9f1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f1c0a2b90 0x7f9f1c0a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f30069b20 tx=0x7f9f2000b390 comp rx=0 tx=0).stop 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 msgr2=0x7f9f3019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 0x7f9f3019f0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800eb10 tx=0x7f9f2800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 shutdown_connections 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f1c0a2b90 0x7f9f1c0a5050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f30101c20 0x7f9f3019f0d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f30101310 0x7f9f30198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 --2- 172.21.15.33:0/374227442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30100950 0x7f9f30197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 >> 172.21.15.33:0/374227442 conn(0x7f9f300fb380 msgr2=0x7f9f300fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 shutdown_connections 2024-04-24T23:12:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:36.956+0000 7f9f35f5a640 1 -- 172.21.15.33:0/374227442 wait complete. 2024-04-24T23:12:36.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 -- 172.21.15.33:0/3519451252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700102990 msgr2=0x7f2700102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 --2- 172.21.15.33:0/3519451252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700102990 0x7f2700102e10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009900 tx=0x7f26f402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 -- 172.21.15.33:0/3519451252 shutdown_connections 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 --2- 172.21.15.33:0/3519451252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2700103350 0x7f2700109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 --2- 172.21.15.33:0/3519451252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700102990 0x7f2700102e10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 --2- 172.21.15.33:0/3519451252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27001017a0 0x7f2700101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 -- 172.21.15.33:0/3519451252 >> 172.21.15.33:0/3519451252 conn(0x7f27000fcd10 msgr2=0x7f27000ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 -- 172.21.15.33:0/3519451252 shutdown_connections 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 -- 172.21.15.33:0/3519451252 wait complete. 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 Processor -- start 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 -- start start 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.072+0000 7f2707415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27001017a0 0x7f2700197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2700102990 0x7f2700198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 0x7f270019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f270010d4a0 con 0x7f2700103350 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f270010d320 con 0x7f2700102990 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f270010d620 con 0x7f27001017a0 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 0x7f270019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 0x7f270019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47950/0 (socket says 172.21.15.33:47950) 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 -- 172.21.15.33:0/1568228834 learned_addr learned my addr 172.21.15.33:0/1568228834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27051b1640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27001017a0 0x7f2700197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27049b0640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2700102990 0x7f2700198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 -- 172.21.15.33:0/1568228834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27001017a0 msgr2=0x7f2700197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27001017a0 0x7f2700197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 -- 172.21.15.33:0/1568228834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2700102990 msgr2=0x7f2700198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2700102990 0x7f2700198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 -- 172.21.15.33:0/1568228834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270019f760 con 0x7f2700103350 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f27059b2640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 0x7f270019f0f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009fd0 tx=0x7f26f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8011830 con 0x7f2700103350 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f8016070 con 0x7f2700103350 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270019fa50 con 0x7f2700103350 2024-04-24T23:12:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f800f2e0 con 0x7f2700103350 2024-04-24T23:12:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.073+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270019fff0 con 0x7f2700103350 2024-04-24T23:12:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.074+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2700108ae0 con 0x7f2700103350 2024-04-24T23:12:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.075+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26f80119d0 con 0x7f2700103350 2024-04-24T23:12:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.076+0000 7f26fe7fc640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26e80a2c10 0x7f26e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.076+0000 7f27051b1640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26e80a2c10 0x7f26e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.076+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f26f80ce950 con 0x7f2700103350 2024-04-24T23:12:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.077+0000 7f27051b1640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26e80a2c10 0x7f26e80a50d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27001027f0 tx=0x7f26f0009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.081+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26f809b520 con 0x7f2700103350 2024-04-24T23:12:37.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:36 smithi117 ceph-mon[32529]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:37.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1289907461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T23:12:37.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:36 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:12:37.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:36 smithi117 ceph-mon[32529]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:37.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:36 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3670839911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-mon[29201]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1289907461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T23:12:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:12:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-mon[29201]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:36 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3670839911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.266+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2700062340 con 0x7f2700103350 2024-04-24T23:12:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.267+0000 7f26fe7fc640 1 -- 172.21.15.33:0/1568228834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f26f809ae20 con 0x7f2700103350 2024-04-24T23:12:37.269 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:37.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26e80a2c10 msgr2=0x7f26e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26e80a2c10 0x7f26e80a50d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27001027f0 tx=0x7f26f0009680 comp rx=0 tx=0).stop 2024-04-24T23:12:37.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 msgr2=0x7f270019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 0x7f270019f0f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009fd0 tx=0x7f26f800b8e0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 shutdown_connections 2024-04-24T23:12:37.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26e80a2c10 0x7f26e80a50d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2700103350 0x7f270019f0f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2700102990 0x7f2700198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 --2- 172.21.15.33:0/1568228834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27001017a0 0x7f2700197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 >> 172.21.15.33:0/1568228834 conn(0x7f27000fcd10 msgr2=0x7f2700105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 shutdown_connections 2024-04-24T23:12:37.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.269+0000 7f2707415640 1 -- 172.21.15.33:0/1568228834 wait complete. 2024-04-24T23:12:37.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 -- 172.21.15.33:0/3071457821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c101890 msgr2=0x7f5d4c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 --2- 172.21.15.33:0/3071457821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c101890 0x7f5d4c101d10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009900 tx=0x7f5d4002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 -- 172.21.15.33:0/3071457821 shutdown_connections 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 --2- 172.21.15.33:0/3071457821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d4c102250 0x7f5d4c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 --2- 172.21.15.33:0/3071457821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c101890 0x7f5d4c101d10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 --2- 172.21.15.33:0/3071457821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4c1006a0 0x7f5d4c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 -- 172.21.15.33:0/3071457821 >> 172.21.15.33:0/3071457821 conn(0x7f5d4c0fbc10 msgr2=0x7f5d4c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 -- 172.21.15.33:0/3071457821 shutdown_connections 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.385+0000 7f5d523b7640 1 -- 172.21.15.33:0/3071457821 wait complete. 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 Processor -- start 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 -- start start 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4c1006a0 0x7f5d4c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d4c101890 0x7f5d4c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 0x7f5d4c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10c320 con 0x7f5d4c102250 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10c1a0 con 0x7f5d4c101890 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d523b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10c4a0 con 0x7f5d4c1006a0 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d50954640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 0x7f5d4c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d50954640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 0x7f5d4c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47962/0 (socket says 172.21.15.33:47962) 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d50954640 1 -- 172.21.15.33:0/4104974616 learned_addr learned my addr 172.21.15.33:0/4104974616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.386+0000 7f5d4bfff640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4c1006a0 0x7f5d4c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d50954640 1 -- 172.21.15.33:0/4104974616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4c1006a0 msgr2=0x7f5d4c1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d50954640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4c1006a0 0x7f5d4c1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d50954640 1 -- 172.21.15.33:0/4104974616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d4c101890 msgr2=0x7f5d4c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d50954640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d4c101890 0x7f5d4c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d50954640 1 -- 172.21.15.33:0/4104974616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c19e600 con 0x7f5d4c102250 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d50954640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 0x7f5d4c19df90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400eaa0 tx=0x7f5d4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400cdb0 con 0x7f5d4c102250 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d44005020 con 0x7f5d4c102250 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d440116b0 con 0x7f5d4c102250 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c19e8f0 con 0x7f5d4c102250 2024-04-24T23:12:37.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.387+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c19edf0 con 0x7f5d4c102250 2024-04-24T23:12:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.388+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c1079c0 con 0x7f5d4c102250 2024-04-24T23:12:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.389+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d44005560 con 0x7f5d4c102250 2024-04-24T23:12:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.390+0000 7f5d497fa640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.390+0000 7f5d4bfff640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.390+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5d440cf700 con 0x7f5d4c102250 2024-04-24T23:12:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.390+0000 7f5d4bfff640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c1016f0 tx=0x7f5d3c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:37.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.394+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d44016030 con 0x7f5d4c102250 2024-04-24T23:12:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:36 smithi038 ceph-mon[32450]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:12:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1289907461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T23:12:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:36 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:12:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:36 smithi038 ceph-mon[32450]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:36 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3670839911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:37.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.577+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5d4c062340 con 0x7f5d4c102250 2024-04-24T23:12:37.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.578+0000 7f5d497fa640 1 -- 172.21.15.33:0/4104974616 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f5d4409c2d0 con 0x7f5d4c102250 2024-04-24T23:12:37.580 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 msgr2=0x7f5d340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c1016f0 tx=0x7f5d3c005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 msgr2=0x7f5d4c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 0x7f5d4c19df90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400eaa0 tx=0x7f5d4400ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 shutdown_connections 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5d340a2b90 0x7f5d340a5050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4c102250 0x7f5d4c19df90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d4c101890 0x7f5d4c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 --2- 172.21.15.33:0/4104974616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4c1006a0 0x7f5d4c1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 >> 172.21.15.33:0/4104974616 conn(0x7f5d4c0fbc10 msgr2=0x7f5d4c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 shutdown_connections 2024-04-24T23:12:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.580+0000 7f5d523b7640 1 -- 172.21.15.33:0/4104974616 wait complete. 2024-04-24T23:12:37.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 -- 172.21.15.33:0/1228440359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 msgr2=0x7f68c4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 --2- 172.21.15.33:0/1228440359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4107000 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009900 tx=0x7f68b402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 -- 172.21.15.33:0/1228440359 shutdown_connections 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 --2- 172.21.15.33:0/1228440359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c4107540 0x7f68c4109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 --2- 172.21.15.33:0/1228440359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4107000 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 --2- 172.21.15.33:0/1228440359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68c41022e0 0x7f68c41046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.696+0000 7f68cb909640 1 -- 172.21.15.33:0/1228440359 >> 172.21.15.33:0/1228440359 conn(0x7f68c40fbc30 msgr2=0x7f68c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 -- 172.21.15.33:0/1228440359 shutdown_connections 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 -- 172.21.15.33:0/1228440359 wait complete. 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 Processor -- start 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 -- start start 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41022e0 0x7f68c4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68c4107540 0x7f68c419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410d450 con 0x7f68c4104c10 2024-04-24T23:12:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410d2d0 con 0x7f68c4107540 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68cb909640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410d5d0 con 0x7f68c41022e0 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68c8ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68c8ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47978/0 (socket says 172.21.15.33:47978) 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68c8ea4640 1 -- 172.21.15.33:0/3610140725 learned_addr learned my addr 172.21.15.33:0/3610140725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.697+0000 7f68c9ea6640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68c4107540 0x7f68c419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c96a5640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41022e0 0x7f68c4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c8ea4640 1 -- 172.21.15.33:0/3610140725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41022e0 msgr2=0x7f68c4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c8ea4640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41022e0 0x7f68c4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c8ea4640 1 -- 172.21.15.33:0/3610140725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68c4107540 msgr2=0x7f68c419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c8ea4640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68c4107540 0x7f68c419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c8ea4640 1 -- 172.21.15.33:0/3610140725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c419f6b0 con 0x7f68c4104c10 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c96a5640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41022e0 0x7f68c4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68c8ea4640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4197fd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68b4031440 tx=0x7f68b4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b403f030 con 0x7f68c4104c10 2024-04-24T23:12:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b4003ce0 con 0x7f68c4104c10 2024-04-24T23:12:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c419f940 con 0x7f68c4104c10 2024-04-24T23:12:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4035070 con 0x7f68c4104c10 2024-04-24T23:12:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.698+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c419fe30 con 0x7f68c4104c10 2024-04-24T23:12:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.699+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c40fd720 con 0x7f68c4104c10 2024-04-24T23:12:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.700+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68b4042030 con 0x7f68c4104c10 2024-04-24T23:12:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.701+0000 7f68ba7fc640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68b00a2b90 0x7f68b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.701+0000 7f68c96a5640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68b00a2b90 0x7f68b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.701+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68b40efa40 con 0x7f68c4104c10 2024-04-24T23:12:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.701+0000 7f68c96a5640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68b00a2b90 0x7f68b00a5050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f68c000abe0 tx=0x7f68c000a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:37.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.706+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f68b40bc5b0 con 0x7f68c4104c10 2024-04-24T23:12:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.889+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68c4062370 con 0x7f68c4104c10 2024-04-24T23:12:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.890+0000 7f68ba7fc640 1 -- 172.21.15.33:0/3610140725 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f68b40bbeb0 con 0x7f68c4104c10 2024-04-24T23:12:37.892 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68b00a2b90 msgr2=0x7f68b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68b00a2b90 0x7f68b00a5050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f68c000abe0 tx=0x7f68c000a5c0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 msgr2=0x7f68c4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4197fd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68b4031440 tx=0x7f68b4032ce0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 shutdown_connections 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68b00a2b90 0x7f68b00a5050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68c4107540 0x7f68c419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c4104c10 0x7f68c4197fd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 --2- 172.21.15.33:0/3610140725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41022e0 0x7f68c4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 >> 172.21.15.33:0/3610140725 conn(0x7f68c40fbc30 msgr2=0x7f68c4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 shutdown_connections 2024-04-24T23:12:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:37.892+0000 7f68cb909640 1 -- 172.21.15.33:0/3610140725 wait complete. 2024-04-24T23:12:37.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:38.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 -- 172.21.15.33:0/2183940503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328100970 msgr2=0x7fb328100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/2183940503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328100970 0x7fb328100dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb310009900 tx=0x7fb31002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:38.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 -- 172.21.15.33:0/2183940503 shutdown_connections 2024-04-24T23:12:38.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/2183940503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb328101310 0x7fb328101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/2183940503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328100970 0x7fb328100dd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/2183940503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 0x7fb328100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 -- 172.21.15.33:0/2183940503 >> 172.21.15.33:0/2183940503 conn(0x7fb3280fbc30 msgr2=0x7fb3280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 -- 172.21.15.33:0/2183940503 shutdown_connections 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.010+0000 7fb32dc9e640 1 -- 172.21.15.33:0/2183940503 wait complete. 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 Processor -- start 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 -- start start 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 0x7fb328199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb328100970 0x7fb32819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 0x7fb3281a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810f630 con 0x7fb328101310 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810f4b0 con 0x7fb328100010 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb32dc9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810f7b0 con 0x7fb328100970 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 0x7fb3281a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 0x7fb3281a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47994/0 (socket says 172.21.15.33:47994) 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 -- 172.21.15.33:0/766295322 learned_addr learned my addr 172.21.15.33:0/766295322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:38.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb326ffd640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb328100970 0x7fb32819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb3277fe640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 0x7fb328199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 -- 172.21.15.33:0/766295322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb328100970 msgr2=0x7fb32819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb328100970 0x7fb32819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 -- 172.21.15.33:0/766295322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 msgr2=0x7fb328199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 0x7fb328199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb327fff640 1 -- 172.21.15.33:0/766295322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3281a18c0 con 0x7fb328101310 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.011+0000 7fb3277fe640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 0x7fb328199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.012+0000 7fb327fff640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 0x7fb3281a1220 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb32000eaa0 tx=0x7fb32000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.012+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32000cdb0 con 0x7fb328101310 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.012+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb320005020 con 0x7fb328101310 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.012+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3200116b0 con 0x7fb328101310 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.012+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3281a1bb0 con 0x7fb328101310 2024-04-24T23:12:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.012+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3281a2080 con 0x7fb328101310 2024-04-24T23:12:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.013+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32810ad60 con 0x7fb328101310 2024-04-24T23:12:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.014+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb320005560 con 0x7fb328101310 2024-04-24T23:12:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.014+0000 7fb324ff9640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3140a2c20 0x7fb3140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.015+0000 7fb3277fe640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3140a2c20 0x7fb3140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.015+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3200cf810 con 0x7fb328101310 2024-04-24T23:12:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.015+0000 7fb3277fe640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3140a2c20 0x7fb3140a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3180058c0 tx=0x7fb318005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.019+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb320016030 con 0x7fb328101310 2024-04-24T23:12:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/374227442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1568228834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4104974616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3610140725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.205+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb32802d390 con 0x7fb328101310 2024-04-24T23:12:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.206+0000 7fb324ff9640 1 -- 172.21.15.33:0/766295322 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7fb32009c220 con 0x7fb328101310 2024-04-24T23:12:38.209 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:38.209 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.208+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3140a2c20 msgr2=0x7fb3140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.208+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3140a2c20 0x7fb3140a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3180058c0 tx=0x7fb318005d50 comp rx=0 tx=0).stop 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.208+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 msgr2=0x7fb3281a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.208+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 0x7fb3281a1220 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb32000eaa0 tx=0x7fb32000ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 shutdown_connections 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3140a2c20 0x7fb3140a50e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb328101310 0x7fb3281a1220 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb328100970 0x7fb32819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 --2- 172.21.15.33:0/766295322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328100010 0x7fb328199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 >> 172.21.15.33:0/766295322 conn(0x7fb3280fbc30 msgr2=0x7fb3280fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 shutdown_connections 2024-04-24T23:12:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.209+0000 7fb32dc9e640 1 -- 172.21.15.33:0/766295322 wait complete. 2024-04-24T23:12:38.222 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:38.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 -- 172.21.15.33:0/3328241334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218101870 msgr2=0x7fa218101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 --2- 172.21.15.33:0/3328241334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218101870 0x7fa218101cf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa208009900 tx=0x7fa20802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 -- 172.21.15.33:0/3328241334 shutdown_connections 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 --2- 172.21.15.33:0/3328241334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218102230 0x7fa218108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 --2- 172.21.15.33:0/3328241334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218101870 0x7fa218101cf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 --2- 172.21.15.33:0/3328241334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 0x7fa218100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 -- 172.21.15.33:0/3328241334 >> 172.21.15.33:0/3328241334 conn(0x7fa2180fbbf0 msgr2=0x7fa2180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 -- 172.21.15.33:0/3328241334 shutdown_connections 2024-04-24T23:12:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 -- 172.21.15.33:0/3328241334 wait complete. 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.326+0000 7fa21e553640 1 Processor -- start 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 -- start start 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 0x7fa2181979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218101870 0x7fa218197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 0x7fa21819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa21810d420 con 0x7fa218102230 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa21810d2a0 con 0x7fa218100680 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21e553640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa21810d5a0 con 0x7fa218101870 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 0x7fa21819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 0x7fa21819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48010/0 (socket says 172.21.15.33:48010) 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 -- 172.21.15.33:0/4217322835 learned_addr learned my addr 172.21.15.33:0/4217322835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa217fff640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 0x7fa2181979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa2177fe640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218101870 0x7fa218197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 -- 172.21.15.33:0/4217322835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218101870 msgr2=0x7fa218197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218101870 0x7fa218197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 -- 172.21.15.33:0/4217322835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 msgr2=0x7fa2181979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 0x7fa2181979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 -- 172.21.15.33:0/4217322835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21819f5e0 con 0x7fa218102230 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa217fff640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 0x7fa2181979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.327+0000 7fa21caf0640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 0x7fa21819ef70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa21000cce0 tx=0x7fa21000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.328+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21000f450 con 0x7fa218102230 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.328+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa21000fad0 con 0x7fa218102230 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.328+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21819f8d0 con 0x7fa218102230 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.328+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210018c10 con 0x7fa218102230 2024-04-24T23:12:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.328+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21819fdd0 con 0x7fa218102230 2024-04-24T23:12:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.329+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2181079a0 con 0x7fa218102230 2024-04-24T23:12:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.330+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa21000f5f0 con 0x7fa218102230 2024-04-24T23:12:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.330+0000 7fa2157fa640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2040a2c20 0x7fa2040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.330+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa2100cfcf0 con 0x7fa218102230 2024-04-24T23:12:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.330+0000 7fa217fff640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2040a2c20 0x7fa2040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.331+0000 7fa217fff640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2040a2c20 0x7fa2040a50e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2181016d0 tx=0x7fa200005db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.335+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa210014030 con 0x7fa218102230 2024-04-24T23:12:38.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/374227442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1568228834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4104974616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3610140725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/374227442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1568228834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4104974616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3610140725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.517+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa21802d390 con 0x7fa218102230 2024-04-24T23:12:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.518+0000 7fa2157fa640 1 -- 172.21.15.33:0/4217322835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fa21009c760 con 0x7fa218102230 2024-04-24T23:12:38.520 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2040a2c20 msgr2=0x7fa2040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2040a2c20 0x7fa2040a50e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2181016d0 tx=0x7fa200005db0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 msgr2=0x7fa21819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 0x7fa21819ef70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa21000cce0 tx=0x7fa21000a620 comp rx=0 tx=0).stop 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 shutdown_connections 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2040a2c20 0x7fa2040a50e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa218102230 0x7fa21819ef70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218101870 0x7fa218197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 --2- 172.21.15.33:0/4217322835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa218100680 0x7fa2181979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 >> 172.21.15.33:0/4217322835 conn(0x7fa2180fbbf0 msgr2=0x7fa218104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 shutdown_connections 2024-04-24T23:12:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.520+0000 7fa21e553640 1 -- 172.21.15.33:0/4217322835 wait complete. 2024-04-24T23:12:38.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:38.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 -- 172.21.15.33:0/976900234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720104bd0 msgr2=0x7fd720106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 --2- 172.21.15.33:0/976900234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720104bd0 0x7fd720106fc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd714009900 tx=0x7fd71402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:38.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 -- 172.21.15.33:0/976900234 shutdown_connections 2024-04-24T23:12:38.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 --2- 172.21.15.33:0/976900234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720107500 0x7fd7201098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 --2- 172.21.15.33:0/976900234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720104bd0 0x7fd720106fc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 --2- 172.21.15.33:0/976900234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7201022a0 0x7fd720104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 -- 172.21.15.33:0/976900234 >> 172.21.15.33:0/976900234 conn(0x7fd7200fbbf0 msgr2=0x7fd7200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 -- 172.21.15.33:0/976900234 shutdown_connections 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.636+0000 7fd727763640 1 -- 172.21.15.33:0/976900234 wait complete. 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 Processor -- start 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 -- start start 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7201022a0 0x7fd720197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720104bd0 0x7fd720198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 0x7fd72019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd72010d420 con 0x7fd720107500 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd72010d2a0 con 0x7fd7201022a0 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd727763640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd72010d5a0 con 0x7fd720104bd0 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd725d00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 0x7fd72019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd725d00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 0x7fd72019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48022/0 (socket says 172.21.15.33:48022) 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd725d00640 1 -- 172.21.15.33:0/3811216759 learned_addr learned my addr 172.21.15.33:0/3811216759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd7254ff640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7201022a0 0x7fd720197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.637+0000 7fd724cfe640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720104bd0 0x7fd720198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd725d00640 1 -- 172.21.15.33:0/3811216759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720104bd0 msgr2=0x7fd720198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd725d00640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720104bd0 0x7fd720198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd725d00640 1 -- 172.21.15.33:0/3811216759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7201022a0 msgr2=0x7fd720197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd725d00640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7201022a0 0x7fd720197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd725d00640 1 -- 172.21.15.33:0/3811216759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72019f720 con 0x7fd720107500 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd725d00640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 0x7fd72019f0b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd710009fd0 tx=0x7fd71000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710017070 con 0x7fd720107500 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd710003e50 con 0x7fd720107500 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100106c0 con 0x7fd720107500 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72019fa10 con 0x7fd720107500 2024-04-24T23:12:38.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.638+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72019ff30 con 0x7fd720107500 2024-04-24T23:12:38.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.639+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7200fd6e0 con 0x7fd720107500 2024-04-24T23:12:38.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.640+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd710003970 con 0x7fd720107500 2024-04-24T23:12:38.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.641+0000 7fd71e7fc640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd7080a2ca0 0x7fd7080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.641+0000 7fd7254ff640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd7080a2ca0 0x7fd7080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.641+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd7100cf2f0 con 0x7fd720107500 2024-04-24T23:12:38.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.641+0000 7fd7254ff640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd7080a2ca0 0x7fd7080a5160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd71800bbe0 tx=0x7fd71800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.645+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd71009bd00 con 0x7fd720107500 2024-04-24T23:12:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.831+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd72002d370 con 0x7fd720107500 2024-04-24T23:12:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.831+0000 7fd71e7fc640 1 -- 172.21.15.33:0/3811216759 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fd71009b600 con 0x7fd720107500 2024-04-24T23:12:38.833 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd7080a2ca0 msgr2=0x7fd7080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd7080a2ca0 0x7fd7080a5160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd71800bbe0 tx=0x7fd71800b390 comp rx=0 tx=0).stop 2024-04-24T23:12:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 msgr2=0x7fd72019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 0x7fd72019f0b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd710009fd0 tx=0x7fd71000b260 comp rx=0 tx=0).stop 2024-04-24T23:12:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 shutdown_connections 2024-04-24T23:12:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd7080a2ca0 0x7fd7080a5160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd720107500 0x7fd72019f0b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720104bd0 0x7fd720198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 --2- 172.21.15.33:0/3811216759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7201022a0 0x7fd720197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 >> 172.21.15.33:0/3811216759 conn(0x7fd7200fbbf0 msgr2=0x7fd720102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 shutdown_connections 2024-04-24T23:12:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.833+0000 7fd727763640 1 -- 172.21.15.33:0/3811216759 wait complete. 2024-04-24T23:12:38.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:38.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 -- 172.21.15.33:0/3144285993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102990 msgr2=0x7fa3c4102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 --2- 172.21.15.33:0/3144285993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102990 0x7fa3c4102e10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009900 tx=0x7fa3ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:38.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 -- 172.21.15.33:0/3144285993 shutdown_connections 2024-04-24T23:12:38.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 --2- 172.21.15.33:0/3144285993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4103350 0x7fa3c4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 --2- 172.21.15.33:0/3144285993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102990 0x7fa3c4102e10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 --2- 172.21.15.33:0/3144285993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41017a0 0x7fa3c4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 -- 172.21.15.33:0/3144285993 >> 172.21.15.33:0/3144285993 conn(0x7fa3c40fcd10 msgr2=0x7fa3c40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 -- 172.21.15.33:0/3144285993 shutdown_connections 2024-04-24T23:12:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.949+0000 7fa3c917e640 1 -- 172.21.15.33:0/3144285993 wait complete. 2024-04-24T23:12:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 Processor -- start 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 -- start start 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41017a0 0x7fa3c4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4102990 0x7fa3c4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 0x7fa3c419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410d510 con 0x7fa3c4103350 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410d390 con 0x7fa3c4102990 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c917e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410d690 con 0x7fa3c41017a0 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 0x7fa3c419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 0x7fa3c419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48040/0 (socket says 172.21.15.33:48040) 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c359e640 1 -- 172.21.15.33:0/1030824715 learned_addr learned my addr 172.21.15.33:0/1030824715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c2d9d640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41017a0 0x7fa3c4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3ba59c640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4102990 0x7fa3c4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c359e640 1 -- 172.21.15.33:0/1030824715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41017a0 msgr2=0x7fa3c4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.950+0000 7fa3c359e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41017a0 0x7fa3c4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c359e640 1 -- 172.21.15.33:0/1030824715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4102990 msgr2=0x7fa3c4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c359e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4102990 0x7fa3c4198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c359e640 1 -- 172.21.15.33:0/1030824715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419f840 con 0x7fa3c4103350 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c359e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 0x7fa3c419f1a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b970 tx=0x7fa3bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc019070 con 0x7fa3c4103350 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc00cb40 con 0x7fa3c4103350 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419fb30 con 0x7fa3c4103350 2024-04-24T23:12:38.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc007650 con 0x7fa3c4103350 2024-04-24T23:12:38.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.951+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a0080 con 0x7fa3c4103350 2024-04-24T23:12:38.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.952+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c4108b60 con 0x7fa3c4103350 2024-04-24T23:12:38.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.953+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa3bc01c070 con 0x7fa3c4103350 2024-04-24T23:12:38.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.954+0000 7fa3c0d99640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2ca0 0x7fa3b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:38.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.954+0000 7fa3c2d9d640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2ca0 0x7fa3b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:38.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.954+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa3bc0ceb10 con 0x7fa3c4103350 2024-04-24T23:12:38.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.954+0000 7fa3c2d9d640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2ca0 0x7fa3b00a5160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4199100 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:38.958+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3bc09b690 con 0x7fa3c4103350 2024-04-24T23:12:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.141+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3c402d320 con 0x7fa3c4103350 2024-04-24T23:12:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.142+0000 7fa3c0d99640 1 -- 172.21.15.33:0/1030824715 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fa3bc09af90 con 0x7fa3c4103350 2024-04-24T23:12:39.144 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2ca0 msgr2=0x7fa3b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2ca0 0x7fa3b00a5160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4199100 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 msgr2=0x7fa3c419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 0x7fa3c419f1a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b970 tx=0x7fa3bc00be30 comp rx=0 tx=0).stop 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 shutdown_connections 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3b00a2ca0 0x7fa3b00a5160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4103350 0x7fa3c419f1a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3c4102990 0x7fa3c4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 --2- 172.21.15.33:0/1030824715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c41017a0 0x7fa3c4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 >> 172.21.15.33:0/1030824715 conn(0x7fa3c40fcd10 msgr2=0x7fa3c4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 shutdown_connections 2024-04-24T23:12:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.144+0000 7fa3c917e640 1 -- 172.21.15.33:0/1030824715 wait complete. 2024-04-24T23:12:39.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:38 smithi033 ceph-mon[29201]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-24T23:12:39.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:38 smithi033 ceph-mon[29201]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-24T23:12:39.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/766295322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:39.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4217322835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:39.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3811216759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:39.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 -- 172.21.15.33:0/137178575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0102230 msgr2=0x7f65b0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 --2- 172.21.15.33:0/137178575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0102230 0x7f65b0108ac0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009ca0 tx=0x7f65a802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 -- 172.21.15.33:0/137178575 shutdown_connections 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 --2- 172.21.15.33:0/137178575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0102230 0x7f65b0108ac0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 --2- 172.21.15.33:0/137178575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0101870 0x7f65b0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 --2- 172.21.15.33:0/137178575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b0100680 0x7f65b0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 -- 172.21.15.33:0/137178575 >> 172.21.15.33:0/137178575 conn(0x7f65b00fbbf0 msgr2=0x7f65b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 -- 172.21.15.33:0/137178575 shutdown_connections 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.261+0000 7f65b6a64640 1 -- 172.21.15.33:0/137178575 wait complete. 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 Processor -- start 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 -- start start 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 0x7f65b0104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b0101870 0x7f65b0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0102230 0x7f65b0103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010c300 con 0x7f65b0100680 2024-04-24T23:12:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010c180 con 0x7f65b0102230 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b6a64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010c480 con 0x7f65b0101870 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b4800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 0x7f65b0104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b4800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 0x7f65b0104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48058/0 (socket says 172.21.15.33:48058) 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b4800640 1 -- 172.21.15.33:0/1252206141 learned_addr learned my addr 172.21.15.33:0/1252206141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65a7fff640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b0101870 0x7f65b0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.262+0000 7f65b5001640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0102230 0x7f65b0103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b4800640 1 -- 172.21.15.33:0/1252206141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b0101870 msgr2=0x7f65b0102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b4800640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b0101870 0x7f65b0102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b4800640 1 -- 172.21.15.33:0/1252206141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0102230 msgr2=0x7f65b0103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b4800640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0102230 0x7f65b0103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b4800640 1 -- 172.21.15.33:0/1252206141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b0103a20 con 0x7f65b0100680 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b5001640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0102230 0x7f65b0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b4800640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 0x7f65b0104610 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e9c0 tx=0x7f65ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00cd80 con 0x7f65b0100680 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b01a1b80 con 0x7f65b0100680 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65ac0035f0 con 0x7f65b0100680 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac010690 con 0x7f65b0100680 2024-04-24T23:12:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.263+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01a2080 con 0x7f65b0100680 2024-04-24T23:12:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.264+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b01a2250 con 0x7f65b0100680 2024-04-24T23:12:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.265+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f65ac0037a0 con 0x7f65b0100680 2024-04-24T23:12:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.266+0000 7f65a5ffb640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65980a2c20 0x7f65980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.266+0000 7f65a7fff640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65980a2c20 0x7f65980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.266+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f65ac0cfb00 con 0x7f65b0100680 2024-04-24T23:12:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.266+0000 7f65a7fff640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65980a2c20 0x7f65980a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f65b0104380 tx=0x7f65a0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.270+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f65ac09c510 con 0x7f65b0100680 2024-04-24T23:12:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:38 smithi038 ceph-mon[32450]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-24T23:12:39.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:38 smithi038 ceph-mon[32450]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-24T23:12:39.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/766295322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:39.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4217322835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:39.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3811216759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:38 smithi117 ceph-mon[32529]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-24T23:12:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:38 smithi117 ceph-mon[32529]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-24T23:12:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/766295322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4217322835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:39.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3811216759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.453+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f65b01a2630 con 0x7f65b0100680 2024-04-24T23:12:39.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.453+0000 7f65a5ffb640 1 -- 172.21.15.33:0/1252206141 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f65ac09be10 con 0x7f65b0100680 2024-04-24T23:12:39.455 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:39.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65980a2c20 msgr2=0x7f65980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65980a2c20 0x7f65980a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f65b0104380 tx=0x7f65a0005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 msgr2=0x7f65b0104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 0x7f65b0104610 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e9c0 tx=0x7f65ac00ee80 comp rx=0 tx=0).stop 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 shutdown_connections 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65980a2c20 0x7f65980a50e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b0102230 0x7f65b0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b0101870 0x7f65b0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 --2- 172.21.15.33:0/1252206141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b0100680 0x7f65b0104610 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 >> 172.21.15.33:0/1252206141 conn(0x7f65b00fbbf0 msgr2=0x7f65b00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 shutdown_connections 2024-04-24T23:12:39.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.455+0000 7f65b6a64640 1 -- 172.21.15.33:0/1252206141 wait complete. 2024-04-24T23:12:39.469 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-24T23:12:39.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:39.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 -- 172.21.15.33:0/2801595836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 msgr2=0x7fa530109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 --2- 172.21.15.33:0/2801595836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530109bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa528009ca0 tx=0x7fa52802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 -- 172.21.15.33:0/2801595836 shutdown_connections 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 --2- 172.21.15.33:0/2801595836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530109bd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 --2- 172.21.15.33:0/2801595836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa530102980 0x7fa530102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 --2- 172.21.15.33:0/2801595836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530101790 0x7fa530101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 -- 172.21.15.33:0/2801595836 >> 172.21.15.33:0/2801595836 conn(0x7fa5300fcd20 msgr2=0x7fa5300ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 -- 172.21.15.33:0/2801595836 shutdown_connections 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 -- 172.21.15.33:0/2801595836 wait complete. 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.570+0000 7fa535d7e640 1 Processor -- start 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 -- start start 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa530101790 0x7fa5300754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530102980 0x7fa530073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010d410 con 0x7fa530103340 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010d290 con 0x7fa530102980 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010d590 con 0x7fa530101790 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48070/0 (socket says 172.21.15.33:48070) 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 -- 172.21.15.33:0/1632276032 learned_addr learned my addr 172.21.15.33:0/1632276032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52f7fe640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa530101790 0x7fa5300754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52effd640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530102980 0x7fa530073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 -- 172.21.15.33:0/1632276032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa530101790 msgr2=0x7fa5300754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa530101790 0x7fa5300754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 -- 172.21.15.33:0/1632276032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530102980 msgr2=0x7fa530073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530102980 0x7fa530073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 -- 172.21.15.33:0/1632276032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa530074900 con 0x7fa530103340 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52effd640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530102980 0x7fa530073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52ffff640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530074070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa528005cc0 tx=0x7fa528009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528003c30 con 0x7fa530103340 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.571+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301a1be0 con 0x7fa530103340 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.572+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa528003dd0 con 0x7fa530103340 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.572+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52802fc50 con 0x7fa530103340 2024-04-24T23:12:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.572+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a2130 con 0x7fa530103340 2024-04-24T23:12:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.573+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530108a50 con 0x7fa530103340 2024-04-24T23:12:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.574+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa528004050 con 0x7fa530103340 2024-04-24T23:12:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.574+0000 7fa52cff9640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5180a2c20 0x7fa5180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.574+0000 7fa52f7fe640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5180a2c20 0x7fa5180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.574+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5280f0860 con 0x7fa530103340 2024-04-24T23:12:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.575+0000 7fa52f7fe640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5180a2c20 0x7fa5180a50e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa520005fd0 tx=0x7fa520005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.579+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5280bd500 con 0x7fa530103340 2024-04-24T23:12:39.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.804+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa530062340 con 0x7fa530103340 2024-04-24T23:12:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.805+0000 7fa52cff9640 1 -- 172.21.15.33:0/1632276032 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7fa5280bce00 con 0x7fa530103340 2024-04-24T23:12:39.807 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:39.807 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:12:35.176325+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:12:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-24T23:12:39.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5180a2c20 msgr2=0x7fa5180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5180a2c20 0x7fa5180a50e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa520005fd0 tx=0x7fa520005850 comp rx=0 tx=0).stop 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 msgr2=0x7fa530074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530074070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa528005cc0 tx=0x7fa528009890 comp rx=0 tx=0).stop 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 shutdown_connections 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5180a2c20 0x7fa5180a50e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa530103340 0x7fa530074070 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa530102980 0x7fa530073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 --2- 172.21.15.33:0/1632276032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa530101790 0x7fa5300754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 >> 172.21.15.33:0/1632276032 conn(0x7fa5300fcd20 msgr2=0x7fa530105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 shutdown_connections 2024-04-24T23:12:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.807+0000 7fa535d7e640 1 -- 172.21.15.33:0/1632276032 wait complete. 2024-04-24T23:12:39.821 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-24T23:12:39.821 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-24T23:12:39.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 -- 172.21.15.33:0/2083581724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f481001f0 msgr2=0x7f6f48109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 --2- 172.21.15.33:0/2083581724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f481001f0 0x7f6f48109da0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ca0 tx=0x7f6f4002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 -- 172.21.15.33:0/2083581724 shutdown_connections 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 --2- 172.21.15.33:0/2083581724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f481001f0 0x7f6f48109da0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 --2- 172.21.15.33:0/2083581724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f480ff830 0x7f6f480ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 --2- 172.21.15.33:0/2083581724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 0x7f6f481031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 -- 172.21.15.33:0/2083581724 >> 172.21.15.33:0/2083581724 conn(0x7f6f480fb3b0 msgr2=0x7f6f480fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 -- 172.21.15.33:0/2083581724 shutdown_connections 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 -- 172.21.15.33:0/2083581724 wait complete. 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 Processor -- start 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 -- start start 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 0x7f6f48197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.921+0000 7f6f50658640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f481001f0 0x7f6f48197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f50658640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 0x7f6f4819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f50658640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d410 con 0x7f6f480ff830 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f50658640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d290 con 0x7f6f481001f0 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f50658640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d590 con 0x7f6f48102dc0 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 0x7f6f48197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 0x7f6f48197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48090/0 (socket says 172.21.15.33:48090) 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 -- 172.21.15.33:0/1226308193 learned_addr learned my addr 172.21.15.33:0/1226308193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4dbf3640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f481001f0 0x7f6f48197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4ebf5640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 0x7f6f4819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 -- 172.21.15.33:0/1226308193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 msgr2=0x7f6f4819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 0x7f6f4819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 -- 172.21.15.33:0/1226308193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f481001f0 msgr2=0x7f6f48197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f481001f0 0x7f6f48197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 -- 172.21.15.33:0/1226308193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4819f650 con 0x7f6f480ff830 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4ebf5640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 0x7f6f4819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f4e3f4640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 0x7f6f48197a30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d950 tx=0x7f6f3800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38017070 con 0x7f6f480ff830 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3800be10 con 0x7f6f480ff830 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4819f940 con 0x7f6f480ff830 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800f620 con 0x7f6f480ff830 2024-04-24T23:12:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.922+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4819fe90 con 0x7f6f480ff830 2024-04-24T23:12:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.923+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f48108bb0 con 0x7f6f480ff830 2024-04-24T23:12:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.924+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6f3801a070 con 0x7f6f480ff830 2024-04-24T23:12:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.925+0000 7f6f3f7fe640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f340a2ca0 0x7f6f340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.925+0000 7f6f4dbf3640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f340a2ca0 0x7f6f340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.925+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f380cef10 con 0x7f6f480ff830 2024-04-24T23:12:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.926+0000 7f6f4dbf3640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f340a2ca0 0x7f6f340a5160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f48198f40 tx=0x7f6f4400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:39.930+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f3809b980 con 0x7f6f480ff830 2024-04-24T23:12:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.111+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f48061b80 con 0x7f6f340a2ca0 2024-04-24T23:12:40.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.120+0000 7f6f3f7fe640 1 -- 172.21.15.33:0/1226308193 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340812 (secure 0 0 0) 0x7f6f48061b80 con 0x7f6f340a2ca0 2024-04-24T23:12:40.127 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:12:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f340a2ca0 msgr2=0x7f6f340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f340a2ca0 0x7f6f340a5160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f48198f40 tx=0x7f6f4400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 msgr2=0x7f6f48197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 0x7f6f48197a30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d950 tx=0x7f6f3800dd50 comp rx=0 tx=0).stop 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 shutdown_connections 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f340a2ca0 0x7f6f340a5160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f48102dc0 0x7f6f4819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f481001f0 0x7f6f48197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 --2- 172.21.15.33:0/1226308193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f480ff830 0x7f6f48197a30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 >> 172.21.15.33:0/1226308193 conn(0x7f6f480fb3b0 msgr2=0x7f6f48105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 shutdown_connections 2024-04-24T23:12:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.124+0000 7f6f50658640 1 -- 172.21.15.33:0/1226308193 wait complete. 2024-04-24T23:12:40.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:39 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1030824715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:39 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1252206141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:39 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1632276032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:40.138 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-24T23:12:39.819005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":865,"num_read_kb":556,"num_write":353,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181934,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357588,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584236,"statfs":{"total":1151940427776,"available":1151574257664,"internally_reserved":0,"allocated":12845056,"data_stored":8230270,"data_compressed":434646,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295913+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:12:08.295913+0000","last_peered":"2024-04-24T23:12:08.295913+0000","last_clean":"2024-04-24T23:12:08.295913+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:12:08.295913+0000","last_undegraded":"2024-04-24T23:12:08.295913+0000","last_fullsized":"2024-04-24T23:12:08.295913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854282+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:12:07.854282+0000","last_peered":"2024-04-24T23:12:07.854282+0000","last_clean":"2024-04-24T23:12:07.854282+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:12:07.854282+0000","last_undegraded":"2024-04-24T23:12:07.854282+0000","last_fullsized":"2024-04-24T23:12:07.854282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854048+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:12:07.854048+0000","last_peered":"2024-04-24T23:12:07.854048+0000","last_clean":"2024-04-24T23:12:07.854048+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:12:07.854048+0000","last_undegraded":"2024-04-24T23:12:07.854048+0000","last_fullsized":"2024-04-24T23:12:07.854048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372122+0000","last_change":"2024-04-24T23:11:07.572557+0000","last_active":"2024-04-24T23:12:08.372122+0000","last_peered":"2024-04-24T23:12:08.372122+0000","last_clean":"2024-04-24T23:12:08.372122+0000","last_became_active":"2024-04-24T23:11:07.572144+0000","last_became_peered":"2024-04-24T23:11:07.572144+0000","last_unstale":"2024-04-24T23:12:08.372122+0000","last_undegraded":"2024-04-24T23:12:08.372122+0000","last_fullsized":"2024-04-24T23:12:08.372122+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:42.681992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373076+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:12:08.373076+0000","last_peered":"2024-04-24T23:12:08.373076+0000","last_clean":"2024-04-24T23:12:08.373076+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:12:08.373076+0000","last_undegraded":"2024-04-24T23:12:08.373076+0000","last_fullsized":"2024-04-24T23:12:08.373076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853784+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:12:07.853784+0000","last_peered":"2024-04-24T23:12:07.853784+0000","last_clean":"2024-04-24T23:12:07.853784+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:12:07.853784+0000","last_undegraded":"2024-04-24T23:12:07.853784+0000","last_fullsized":"2024-04-24T23:12:07.853784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372524+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:12:08.372524+0000","last_peered":"2024-04-24T23:12:08.372524+0000","last_clean":"2024-04-24T23:12:08.372524+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:12:08.372524+0000","last_undegraded":"2024-04-24T23:12:08.372524+0000","last_fullsized":"2024-04-24T23:12:08.372524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850067+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:12:07.850067+0000","last_peered":"2024-04-24T23:12:07.850067+0000","last_clean":"2024-04-24T23:12:07.850067+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:12:07.850067+0000","last_undegraded":"2024-04-24T23:12:07.850067+0000","last_fullsized":"2024-04-24T23:12:07.850067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281392+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:12:08.281392+0000","last_peered":"2024-04-24T23:12:08.281392+0000","last_clean":"2024-04-24T23:12:08.281392+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:12:08.281392+0000","last_undegraded":"2024-04-24T23:12:08.281392+0000","last_fullsized":"2024-04-24T23:12:08.281392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854555+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:12:07.854555+0000","last_peered":"2024-04-24T23:12:07.854555+0000","last_clean":"2024-04-24T23:12:07.854555+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:12:07.854555+0000","last_undegraded":"2024-04-24T23:12:07.854555+0000","last_fullsized":"2024-04-24T23:12:07.854555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373106+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:12:08.373106+0000","last_peered":"2024-04-24T23:12:08.373106+0000","last_clean":"2024-04-24T23:12:08.373106+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:12:08.373106+0000","last_undegraded":"2024-04-24T23:12:08.373106+0000","last_fullsized":"2024-04-24T23:12:08.373106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371900+0000","last_change":"2024-04-24T23:11:07.568947+0000","last_active":"2024-04-24T23:12:08.371900+0000","last_peered":"2024-04-24T23:12:08.371900+0000","last_clean":"2024-04-24T23:12:08.371900+0000","last_became_active":"2024-04-24T23:11:07.568528+0000","last_became_peered":"2024-04-24T23:11:07.568528+0000","last_unstale":"2024-04-24T23:12:08.371900+0000","last_undegraded":"2024-04-24T23:12:08.371900+0000","last_fullsized":"2024-04-24T23:12:08.371900+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:23.452961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352070+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:12:08.352070+0000","last_peered":"2024-04-24T23:12:08.352070+0000","last_clean":"2024-04-24T23:12:08.352070+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:12:08.352070+0000","last_undegraded":"2024-04-24T23:12:08.352070+0000","last_fullsized":"2024-04-24T23:12:08.352070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853752+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:12:07.853752+0000","last_peered":"2024-04-24T23:12:07.853752+0000","last_clean":"2024-04-24T23:12:07.853752+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:12:07.853752+0000","last_undegraded":"2024-04-24T23:12:07.853752+0000","last_fullsized":"2024-04-24T23:12:07.853752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853692+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:12:07.853692+0000","last_peered":"2024-04-24T23:12:07.853692+0000","last_clean":"2024-04-24T23:12:07.853692+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:12:07.853692+0000","last_undegraded":"2024-04-24T23:12:07.853692+0000","last_fullsized":"2024-04-24T23:12:07.853692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280637+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:12:08.280637+0000","last_peered":"2024-04-24T23:12:08.280637+0000","last_clean":"2024-04-24T23:12:08.280637+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:12:08.280637+0000","last_undegraded":"2024-04-24T23:12:08.280637+0000","last_fullsized":"2024-04-24T23:12:08.280637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295782+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:12:08.295782+0000","last_peered":"2024-04-24T23:12:08.295782+0000","last_clean":"2024-04-24T23:12:08.295782+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:12:08.295782+0000","last_undegraded":"2024-04-24T23:12:08.295782+0000","last_fullsized":"2024-04-24T23:12:08.295782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372278+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:12:08.372278+0000","last_peered":"2024-04-24T23:12:08.372278+0000","last_clean":"2024-04-24T23:12:08.372278+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:12:08.372278+0000","last_undegraded":"2024-04-24T23:12:08.372278+0000","last_fullsized":"2024-04-24T23:12:08.372278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855452+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:12:07.855452+0000","last_peered":"2024-04-24T23:12:07.855452+0000","last_clean":"2024-04-24T23:12:07.855452+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:12:07.855452+0000","last_undegraded":"2024-04-24T23:12:07.855452+0000","last_fullsized":"2024-04-24T23:12:07.855452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855027+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:12:07.855027+0000","last_peered":"2024-04-24T23:12:07.855027+0000","last_clean":"2024-04-24T23:12:07.855027+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:12:07.855027+0000","last_undegraded":"2024-04-24T23:12:07.855027+0000","last_fullsized":"2024-04-24T23:12:07.855027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372315+0000","last_change":"2024-04-24T23:11:07.769518+0000","last_active":"2024-04-24T23:12:08.372315+0000","last_peered":"2024-04-24T23:12:08.372315+0000","last_clean":"2024-04-24T23:12:08.372315+0000","last_became_active":"2024-04-24T23:11:07.763040+0000","last_became_peered":"2024-04-24T23:11:07.763040+0000","last_unstale":"2024-04-24T23:12:08.372315+0000","last_undegraded":"2024-04-24T23:12:08.372315+0000","last_fullsized":"2024-04-24T23:12:08.372315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.056568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850728+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:12:07.850728+0000","last_peered":"2024-04-24T23:12:07.850728+0000","last_clean":"2024-04-24T23:12:07.850728+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:12:07.850728+0000","last_undegraded":"2024-04-24T23:12:07.850728+0000","last_fullsized":"2024-04-24T23:12:07.850728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192411+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:12:08.192411+0000","last_peered":"2024-04-24T23:12:08.192411+0000","last_clean":"2024-04-24T23:12:08.192411+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:12:08.192411+0000","last_undegraded":"2024-04-24T23:12:08.192411+0000","last_fullsized":"2024-04-24T23:12:08.192411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850867+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:12:07.850867+0000","last_peered":"2024-04-24T23:12:07.850867+0000","last_clean":"2024-04-24T23:12:07.850867+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:12:07.850867+0000","last_undegraded":"2024-04-24T23:12:07.850867+0000","last_fullsized":"2024-04-24T23:12:07.850867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854113+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:12:07.854113+0000","last_peered":"2024-04-24T23:12:07.854113+0000","last_clean":"2024-04-24T23:12:07.854113+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:12:07.854113+0000","last_undegraded":"2024-04-24T23:12:07.854113+0000","last_fullsized":"2024-04-24T23:12:07.854113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192900+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:12:08.192900+0000","last_peered":"2024-04-24T23:12:08.192900+0000","last_clean":"2024-04-24T23:12:08.192900+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:12:08.192900+0000","last_undegraded":"2024-04-24T23:12:08.192900+0000","last_fullsized":"2024-04-24T23:12:08.192900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295762+0000","last_change":"2024-04-24T23:11:07.583051+0000","last_active":"2024-04-24T23:12:08.295762+0000","last_peered":"2024-04-24T23:12:08.295762+0000","last_clean":"2024-04-24T23:12:08.295762+0000","last_became_active":"2024-04-24T23:11:07.572502+0000","last_became_peered":"2024-04-24T23:11:07.572502+0000","last_unstale":"2024-04-24T23:12:08.295762+0000","last_undegraded":"2024-04-24T23:12:08.295762+0000","last_fullsized":"2024-04-24T23:12:08.295762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:21.581198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280792+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:12:08.280792+0000","last_peered":"2024-04-24T23:12:08.280792+0000","last_clean":"2024-04-24T23:12:08.280792+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:12:08.280792+0000","last_undegraded":"2024-04-24T23:12:08.280792+0000","last_fullsized":"2024-04-24T23:12:08.280792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192430+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:12:08.192430+0000","last_peered":"2024-04-24T23:12:08.192430+0000","last_clean":"2024-04-24T23:12:08.192430+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:12:08.192430+0000","last_undegraded":"2024-04-24T23:12:08.192430+0000","last_fullsized":"2024-04-24T23:12:08.192430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373278+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:12:08.373278+0000","last_peered":"2024-04-24T23:12:08.373278+0000","last_clean":"2024-04-24T23:12:08.373278+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:12:08.373278+0000","last_undegraded":"2024-04-24T23:12:08.373278+0000","last_fullsized":"2024-04-24T23:12:08.373278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372321+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:12:08.372321+0000","last_peered":"2024-04-24T23:12:08.372321+0000","last_clean":"2024-04-24T23:12:08.372321+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:12:08.372321+0000","last_undegraded":"2024-04-24T23:12:08.372321+0000","last_fullsized":"2024-04-24T23:12:08.372321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855307+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:12:07.855307+0000","last_peered":"2024-04-24T23:12:07.855307+0000","last_clean":"2024-04-24T23:12:07.855307+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:12:07.855307+0000","last_undegraded":"2024-04-24T23:12:07.855307+0000","last_fullsized":"2024-04-24T23:12:07.855307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.679678+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:12:33.679678+0000","last_peered":"2024-04-24T23:12:33.679678+0000","last_clean":"2024-04-24T23:12:33.679678+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:12:33.679678+0000","last_undegraded":"2024-04-24T23:12:33.679678+0000","last_fullsized":"2024-04-24T23:12:33.679678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373431+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:12:08.373431+0000","last_peered":"2024-04-24T23:12:08.373431+0000","last_clean":"2024-04-24T23:12:08.373431+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:12:08.373431+0000","last_undegraded":"2024-04-24T23:12:08.373431+0000","last_fullsized":"2024-04-24T23:12:08.373431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855174+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:12:07.855174+0000","last_peered":"2024-04-24T23:12:07.855174+0000","last_clean":"2024-04-24T23:12:07.855174+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:12:07.855174+0000","last_undegraded":"2024-04-24T23:12:07.855174+0000","last_fullsized":"2024-04-24T23:12:07.855174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853715+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:12:07.853715+0000","last_peered":"2024-04-24T23:12:07.853715+0000","last_clean":"2024-04-24T23:12:07.853715+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:12:07.853715+0000","last_undegraded":"2024-04-24T23:12:07.853715+0000","last_fullsized":"2024-04-24T23:12:07.853715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372610+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:12:08.372610+0000","last_peered":"2024-04-24T23:12:08.372610+0000","last_clean":"2024-04-24T23:12:08.372610+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:12:08.372610+0000","last_undegraded":"2024-04-24T23:12:08.372610+0000","last_fullsized":"2024-04-24T23:12:08.372610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295780+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:12:08.295780+0000","last_peered":"2024-04-24T23:12:08.295780+0000","last_clean":"2024-04-24T23:12:08.295780+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:12:08.295780+0000","last_undegraded":"2024-04-24T23:12:08.295780+0000","last_fullsized":"2024-04-24T23:12:08.295780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.351967+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:12:08.351967+0000","last_peered":"2024-04-24T23:12:08.351967+0000","last_clean":"2024-04-24T23:12:08.351967+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:12:08.351967+0000","last_undegraded":"2024-04-24T23:12:08.351967+0000","last_fullsized":"2024-04-24T23:12:08.351967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.849936+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:12:07.849936+0000","last_peered":"2024-04-24T23:12:07.849936+0000","last_clean":"2024-04-24T23:12:07.849936+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:12:07.849936+0000","last_undegraded":"2024-04-24T23:12:07.849936+0000","last_fullsized":"2024-04-24T23:12:07.849936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854277+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:12:07.854277+0000","last_peered":"2024-04-24T23:12:07.854277+0000","last_clean":"2024-04-24T23:12:07.854277+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:12:07.854277+0000","last_undegraded":"2024-04-24T23:12:07.854277+0000","last_fullsized":"2024-04-24T23:12:07.854277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371822+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:12:08.371822+0000","last_peered":"2024-04-24T23:12:08.371822+0000","last_clean":"2024-04-24T23:12:08.371822+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:12:08.371822+0000","last_undegraded":"2024-04-24T23:12:08.371822+0000","last_fullsized":"2024-04-24T23:12:08.371822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371791+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:12:08.371791+0000","last_peered":"2024-04-24T23:12:08.371791+0000","last_clean":"2024-04-24T23:12:08.371791+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:12:08.371791+0000","last_undegraded":"2024-04-24T23:12:08.371791+0000","last_fullsized":"2024-04-24T23:12:08.371791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850098+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:12:07.850098+0000","last_peered":"2024-04-24T23:12:07.850098+0000","last_clean":"2024-04-24T23:12:07.850098+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:12:07.850098+0000","last_undegraded":"2024-04-24T23:12:07.850098+0000","last_fullsized":"2024-04-24T23:12:07.850098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:36.684142+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:12:36.684142+0000","last_peered":"2024-04-24T23:12:36.684142+0000","last_clean":"2024-04-24T23:12:36.684142+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:12:36.684142+0000","last_undegraded":"2024-04-24T23:12:36.684142+0000","last_fullsized":"2024-04-24T23:12:36.684142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281321+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:12:08.281321+0000","last_peered":"2024-04-24T23:12:08.281321+0000","last_clean":"2024-04-24T23:12:08.281321+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:12:08.281321+0000","last_undegraded":"2024-04-24T23:12:08.281321+0000","last_fullsized":"2024-04-24T23:12:08.281321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371692+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:12:08.371692+0000","last_peered":"2024-04-24T23:12:08.371692+0000","last_clean":"2024-04-24T23:12:08.371692+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:12:08.371692+0000","last_undegraded":"2024-04-24T23:12:08.371692+0000","last_fullsized":"2024-04-24T23:12:08.371692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373045+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:12:08.373045+0000","last_peered":"2024-04-24T23:12:08.373045+0000","last_clean":"2024-04-24T23:12:08.373045+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:12:08.373045+0000","last_undegraded":"2024-04-24T23:12:08.373045+0000","last_fullsized":"2024-04-24T23:12:08.373045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854248+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:12:07.854248+0000","last_peered":"2024-04-24T23:12:07.854248+0000","last_clean":"2024-04-24T23:12:07.854248+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:12:07.854248+0000","last_undegraded":"2024-04-24T23:12:07.854248+0000","last_fullsized":"2024-04-24T23:12:07.854248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193177+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:12:08.193177+0000","last_peered":"2024-04-24T23:12:08.193177+0000","last_clean":"2024-04-24T23:12:08.193177+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:12:08.193177+0000","last_undegraded":"2024-04-24T23:12:08.193177+0000","last_fullsized":"2024-04-24T23:12:08.193177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372194+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:12:08.372194+0000","last_peered":"2024-04-24T23:12:08.372194+0000","last_clean":"2024-04-24T23:12:08.372194+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:12:08.372194+0000","last_undegraded":"2024-04-24T23:12:08.372194+0000","last_fullsized":"2024-04-24T23:12:08.372194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281176+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:12:08.281176+0000","last_peered":"2024-04-24T23:12:08.281176+0000","last_clean":"2024-04-24T23:12:08.281176+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:12:08.281176+0000","last_undegraded":"2024-04-24T23:12:08.281176+0000","last_fullsized":"2024-04-24T23:12:08.281176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371789+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:12:08.371789+0000","last_peered":"2024-04-24T23:12:08.371789+0000","last_clean":"2024-04-24T23:12:08.371789+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:12:08.371789+0000","last_undegraded":"2024-04-24T23:12:08.371789+0000","last_fullsized":"2024-04-24T23:12:08.371789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'37","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:36.667224+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:12:36.667224+0000","last_peered":"2024-04-24T23:12:36.667224+0000","last_clean":"2024-04-24T23:12:36.667224+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:12:36.667224+0000","last_undegraded":"2024-04-24T23:12:36.667224+0000","last_fullsized":"2024-04-24T23:12:36.667224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":248,"num_read_kb":191,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373386+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:12:08.373386+0000","last_peered":"2024-04-24T23:12:08.373386+0000","last_clean":"2024-04-24T23:12:08.373386+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:12:08.373386+0000","last_undegraded":"2024-04-24T23:12:08.373386+0000","last_fullsized":"2024-04-24T23:12:08.373386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.375570+0000","last_change":"2024-04-24T23:12:08.375043+0000","last_active":"2024-04-24T23:12:08.375570+0000","last_peered":"2024-04-24T23:12:08.375570+0000","last_clean":"2024-04-24T23:12:08.375570+0000","last_became_active":"2024-04-24T23:12:08.374204+0000","last_became_peered":"2024-04-24T23:12:08.374204+0000","last_unstale":"2024-04-24T23:12:08.375570+0000","last_undegraded":"2024-04-24T23:12:08.375570+0000","last_fullsized":"2024-04-24T23:12:08.375570+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:36.596328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295494+0000","last_change":"2024-04-24T23:11:07.579273+0000","last_active":"2024-04-24T23:12:08.295494+0000","last_peered":"2024-04-24T23:12:08.295494+0000","last_clean":"2024-04-24T23:12:08.295494+0000","last_became_active":"2024-04-24T23:11:07.570480+0000","last_became_peered":"2024-04-24T23:11:07.570480+0000","last_unstale":"2024-04-24T23:12:08.295494+0000","last_undegraded":"2024-04-24T23:12:08.295494+0000","last_fullsized":"2024-04-24T23:12:08.295494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:21.291453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352055+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:12:08.352055+0000","last_peered":"2024-04-24T23:12:08.352055+0000","last_clean":"2024-04-24T23:12:08.352055+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:12:08.352055+0000","last_undegraded":"2024-04-24T23:12:08.352055+0000","last_fullsized":"2024-04-24T23:12:08.352055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295559+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:12:08.295559+0000","last_peered":"2024-04-24T23:12:08.295559+0000","last_clean":"2024-04-24T23:12:08.295559+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:12:08.295559+0000","last_undegraded":"2024-04-24T23:12:08.295559+0000","last_fullsized":"2024-04-24T23:12:08.295559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373038+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:12:08.373038+0000","last_peered":"2024-04-24T23:12:08.373038+0000","last_clean":"2024-04-24T23:12:08.373038+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:12:08.373038+0000","last_undegraded":"2024-04-24T23:12:08.373038+0000","last_fullsized":"2024-04-24T23:12:08.373038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373266+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:12:08.373266+0000","last_peered":"2024-04-24T23:12:08.373266+0000","last_clean":"2024-04-24T23:12:08.373266+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:12:08.373266+0000","last_undegraded":"2024-04-24T23:12:08.373266+0000","last_fullsized":"2024-04-24T23:12:08.373266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372070+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:12:08.372070+0000","last_peered":"2024-04-24T23:12:08.372070+0000","last_clean":"2024-04-24T23:12:08.372070+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:12:08.372070+0000","last_undegraded":"2024-04-24T23:12:08.372070+0000","last_fullsized":"2024-04-24T23:12:08.372070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:34.651637+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:12:34.651637+0000","last_peered":"2024-04-24T23:12:34.651637+0000","last_clean":"2024-04-24T23:12:34.651637+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:12:34.651637+0000","last_undegraded":"2024-04-24T23:12:34.651637+0000","last_fullsized":"2024-04-24T23:12:34.651637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372711+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:12:08.372711+0000","last_peered":"2024-04-24T23:12:08.372711+0000","last_clean":"2024-04-24T23:12:08.372711+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:12:08.372711+0000","last_undegraded":"2024-04-24T23:12:08.372711+0000","last_fullsized":"2024-04-24T23:12:08.372711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352276+0000","last_change":"2024-04-24T23:11:07.570768+0000","last_active":"2024-04-24T23:12:08.352276+0000","last_peered":"2024-04-24T23:12:08.352276+0000","last_clean":"2024-04-24T23:12:08.352276+0000","last_became_active":"2024-04-24T23:11:07.570306+0000","last_became_peered":"2024-04-24T23:11:07.570306+0000","last_unstale":"2024-04-24T23:12:08.352276+0000","last_undegraded":"2024-04-24T23:12:08.352276+0000","last_fullsized":"2024-04-24T23:12:08.352276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:10.450660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373058+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:12:08.373058+0000","last_peered":"2024-04-24T23:12:08.373058+0000","last_clean":"2024-04-24T23:12:08.373058+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:12:08.373058+0000","last_undegraded":"2024-04-24T23:12:08.373058+0000","last_fullsized":"2024-04-24T23:12:08.373058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371872+0000","last_change":"2024-04-24T23:11:07.762601+0000","last_active":"2024-04-24T23:12:08.371872+0000","last_peered":"2024-04-24T23:12:08.371872+0000","last_clean":"2024-04-24T23:12:08.371872+0000","last_became_active":"2024-04-24T23:11:07.762305+0000","last_became_peered":"2024-04-24T23:11:07.762305+0000","last_unstale":"2024-04-24T23:12:08.371872+0000","last_undegraded":"2024-04-24T23:12:08.371872+0000","last_fullsized":"2024-04-24T23:12:08.371872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:36.209046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372017+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:12:08.372017+0000","last_peered":"2024-04-24T23:12:08.372017+0000","last_clean":"2024-04-24T23:12:08.372017+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:12:08.372017+0000","last_undegraded":"2024-04-24T23:12:08.372017+0000","last_fullsized":"2024-04-24T23:12:08.372017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373271+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:12:08.373271+0000","last_peered":"2024-04-24T23:12:08.373271+0000","last_clean":"2024-04-24T23:12:08.373271+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:12:08.373271+0000","last_undegraded":"2024-04-24T23:12:08.373271+0000","last_fullsized":"2024-04-24T23:12:08.373271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295800+0000","last_change":"2024-04-24T23:11:07.569318+0000","last_active":"2024-04-24T23:12:08.295800+0000","last_peered":"2024-04-24T23:12:08.295800+0000","last_clean":"2024-04-24T23:12:08.295800+0000","last_became_active":"2024-04-24T23:11:07.568799+0000","last_became_peered":"2024-04-24T23:11:07.568799+0000","last_unstale":"2024-04-24T23:12:08.295800+0000","last_undegraded":"2024-04-24T23:12:08.295800+0000","last_fullsized":"2024-04-24T23:12:08.295800+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:40.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373085+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:12:08.373085+0000","last_peered":"2024-04-24T23:12:08.373085+0000","last_clean":"2024-04-24T23:12:08.373085+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:12:08.373085+0000","last_undegraded":"2024-04-24T23:12:08.373085+0000","last_fullsized":"2024-04-24T23:12:08.373085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372189+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:12:08.372189+0000","last_peered":"2024-04-24T23:12:08.372189+0000","last_clean":"2024-04-24T23:12:08.372189+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:12:08.372189+0000","last_undegraded":"2024-04-24T23:12:08.372189+0000","last_fullsized":"2024-04-24T23:12:08.372189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854317+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:12:07.854317+0000","last_peered":"2024-04-24T23:12:07.854317+0000","last_clean":"2024-04-24T23:12:07.854317+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:12:07.854317+0000","last_undegraded":"2024-04-24T23:12:07.854317+0000","last_fullsized":"2024-04-24T23:12:07.854317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352055+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:12:08.352055+0000","last_peered":"2024-04-24T23:12:08.352055+0000","last_clean":"2024-04-24T23:12:08.352055+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:12:08.352055+0000","last_undegraded":"2024-04-24T23:12:08.352055+0000","last_fullsized":"2024-04-24T23:12:08.352055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372748+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:12:08.372748+0000","last_peered":"2024-04-24T23:12:08.372748+0000","last_clean":"2024-04-24T23:12:08.372748+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:12:08.372748+0000","last_undegraded":"2024-04-24T23:12:08.372748+0000","last_fullsized":"2024-04-24T23:12:08.372748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853997+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:12:07.853997+0000","last_peered":"2024-04-24T23:12:07.853997+0000","last_clean":"2024-04-24T23:12:07.853997+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:12:07.853997+0000","last_undegraded":"2024-04-24T23:12:07.853997+0000","last_fullsized":"2024-04-24T23:12:07.853997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352356+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:12:08.352356+0000","last_peered":"2024-04-24T23:12:08.352356+0000","last_clean":"2024-04-24T23:12:08.352356+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:12:08.352356+0000","last_undegraded":"2024-04-24T23:12:08.352356+0000","last_fullsized":"2024-04-24T23:12:08.352356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372205+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:12:08.372205+0000","last_peered":"2024-04-24T23:12:08.372205+0000","last_clean":"2024-04-24T23:12:08.372205+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:12:08.372205+0000","last_undegraded":"2024-04-24T23:12:08.372205+0000","last_fullsized":"2024-04-24T23:12:08.372205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:34.650960+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:12:34.650960+0000","last_peered":"2024-04-24T23:12:34.650960+0000","last_clean":"2024-04-24T23:12:34.650960+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:12:34.650960+0000","last_undegraded":"2024-04-24T23:12:34.650960+0000","last_fullsized":"2024-04-24T23:12:34.650960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372870+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:12:08.372870+0000","last_peered":"2024-04-24T23:12:08.372870+0000","last_clean":"2024-04-24T23:12:08.372870+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:12:08.372870+0000","last_undegraded":"2024-04-24T23:12:08.372870+0000","last_fullsized":"2024-04-24T23:12:08.372870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850177+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:12:07.850177+0000","last_peered":"2024-04-24T23:12:07.850177+0000","last_clean":"2024-04-24T23:12:07.850177+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:12:07.850177+0000","last_undegraded":"2024-04-24T23:12:07.850177+0000","last_fullsized":"2024-04-24T23:12:07.850177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193019+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:12:08.193019+0000","last_peered":"2024-04-24T23:12:08.193019+0000","last_clean":"2024-04-24T23:12:08.193019+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:12:08.193019+0000","last_undegraded":"2024-04-24T23:12:08.193019+0000","last_fullsized":"2024-04-24T23:12:08.193019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372433+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:12:08.372433+0000","last_peered":"2024-04-24T23:12:08.372433+0000","last_clean":"2024-04-24T23:12:08.372433+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:12:08.372433+0000","last_undegraded":"2024-04-24T23:12:08.372433+0000","last_fullsized":"2024-04-24T23:12:08.372433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372229+0000","last_change":"2024-04-24T23:11:07.571671+0000","last_active":"2024-04-24T23:12:08.372229+0000","last_peered":"2024-04-24T23:12:08.372229+0000","last_clean":"2024-04-24T23:12:08.372229+0000","last_became_active":"2024-04-24T23:11:07.571082+0000","last_became_peered":"2024-04-24T23:11:07.571082+0000","last_unstale":"2024-04-24T23:12:08.372229+0000","last_undegraded":"2024-04-24T23:12:08.372229+0000","last_fullsized":"2024-04-24T23:12:08.372229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:11.106931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.351810+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:12:08.351810+0000","last_peered":"2024-04-24T23:12:08.351810+0000","last_clean":"2024-04-24T23:12:08.351810+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:12:08.351810+0000","last_undegraded":"2024-04-24T23:12:08.351810+0000","last_fullsized":"2024-04-24T23:12:08.351810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372678+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:12:08.372678+0000","last_peered":"2024-04-24T23:12:08.372678+0000","last_clean":"2024-04-24T23:12:08.372678+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:12:08.372678+0000","last_undegraded":"2024-04-24T23:12:08.372678+0000","last_fullsized":"2024-04-24T23:12:08.372678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295482+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:12:08.295482+0000","last_peered":"2024-04-24T23:12:08.295482+0000","last_clean":"2024-04-24T23:12:08.295482+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:12:08.295482+0000","last_undegraded":"2024-04-24T23:12:08.295482+0000","last_fullsized":"2024-04-24T23:12:08.295482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373489+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:12:08.373489+0000","last_peered":"2024-04-24T23:12:08.373489+0000","last_clean":"2024-04-24T23:12:08.373489+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:12:08.373489+0000","last_undegraded":"2024-04-24T23:12:08.373489+0000","last_fullsized":"2024-04-24T23:12:08.373489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352204+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:12:08.352204+0000","last_peered":"2024-04-24T23:12:08.352204+0000","last_clean":"2024-04-24T23:12:08.352204+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:12:08.352204+0000","last_undegraded":"2024-04-24T23:12:08.352204+0000","last_fullsized":"2024-04-24T23:12:08.352204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850006+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:12:07.850006+0000","last_peered":"2024-04-24T23:12:07.850006+0000","last_clean":"2024-04-24T23:12:07.850006+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:12:07.850006+0000","last_undegraded":"2024-04-24T23:12:07.850006+0000","last_fullsized":"2024-04-24T23:12:07.850006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372243+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:12:08.372243+0000","last_peered":"2024-04-24T23:12:08.372243+0000","last_clean":"2024-04-24T23:12:08.372243+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:12:08.372243+0000","last_undegraded":"2024-04-24T23:12:08.372243+0000","last_fullsized":"2024-04-24T23:12:08.372243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850752+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:12:07.850752+0000","last_peered":"2024-04-24T23:12:07.850752+0000","last_clean":"2024-04-24T23:12:07.850752+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:12:07.850752+0000","last_undegraded":"2024-04-24T23:12:07.850752+0000","last_fullsized":"2024-04-24T23:12:07.850752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854907+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:12:07.854907+0000","last_peered":"2024-04-24T23:12:07.854907+0000","last_clean":"2024-04-24T23:12:07.854907+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:12:07.854907+0000","last_undegraded":"2024-04-24T23:12:07.854907+0000","last_fullsized":"2024-04-24T23:12:07.854907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193280+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:12:08.193280+0000","last_peered":"2024-04-24T23:12:08.193280+0000","last_clean":"2024-04-24T23:12:08.193280+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:12:08.193280+0000","last_undegraded":"2024-04-24T23:12:08.193280+0000","last_fullsized":"2024-04-24T23:12:08.193280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854403+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:12:07.854403+0000","last_peered":"2024-04-24T23:12:07.854403+0000","last_clean":"2024-04-24T23:12:07.854403+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:12:07.854403+0000","last_undegraded":"2024-04-24T23:12:07.854403+0000","last_fullsized":"2024-04-24T23:12:07.854403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192900+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:12:08.192900+0000","last_peered":"2024-04-24T23:12:08.192900+0000","last_clean":"2024-04-24T23:12:08.192900+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:12:08.192900+0000","last_undegraded":"2024-04-24T23:12:08.192900+0000","last_fullsized":"2024-04-24T23:12:08.192900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280633+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:12:08.280633+0000","last_peered":"2024-04-24T23:12:08.280633+0000","last_clean":"2024-04-24T23:12:08.280633+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:12:08.280633+0000","last_undegraded":"2024-04-24T23:12:08.280633+0000","last_fullsized":"2024-04-24T23:12:08.280633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373314+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:12:08.373314+0000","last_peered":"2024-04-24T23:12:08.373314+0000","last_clean":"2024-04-24T23:12:08.373314+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:12:08.373314+0000","last_undegraded":"2024-04-24T23:12:08.373314+0000","last_fullsized":"2024-04-24T23:12:08.373314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372313+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:12:08.372313+0000","last_peered":"2024-04-24T23:12:08.372313+0000","last_clean":"2024-04-24T23:12:08.372313+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:12:08.372313+0000","last_undegraded":"2024-04-24T23:12:08.372313+0000","last_fullsized":"2024-04-24T23:12:08.372313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850177+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:12:07.850177+0000","last_peered":"2024-04-24T23:12:07.850177+0000","last_clean":"2024-04-24T23:12:07.850177+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:12:07.850177+0000","last_undegraded":"2024-04-24T23:12:07.850177+0000","last_fullsized":"2024-04-24T23:12:07.850177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295519+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:12:08.295519+0000","last_peered":"2024-04-24T23:12:08.295519+0000","last_clean":"2024-04-24T23:12:08.295519+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:12:08.295519+0000","last_undegraded":"2024-04-24T23:12:08.295519+0000","last_fullsized":"2024-04-24T23:12:08.295519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192876+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:12:08.192876+0000","last_peered":"2024-04-24T23:12:08.192876+0000","last_clean":"2024-04-24T23:12:08.192876+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:12:08.192876+0000","last_undegraded":"2024-04-24T23:12:08.192876+0000","last_fullsized":"2024-04-24T23:12:08.192876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855100+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:12:07.855100+0000","last_peered":"2024-04-24T23:12:07.855100+0000","last_clean":"2024-04-24T23:12:07.855100+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:12:07.855100+0000","last_undegraded":"2024-04-24T23:12:07.855100+0000","last_fullsized":"2024-04-24T23:12:07.855100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850423+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:12:07.850423+0000","last_peered":"2024-04-24T23:12:07.850423+0000","last_clean":"2024-04-24T23:12:07.850423+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:12:07.850423+0000","last_undegraded":"2024-04-24T23:12:07.850423+0000","last_fullsized":"2024-04-24T23:12:07.850423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192912+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:12:08.192912+0000","last_peered":"2024-04-24T23:12:08.192912+0000","last_clean":"2024-04-24T23:12:08.192912+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:12:08.192912+0000","last_undegraded":"2024-04-24T23:12:08.192912+0000","last_fullsized":"2024-04-24T23:12:08.192912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372997+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:12:08.372997+0000","last_peered":"2024-04-24T23:12:08.372997+0000","last_clean":"2024-04-24T23:12:08.372997+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:12:08.372997+0000","last_undegraded":"2024-04-24T23:12:08.372997+0000","last_fullsized":"2024-04-24T23:12:08.372997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372923+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:12:08.372923+0000","last_peered":"2024-04-24T23:12:08.372923+0000","last_clean":"2024-04-24T23:12:08.372923+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:12:08.372923+0000","last_undegraded":"2024-04-24T23:12:08.372923+0000","last_fullsized":"2024-04-24T23:12:08.372923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280525+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:12:08.280525+0000","last_peered":"2024-04-24T23:12:08.280525+0000","last_clean":"2024-04-24T23:12:08.280525+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:12:08.280525+0000","last_undegraded":"2024-04-24T23:12:08.280525+0000","last_fullsized":"2024-04-24T23:12:08.280525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280781+0000","last_change":"2024-04-24T23:11:07.579626+0000","last_active":"2024-04-24T23:12:08.280781+0000","last_peered":"2024-04-24T23:12:08.280781+0000","last_clean":"2024-04-24T23:12:08.280781+0000","last_became_active":"2024-04-24T23:11:07.574907+0000","last_became_peered":"2024-04-24T23:11:07.574907+0000","last_unstale":"2024-04-24T23:12:08.280781+0000","last_undegraded":"2024-04-24T23:12:08.280781+0000","last_fullsized":"2024-04-24T23:12:08.280781+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:56.428190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373144+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:12:08.373144+0000","last_peered":"2024-04-24T23:12:08.373144+0000","last_clean":"2024-04-24T23:12:08.373144+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:12:08.373144+0000","last_undegraded":"2024-04-24T23:12:08.373144+0000","last_fullsized":"2024-04-24T23:12:08.373144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.375191+0000","last_change":"2024-04-24T23:12:08.374812+0000","last_active":"2024-04-24T23:12:08.375191+0000","last_peered":"2024-04-24T23:12:08.375191+0000","last_clean":"2024-04-24T23:12:08.375191+0000","last_became_active":"2024-04-24T23:12:08.374353+0000","last_became_peered":"2024-04-24T23:12:08.374353+0000","last_unstale":"2024-04-24T23:12:08.375191+0000","last_undegraded":"2024-04-24T23:12:08.375191+0000","last_fullsized":"2024-04-24T23:12:08.375191+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:58.241681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372501+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:12:08.372501+0000","last_peered":"2024-04-24T23:12:08.372501+0000","last_clean":"2024-04-24T23:12:08.372501+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:12:08.372501+0000","last_undegraded":"2024-04-24T23:12:08.372501+0000","last_fullsized":"2024-04-24T23:12:08.372501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373096+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:12:08.373096+0000","last_peered":"2024-04-24T23:12:08.373096+0000","last_clean":"2024-04-24T23:12:08.373096+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:12:08.373096+0000","last_undegraded":"2024-04-24T23:12:08.373096+0000","last_fullsized":"2024-04-24T23:12:08.373096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371788+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:12:08.371788+0000","last_peered":"2024-04-24T23:12:08.371788+0000","last_clean":"2024-04-24T23:12:08.371788+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:12:08.371788+0000","last_undegraded":"2024-04-24T23:12:08.371788+0000","last_fullsized":"2024-04-24T23:12:08.371788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352331+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:12:08.352331+0000","last_peered":"2024-04-24T23:12:08.352331+0000","last_clean":"2024-04-24T23:12:08.352331+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:12:08.352331+0000","last_undegraded":"2024-04-24T23:12:08.352331+0000","last_fullsized":"2024-04-24T23:12:08.352331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373318+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:12:08.373318+0000","last_peered":"2024-04-24T23:12:08.373318+0000","last_clean":"2024-04-24T23:12:08.373318+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:12:08.373318+0000","last_undegraded":"2024-04-24T23:12:08.373318+0000","last_fullsized":"2024-04-24T23:12:08.373318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295479+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:12:08.295479+0000","last_peered":"2024-04-24T23:12:08.295479+0000","last_clean":"2024-04-24T23:12:08.295479+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:12:08.295479+0000","last_undegraded":"2024-04-24T23:12:08.295479+0000","last_fullsized":"2024-04-24T23:12:08.295479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855299+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:12:07.855299+0000","last_peered":"2024-04-24T23:12:07.855299+0000","last_clean":"2024-04-24T23:12:07.855299+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:12:07.855299+0000","last_undegraded":"2024-04-24T23:12:07.855299+0000","last_fullsized":"2024-04-24T23:12:07.855299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854984+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:12:07.854984+0000","last_peered":"2024-04-24T23:12:07.854984+0000","last_clean":"2024-04-24T23:12:07.854984+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:12:07.854984+0000","last_undegraded":"2024-04-24T23:12:07.854984+0000","last_fullsized":"2024-04-24T23:12:07.854984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853900+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:12:07.853900+0000","last_peered":"2024-04-24T23:12:07.853900+0000","last_clean":"2024-04-24T23:12:07.853900+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:12:07.853900+0000","last_undegraded":"2024-04-24T23:12:07.853900+0000","last_fullsized":"2024-04-24T23:12:07.853900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352229+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:12:08.352229+0000","last_peered":"2024-04-24T23:12:08.352229+0000","last_clean":"2024-04-24T23:12:08.352229+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:12:08.352229+0000","last_undegraded":"2024-04-24T23:12:08.352229+0000","last_fullsized":"2024-04-24T23:12:08.352229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295817+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:12:08.295817+0000","last_peered":"2024-04-24T23:12:08.295817+0000","last_clean":"2024-04-24T23:12:08.295817+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:12:08.295817+0000","last_undegraded":"2024-04-24T23:12:08.295817+0000","last_fullsized":"2024-04-24T23:12:08.295817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855547+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:12:07.855547+0000","last_peered":"2024-04-24T23:12:07.855547+0000","last_clean":"2024-04-24T23:12:07.855547+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:12:07.855547+0000","last_undegraded":"2024-04-24T23:12:07.855547+0000","last_fullsized":"2024-04-24T23:12:07.855547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850816+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:12:07.850816+0000","last_peered":"2024-04-24T23:12:07.850816+0000","last_clean":"2024-04-24T23:12:07.850816+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:12:07.850816+0000","last_undegraded":"2024-04-24T23:12:07.850816+0000","last_fullsized":"2024-04-24T23:12:07.850816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371911+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:12:08.371911+0000","last_peered":"2024-04-24T23:12:08.371911+0000","last_clean":"2024-04-24T23:12:08.371911+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:12:08.371911+0000","last_undegraded":"2024-04-24T23:12:08.371911+0000","last_fullsized":"2024-04-24T23:12:08.371911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850318+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:12:07.850318+0000","last_peered":"2024-04-24T23:12:07.850318+0000","last_clean":"2024-04-24T23:12:07.850318+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:12:07.850318+0000","last_undegraded":"2024-04-24T23:12:07.850318+0000","last_fullsized":"2024-04-24T23:12:07.850318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280809+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:12:08.280809+0000","last_peered":"2024-04-24T23:12:08.280809+0000","last_clean":"2024-04-24T23:12:08.280809+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:12:08.280809+0000","last_undegraded":"2024-04-24T23:12:08.280809+0000","last_fullsized":"2024-04-24T23:12:08.280809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352027+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:12:08.352027+0000","last_peered":"2024-04-24T23:12:08.352027+0000","last_clean":"2024-04-24T23:12:08.352027+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:12:08.352027+0000","last_undegraded":"2024-04-24T23:12:08.352027+0000","last_fullsized":"2024-04-24T23:12:08.352027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295519+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:12:08.295519+0000","last_peered":"2024-04-24T23:12:08.295519+0000","last_clean":"2024-04-24T23:12:08.295519+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:12:08.295519+0000","last_undegraded":"2024-04-24T23:12:08.295519+0000","last_fullsized":"2024-04-24T23:12:08.295519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":739,"num_read_kb":447,"num_write":170,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1198,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":213861,"data_compressed":4821,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1178528,"data_compressed":64827,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":526964,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":522206,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1118838,"data_compressed":63220,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":518044,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":586632,"data_compressed":28292,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":581020,"data_compressed":28292,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":518238,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":523808,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1113894,"data_compressed":63220,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71198,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63066,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70206,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:40.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:12:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/2180253360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81074e0 msgr2=0x7fa4a81098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/2180253360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81074e0 0x7fa4a81098d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009ca0 tx=0x7fa4a002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/2180253360 shutdown_connections 2024-04-24T23:12:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/2180253360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a81074e0 0x7fa4a81098d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/2180253360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a8104bb0 0x7fa4a8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/2180253360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a8102280 0x7fa4a8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/2180253360 >> 172.21.15.33:0/2180253360 conn(0x7fa4a80fbc30 msgr2=0x7fa4a80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/2180253360 shutdown_connections 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.278+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/2180253360 wait complete. 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 Processor -- start 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 -- start start 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 0x7fa4a8101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a8104bb0 0x7fa4a8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a81074e0 0x7fa4a8100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810d440 con 0x7fa4a8102280 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810d2c0 con 0x7fa4a8104bb0 2024-04-24T23:12:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4ad0a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810d5c0 con 0x7fa4a81074e0 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 0x7fa4a8101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 0x7fa4a8101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48120/0 (socket says 172.21.15.33:48120) 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4a6d9d640 1 -- 172.21.15.33:0/3007046572 learned_addr learned my addr 172.21.15.33:0/3007046572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa49ffff640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a8104bb0 0x7fa4a8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.279+0000 7fa4a759e640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a81074e0 0x7fa4a8100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a6d9d640 1 -- 172.21.15.33:0/3007046572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a81074e0 msgr2=0x7fa4a8100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a6d9d640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a81074e0 0x7fa4a8100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a6d9d640 1 -- 172.21.15.33:0/3007046572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a8104bb0 msgr2=0x7fa4a8100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a6d9d640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a8104bb0 0x7fa4a8100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a6d9d640 1 -- 172.21.15.33:0/3007046572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a8101000 con 0x7fa4a8102280 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a6d9d640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 0x7fa4a8101bf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d950 tx=0x7fa49800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498017070 con 0x7fa4a8102280 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a81a1b70 con 0x7fa4a8102280 2024-04-24T23:12:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498009c30 con 0x7fa4a8102280 2024-04-24T23:12:40.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49800f6b0 con 0x7fa4a8102280 2024-04-24T23:12:40.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.280+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a2110 con 0x7fa4a8102280 2024-04-24T23:12:40.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.281+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a80fd720 con 0x7fa4a8102280 2024-04-24T23:12:40.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.282+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa4980109d0 con 0x7fa4a8102280 2024-04-24T23:12:40.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.283+0000 7fa4a4d99640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4940a2c20 0x7fa4940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.283+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4980cf0e0 con 0x7fa4a8102280 2024-04-24T23:12:40.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.283+0000 7fa49ffff640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4940a2c20 0x7fa4940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.283+0000 7fa49ffff640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4940a2c20 0x7fa4940a50e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8101960 tx=0x7fa490005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.286+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa498009de0 con 0x7fa4a8102280 2024-04-24T23:12:40.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:39 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1030824715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:39 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1252206141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:39 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1632276032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:39 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1030824715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:39 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1252206141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:39 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1632276032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:40.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.471+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4a8061b80 con 0x7fa4940a2c20 2024-04-24T23:12:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.481+0000 7fa4a4d99640 1 -- 172.21.15.33:0/3007046572 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340812 (secure 0 0 0) 0x7fa4a8061b80 con 0x7fa4940a2c20 2024-04-24T23:12:40.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:40.489 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4940a2c20 msgr2=0x7fa4940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4940a2c20 0x7fa4940a50e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8101960 tx=0x7fa490005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 msgr2=0x7fa4a8101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 0x7fa4a8101bf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d950 tx=0x7fa49800de10 comp rx=0 tx=0).stop 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 shutdown_connections 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4940a2c20 0x7fa4940a50e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a81074e0 0x7fa4a8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a8104bb0 0x7fa4a8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 --2- 172.21.15.33:0/3007046572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4a8102280 0x7fa4a8101bf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 >> 172.21.15.33:0/3007046572 conn(0x7fa4a80fbc30 msgr2=0x7fa4a8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 shutdown_connections 2024-04-24T23:12:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.484+0000 7fa4ad0a5640 1 -- 172.21.15.33:0/3007046572 wait complete. 2024-04-24T23:12:40.498 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-24T23:12:39.819005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":865,"num_read_kb":556,"num_write":353,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181934,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357588,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584236,"statfs":{"total":1151940427776,"available":1151574257664,"internally_reserved":0,"allocated":12845056,"data_stored":8230270,"data_compressed":434646,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295913+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:12:08.295913+0000","last_peered":"2024-04-24T23:12:08.295913+0000","last_clean":"2024-04-24T23:12:08.295913+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:12:08.295913+0000","last_undegraded":"2024-04-24T23:12:08.295913+0000","last_fullsized":"2024-04-24T23:12:08.295913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854282+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:12:07.854282+0000","last_peered":"2024-04-24T23:12:07.854282+0000","last_clean":"2024-04-24T23:12:07.854282+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:12:07.854282+0000","last_undegraded":"2024-04-24T23:12:07.854282+0000","last_fullsized":"2024-04-24T23:12:07.854282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854048+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:12:07.854048+0000","last_peered":"2024-04-24T23:12:07.854048+0000","last_clean":"2024-04-24T23:12:07.854048+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:12:07.854048+0000","last_undegraded":"2024-04-24T23:12:07.854048+0000","last_fullsized":"2024-04-24T23:12:07.854048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372122+0000","last_change":"2024-04-24T23:11:07.572557+0000","last_active":"2024-04-24T23:12:08.372122+0000","last_peered":"2024-04-24T23:12:08.372122+0000","last_clean":"2024-04-24T23:12:08.372122+0000","last_became_active":"2024-04-24T23:11:07.572144+0000","last_became_peered":"2024-04-24T23:11:07.572144+0000","last_unstale":"2024-04-24T23:12:08.372122+0000","last_undegraded":"2024-04-24T23:12:08.372122+0000","last_fullsized":"2024-04-24T23:12:08.372122+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:42.681992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373076+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:12:08.373076+0000","last_peered":"2024-04-24T23:12:08.373076+0000","last_clean":"2024-04-24T23:12:08.373076+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:12:08.373076+0000","last_undegraded":"2024-04-24T23:12:08.373076+0000","last_fullsized":"2024-04-24T23:12:08.373076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853784+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:12:07.853784+0000","last_peered":"2024-04-24T23:12:07.853784+0000","last_clean":"2024-04-24T23:12:07.853784+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:12:07.853784+0000","last_undegraded":"2024-04-24T23:12:07.853784+0000","last_fullsized":"2024-04-24T23:12:07.853784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372524+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:12:08.372524+0000","last_peered":"2024-04-24T23:12:08.372524+0000","last_clean":"2024-04-24T23:12:08.372524+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:12:08.372524+0000","last_undegraded":"2024-04-24T23:12:08.372524+0000","last_fullsized":"2024-04-24T23:12:08.372524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850067+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:12:07.850067+0000","last_peered":"2024-04-24T23:12:07.850067+0000","last_clean":"2024-04-24T23:12:07.850067+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:12:07.850067+0000","last_undegraded":"2024-04-24T23:12:07.850067+0000","last_fullsized":"2024-04-24T23:12:07.850067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281392+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:12:08.281392+0000","last_peered":"2024-04-24T23:12:08.281392+0000","last_clean":"2024-04-24T23:12:08.281392+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:12:08.281392+0000","last_undegraded":"2024-04-24T23:12:08.281392+0000","last_fullsized":"2024-04-24T23:12:08.281392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854555+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:12:07.854555+0000","last_peered":"2024-04-24T23:12:07.854555+0000","last_clean":"2024-04-24T23:12:07.854555+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:12:07.854555+0000","last_undegraded":"2024-04-24T23:12:07.854555+0000","last_fullsized":"2024-04-24T23:12:07.854555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373106+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:12:08.373106+0000","last_peered":"2024-04-24T23:12:08.373106+0000","last_clean":"2024-04-24T23:12:08.373106+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:12:08.373106+0000","last_undegraded":"2024-04-24T23:12:08.373106+0000","last_fullsized":"2024-04-24T23:12:08.373106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371900+0000","last_change":"2024-04-24T23:11:07.568947+0000","last_active":"2024-04-24T23:12:08.371900+0000","last_peered":"2024-04-24T23:12:08.371900+0000","last_clean":"2024-04-24T23:12:08.371900+0000","last_became_active":"2024-04-24T23:11:07.568528+0000","last_became_peered":"2024-04-24T23:11:07.568528+0000","last_unstale":"2024-04-24T23:12:08.371900+0000","last_undegraded":"2024-04-24T23:12:08.371900+0000","last_fullsized":"2024-04-24T23:12:08.371900+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:23.452961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352070+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:12:08.352070+0000","last_peered":"2024-04-24T23:12:08.352070+0000","last_clean":"2024-04-24T23:12:08.352070+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:12:08.352070+0000","last_undegraded":"2024-04-24T23:12:08.352070+0000","last_fullsized":"2024-04-24T23:12:08.352070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853752+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:12:07.853752+0000","last_peered":"2024-04-24T23:12:07.853752+0000","last_clean":"2024-04-24T23:12:07.853752+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:12:07.853752+0000","last_undegraded":"2024-04-24T23:12:07.853752+0000","last_fullsized":"2024-04-24T23:12:07.853752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853692+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:12:07.853692+0000","last_peered":"2024-04-24T23:12:07.853692+0000","last_clean":"2024-04-24T23:12:07.853692+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:12:07.853692+0000","last_undegraded":"2024-04-24T23:12:07.853692+0000","last_fullsized":"2024-04-24T23:12:07.853692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280637+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:12:08.280637+0000","last_peered":"2024-04-24T23:12:08.280637+0000","last_clean":"2024-04-24T23:12:08.280637+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:12:08.280637+0000","last_undegraded":"2024-04-24T23:12:08.280637+0000","last_fullsized":"2024-04-24T23:12:08.280637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295782+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:12:08.295782+0000","last_peered":"2024-04-24T23:12:08.295782+0000","last_clean":"2024-04-24T23:12:08.295782+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:12:08.295782+0000","last_undegraded":"2024-04-24T23:12:08.295782+0000","last_fullsized":"2024-04-24T23:12:08.295782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372278+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:12:08.372278+0000","last_peered":"2024-04-24T23:12:08.372278+0000","last_clean":"2024-04-24T23:12:08.372278+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:12:08.372278+0000","last_undegraded":"2024-04-24T23:12:08.372278+0000","last_fullsized":"2024-04-24T23:12:08.372278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855452+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:12:07.855452+0000","last_peered":"2024-04-24T23:12:07.855452+0000","last_clean":"2024-04-24T23:12:07.855452+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:12:07.855452+0000","last_undegraded":"2024-04-24T23:12:07.855452+0000","last_fullsized":"2024-04-24T23:12:07.855452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855027+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:12:07.855027+0000","last_peered":"2024-04-24T23:12:07.855027+0000","last_clean":"2024-04-24T23:12:07.855027+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:12:07.855027+0000","last_undegraded":"2024-04-24T23:12:07.855027+0000","last_fullsized":"2024-04-24T23:12:07.855027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372315+0000","last_change":"2024-04-24T23:11:07.769518+0000","last_active":"2024-04-24T23:12:08.372315+0000","last_peered":"2024-04-24T23:12:08.372315+0000","last_clean":"2024-04-24T23:12:08.372315+0000","last_became_active":"2024-04-24T23:11:07.763040+0000","last_became_peered":"2024-04-24T23:11:07.763040+0000","last_unstale":"2024-04-24T23:12:08.372315+0000","last_undegraded":"2024-04-24T23:12:08.372315+0000","last_fullsized":"2024-04-24T23:12:08.372315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.056568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850728+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:12:07.850728+0000","last_peered":"2024-04-24T23:12:07.850728+0000","last_clean":"2024-04-24T23:12:07.850728+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:12:07.850728+0000","last_undegraded":"2024-04-24T23:12:07.850728+0000","last_fullsized":"2024-04-24T23:12:07.850728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192411+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:12:08.192411+0000","last_peered":"2024-04-24T23:12:08.192411+0000","last_clean":"2024-04-24T23:12:08.192411+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:12:08.192411+0000","last_undegraded":"2024-04-24T23:12:08.192411+0000","last_fullsized":"2024-04-24T23:12:08.192411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850867+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:12:07.850867+0000","last_peered":"2024-04-24T23:12:07.850867+0000","last_clean":"2024-04-24T23:12:07.850867+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:12:07.850867+0000","last_undegraded":"2024-04-24T23:12:07.850867+0000","last_fullsized":"2024-04-24T23:12:07.850867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854113+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:12:07.854113+0000","last_peered":"2024-04-24T23:12:07.854113+0000","last_clean":"2024-04-24T23:12:07.854113+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:12:07.854113+0000","last_undegraded":"2024-04-24T23:12:07.854113+0000","last_fullsized":"2024-04-24T23:12:07.854113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192900+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:12:08.192900+0000","last_peered":"2024-04-24T23:12:08.192900+0000","last_clean":"2024-04-24T23:12:08.192900+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:12:08.192900+0000","last_undegraded":"2024-04-24T23:12:08.192900+0000","last_fullsized":"2024-04-24T23:12:08.192900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295762+0000","last_change":"2024-04-24T23:11:07.583051+0000","last_active":"2024-04-24T23:12:08.295762+0000","last_peered":"2024-04-24T23:12:08.295762+0000","last_clean":"2024-04-24T23:12:08.295762+0000","last_became_active":"2024-04-24T23:11:07.572502+0000","last_became_peered":"2024-04-24T23:11:07.572502+0000","last_unstale":"2024-04-24T23:12:08.295762+0000","last_undegraded":"2024-04-24T23:12:08.295762+0000","last_fullsized":"2024-04-24T23:12:08.295762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:21.581198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280792+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:12:08.280792+0000","last_peered":"2024-04-24T23:12:08.280792+0000","last_clean":"2024-04-24T23:12:08.280792+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:12:08.280792+0000","last_undegraded":"2024-04-24T23:12:08.280792+0000","last_fullsized":"2024-04-24T23:12:08.280792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192430+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:12:08.192430+0000","last_peered":"2024-04-24T23:12:08.192430+0000","last_clean":"2024-04-24T23:12:08.192430+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:12:08.192430+0000","last_undegraded":"2024-04-24T23:12:08.192430+0000","last_fullsized":"2024-04-24T23:12:08.192430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373278+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:12:08.373278+0000","last_peered":"2024-04-24T23:12:08.373278+0000","last_clean":"2024-04-24T23:12:08.373278+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:12:08.373278+0000","last_undegraded":"2024-04-24T23:12:08.373278+0000","last_fullsized":"2024-04-24T23:12:08.373278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372321+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:12:08.372321+0000","last_peered":"2024-04-24T23:12:08.372321+0000","last_clean":"2024-04-24T23:12:08.372321+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:12:08.372321+0000","last_undegraded":"2024-04-24T23:12:08.372321+0000","last_fullsized":"2024-04-24T23:12:08.372321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855307+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:12:07.855307+0000","last_peered":"2024-04-24T23:12:07.855307+0000","last_clean":"2024-04-24T23:12:07.855307+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:12:07.855307+0000","last_undegraded":"2024-04-24T23:12:07.855307+0000","last_fullsized":"2024-04-24T23:12:07.855307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.679678+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:12:33.679678+0000","last_peered":"2024-04-24T23:12:33.679678+0000","last_clean":"2024-04-24T23:12:33.679678+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:12:33.679678+0000","last_undegraded":"2024-04-24T23:12:33.679678+0000","last_fullsized":"2024-04-24T23:12:33.679678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373431+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:12:08.373431+0000","last_peered":"2024-04-24T23:12:08.373431+0000","last_clean":"2024-04-24T23:12:08.373431+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:12:08.373431+0000","last_undegraded":"2024-04-24T23:12:08.373431+0000","last_fullsized":"2024-04-24T23:12:08.373431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855174+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:12:07.855174+0000","last_peered":"2024-04-24T23:12:07.855174+0000","last_clean":"2024-04-24T23:12:07.855174+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:12:07.855174+0000","last_undegraded":"2024-04-24T23:12:07.855174+0000","last_fullsized":"2024-04-24T23:12:07.855174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853715+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:12:07.853715+0000","last_peered":"2024-04-24T23:12:07.853715+0000","last_clean":"2024-04-24T23:12:07.853715+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:12:07.853715+0000","last_undegraded":"2024-04-24T23:12:07.853715+0000","last_fullsized":"2024-04-24T23:12:07.853715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372610+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:12:08.372610+0000","last_peered":"2024-04-24T23:12:08.372610+0000","last_clean":"2024-04-24T23:12:08.372610+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:12:08.372610+0000","last_undegraded":"2024-04-24T23:12:08.372610+0000","last_fullsized":"2024-04-24T23:12:08.372610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295780+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:12:08.295780+0000","last_peered":"2024-04-24T23:12:08.295780+0000","last_clean":"2024-04-24T23:12:08.295780+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:12:08.295780+0000","last_undegraded":"2024-04-24T23:12:08.295780+0000","last_fullsized":"2024-04-24T23:12:08.295780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.351967+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:12:08.351967+0000","last_peered":"2024-04-24T23:12:08.351967+0000","last_clean":"2024-04-24T23:12:08.351967+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:12:08.351967+0000","last_undegraded":"2024-04-24T23:12:08.351967+0000","last_fullsized":"2024-04-24T23:12:08.351967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.849936+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:12:07.849936+0000","last_peered":"2024-04-24T23:12:07.849936+0000","last_clean":"2024-04-24T23:12:07.849936+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:12:07.849936+0000","last_undegraded":"2024-04-24T23:12:07.849936+0000","last_fullsized":"2024-04-24T23:12:07.849936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854277+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:12:07.854277+0000","last_peered":"2024-04-24T23:12:07.854277+0000","last_clean":"2024-04-24T23:12:07.854277+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:12:07.854277+0000","last_undegraded":"2024-04-24T23:12:07.854277+0000","last_fullsized":"2024-04-24T23:12:07.854277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371822+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:12:08.371822+0000","last_peered":"2024-04-24T23:12:08.371822+0000","last_clean":"2024-04-24T23:12:08.371822+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:12:08.371822+0000","last_undegraded":"2024-04-24T23:12:08.371822+0000","last_fullsized":"2024-04-24T23:12:08.371822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371791+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:12:08.371791+0000","last_peered":"2024-04-24T23:12:08.371791+0000","last_clean":"2024-04-24T23:12:08.371791+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:12:08.371791+0000","last_undegraded":"2024-04-24T23:12:08.371791+0000","last_fullsized":"2024-04-24T23:12:08.371791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850098+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:12:07.850098+0000","last_peered":"2024-04-24T23:12:07.850098+0000","last_clean":"2024-04-24T23:12:07.850098+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:12:07.850098+0000","last_undegraded":"2024-04-24T23:12:07.850098+0000","last_fullsized":"2024-04-24T23:12:07.850098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:36.684142+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:12:36.684142+0000","last_peered":"2024-04-24T23:12:36.684142+0000","last_clean":"2024-04-24T23:12:36.684142+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:12:36.684142+0000","last_undegraded":"2024-04-24T23:12:36.684142+0000","last_fullsized":"2024-04-24T23:12:36.684142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281321+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:12:08.281321+0000","last_peered":"2024-04-24T23:12:08.281321+0000","last_clean":"2024-04-24T23:12:08.281321+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:12:08.281321+0000","last_undegraded":"2024-04-24T23:12:08.281321+0000","last_fullsized":"2024-04-24T23:12:08.281321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371692+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:12:08.371692+0000","last_peered":"2024-04-24T23:12:08.371692+0000","last_clean":"2024-04-24T23:12:08.371692+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:12:08.371692+0000","last_undegraded":"2024-04-24T23:12:08.371692+0000","last_fullsized":"2024-04-24T23:12:08.371692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373045+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:12:08.373045+0000","last_peered":"2024-04-24T23:12:08.373045+0000","last_clean":"2024-04-24T23:12:08.373045+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:12:08.373045+0000","last_undegraded":"2024-04-24T23:12:08.373045+0000","last_fullsized":"2024-04-24T23:12:08.373045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854248+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:12:07.854248+0000","last_peered":"2024-04-24T23:12:07.854248+0000","last_clean":"2024-04-24T23:12:07.854248+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:12:07.854248+0000","last_undegraded":"2024-04-24T23:12:07.854248+0000","last_fullsized":"2024-04-24T23:12:07.854248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193177+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:12:08.193177+0000","last_peered":"2024-04-24T23:12:08.193177+0000","last_clean":"2024-04-24T23:12:08.193177+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:12:08.193177+0000","last_undegraded":"2024-04-24T23:12:08.193177+0000","last_fullsized":"2024-04-24T23:12:08.193177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372194+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:12:08.372194+0000","last_peered":"2024-04-24T23:12:08.372194+0000","last_clean":"2024-04-24T23:12:08.372194+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:12:08.372194+0000","last_undegraded":"2024-04-24T23:12:08.372194+0000","last_fullsized":"2024-04-24T23:12:08.372194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281176+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:12:08.281176+0000","last_peered":"2024-04-24T23:12:08.281176+0000","last_clean":"2024-04-24T23:12:08.281176+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:12:08.281176+0000","last_undegraded":"2024-04-24T23:12:08.281176+0000","last_fullsized":"2024-04-24T23:12:08.281176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371789+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:12:08.371789+0000","last_peered":"2024-04-24T23:12:08.371789+0000","last_clean":"2024-04-24T23:12:08.371789+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:12:08.371789+0000","last_undegraded":"2024-04-24T23:12:08.371789+0000","last_fullsized":"2024-04-24T23:12:08.371789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'37","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:36.667224+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:12:36.667224+0000","last_peered":"2024-04-24T23:12:36.667224+0000","last_clean":"2024-04-24T23:12:36.667224+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:12:36.667224+0000","last_undegraded":"2024-04-24T23:12:36.667224+0000","last_fullsized":"2024-04-24T23:12:36.667224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":248,"num_read_kb":191,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373386+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:12:08.373386+0000","last_peered":"2024-04-24T23:12:08.373386+0000","last_clean":"2024-04-24T23:12:08.373386+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:12:08.373386+0000","last_undegraded":"2024-04-24T23:12:08.373386+0000","last_fullsized":"2024-04-24T23:12:08.373386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.375570+0000","last_change":"2024-04-24T23:12:08.375043+0000","last_active":"2024-04-24T23:12:08.375570+0000","last_peered":"2024-04-24T23:12:08.375570+0000","last_clean":"2024-04-24T23:12:08.375570+0000","last_became_active":"2024-04-24T23:12:08.374204+0000","last_became_peered":"2024-04-24T23:12:08.374204+0000","last_unstale":"2024-04-24T23:12:08.375570+0000","last_undegraded":"2024-04-24T23:12:08.375570+0000","last_fullsized":"2024-04-24T23:12:08.375570+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:36.596328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295494+0000","last_change":"2024-04-24T23:11:07.579273+0000","last_active":"2024-04-24T23:12:08.295494+0000","last_peered":"2024-04-24T23:12:08.295494+0000","last_clean":"2024-04-24T23:12:08.295494+0000","last_became_active":"2024-04-24T23:11:07.570480+0000","last_became_peered":"2024-04-24T23:11:07.570480+0000","last_unstale":"2024-04-24T23:12:08.295494+0000","last_undegraded":"2024-04-24T23:12:08.295494+0000","last_fullsized":"2024-04-24T23:12:08.295494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:21.291453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352055+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:12:08.352055+0000","last_peered":"2024-04-24T23:12:08.352055+0000","last_clean":"2024-04-24T23:12:08.352055+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:12:08.352055+0000","last_undegraded":"2024-04-24T23:12:08.352055+0000","last_fullsized":"2024-04-24T23:12:08.352055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295559+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:12:08.295559+0000","last_peered":"2024-04-24T23:12:08.295559+0000","last_clean":"2024-04-24T23:12:08.295559+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:12:08.295559+0000","last_undegraded":"2024-04-24T23:12:08.295559+0000","last_fullsized":"2024-04-24T23:12:08.295559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373038+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:12:08.373038+0000","last_peered":"2024-04-24T23:12:08.373038+0000","last_clean":"2024-04-24T23:12:08.373038+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:12:08.373038+0000","last_undegraded":"2024-04-24T23:12:08.373038+0000","last_fullsized":"2024-04-24T23:12:08.373038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373266+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:12:08.373266+0000","last_peered":"2024-04-24T23:12:08.373266+0000","last_clean":"2024-04-24T23:12:08.373266+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:12:08.373266+0000","last_undegraded":"2024-04-24T23:12:08.373266+0000","last_fullsized":"2024-04-24T23:12:08.373266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372070+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:12:08.372070+0000","last_peered":"2024-04-24T23:12:08.372070+0000","last_clean":"2024-04-24T23:12:08.372070+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:12:08.372070+0000","last_undegraded":"2024-04-24T23:12:08.372070+0000","last_fullsized":"2024-04-24T23:12:08.372070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:34.651637+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:12:34.651637+0000","last_peered":"2024-04-24T23:12:34.651637+0000","last_clean":"2024-04-24T23:12:34.651637+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:12:34.651637+0000","last_undegraded":"2024-04-24T23:12:34.651637+0000","last_fullsized":"2024-04-24T23:12:34.651637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372711+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:12:08.372711+0000","last_peered":"2024-04-24T23:12:08.372711+0000","last_clean":"2024-04-24T23:12:08.372711+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:12:08.372711+0000","last_undegraded":"2024-04-24T23:12:08.372711+0000","last_fullsized":"2024-04-24T23:12:08.372711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352276+0000","last_change":"2024-04-24T23:11:07.570768+0000","last_active":"2024-04-24T23:12:08.352276+0000","last_peered":"2024-04-24T23:12:08.352276+0000","last_clean":"2024-04-24T23:12:08.352276+0000","last_became_active":"2024-04-24T23:11:07.570306+0000","last_became_peered":"2024-04-24T23:11:07.570306+0000","last_unstale":"2024-04-24T23:12:08.352276+0000","last_undegraded":"2024-04-24T23:12:08.352276+0000","last_fullsized":"2024-04-24T23:12:08.352276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:10.450660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373058+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:12:08.373058+0000","last_peered":"2024-04-24T23:12:08.373058+0000","last_clean":"2024-04-24T23:12:08.373058+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:12:08.373058+0000","last_undegraded":"2024-04-24T23:12:08.373058+0000","last_fullsized":"2024-04-24T23:12:08.373058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371872+0000","last_change":"2024-04-24T23:11:07.762601+0000","last_active":"2024-04-24T23:12:08.371872+0000","last_peered":"2024-04-24T23:12:08.371872+0000","last_clean":"2024-04-24T23:12:08.371872+0000","last_became_active":"2024-04-24T23:11:07.762305+0000","last_became_peered":"2024-04-24T23:11:07.762305+0000","last_unstale":"2024-04-24T23:12:08.371872+0000","last_undegraded":"2024-04-24T23:12:08.371872+0000","last_fullsized":"2024-04-24T23:12:08.371872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:36.209046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372017+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:12:08.372017+0000","last_peered":"2024-04-24T23:12:08.372017+0000","last_clean":"2024-04-24T23:12:08.372017+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:12:08.372017+0000","last_undegraded":"2024-04-24T23:12:08.372017+0000","last_fullsized":"2024-04-24T23:12:08.372017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373271+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:12:08.373271+0000","last_peered":"2024-04-24T23:12:08.373271+0000","last_clean":"2024-04-24T23:12:08.373271+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:12:08.373271+0000","last_undegraded":"2024-04-24T23:12:08.373271+0000","last_fullsized":"2024-04-24T23:12:08.373271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295800+0000","last_change":"2024-04-24T23:11:07.569318+0000","last_active":"2024-04-24T23:12:08.295800+0000","last_peered":"2024-04-24T23:12:08.295800+0000","last_clean":"2024-04-24T23:12:08.295800+0000","last_became_active":"2024-04-24T23:11:07.568799+0000","last_became_peered":"2024-04-24T23:11:07.568799+0000","last_unstale":"2024-04-24T23:12:08.295800+0000","last_undegraded":"2024-04-24T23:12:08.295800+0000","last_fullsized":"2024-04-24T23:12:08.295800+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:40.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373085+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:12:08.373085+0000","last_peered":"2024-04-24T23:12:08.373085+0000","last_clean":"2024-04-24T23:12:08.373085+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:12:08.373085+0000","last_undegraded":"2024-04-24T23:12:08.373085+0000","last_fullsized":"2024-04-24T23:12:08.373085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372189+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:12:08.372189+0000","last_peered":"2024-04-24T23:12:08.372189+0000","last_clean":"2024-04-24T23:12:08.372189+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:12:08.372189+0000","last_undegraded":"2024-04-24T23:12:08.372189+0000","last_fullsized":"2024-04-24T23:12:08.372189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854317+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:12:07.854317+0000","last_peered":"2024-04-24T23:12:07.854317+0000","last_clean":"2024-04-24T23:12:07.854317+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:12:07.854317+0000","last_undegraded":"2024-04-24T23:12:07.854317+0000","last_fullsized":"2024-04-24T23:12:07.854317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352055+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:12:08.352055+0000","last_peered":"2024-04-24T23:12:08.352055+0000","last_clean":"2024-04-24T23:12:08.352055+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:12:08.352055+0000","last_undegraded":"2024-04-24T23:12:08.352055+0000","last_fullsized":"2024-04-24T23:12:08.352055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372748+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:12:08.372748+0000","last_peered":"2024-04-24T23:12:08.372748+0000","last_clean":"2024-04-24T23:12:08.372748+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:12:08.372748+0000","last_undegraded":"2024-04-24T23:12:08.372748+0000","last_fullsized":"2024-04-24T23:12:08.372748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853997+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:12:07.853997+0000","last_peered":"2024-04-24T23:12:07.853997+0000","last_clean":"2024-04-24T23:12:07.853997+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:12:07.853997+0000","last_undegraded":"2024-04-24T23:12:07.853997+0000","last_fullsized":"2024-04-24T23:12:07.853997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352356+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:12:08.352356+0000","last_peered":"2024-04-24T23:12:08.352356+0000","last_clean":"2024-04-24T23:12:08.352356+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:12:08.352356+0000","last_undegraded":"2024-04-24T23:12:08.352356+0000","last_fullsized":"2024-04-24T23:12:08.352356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372205+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:12:08.372205+0000","last_peered":"2024-04-24T23:12:08.372205+0000","last_clean":"2024-04-24T23:12:08.372205+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:12:08.372205+0000","last_undegraded":"2024-04-24T23:12:08.372205+0000","last_fullsized":"2024-04-24T23:12:08.372205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:34.650960+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:12:34.650960+0000","last_peered":"2024-04-24T23:12:34.650960+0000","last_clean":"2024-04-24T23:12:34.650960+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:12:34.650960+0000","last_undegraded":"2024-04-24T23:12:34.650960+0000","last_fullsized":"2024-04-24T23:12:34.650960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372870+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:12:08.372870+0000","last_peered":"2024-04-24T23:12:08.372870+0000","last_clean":"2024-04-24T23:12:08.372870+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:12:08.372870+0000","last_undegraded":"2024-04-24T23:12:08.372870+0000","last_fullsized":"2024-04-24T23:12:08.372870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850177+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:12:07.850177+0000","last_peered":"2024-04-24T23:12:07.850177+0000","last_clean":"2024-04-24T23:12:07.850177+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:12:07.850177+0000","last_undegraded":"2024-04-24T23:12:07.850177+0000","last_fullsized":"2024-04-24T23:12:07.850177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193019+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:12:08.193019+0000","last_peered":"2024-04-24T23:12:08.193019+0000","last_clean":"2024-04-24T23:12:08.193019+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:12:08.193019+0000","last_undegraded":"2024-04-24T23:12:08.193019+0000","last_fullsized":"2024-04-24T23:12:08.193019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372433+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:12:08.372433+0000","last_peered":"2024-04-24T23:12:08.372433+0000","last_clean":"2024-04-24T23:12:08.372433+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:12:08.372433+0000","last_undegraded":"2024-04-24T23:12:08.372433+0000","last_fullsized":"2024-04-24T23:12:08.372433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372229+0000","last_change":"2024-04-24T23:11:07.571671+0000","last_active":"2024-04-24T23:12:08.372229+0000","last_peered":"2024-04-24T23:12:08.372229+0000","last_clean":"2024-04-24T23:12:08.372229+0000","last_became_active":"2024-04-24T23:11:07.571082+0000","last_became_peered":"2024-04-24T23:11:07.571082+0000","last_unstale":"2024-04-24T23:12:08.372229+0000","last_undegraded":"2024-04-24T23:12:08.372229+0000","last_fullsized":"2024-04-24T23:12:08.372229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:11.106931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.351810+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:12:08.351810+0000","last_peered":"2024-04-24T23:12:08.351810+0000","last_clean":"2024-04-24T23:12:08.351810+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:12:08.351810+0000","last_undegraded":"2024-04-24T23:12:08.351810+0000","last_fullsized":"2024-04-24T23:12:08.351810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372678+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:12:08.372678+0000","last_peered":"2024-04-24T23:12:08.372678+0000","last_clean":"2024-04-24T23:12:08.372678+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:12:08.372678+0000","last_undegraded":"2024-04-24T23:12:08.372678+0000","last_fullsized":"2024-04-24T23:12:08.372678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295482+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:12:08.295482+0000","last_peered":"2024-04-24T23:12:08.295482+0000","last_clean":"2024-04-24T23:12:08.295482+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:12:08.295482+0000","last_undegraded":"2024-04-24T23:12:08.295482+0000","last_fullsized":"2024-04-24T23:12:08.295482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373489+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:12:08.373489+0000","last_peered":"2024-04-24T23:12:08.373489+0000","last_clean":"2024-04-24T23:12:08.373489+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:12:08.373489+0000","last_undegraded":"2024-04-24T23:12:08.373489+0000","last_fullsized":"2024-04-24T23:12:08.373489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352204+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:12:08.352204+0000","last_peered":"2024-04-24T23:12:08.352204+0000","last_clean":"2024-04-24T23:12:08.352204+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:12:08.352204+0000","last_undegraded":"2024-04-24T23:12:08.352204+0000","last_fullsized":"2024-04-24T23:12:08.352204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850006+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:12:07.850006+0000","last_peered":"2024-04-24T23:12:07.850006+0000","last_clean":"2024-04-24T23:12:07.850006+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:12:07.850006+0000","last_undegraded":"2024-04-24T23:12:07.850006+0000","last_fullsized":"2024-04-24T23:12:07.850006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372243+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:12:08.372243+0000","last_peered":"2024-04-24T23:12:08.372243+0000","last_clean":"2024-04-24T23:12:08.372243+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:12:08.372243+0000","last_undegraded":"2024-04-24T23:12:08.372243+0000","last_fullsized":"2024-04-24T23:12:08.372243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850752+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:12:07.850752+0000","last_peered":"2024-04-24T23:12:07.850752+0000","last_clean":"2024-04-24T23:12:07.850752+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:12:07.850752+0000","last_undegraded":"2024-04-24T23:12:07.850752+0000","last_fullsized":"2024-04-24T23:12:07.850752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854907+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:12:07.854907+0000","last_peered":"2024-04-24T23:12:07.854907+0000","last_clean":"2024-04-24T23:12:07.854907+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:12:07.854907+0000","last_undegraded":"2024-04-24T23:12:07.854907+0000","last_fullsized":"2024-04-24T23:12:07.854907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193280+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:12:08.193280+0000","last_peered":"2024-04-24T23:12:08.193280+0000","last_clean":"2024-04-24T23:12:08.193280+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:12:08.193280+0000","last_undegraded":"2024-04-24T23:12:08.193280+0000","last_fullsized":"2024-04-24T23:12:08.193280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854403+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:12:07.854403+0000","last_peered":"2024-04-24T23:12:07.854403+0000","last_clean":"2024-04-24T23:12:07.854403+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:12:07.854403+0000","last_undegraded":"2024-04-24T23:12:07.854403+0000","last_fullsized":"2024-04-24T23:12:07.854403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192900+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:12:08.192900+0000","last_peered":"2024-04-24T23:12:08.192900+0000","last_clean":"2024-04-24T23:12:08.192900+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:12:08.192900+0000","last_undegraded":"2024-04-24T23:12:08.192900+0000","last_fullsized":"2024-04-24T23:12:08.192900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280633+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:12:08.280633+0000","last_peered":"2024-04-24T23:12:08.280633+0000","last_clean":"2024-04-24T23:12:08.280633+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:12:08.280633+0000","last_undegraded":"2024-04-24T23:12:08.280633+0000","last_fullsized":"2024-04-24T23:12:08.280633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373314+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:12:08.373314+0000","last_peered":"2024-04-24T23:12:08.373314+0000","last_clean":"2024-04-24T23:12:08.373314+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:12:08.373314+0000","last_undegraded":"2024-04-24T23:12:08.373314+0000","last_fullsized":"2024-04-24T23:12:08.373314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372313+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:12:08.372313+0000","last_peered":"2024-04-24T23:12:08.372313+0000","last_clean":"2024-04-24T23:12:08.372313+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:12:08.372313+0000","last_undegraded":"2024-04-24T23:12:08.372313+0000","last_fullsized":"2024-04-24T23:12:08.372313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850177+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:12:07.850177+0000","last_peered":"2024-04-24T23:12:07.850177+0000","last_clean":"2024-04-24T23:12:07.850177+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:12:07.850177+0000","last_undegraded":"2024-04-24T23:12:07.850177+0000","last_fullsized":"2024-04-24T23:12:07.850177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295519+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:12:08.295519+0000","last_peered":"2024-04-24T23:12:08.295519+0000","last_clean":"2024-04-24T23:12:08.295519+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:12:08.295519+0000","last_undegraded":"2024-04-24T23:12:08.295519+0000","last_fullsized":"2024-04-24T23:12:08.295519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192876+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:12:08.192876+0000","last_peered":"2024-04-24T23:12:08.192876+0000","last_clean":"2024-04-24T23:12:08.192876+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:12:08.192876+0000","last_undegraded":"2024-04-24T23:12:08.192876+0000","last_fullsized":"2024-04-24T23:12:08.192876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855100+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:12:07.855100+0000","last_peered":"2024-04-24T23:12:07.855100+0000","last_clean":"2024-04-24T23:12:07.855100+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:12:07.855100+0000","last_undegraded":"2024-04-24T23:12:07.855100+0000","last_fullsized":"2024-04-24T23:12:07.855100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850423+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:12:07.850423+0000","last_peered":"2024-04-24T23:12:07.850423+0000","last_clean":"2024-04-24T23:12:07.850423+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:12:07.850423+0000","last_undegraded":"2024-04-24T23:12:07.850423+0000","last_fullsized":"2024-04-24T23:12:07.850423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192912+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:12:08.192912+0000","last_peered":"2024-04-24T23:12:08.192912+0000","last_clean":"2024-04-24T23:12:08.192912+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:12:08.192912+0000","last_undegraded":"2024-04-24T23:12:08.192912+0000","last_fullsized":"2024-04-24T23:12:08.192912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372997+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:12:08.372997+0000","last_peered":"2024-04-24T23:12:08.372997+0000","last_clean":"2024-04-24T23:12:08.372997+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:12:08.372997+0000","last_undegraded":"2024-04-24T23:12:08.372997+0000","last_fullsized":"2024-04-24T23:12:08.372997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372923+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:12:08.372923+0000","last_peered":"2024-04-24T23:12:08.372923+0000","last_clean":"2024-04-24T23:12:08.372923+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:12:08.372923+0000","last_undegraded":"2024-04-24T23:12:08.372923+0000","last_fullsized":"2024-04-24T23:12:08.372923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280525+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:12:08.280525+0000","last_peered":"2024-04-24T23:12:08.280525+0000","last_clean":"2024-04-24T23:12:08.280525+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:12:08.280525+0000","last_undegraded":"2024-04-24T23:12:08.280525+0000","last_fullsized":"2024-04-24T23:12:08.280525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280781+0000","last_change":"2024-04-24T23:11:07.579626+0000","last_active":"2024-04-24T23:12:08.280781+0000","last_peered":"2024-04-24T23:12:08.280781+0000","last_clean":"2024-04-24T23:12:08.280781+0000","last_became_active":"2024-04-24T23:11:07.574907+0000","last_became_peered":"2024-04-24T23:11:07.574907+0000","last_unstale":"2024-04-24T23:12:08.280781+0000","last_undegraded":"2024-04-24T23:12:08.280781+0000","last_fullsized":"2024-04-24T23:12:08.280781+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:56.428190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373144+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:12:08.373144+0000","last_peered":"2024-04-24T23:12:08.373144+0000","last_clean":"2024-04-24T23:12:08.373144+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:12:08.373144+0000","last_undegraded":"2024-04-24T23:12:08.373144+0000","last_fullsized":"2024-04-24T23:12:08.373144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.375191+0000","last_change":"2024-04-24T23:12:08.374812+0000","last_active":"2024-04-24T23:12:08.375191+0000","last_peered":"2024-04-24T23:12:08.375191+0000","last_clean":"2024-04-24T23:12:08.375191+0000","last_became_active":"2024-04-24T23:12:08.374353+0000","last_became_peered":"2024-04-24T23:12:08.374353+0000","last_unstale":"2024-04-24T23:12:08.375191+0000","last_undegraded":"2024-04-24T23:12:08.375191+0000","last_fullsized":"2024-04-24T23:12:08.375191+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:58.241681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372501+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:12:08.372501+0000","last_peered":"2024-04-24T23:12:08.372501+0000","last_clean":"2024-04-24T23:12:08.372501+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:12:08.372501+0000","last_undegraded":"2024-04-24T23:12:08.372501+0000","last_fullsized":"2024-04-24T23:12:08.372501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373096+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:12:08.373096+0000","last_peered":"2024-04-24T23:12:08.373096+0000","last_clean":"2024-04-24T23:12:08.373096+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:12:08.373096+0000","last_undegraded":"2024-04-24T23:12:08.373096+0000","last_fullsized":"2024-04-24T23:12:08.373096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371788+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:12:08.371788+0000","last_peered":"2024-04-24T23:12:08.371788+0000","last_clean":"2024-04-24T23:12:08.371788+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:12:08.371788+0000","last_undegraded":"2024-04-24T23:12:08.371788+0000","last_fullsized":"2024-04-24T23:12:08.371788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352331+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:12:08.352331+0000","last_peered":"2024-04-24T23:12:08.352331+0000","last_clean":"2024-04-24T23:12:08.352331+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:12:08.352331+0000","last_undegraded":"2024-04-24T23:12:08.352331+0000","last_fullsized":"2024-04-24T23:12:08.352331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373318+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:12:08.373318+0000","last_peered":"2024-04-24T23:12:08.373318+0000","last_clean":"2024-04-24T23:12:08.373318+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:12:08.373318+0000","last_undegraded":"2024-04-24T23:12:08.373318+0000","last_fullsized":"2024-04-24T23:12:08.373318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295479+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:12:08.295479+0000","last_peered":"2024-04-24T23:12:08.295479+0000","last_clean":"2024-04-24T23:12:08.295479+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:12:08.295479+0000","last_undegraded":"2024-04-24T23:12:08.295479+0000","last_fullsized":"2024-04-24T23:12:08.295479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855299+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:12:07.855299+0000","last_peered":"2024-04-24T23:12:07.855299+0000","last_clean":"2024-04-24T23:12:07.855299+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:12:07.855299+0000","last_undegraded":"2024-04-24T23:12:07.855299+0000","last_fullsized":"2024-04-24T23:12:07.855299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854984+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:12:07.854984+0000","last_peered":"2024-04-24T23:12:07.854984+0000","last_clean":"2024-04-24T23:12:07.854984+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:12:07.854984+0000","last_undegraded":"2024-04-24T23:12:07.854984+0000","last_fullsized":"2024-04-24T23:12:07.854984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853900+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:12:07.853900+0000","last_peered":"2024-04-24T23:12:07.853900+0000","last_clean":"2024-04-24T23:12:07.853900+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:12:07.853900+0000","last_undegraded":"2024-04-24T23:12:07.853900+0000","last_fullsized":"2024-04-24T23:12:07.853900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352229+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:12:08.352229+0000","last_peered":"2024-04-24T23:12:08.352229+0000","last_clean":"2024-04-24T23:12:08.352229+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:12:08.352229+0000","last_undegraded":"2024-04-24T23:12:08.352229+0000","last_fullsized":"2024-04-24T23:12:08.352229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295817+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:12:08.295817+0000","last_peered":"2024-04-24T23:12:08.295817+0000","last_clean":"2024-04-24T23:12:08.295817+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:12:08.295817+0000","last_undegraded":"2024-04-24T23:12:08.295817+0000","last_fullsized":"2024-04-24T23:12:08.295817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855547+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:12:07.855547+0000","last_peered":"2024-04-24T23:12:07.855547+0000","last_clean":"2024-04-24T23:12:07.855547+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:12:07.855547+0000","last_undegraded":"2024-04-24T23:12:07.855547+0000","last_fullsized":"2024-04-24T23:12:07.855547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850816+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:12:07.850816+0000","last_peered":"2024-04-24T23:12:07.850816+0000","last_clean":"2024-04-24T23:12:07.850816+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:12:07.850816+0000","last_undegraded":"2024-04-24T23:12:07.850816+0000","last_fullsized":"2024-04-24T23:12:07.850816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371911+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:12:08.371911+0000","last_peered":"2024-04-24T23:12:08.371911+0000","last_clean":"2024-04-24T23:12:08.371911+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:12:08.371911+0000","last_undegraded":"2024-04-24T23:12:08.371911+0000","last_fullsized":"2024-04-24T23:12:08.371911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850318+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:12:07.850318+0000","last_peered":"2024-04-24T23:12:07.850318+0000","last_clean":"2024-04-24T23:12:07.850318+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:12:07.850318+0000","last_undegraded":"2024-04-24T23:12:07.850318+0000","last_fullsized":"2024-04-24T23:12:07.850318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280809+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:12:08.280809+0000","last_peered":"2024-04-24T23:12:08.280809+0000","last_clean":"2024-04-24T23:12:08.280809+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:12:08.280809+0000","last_undegraded":"2024-04-24T23:12:08.280809+0000","last_fullsized":"2024-04-24T23:12:08.280809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352027+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:12:08.352027+0000","last_peered":"2024-04-24T23:12:08.352027+0000","last_clean":"2024-04-24T23:12:08.352027+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:12:08.352027+0000","last_undegraded":"2024-04-24T23:12:08.352027+0000","last_fullsized":"2024-04-24T23:12:08.352027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295519+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:12:08.295519+0000","last_peered":"2024-04-24T23:12:08.295519+0000","last_clean":"2024-04-24T23:12:08.295519+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:12:08.295519+0000","last_undegraded":"2024-04-24T23:12:08.295519+0000","last_fullsized":"2024-04-24T23:12:08.295519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":739,"num_read_kb":447,"num_write":170,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1198,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":213861,"data_compressed":4821,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1178528,"data_compressed":64827,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":526964,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":522206,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1118838,"data_compressed":63220,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":518044,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":586632,"data_compressed":28292,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":581020,"data_compressed":28292,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":518238,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":523808,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1113894,"data_compressed":63220,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71198,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63066,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70206,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:40.503 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-24T23:12:40.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:12:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.639+0000 7fcb1ec41640 1 -- 172.21.15.33:0/4090164178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18101790 msgr2=0x7fcb18101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.639+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/4090164178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18101790 0x7fcb18101bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009900 tx=0x7fcb1402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.639+0000 7fcb1ec41640 1 -- 172.21.15.33:0/4090164178 shutdown_connections 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.639+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/4090164178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb18103340 0x7fcb18109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.639+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/4090164178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb18102980 0x7fcb18102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/4090164178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18101790 0x7fcb18101bb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- 172.21.15.33:0/4090164178 >> 172.21.15.33:0/4090164178 conn(0x7fcb180fcd40 msgr2=0x7fcb180ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- 172.21.15.33:0/4090164178 shutdown_connections 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- 172.21.15.33:0/4090164178 wait complete. 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 Processor -- start 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- start start 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb18101790 0x7fcb18197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb18102980 0x7fcb18197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 0x7fcb1819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1810d3e0 con 0x7fcb18103340 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1810d260 con 0x7fcb18102980 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1ec41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1810d560 con 0x7fcb18101790 2024-04-24T23:12:40.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1d1de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 0x7fcb1819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1d1de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 0x7fcb1819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48140/0 (socket says 172.21.15.33:48140) 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1d1de640 1 -- 172.21.15.33:0/2703759188 learned_addr learned my addr 172.21.15.33:0/2703759188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.640+0000 7fcb1c9dd640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb18101790 0x7fcb18197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb0ffff640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb18102980 0x7fcb18197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1d1de640 1 -- 172.21.15.33:0/2703759188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb18101790 msgr2=0x7fcb18197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1d1de640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb18101790 0x7fcb18197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1d1de640 1 -- 172.21.15.33:0/2703759188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb18102980 msgr2=0x7fcb18197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1d1de640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb18102980 0x7fcb18197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1d1de640 1 -- 172.21.15.33:0/2703759188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1819f650 con 0x7fcb18103340 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1d1de640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 0x7fcb1819efe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000cce0 tx=0x7fcb1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000f450 con 0x7fcb18103340 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1819f940 con 0x7fcb18103340 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1000fad0 con 0x7fcb18103340 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10018c10 con 0x7fcb18103340 2024-04-24T23:12:40.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.641+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1819fe90 con 0x7fcb18103340 2024-04-24T23:12:40.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.642+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb18108ab0 con 0x7fcb18103340 2024-04-24T23:12:40.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.643+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fcb1000f5f0 con 0x7fcb18103340 2024-04-24T23:12:40.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.644+0000 7fcb0dffb640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb000a2c20 0x7fcb000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.644+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcb100cfbe0 con 0x7fcb18103340 2024-04-24T23:12:40.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.644+0000 7fcb1c9dd640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb000a2c20 0x7fcb000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.644+0000 7fcb1c9dd640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb000a2c20 0x7fcb000a50e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005ab0 tx=0x7fcb14000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.648+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcb10014030 con 0x7fcb18103340 2024-04-24T23:12:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.830+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 --> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb18061b80 con 0x7fcb000a2c20 2024-04-24T23:12:40.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.839+0000 7fcb0dffb640 1 -- 172.21.15.33:0/2703759188 <== mgr.14148 v2:172.21.15.33:6800/3494091302 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340812 (secure 0 0 0) 0x7fcb18061b80 con 0x7fcb000a2c20 2024-04-24T23:12:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:40.846 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb000a2c20 msgr2=0x7fcb000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb000a2c20 0x7fcb000a50e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005ab0 tx=0x7fcb14000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 msgr2=0x7fcb1819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 0x7fcb1819efe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000cce0 tx=0x7fcb1000a620 comp rx=0 tx=0).stop 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 shutdown_connections 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcb000a2c20 0x7fcb000a50e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb18103340 0x7fcb1819efe0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb18102980 0x7fcb18197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 --2- 172.21.15.33:0/2703759188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb18101790 0x7fcb18197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 >> 172.21.15.33:0/2703759188 conn(0x7fcb180fcd40 msgr2=0x7fcb18105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 shutdown_connections 2024-04-24T23:12:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.843+0000 7fcb1ec41640 1 -- 172.21.15.33:0/2703759188 wait complete. 2024-04-24T23:12:40.856 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-24T23:12:39.819005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":865,"num_read_kb":556,"num_write":353,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181934,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357588,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584236,"statfs":{"total":1151940427776,"available":1151574257664,"internally_reserved":0,"allocated":12845056,"data_stored":8230270,"data_compressed":434646,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295913+0000","last_change":"2024-04-24T23:10:12.350152+0000","last_active":"2024-04-24T23:12:08.295913+0000","last_peered":"2024-04-24T23:12:08.295913+0000","last_clean":"2024-04-24T23:12:08.295913+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:12:08.295913+0000","last_undegraded":"2024-04-24T23:12:08.295913+0000","last_fullsized":"2024-04-24T23:12:08.295913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:45.190527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854282+0000","last_change":"2024-04-24T23:10:12.343536+0000","last_active":"2024-04-24T23:12:07.854282+0000","last_peered":"2024-04-24T23:12:07.854282+0000","last_clean":"2024-04-24T23:12:07.854282+0000","last_became_active":"2024-04-24T23:10:12.343207+0000","last_became_peered":"2024-04-24T23:10:12.343207+0000","last_unstale":"2024-04-24T23:12:07.854282+0000","last_undegraded":"2024-04-24T23:12:07.854282+0000","last_fullsized":"2024-04-24T23:12:07.854282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:20.264012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854048+0000","last_change":"2024-04-24T23:10:12.355169+0000","last_active":"2024-04-24T23:12:07.854048+0000","last_peered":"2024-04-24T23:12:07.854048+0000","last_clean":"2024-04-24T23:12:07.854048+0000","last_became_active":"2024-04-24T23:10:12.354039+0000","last_became_peered":"2024-04-24T23:10:12.354039+0000","last_unstale":"2024-04-24T23:12:07.854048+0000","last_undegraded":"2024-04-24T23:12:07.854048+0000","last_fullsized":"2024-04-24T23:12:07.854048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:59.200919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372122+0000","last_change":"2024-04-24T23:11:07.572557+0000","last_active":"2024-04-24T23:12:08.372122+0000","last_peered":"2024-04-24T23:12:08.372122+0000","last_clean":"2024-04-24T23:12:08.372122+0000","last_became_active":"2024-04-24T23:11:07.572144+0000","last_became_peered":"2024-04-24T23:11:07.572144+0000","last_unstale":"2024-04-24T23:12:08.372122+0000","last_undegraded":"2024-04-24T23:12:08.372122+0000","last_fullsized":"2024-04-24T23:12:08.372122+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:42.681992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373076+0000","last_change":"2024-04-24T23:10:12.339453+0000","last_active":"2024-04-24T23:12:08.373076+0000","last_peered":"2024-04-24T23:12:08.373076+0000","last_clean":"2024-04-24T23:12:08.373076+0000","last_became_active":"2024-04-24T23:10:12.339118+0000","last_became_peered":"2024-04-24T23:10:12.339118+0000","last_unstale":"2024-04-24T23:12:08.373076+0000","last_undegraded":"2024-04-24T23:12:08.373076+0000","last_fullsized":"2024-04-24T23:12:08.373076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:16.013202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853784+0000","last_change":"2024-04-24T23:10:12.342170+0000","last_active":"2024-04-24T23:12:07.853784+0000","last_peered":"2024-04-24T23:12:07.853784+0000","last_clean":"2024-04-24T23:12:07.853784+0000","last_became_active":"2024-04-24T23:10:12.341791+0000","last_became_peered":"2024-04-24T23:10:12.341791+0000","last_unstale":"2024-04-24T23:12:07.853784+0000","last_undegraded":"2024-04-24T23:12:07.853784+0000","last_fullsized":"2024-04-24T23:12:07.853784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:23.847984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372524+0000","last_change":"2024-04-24T23:10:12.348048+0000","last_active":"2024-04-24T23:12:08.372524+0000","last_peered":"2024-04-24T23:12:08.372524+0000","last_clean":"2024-04-24T23:12:08.372524+0000","last_became_active":"2024-04-24T23:10:12.347447+0000","last_became_peered":"2024-04-24T23:10:12.347447+0000","last_unstale":"2024-04-24T23:12:08.372524+0000","last_undegraded":"2024-04-24T23:12:08.372524+0000","last_fullsized":"2024-04-24T23:12:08.372524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:42.922390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850067+0000","last_change":"2024-04-24T23:10:12.349068+0000","last_active":"2024-04-24T23:12:07.850067+0000","last_peered":"2024-04-24T23:12:07.850067+0000","last_clean":"2024-04-24T23:12:07.850067+0000","last_became_active":"2024-04-24T23:10:12.348716+0000","last_became_peered":"2024-04-24T23:10:12.348716+0000","last_unstale":"2024-04-24T23:12:07.850067+0000","last_undegraded":"2024-04-24T23:12:07.850067+0000","last_fullsized":"2024-04-24T23:12:07.850067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281392+0000","last_change":"2024-04-24T23:10:12.340512+0000","last_active":"2024-04-24T23:12:08.281392+0000","last_peered":"2024-04-24T23:12:08.281392+0000","last_clean":"2024-04-24T23:12:08.281392+0000","last_became_active":"2024-04-24T23:10:12.338786+0000","last_became_peered":"2024-04-24T23:10:12.338786+0000","last_unstale":"2024-04-24T23:12:08.281392+0000","last_undegraded":"2024-04-24T23:12:08.281392+0000","last_fullsized":"2024-04-24T23:12:08.281392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:00.416934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854555+0000","last_change":"2024-04-24T23:10:12.352586+0000","last_active":"2024-04-24T23:12:07.854555+0000","last_peered":"2024-04-24T23:12:07.854555+0000","last_clean":"2024-04-24T23:12:07.854555+0000","last_became_active":"2024-04-24T23:10:12.351649+0000","last_became_peered":"2024-04-24T23:10:12.351649+0000","last_unstale":"2024-04-24T23:12:07.854555+0000","last_undegraded":"2024-04-24T23:12:07.854555+0000","last_fullsized":"2024-04-24T23:12:07.854555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:50.630673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373106+0000","last_change":"2024-04-24T23:10:12.343762+0000","last_active":"2024-04-24T23:12:08.373106+0000","last_peered":"2024-04-24T23:12:08.373106+0000","last_clean":"2024-04-24T23:12:08.373106+0000","last_became_active":"2024-04-24T23:10:12.343245+0000","last_became_peered":"2024-04-24T23:10:12.343245+0000","last_unstale":"2024-04-24T23:12:08.373106+0000","last_undegraded":"2024-04-24T23:12:08.373106+0000","last_fullsized":"2024-04-24T23:12:08.373106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:05.843677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371900+0000","last_change":"2024-04-24T23:11:07.568947+0000","last_active":"2024-04-24T23:12:08.371900+0000","last_peered":"2024-04-24T23:12:08.371900+0000","last_clean":"2024-04-24T23:12:08.371900+0000","last_became_active":"2024-04-24T23:11:07.568528+0000","last_became_peered":"2024-04-24T23:11:07.568528+0000","last_unstale":"2024-04-24T23:12:08.371900+0000","last_undegraded":"2024-04-24T23:12:08.371900+0000","last_fullsized":"2024-04-24T23:12:08.371900+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:23.452961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352070+0000","last_change":"2024-04-24T23:10:12.352110+0000","last_active":"2024-04-24T23:12:08.352070+0000","last_peered":"2024-04-24T23:12:08.352070+0000","last_clean":"2024-04-24T23:12:08.352070+0000","last_became_active":"2024-04-24T23:10:12.350409+0000","last_became_peered":"2024-04-24T23:10:12.350409+0000","last_unstale":"2024-04-24T23:12:08.352070+0000","last_undegraded":"2024-04-24T23:12:08.352070+0000","last_fullsized":"2024-04-24T23:12:08.352070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:44.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853752+0000","last_change":"2024-04-24T23:10:12.355401+0000","last_active":"2024-04-24T23:12:07.853752+0000","last_peered":"2024-04-24T23:12:07.853752+0000","last_clean":"2024-04-24T23:12:07.853752+0000","last_became_active":"2024-04-24T23:10:12.351542+0000","last_became_peered":"2024-04-24T23:10:12.351542+0000","last_unstale":"2024-04-24T23:12:07.853752+0000","last_undegraded":"2024-04-24T23:12:07.853752+0000","last_fullsized":"2024-04-24T23:12:07.853752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:42.082917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853692+0000","last_change":"2024-04-24T23:10:12.345197+0000","last_active":"2024-04-24T23:12:07.853692+0000","last_peered":"2024-04-24T23:12:07.853692+0000","last_clean":"2024-04-24T23:12:07.853692+0000","last_became_active":"2024-04-24T23:10:12.344800+0000","last_became_peered":"2024-04-24T23:10:12.344800+0000","last_unstale":"2024-04-24T23:12:07.853692+0000","last_undegraded":"2024-04-24T23:12:07.853692+0000","last_fullsized":"2024-04-24T23:12:07.853692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:33.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280637+0000","last_change":"2024-04-24T23:10:12.337426+0000","last_active":"2024-04-24T23:12:08.280637+0000","last_peered":"2024-04-24T23:12:08.280637+0000","last_clean":"2024-04-24T23:12:08.280637+0000","last_became_active":"2024-04-24T23:10:12.336716+0000","last_became_peered":"2024-04-24T23:10:12.336716+0000","last_unstale":"2024-04-24T23:12:08.280637+0000","last_undegraded":"2024-04-24T23:12:08.280637+0000","last_fullsized":"2024-04-24T23:12:08.280637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:47.742756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295782+0000","last_change":"2024-04-24T23:10:12.350190+0000","last_active":"2024-04-24T23:12:08.295782+0000","last_peered":"2024-04-24T23:12:08.295782+0000","last_clean":"2024-04-24T23:12:08.295782+0000","last_became_active":"2024-04-24T23:10:12.349815+0000","last_became_peered":"2024-04-24T23:10:12.349815+0000","last_unstale":"2024-04-24T23:12:08.295782+0000","last_undegraded":"2024-04-24T23:12:08.295782+0000","last_fullsized":"2024-04-24T23:12:08.295782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.888695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372278+0000","last_change":"2024-04-24T23:10:12.349052+0000","last_active":"2024-04-24T23:12:08.372278+0000","last_peered":"2024-04-24T23:12:08.372278+0000","last_clean":"2024-04-24T23:12:08.372278+0000","last_became_active":"2024-04-24T23:10:12.348678+0000","last_became_peered":"2024-04-24T23:10:12.348678+0000","last_unstale":"2024-04-24T23:12:08.372278+0000","last_undegraded":"2024-04-24T23:12:08.372278+0000","last_fullsized":"2024-04-24T23:12:08.372278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:35.906344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855452+0000","last_change":"2024-04-24T23:10:12.343911+0000","last_active":"2024-04-24T23:12:07.855452+0000","last_peered":"2024-04-24T23:12:07.855452+0000","last_clean":"2024-04-24T23:12:07.855452+0000","last_became_active":"2024-04-24T23:10:12.343529+0000","last_became_peered":"2024-04-24T23:10:12.343529+0000","last_unstale":"2024-04-24T23:12:07.855452+0000","last_undegraded":"2024-04-24T23:12:07.855452+0000","last_fullsized":"2024-04-24T23:12:07.855452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:15.870685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855027+0000","last_change":"2024-04-24T23:10:16.382384+0000","last_active":"2024-04-24T23:12:07.855027+0000","last_peered":"2024-04-24T23:12:07.855027+0000","last_clean":"2024-04-24T23:12:07.855027+0000","last_became_active":"2024-04-24T23:10:16.382019+0000","last_became_peered":"2024-04-24T23:10:16.382019+0000","last_unstale":"2024-04-24T23:12:07.855027+0000","last_undegraded":"2024-04-24T23:12:07.855027+0000","last_fullsized":"2024-04-24T23:12:07.855027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:01.942253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372315+0000","last_change":"2024-04-24T23:11:07.769518+0000","last_active":"2024-04-24T23:12:08.372315+0000","last_peered":"2024-04-24T23:12:08.372315+0000","last_clean":"2024-04-24T23:12:08.372315+0000","last_became_active":"2024-04-24T23:11:07.763040+0000","last_became_peered":"2024-04-24T23:11:07.763040+0000","last_unstale":"2024-04-24T23:12:08.372315+0000","last_undegraded":"2024-04-24T23:12:08.372315+0000","last_fullsized":"2024-04-24T23:12:08.372315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.056568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850728+0000","last_change":"2024-04-24T23:10:12.353196+0000","last_active":"2024-04-24T23:12:07.850728+0000","last_peered":"2024-04-24T23:12:07.850728+0000","last_clean":"2024-04-24T23:12:07.850728+0000","last_became_active":"2024-04-24T23:10:12.352464+0000","last_became_peered":"2024-04-24T23:10:12.352464+0000","last_unstale":"2024-04-24T23:12:07.850728+0000","last_undegraded":"2024-04-24T23:12:07.850728+0000","last_fullsized":"2024-04-24T23:12:07.850728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:45.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192411+0000","last_change":"2024-04-24T23:10:16.381498+0000","last_active":"2024-04-24T23:12:08.192411+0000","last_peered":"2024-04-24T23:12:08.192411+0000","last_clean":"2024-04-24T23:12:08.192411+0000","last_became_active":"2024-04-24T23:10:16.381029+0000","last_became_peered":"2024-04-24T23:10:16.381029+0000","last_unstale":"2024-04-24T23:12:08.192411+0000","last_undegraded":"2024-04-24T23:12:08.192411+0000","last_fullsized":"2024-04-24T23:12:08.192411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:37.272827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850867+0000","last_change":"2024-04-24T23:10:11.318371+0000","last_active":"2024-04-24T23:12:07.850867+0000","last_peered":"2024-04-24T23:12:07.850867+0000","last_clean":"2024-04-24T23:12:07.850867+0000","last_became_active":"2024-04-24T23:10:11.318074+0000","last_became_peered":"2024-04-24T23:10:11.318074+0000","last_unstale":"2024-04-24T23:12:07.850867+0000","last_undegraded":"2024-04-24T23:12:07.850867+0000","last_fullsized":"2024-04-24T23:12:07.850867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854113+0000","last_change":"2024-04-24T23:10:12.343411+0000","last_active":"2024-04-24T23:12:07.854113+0000","last_peered":"2024-04-24T23:12:07.854113+0000","last_clean":"2024-04-24T23:12:07.854113+0000","last_became_active":"2024-04-24T23:10:12.343069+0000","last_became_peered":"2024-04-24T23:10:12.343069+0000","last_unstale":"2024-04-24T23:12:07.854113+0000","last_undegraded":"2024-04-24T23:12:07.854113+0000","last_fullsized":"2024-04-24T23:12:07.854113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:00.448956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192900+0000","last_change":"2024-04-24T23:10:16.382007+0000","last_active":"2024-04-24T23:12:08.192900+0000","last_peered":"2024-04-24T23:12:08.192900+0000","last_clean":"2024-04-24T23:12:08.192900+0000","last_became_active":"2024-04-24T23:10:16.381448+0000","last_became_peered":"2024-04-24T23:10:16.381448+0000","last_unstale":"2024-04-24T23:12:08.192900+0000","last_undegraded":"2024-04-24T23:12:08.192900+0000","last_fullsized":"2024-04-24T23:12:08.192900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:11.011256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295762+0000","last_change":"2024-04-24T23:11:07.583051+0000","last_active":"2024-04-24T23:12:08.295762+0000","last_peered":"2024-04-24T23:12:08.295762+0000","last_clean":"2024-04-24T23:12:08.295762+0000","last_became_active":"2024-04-24T23:11:07.572502+0000","last_became_peered":"2024-04-24T23:11:07.572502+0000","last_unstale":"2024-04-24T23:12:08.295762+0000","last_undegraded":"2024-04-24T23:12:08.295762+0000","last_fullsized":"2024-04-24T23:12:08.295762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:21.581198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280792+0000","last_change":"2024-04-24T23:10:12.344216+0000","last_active":"2024-04-24T23:12:08.280792+0000","last_peered":"2024-04-24T23:12:08.280792+0000","last_clean":"2024-04-24T23:12:08.280792+0000","last_became_active":"2024-04-24T23:10:12.343896+0000","last_became_peered":"2024-04-24T23:10:12.343896+0000","last_unstale":"2024-04-24T23:12:08.280792+0000","last_undegraded":"2024-04-24T23:12:08.280792+0000","last_fullsized":"2024-04-24T23:12:08.280792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:49.065546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192430+0000","last_change":"2024-04-24T23:10:16.378081+0000","last_active":"2024-04-24T23:12:08.192430+0000","last_peered":"2024-04-24T23:12:08.192430+0000","last_clean":"2024-04-24T23:12:08.192430+0000","last_became_active":"2024-04-24T23:10:16.377550+0000","last_became_peered":"2024-04-24T23:10:16.377550+0000","last_unstale":"2024-04-24T23:12:08.192430+0000","last_undegraded":"2024-04-24T23:12:08.192430+0000","last_fullsized":"2024-04-24T23:12:08.192430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.234669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373278+0000","last_change":"2024-04-24T23:10:11.326286+0000","last_active":"2024-04-24T23:12:08.373278+0000","last_peered":"2024-04-24T23:12:08.373278+0000","last_clean":"2024-04-24T23:12:08.373278+0000","last_became_active":"2024-04-24T23:10:11.325908+0000","last_became_peered":"2024-04-24T23:10:11.325908+0000","last_unstale":"2024-04-24T23:12:08.373278+0000","last_undegraded":"2024-04-24T23:12:08.373278+0000","last_fullsized":"2024-04-24T23:12:08.373278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:45.229900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372321+0000","last_change":"2024-04-24T23:10:12.346066+0000","last_active":"2024-04-24T23:12:08.372321+0000","last_peered":"2024-04-24T23:12:08.372321+0000","last_clean":"2024-04-24T23:12:08.372321+0000","last_became_active":"2024-04-24T23:10:12.345731+0000","last_became_peered":"2024-04-24T23:10:12.345731+0000","last_unstale":"2024-04-24T23:12:08.372321+0000","last_undegraded":"2024-04-24T23:12:08.372321+0000","last_fullsized":"2024-04-24T23:12:08.372321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:42.276585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855307+0000","last_change":"2024-04-24T23:10:16.373328+0000","last_active":"2024-04-24T23:12:07.855307+0000","last_peered":"2024-04-24T23:12:07.855307+0000","last_clean":"2024-04-24T23:12:07.855307+0000","last_became_active":"2024-04-24T23:10:16.372646+0000","last_became_peered":"2024-04-24T23:10:16.372646+0000","last_unstale":"2024-04-24T23:12:07.855307+0000","last_undegraded":"2024-04-24T23:12:07.855307+0000","last_fullsized":"2024-04-24T23:12:07.855307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:52.100838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.679678+0000","last_change":"2024-04-24T23:10:11.321038+0000","last_active":"2024-04-24T23:12:33.679678+0000","last_peered":"2024-04-24T23:12:33.679678+0000","last_clean":"2024-04-24T23:12:33.679678+0000","last_became_active":"2024-04-24T23:10:11.320572+0000","last_became_peered":"2024-04-24T23:10:11.320572+0000","last_unstale":"2024-04-24T23:12:33.679678+0000","last_undegraded":"2024-04-24T23:12:33.679678+0000","last_fullsized":"2024-04-24T23:12:33.679678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:52.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373431+0000","last_change":"2024-04-24T23:10:12.343348+0000","last_active":"2024-04-24T23:12:08.373431+0000","last_peered":"2024-04-24T23:12:08.373431+0000","last_clean":"2024-04-24T23:12:08.373431+0000","last_became_active":"2024-04-24T23:10:12.340470+0000","last_became_peered":"2024-04-24T23:10:12.340470+0000","last_unstale":"2024-04-24T23:12:08.373431+0000","last_undegraded":"2024-04-24T23:12:08.373431+0000","last_fullsized":"2024-04-24T23:12:08.373431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.555110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855174+0000","last_change":"2024-04-24T23:10:16.375364+0000","last_active":"2024-04-24T23:12:07.855174+0000","last_peered":"2024-04-24T23:12:07.855174+0000","last_clean":"2024-04-24T23:12:07.855174+0000","last_became_active":"2024-04-24T23:10:16.374724+0000","last_became_peered":"2024-04-24T23:10:16.374724+0000","last_unstale":"2024-04-24T23:12:07.855174+0000","last_undegraded":"2024-04-24T23:12:07.855174+0000","last_fullsized":"2024-04-24T23:12:07.855174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:59.085567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853715+0000","last_change":"2024-04-24T23:10:11.322039+0000","last_active":"2024-04-24T23:12:07.853715+0000","last_peered":"2024-04-24T23:12:07.853715+0000","last_clean":"2024-04-24T23:12:07.853715+0000","last_became_active":"2024-04-24T23:10:11.321693+0000","last_became_peered":"2024-04-24T23:10:11.321693+0000","last_unstale":"2024-04-24T23:12:07.853715+0000","last_undegraded":"2024-04-24T23:12:07.853715+0000","last_fullsized":"2024-04-24T23:12:07.853715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:34.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372610+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:12:08.372610+0000","last_peered":"2024-04-24T23:12:08.372610+0000","last_clean":"2024-04-24T23:12:08.372610+0000","last_became_active":"2024-04-24T23:10:12.343704+0000","last_became_peered":"2024-04-24T23:10:12.343704+0000","last_unstale":"2024-04-24T23:12:08.372610+0000","last_undegraded":"2024-04-24T23:12:08.372610+0000","last_fullsized":"2024-04-24T23:12:08.372610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:36.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295780+0000","last_change":"2024-04-24T23:10:16.384894+0000","last_active":"2024-04-24T23:12:08.295780+0000","last_peered":"2024-04-24T23:12:08.295780+0000","last_clean":"2024-04-24T23:12:08.295780+0000","last_became_active":"2024-04-24T23:10:16.384526+0000","last_became_peered":"2024-04-24T23:10:16.384526+0000","last_unstale":"2024-04-24T23:12:08.295780+0000","last_undegraded":"2024-04-24T23:12:08.295780+0000","last_fullsized":"2024-04-24T23:12:08.295780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:03.744789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.351967+0000","last_change":"2024-04-24T23:10:11.313868+0000","last_active":"2024-04-24T23:12:08.351967+0000","last_peered":"2024-04-24T23:12:08.351967+0000","last_clean":"2024-04-24T23:12:08.351967+0000","last_became_active":"2024-04-24T23:10:11.312350+0000","last_became_peered":"2024-04-24T23:10:11.312350+0000","last_unstale":"2024-04-24T23:12:08.351967+0000","last_undegraded":"2024-04-24T23:12:08.351967+0000","last_fullsized":"2024-04-24T23:12:08.351967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:56.908642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.849936+0000","last_change":"2024-04-24T23:10:12.336879+0000","last_active":"2024-04-24T23:12:07.849936+0000","last_peered":"2024-04-24T23:12:07.849936+0000","last_clean":"2024-04-24T23:12:07.849936+0000","last_became_active":"2024-04-24T23:10:12.336546+0000","last_became_peered":"2024-04-24T23:10:12.336546+0000","last_unstale":"2024-04-24T23:12:07.849936+0000","last_undegraded":"2024-04-24T23:12:07.849936+0000","last_fullsized":"2024-04-24T23:12:07.849936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:27.606471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854277+0000","last_change":"2024-04-24T23:10:16.389263+0000","last_active":"2024-04-24T23:12:07.854277+0000","last_peered":"2024-04-24T23:12:07.854277+0000","last_clean":"2024-04-24T23:12:07.854277+0000","last_became_active":"2024-04-24T23:10:16.388807+0000","last_became_peered":"2024-04-24T23:10:16.388807+0000","last_unstale":"2024-04-24T23:12:07.854277+0000","last_undegraded":"2024-04-24T23:12:07.854277+0000","last_fullsized":"2024-04-24T23:12:07.854277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:23.240012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371822+0000","last_change":"2024-04-24T23:10:11.316739+0000","last_active":"2024-04-24T23:12:08.371822+0000","last_peered":"2024-04-24T23:12:08.371822+0000","last_clean":"2024-04-24T23:12:08.371822+0000","last_became_active":"2024-04-24T23:10:11.316420+0000","last_became_peered":"2024-04-24T23:10:11.316420+0000","last_unstale":"2024-04-24T23:12:08.371822+0000","last_undegraded":"2024-04-24T23:12:08.371822+0000","last_fullsized":"2024-04-24T23:12:08.371822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:39.846017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371791+0000","last_change":"2024-04-24T23:10:12.349911+0000","last_active":"2024-04-24T23:12:08.371791+0000","last_peered":"2024-04-24T23:12:08.371791+0000","last_clean":"2024-04-24T23:12:08.371791+0000","last_became_active":"2024-04-24T23:10:12.349585+0000","last_became_peered":"2024-04-24T23:10:12.349585+0000","last_unstale":"2024-04-24T23:12:08.371791+0000","last_undegraded":"2024-04-24T23:12:08.371791+0000","last_fullsized":"2024-04-24T23:12:08.371791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.711126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850098+0000","last_change":"2024-04-24T23:10:16.378057+0000","last_active":"2024-04-24T23:12:07.850098+0000","last_peered":"2024-04-24T23:12:07.850098+0000","last_clean":"2024-04-24T23:12:07.850098+0000","last_became_active":"2024-04-24T23:10:16.377610+0000","last_became_peered":"2024-04-24T23:10:16.377610+0000","last_unstale":"2024-04-24T23:12:07.850098+0000","last_undegraded":"2024-04-24T23:12:07.850098+0000","last_fullsized":"2024-04-24T23:12:07.850098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:28.985870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:36.684142+0000","last_change":"2024-04-24T23:10:11.315317+0000","last_active":"2024-04-24T23:12:36.684142+0000","last_peered":"2024-04-24T23:12:36.684142+0000","last_clean":"2024-04-24T23:12:36.684142+0000","last_became_active":"2024-04-24T23:10:11.314989+0000","last_became_peered":"2024-04-24T23:10:11.314989+0000","last_unstale":"2024-04-24T23:12:36.684142+0000","last_undegraded":"2024-04-24T23:12:36.684142+0000","last_fullsized":"2024-04-24T23:12:36.684142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:13.462987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281321+0000","last_change":"2024-04-24T23:10:12.343946+0000","last_active":"2024-04-24T23:12:08.281321+0000","last_peered":"2024-04-24T23:12:08.281321+0000","last_clean":"2024-04-24T23:12:08.281321+0000","last_became_active":"2024-04-24T23:10:12.343555+0000","last_became_peered":"2024-04-24T23:10:12.343555+0000","last_unstale":"2024-04-24T23:12:08.281321+0000","last_undegraded":"2024-04-24T23:12:08.281321+0000","last_fullsized":"2024-04-24T23:12:08.281321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:34.596288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371692+0000","last_change":"2024-04-24T23:10:16.383567+0000","last_active":"2024-04-24T23:12:08.371692+0000","last_peered":"2024-04-24T23:12:08.371692+0000","last_clean":"2024-04-24T23:12:08.371692+0000","last_became_active":"2024-04-24T23:10:16.383145+0000","last_became_peered":"2024-04-24T23:10:16.383145+0000","last_unstale":"2024-04-24T23:12:08.371692+0000","last_undegraded":"2024-04-24T23:12:08.371692+0000","last_fullsized":"2024-04-24T23:12:08.371692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:13.278209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373045+0000","last_change":"2024-04-24T23:10:11.320217+0000","last_active":"2024-04-24T23:12:08.373045+0000","last_peered":"2024-04-24T23:12:08.373045+0000","last_clean":"2024-04-24T23:12:08.373045+0000","last_became_active":"2024-04-24T23:10:11.319864+0000","last_became_peered":"2024-04-24T23:10:11.319864+0000","last_unstale":"2024-04-24T23:12:08.373045+0000","last_undegraded":"2024-04-24T23:12:08.373045+0000","last_fullsized":"2024-04-24T23:12:08.373045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:04.514457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854248+0000","last_change":"2024-04-24T23:10:12.350610+0000","last_active":"2024-04-24T23:12:07.854248+0000","last_peered":"2024-04-24T23:12:07.854248+0000","last_clean":"2024-04-24T23:12:07.854248+0000","last_became_active":"2024-04-24T23:10:12.349317+0000","last_became_peered":"2024-04-24T23:10:12.349317+0000","last_unstale":"2024-04-24T23:12:07.854248+0000","last_undegraded":"2024-04-24T23:12:07.854248+0000","last_fullsized":"2024-04-24T23:12:07.854248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193177+0000","last_change":"2024-04-24T23:10:16.381532+0000","last_active":"2024-04-24T23:12:08.193177+0000","last_peered":"2024-04-24T23:12:08.193177+0000","last_clean":"2024-04-24T23:12:08.193177+0000","last_became_active":"2024-04-24T23:10:16.381068+0000","last_became_peered":"2024-04-24T23:10:16.381068+0000","last_unstale":"2024-04-24T23:12:08.193177+0000","last_undegraded":"2024-04-24T23:12:08.193177+0000","last_fullsized":"2024-04-24T23:12:08.193177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:29.473228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372194+0000","last_change":"2024-04-24T23:10:11.319408+0000","last_active":"2024-04-24T23:12:08.372194+0000","last_peered":"2024-04-24T23:12:08.372194+0000","last_clean":"2024-04-24T23:12:08.372194+0000","last_became_active":"2024-04-24T23:10:11.319076+0000","last_became_peered":"2024-04-24T23:10:11.319076+0000","last_unstale":"2024-04-24T23:12:08.372194+0000","last_undegraded":"2024-04-24T23:12:08.372194+0000","last_fullsized":"2024-04-24T23:12:08.372194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:49.164995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.281176+0000","last_change":"2024-04-24T23:10:12.344210+0000","last_active":"2024-04-24T23:12:08.281176+0000","last_peered":"2024-04-24T23:12:08.281176+0000","last_clean":"2024-04-24T23:12:08.281176+0000","last_became_active":"2024-04-24T23:10:12.343403+0000","last_became_peered":"2024-04-24T23:10:12.343403+0000","last_unstale":"2024-04-24T23:12:08.281176+0000","last_undegraded":"2024-04-24T23:12:08.281176+0000","last_fullsized":"2024-04-24T23:12:08.281176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:57.595206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371789+0000","last_change":"2024-04-24T23:10:16.378666+0000","last_active":"2024-04-24T23:12:08.371789+0000","last_peered":"2024-04-24T23:12:08.371789+0000","last_clean":"2024-04-24T23:12:08.371789+0000","last_became_active":"2024-04-24T23:10:16.378178+0000","last_became_peered":"2024-04-24T23:10:16.378178+0000","last_unstale":"2024-04-24T23:12:08.371789+0000","last_undegraded":"2024-04-24T23:12:08.371789+0000","last_fullsized":"2024-04-24T23:12:08.371789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:48.468771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'37","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:36.667224+0000","last_change":"2024-04-24T23:10:11.313493+0000","last_active":"2024-04-24T23:12:36.667224+0000","last_peered":"2024-04-24T23:12:36.667224+0000","last_clean":"2024-04-24T23:12:36.667224+0000","last_became_active":"2024-04-24T23:10:11.313047+0000","last_became_peered":"2024-04-24T23:10:11.313047+0000","last_unstale":"2024-04-24T23:12:36.667224+0000","last_undegraded":"2024-04-24T23:12:36.667224+0000","last_fullsized":"2024-04-24T23:12:36.667224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:15.362025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":248,"num_read_kb":191,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373386+0000","last_change":"2024-04-24T23:10:12.337130+0000","last_active":"2024-04-24T23:12:08.373386+0000","last_peered":"2024-04-24T23:12:08.373386+0000","last_clean":"2024-04-24T23:12:08.373386+0000","last_became_active":"2024-04-24T23:10:12.336771+0000","last_became_peered":"2024-04-24T23:10:12.336771+0000","last_unstale":"2024-04-24T23:12:08.373386+0000","last_undegraded":"2024-04-24T23:12:08.373386+0000","last_fullsized":"2024-04-24T23:12:08.373386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:06.114141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.375570+0000","last_change":"2024-04-24T23:12:08.375043+0000","last_active":"2024-04-24T23:12:08.375570+0000","last_peered":"2024-04-24T23:12:08.375570+0000","last_clean":"2024-04-24T23:12:08.375570+0000","last_became_active":"2024-04-24T23:12:08.374204+0000","last_became_peered":"2024-04-24T23:12:08.374204+0000","last_unstale":"2024-04-24T23:12:08.375570+0000","last_undegraded":"2024-04-24T23:12:08.375570+0000","last_fullsized":"2024-04-24T23:12:08.375570+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:36.596328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295494+0000","last_change":"2024-04-24T23:11:07.579273+0000","last_active":"2024-04-24T23:12:08.295494+0000","last_peered":"2024-04-24T23:12:08.295494+0000","last_clean":"2024-04-24T23:12:08.295494+0000","last_became_active":"2024-04-24T23:11:07.570480+0000","last_became_peered":"2024-04-24T23:11:07.570480+0000","last_unstale":"2024-04-24T23:12:08.295494+0000","last_undegraded":"2024-04-24T23:12:08.295494+0000","last_fullsized":"2024-04-24T23:12:08.295494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:21.291453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352055+0000","last_change":"2024-04-24T23:10:12.339348+0000","last_active":"2024-04-24T23:12:08.352055+0000","last_peered":"2024-04-24T23:12:08.352055+0000","last_clean":"2024-04-24T23:12:08.352055+0000","last_became_active":"2024-04-24T23:10:12.338922+0000","last_became_peered":"2024-04-24T23:10:12.338922+0000","last_unstale":"2024-04-24T23:12:08.352055+0000","last_undegraded":"2024-04-24T23:12:08.352055+0000","last_fullsized":"2024-04-24T23:12:08.352055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:26.755465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295559+0000","last_change":"2024-04-24T23:10:16.377674+0000","last_active":"2024-04-24T23:12:08.295559+0000","last_peered":"2024-04-24T23:12:08.295559+0000","last_clean":"2024-04-24T23:12:08.295559+0000","last_became_active":"2024-04-24T23:10:16.377275+0000","last_became_peered":"2024-04-24T23:10:16.377275+0000","last_unstale":"2024-04-24T23:12:08.295559+0000","last_undegraded":"2024-04-24T23:12:08.295559+0000","last_fullsized":"2024-04-24T23:12:08.295559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:41.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373038+0000","last_change":"2024-04-24T23:10:11.319638+0000","last_active":"2024-04-24T23:12:08.373038+0000","last_peered":"2024-04-24T23:12:08.373038+0000","last_clean":"2024-04-24T23:12:08.373038+0000","last_became_active":"2024-04-24T23:10:11.319324+0000","last_became_peered":"2024-04-24T23:10:11.319324+0000","last_unstale":"2024-04-24T23:12:08.373038+0000","last_undegraded":"2024-04-24T23:12:08.373038+0000","last_fullsized":"2024-04-24T23:12:08.373038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:38.744698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373266+0000","last_change":"2024-04-24T23:10:12.339788+0000","last_active":"2024-04-24T23:12:08.373266+0000","last_peered":"2024-04-24T23:12:08.373266+0000","last_clean":"2024-04-24T23:12:08.373266+0000","last_became_active":"2024-04-24T23:10:12.339439+0000","last_became_peered":"2024-04-24T23:10:12.339439+0000","last_unstale":"2024-04-24T23:12:08.373266+0000","last_undegraded":"2024-04-24T23:12:08.373266+0000","last_fullsized":"2024-04-24T23:12:08.373266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:15.724995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372070+0000","last_change":"2024-04-24T23:10:16.384367+0000","last_active":"2024-04-24T23:12:08.372070+0000","last_peered":"2024-04-24T23:12:08.372070+0000","last_clean":"2024-04-24T23:12:08.372070+0000","last_became_active":"2024-04-24T23:10:16.383985+0000","last_became_peered":"2024-04-24T23:10:16.383985+0000","last_unstale":"2024-04-24T23:12:08.372070+0000","last_undegraded":"2024-04-24T23:12:08.372070+0000","last_fullsized":"2024-04-24T23:12:08.372070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:34.651637+0000","last_change":"2024-04-24T23:10:11.319370+0000","last_active":"2024-04-24T23:12:34.651637+0000","last_peered":"2024-04-24T23:12:34.651637+0000","last_clean":"2024-04-24T23:12:34.651637+0000","last_became_active":"2024-04-24T23:10:11.319034+0000","last_became_peered":"2024-04-24T23:10:11.319034+0000","last_unstale":"2024-04-24T23:12:34.651637+0000","last_undegraded":"2024-04-24T23:12:34.651637+0000","last_fullsized":"2024-04-24T23:12:34.651637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:54.940965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372711+0000","last_change":"2024-04-24T23:10:12.348868+0000","last_active":"2024-04-24T23:12:08.372711+0000","last_peered":"2024-04-24T23:12:08.372711+0000","last_clean":"2024-04-24T23:12:08.372711+0000","last_became_active":"2024-04-24T23:10:12.348516+0000","last_became_peered":"2024-04-24T23:10:12.348516+0000","last_unstale":"2024-04-24T23:12:08.372711+0000","last_undegraded":"2024-04-24T23:12:08.372711+0000","last_fullsized":"2024-04-24T23:12:08.372711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:28.695559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352276+0000","last_change":"2024-04-24T23:11:07.570768+0000","last_active":"2024-04-24T23:12:08.352276+0000","last_peered":"2024-04-24T23:12:08.352276+0000","last_clean":"2024-04-24T23:12:08.352276+0000","last_became_active":"2024-04-24T23:11:07.570306+0000","last_became_peered":"2024-04-24T23:11:07.570306+0000","last_unstale":"2024-04-24T23:12:08.352276+0000","last_undegraded":"2024-04-24T23:12:08.352276+0000","last_fullsized":"2024-04-24T23:12:08.352276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:10.450660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373058+0000","last_change":"2024-04-24T23:10:11.323102+0000","last_active":"2024-04-24T23:12:08.373058+0000","last_peered":"2024-04-24T23:12:08.373058+0000","last_clean":"2024-04-24T23:12:08.373058+0000","last_became_active":"2024-04-24T23:10:11.322795+0000","last_became_peered":"2024-04-24T23:10:11.322795+0000","last_unstale":"2024-04-24T23:12:08.373058+0000","last_undegraded":"2024-04-24T23:12:08.373058+0000","last_fullsized":"2024-04-24T23:12:08.373058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:37.182662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371872+0000","last_change":"2024-04-24T23:11:07.762601+0000","last_active":"2024-04-24T23:12:08.371872+0000","last_peered":"2024-04-24T23:12:08.371872+0000","last_clean":"2024-04-24T23:12:08.371872+0000","last_became_active":"2024-04-24T23:11:07.762305+0000","last_became_peered":"2024-04-24T23:11:07.762305+0000","last_unstale":"2024-04-24T23:12:08.371872+0000","last_undegraded":"2024-04-24T23:12:08.371872+0000","last_fullsized":"2024-04-24T23:12:08.371872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:36.209046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372017+0000","last_change":"2024-04-24T23:10:16.380843+0000","last_active":"2024-04-24T23:12:08.372017+0000","last_peered":"2024-04-24T23:12:08.372017+0000","last_clean":"2024-04-24T23:12:08.372017+0000","last_became_active":"2024-04-24T23:10:16.380509+0000","last_became_peered":"2024-04-24T23:10:16.380509+0000","last_unstale":"2024-04-24T23:12:08.372017+0000","last_undegraded":"2024-04-24T23:12:08.372017+0000","last_fullsized":"2024-04-24T23:12:08.372017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:41.699710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373271+0000","last_change":"2024-04-24T23:10:11.319511+0000","last_active":"2024-04-24T23:12:08.373271+0000","last_peered":"2024-04-24T23:12:08.373271+0000","last_clean":"2024-04-24T23:12:08.373271+0000","last_became_active":"2024-04-24T23:10:11.319157+0000","last_became_peered":"2024-04-24T23:10:11.319157+0000","last_unstale":"2024-04-24T23:12:08.373271+0000","last_undegraded":"2024-04-24T23:12:08.373271+0000","last_fullsized":"2024-04-24T23:12:08.373271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:46.270959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295800+0000","last_change":"2024-04-24T23:11:07.569318+0000","last_active":"2024-04-24T23:12:08.295800+0000","last_peered":"2024-04-24T23:12:08.295800+0000","last_clean":"2024-04-24T23:12:08.295800+0000","last_became_active":"2024-04-24T23:11:07.568799+0000","last_became_peered":"2024-04-24T23:11:07.568799+0000","last_unstale":"2024-04-24T23:12:08.295800+0000","last_undegraded":"2024-04-24T23:12:08.295800+0000","last_fullsized":"2024-04-24T23:12:08.295800+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:40.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373085+0000","last_change":"2024-04-24T23:10:16.447726+0000","last_active":"2024-04-24T23:12:08.373085+0000","last_peered":"2024-04-24T23:12:08.373085+0000","last_clean":"2024-04-24T23:12:08.373085+0000","last_became_active":"2024-04-24T23:10:16.447323+0000","last_became_peered":"2024-04-24T23:10:16.447323+0000","last_unstale":"2024-04-24T23:12:08.373085+0000","last_undegraded":"2024-04-24T23:12:08.373085+0000","last_fullsized":"2024-04-24T23:12:08.373085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:25.723783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372189+0000","last_change":"2024-04-24T23:10:11.315055+0000","last_active":"2024-04-24T23:12:08.372189+0000","last_peered":"2024-04-24T23:12:08.372189+0000","last_clean":"2024-04-24T23:12:08.372189+0000","last_became_active":"2024-04-24T23:10:11.314695+0000","last_became_peered":"2024-04-24T23:10:11.314695+0000","last_unstale":"2024-04-24T23:12:08.372189+0000","last_undegraded":"2024-04-24T23:12:08.372189+0000","last_fullsized":"2024-04-24T23:12:08.372189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:26.914523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854317+0000","last_change":"2024-04-24T23:10:12.352964+0000","last_active":"2024-04-24T23:12:07.854317+0000","last_peered":"2024-04-24T23:12:07.854317+0000","last_clean":"2024-04-24T23:12:07.854317+0000","last_became_active":"2024-04-24T23:10:12.352608+0000","last_became_peered":"2024-04-24T23:10:12.352608+0000","last_unstale":"2024-04-24T23:12:07.854317+0000","last_undegraded":"2024-04-24T23:12:07.854317+0000","last_fullsized":"2024-04-24T23:12:07.854317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:21.552653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352055+0000","last_change":"2024-04-24T23:10:16.387791+0000","last_active":"2024-04-24T23:12:08.352055+0000","last_peered":"2024-04-24T23:12:08.352055+0000","last_clean":"2024-04-24T23:12:08.352055+0000","last_became_active":"2024-04-24T23:10:16.387389+0000","last_became_peered":"2024-04-24T23:10:16.387389+0000","last_unstale":"2024-04-24T23:12:08.352055+0000","last_undegraded":"2024-04-24T23:12:08.352055+0000","last_fullsized":"2024-04-24T23:12:08.352055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:19.248990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372748+0000","last_change":"2024-04-24T23:10:11.324565+0000","last_active":"2024-04-24T23:12:08.372748+0000","last_peered":"2024-04-24T23:12:08.372748+0000","last_clean":"2024-04-24T23:12:08.372748+0000","last_became_active":"2024-04-24T23:10:11.323874+0000","last_became_peered":"2024-04-24T23:10:11.323874+0000","last_unstale":"2024-04-24T23:12:08.372748+0000","last_undegraded":"2024-04-24T23:12:08.372748+0000","last_fullsized":"2024-04-24T23:12:08.372748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.403486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853997+0000","last_change":"2024-04-24T23:10:12.350055+0000","last_active":"2024-04-24T23:12:07.853997+0000","last_peered":"2024-04-24T23:12:07.853997+0000","last_clean":"2024-04-24T23:12:07.853997+0000","last_became_active":"2024-04-24T23:10:12.349726+0000","last_became_peered":"2024-04-24T23:10:12.349726+0000","last_unstale":"2024-04-24T23:12:07.853997+0000","last_undegraded":"2024-04-24T23:12:07.853997+0000","last_fullsized":"2024-04-24T23:12:07.853997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:26.062992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352356+0000","last_change":"2024-04-24T23:10:16.447923+0000","last_active":"2024-04-24T23:12:08.352356+0000","last_peered":"2024-04-24T23:12:08.352356+0000","last_clean":"2024-04-24T23:12:08.352356+0000","last_became_active":"2024-04-24T23:10:16.447362+0000","last_became_peered":"2024-04-24T23:10:16.447362+0000","last_unstale":"2024-04-24T23:12:08.352356+0000","last_undegraded":"2024-04-24T23:12:08.352356+0000","last_fullsized":"2024-04-24T23:12:08.352356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:38.381445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372205+0000","last_change":"2024-04-24T23:10:12.345334+0000","last_active":"2024-04-24T23:12:08.372205+0000","last_peered":"2024-04-24T23:12:08.372205+0000","last_clean":"2024-04-24T23:12:08.372205+0000","last_became_active":"2024-04-24T23:10:12.345007+0000","last_became_peered":"2024-04-24T23:10:12.345007+0000","last_unstale":"2024-04-24T23:12:08.372205+0000","last_undegraded":"2024-04-24T23:12:08.372205+0000","last_fullsized":"2024-04-24T23:12:08.372205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.319834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:34.650960+0000","last_change":"2024-04-24T23:10:11.318756+0000","last_active":"2024-04-24T23:12:34.650960+0000","last_peered":"2024-04-24T23:12:34.650960+0000","last_clean":"2024-04-24T23:12:34.650960+0000","last_became_active":"2024-04-24T23:10:11.318441+0000","last_became_peered":"2024-04-24T23:10:11.318441+0000","last_unstale":"2024-04-24T23:12:34.650960+0000","last_undegraded":"2024-04-24T23:12:34.650960+0000","last_fullsized":"2024-04-24T23:12:34.650960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:17.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372870+0000","last_change":"2024-04-24T23:10:16.375644+0000","last_active":"2024-04-24T23:12:08.372870+0000","last_peered":"2024-04-24T23:12:08.372870+0000","last_clean":"2024-04-24T23:12:08.372870+0000","last_became_active":"2024-04-24T23:10:16.375061+0000","last_became_peered":"2024-04-24T23:10:16.375061+0000","last_unstale":"2024-04-24T23:12:08.372870+0000","last_undegraded":"2024-04-24T23:12:08.372870+0000","last_fullsized":"2024-04-24T23:12:08.372870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:24.298205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850177+0000","last_change":"2024-04-24T23:10:12.336145+0000","last_active":"2024-04-24T23:12:07.850177+0000","last_peered":"2024-04-24T23:12:07.850177+0000","last_clean":"2024-04-24T23:12:07.850177+0000","last_became_active":"2024-04-24T23:10:12.335793+0000","last_became_peered":"2024-04-24T23:10:12.335793+0000","last_unstale":"2024-04-24T23:12:07.850177+0000","last_undegraded":"2024-04-24T23:12:07.850177+0000","last_fullsized":"2024-04-24T23:12:07.850177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:04.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193019+0000","last_change":"2024-04-24T23:10:11.315082+0000","last_active":"2024-04-24T23:12:08.193019+0000","last_peered":"2024-04-24T23:12:08.193019+0000","last_clean":"2024-04-24T23:12:08.193019+0000","last_became_active":"2024-04-24T23:10:11.314757+0000","last_became_peered":"2024-04-24T23:10:11.314757+0000","last_unstale":"2024-04-24T23:12:08.193019+0000","last_undegraded":"2024-04-24T23:12:08.193019+0000","last_fullsized":"2024-04-24T23:12:08.193019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:33.698757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372433+0000","last_change":"2024-04-24T23:08:59.734921+0000","last_active":"2024-04-24T23:12:08.372433+0000","last_peered":"2024-04-24T23:12:08.372433+0000","last_clean":"2024-04-24T23:12:08.372433+0000","last_became_active":"2024-04-24T23:08:59.734536+0000","last_became_peered":"2024-04-24T23:08:59.734536+0000","last_unstale":"2024-04-24T23:12:08.372433+0000","last_undegraded":"2024-04-24T23:12:08.372433+0000","last_fullsized":"2024-04-24T23:12:08.372433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.474769+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.474769+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:43.242676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372229+0000","last_change":"2024-04-24T23:11:07.571671+0000","last_active":"2024-04-24T23:12:08.372229+0000","last_peered":"2024-04-24T23:12:08.372229+0000","last_clean":"2024-04-24T23:12:08.372229+0000","last_became_active":"2024-04-24T23:11:07.571082+0000","last_became_peered":"2024-04-24T23:11:07.571082+0000","last_unstale":"2024-04-24T23:12:08.372229+0000","last_undegraded":"2024-04-24T23:12:08.372229+0000","last_fullsized":"2024-04-24T23:12:08.372229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:11.106931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.351810+0000","last_change":"2024-04-24T23:10:12.340012+0000","last_active":"2024-04-24T23:12:08.351810+0000","last_peered":"2024-04-24T23:12:08.351810+0000","last_clean":"2024-04-24T23:12:08.351810+0000","last_became_active":"2024-04-24T23:10:12.339450+0000","last_became_peered":"2024-04-24T23:10:12.339450+0000","last_unstale":"2024-04-24T23:12:08.351810+0000","last_undegraded":"2024-04-24T23:12:08.351810+0000","last_fullsized":"2024-04-24T23:12:08.351810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:35.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372678+0000","last_change":"2024-04-24T23:10:11.324617+0000","last_active":"2024-04-24T23:12:08.372678+0000","last_peered":"2024-04-24T23:12:08.372678+0000","last_clean":"2024-04-24T23:12:08.372678+0000","last_became_active":"2024-04-24T23:10:11.323980+0000","last_became_peered":"2024-04-24T23:10:11.323980+0000","last_unstale":"2024-04-24T23:12:08.372678+0000","last_undegraded":"2024-04-24T23:12:08.372678+0000","last_fullsized":"2024-04-24T23:12:08.372678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:29.118867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295482+0000","last_change":"2024-04-24T23:10:16.376576+0000","last_active":"2024-04-24T23:12:08.295482+0000","last_peered":"2024-04-24T23:12:08.295482+0000","last_clean":"2024-04-24T23:12:08.295482+0000","last_became_active":"2024-04-24T23:10:16.376194+0000","last_became_peered":"2024-04-24T23:10:16.376194+0000","last_unstale":"2024-04-24T23:12:08.295482+0000","last_undegraded":"2024-04-24T23:12:08.295482+0000","last_fullsized":"2024-04-24T23:12:08.295482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:57.693223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373489+0000","last_change":"2024-04-24T23:10:11.325689+0000","last_active":"2024-04-24T23:12:08.373489+0000","last_peered":"2024-04-24T23:12:08.373489+0000","last_clean":"2024-04-24T23:12:08.373489+0000","last_became_active":"2024-04-24T23:10:11.324220+0000","last_became_peered":"2024-04-24T23:10:11.324220+0000","last_unstale":"2024-04-24T23:12:08.373489+0000","last_undegraded":"2024-04-24T23:12:08.373489+0000","last_fullsized":"2024-04-24T23:12:08.373489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:11.601286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352204+0000","last_change":"2024-04-24T23:10:12.334866+0000","last_active":"2024-04-24T23:12:08.352204+0000","last_peered":"2024-04-24T23:12:08.352204+0000","last_clean":"2024-04-24T23:12:08.352204+0000","last_became_active":"2024-04-24T23:10:12.334483+0000","last_became_peered":"2024-04-24T23:10:12.334483+0000","last_unstale":"2024-04-24T23:12:08.352204+0000","last_undegraded":"2024-04-24T23:12:08.352204+0000","last_fullsized":"2024-04-24T23:12:08.352204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:47.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850006+0000","last_change":"2024-04-24T23:10:16.447837+0000","last_active":"2024-04-24T23:12:07.850006+0000","last_peered":"2024-04-24T23:12:07.850006+0000","last_clean":"2024-04-24T23:12:07.850006+0000","last_became_active":"2024-04-24T23:10:16.447340+0000","last_became_peered":"2024-04-24T23:10:16.447340+0000","last_unstale":"2024-04-24T23:12:07.850006+0000","last_undegraded":"2024-04-24T23:12:07.850006+0000","last_fullsized":"2024-04-24T23:12:07.850006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:20.678595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372243+0000","last_change":"2024-04-24T23:10:11.321645+0000","last_active":"2024-04-24T23:12:08.372243+0000","last_peered":"2024-04-24T23:12:08.372243+0000","last_clean":"2024-04-24T23:12:08.372243+0000","last_became_active":"2024-04-24T23:10:11.321286+0000","last_became_peered":"2024-04-24T23:10:11.321286+0000","last_unstale":"2024-04-24T23:12:08.372243+0000","last_undegraded":"2024-04-24T23:12:08.372243+0000","last_fullsized":"2024-04-24T23:12:08.372243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:51.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850752+0000","last_change":"2024-04-24T23:10:12.338597+0000","last_active":"2024-04-24T23:12:07.850752+0000","last_peered":"2024-04-24T23:12:07.850752+0000","last_clean":"2024-04-24T23:12:07.850752+0000","last_became_active":"2024-04-24T23:10:12.338069+0000","last_became_peered":"2024-04-24T23:10:12.338069+0000","last_unstale":"2024-04-24T23:12:07.850752+0000","last_undegraded":"2024-04-24T23:12:07.850752+0000","last_fullsized":"2024-04-24T23:12:07.850752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:20.543211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854907+0000","last_change":"2024-04-24T23:10:16.376635+0000","last_active":"2024-04-24T23:12:07.854907+0000","last_peered":"2024-04-24T23:12:07.854907+0000","last_clean":"2024-04-24T23:12:07.854907+0000","last_became_active":"2024-04-24T23:10:16.375884+0000","last_became_peered":"2024-04-24T23:10:16.375884+0000","last_unstale":"2024-04-24T23:12:07.854907+0000","last_undegraded":"2024-04-24T23:12:07.854907+0000","last_fullsized":"2024-04-24T23:12:07.854907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:25.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.193280+0000","last_change":"2024-04-24T23:10:11.314135+0000","last_active":"2024-04-24T23:12:08.193280+0000","last_peered":"2024-04-24T23:12:08.193280+0000","last_clean":"2024-04-24T23:12:08.193280+0000","last_became_active":"2024-04-24T23:10:11.313732+0000","last_became_peered":"2024-04-24T23:10:11.313732+0000","last_unstale":"2024-04-24T23:12:08.193280+0000","last_undegraded":"2024-04-24T23:12:08.193280+0000","last_fullsized":"2024-04-24T23:12:08.193280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854403+0000","last_change":"2024-04-24T23:10:12.347021+0000","last_active":"2024-04-24T23:12:07.854403+0000","last_peered":"2024-04-24T23:12:07.854403+0000","last_clean":"2024-04-24T23:12:07.854403+0000","last_became_active":"2024-04-24T23:10:12.346680+0000","last_became_peered":"2024-04-24T23:10:12.346680+0000","last_unstale":"2024-04-24T23:12:07.854403+0000","last_undegraded":"2024-04-24T23:12:07.854403+0000","last_fullsized":"2024-04-24T23:12:07.854403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.165040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192900+0000","last_change":"2024-04-24T23:10:16.448858+0000","last_active":"2024-04-24T23:12:08.192900+0000","last_peered":"2024-04-24T23:12:08.192900+0000","last_clean":"2024-04-24T23:12:08.192900+0000","last_became_active":"2024-04-24T23:10:16.447753+0000","last_became_peered":"2024-04-24T23:10:16.447753+0000","last_unstale":"2024-04-24T23:12:08.192900+0000","last_undegraded":"2024-04-24T23:12:08.192900+0000","last_fullsized":"2024-04-24T23:12:08.192900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:43.422944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280633+0000","last_change":"2024-04-24T23:10:11.314874+0000","last_active":"2024-04-24T23:12:08.280633+0000","last_peered":"2024-04-24T23:12:08.280633+0000","last_clean":"2024-04-24T23:12:08.280633+0000","last_became_active":"2024-04-24T23:10:11.314333+0000","last_became_peered":"2024-04-24T23:10:11.314333+0000","last_unstale":"2024-04-24T23:12:08.280633+0000","last_undegraded":"2024-04-24T23:12:08.280633+0000","last_fullsized":"2024-04-24T23:12:08.280633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:03.126103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373314+0000","last_change":"2024-04-24T23:10:12.352070+0000","last_active":"2024-04-24T23:12:08.373314+0000","last_peered":"2024-04-24T23:12:08.373314+0000","last_clean":"2024-04-24T23:12:08.373314+0000","last_became_active":"2024-04-24T23:10:12.350643+0000","last_became_peered":"2024-04-24T23:10:12.350643+0000","last_unstale":"2024-04-24T23:12:08.373314+0000","last_undegraded":"2024-04-24T23:12:08.373314+0000","last_fullsized":"2024-04-24T23:12:08.373314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:03.180852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372313+0000","last_change":"2024-04-24T23:10:16.378390+0000","last_active":"2024-04-24T23:12:08.372313+0000","last_peered":"2024-04-24T23:12:08.372313+0000","last_clean":"2024-04-24T23:12:08.372313+0000","last_became_active":"2024-04-24T23:10:16.378005+0000","last_became_peered":"2024-04-24T23:10:16.378005+0000","last_unstale":"2024-04-24T23:12:08.372313+0000","last_undegraded":"2024-04-24T23:12:08.372313+0000","last_fullsized":"2024-04-24T23:12:08.372313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:39.775231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850177+0000","last_change":"2024-04-24T23:10:11.324117+0000","last_active":"2024-04-24T23:12:07.850177+0000","last_peered":"2024-04-24T23:12:07.850177+0000","last_clean":"2024-04-24T23:12:07.850177+0000","last_became_active":"2024-04-24T23:10:11.323746+0000","last_became_peered":"2024-04-24T23:10:11.323746+0000","last_unstale":"2024-04-24T23:12:07.850177+0000","last_undegraded":"2024-04-24T23:12:07.850177+0000","last_fullsized":"2024-04-24T23:12:07.850177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:24.368018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295519+0000","last_change":"2024-04-24T23:10:12.350257+0000","last_active":"2024-04-24T23:12:08.295519+0000","last_peered":"2024-04-24T23:12:08.295519+0000","last_clean":"2024-04-24T23:12:08.295519+0000","last_became_active":"2024-04-24T23:10:12.349894+0000","last_became_peered":"2024-04-24T23:10:12.349894+0000","last_unstale":"2024-04-24T23:12:08.295519+0000","last_undegraded":"2024-04-24T23:12:08.295519+0000","last_fullsized":"2024-04-24T23:12:08.295519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:17.763714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192876+0000","last_change":"2024-04-24T23:10:16.381507+0000","last_active":"2024-04-24T23:12:08.192876+0000","last_peered":"2024-04-24T23:12:08.192876+0000","last_clean":"2024-04-24T23:12:08.192876+0000","last_became_active":"2024-04-24T23:10:16.381052+0000","last_became_peered":"2024-04-24T23:10:16.381052+0000","last_unstale":"2024-04-24T23:12:08.192876+0000","last_undegraded":"2024-04-24T23:12:08.192876+0000","last_fullsized":"2024-04-24T23:12:08.192876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:33.351469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855100+0000","last_change":"2024-04-24T23:10:11.321872+0000","last_active":"2024-04-24T23:12:07.855100+0000","last_peered":"2024-04-24T23:12:07.855100+0000","last_clean":"2024-04-24T23:12:07.855100+0000","last_became_active":"2024-04-24T23:10:11.321553+0000","last_became_peered":"2024-04-24T23:10:11.321553+0000","last_unstale":"2024-04-24T23:12:07.855100+0000","last_undegraded":"2024-04-24T23:12:07.855100+0000","last_fullsized":"2024-04-24T23:12:07.855100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:56.809799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850423+0000","last_change":"2024-04-24T23:10:12.349264+0000","last_active":"2024-04-24T23:12:07.850423+0000","last_peered":"2024-04-24T23:12:07.850423+0000","last_clean":"2024-04-24T23:12:07.850423+0000","last_became_active":"2024-04-24T23:10:12.348071+0000","last_became_peered":"2024-04-24T23:10:12.348071+0000","last_unstale":"2024-04-24T23:12:07.850423+0000","last_undegraded":"2024-04-24T23:12:07.850423+0000","last_fullsized":"2024-04-24T23:12:07.850423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:39.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.192912+0000","last_change":"2024-04-24T23:10:16.448346+0000","last_active":"2024-04-24T23:12:08.192912+0000","last_peered":"2024-04-24T23:12:08.192912+0000","last_clean":"2024-04-24T23:12:08.192912+0000","last_became_active":"2024-04-24T23:10:16.447546+0000","last_became_peered":"2024-04-24T23:10:16.447546+0000","last_unstale":"2024-04-24T23:12:08.192912+0000","last_undegraded":"2024-04-24T23:12:08.192912+0000","last_fullsized":"2024-04-24T23:12:08.192912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:53.909720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372997+0000","last_change":"2024-04-24T23:10:11.327735+0000","last_active":"2024-04-24T23:12:08.372997+0000","last_peered":"2024-04-24T23:12:08.372997+0000","last_clean":"2024-04-24T23:12:08.372997+0000","last_became_active":"2024-04-24T23:10:11.327368+0000","last_became_peered":"2024-04-24T23:10:11.327368+0000","last_unstale":"2024-04-24T23:12:08.372997+0000","last_undegraded":"2024-04-24T23:12:08.372997+0000","last_fullsized":"2024-04-24T23:12:08.372997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:04.914576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372923+0000","last_change":"2024-04-24T23:10:12.338594+0000","last_active":"2024-04-24T23:12:08.372923+0000","last_peered":"2024-04-24T23:12:08.372923+0000","last_clean":"2024-04-24T23:12:08.372923+0000","last_became_active":"2024-04-24T23:10:12.338261+0000","last_became_peered":"2024-04-24T23:10:12.338261+0000","last_unstale":"2024-04-24T23:12:08.372923+0000","last_undegraded":"2024-04-24T23:12:08.372923+0000","last_fullsized":"2024-04-24T23:12:08.372923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:26.930847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280525+0000","last_change":"2024-04-24T23:10:16.376862+0000","last_active":"2024-04-24T23:12:08.280525+0000","last_peered":"2024-04-24T23:12:08.280525+0000","last_clean":"2024-04-24T23:12:08.280525+0000","last_became_active":"2024-04-24T23:10:16.376369+0000","last_became_peered":"2024-04-24T23:10:16.376369+0000","last_unstale":"2024-04-24T23:12:08.280525+0000","last_undegraded":"2024-04-24T23:12:08.280525+0000","last_fullsized":"2024-04-24T23:12:08.280525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:38.716231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280781+0000","last_change":"2024-04-24T23:11:07.579626+0000","last_active":"2024-04-24T23:12:08.280781+0000","last_peered":"2024-04-24T23:12:08.280781+0000","last_clean":"2024-04-24T23:12:08.280781+0000","last_became_active":"2024-04-24T23:11:07.574907+0000","last_became_peered":"2024-04-24T23:11:07.574907+0000","last_unstale":"2024-04-24T23:12:08.280781+0000","last_undegraded":"2024-04-24T23:12:08.280781+0000","last_fullsized":"2024-04-24T23:12:08.280781+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:56.428190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373144+0000","last_change":"2024-04-24T23:10:12.341473+0000","last_active":"2024-04-24T23:12:08.373144+0000","last_peered":"2024-04-24T23:12:08.373144+0000","last_clean":"2024-04-24T23:12:08.373144+0000","last_became_active":"2024-04-24T23:10:12.341117+0000","last_became_peered":"2024-04-24T23:10:12.341117+0000","last_unstale":"2024-04-24T23:12:08.373144+0000","last_undegraded":"2024-04-24T23:12:08.373144+0000","last_fullsized":"2024-04-24T23:12:08.373144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:18.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.375191+0000","last_change":"2024-04-24T23:12:08.374812+0000","last_active":"2024-04-24T23:12:08.375191+0000","last_peered":"2024-04-24T23:12:08.375191+0000","last_clean":"2024-04-24T23:12:08.375191+0000","last_became_active":"2024-04-24T23:12:08.374353+0000","last_became_peered":"2024-04-24T23:12:08.374353+0000","last_unstale":"2024-04-24T23:12:08.375191+0000","last_undegraded":"2024-04-24T23:12:08.375191+0000","last_fullsized":"2024-04-24T23:12:08.375191+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:58.241681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.372501+0000","last_change":"2024-04-24T23:10:11.320904+0000","last_active":"2024-04-24T23:12:08.372501+0000","last_peered":"2024-04-24T23:12:08.372501+0000","last_clean":"2024-04-24T23:12:08.372501+0000","last_became_active":"2024-04-24T23:10:11.320379+0000","last_became_peered":"2024-04-24T23:10:11.320379+0000","last_unstale":"2024-04-24T23:12:08.372501+0000","last_undegraded":"2024-04-24T23:12:08.372501+0000","last_fullsized":"2024-04-24T23:12:08.372501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:31.882273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373096+0000","last_change":"2024-04-24T23:10:12.346171+0000","last_active":"2024-04-24T23:12:08.373096+0000","last_peered":"2024-04-24T23:12:08.373096+0000","last_clean":"2024-04-24T23:12:08.373096+0000","last_became_active":"2024-04-24T23:10:12.345789+0000","last_became_peered":"2024-04-24T23:10:12.345789+0000","last_unstale":"2024-04-24T23:12:08.373096+0000","last_undegraded":"2024-04-24T23:12:08.373096+0000","last_fullsized":"2024-04-24T23:12:08.373096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:46.193055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371788+0000","last_change":"2024-04-24T23:10:16.375784+0000","last_active":"2024-04-24T23:12:08.371788+0000","last_peered":"2024-04-24T23:12:08.371788+0000","last_clean":"2024-04-24T23:12:08.371788+0000","last_became_active":"2024-04-24T23:10:16.375404+0000","last_became_peered":"2024-04-24T23:10:16.375404+0000","last_unstale":"2024-04-24T23:12:08.371788+0000","last_undegraded":"2024-04-24T23:12:08.371788+0000","last_fullsized":"2024-04-24T23:12:08.371788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:15.342352+0000","last_clean_scrub_stamp":"2024-04-24T23:10:15.342352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:57.895339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352331+0000","last_change":"2024-04-24T23:10:11.313036+0000","last_active":"2024-04-24T23:12:08.352331+0000","last_peered":"2024-04-24T23:12:08.352331+0000","last_clean":"2024-04-24T23:12:08.352331+0000","last_became_active":"2024-04-24T23:10:11.312679+0000","last_became_peered":"2024-04-24T23:10:11.312679+0000","last_unstale":"2024-04-24T23:12:08.352331+0000","last_undegraded":"2024-04-24T23:12:08.352331+0000","last_fullsized":"2024-04-24T23:12:08.352331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:10.294734+0000","last_clean_scrub_stamp":"2024-04-24T23:10:10.294734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:30.585262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.373318+0000","last_change":"2024-04-24T23:10:12.337209+0000","last_active":"2024-04-24T23:12:08.373318+0000","last_peered":"2024-04-24T23:12:08.373318+0000","last_clean":"2024-04-24T23:12:08.373318+0000","last_became_active":"2024-04-24T23:10:12.336844+0000","last_became_peered":"2024-04-24T23:10:12.336844+0000","last_unstale":"2024-04-24T23:12:08.373318+0000","last_undegraded":"2024-04-24T23:12:08.373318+0000","last_fullsized":"2024-04-24T23:12:08.373318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:29.292956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295479+0000","last_change":"2024-04-24T23:10:12.342330+0000","last_active":"2024-04-24T23:12:08.295479+0000","last_peered":"2024-04-24T23:12:08.295479+0000","last_clean":"2024-04-24T23:12:08.295479+0000","last_became_active":"2024-04-24T23:10:12.341917+0000","last_became_peered":"2024-04-24T23:10:12.341917+0000","last_unstale":"2024-04-24T23:12:08.295479+0000","last_undegraded":"2024-04-24T23:12:08.295479+0000","last_fullsized":"2024-04-24T23:12:08.295479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.013265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855299+0000","last_change":"2024-04-24T23:10:12.344112+0000","last_active":"2024-04-24T23:12:07.855299+0000","last_peered":"2024-04-24T23:12:07.855299+0000","last_clean":"2024-04-24T23:12:07.855299+0000","last_became_active":"2024-04-24T23:10:12.341620+0000","last_became_peered":"2024-04-24T23:10:12.341620+0000","last_unstale":"2024-04-24T23:12:07.855299+0000","last_undegraded":"2024-04-24T23:12:07.855299+0000","last_fullsized":"2024-04-24T23:12:07.855299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.699534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.854984+0000","last_change":"2024-04-24T23:10:12.346102+0000","last_active":"2024-04-24T23:12:07.854984+0000","last_peered":"2024-04-24T23:12:07.854984+0000","last_clean":"2024-04-24T23:12:07.854984+0000","last_became_active":"2024-04-24T23:10:12.345703+0000","last_became_peered":"2024-04-24T23:10:12.345703+0000","last_unstale":"2024-04-24T23:12:07.854984+0000","last_undegraded":"2024-04-24T23:12:07.854984+0000","last_fullsized":"2024-04-24T23:12:07.854984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:42.575512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.853900+0000","last_change":"2024-04-24T23:10:12.347549+0000","last_active":"2024-04-24T23:12:07.853900+0000","last_peered":"2024-04-24T23:12:07.853900+0000","last_clean":"2024-04-24T23:12:07.853900+0000","last_became_active":"2024-04-24T23:10:12.344887+0000","last_became_peered":"2024-04-24T23:10:12.344887+0000","last_unstale":"2024-04-24T23:12:07.853900+0000","last_undegraded":"2024-04-24T23:12:07.853900+0000","last_fullsized":"2024-04-24T23:12:07.853900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.523954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352229+0000","last_change":"2024-04-24T23:10:12.344512+0000","last_active":"2024-04-24T23:12:08.352229+0000","last_peered":"2024-04-24T23:12:08.352229+0000","last_clean":"2024-04-24T23:12:08.352229+0000","last_became_active":"2024-04-24T23:10:12.344190+0000","last_became_peered":"2024-04-24T23:10:12.344190+0000","last_unstale":"2024-04-24T23:12:08.352229+0000","last_undegraded":"2024-04-24T23:12:08.352229+0000","last_fullsized":"2024-04-24T23:12:08.352229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295817+0000","last_change":"2024-04-24T23:10:12.336346+0000","last_active":"2024-04-24T23:12:08.295817+0000","last_peered":"2024-04-24T23:12:08.295817+0000","last_clean":"2024-04-24T23:12:08.295817+0000","last_became_active":"2024-04-24T23:10:12.336010+0000","last_became_peered":"2024-04-24T23:10:12.336010+0000","last_unstale":"2024-04-24T23:12:08.295817+0000","last_undegraded":"2024-04-24T23:12:08.295817+0000","last_fullsized":"2024-04-24T23:12:08.295817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:56.862869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.855547+0000","last_change":"2024-04-24T23:10:12.336181+0000","last_active":"2024-04-24T23:12:07.855547+0000","last_peered":"2024-04-24T23:12:07.855547+0000","last_clean":"2024-04-24T23:12:07.855547+0000","last_became_active":"2024-04-24T23:10:12.335755+0000","last_became_peered":"2024-04-24T23:10:12.335755+0000","last_unstale":"2024-04-24T23:12:07.855547+0000","last_undegraded":"2024-04-24T23:12:07.855547+0000","last_fullsized":"2024-04-24T23:12:07.855547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:04.507718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850816+0000","last_change":"2024-04-24T23:10:12.342544+0000","last_active":"2024-04-24T23:12:07.850816+0000","last_peered":"2024-04-24T23:12:07.850816+0000","last_clean":"2024-04-24T23:12:07.850816+0000","last_became_active":"2024-04-24T23:10:12.342206+0000","last_became_peered":"2024-04-24T23:10:12.342206+0000","last_unstale":"2024-04-24T23:12:07.850816+0000","last_undegraded":"2024-04-24T23:12:07.850816+0000","last_fullsized":"2024-04-24T23:12:07.850816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:07.167391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.371911+0000","last_change":"2024-04-24T23:10:12.344303+0000","last_active":"2024-04-24T23:12:08.371911+0000","last_peered":"2024-04-24T23:12:08.371911+0000","last_clean":"2024-04-24T23:12:08.371911+0000","last_became_active":"2024-04-24T23:10:12.343718+0000","last_became_peered":"2024-04-24T23:10:12.343718+0000","last_unstale":"2024-04-24T23:12:08.371911+0000","last_undegraded":"2024-04-24T23:12:08.371911+0000","last_fullsized":"2024-04-24T23:12:08.371911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:57.828583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:07.850318+0000","last_change":"2024-04-24T23:10:12.335099+0000","last_active":"2024-04-24T23:12:07.850318+0000","last_peered":"2024-04-24T23:12:07.850318+0000","last_clean":"2024-04-24T23:12:07.850318+0000","last_became_active":"2024-04-24T23:10:12.334753+0000","last_became_peered":"2024-04-24T23:10:12.334753+0000","last_unstale":"2024-04-24T23:12:07.850318+0000","last_undegraded":"2024-04-24T23:12:07.850318+0000","last_fullsized":"2024-04-24T23:12:07.850318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:52.256402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.280809+0000","last_change":"2024-04-24T23:10:12.342035+0000","last_active":"2024-04-24T23:12:08.280809+0000","last_peered":"2024-04-24T23:12:08.280809+0000","last_clean":"2024-04-24T23:12:08.280809+0000","last_became_active":"2024-04-24T23:10:12.341491+0000","last_became_peered":"2024-04-24T23:10:12.341491+0000","last_unstale":"2024-04-24T23:12:08.280809+0000","last_undegraded":"2024-04-24T23:12:08.280809+0000","last_fullsized":"2024-04-24T23:12:08.280809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:15.380605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.352027+0000","last_change":"2024-04-24T23:10:12.338035+0000","last_active":"2024-04-24T23:12:08.352027+0000","last_peered":"2024-04-24T23:12:08.352027+0000","last_clean":"2024-04-24T23:12:08.352027+0000","last_became_active":"2024-04-24T23:10:12.337708+0000","last_became_peered":"2024-04-24T23:10:12.337708+0000","last_unstale":"2024-04-24T23:12:08.352027+0000","last_undegraded":"2024-04-24T23:12:08.352027+0000","last_fullsized":"2024-04-24T23:12:08.352027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:12:08.295519+0000","last_change":"2024-04-24T23:10:12.345283+0000","last_active":"2024-04-24T23:12:08.295519+0000","last_peered":"2024-04-24T23:12:08.295519+0000","last_clean":"2024-04-24T23:12:08.295519+0000","last_became_active":"2024-04-24T23:10:12.344956+0000","last_became_peered":"2024-04-24T23:10:12.344956+0000","last_unstale":"2024-04-24T23:12:08.295519+0000","last_undegraded":"2024-04-24T23:12:08.295519+0000","last_fullsized":"2024-04-24T23:12:08.295519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:11.298479+0000","last_clean_scrub_stamp":"2024-04-24T23:10:11.298479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.937393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287366,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":739,"num_read_kb":447,"num_write":170,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1198,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":213861,"data_compressed":4821,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1178528,"data_compressed":64827,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":526964,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":522206,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1118838,"data_compressed":63220,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":518044,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":586632,"data_compressed":28292,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":581020,"data_compressed":28292,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":518238,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":523808,"data_compressed":26685,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1113894,"data_compressed":63220,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34897,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36535,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71198,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63066,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70206,"data_compressed":1607,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:40.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 -- 172.21.15.33:0/603508697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94241006a0 msgr2=0x7f9424100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 --2- 172.21.15.33:0/603508697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94241006a0 0x7f9424100ac0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f941c009900 tx=0x7f941c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 -- 172.21.15.33:0/603508697 shutdown_connections 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 --2- 172.21.15.33:0/603508697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424102250 0x7f9424108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 --2- 172.21.15.33:0/603508697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9424101890 0x7f9424101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 --2- 172.21.15.33:0/603508697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94241006a0 0x7f9424100ac0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 -- 172.21.15.33:0/603508697 >> 172.21.15.33:0/603508697 conn(0x7f94240fbc30 msgr2=0x7f94240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 -- 172.21.15.33:0/603508697 shutdown_connections 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.998+0000 7f94289a5640 1 -- 172.21.15.33:0/603508697 wait complete. 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 Processor -- start 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 -- start start 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94241006a0 0x7f94241969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424101890 0x7f9424196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 0x7f942419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f942410c370 con 0x7f9424102250 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f942410c1f0 con 0x7f9424101890 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f94289a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f942410c4f0 con 0x7f94241006a0 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 0x7f942419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 0x7f942419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48150/0 (socket says 172.21.15.33:48150) 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 -- 172.21.15.33:0/4054742180 learned_addr learned my addr 172.21.15.33:0/4054742180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9421d9b640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424101890 0x7f9424196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f942259c640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94241006a0 0x7f94241969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 -- 172.21.15.33:0/4054742180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94241006a0 msgr2=0x7f94241969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94241006a0 0x7f94241969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 -- 172.21.15.33:0/4054742180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424101890 msgr2=0x7f9424196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424101890 0x7f9424196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9422d9d640 1 -- 172.21.15.33:0/4054742180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942419e600 con 0x7f9424102250 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:40.999+0000 7f9421d9b640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424101890 0x7f9424196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.000+0000 7f9422d9d640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 0x7f942419df90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f941400cce0 tx=0x7f941400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.000+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941400f450 con 0x7f9424102250 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.000+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f941400fad0 con 0x7f9424102250 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.000+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942419e8f0 con 0x7f9424102250 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.000+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414018c10 con 0x7f9424102250 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.000+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942419ee70 con 0x7f9424102250 2024-04-24T23:12:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.001+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94241079c0 con 0x7f9424102250 2024-04-24T23:12:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.002+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f941400f5f0 con 0x7f9424102250 2024-04-24T23:12:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.002+0000 7f941b7fe640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94100a2c20 0x7f94100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.003+0000 7f942259c640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94100a2c20 0x7f94100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.003+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94140cfcd0 con 0x7f9424102250 2024-04-24T23:12:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.003+0000 7f942259c640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94100a2c20 0x7f94100a50e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f941c006010 tx=0x7f941c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.007+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9414014030 con 0x7f9424102250 2024-04-24T23:12:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.195+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9424062340 con 0x7f9424102250 2024-04-24T23:12:41.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.197+0000 7f941b7fe640 1 -- 172.21.15.33:0/4054742180 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7f941409c740 con 0x7f9424102250 2024-04-24T23:12:41.199 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:41.199 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94100a2c20 msgr2=0x7f94100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94100a2c20 0x7f94100a50e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f941c006010 tx=0x7f941c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 msgr2=0x7f942419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 0x7f942419df90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f941400cce0 tx=0x7f941400a620 comp rx=0 tx=0).stop 2024-04-24T23:12:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 shutdown_connections 2024-04-24T23:12:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94100a2c20 0x7f94100a50e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9424102250 0x7f942419df90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9424101890 0x7f9424196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 --2- 172.21.15.33:0/4054742180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94241006a0 0x7f94241969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 >> 172.21.15.33:0/4054742180 conn(0x7f94240fbc30 msgr2=0x7f9424104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 shutdown_connections 2024-04-24T23:12:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.199+0000 7f94289a5640 1 -- 172.21.15.33:0/4054742180 wait complete. 2024-04-24T23:12:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:40 smithi033 ceph-mon[29201]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T23:12:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:40 smithi033 ceph-mon[29201]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:40 smithi033 ceph-mon[29201]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:41.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:41.214 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:41.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 -- 172.21.15.33:0/2227052735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 msgr2=0x7f4fa0100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/2227052735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0100470 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009900 tx=0x7f4f9c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 -- 172.21.15.33:0/2227052735 shutdown_connections 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/2227052735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa0101350 0x7f4fa01017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/2227052735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa01009b0 0x7f4fa0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/2227052735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0100470 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.317+0000 7f4fa72bd640 1 -- 172.21.15.33:0/2227052735 >> 172.21.15.33:0/2227052735 conn(0x7f4fa00fbbf0 msgr2=0x7f4fa00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 -- 172.21.15.33:0/2227052735 shutdown_connections 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 -- 172.21.15.33:0/2227052735 wait complete. 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 Processor -- start 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 -- start start 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa01009b0 0x7f4fa019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa0101350 0x7f4fa01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa010f5c0 con 0x7f4fa0100050 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa010f440 con 0x7f4fa0101350 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa72bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa010f740 con 0x7f4fa01009b0 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa5059640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa5059640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48166/0 (socket says 172.21.15.33:48166) 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.318+0000 7f4fa5059640 1 -- 172.21.15.33:0/3954318456 learned_addr learned my addr 172.21.15.33:0/3954318456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa4858640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa01009b0 0x7f4fa019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa585a640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa0101350 0x7f4fa01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa5059640 1 -- 172.21.15.33:0/3954318456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa01009b0 msgr2=0x7f4fa019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa5059640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa01009b0 0x7f4fa019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa5059640 1 -- 172.21.15.33:0/3954318456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa0101350 msgr2=0x7f4fa01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa5059640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa0101350 0x7f4fa01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa5059640 1 -- 172.21.15.33:0/3954318456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa01a1920 con 0x7f4fa0100050 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa4858640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa01009b0 0x7f4fa019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa585a640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa0101350 0x7f4fa01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa5059640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0199d10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009230 tx=0x7f4f9c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c03f030 con 0x7f4fa0100050 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa01a1bb0 con 0x7f4fa0100050 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c003ce0 con 0x7f4fa0100050 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c035070 con 0x7f4fa0100050 2024-04-24T23:12:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.319+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa01a20f0 con 0x7f4fa0100050 2024-04-24T23:12:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.320+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa010b030 con 0x7f4fa0100050 2024-04-24T23:12:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.321+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4f9c042030 con 0x7f4fa0100050 2024-04-24T23:12:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.322+0000 7f4f967fc640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f8c0a2c20 0x7f4f8c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.322+0000 7f4fa4858640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f8c0a2c20 0x7f4f8c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.322+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4f9c0f0180 con 0x7f4fa0100050 2024-04-24T23:12:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.323+0000 7f4fa4858640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f8c0a2c20 0x7f4f8c0a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4f90005d00 tx=0x7f4f90005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.326+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f9c0bcb90 con 0x7f4fa0100050 2024-04-24T23:12:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:40 smithi038 ceph-mon[32450]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T23:12:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:40 smithi038 ceph-mon[32450]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:41.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:40 smithi038 ceph-mon[32450]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:40 smithi117 ceph-mon[32529]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T23:12:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:40 smithi117 ceph-mon[32529]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:40 smithi117 ceph-mon[32529]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:41.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.509+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fa002d170 con 0x7f4fa0100050 2024-04-24T23:12:41.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.509+0000 7f4f967fc640 1 -- 172.21.15.33:0/3954318456 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f4fa002d170 con 0x7f4fa0100050 2024-04-24T23:12:41.512 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:12:41.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f8c0a2c20 msgr2=0x7f4f8c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f8c0a2c20 0x7f4f8c0a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4f90005d00 tx=0x7f4f90005c70 comp rx=0 tx=0).stop 2024-04-24T23:12:41.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 msgr2=0x7f4fa0199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0199d10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009230 tx=0x7f4f9c004360 comp rx=0 tx=0).stop 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 shutdown_connections 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f8c0a2c20 0x7f4f8c0a50e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fa0101350 0x7f4fa01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa01009b0 0x7f4fa019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 --2- 172.21.15.33:0/3954318456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa0100050 0x7f4fa0199d10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 >> 172.21.15.33:0/3954318456 conn(0x7f4fa00fbbf0 msgr2=0x7f4fa00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 shutdown_connections 2024-04-24T23:12:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.511+0000 7f4fa72bd640 1 -- 172.21.15.33:0/3954318456 wait complete. 2024-04-24T23:12:41.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 -- 172.21.15.33:0/2690020544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 msgr2=0x7f118c104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 --2- 172.21.15.33:0/2690020544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c104670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 -- 172.21.15.33:0/2690020544 shutdown_connections 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 --2- 172.21.15.33:0/2690020544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f118c1074e0 0x7f118c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 --2- 172.21.15.33:0/2690020544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c104bb0 0x7f118c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 --2- 172.21.15.33:0/2690020544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c104670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 -- 172.21.15.33:0/2690020544 >> 172.21.15.33:0/2690020544 conn(0x7f118c0fbc30 msgr2=0x7f118c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 -- 172.21.15.33:0/2690020544 shutdown_connections 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.627+0000 7f1192c7d640 1 -- 172.21.15.33:0/2690020544 wait complete. 2024-04-24T23:12:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 Processor -- start 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 -- start start 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f118c104bb0 0x7f118c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c1074e0 0x7f118c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f118c10d440 con 0x7f118c102280 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f118c10d2c0 con 0x7f118c1074e0 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1192c7d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f118c10d5c0 con 0x7f118c104bb0 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1190a19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1190a19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48176/0 (socket says 172.21.15.33:48176) 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f1190a19640 1 -- 172.21.15.33:0/4209989957 learned_addr learned my addr 172.21.15.33:0/4209989957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.628+0000 7f118bfff640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f118c104bb0 0x7f118c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f119121a640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c1074e0 0x7f118c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1190a19640 1 -- 172.21.15.33:0/4209989957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f118c104bb0 msgr2=0x7f118c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1190a19640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f118c104bb0 0x7f118c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1190a19640 1 -- 172.21.15.33:0/4209989957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c1074e0 msgr2=0x7f118c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1190a19640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c1074e0 0x7f118c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1190a19640 1 -- 172.21.15.33:0/4209989957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c19f6d0 con 0x7f118c102280 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f119121a640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c1074e0 0x7f118c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1190a19640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c197ab0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1180009230 tx=0x7f11800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118003f030 con 0x7f118c102280 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c19f960 con 0x7f118c102280 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1180003ce0 con 0x7f118c102280 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180035070 con 0x7f118c102280 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.629+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c19fe50 con 0x7f118c102280 2024-04-24T23:12:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.630+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118c0fd720 con 0x7f118c102280 2024-04-24T23:12:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.631+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f1180042030 con 0x7f118c102280 2024-04-24T23:12:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.632+0000 7f1189ffb640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f117c0a2c20 0x7f117c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.632+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11800efb40 con 0x7f118c102280 2024-04-24T23:12:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.632+0000 7f118bfff640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f117c0a2c20 0x7f117c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.632+0000 7f118bfff640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f117c0a2c20 0x7f117c0a50e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f118c198fc0 tx=0x7f1178005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.636+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f11800bc550 con 0x7f118c102280 2024-04-24T23:12:41.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.824+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f118c02d400 con 0x7f118c102280 2024-04-24T23:12:41.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.825+0000 7f1189ffb640 1 -- 172.21.15.33:0/4209989957 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f11800bbe50 con 0x7f118c102280 2024-04-24T23:12:41.826 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f117c0a2c20 msgr2=0x7f117c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f117c0a2c20 0x7f117c0a50e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f118c198fc0 tx=0x7f1178005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 msgr2=0x7f118c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c197ab0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1180009230 tx=0x7f11800042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 shutdown_connections 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f117c0a2c20 0x7f117c0a50e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f118c1074e0 0x7f118c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f118c104bb0 0x7f118c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 --2- 172.21.15.33:0/4209989957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f118c102280 0x7f118c197ab0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 >> 172.21.15.33:0/4209989957 conn(0x7f118c0fbc30 msgr2=0x7f118c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 shutdown_connections 2024-04-24T23:12:41.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.827+0000 7f1192c7d640 1 -- 172.21.15.33:0/4209989957 wait complete. 2024-04-24T23:12:41.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 -- 172.21.15.33:0/1930853014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408102250 msgr2=0x7f7408108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 --2- 172.21.15.33:0/1930853014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408102250 0x7f7408108ae0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7400009ca0 tx=0x7f740002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 -- 172.21.15.33:0/1930853014 shutdown_connections 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 --2- 172.21.15.33:0/1930853014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408102250 0x7f7408108ae0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 --2- 172.21.15.33:0/1930853014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7408101890 0x7f7408101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 --2- 172.21.15.33:0/1930853014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74081006a0 0x7f7408100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 -- 172.21.15.33:0/1930853014 >> 172.21.15.33:0/1930853014 conn(0x7f74080fbc30 msgr2=0x7f74080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 -- 172.21.15.33:0/1930853014 shutdown_connections 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.943+0000 7f740d342640 1 -- 172.21.15.33:0/1930853014 wait complete. 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 Processor -- start 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 -- start start 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74081006a0 0x7f7408196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 0x7f7408196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7408102250 0x7f740819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f740810c320 con 0x7f7408101890 2024-04-24T23:12:41.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f740810c1a0 con 0x7f7408102250 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f740d342640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f740810c4a0 con 0x7f74081006a0 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 0x7f7408196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 0x7f7408196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48196/0 (socket says 172.21.15.33:48196) 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 -- 172.21.15.33:0/3147102386 learned_addr learned my addr 172.21.15.33:0/3147102386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f7406ffd640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74081006a0 0x7f7408196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 -- 172.21.15.33:0/3147102386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74081006a0 msgr2=0x7f7408196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74081006a0 0x7f7408196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 -- 172.21.15.33:0/3147102386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7408102250 msgr2=0x7f740819df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f74077fe640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7408102250 0x7f740819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7408102250 0x7f740819df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 -- 172.21.15.33:0/3147102386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740819e580 con 0x7f7408101890 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f7406ffd640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74081006a0 0x7f7408196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.944+0000 7f73fffff640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 0x7f7408196ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d770 tx=0x7f73f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.945+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0017070 con 0x7f7408101890 2024-04-24T23:12:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.945+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0005500 con 0x7f7408101890 2024-04-24T23:12:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.945+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740819e870 con 0x7f7408101890 2024-04-24T23:12:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.945+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0004e00 con 0x7f7408101890 2024-04-24T23:12:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.945+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740819edc0 con 0x7f7408101890 2024-04-24T23:12:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.946+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74081079c0 con 0x7f7408101890 2024-04-24T23:12:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.947+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f73f0004fa0 con 0x7f7408101890 2024-04-24T23:12:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.948+0000 7f7404ff9640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73f40a2ca0 0x7f73f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.948+0000 7f7406ffd640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73f40a2ca0 0x7f73f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.948+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f73f00cf450 con 0x7f7408101890 2024-04-24T23:12:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.948+0000 7f7406ffd640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73f40a2ca0 0x7f73f40a5160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f74081016f0 tx=0x7f73f800a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:41.951+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73f009bf40 con 0x7f7408101890 2024-04-24T23:12:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.135+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f740802d300 con 0x7f7408101890 2024-04-24T23:12:42.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.136+0000 7f7404ff9640 1 -- 172.21.15.33:0/3147102386 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f73f009b840 con 0x7f7408101890 2024-04-24T23:12:42.138 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73f40a2ca0 msgr2=0x7f73f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73f40a2ca0 0x7f73f40a5160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f74081016f0 tx=0x7f73f800a560 comp rx=0 tx=0).stop 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 msgr2=0x7f7408196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 0x7f7408196ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d770 tx=0x7f73f000dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 shutdown_connections 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73f40a2ca0 0x7f73f40a5160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7408102250 0x7f740819df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7408101890 0x7f7408196ea0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 --2- 172.21.15.33:0/3147102386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74081006a0 0x7f7408196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 >> 172.21.15.33:0/3147102386 conn(0x7f74080fbc30 msgr2=0x7f7408104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 shutdown_connections 2024-04-24T23:12:42.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.138+0000 7f740d342640 1 -- 172.21.15.33:0/3147102386 wait complete. 2024-04-24T23:12:42.150 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:41 smithi033 ceph-mon[29201]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:42.150 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:41 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4054742180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:42.150 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:41 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3954318456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.150 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:41 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4209989957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 -- 172.21.15.33:0/1381191452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 msgr2=0x7f5928101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 --2- 172.21.15.33:0/1381191452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928101d10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5918009900 tx=0x7f591802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 -- 172.21.15.33:0/1381191452 shutdown_connections 2024-04-24T23:12:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 --2- 172.21.15.33:0/1381191452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5928102250 0x7f5928108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 --2- 172.21.15.33:0/1381191452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928101d10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 --2- 172.21.15.33:0/1381191452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59281006a0 0x7f5928100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 -- 172.21.15.33:0/1381191452 >> 172.21.15.33:0/1381191452 conn(0x7f59280fbc30 msgr2=0x7f59280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 -- 172.21.15.33:0/1381191452 shutdown_connections 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.255+0000 7f593036a640 1 -- 172.21.15.33:0/1381191452 wait complete. 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 Processor -- start 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 -- start start 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59281006a0 0x7f5928197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5928102250 0x7f592819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f592810c3d0 con 0x7f5928101890 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f592810c250 con 0x7f59281006a0 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f593036a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f592810c550 con 0x7f5928102250 2024-04-24T23:12:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592d905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592d905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48220/0 (socket says 172.21.15.33:48220) 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592d905640 1 -- 172.21.15.33:0/3270641788 learned_addr learned my addr 172.21.15.33:0/3270641788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592e907640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5928102250 0x7f592819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592e106640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59281006a0 0x7f5928197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592d905640 1 -- 172.21.15.33:0/3270641788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5928102250 msgr2=0x7f592819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.256+0000 7f592d905640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5928102250 0x7f592819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f592d905640 1 -- 172.21.15.33:0/3270641788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59281006a0 msgr2=0x7f5928197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f592d905640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59281006a0 0x7f5928197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f592d905640 1 -- 172.21.15.33:0/3270641788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592819f700 con 0x7f5928101890 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f592d905640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928198020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f591802aa00 tx=0x7f5918032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591803f030 con 0x7f5928101890 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5918003ce0 con 0x7f5928101890 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592819f990 con 0x7f5928101890 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918035070 con 0x7f5928101890 2024-04-24T23:12:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.257+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592819ff50 con 0x7f5928101890 2024-04-24T23:12:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.258+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59281a0120 con 0x7f5928101890 2024-04-24T23:12:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.259+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5918042030 con 0x7f5928101890 2024-04-24T23:12:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.260+0000 7f591f7fe640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59140a2ca0 0x7f59140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.260+0000 7f592e106640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59140a2ca0 0x7f59140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.260+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59180efd60 con 0x7f5928101890 2024-04-24T23:12:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.260+0000 7f592e106640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59140a2ca0 0x7f59140a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f59240058c0 tx=0x7f5924005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.264+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59180bc770 con 0x7f5928101890 2024-04-24T23:12:42.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:41 smithi038 ceph-mon[32450]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:42.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:41 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4054742180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:42.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:41 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3954318456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:41 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4209989957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:41 smithi117 ceph-mon[32529]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:42.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:41 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4054742180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:42.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:41 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3954318456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:41 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4209989957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.451+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f59281a04b0 con 0x7f5928101890 2024-04-24T23:12:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.451+0000 7f591f7fe640 1 -- 172.21.15.33:0/3270641788 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f59180bc070 con 0x7f5928101890 2024-04-24T23:12:42.453 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:12:42.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59140a2ca0 msgr2=0x7f59140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59140a2ca0 0x7f59140a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f59240058c0 tx=0x7f5924005850 comp rx=0 tx=0).stop 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 msgr2=0x7f5928198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928198020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f591802aa00 tx=0x7f5918032da0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 shutdown_connections 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59140a2ca0 0x7f59140a5160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5928102250 0x7f592819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5928101890 0x7f5928198020 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 --2- 172.21.15.33:0/3270641788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59281006a0 0x7f5928197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 >> 172.21.15.33:0/3270641788 conn(0x7f59280fbc30 msgr2=0x7f5928069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 shutdown_connections 2024-04-24T23:12:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.454+0000 7f593036a640 1 -- 172.21.15.33:0/3270641788 wait complete. 2024-04-24T23:12:42.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 -- 172.21.15.33:0/3668532301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac101870 msgr2=0x7f15ac101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 --2- 172.21.15.33:0/3668532301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac101870 0x7f15ac101cf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009900 tx=0x7f15a002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 -- 172.21.15.33:0/3668532301 shutdown_connections 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 --2- 172.21.15.33:0/3668532301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac102230 0x7f15ac108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 --2- 172.21.15.33:0/3668532301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac101870 0x7f15ac101cf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 --2- 172.21.15.33:0/3668532301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15ac100680 0x7f15ac100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 -- 172.21.15.33:0/3668532301 >> 172.21.15.33:0/3668532301 conn(0x7f15ac0fbbf0 msgr2=0x7f15ac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 -- 172.21.15.33:0/3668532301 shutdown_connections 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.567+0000 7f15b19f9640 1 -- 172.21.15.33:0/3668532301 wait complete. 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 Processor -- start 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 -- start start 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 0x7f15ac104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac101870 0x7f15ac102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15ac102230 0x7f15ac103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10c330 con 0x7f15ac100680 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10c1b0 con 0x7f15ac101870 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15b19f9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10c4b0 con 0x7f15ac102230 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 0x7f15ac104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 0x7f15ac104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48240/0 (socket says 172.21.15.33:48240) 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.568+0000 7f15aaffd640 1 -- 172.21.15.33:0/2079479443 learned_addr learned my addr 172.21.15.33:0/2079479443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15ab7fe640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15ac102230 0x7f15ac103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aa7fc640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac101870 0x7f15ac102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aaffd640 1 -- 172.21.15.33:0/2079479443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15ac102230 msgr2=0x7f15ac103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aaffd640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15ac102230 0x7f15ac103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aaffd640 1 -- 172.21.15.33:0/2079479443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac101870 msgr2=0x7f15ac102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aaffd640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac101870 0x7f15ac102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aaffd640 1 -- 172.21.15.33:0/2079479443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac103a20 con 0x7f15ac100680 2024-04-24T23:12:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15aaffd640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 0x7f15ac104610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f159c00d770 tx=0x7f159c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c017070 con 0x7f15ac100680 2024-04-24T23:12:42.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac1a1bf0 con 0x7f15ac100680 2024-04-24T23:12:42.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c009c30 con 0x7f15ac100680 2024-04-24T23:12:42.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c003dd0 con 0x7f15ac100680 2024-04-24T23:12:42.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.569+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a2190 con 0x7f15ac100680 2024-04-24T23:12:42.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.570+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac1a2360 con 0x7f15ac100680 2024-04-24T23:12:42.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.571+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f159c003f70 con 0x7f15ac100680 2024-04-24T23:12:42.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.572+0000 7f15b09f7640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15940a2c20 0x7f15940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.572+0000 7f15aa7fc640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15940a2c20 0x7f15940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.572+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f159c0ceff0 con 0x7f15ac100680 2024-04-24T23:12:42.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.572+0000 7f15aa7fc640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15940a2c20 0x7f15940a50e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15a0031610 tx=0x7f15a0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.577+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f159c09ba40 con 0x7f15ac100680 2024-04-24T23:12:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.800+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15ac1a2530 con 0x7f15ac100680 2024-04-24T23:12:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.800+0000 7f15b09f7640 1 -- 172.21.15.33:0/2079479443 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f159c009de0 con 0x7f15ac100680 2024-04-24T23:12:42.802 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:42.802 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:12:35.176325+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:12:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-24T23:12:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15940a2c20 msgr2=0x7f15940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15940a2c20 0x7f15940a50e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15a0031610 tx=0x7f15a0005fd0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 msgr2=0x7f15ac104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 0x7f15ac104610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f159c00d770 tx=0x7f159c00dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 shutdown_connections 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15940a2c20 0x7f15940a50e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15ac102230 0x7f15ac103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac101870 0x7f15ac102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 --2- 172.21.15.33:0/2079479443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15ac100680 0x7f15ac104610 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 >> 172.21.15.33:0/2079479443 conn(0x7f15ac0fbbf0 msgr2=0x7f15ac0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 shutdown_connections 2024-04-24T23:12:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.802+0000 7f15b19f9640 1 -- 172.21.15.33:0/2079479443 wait complete. 2024-04-24T23:12:42.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/1331486854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 msgr2=0x7fd2cc100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/1331486854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc100ac0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009900 tx=0x7fd2bc02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/1331486854 shutdown_connections 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/1331486854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 0x7fd2cc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/1331486854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc101890 0x7fd2cc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/1331486854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc100ac0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/1331486854 >> 172.21.15.33:0/1331486854 conn(0x7fd2cc0fbc30 msgr2=0x7fd2cc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.917+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/1331486854 shutdown_connections 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/1331486854 wait complete. 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 Processor -- start 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 -- start start 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc101890 0x7fd2cc198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 0x7fd2cc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc10c320 con 0x7fd2cc1006a0 2024-04-24T23:12:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc10c1a0 con 0x7fd2cc102250 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d2d9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc10c4a0 con 0x7fd2cc101890 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d0b3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d0b3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48260/0 (socket says 172.21.15.33:48260) 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2d0b3a640 1 -- 172.21.15.33:0/3777569655 learned_addr learned my addr 172.21.15.33:0/3777569655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.918+0000 7fd2cbfff640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc101890 0x7fd2cc198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d133b640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 0x7fd2cc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d0b3a640 1 -- 172.21.15.33:0/3777569655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc101890 msgr2=0x7fd2cc198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d0b3a640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc101890 0x7fd2cc198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d0b3a640 1 -- 172.21.15.33:0/3777569655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 msgr2=0x7fd2cc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d0b3a640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 0x7fd2cc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d0b3a640 1 -- 172.21.15.33:0/3777569655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc19f730 con 0x7fd2cc1006a0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d133b640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 0x7fd2cc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d0b3a640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc197bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc031610 tx=0x7fd2bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc03f030 con 0x7fd2cc1006a0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc003ad0 con 0x7fd2cc1006a0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc19f9c0 con 0x7fd2cc1006a0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc035070 con 0x7fd2cc1006a0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.919+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc19feb0 con 0x7fd2cc1006a0 2024-04-24T23:12:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.920+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc1a0080 con 0x7fd2cc1006a0 2024-04-24T23:12:42.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.921+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd2bc042030 con 0x7fd2cc1006a0 2024-04-24T23:12:42.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.922+0000 7fd2c9ffb640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2b40a2c20 0x7fd2b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.922+0000 7fd2cbfff640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2b40a2c20 0x7fd2b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.922+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd2bc0efc50 con 0x7fd2cc1006a0 2024-04-24T23:12:42.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.923+0000 7fd2cbfff640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2b40a2c20 0x7fd2b40a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc199020 tx=0x7fd2c0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:42.927+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd2bc0bc6c0 con 0x7fd2cc1006a0 2024-04-24T23:12:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.113+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2cc1a0370 con 0x7fd2cc1006a0 2024-04-24T23:12:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.114+0000 7fd2c9ffb640 1 -- 172.21.15.33:0/3777569655 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7fd2bc0bbfc0 con 0x7fd2cc1006a0 2024-04-24T23:12:43.116 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:12:43.117 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2b40a2c20 msgr2=0x7fd2b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2b40a2c20 0x7fd2b40a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc199020 tx=0x7fd2c0005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 msgr2=0x7fd2cc197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc197bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc031610 tx=0x7fd2bc004270 comp rx=0 tx=0).stop 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 shutdown_connections 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2b40a2c20 0x7fd2b40a50e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2cc102250 0x7fd2cc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc101890 0x7fd2cc198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 --2- 172.21.15.33:0/3777569655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1006a0 0x7fd2cc197bd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 >> 172.21.15.33:0/3777569655 conn(0x7fd2cc0fbc30 msgr2=0x7fd2cc0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 shutdown_connections 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.116+0000 7fd2d2d9e640 1 -- 172.21.15.33:0/3777569655 wait complete. 2024-04-24T23:12:43.128 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:42 smithi033 ceph-mon[29201]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-24T23:12:43.128 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:42 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3147102386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.128 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:42 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3270641788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:42 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2079479443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:43.130 DEBUG:tasks.fwd_scrub:Yielding 2024-04-24T23:12:43.130 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T23:12:43.147 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:12:43.147 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:12:43.147 INFO:tasks.workunit:timeout=3h 2024-04-24T23:12:43.147 INFO:tasks.workunit:cleanup=True 2024-04-24T23:12:43.148 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:43.148 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-24T23:12:43.148 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:12:43.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:43.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-24 23:10:39.909153317 +0000 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-24 23:10:39.909153317 +0000 2024-04-24T23:12:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-24 23:12:34.486560905 +0000 2024-04-24T23:12:43.165 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-24 23:10:39.909153317 +0000 2024-04-24T23:12:43.165 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:43.165 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T23:12:43.240 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:12:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 -- 172.21.15.33:0/4203593708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028101870 msgr2=0x7f4028101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 --2- 172.21.15.33:0/4203593708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028101870 0x7f4028101cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f401c009900 tx=0x7f401c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 -- 172.21.15.33:0/4203593708 shutdown_connections 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 --2- 172.21.15.33:0/4203593708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4028102230 0x7f4028108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 --2- 172.21.15.33:0/4203593708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028101870 0x7f4028101cf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 --2- 172.21.15.33:0/4203593708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4028100680 0x7f4028100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 -- 172.21.15.33:0/4203593708 >> 172.21.15.33:0/4203593708 conn(0x7f40280fbbf0 msgr2=0x7f40280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 -- 172.21.15.33:0/4203593708 shutdown_connections 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.254+0000 7f402d629640 1 -- 172.21.15.33:0/4203593708 wait complete. 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 Processor -- start 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 -- start start 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 0x7f4028196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4028101870 0x7f4028196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4028102230 0x7f402819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f402810c300 con 0x7f4028100680 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f402810c180 con 0x7f4028101870 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f402d629640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f402810c480 con 0x7f4028102230 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 0x7f4028196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 0x7f4028196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48272/0 (socket says 172.21.15.33:48272) 2024-04-24T23:12:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.255+0000 7f4026ffd640 1 -- 172.21.15.33:0/1667036993 learned_addr learned my addr 172.21.15.33:0/1667036993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f40277fe640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4028102230 0x7f402819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f40267fc640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4028101870 0x7f4028196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f4026ffd640 1 -- 172.21.15.33:0/1667036993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4028102230 msgr2=0x7f402819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f4026ffd640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4028102230 0x7f402819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f4026ffd640 1 -- 172.21.15.33:0/1667036993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4028101870 msgr2=0x7f4028196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f4026ffd640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4028101870 0x7f4028196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f4026ffd640 1 -- 172.21.15.33:0/1667036993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402819e530 con 0x7f4028100680 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f4026ffd640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 0x7f4028196910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4018007b50 tx=0x7f401800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f400ffff640 1 -- 172.21.15.33:0/1667036993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018007c40 con 0x7f4028100680 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f400ffff640 1 -- 172.21.15.33:0/1667036993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4018007de0 con 0x7f4028100680 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4028068930 con 0x7f4028100680 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f400ffff640 1 -- 172.21.15.33:0/1667036993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401800adb0 con 0x7f4028100680 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.256+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4028068e50 con 0x7f4028100680 2024-04-24T23:12:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 Processor -- start 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f400ffff640 1 -- 172.21.15.33:0/1667036993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f401800a440 con 0x7f4028100680 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 -- start start 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf387780 0x55d3cf38c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf38ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3cf2ea2c0 con 0x55d3cf386d90 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d3cf292910 con 0x55d3cf387780 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f403040c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d3cf2e6280 con 0x55d3cf38c8f0 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48274/0 (socket says 172.21.15.33:48274) 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 -- 172.21.15.33:0/1211384187 learned_addr learned my addr 172.21.15.33:0/1211384187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40267fc640 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf38ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f4026ffd640 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf387780 0x55d3cf38c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f400ffff640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40100a2c20 0x7f40100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 -- 172.21.15.33:0/1211384187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 msgr2=0x55d3cf38ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf38ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 -- 172.21.15.33:0/1211384187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf387780 msgr2=0x55d3cf38c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf387780 0x55d3cf38c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f40277fe640 1 -- 172.21.15.33:0/1211384187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3cf2a3fb0 con 0x55d3cf386d90 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.259+0000 7f4026ffd640 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf387780 0x55d3cf38c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f400ffff640 1 -- 172.21.15.33:0/1667036993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40180d0c40 con 0x7f4028100680 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f40277fe640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40100a2c20 0x7f40100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f40277fe640 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3871b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4020010370 tx=0x7f4020010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebdc8dbe0a5faf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f400d7fa640 1 -- 172.21.15.33:0/1211384187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020022070 con 0x55d3cf386d90 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f400d7fa640 1 -- 172.21.15.33:0/1211384187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40200127b0 con 0x55d3cf386d90 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f400d7fa640 1 -- 172.21.15.33:0/1211384187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020016070 con 0x55d3cf386d90 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.260+0000 7f40277fe640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40100a2c20 0x7f40100a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f402001f000 tx=0x7f4020022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 -- 172.21.15.33:0/1211384187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 msgr2=0x55d3cf3871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3871b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4020010370 tx=0x7f4020010830 comp rx=0 tx=0).stop 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 -- 172.21.15.33:0/1211384187 shutdown_connections 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf38ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf387780 0x55d3cf38c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 --2- 172.21.15.33:0/1211384187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3871b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 -- 172.21.15.33:0/1211384187 >> 172.21.15.33:0/1211384187 conn(0x55d3cf34e950 msgr2=0x55d3cf382810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 -- 172.21.15.33:0/1211384187 shutdown_connections 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 -- 172.21.15.33:0/1211384187 wait complete. 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 -- start start 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.261+0000 7f403040c740 1 Processor -- start 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf386d90 0x55d3cf391190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf3916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf398710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3cf38f8a0 con 0x55d3cf387780 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d3cf2a5e90 con 0x55d3cf386d90 2024-04-24T23:12:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d3cf38fa20 con 0x55d3cf38c8f0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 10 client.0 _ll_register_callbacks cb 0x55d3cf2e1970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf3916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf386d90 0x55d3cf391190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf3916d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48288/0 (socket says 172.21.15.33:48288) 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 -- 172.21.15.33:0/353256894 learned_addr learned my addr 172.21.15.33:0/353256894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40277fe640 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf398710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 -- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 msgr2=0x55d3cf398710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf398710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 -- 172.21.15.33:0/353256894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf386d90 msgr2=0x55d3cf391190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf386d90 0x55d3cf391190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 -- 172.21.15.33:0/353256894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3cf0c67e0 con 0x55d3cf387780 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f40267fc640 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf3916d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f401c02a930 tx=0x7f401c031aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d9f05474fb738a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f400aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f400aff5640 1 -- 172.21.15.33:0/353256894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c035070 con 0x55d3cf387780 2024-04-24T23:12:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f400aff5640 1 -- 172.21.15.33:0/353256894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f401c004340 con 0x55d3cf387780 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.262+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40200127b0 con 0x55d3cf387780 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f400aff5640 1 -- 172.21.15.33:0/353256894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f401c04d050 con 0x55d3cf387780 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3cf126f70 con 0x55d3cf387780 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f400aff5640 1 -- 172.21.15.33:0/353256894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40200127b0 con 0x55d3cf387780 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f400aff5640 1 -- 172.21.15.33:0/353256894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f401c0035f0 con 0x55d3cf387780 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 20 client.15600 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf39cbb0 0x55d3cf39f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 4 client.15600 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:43.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3cf2be5a0 con 0x55d3cf39cbb0 2024-04-24T23:12:43.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.263+0000 7f4026ffd640 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf39cbb0 0x55d3cf39f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.264+0000 7f4026ffd640 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf39cbb0 0x55d3cf39f070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad61843304e9fba server_cookie=dd43efdd396521fb in_seq=0 out_seq=0 2024-04-24T23:12:43.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.264+0000 7f400aff5640 10 client.15600 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.264+0000 7f400aff5640 1 -- 172.21.15.33:0/353256894 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3cf2be5a0 con 0x55d3cf39cbb0 2024-04-24T23:12:43.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.264+0000 7f400aff5640 10 client.15600 handle_command_reply: tid=0 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 client.15600 shutdown 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 msgr2=0x55d3cf3916d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf3916d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f401c02a930 tx=0x7f401c031aa0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 shutdown_connections 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf39cbb0 0x55d3cf39f070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf38c8f0 0x55d3cf398710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf3916d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 --2- 172.21.15.33:0/353256894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf386d90 0x55d3cf391190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 >> 172.21.15.33:0/353256894 conn(0x55d3cf34e950 msgr2=0x55d3cf381950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f400aff5640 0 client.15600 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 shutdown_connections 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 1 -- 172.21.15.33:0/353256894 wait complete. 2024-04-24T23:12:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.265+0000 7f403040c740 20 client.15600 trim_cache size 0 max 16384 2024-04-24T23:12:43.276 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T23:12:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 Processor -- start 2024-04-24T23:12:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 -- start start 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf386d90 0x55d3cf3854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf385a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf385f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40281aa9b0 con 0x55d3cf387780 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40281a8280 con 0x55d3cf38c8f0 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.283+0000 7f403040c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40281a60d0 con 0x55d3cf386d90 2024-04-24T23:12:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf385a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf385a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48294/0 (socket says 172.21.15.33:48294) 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 -- 172.21.15.33:0/1497388929 learned_addr learned my addr 172.21.15.33:0/1497388929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f4026ffd640 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf386d90 0x55d3cf3854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40277fe640 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf385f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 -- 172.21.15.33:0/1497388929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf386d90 msgr2=0x55d3cf3854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf386d90 0x55d3cf3854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 -- 172.21.15.33:0/1497388929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 msgr2=0x55d3cf385f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf385f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 -- 172.21.15.33:0/1497388929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3cefbe170 con 0x55d3cf387780 2024-04-24T23:12:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f40267fc640 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf385a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f401c031440 tx=0x7f401c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc31e2392d49998 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f400cff9640 1 -- 172.21.15.33:0/1497388929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c035040 con 0x55d3cf387780 2024-04-24T23:12:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f400cff9640 1 -- 172.21.15.33:0/1497388929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401c003c00 con 0x55d3cf387780 2024-04-24T23:12:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.284+0000 7f400cff9640 1 -- 172.21.15.33:0/1497388929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c0446d0 con 0x55d3cf387780 2024-04-24T23:12:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 -- 172.21.15.33:0/1497388929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 msgr2=0x55d3cf385a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf385a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f401c031440 tx=0x7f401c009230 comp rx=0 tx=0).stop 2024-04-24T23:12:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 -- 172.21.15.33:0/1497388929 shutdown_connections 2024-04-24T23:12:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf385f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf387780 0x55d3cf385a10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 --2- 172.21.15.33:0/1497388929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf386d90 0x55d3cf3854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 -- 172.21.15.33:0/1497388929 >> 172.21.15.33:0/1497388929 conn(0x55d3cf34e950 msgr2=0x55d3cf388be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 -- 172.21.15.33:0/1497388929 shutdown_connections 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 -- 172.21.15.33:0/1497388929 wait complete. 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.285+0000 7f403040c740 1 -- start start 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 Processor -- start 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf387780 0x55d3cf3feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf405de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40281a6740 con 0x55d3cf386d90 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40281a7bb0 con 0x55d3cf38c8f0 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40281a6e10 con 0x55d3cf387780 2024-04-24T23:12:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 10 client.0 _ll_register_callbacks cb 0x7f401c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3fe860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48302/0 (socket says 172.21.15.33:48302) 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 -- 172.21.15.33:0/3515667155 learned_addr learned my addr 172.21.15.33:0/3515667155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f40277fe640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf405de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 -- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf387780 msgr2=0x55d3cf3feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf387780 0x55d3cf3feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 -- 172.21.15.33:0/3515667155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 msgr2=0x55d3cf405de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf405de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 -- 172.21.15.33:0/3515667155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3cf4063d0 con 0x55d3cf386d90 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f40277fe640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf405de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f4026ffd640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3fe860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f401809cfd0 tx=0x7f401800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed20b6d4492728c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f400aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f400aff5640 1 -- 172.21.15.33:0/3515667155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401809d040 con 0x55d3cf386d90 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f400aff5640 1 -- 172.21.15.33:0/3515667155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40180996b0 con 0x55d3cf386d90 2024-04-24T23:12:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.286+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401c003c00 con 0x55d3cf386d90 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f400aff5640 1 -- 172.21.15.33:0/3515667155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40180a13f0 con 0x55d3cf386d90 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3cf4077e0 con 0x55d3cf386d90 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f400aff5640 1 -- 172.21.15.33:0/3515667155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401c003c00 con 0x55d3cf386d90 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f400aff5640 1 -- 172.21.15.33:0/3515667155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4018014070 con 0x55d3cf386d90 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 20 client.15606 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf4081e0 0x55d3cf40a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 4 client.15606 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d3cf2f7800 con 0x55d3cf4081e0 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.287+0000 7f40267fc640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf4081e0 0x55d3cf40a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.288+0000 7f40267fc640 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf4081e0 0x55d3cf40a620 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2bfedfa02451085 server_cookie=7737432c3d5af2e5 in_seq=0 out_seq=0 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.288+0000 7f400aff5640 10 client.15606 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f400aff5640 1 -- 172.21.15.33:0/3515667155 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d3cf2f7800 con 0x55d3cf4081e0 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f400aff5640 10 client.15606 handle_command_reply: tid=0 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 client.15606 shutdown 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 msgr2=0x55d3cf3fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3fe860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f401809cfd0 tx=0x7f401800ef30 comp rx=0 tx=0).stop 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 shutdown_connections 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d3cf4081e0 0x55d3cf40a620 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3cf38c8f0 0x55d3cf405de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3cf387780 0x55d3cf3feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 --2- 172.21.15.33:0/3515667155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3cf386d90 0x55d3cf3fe860 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f400aff5640 0 client.15606 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 >> 172.21.15.33:0/3515667155 conn(0x55d3cf34e950 msgr2=0x55d3cf3810c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 shutdown_connections 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 1 -- 172.21.15.33:0/3515667155 wait complete. 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.289+0000 7f403040c740 20 client.15606 trim_cache size 0 max 16384 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "14466560-e98c-43a1-b1ca-5a5c143d6cc1", 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:12:43.299 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40100a2c20 msgr2=0x7f40100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40100a2c20 0x7f40100a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f402001f000 tx=0x7f4020022000 comp rx=0 tx=0).stop 2024-04-24T23:12:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 msgr2=0x7f4028196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 0x7f4028196910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4018007b50 tx=0x7f401800e420 comp rx=0 tx=0).stop 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 shutdown_connections 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40100a2c20 0x7f40100a50e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4028102230 0x7f402819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4028101870 0x7f4028196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 --2- 172.21.15.33:0/1667036993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4028100680 0x7f4028196910 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 >> 172.21.15.33:0/1667036993 conn(0x7f40280fbbf0 msgr2=0x7f4028104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 shutdown_connections 2024-04-24T23:12:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.290+0000 7f402d629640 1 -- 172.21.15.33:0/1667036993 wait complete. 2024-04-24T23:12:43.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:12:43.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:12:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:42 smithi038 ceph-mon[32450]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-24T23:12:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:42 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3147102386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:42 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3270641788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:42 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2079479443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:42 smithi117 ceph-mon[32529]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-24T23:12:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:42 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3147102386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:42 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3270641788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:42 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2079479443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 -- 172.21.15.33:0/3572585380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 msgr2=0x7f64ec1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 --2- 172.21.15.33:0/3572585380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec1098d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009ca0 tx=0x7f64e802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 -- 172.21.15.33:0/3572585380 shutdown_connections 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 --2- 172.21.15.33:0/3572585380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec1098d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 --2- 172.21.15.33:0/3572585380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ec104b50 0x7f64ec106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 --2- 172.21.15.33:0/3572585380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec102220 0x7f64ec104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 -- 172.21.15.33:0/3572585380 >> 172.21.15.33:0/3572585380 conn(0x7f64ec0fbc30 msgr2=0x7f64ec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 -- 172.21.15.33:0/3572585380 shutdown_connections 2024-04-24T23:12:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 -- 172.21.15.33:0/3572585380 wait complete. 2024-04-24T23:12:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 Processor -- start 2024-04-24T23:12:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.448+0000 7f64f3aba640 1 -- start start 2024-04-24T23:12:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f3aba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ec102220 0x7f64ec197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f3aba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec104b50 0x7f64ec198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f3aba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f3aba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec10d420 con 0x7f64ec107480 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f3aba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec10d2a0 con 0x7f64ec104b50 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f3aba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec10d5a0 con 0x7f64ec102220 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48306/0 (socket says 172.21.15.33:48306) 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 -- 172.21.15.33:0/1841143499 learned_addr learned my addr 172.21.15.33:0/1841143499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f1055640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec104b50 0x7f64ec198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f1856640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ec102220 0x7f64ec197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 -- 172.21.15.33:0/1841143499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ec102220 msgr2=0x7f64ec197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ec102220 0x7f64ec197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 -- 172.21.15.33:0/1841143499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec104b50 msgr2=0x7f64ec198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec104b50 0x7f64ec198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 -- 172.21.15.33:0/1841143499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec19f6e0 con 0x7f64ec107480 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f1055640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec104b50 0x7f64ec198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.449+0000 7f64f2057640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec19f070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f64e8031ee0 tx=0x7f64e8031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.450+0000 7f64e2ffd640 1 -- 172.21.15.33:0/1841143499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8003c30 con 0x7f64ec107480 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.450+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec19f970 con 0x7f64ec107480 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.450+0000 7f64e2ffd640 1 -- 172.21.15.33:0/1841143499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8003dd0 con 0x7f64ec107480 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.450+0000 7f64e2ffd640 1 -- 172.21.15.33:0/1841143499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e802fc10 con 0x7f64ec107480 2024-04-24T23:12:43.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.450+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec19fec0 con 0x7f64ec107480 2024-04-24T23:12:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.452+0000 7f64e2ffd640 1 -- 172.21.15.33:0/1841143499 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f64e80046c0 con 0x7f64ec107480 2024-04-24T23:12:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 Processor -- start 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 -- start start 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919240 0x558955919660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895591eda0 0x558955921230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55895587d6b0 con 0x558955919c30 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558955825880 con 0x558955919240 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f67ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558955879670 con 0x55895591eda0 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64e2ffd640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64d80a2c20 0x7f64d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f1856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f2057640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64d80a2c20 0x7f64d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f1856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48318/0 (socket says 172.21.15.33:48318) 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.453+0000 7f64f1856640 1 -- 172.21.15.33:0/2024393484 learned_addr learned my addr 172.21.15.33:0/2024393484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1055640 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919240 0x558955919660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1055640 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895591eda0 0x558955921230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64e2ffd640 1 -- 172.21.15.33:0/1841143499 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64e80f0700 con 0x7f64ec107480 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1856640 1 -- 172.21.15.33:0/2024393484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895591eda0 msgr2=0x558955921230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1856640 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895591eda0 0x558955921230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1856640 1 -- 172.21.15.33:0/2024393484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919240 msgr2=0x558955919660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1856640 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919240 0x558955919660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1856640 1 -- 172.21.15.33:0/2024393484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895565b7d0 con 0x558955919c30 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1055640 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895591eda0 0x558955921230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f2057640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64d80a2c20 0x7f64d80a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64e80bc820 tx=0x7f64e80b8720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64f1856640 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591e730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f64e400bd10 tx=0x7f64e40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d44a20294f248f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64d7fff640 1 -- 172.21.15.33:0/2024393484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004030 con 0x558955919c30 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64d7fff640 1 -- 172.21.15.33:0/2024393484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e40041d0 con 0x558955919c30 2024-04-24T23:12:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.454+0000 7f64d7fff640 1 -- 172.21.15.33:0/2024393484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40117c0 con 0x558955919c30 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 -- 172.21.15.33:0/2024393484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 msgr2=0x55895591e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591e730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f64e400bd10 tx=0x7f64e40107a0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 -- 172.21.15.33:0/2024393484 shutdown_connections 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895591eda0 0x558955921230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591e730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2024393484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919240 0x558955919660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 -- 172.21.15.33:0/2024393484 >> 172.21.15.33:0/2024393484 conn(0x5589559148b0 msgr2=0x558955914cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 -- 172.21.15.33:0/2024393484 shutdown_connections 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 -- 172.21.15.33:0/2024393484 wait complete. 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.455+0000 7f64f67ca740 1 -- start start 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 Processor -- start 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x5589559236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x558955923c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558955921d50 con 0x558955919c30 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558955838f50 con 0x55895591eda0 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558955921ed0 con 0x558955919240 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f67ca740 10 client.0 _ll_register_callbacks cb 0x558955874000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x558955923c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x558955923c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48328/0 (socket says 172.21.15.33:48328) 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 -- 172.21.15.33:0/2992015986 learned_addr learned my addr 172.21.15.33:0/2992015986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1055640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x5589559236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1055640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 -- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 msgr2=0x5589559236c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x5589559236c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 -- 172.21.15.33:0/2992015986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 msgr2=0x55895592ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1856640 1 -- 172.21.15.33:0/2992015986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589555913f0 con 0x558955919c30 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.456+0000 7f64f1055640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x5589559236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64f1856640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x558955923c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f64e4010ca0 tx=0x7f64e400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beb54715e3ed860 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64d57fa640 1 -- 172.21.15.33:0/2992015986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4029060 con 0x558955919c30 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64d57fa640 1 -- 172.21.15.33:0/2992015986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64e40035f0 con 0x558955919c30 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e40041d0 con 0x558955919c30 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64d57fa640 1 -- 172.21.15.33:0/2992015986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64e4030050 con 0x558955919c30 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64f67ca740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-04-24T23:12:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589555510f0 con 0x558955919c30 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.457+0000 7f64d57fa640 1 -- 172.21.15.33:0/2992015986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e40041d0 con 0x558955919c30 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64d57fa640 1 -- 172.21.15.33:0/2992015986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f64e40046c0 con 0x558955919c30 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f67ca740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f67ca740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f67ca740 20 client.15618 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55895592f240 0x558955931630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f67ca740 4 client.15618 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895588abf0 con 0x55895592f240 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f1055640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55895592f240 0x558955931630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64d57fa640 10 client.15618 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.458+0000 7f64f1055640 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55895592f240 0x558955931630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365477baedb00815 server_cookie=efa90b31c79b9a6c in_seq=0 out_seq=0 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64d57fa640 1 -- 172.21.15.33:0/2992015986 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55895588abf0 con 0x55895592f240 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64d57fa640 10 client.15618 handle_command_reply: tid=0 2024-04-24T23:12:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 client.15618 shutdown 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 msgr2=0x558955923c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x558955923c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f64e4010ca0 tx=0x7f64e400b950 comp rx=0 tx=0).stop 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 shutdown_connections 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55895592f240 0x558955931630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x558955923c00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64d57fa640 0 client.15618 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 --2- 172.21.15.33:0/2992015986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x5589559236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.459+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 >> 172.21.15.33:0/2992015986 conn(0x5589559148b0 msgr2=0x558955913af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.460+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 shutdown_connections 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.460+0000 7f64f67ca740 1 -- 172.21.15.33:0/2992015986 wait complete. 2024-04-24T23:12:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.460+0000 7f64f67ca740 20 client.15618 trim_cache size 0 max 16384 2024-04-24T23:12:43.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 Processor -- start 2024-04-24T23:12:43.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 -- start start 2024-04-24T23:12:43.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x55895591b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1acd20 con 0x558955919c30 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1aa5f0 con 0x55895591eda0 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f67ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1a8440 con 0x558955919240 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48334/0 (socket says 172.21.15.33:48334) 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 -- 172.21.15.33:0/3822821968 learned_addr learned my addr 172.21.15.33:0/3822821968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1055640 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x55895591b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1055640 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 -- 172.21.15.33:0/3822821968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 msgr2=0x55895591b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x55895591b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 -- 172.21.15.33:0/3822821968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 msgr2=0x55895592ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1856640 1 -- 172.21.15.33:0/3822821968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589559319a0 con 0x558955919c30 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.478+0000 7f64f1055640 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.479+0000 7f64f1856640 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591b5d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005d00 tx=0x7f64e40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b26960e17f22fca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.479+0000 7f64d77fe640 1 -- 172.21.15.33:0/3822821968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4029070 con 0x558955919c30 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.479+0000 7f64d77fe640 1 -- 172.21.15.33:0/3822821968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e4004590 con 0x558955919c30 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.479+0000 7f64d77fe640 1 -- 172.21.15.33:0/3822821968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011640 con 0x558955919c30 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- 172.21.15.33:0/3822821968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 msgr2=0x55895591b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591b5d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005d00 tx=0x7f64e40107a0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- 172.21.15.33:0/3822821968 shutdown_connections 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55895591eda0 0x55895592ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955919c30 0x55895591b5d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- 172.21.15.33:0/3822821968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x55895591b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- 172.21.15.33:0/3822821968 >> 172.21.15.33:0/3822821968 conn(0x5589559148b0 msgr2=0x5589559236c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- 172.21.15.33:0/3822821968 shutdown_connections 2024-04-24T23:12:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- 172.21.15.33:0/3822821968 wait complete. 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- start start 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 Processor -- start 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x558955928650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919c30 0x558955928b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 0x558955929ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1a8ab0 con 0x55895591eda0 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1a9f20 con 0x558955919c30 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1a9180 con 0x558955919240 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.480+0000 7f64f67ca740 10 client.0 _ll_register_callbacks cb 0x7f64dc00bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 0x558955929ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 0x558955929ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48342/0 (socket says 172.21.15.33:48342) 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 -- 172.21.15.33:0/1054008965 learned_addr learned my addr 172.21.15.33:0/1054008965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x558955928650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1856640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919c30 0x558955928b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 -- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 msgr2=0x558955928650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x558955928650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 -- 172.21.15.33:0/1054008965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919c30 msgr2=0x558955928b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919c30 0x558955928b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 -- 172.21.15.33:0/1054008965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895592a190 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f1055640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 0x558955929ba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64dc021010 tx=0x7f64e4003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29641a97b08be40c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64d57fa640 1 -- 172.21.15.33:0/1054008965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0128c0 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64d57fa640 1 -- 172.21.15.33:0/1054008965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64dc012a60 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e4004590 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.481+0000 7f64d57fa640 1 -- 172.21.15.33:0/1054008965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64dc012c60 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558955995b20 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64d57fa640 1 -- 172.21.15.33:0/1054008965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e4004590 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64d57fa640 1 -- 172.21.15.33:0/1054008965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f64dc02b3b0 con 0x55895591eda0 2024-04-24T23:12:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 20 client.15624 populate_metadata read hostname 'smithi033' 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558955996550 0x558955998970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 4 client.15624 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589559234b0 con 0x558955996550 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.482+0000 7f64f1856640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558955996550 0x558955998970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64d57fa640 10 client.15624 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f1856640 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558955996550 0x558955998970 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8479c4b69b1a122 server_cookie=8396fbc1cc7b1cf4 in_seq=0 out_seq=0 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64d57fa640 1 -- 172.21.15.33:0/1054008965 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5589559234b0 con 0x558955996550 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64d57fa640 10 client.15624 handle_command_reply: tid=0 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 client.15624 shutdown 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 msgr2=0x558955929ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 0x558955929ba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64dc021010 tx=0x7f64e4003d30 comp rx=0 tx=0).stop 2024-04-24T23:12:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 shutdown_connections 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558955996550 0x558955998970 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55895591eda0 0x558955929ba0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558955919c30 0x558955928b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64d57fa640 0 client.15624 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 --2- 172.21.15.33:0/1054008965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558955919240 0x558955928650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 >> 172.21.15.33:0/1054008965 conn(0x5589559148b0 msgr2=0x558955923660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 shutdown_connections 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 1 -- 172.21.15.33:0/1054008965 wait complete. 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.483+0000 7f64f67ca740 20 client.15624 trim_cache size 0 max 16384 2024-04-24T23:12:43.491 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "14466560-e98c-43a1-b1ca-5a5c143d6cc1": { 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "14466560-e98c-43a1-b1ca-5a5c143d6cc1", 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:12:43.492 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64d80a2c20 msgr2=0x7f64d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64d80a2c20 0x7f64d80a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64e80bc820 tx=0x7f64e80b8720 comp rx=0 tx=0).stop 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 msgr2=0x7f64ec19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec19f070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f64e8031ee0 tx=0x7f64e8031f10 comp rx=0 tx=0).stop 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 shutdown_connections 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64d80a2c20 0x7f64d80a50e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec107480 0x7f64ec19f070 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64ec104b50 0x7f64ec198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 --2- 172.21.15.33:0/1841143499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ec102220 0x7f64ec197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 >> 172.21.15.33:0/1841143499 conn(0x7f64ec0fbc30 msgr2=0x7f64ec102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 shutdown_connections 2024-04-24T23:12:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:12:43.484+0000 7f64f3aba640 1 -- 172.21.15.33:0/1841143499 wait complete. 2024-04-24T23:12:43.497 INFO:tasks.cephfs.filesystem:scrub status for tag:14466560-e98c-43a1-b1ca-5a5c143d6cc1 - {'path': '/', 'tag': '14466560-e98c-43a1-b1ca-5a5c143d6cc1', 'options': 'recursive,force'} 2024-04-24T23:12:44.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:43 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3777569655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:44.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:43 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3777569655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:44.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:43 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3777569655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:44 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:12:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:44 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:12:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:44 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:12:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:44 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:12:45.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:44 smithi117 ceph-mon[32529]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-24T23:12:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:44 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:12:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:44 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:12:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:44 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:12:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:44 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:12:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:44 smithi038 ceph-mon[32450]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-24T23:12:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:44 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:12:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:44 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:12:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:44 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:12:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:44 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:12:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:44 smithi033 ceph-mon[29201]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-24T23:12:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:46 smithi038 ceph-mon[32450]: pgmap v374: 129 pgs: 129 active+clean; 651 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-24T23:12:47.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:46 smithi117 ceph-mon[32529]: pgmap v374: 129 pgs: 129 active+clean; 651 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-24T23:12:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:46 smithi033 ceph-mon[29201]: pgmap v374: 129 pgs: 129 active+clean; 651 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-24T23:12:49.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:48 smithi038 ceph-mon[32450]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-24T23:12:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:48 smithi117 ceph-mon[32529]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-24T23:12:49.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:48 smithi033 ceph-mon[29201]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-24T23:12:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:50 smithi038 ceph-mon[32450]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-24T23:12:51.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:50 smithi117 ceph-mon[32529]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-24T23:12:51.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:50 smithi033 ceph-mon[29201]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-24T23:12:53.335 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:52 smithi038 ceph-mon[32450]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-24T23:12:53.350 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:52 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:52 smithi033 ceph-mon[29201]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-24T23:12:53.351 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:52 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:52 smithi117 ceph-mon[32529]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-24T23:12:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:52 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:55.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-24T23:12:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:55.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:55.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:55.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:55.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:55.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:12:57.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:56 smithi038 ceph-mon[32450]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T23:12:57.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:56 smithi117 ceph-mon[32529]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T23:12:57.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:56 smithi033 ceph-mon[29201]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T23:12:58.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:57 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:12:58.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:57 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:12:58.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:57 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:12:59.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:12:58 smithi038 ceph-mon[32450]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T23:12:59.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:12:58 smithi117 ceph-mon[32529]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T23:12:59.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:12:58 smithi033 ceph-mon[29201]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T23:13:01.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:00 smithi038 ceph-mon[32450]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:01.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:00 smithi117 ceph-mon[32529]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:01.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:00 smithi033 ceph-mon[29201]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:03.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:02 smithi038 ceph-mon[32450]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:03.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:02 smithi117 ceph-mon[32529]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:03.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:02 smithi033 ceph-mon[29201]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:05.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:04 smithi038 ceph-mon[32450]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:05.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:04 smithi117 ceph-mon[32529]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:05.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:04 smithi033 ceph-mon[29201]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:07.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:06 smithi038 ceph-mon[32450]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:07.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:06 smithi117 ceph-mon[32529]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:07.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:06 smithi033 ceph-mon[29201]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:09.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:08 smithi038 ceph-mon[32450]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:09.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:08 smithi117 ceph-mon[32529]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:09.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:08 smithi033 ceph-mon[29201]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:11.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:10 smithi038 ceph-mon[32450]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:10 smithi117 ceph-mon[32529]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:10 smithi033 ceph-mon[29201]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:12 smithi038 ceph-mon[32450]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:13.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:12 smithi117 ceph-mon[32529]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:13.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:12 smithi033 ceph-mon[29201]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:13.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:13:13.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:13:13.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 -- 172.21.15.33:0/1370425835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024101870 msgr2=0x7f0024101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 --2- 172.21.15.33:0/1370425835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024101870 0x7f0024101cf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0018009900 tx=0x7f001802a410 comp rx=0 tx=0).stop 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 -- 172.21.15.33:0/1370425835 shutdown_connections 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 --2- 172.21.15.33:0/1370425835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0024102230 0x7f0024108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 --2- 172.21.15.33:0/1370425835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024101870 0x7f0024101cf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 --2- 172.21.15.33:0/1370425835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0024100680 0x7f0024100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 -- 172.21.15.33:0/1370425835 >> 172.21.15.33:0/1370425835 conn(0x7f00240fbbf0 msgr2=0x7f00240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 -- 172.21.15.33:0/1370425835 shutdown_connections 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 -- 172.21.15.33:0/1370425835 wait complete. 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 Processor -- start 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.606+0000 7f0029c9a640 1 -- start start 2024-04-24T23:13:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0024100680 0x7f0024197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0024101870 0x7f0024198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 0x7f002419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f002410c300 con 0x7f0024102230 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f002410c180 con 0x7f0024100680 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f002410c480 con 0x7f0024101870 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 0x7f002419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 0x7f002419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36902/0 (socket says 172.21.15.33:36902) 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 -- 172.21.15.33:0/45307471 learned_addr learned my addr 172.21.15.33:0/45307471 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f00237fe640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0024100680 0x7f0024197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0022ffd640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0024101870 0x7f0024198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 -- 172.21.15.33:0/45307471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0024101870 msgr2=0x7f0024198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0024101870 0x7f0024198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 -- 172.21.15.33:0/45307471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0024100680 msgr2=0x7f0024197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0024100680 0x7f0024197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 -- 172.21.15.33:0/45307471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002419f710 con 0x7f0024102230 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0023fff640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 0x7f002419f0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f001c00eaa0 tx=0x7f001c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0020ff9640 1 -- 172.21.15.33:0/45307471 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c00cdb0 con 0x7f0024102230 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002419fa00 con 0x7f0024102230 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0020ff9640 1 -- 172.21.15.33:0/45307471 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f001c005020 con 0x7f0024102230 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0020ff9640 1 -- 172.21.15.33:0/45307471 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c0116b0 con 0x7f0024102230 2024-04-24T23:13:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.607+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002419fed0 con 0x7f0024102230 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f0020ff9640 1 -- 172.21.15.33:0/45307471 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f001c005560 con 0x7f0024102230 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 Processor -- start 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 -- start start 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa521a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa526b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa5271b0 0x564bfa529640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564bfa4816b0 con 0x564bfa521650 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564bfa429880 con 0x564bfa522040 2024-04-24T23:13:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f002c9a7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564bfa47d670 con 0x564bfa5271b0 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f0022ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa521a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f0020ff9640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f000c0a2c20 0x7f000c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f0022ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa521a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36916/0 (socket says 172.21.15.33:36916) 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.610+0000 7f0022ffd640 1 -- 172.21.15.33:0/1433856196 learned_addr learned my addr 172.21.15.33:0/1433856196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa5271b0 0x564bfa529640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f00237fe640 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa526b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0023fff640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f000c0a2c20 0x7f000c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0020ff9640 1 -- 172.21.15.33:0/45307471 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f001c0cf700 con 0x7f0024102230 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 -- 172.21.15.33:0/1433856196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa5271b0 msgr2=0x564bfa529640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa5271b0 0x564bfa529640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 -- 172.21.15.33:0/1433856196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 msgr2=0x564bfa526b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa526b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 -- 172.21.15.33:0/1433856196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfa25f7d0 con 0x564bfa521650 2024-04-24T23:13:13.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0022ffd640 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa521a70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0018031610 tx=0x7f0018005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508cf9766c6918fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0023fff640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f000c0a2c20 0x7f000c0a50e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f001c09b820 tx=0x7f001c0054d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0009ffb640 1 -- 172.21.15.33:0/1433856196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018035070 con 0x564bfa521650 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0009ffb640 1 -- 172.21.15.33:0/1433856196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018005510 con 0x564bfa521650 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.611+0000 7f0009ffb640 1 -- 172.21.15.33:0/1433856196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018004c00 con 0x564bfa521650 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 -- 172.21.15.33:0/1433856196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 msgr2=0x564bfa521a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa521a70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0018031610 tx=0x7f0018005fd0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 -- 172.21.15.33:0/1433856196 shutdown_connections 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa5271b0 0x564bfa529640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa526b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 --2- 172.21.15.33:0/1433856196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa521a70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 -- 172.21.15.33:0/1433856196 >> 172.21.15.33:0/1433856196 conn(0x564bfa5188b0 msgr2=0x564bfa518cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 -- 172.21.15.33:0/1433856196 shutdown_connections 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 -- 172.21.15.33:0/1433856196 wait complete. 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.612+0000 7f002c9a7740 1 -- start start 2024-04-24T23:13:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 Processor -- start 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa52b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa52bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 0x564bfa532f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564bfa52a100 con 0x564bfa5271b0 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564bfa43cf50 con 0x564bfa522040 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564bfa52a280 con 0x564bfa521650 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f002c9a7740 10 client.0 _ll_register_callbacks cb 0x564bfa478000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f0022ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 0x564bfa532f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f00237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa52bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f0022ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 0x564bfa532f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36932/0 (socket says 172.21.15.33:36932) 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f0022ffd640 1 -- 172.21.15.33:0/963606134 learned_addr learned my addr 172.21.15.33:0/963606134 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f0022ffd640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa52b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f00237fe640 1 -- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 msgr2=0x564bfa52b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f00237fe640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa52b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f00237fe640 1 -- 172.21.15.33:0/963606134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 msgr2=0x564bfa532f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f00237fe640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 0x564bfa532f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f00237fe640 1 -- 172.21.15.33:0/963606134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfa1953f0 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.613+0000 7f0022ffd640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 0x564bfa532f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f00237fe640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa52bf30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x564bfa522440 tx=0x7f0014010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330c086be9ca6ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f0002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f0002ffd640 1 -- 172.21.15.33:0/963606134 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014004240 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0018005510 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f0002ffd640 1 -- 172.21.15.33:0/963606134 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00140043e0 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f0002ffd640 1 -- 172.21.15.33:0/963606134 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f001400f8d0 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f002c9a7740 10 client.25214 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564bfa1550f0 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.614+0000 7f0002ffd640 1 -- 172.21.15.33:0/963606134 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018005510 con 0x564bfa522040 2024-04-24T23:13:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f0002ffd640 1 -- 172.21.15.33:0/963606134 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f001401a420 con 0x564bfa522040 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f002c9a7740 10 client.25214 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f002c9a7740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f002c9a7740 20 client.25214 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f002c9a7740 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa5378d0 0x564bfa539cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f002c9a7740 4 client.25214 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bfa48ebf0 con 0x564bfa5378d0 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f0022ffd640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa5378d0 0x564bfa539cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f0002ffd640 10 client.25214 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.615+0000 7f0022ffd640 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa5378d0 0x564bfa539cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6792db4b16ac772a server_cookie=f964d657b0d8cf64 in_seq=0 out_seq=0 2024-04-24T23:13:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f0002ffd640 1 -- 172.21.15.33:0/963606134 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bfa48ebf0 con 0x564bfa5378d0 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f0002ffd640 10 client.25214 handle_command_reply: tid=0 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 client.25214 shutdown 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 msgr2=0x564bfa52bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa52bf30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x564bfa522440 tx=0x7f0014010c80 comp rx=0 tx=0).stop 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 shutdown_connections 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa5378d0 0x564bfa539cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa5271b0 0x564bfa532f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f0002ffd640 0 client.25214 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa522040 0x564bfa52bf30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 --2- 172.21.15.33:0/963606134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa52b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 >> 172.21.15.33:0/963606134 conn(0x564bfa5188b0 msgr2=0x564bfa517af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 shutdown_connections 2024-04-24T23:13:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 1 -- 172.21.15.33:0/963606134 wait complete. 2024-04-24T23:13:13.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.616+0000 7f002c9a7740 20 client.25214 trim_cache size 0 max 16384 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 Processor -- start 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 -- start start 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa518c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa522040 0x564bfa532f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa51b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00241aaab0 con 0x564bfa521650 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00241a8380 con 0x564bfa5271b0 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f002c9a7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00241a61d0 con 0x564bfa522040 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa518c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa518c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36942/0 (socket says 172.21.15.33:36942) 2024-04-24T23:13:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 -- 172.21.15.33:0/3328025123 learned_addr learned my addr 172.21.15.33:0/3328025123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f00237fe640 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa522040 0x564bfa532f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 -- 172.21.15.33:0/3328025123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa522040 msgr2=0x564bfa532f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa522040 0x564bfa532f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 -- 172.21.15.33:0/3328025123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 msgr2=0x564bfa51b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa51b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f0022ffd640 1 -- 172.21.15.33:0/3328025123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfa50a090 con 0x564bfa521650 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.637+0000 7f00237fe640 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa522040 0x564bfa532f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.638+0000 7f0022ffd640 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa518c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f00180314e0 tx=0x7f0018031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178e6bd53059c608 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.638+0000 7f00097fa640 1 -- 172.21.15.33:0/3328025123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001805b070 con 0x564bfa521650 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.638+0000 7f00097fa640 1 -- 172.21.15.33:0/3328025123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018056bd0 con 0x564bfa521650 2024-04-24T23:13:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.638+0000 7f00097fa640 1 -- 172.21.15.33:0/3328025123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018035040 con 0x564bfa521650 2024-04-24T23:13:13.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.638+0000 7f002c9a7740 1 -- 172.21.15.33:0/3328025123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 msgr2=0x564bfa518c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.638+0000 7f002c9a7740 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa518c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f00180314e0 tx=0x7f0018031b30 comp rx=0 tx=0).stop 2024-04-24T23:13:13.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- 172.21.15.33:0/3328025123 shutdown_connections 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa51b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa522040 0x564bfa532f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 --2- 172.21.15.33:0/3328025123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa521650 0x564bfa518c90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- 172.21.15.33:0/3328025123 >> 172.21.15.33:0/3328025123 conn(0x564bfa5188b0 msgr2=0x564bfa5233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- 172.21.15.33:0/3328025123 shutdown_connections 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- 172.21.15.33:0/3328025123 wait complete. 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- start start 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 Processor -- start 2024-04-24T23:13:13.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa530c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 0x564bfa531140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa532150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00241a6840 con 0x564bfa522040 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00241a7cb0 con 0x564bfa5271b0 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00241a6f10 con 0x564bfa521650 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f002c9a7740 10 client.0 _ll_register_callbacks cb 0x7f0014010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f00237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 0x564bfa531140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f00237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 0x564bfa531140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36948/0 (socket says 172.21.15.33:36948) 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.639+0000 7f00237fe640 1 -- 172.21.15.33:0/2299448443 learned_addr learned my addr 172.21.15.33:0/2299448443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:13.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0022ffd640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa530c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0022ffd640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa532150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f00237fe640 1 -- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 msgr2=0x564bfa530c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f00237fe640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa530c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f00237fe640 1 -- 172.21.15.33:0/2299448443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 msgr2=0x564bfa532150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f00237fe640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa532150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f00237fe640 1 -- 172.21.15.33:0/2299448443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfa532740 con 0x564bfa522040 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0022ffd640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa532150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f00237fe640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 0x564bfa531140 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0014005710 tx=0x7f0014005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4ec4ca497ba065 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0002ffd640 1 -- 172.21.15.33:0/2299448443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001401a420 con 0x564bfa522040 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0002ffd640 1 -- 172.21.15.33:0/2299448443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0014016040 con 0x564bfa522040 2024-04-24T23:13:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0018056bd0 con 0x564bfa522040 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0002ffd640 1 -- 172.21.15.33:0/2299448443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0014036050 con 0x564bfa522040 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f002c9a7740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564bfa59e310 con 0x564bfa522040 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.640+0000 7f0002ffd640 1 -- 172.21.15.33:0/2299448443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018056bd0 con 0x564bfa522040 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f0002ffd640 1 -- 172.21.15.33:0/2299448443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0014003b20 con 0x564bfa522040 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f002c9a7740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f002c9a7740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f002c9a7740 20 client.15642 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f002c9a7740 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa59ed90 0x564bfa5a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f002c9a7740 4 client.15642 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:13:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bfa488f40 con 0x564bfa59ed90 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f0022ffd640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa59ed90 0x564bfa5a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f0002ffd640 10 client.15642 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.641+0000 7f0022ffd640 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa59ed90 0x564bfa5a11b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b855b6b12547b66 server_cookie=2fc886f86d08e1a5 in_seq=0 out_seq=0 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0002ffd640 1 -- 172.21.15.33:0/2299448443 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564bfa488f40 con 0x564bfa59ed90 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0002ffd640 10 client.15642 handle_command_reply: tid=0 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 client.15642 shutdown 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 msgr2=0x564bfa531140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 0x564bfa531140 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0014005710 tx=0x7f0014005740 comp rx=0 tx=0).stop 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 shutdown_connections 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564bfa59ed90 0x564bfa5a11b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564bfa5271b0 0x564bfa532150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bfa522040 0x564bfa531140 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0002ffd640 0 client.15642 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 --2- 172.21.15.33:0/2299448443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bfa521650 0x564bfa530c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 >> 172.21.15.33:0/2299448443 conn(0x564bfa5188b0 msgr2=0x564bfa517260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 shutdown_connections 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 1 -- 172.21.15.33:0/2299448443 wait complete. 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f002c9a7740 20 client.15642 trim_cache size 0 max 16384 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:13:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:13:13.652 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:13:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f000c0a2c20 msgr2=0x7f000c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0029c9a640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f000c0a2c20 0x7f000c0a50e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f001c09b820 tx=0x7f001c0054d0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 msgr2=0x7f002419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0029c9a640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 0x7f002419f0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f001c00eaa0 tx=0x7f001c00ef60 comp rx=0 tx=0).stop 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.642+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 shutdown_connections 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f000c0a2c20 0x7f000c0a50e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0024102230 0x7f002419f0a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0024101870 0x7f0024198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 --2- 172.21.15.33:0/45307471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0024100680 0x7f0024197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 >> 172.21.15.33:0/45307471 conn(0x7f00240fbbf0 msgr2=0x7f00240688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 shutdown_connections 2024-04-24T23:13:13.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.643+0000 7f0029c9a640 1 -- 172.21.15.33:0/45307471 wait complete. 2024-04-24T23:13:13.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:13:13.659 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 -- 172.21.15.33:0/2578783431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8102230 msgr2=0x7f47e8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 --2- 172.21.15.33:0/2578783431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8102230 0x7f47e8108ac0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009ca0 tx=0x7f47e002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 -- 172.21.15.33:0/2578783431 shutdown_connections 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 --2- 172.21.15.33:0/2578783431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8102230 0x7f47e8108ac0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 --2- 172.21.15.33:0/2578783431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e8101870 0x7f47e8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 --2- 172.21.15.33:0/2578783431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e8100680 0x7f47e8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.763+0000 7f47f0053640 1 -- 172.21.15.33:0/2578783431 >> 172.21.15.33:0/2578783431 conn(0x7f47e80fbbf0 msgr2=0x7f47e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 -- 172.21.15.33:0/2578783431 shutdown_connections 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 -- 172.21.15.33:0/2578783431 wait complete. 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 Processor -- start 2024-04-24T23:13:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 -- start start 2024-04-24T23:13:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e8100680 0x7f47e8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 0x7f47e8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e8102230 0x7f47e819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810c300 con 0x7f47e8101870 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810c180 con 0x7f47e8102230 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47f0053640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810c480 con 0x7f47e8100680 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.764+0000 7f47ed5ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 0x7f47e8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 0x7f47e8197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36970/0 (socket says 172.21.15.33:36970) 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 -- 172.21.15.33:0/1542615476 learned_addr learned my addr 172.21.15.33:0/1542615476 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47eddef640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e8100680 0x7f47e8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ee5f0640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e8102230 0x7f47e819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 -- 172.21.15.33:0/1542615476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e8100680 msgr2=0x7f47e8197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e8100680 0x7f47e8197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 -- 172.21.15.33:0/1542615476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e8102230 msgr2=0x7f47e819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e8102230 0x7f47e819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 -- 172.21.15.33:0/1542615476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e819f620 con 0x7f47e8101870 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47ed5ee640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 0x7f47e8197f40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c970 tx=0x7f47d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8007bb0 con 0x7f47e8101870 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e819f910 con 0x7f47e8101870 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d8007d50 con 0x7f47e8101870 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d800f450 con 0x7f47e8101870 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.765+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e819fe60 con 0x7f47e8101870 2024-04-24T23:13:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.766+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e81a00f0 con 0x7f47e8101870 2024-04-24T23:13:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.768+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f47d8005020 con 0x7f47e8101870 2024-04-24T23:13:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.768+0000 7f47deffd640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47d40a2c20 0x7f47d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.768+0000 7f47eddef640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47d40a2c20 0x7f47d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.768+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47d80cf340 con 0x7f47e8101870 2024-04-24T23:13:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.769+0000 7f47eddef640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47d40a2c20 0x7f47d40a50e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f47e81016d0 tx=0x7f47e4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:13.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:13.773+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f47d8014030 con 0x7f47e8101870 2024-04-24T23:13:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.013+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47e81a02c0 con 0x7f47e8101870 2024-04-24T23:13:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.014+0000 7f47deffd640 1 -- 172.21.15.33:0/1542615476 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f47d809bdb0 con 0x7f47e8101870 2024-04-24T23:13:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:13:14.017 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:12:35.176325+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:13:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-24T23:13:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47d40a2c20 msgr2=0x7f47d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47d40a2c20 0x7f47d40a50e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f47e81016d0 tx=0x7f47e4005f90 comp rx=0 tx=0).stop 2024-04-24T23:13:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 msgr2=0x7f47e8197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 0x7f47e8197f40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c970 tx=0x7f47d800ce30 comp rx=0 tx=0).stop 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 shutdown_connections 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47d40a2c20 0x7f47d40a50e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e8102230 0x7f47e819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8101870 0x7f47e8197f40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 --2- 172.21.15.33:0/1542615476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e8100680 0x7f47e8197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 >> 172.21.15.33:0/1542615476 conn(0x7f47e80fbbf0 msgr2=0x7f47e80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 shutdown_connections 2024-04-24T23:13:14.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.016+0000 7f47f0053640 1 -- 172.21.15.33:0/1542615476 wait complete. 2024-04-24T23:13:14.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:13:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 -- 172.21.15.33:0/2781024605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828100680 msgr2=0x7fa828100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/2781024605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828100680 0x7fa828100aa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa824009900 tx=0x7fa82402a410 comp rx=0 tx=0).stop 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 -- 172.21.15.33:0/2781024605 shutdown_connections 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/2781024605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa828102230 0x7fa828108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/2781024605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa828101870 0x7fa828101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/2781024605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828100680 0x7fa828100aa0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.136+0000 7fa82f8a5640 1 -- 172.21.15.33:0/2781024605 >> 172.21.15.33:0/2781024605 conn(0x7fa8280fbbf0 msgr2=0x7fa8280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 -- 172.21.15.33:0/2781024605 shutdown_connections 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 -- 172.21.15.33:0/2781024605 wait complete. 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 Processor -- start 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 -- start start 2024-04-24T23:13:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa828100680 0x7fa828069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa828101870 0x7fa82819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 0x7fa82819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa82810c2d0 con 0x7fa828102230 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa82810c150 con 0x7fa828100680 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82f8a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa82810c450 con 0x7fa828101870 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82de42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 0x7fa82819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82de42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 0x7fa82819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36974/0 (socket says 172.21.15.33:36974) 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.137+0000 7fa82de42640 1 -- 172.21.15.33:0/3588270652 learned_addr learned my addr 172.21.15.33:0/3588270652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa828101870 0x7fa82819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82d641640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa828100680 0x7fa828069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82de42640 1 -- 172.21.15.33:0/3588270652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa828101870 msgr2=0x7fa82819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82de42640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa828101870 0x7fa82819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82de42640 1 -- 172.21.15.33:0/3588270652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa828100680 msgr2=0x7fa828069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82de42640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa828100680 0x7fa828069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82de42640 1 -- 172.21.15.33:0/3588270652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82819f720 con 0x7fa828102230 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82de42640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 0x7fa82819ee10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa820009fd0 tx=0x7fa82000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa81e7fc640 1 -- 172.21.15.33:0/3588270652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820017070 con 0x7fa828102230 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa81e7fc640 1 -- 172.21.15.33:0/3588270652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820003e50 con 0x7fa828102230 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82819f9b0 con 0x7fa828102230 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa81e7fc640 1 -- 172.21.15.33:0/3588270652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820010460 con 0x7fa828102230 2024-04-24T23:13:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.138+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82819feb0 con 0x7fa828102230 2024-04-24T23:13:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.141+0000 7fa8325b3740 1 Processor -- start 2024-04-24T23:13:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.141+0000 7fa81e7fc640 1 -- 172.21.15.33:0/3588270652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa820003970 con 0x7fa828102230 2024-04-24T23:13:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.141+0000 7fa8325b3740 1 -- start start 2024-04-24T23:13:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.141+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac1698a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac169db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa8325b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fac15f8670 con 0x55fac169e1a0 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa8325b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fac15a0970 con 0x55fac1699030 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa8325b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fac15f4630 con 0x55fac1698640 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82d641640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac169db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36984/0 (socket says 172.21.15.33:36984) 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82ce40640 1 -- 172.21.15.33:0/3860454599 learned_addr learned my addr 172.21.15.33:0/3860454599 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac1698a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa81e7fc640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8140a2c20 0x7fa8140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82d641640 1 -- 172.21.15.33:0/3860454599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 msgr2=0x55fac1698a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82d641640 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac1698a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82de42640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8140a2c20 0x7fa8140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82d641640 1 -- 172.21.15.33:0/3860454599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 msgr2=0x55fac16a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82d641640 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82d641640 1 -- 172.21.15.33:0/3860454599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac15c4400 con 0x55fac1699030 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.142+0000 7fa81e7fc640 1 -- 172.21.15.33:0/3588270652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8200cf280 con 0x7fa828102230 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.143+0000 7fa82d641640 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac169db30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa824009230 tx=0x7fa824032d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a556665501d14b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.143+0000 7fa82de42640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8140a2c20 0x7fa8140a50e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa820096090 tx=0x7fa8200971f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.143+0000 7fa8137fe640 1 -- 172.21.15.33:0/3860454599 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82403f070 con 0x55fac1699030 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.143+0000 7fa8137fe640 1 -- 172.21.15.33:0/3860454599 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824031970 con 0x55fac1699030 2024-04-24T23:13:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.143+0000 7fa8137fe640 1 -- 172.21.15.33:0/3860454599 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824035070 con 0x55fac1699030 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 -- 172.21.15.33:0/3860454599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 msgr2=0x55fac169db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac169db30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa824009230 tx=0x7fa824032d80 comp rx=0 tx=0).stop 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 -- 172.21.15.33:0/3860454599 shutdown_connections 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac169db30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3860454599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac1698a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 -- 172.21.15.33:0/3860454599 >> 172.21.15.33:0/3860454599 conn(0x55fac168f8a0 msgr2=0x55fac168fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 -- 172.21.15.33:0/3860454599 shutdown_connections 2024-04-24T23:13:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 -- 172.21.15.33:0/3860454599 wait complete. 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 -- start start 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 Processor -- start 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.144+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 0x55fac16a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac169e1a0 0x55fac16a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fac16a10f0 con 0x55fac1698640 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fac15b4050 con 0x55fac1699030 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fac16a1270 con 0x55fac169e1a0 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 10 client.0 _ll_register_callbacks cb 0x55fac130bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 0x55fac16a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 0x55fac16a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37000/0 (socket says 172.21.15.33:37000) 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 -- 172.21.15.33:0/3881481229 learned_addr learned my addr 172.21.15.33:0/3881481229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82d641640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 -- 172.21.15.33:0/3881481229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac169e1a0 msgr2=0x55fac16a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac169e1a0 0x55fac16a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 -- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 msgr2=0x55fac16a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 -- 172.21.15.33:0/3881481229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac1585dc0 con 0x55fac1698640 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82d641640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 0x55fac16a2730 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa81800afc0 tx=0x7fa81800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e97aa5c5406f20c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa810ff9640 1 -- 172.21.15.33:0/3881481229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81800e670 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa810ff9640 1 -- 172.21.15.33:0/3881481229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa81800e810 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa824031970 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa810ff9640 1 -- 172.21.15.33:0/3881481229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa818028050 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.145+0000 7fa8325b3740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fac130c3f0 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa810ff9640 1 -- 172.21.15.33:0/3881481229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824031970 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa810ff9640 1 -- 172.21.15.33:0/3881481229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa81801faf0 con 0x55fac1698640 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:13:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 20 client.15660 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac16ae4e0 0x55fac16b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 4 client.15660 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac1605bb0 con 0x55fac16ae4e0 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.146+0000 7fa82d641640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac16ae4e0 0x55fac16b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.147+0000 7fa82d641640 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac16ae4e0 0x55fac16b09a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5840b9ea21da1e69 server_cookie=4a226a3f0a5245da in_seq=0 out_seq=0 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.147+0000 7fa810ff9640 10 client.15660 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.147+0000 7fa810ff9640 1 -- 172.21.15.33:0/3881481229 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fac1605bb0 con 0x55fac16ae4e0 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.147+0000 7fa810ff9640 10 client.15660 handle_command_reply: tid=0 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 client.15660 shutdown 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 msgr2=0x55fac16a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 0x55fac16a2730 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa81800afc0 tx=0x7fa81800f7f0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 shutdown_connections 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac16ae4e0 0x55fac16b09a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac169e1a0 0x55fac16a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa810ff9640 0 client.15660 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3881481229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac1698640 0x55fac16a2730 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 >> 172.21.15.33:0/3881481229 conn(0x55fac168f8a0 msgr2=0x55fac168eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 shutdown_connections 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 1 -- 172.21.15.33:0/3881481229 wait complete. 2024-04-24T23:13:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.148+0000 7fa8325b3740 20 client.15660 trim_cache size 0 max 16384 2024-04-24T23:13:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.161+0000 7fa8325b3740 1 Processor -- start 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.161+0000 7fa8325b3740 1 -- start start 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac1692cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa8325b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281aa9a0 con 0x55fac169e1a0 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa8325b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281a8270 con 0x55fac1699030 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa8325b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281a60c0 con 0x55fac1698640 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac1692cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac1692cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37006/0 (socket says 172.21.15.33:37006) 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 -- 172.21.15.33:0/3395347219 learned_addr learned my addr 172.21.15.33:0/3395347219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82d641640 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 -- 172.21.15.33:0/3395347219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 msgr2=0x55fac16927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 -- 172.21.15.33:0/3395347219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 msgr2=0x55fac16a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 -- 172.21.15.33:0/3395347219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac13d67d0 con 0x55fac169e1a0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa82ce40640 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac1692cf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa818022910 tx=0x7fa81800ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18257830a7562b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.162+0000 7fa812ffd640 1 -- 172.21.15.33:0/3395347219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818032070 con 0x55fac169e1a0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa812ffd640 1 -- 172.21.15.33:0/3395347219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81801c070 con 0x55fac169e1a0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa812ffd640 1 -- 172.21.15.33:0/3395347219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81801f9e0 con 0x55fac169e1a0 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 -- 172.21.15.33:0/3395347219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 msgr2=0x55fac1692cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac1692cf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa818022910 tx=0x7fa81800ffb0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 -- 172.21.15.33:0/3395347219 shutdown_connections 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac1692cf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 --2- 172.21.15.33:0/3395347219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 -- 172.21.15.33:0/3395347219 >> 172.21.15.33:0/3395347219 conn(0x55fac168f8a0 msgr2=0x55fac168eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 -- 172.21.15.33:0/3395347219 shutdown_connections 2024-04-24T23:13:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.163+0000 7fa8325b3740 1 -- 172.21.15.33:0/3395347219 wait complete. 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 -- start start 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 Processor -- start 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281a6730 con 0x55fac169e1a0 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281a7ba0 con 0x55fac1699030 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281a6e00 con 0x55fac1698640 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa8325b3740 10 client.0 _ll_register_callbacks cb 0x7fa818011ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82ce40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37012/0 (socket says 172.21.15.33:37012) 2024-04-24T23:13:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82ce40640 1 -- 172.21.15.33:0/2560553980 learned_addr learned my addr 172.21.15.33:0/2560553980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82d641640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82ce40640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82ce40640 1 -- 172.21.15.33:0/2560553980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 msgr2=0x55fac16a6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.164+0000 7fa82ce40640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16a6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa82ce40640 1 -- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 msgr2=0x55fac16a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa82ce40640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa82ce40640 1 -- 172.21.15.33:0/2560553980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac16a8a00 con 0x55fac169e1a0 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa82ce40640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a8410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa818000be0 tx=0x7fa818014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6128afdbe7c2da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa810ff9640 1 -- 172.21.15.33:0/2560553980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818032040 con 0x55fac169e1a0 2024-04-24T23:13:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa810ff9640 1 -- 172.21.15.33:0/2560553980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa81801fb90 con 0x55fac169e1a0 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81801c070 con 0x55fac169e1a0 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa810ff9640 1 -- 172.21.15.33:0/2560553980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa818022a30 con 0x55fac169e1a0 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa8325b3740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fac16a9d00 con 0x55fac169e1a0 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.165+0000 7fa810ff9640 1 -- 172.21.15.33:0/2560553980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81801c070 con 0x55fac169e1a0 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa810ff9640 1 -- 172.21.15.33:0/2560553980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa818022bd0 con 0x55fac169e1a0 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 20 client.15666 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac1714ff0 0x55fac1717410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 4 client.15666 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fac15fff00 con 0x55fac1714ff0 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa82d641640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac1714ff0 0x55fac1717410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa82d641640 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac1714ff0 0x55fac1717410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1b894189326c031 server_cookie=30aee19790024a5 in_seq=0 out_seq=0 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa810ff9640 10 client.15666 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa810ff9640 1 -- 172.21.15.33:0/2560553980 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fac15fff00 con 0x55fac1714ff0 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa810ff9640 10 client.15666 handle_command_reply: tid=0 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.166+0000 7fa8325b3740 1 client.15666 shutdown 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 msgr2=0x55fac16a8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a8410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa818000be0 tx=0x7fa818014040 comp rx=0 tx=0).stop 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 shutdown_connections 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fac1714ff0 0x55fac1717410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac169e1a0 0x55fac16a8410 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fac1699030 0x55fac16a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa810ff9640 0 client.15666 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 --2- 172.21.15.33:0/2560553980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fac1698640 0x55fac16a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 >> 172.21.15.33:0/2560553980 conn(0x55fac168f8a0 msgr2=0x55fac1699ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 shutdown_connections 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 1 -- 172.21.15.33:0/2560553980 wait complete. 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa8325b3740 20 client.15666 trim_cache size 0 max 16384 2024-04-24T23:13:14.177 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8140a2c20 msgr2=0x7fa8140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8140a2c20 0x7fa8140a50e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa820096090 tx=0x7fa8200971f0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 msgr2=0x7fa82819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 0x7fa82819ee10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa820009fd0 tx=0x7fa82000def0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 shutdown_connections 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8140a2c20 0x7fa8140a50e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828102230 0x7fa82819ee10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa828101870 0x7fa82819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 --2- 172.21.15.33:0/3588270652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa828100680 0x7fa828069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.167+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 >> 172.21.15.33:0/3588270652 conn(0x7fa8280fbbf0 msgr2=0x7fa828104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.168+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 shutdown_connections 2024-04-24T23:13:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.168+0000 7fa82f8a5640 1 -- 172.21.15.33:0/3588270652 wait complete. 2024-04-24T23:13:14.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:13:14.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 -- 172.21.15.33:0/338753761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c81017a0 msgr2=0x7f23c8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 --2- 172.21.15.33:0/338753761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c81017a0 0x7f23c8101bc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009900 tx=0x7f23b802a410 comp rx=0 tx=0).stop 2024-04-24T23:13:14.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 -- 172.21.15.33:0/338753761 shutdown_connections 2024-04-24T23:13:14.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 --2- 172.21.15.33:0/338753761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c8103350 0x7f23c8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 --2- 172.21.15.33:0/338753761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c8102990 0x7f23c8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 --2- 172.21.15.33:0/338753761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c81017a0 0x7f23c8101bc0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 -- 172.21.15.33:0/338753761 >> 172.21.15.33:0/338753761 conn(0x7f23c80fcd10 msgr2=0x7f23c80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 -- 172.21.15.33:0/338753761 shutdown_connections 2024-04-24T23:13:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.302+0000 7f23ce6da640 1 -- 172.21.15.33:0/338753761 wait complete. 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.303+0000 7f23ce6da640 1 Processor -- start 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.303+0000 7f23ce6da640 1 -- start start 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.303+0000 7f23ce6da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c81017a0 0x7f23c8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ce6da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c8102990 0x7f23c8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ce6da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 0x7f23c819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ce6da640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23c810d410 con 0x7f23c8103350 2024-04-24T23:13:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ce6da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23c810d290 con 0x7f23c8102990 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ce6da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23c810d590 con 0x7f23c81017a0 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 0x7f23c819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 0x7f23c819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37020/0 (socket says 172.21.15.33:37020) 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 -- 172.21.15.33:0/2288957193 learned_addr learned my addr 172.21.15.33:0/2288957193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c81017a0 0x7f23c8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c8102990 0x7f23c8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 -- 172.21.15.33:0/2288957193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c81017a0 msgr2=0x7f23c8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c81017a0 0x7f23c8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 -- 172.21.15.33:0/2288957193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c8102990 msgr2=0x7f23c8198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c8102990 0x7f23c8198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 -- 172.21.15.33:0/2288957193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c819f6e0 con 0x7f23c8103350 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c81017a0 0x7f23c8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ccc77640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 0x7f23c819f070 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f23c000eaa0 tx=0x7f23c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23c57fa640 1 -- 172.21.15.33:0/2288957193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c000cdb0 con 0x7f23c8103350 2024-04-24T23:13:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c819f9d0 con 0x7f23c8103350 2024-04-24T23:13:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23c57fa640 1 -- 172.21.15.33:0/2288957193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23c0005020 con 0x7f23c8103350 2024-04-24T23:13:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.304+0000 7f23c57fa640 1 -- 172.21.15.33:0/2288957193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c00116b0 con 0x7f23c8103350 2024-04-24T23:13:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.305+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c819ff20 con 0x7f23c8103350 2024-04-24T23:13:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23c57fa640 1 -- 172.21.15.33:0/2288957193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f23c0005560 con 0x7f23c8103350 2024-04-24T23:13:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23d13eb740 1 Processor -- start 2024-04-24T23:13:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23d13eb740 1 -- start start 2024-04-24T23:13:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314be640 0x556c314bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23c57fa640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23b40a2c20 0x7f23b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.307+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314c41a0 0x556c314c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23d13eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c3141e670 con 0x556c314c41a0 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23d13eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556c313c6970 con 0x556c314bf030 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23d13eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c3141a630 con 0x556c314be640 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23ccc77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314c41a0 0x556c314c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60932/0 (socket says 172.21.15.33:60932) 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 -- 172.21.15.33:0/4198033286 learned_addr learned my addr 172.21.15.33:0/4198033286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23b40a2c20 0x7f23b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c57fa640 1 -- 172.21.15.33:0/2288957193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23c00cf890 con 0x7f23c8103350 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c77fe640 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314be640 0x556c314bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 -- 172.21.15.33:0/4198033286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314be640 msgr2=0x556c314bea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314be640 0x556c314bea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 -- 172.21.15.33:0/4198033286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314c41a0 msgr2=0x556c314c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314c41a0 0x556c314c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 -- 172.21.15.33:0/4198033286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c313ea400 con 0x556c314bf030 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23ccc77640 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314c41a0 0x556c314c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23b40a2c20 0x7f23b40a50e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f23c8198fd0 tx=0x7f23b000ba80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.308+0000 7f23c7fff640 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c3b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23b8031440 tx=0x7f23b8032de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c271a86a5e11e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23be7fc640 1 -- 172.21.15.33:0/4198033286 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b803f070 con 0x556c314bf030 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23be7fc640 1 -- 172.21.15.33:0/4198033286 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b80318b0 con 0x556c314bf030 2024-04-24T23:13:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23be7fc640 1 -- 172.21.15.33:0/4198033286 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8035070 con 0x556c314bf030 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 -- 172.21.15.33:0/4198033286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 msgr2=0x556c314c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c3b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23b8031440 tx=0x7f23b8032de0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 -- 172.21.15.33:0/4198033286 shutdown_connections 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314c41a0 0x556c314c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c3b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 --2- 172.21.15.33:0/4198033286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314be640 0x556c314bea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 -- 172.21.15.33:0/4198033286 >> 172.21.15.33:0/4198033286 conn(0x556c314b58a0 msgr2=0x556c314b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 -- 172.21.15.33:0/4198033286 shutdown_connections 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.309+0000 7f23d13eb740 1 -- 172.21.15.33:0/4198033286 wait complete. 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 -- start start 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 Processor -- start 2024-04-24T23:13:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c314c70f0 con 0x556c314be640 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556c313da050 con 0x556c314bf030 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c314c7270 con 0x556c314c41a0 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23d13eb740 10 client.0 _ll_register_callbacks cb 0x556c31131be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37040/0 (socket says 172.21.15.33:37040) 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23c7fff640 1 -- 172.21.15.33:0/2881591338 learned_addr learned my addr 172.21.15.33:0/2881591338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.310+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23ccc77640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23c7fff640 1 -- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 msgr2=0x556c314cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23c7fff640 1 -- 172.21.15.33:0/2881591338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 msgr2=0x556c314c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23c7fff640 1 -- 172.21.15.33:0/2881591338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c313abdc0 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23ccc77640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314c8790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f23b802a930 tx=0x7f23b8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827b2c993f6f809c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23a77fe640 1 -- 172.21.15.33:0/2881591338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b804a070 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23a77fe640 1 -- 172.21.15.33:0/2881591338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23b8003a00 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b80318b0 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23a77fe640 1 -- 172.21.15.33:0/2881591338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23b8056050 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23d13eb740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.311+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c311323f0 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23a77fe640 1 -- 172.21.15.33:0/2881591338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b80318b0 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23a77fe640 1 -- 172.21.15.33:0/2881591338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23b803bcb0 con 0x556c314be640 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23d13eb740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23d13eb740 10 client.15678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23d13eb740 20 client.15678 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c314d4190 0x556c314d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23d13eb740 4 client.15678 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c3142bbb0 con 0x556c314d4190 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c314d4190 0x556c314d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c314d4190 0x556c314d6580 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d399a206485592eb server_cookie=9d1bd3ca276e44bc in_seq=0 out_seq=0 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.312+0000 7f23a77fe640 10 client.15678 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23a77fe640 1 -- 172.21.15.33:0/2881591338 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c3142bbb0 con 0x556c314d4190 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23a77fe640 10 client.15678 handle_command_reply: tid=0 2024-04-24T23:13:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 client.15678 shutdown 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 msgr2=0x556c314c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314c8790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f23b802a930 tx=0x7f23b8004530 comp rx=0 tx=0).stop 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 shutdown_connections 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c314d4190 0x556c314d6580 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23a77fe640 0 client.15678 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2881591338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314c8790 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.313+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 >> 172.21.15.33:0/2881591338 conn(0x556c314b58a0 msgr2=0x556c314b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.314+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 shutdown_connections 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.314+0000 7f23d13eb740 1 -- 172.21.15.33:0/2881591338 wait complete. 2024-04-24T23:13:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.314+0000 7f23d13eb740 20 client.15678 trim_cache size 0 max 16384 2024-04-24T23:13:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 Processor -- start 2024-04-24T23:13:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 -- start start 2024-04-24T23:13:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314be640 0x556c314b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 0x556c314cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23c81aa9b0 con 0x556c314bf030 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23c81a8280 con 0x556c314be640 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23d13eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23c81a60d0 con 0x556c314c41a0 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23c77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 0x556c314cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23c77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 0x556c314cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37044/0 (socket says 172.21.15.33:37044) 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.327+0000 7f23c77fe640 1 -- 172.21.15.33:0/155152152 learned_addr learned my addr 172.21.15.33:0/155152152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23ccc77640 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c77fe640 1 -- 172.21.15.33:0/155152152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 msgr2=0x556c314c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c77fe640 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c7fff640 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314be640 0x556c314b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c77fe640 1 -- 172.21.15.33:0/155152152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314be640 msgr2=0x556c314b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c77fe640 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314be640 0x556c314b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c77fe640 1 -- 172.21.15.33:0/155152152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c314d68f0 con 0x556c314bf030 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23c77fe640 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 0x556c314cfbc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005bd0 tx=0x7f23b000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209e544df2e8aba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23bdffb640 1 -- 172.21.15.33:0/155152152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0010b20 con 0x556c314bf030 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23bdffb640 1 -- 172.21.15.33:0/155152152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b0029820 con 0x556c314bf030 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.328+0000 7f23bdffb640 1 -- 172.21.15.33:0/155152152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b00047a0 con 0x556c314bf030 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 -- 172.21.15.33:0/155152152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 msgr2=0x556c314cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 0x556c314cfbc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005bd0 tx=0x7f23b000a000 comp rx=0 tx=0).stop 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 -- 172.21.15.33:0/155152152 shutdown_connections 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314bf030 0x556c314cfbc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 --2- 172.21.15.33:0/155152152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314be640 0x556c314b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 -- 172.21.15.33:0/155152152 >> 172.21.15.33:0/155152152 conn(0x556c314b58a0 msgr2=0x556c314c0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 -- 172.21.15.33:0/155152152 shutdown_connections 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 -- 172.21.15.33:0/155152152 wait complete. 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 -- start start 2024-04-24T23:13:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.329+0000 7f23d13eb740 1 Processor -- start 2024-04-24T23:13:14.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23c81a6740 con 0x556c314be640 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23c81a7bb0 con 0x556c314bf030 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23c81a6e10 con 0x556c314c41a0 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 10 client.0 _ll_register_callbacks cb 0x7f23c0097780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37050/0 (socket says 172.21.15.33:37050) 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 -- 172.21.15.33:0/2770258474 learned_addr learned my addr 172.21.15.33:0/2770258474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23ccc77640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 -- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 msgr2=0x556c314ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 -- 172.21.15.33:0/2770258474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 msgr2=0x556c314cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 -- 172.21.15.33:0/2770258474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c311fc7d0 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23c7fff640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314cd010 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f23b802a450 tx=0x7f23b8004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da750dcb8be88fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23a77fe640 1 -- 172.21.15.33:0/2770258474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b804a040 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23a77fe640 1 -- 172.21.15.33:0/2770258474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23b80439c0 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b0029820 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.330+0000 7f23a77fe640 1 -- 172.21.15.33:0/2770258474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23b803f030 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c314cfc80 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23a77fe640 1 -- 172.21.15.33:0/2770258474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b0029820 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23a77fe640 1 -- 172.21.15.33:0/2770258474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23b804f440 con 0x556c314be640 2024-04-24T23:13:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 10 client.15684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 20 client.15684 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c3153a880 0x556c3153cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 4 client.15684 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c31425f00 con 0x556c3153a880 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.331+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c3153a880 0x556c3153cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23c77fe640 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c3153a880 0x556c3153cc70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=673793ef46c5c234 server_cookie=2ab9f0617fc4e20 in_seq=0 out_seq=0 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23a77fe640 10 client.15684 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23a77fe640 1 -- 172.21.15.33:0/2770258474 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c31425f00 con 0x556c3153a880 2024-04-24T23:13:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23a77fe640 10 client.15684 handle_command_reply: tid=0 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 client.15684 shutdown 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 msgr2=0x556c314cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314cd010 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f23b802a450 tx=0x7f23b8004420 comp rx=0 tx=0).stop 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 shutdown_connections 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556c3153a880 0x556c3153cc70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c314c41a0 0x556c314ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c314bf030 0x556c314cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23a77fe640 0 client.15684 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 --2- 172.21.15.33:0/2770258474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c314be640 0x556c314cd010 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 >> 172.21.15.33:0/2770258474 conn(0x556c314b58a0 msgr2=0x556c314bfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 shutdown_connections 2024-04-24T23:13:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 1 -- 172.21.15.33:0/2770258474 wait complete. 2024-04-24T23:13:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.332+0000 7f23d13eb740 20 client.15684 trim_cache size 0 max 16384 2024-04-24T23:13:14.342 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23b40a2c20 msgr2=0x7f23b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23b40a2c20 0x7f23b40a50e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f23c8198fd0 tx=0x7f23b000ba80 comp rx=0 tx=0).stop 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 msgr2=0x7f23c819f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 0x7f23c819f070 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f23c000eaa0 tx=0x7f23c000ef60 comp rx=0 tx=0).stop 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 shutdown_connections 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23b40a2c20 0x7f23b40a50e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23c8103350 0x7f23c819f070 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c8102990 0x7f23c8198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 --2- 172.21.15.33:0/2288957193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c81017a0 0x7f23c8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 >> 172.21.15.33:0/2288957193 conn(0x7f23c80fcd10 msgr2=0x7f23c8105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 shutdown_connections 2024-04-24T23:13:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.333+0000 7f23ce6da640 1 -- 172.21.15.33:0/2288957193 wait complete. 2024-04-24T23:13:14.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 -- 172.21.15.33:0/2182766924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8101870 msgr2=0x7fd9d8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 --2- 172.21.15.33:0/2182766924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8101870 0x7fd9d8101cf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009900 tx=0x7fd9c402a410 comp rx=0 tx=0).stop 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 -- 172.21.15.33:0/2182766924 shutdown_connections 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 --2- 172.21.15.33:0/2182766924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8102230 0x7fd9d8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 --2- 172.21.15.33:0/2182766924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8101870 0x7fd9d8101cf0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 --2- 172.21.15.33:0/2182766924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9d8100680 0x7fd9d8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 -- 172.21.15.33:0/2182766924 >> 172.21.15.33:0/2182766924 conn(0x7fd9d80fbbf0 msgr2=0x7fd9d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 -- 172.21.15.33:0/2182766924 shutdown_connections 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.455+0000 7fd9dec91640 1 -- 172.21.15.33:0/2182766924 wait complete. 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 Processor -- start 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 -- start start 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 0x7fd9d8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9d8101870 0x7fd9d8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8102230 0x7fd9d819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810c330 con 0x7fd9d8100680 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810c1b0 con 0x7fd9d8101870 2024-04-24T23:13:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dec91640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810c4b0 con 0x7fd9d8102230 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dca2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 0x7fd9d8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dca2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 0x7fd9d8197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37078/0 (socket says 172.21.15.33:37078) 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1184555788 learned_addr learned my addr 172.21.15.33:0/1184555788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9d8101870 0x7fd9d8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.456+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8102230 0x7fd9d819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1184555788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8102230 msgr2=0x7fd9d819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8102230 0x7fd9d819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1184555788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9d8101870 msgr2=0x7fd9d8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9d8101870 0x7fd9d8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1184555788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d819f740 con 0x7fd9d8100680 2024-04-24T23:13:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 0x7fd9d8197b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00d770 tx=0x7fd9cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9d5ffb640 1 -- 172.21.15.33:0/1184555788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc017070 con 0x7fd9d8100680 2024-04-24T23:13:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9d5ffb640 1 -- 172.21.15.33:0/1184555788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc009c30 con 0x7fd9d8100680 2024-04-24T23:13:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d819fa30 con 0x7fd9d8100680 2024-04-24T23:13:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9d5ffb640 1 -- 172.21.15.33:0/1184555788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc003dd0 con 0x7fd9d8100680 2024-04-24T23:13:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.457+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d819ff50 con 0x7fd9d8100680 2024-04-24T23:13:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9d5ffb640 1 -- 172.21.15.33:0/1184555788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd9cc003fb0 con 0x7fd9d8100680 2024-04-24T23:13:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 Processor -- start 2024-04-24T23:13:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 -- start start 2024-04-24T23:13:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f6eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f71630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a50ec9670 con 0x562a50f6a030 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562a50e71970 con 0x562a50f69640 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9e1995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a50ec5630 con 0x562a50f6f1a0 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9d5ffb640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd9c80a2c20 0x7fd9c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9dca2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f6eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd9c80a2c20 0x7fd9c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9dca2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f6eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37080/0 (socket says 172.21.15.33:37080) 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.460+0000 7fd9dca2d640 1 -- 172.21.15.33:0/135762006 learned_addr learned my addr 172.21.15.33:0/135762006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f71630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9d5ffb640 1 -- 172.21.15.33:0/1184555788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd9cc0ceea0 con 0x7fd9d8100680 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dca2d640 1 -- 172.21.15.33:0/135762006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 msgr2=0x562a50f71630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f71630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dca2d640 1 -- 172.21.15.33:0/135762006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 msgr2=0x562a50f69a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f69a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dca2d640 1 -- 172.21.15.33:0/135762006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a50e95400 con 0x562a50f6a030 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f6eb30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0a38f0 tx=0x7fd9cc06e320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a674a5afce86c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd9c80a2c20 0x7fd9c80a50e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0005fd0 tx=0x7fd9d0009de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9c2ffd640 1 -- 172.21.15.33:0/135762006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc020070 con 0x562a50f6a030 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9c2ffd640 1 -- 172.21.15.33:0/135762006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc009de0 con 0x562a50f6a030 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.461+0000 7fd9c2ffd640 1 -- 172.21.15.33:0/135762006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0042a0 con 0x562a50f6a030 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 -- 172.21.15.33:0/135762006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 msgr2=0x562a50f6eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f6eb30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0a38f0 tx=0x7fd9cc06e320 comp rx=0 tx=0).stop 2024-04-24T23:13:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 -- 172.21.15.33:0/135762006 shutdown_connections 2024-04-24T23:13:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f71630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f6eb30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 --2- 172.21.15.33:0/135762006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f69a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 -- 172.21.15.33:0/135762006 >> 172.21.15.33:0/135762006 conn(0x562a50f608a0 msgr2=0x562a50f60ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 -- 172.21.15.33:0/135762006 shutdown_connections 2024-04-24T23:13:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.462+0000 7fd9e1995740 1 -- 172.21.15.33:0/135762006 wait complete. 2024-04-24T23:13:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 -- start start 2024-04-24T23:13:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 Processor -- start 2024-04-24T23:13:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 0x562a50f736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6a030 0x562a50f73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f7ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a50f72090 con 0x562a50f69640 2024-04-24T23:13:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562a50e85050 con 0x562a50f6a030 2024-04-24T23:13:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a50f72210 con 0x562a50f6f1a0 2024-04-24T23:13:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9e1995740 10 client.0 _ll_register_callbacks cb 0x562a50bdcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9dca2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 0x562a50f736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9dca2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 0x562a50f736b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37090/0 (socket says 172.21.15.33:37090) 2024-04-24T23:13:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1662717974 learned_addr learned my addr 172.21.15.33:0/1662717974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6a030 0x562a50f73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.463+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f7ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 msgr2=0x562a50f7ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f7ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1662717974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6a030 msgr2=0x562a50f73bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6a030 0x562a50f73bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dca2d640 1 -- 172.21.15.33:0/1662717974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a50e56dc0 con 0x562a50f69640 2024-04-24T23:13:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f7ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6a030 0x562a50f73bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:14.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 0x562a50f736b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009610 tx=0x7fd9cc005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf104b5e915e384 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:14.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9b7fff640 1 -- 172.21.15.33:0/1662717974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc020040 con 0x562a50f69640 2024-04-24T23:13:14.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9b7fff640 1 -- 172.21.15.33:0/1662717974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd9cc0978d0 con 0x562a50f69640 2024-04-24T23:13:14.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9cc009de0 con 0x562a50f69640 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9b7fff640 1 -- 172.21.15.33:0/1662717974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9cc09b260 con 0x562a50f69640 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9e1995740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a50bdd3f0 con 0x562a50f69640 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.464+0000 7fd9b7fff640 1 -- 172.21.15.33:0/1662717974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc009de0 con 0x562a50f69640 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9b7fff640 1 -- 172.21.15.33:0/1662717974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd9cc0bb080 con 0x562a50f69640 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9e1995740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:14.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9e1995740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9e1995740 20 client.15696 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9e1995740 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50f7f120 0x562a50f81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9e1995740 4 client.15696 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a50ed6bb0 con 0x562a50f7f120 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50f7f120 0x562a50f81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50f7f120 0x562a50f81510 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8348cdc237f79d server_cookie=ac1f81eae0998cad in_seq=0 out_seq=0 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.465+0000 7fd9b7fff640 10 client.15696 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9b7fff640 1 -- 172.21.15.33:0/1662717974 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a50ed6bb0 con 0x562a50f7f120 2024-04-24T23:13:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9b7fff640 10 client.15696 handle_command_reply: tid=0 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 client.15696 shutdown 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 msgr2=0x562a50f736b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 0x562a50f736b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009610 tx=0x7fd9cc005d10 comp rx=0 tx=0).stop 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 shutdown_connections 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50f7f120 0x562a50f81510 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6f1a0 0x562a50f7ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6a030 0x562a50f73bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 --2- 172.21.15.33:0/1662717974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f69640 0x562a50f736b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 >> 172.21.15.33:0/1662717974 conn(0x562a50f608a0 msgr2=0x562a50f5fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.466+0000 7fd9b7fff640 0 client.15696 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:14.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.467+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 shutdown_connections 2024-04-24T23:13:14.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.467+0000 7fd9e1995740 1 -- 172.21.15.33:0/1662717974 wait complete. 2024-04-24T23:13:14.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.467+0000 7fd9e1995740 20 client.15696 trim_cache size 0 max 16384 2024-04-24T23:13:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 Processor -- start 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 -- start start 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f6a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6a030 0x562a50f7abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 0x562a50f6ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d81aab70 con 0x562a50f6f1a0 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d81a8440 con 0x562a50f69640 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9e1995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d81a6290 con 0x562a50f6a030 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 0x562a50f6ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 0x562a50f6ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37106/0 (socket says 172.21.15.33:37106) 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 -- 172.21.15.33:0/4144881708 learned_addr learned my addr 172.21.15.33:0/4144881708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f6a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6a030 0x562a50f7abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 -- 172.21.15.33:0/4144881708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6a030 msgr2=0x562a50f7abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6a030 0x562a50f7abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 -- 172.21.15.33:0/4144881708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 msgr2=0x562a50f6a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f6a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dd22e640 1 -- 172.21.15.33:0/4144881708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a50ca77d0 con 0x562a50f6f1a0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.494+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f6a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.495+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 0x562a50f6ae00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00126c0 tx=0x7fd9d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e05f0ce37fdb52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.495+0000 7fd9c27fc640 1 -- 172.21.15.33:0/4144881708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d001e720 con 0x562a50f6f1a0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.495+0000 7fd9c27fc640 1 -- 172.21.15.33:0/4144881708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d001eda0 con 0x562a50f6f1a0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.495+0000 7fd9c27fc640 1 -- 172.21.15.33:0/4144881708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0021410 con 0x562a50f6f1a0 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 -- 172.21.15.33:0/4144881708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 msgr2=0x562a50f6ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 0x562a50f6ae00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00126c0 tx=0x7fd9d0004ab0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 -- 172.21.15.33:0/4144881708 shutdown_connections 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6f1a0 0x562a50f6ae00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f6a030 0x562a50f7abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 --2- 172.21.15.33:0/4144881708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f69640 0x562a50f6a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 -- 172.21.15.33:0/4144881708 >> 172.21.15.33:0/4144881708 conn(0x562a50f608a0 msgr2=0x562a50f5fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 -- 172.21.15.33:0/4144881708 shutdown_connections 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 -- 172.21.15.33:0/4144881708 wait complete. 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 -- start start 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.496+0000 7fd9e1995740 1 Processor -- start 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f69640 0x562a50f77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f78180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6f1a0 0x562a50f79190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d81a6900 con 0x562a50f6a030 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d81a7d70 con 0x562a50f6f1a0 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d81a6fd0 con 0x562a50f69640 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9e1995740 10 client.0 _ll_register_callbacks cb 0x7fd9d0012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f78180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f78180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37122/0 (socket says 172.21.15.33:37122) 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 -- 172.21.15.33:0/786785970 learned_addr learned my addr 172.21.15.33:0/786785970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9dd22e640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6f1a0 0x562a50f79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f69640 0x562a50f77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 -- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f69640 msgr2=0x562a50f77c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f69640 0x562a50f77c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 -- 172.21.15.33:0/786785970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6f1a0 msgr2=0x562a50f79190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6f1a0 0x562a50f79190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 -- 172.21.15.33:0/786785970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a50f79780 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9d7fff640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f78180 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4006010 tx=0x7fd9c4004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c717e773485c1cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.497+0000 7fd9b7fff640 1 -- 172.21.15.33:0/786785970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4048020 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d001eda0 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9b7fff640 1 -- 172.21.15.33:0/786785970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd9c4044330 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9b7fff640 1 -- 172.21.15.33:0/786785970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9c4053050 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a50f7aa60 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9b7fff640 1 -- 172.21.15.33:0/786785970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d001eda0 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9b7fff640 1 -- 172.21.15.33:0/786785970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd9c4004770 con 0x562a50f6a030 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 20 client.15702 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:14.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50fe5850 0x562a50fe7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 4 client.15702 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a50ed0f00 con 0x562a50fe5850 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.498+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50fe5850 0x562a50fe7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9b7fff640 10 client.15702 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9dca2d640 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50fe5850 0x562a50fe7c70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c20533d4c2961e server_cookie=55e956823cb927f2 in_seq=0 out_seq=0 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9b7fff640 1 -- 172.21.15.33:0/786785970 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a50ed0f00 con 0x562a50fe5850 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9b7fff640 10 client.15702 handle_command_reply: tid=0 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 client.15702 shutdown 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 msgr2=0x562a50f78180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f78180 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4006010 tx=0x7fd9c4004480 comp rx=0 tx=0).stop 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 shutdown_connections 2024-04-24T23:13:14.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562a50fe5850 0x562a50fe7c70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562a50f6f1a0 0x562a50f79190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9b7fff640 0 client.15702 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a50f6a030 0x562a50f78180 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 --2- 172.21.15.33:0/786785970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a50f69640 0x562a50f77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 >> 172.21.15.33:0/786785970 conn(0x562a50f608a0 msgr2=0x562a50f5f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 shutdown_connections 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.499+0000 7fd9e1995740 1 -- 172.21.15.33:0/786785970 wait complete. 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9e1995740 20 client.15702 trim_cache size 0 max 16384 2024-04-24T23:13:14.509 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:13:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd9c80a2c20 msgr2=0x7fd9c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd9c80a2c20 0x7fd9c80a50e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0005fd0 tx=0x7fd9d0009de0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 msgr2=0x7fd9d8197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 0x7fd9d8197b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00d770 tx=0x7fd9cc00dc30 comp rx=0 tx=0).stop 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 shutdown_connections 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd9c80a2c20 0x7fd9c80a50e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8102230 0x7fd9d819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9d8101870 0x7fd9d8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 --2- 172.21.15.33:0/1184555788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d8100680 0x7fd9d8197b20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 >> 172.21.15.33:0/1184555788 conn(0x7fd9d80fbbf0 msgr2=0x7fd9d80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 shutdown_connections 2024-04-24T23:13:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:14.500+0000 7fd9dec91640 1 -- 172.21.15.33:0/1184555788 wait complete. 2024-04-24T23:13:15.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:14 smithi038 ceph-mon[32450]: pgmap v388: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:15.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:14 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1542615476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:13:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:14 smithi117 ceph-mon[32529]: pgmap v388: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:14 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1542615476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:13:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:14 smithi033 ceph-mon[29201]: pgmap v388: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T23:13:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:14 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1542615476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:13:15.513 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:13:15.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:13:15.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/453070783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98102230 msgr2=0x7fdf98108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/453070783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98102230 0x7fdf98108ac0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009ca0 tx=0x7fdf9002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/453070783 shutdown_connections 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/453070783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98102230 0x7fdf98108ac0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/453070783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf98101870 0x7fdf98101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/453070783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf98100680 0x7fdf98100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/453070783 >> 172.21.15.33:0/453070783 conn(0x7fdf980fbbf0 msgr2=0x7fdf980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/453070783 shutdown_connections 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.624+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/453070783 wait complete. 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 Processor -- start 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 -- start start 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf98100680 0x7fdf98104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 0x7fdf98102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf98102230 0x7fdf98103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810c2e0 con 0x7fdf98101870 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810c160 con 0x7fdf98102230 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9cd4f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810c460 con 0x7fdf98100680 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 0x7fdf98102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 0x7fdf98102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37134/0 (socket says 172.21.15.33:37134) 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf95d9b640 1 -- 172.21.15.33:0/3027560598 learned_addr learned my addr 172.21.15.33:0/3027560598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf9659c640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf98100680 0x7fdf98104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.625+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf98102230 0x7fdf98103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf95d9b640 1 -- 172.21.15.33:0/3027560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf98100680 msgr2=0x7fdf98104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf98100680 0x7fdf98104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf95d9b640 1 -- 172.21.15.33:0/3027560598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf98102230 msgr2=0x7fdf98103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf98102230 0x7fdf98103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf95d9b640 1 -- 172.21.15.33:0/3027560598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf98103a20 con 0x7fdf98101870 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 0x7fdf98102c50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00e9f0 tx=0x7fdf8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf877fe640 1 -- 172.21.15.33:0/3027560598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00cd80 con 0x7fdf98101870 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf877fe640 1 -- 172.21.15.33:0/3027560598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c010910 con 0x7fdf98101870 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf981a1b40 con 0x7fdf98101870 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf877fe640 1 -- 172.21.15.33:0/3027560598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c018990 con 0x7fdf98101870 2024-04-24T23:13:15.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.626+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf981a20e0 con 0x7fdf98101870 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.628+0000 7fdf877fe640 1 -- 172.21.15.33:0/3027560598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fdf8c010ac0 con 0x7fdf98101870 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.628+0000 7fdf9fa59740 1 Processor -- start 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 -- start start 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202dd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 0x5600202e2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600202402c0 con 0x5600202dd780 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5600201e8910 con 0x5600202e28f0 2024-04-24T23:13:15.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56002023c280 con 0x5600202dcd90 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 0x5600202e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 0x5600202e2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37148/0 (socket says 172.21.15.33:37148) 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf9659c640 1 -- 172.21.15.33:0/3002132820 learned_addr learned my addr 172.21.15.33:0/3002132820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf877fe640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdf800a2ca0 0x7fdf800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.629+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202dd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf9659c640 1 -- 172.21.15.33:0/3002132820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 msgr2=0x5600202dd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf9659c640 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202dd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf9659c640 1 -- 172.21.15.33:0/3002132820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 msgr2=0x5600202e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf9659c640 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf9659c640 1 -- 172.21.15.33:0/3002132820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600201f9fb0 con 0x5600202dd780 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202dd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdf800a2ca0 0x7fdf800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf877fe640 1 -- 172.21.15.33:0/3027560598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdf8c0cf7e0 con 0x7fdf98101870 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf9659c640 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 0x5600202e2280 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800ac70 tx=0x7fdf8800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6855cfe612327dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf84ff9640 1 -- 172.21.15.33:0/3002132820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88005220 con 0x5600202dd780 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf84ff9640 1 -- 172.21.15.33:0/3002132820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88004c80 con 0x5600202dd780 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.630+0000 7fdf84ff9640 1 -- 172.21.15.33:0/3002132820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88018940 con 0x5600202dd780 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 -- 172.21.15.33:0/3002132820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 msgr2=0x5600202e2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdf800a2ca0 0x7fdf800a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5600202ddfe0 tx=0x7fdf900095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 0x5600202e2280 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800ac70 tx=0x7fdf8800f790 comp rx=0 tx=0).stop 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 -- 172.21.15.33:0/3002132820 shutdown_connections 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dd780 0x5600202e2280 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/3002132820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202dd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 -- 172.21.15.33:0/3002132820 >> 172.21.15.33:0/3002132820 conn(0x5600202a4950 msgr2=0x5600202d8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 -- 172.21.15.33:0/3002132820 shutdown_connections 2024-04-24T23:13:15.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 -- 172.21.15.33:0/3002132820 wait complete. 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 -- start start 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.631+0000 7fdf9fa59740 1 Processor -- start 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dcd90 0x5600202e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600202e58a0 con 0x5600202e28f0 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5600201fbe90 con 0x5600202dcd90 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5600202e5a20 con 0x5600202dd780 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 10 client.0 _ll_register_callbacks cb 0x560020237970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37164/0 (socket says 172.21.15.33:37164) 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 -- 172.21.15.33:0/1419744022 learned_addr learned my addr 172.21.15.33:0/1419744022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9659c640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dcd90 0x5600202e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 -- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 msgr2=0x5600202e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 -- 172.21.15.33:0/1419744022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dcd90 msgr2=0x5600202e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dcd90 0x5600202e71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 -- 172.21.15.33:0/1419744022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56002001c7e0 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ee770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009ca0 tx=0x7fdf9002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a63ee4403d17171 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf7dffb640 1 -- 172.21.15.33:0/1419744022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004040 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf7dffb640 1 -- 172.21.15.33:0/1419744022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdf900041e0 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.632+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf88004c80 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf7dffb640 1 -- 172.21.15.33:0/1419744022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf9005a050 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 10 client.15714 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56002007cf70 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf7dffb640 1 -- 172.21.15.33:0/1419744022 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88004c80 con 0x5600202e28f0 2024-04-24T23:13:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf7dffb640 1 -- 172.21.15.33:0/1419744022 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdf9002fca0 con 0x5600202e28f0 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 20 client.15714 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5600202f2ca0 0x5600202f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 4 client.15714 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600202145a0 con 0x5600202f2ca0 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.633+0000 7fdf9659c640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5600202f2ca0 0x5600202f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.634+0000 7fdf9659c640 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5600202f2ca0 0x5600202f5160 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367705952643d8e2 server_cookie=a28d24cc36a3bae5 in_seq=0 out_seq=0 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.634+0000 7fdf7dffb640 10 client.15714 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf7dffb640 1 -- 172.21.15.33:0/1419744022 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600202145a0 con 0x5600202f2ca0 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf7dffb640 10 client.15714 handle_command_reply: tid=0 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 client.15714 shutdown 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 msgr2=0x5600202ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ee770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009ca0 tx=0x7fdf9002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 shutdown_connections 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5600202f2ca0 0x5600202f5160 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ee770 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf7dffb640 0 client.15714 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/1419744022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dcd90 0x5600202e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 >> 172.21.15.33:0/1419744022 conn(0x5600202a4950 msgr2=0x5600202d7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 shutdown_connections 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 1 -- 172.21.15.33:0/1419744022 wait complete. 2024-04-24T23:13:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.635+0000 7fdf9fa59740 20 client.15714 trim_cache size 0 max 16384 2024-04-24T23:13:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 Processor -- start 2024-04-24T23:13:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 -- start start 2024-04-24T23:13:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 0x5600202d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981aedf0 con 0x5600202dcd90 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981ac6c0 con 0x5600202e28f0 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981aa510 con 0x5600202dd780 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 0x5600202d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 0x5600202d77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37176/0 (socket says 172.21.15.33:37176) 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.653+0000 7fdf9659c640 1 -- 172.21.15.33:0/4055619702 learned_addr learned my addr 172.21.15.33:0/4055619702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf9659c640 1 -- 172.21.15.33:0/4055619702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 msgr2=0x5600202d7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf9659c640 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202d7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf9659c640 1 -- 172.21.15.33:0/4055619702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 msgr2=0x5600202ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf9659c640 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202ee730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf9659c640 1 -- 172.21.15.33:0/4055619702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600202df220 con 0x5600202dcd90 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf95d9b640 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202d7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf9659c640 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 0x5600202d77d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800ac70 tx=0x7fdf88000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b327a7491435d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf7ffff640 1 -- 172.21.15.33:0/4055619702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802bb50 con 0x5600202dcd90 2024-04-24T23:13:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf7ffff640 1 -- 172.21.15.33:0/4055619702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8802bcf0 con 0x5600202dcd90 2024-04-24T23:13:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.654+0000 7fdf7ffff640 1 -- 172.21.15.33:0/4055619702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802b2d0 con 0x5600202dcd90 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 -- 172.21.15.33:0/4055619702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 msgr2=0x5600202d77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 0x5600202d77d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800ac70 tx=0x7fdf88000bc0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 -- 172.21.15.33:0/4055619702 shutdown_connections 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202e28f0 0x5600202ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dd780 0x5600202d7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/4055619702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202dcd90 0x5600202d77d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 -- 172.21.15.33:0/4055619702 >> 172.21.15.33:0/4055619702 conn(0x5600202a4950 msgr2=0x5600202d70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 -- 172.21.15.33:0/4055619702 shutdown_connections 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 -- 172.21.15.33:0/4055619702 wait complete. 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 -- start start 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.655+0000 7fdf9fa59740 1 Processor -- start 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dd780 0x5600202ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981aab80 con 0x5600202e28f0 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981abff0 con 0x5600202dd780 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981ab250 con 0x5600202dcd90 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9fa59740 10 client.0 _ll_register_callbacks cb 0x7fdf90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dd780 0x5600202ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42830/0 (socket says 172.21.15.33:42830) 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 -- 172.21.15.33:0/684136977 learned_addr learned my addr 172.21.15.33:0/684136977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9659c640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 -- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 msgr2=0x5600202edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202edfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 -- 172.21.15.33:0/684136977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dd780 msgr2=0x5600202ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dd780 0x5600202ee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 -- 172.21.15.33:0/684136977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001ff14170 con 0x5600202e28f0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf9659c640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf96d9d640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ec620 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf90030780 tx=0x7fdf90003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6d9a0d1786e406 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.656+0000 7fdf7dffb640 1 -- 172.21.15.33:0/684136977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90062020 con 0x5600202e28f0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf7dffb640 1 -- 172.21.15.33:0/684136977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdf90030800 con 0x5600202e28f0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8802bcf0 con 0x5600202e28f0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf7dffb640 1 -- 172.21.15.33:0/684136977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf9006d050 con 0x5600202e28f0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600203599e0 con 0x5600202e28f0 2024-04-24T23:13:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf7dffb640 1 -- 172.21.15.33:0/684136977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8802bcf0 con 0x5600202e28f0 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf7dffb640 1 -- 172.21.15.33:0/684136977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdf90003f80 con 0x5600202e28f0 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 20 client.15720 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56002035a440 0x56002035c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 4 client.15720 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56002024d800 con 0x56002035a440 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.657+0000 7fdf9659c640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56002035a440 0x56002035c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.658+0000 7fdf7dffb640 10 client.15720 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.658+0000 7fdf9659c640 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56002035a440 0x56002035c860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0809af014acccc server_cookie=a9e31133cd730ed6 in_seq=0 out_seq=0 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf7dffb640 1 -- 172.21.15.33:0/684136977 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56002024d800 con 0x56002035a440 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf7dffb640 10 client.15720 handle_command_reply: tid=0 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 client.15720 shutdown 2024-04-24T23:13:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 msgr2=0x5600202ec620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ec620 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf90030780 tx=0x7fdf90003de0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 shutdown_connections 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56002035a440 0x56002035c860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600202e28f0 0x5600202ec620 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5600202dd780 0x5600202ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 --2- 172.21.15.33:0/684136977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600202dcd90 0x5600202edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 >> 172.21.15.33:0/684136977 conn(0x5600202a4950 msgr2=0x5600202d7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf7dffb640 0 client.15720 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.659+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 shutdown_connections 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9fa59740 1 -- 172.21.15.33:0/684136977 wait complete. 2024-04-24T23:13:15.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9fa59740 20 client.15720 trim_cache size 0 max 16384 2024-04-24T23:13:15.667 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:13:15.667 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:13:15.667 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d71f3e4d-d9ac-46f4-a9de-96fce6d37f5a", 2024-04-24T23:13:15.667 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:13:15.667 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdf800a2ca0 msgr2=0x7fdf800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdf800a2ca0 0x7fdf800a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5600202ddfe0 tx=0x7fdf900095c0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 msgr2=0x7fdf98102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 0x7fdf98102c50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00e9f0 tx=0x7fdf8c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 shutdown_connections 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdf800a2ca0 0x7fdf800a5160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf98102230 0x7fdf98103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf98101870 0x7fdf98102c50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 --2- 172.21.15.33:0/3027560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf98100680 0x7fdf98104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 >> 172.21.15.33:0/3027560598 conn(0x7fdf980fbbf0 msgr2=0x7fdf980fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 shutdown_connections 2024-04-24T23:13:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.660+0000 7fdf9cd4f640 1 -- 172.21.15.33:0/3027560598 wait complete. 2024-04-24T23:13:15.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:13:15.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 -- 172.21.15.33:0/1448564898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8102230 msgr2=0x7f5ac8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 --2- 172.21.15.33:0/1448564898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8102230 0x7f5ac8108ac0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009ca0 tx=0x7f5ac002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 -- 172.21.15.33:0/1448564898 shutdown_connections 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 --2- 172.21.15.33:0/1448564898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8102230 0x7f5ac8108ac0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 --2- 172.21.15.33:0/1448564898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8101870 0x7f5ac8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 --2- 172.21.15.33:0/1448564898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8100680 0x7f5ac8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 -- 172.21.15.33:0/1448564898 >> 172.21.15.33:0/1448564898 conn(0x7f5ac80fbbf0 msgr2=0x7f5ac80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 -- 172.21.15.33:0/1448564898 shutdown_connections 2024-04-24T23:13:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.781+0000 7f5ad0612640 1 -- 172.21.15.33:0/1448564898 wait complete. 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 Processor -- start 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 -- start start 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8100680 0x7f5ac8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 0x7f5ac8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8102230 0x7f5ac819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810c2e0 con 0x7f5ac8101870 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810c160 con 0x7f5ac8100680 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ad0612640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810c460 con 0x7f5ac8102230 2024-04-24T23:13:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 0x7f5ac8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 0x7f5ac8197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42848/0 (socket says 172.21.15.33:42848) 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 -- 172.21.15.33:0/2066941855 learned_addr learned my addr 172.21.15.33:0/2066941855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8100680 0x7f5ac8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 -- 172.21.15.33:0/2066941855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8102230 msgr2=0x7f5ac819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8102230 0x7f5ac819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8102230 0x7f5ac819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 -- 172.21.15.33:0/2066941855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8100680 msgr2=0x7f5ac8197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8100680 0x7f5ac8197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 -- 172.21.15.33:0/2066941855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac819f660 con 0x7f5ac8101870 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8100680 0x7f5ac8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8102230 0x7f5ac819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.782+0000 7f5acdbad640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 0x7f5ac8197f80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800d950 tx=0x7f5ab800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.783+0000 7f5abf7fe640 1 -- 172.21.15.33:0/2066941855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8017030 con 0x7f5ac8101870 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.783+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac819f950 con 0x7f5ac8101870 2024-04-24T23:13:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.783+0000 7f5abf7fe640 1 -- 172.21.15.33:0/2066941855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab8004020 con 0x7f5ac8101870 2024-04-24T23:13:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.783+0000 7f5abf7fe640 1 -- 172.21.15.33:0/2066941855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80047a0 con 0x7f5ac8101870 2024-04-24T23:13:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.783+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac819fef0 con 0x7f5ac8101870 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.785+0000 7f5abf7fe640 1 -- 172.21.15.33:0/2066941855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5ab80041d0 con 0x7f5ac8101870 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.785+0000 7f5ad33f6740 1 Processor -- start 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 -- start start 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabe240 0x55d5edabe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabec30 0x55d5edac3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edac6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d5eda226b0 con 0x55d5edac3da0 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed9ca880 con 0x55d5edabe240 2024-04-24T23:13:15.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5eda1e670 con 0x55d5edabec30 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edac6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edac6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42852/0 (socket says 172.21.15.33:42852) 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 -- 172.21.15.33:0/3737186411 learned_addr learned my addr 172.21.15.33:0/3737186411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acebaf640 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabe240 0x55d5edabe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabec30 0x55d5edac3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5abf7fe640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ab40a2c20 0x7f5ab40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ab40a2c20 0x7f5ab40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5abf7fe640 1 -- 172.21.15.33:0/2066941855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ab80cf020 con 0x7f5ac8101870 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 -- 172.21.15.33:0/3737186411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabec30 msgr2=0x55d5edac3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabec30 0x55d5edac3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 -- 172.21.15.33:0/3737186411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabe240 msgr2=0x55d5edabe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.786+0000 7f5acdbad640 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabe240 0x55d5edabe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.787+0000 7f5acdbad640 1 -- 172.21.15.33:0/3737186411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ed8007d0 con 0x55d5edac3da0 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.787+0000 7f5acdbad640 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edac6230 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5ab809b320 tx=0x7f5ab8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453f7b20005c44e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.787+0000 7f5abcbfe640 1 -- 172.21.15.33:0/3737186411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab809aed0 con 0x55d5edac3da0 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.787+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ab40a2c20 0x7f5ab40a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55d5edabf490 tx=0x7f5ac002a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.787+0000 7f5abcbfe640 1 -- 172.21.15.33:0/3737186411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab809ba90 con 0x55d5edac3da0 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.787+0000 7f5abcbfe640 1 -- 172.21.15.33:0/3737186411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab809acf0 con 0x55d5edac3da0 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- 172.21.15.33:0/3737186411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 msgr2=0x55d5edac6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edac6230 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5ab809b320 tx=0x7f5ab8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- 172.21.15.33:0/3737186411 shutdown_connections 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edac6230 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabec30 0x55d5edac3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/3737186411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabe240 0x55d5edabe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- 172.21.15.33:0/3737186411 >> 172.21.15.33:0/3737186411 conn(0x55d5edab98b0 msgr2=0x55d5edab9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- 172.21.15.33:0/3737186411 shutdown_connections 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- 172.21.15.33:0/3737186411 wait complete. 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- start start 2024-04-24T23:13:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 Processor -- start 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edac8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edac8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edacfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d5edac6c90 con 0x55d5edac3da0 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed9ddf50 con 0x55d5edabec30 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.788+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5edac6e10 con 0x55d5edabe240 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ad33f6740 10 client.0 _ll_register_callbacks cb 0x55d5eda19000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edacfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edacfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42868/0 (socket says 172.21.15.33:42868) 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 -- 172.21.15.33:0/1402141331 learned_addr learned my addr 172.21.15.33:0/1402141331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edac8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acdbad640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edac8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 -- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 msgr2=0x55d5edac8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edac8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 -- 172.21.15.33:0/1402141331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 msgr2=0x55d5edac8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edac8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 -- 172.21.15.33:0/1402141331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ed7363f0 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5acebaf640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edacfc10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5ac002aeb0 tx=0x7f5ac00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c61c382db721df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1402141331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac004f070 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1402141331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ac0004710 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab809ba90 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1402141331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ac005b050 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.789+0000 7f5ad33f6740 10 client.15732 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5ed6f60f0 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1402141331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab809ba90 con 0x55d5edac3da0 2024-04-24T23:13:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1402141331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ac0030650 con 0x55d5edac3da0 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 20 client.15732 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edad44d0 0x55d5edad6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 4 client.15732 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5eda2fbf0 con 0x55d5edad44d0 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.790+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edad44d0 0x55d5edad6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ab1ffb640 10 client.15732 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edad44d0 0x55d5edad6950 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a101875552b44fe9 server_cookie=701576fe621a5c48 in_seq=0 out_seq=0 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1402141331 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5eda2fbf0 con 0x55d5edad44d0 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ab1ffb640 10 client.15732 handle_command_reply: tid=0 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ad33f6740 1 client.15732 shutdown 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 msgr2=0x55d5edacfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edacfc10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5ac002aeb0 tx=0x7f5ac00095c0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 shutdown_connections 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edad44d0 0x55d5edad6950 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.791+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edacfc10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edac8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1402141331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edac8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 >> 172.21.15.33:0/1402141331 conn(0x55d5edab98b0 msgr2=0x55d5edab8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ab1ffb640 0 client.15732 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 shutdown_connections 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1402141331 wait complete. 2024-04-24T23:13:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.792+0000 7f5ad33f6740 20 client.15732 trim_cache size 0 max 16384 2024-04-24T23:13:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 Processor -- start 2024-04-24T23:13:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 -- start start 2024-04-24T23:13:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edad6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edacfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edabc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81acd10 con 0x55d5edac3da0 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81aa5e0 con 0x55d5edabec30 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81a8430 con 0x55d5edabe240 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edabc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edabc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42880/0 (socket says 172.21.15.33:42880) 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 -- 172.21.15.33:0/2996171600 learned_addr learned my addr 172.21.15.33:0/2996171600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acdbad640 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edacfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edad6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 -- 172.21.15.33:0/2996171600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 msgr2=0x55d5edad6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edad6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 -- 172.21.15.33:0/2996171600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 msgr2=0x55d5edacfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edacfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 -- 172.21.15.33:0/2996171600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5edaab090 con 0x55d5edac3da0 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acdbad640 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edacfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.810+0000 7f5acebaf640 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edabc860 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009bf0 tx=0x7f5ac0031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d810774ae2528a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ab3fff640 1 -- 172.21.15.33:0/2996171600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0064070 con 0x55d5edac3da0 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ab3fff640 1 -- 172.21.15.33:0/2996171600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac0058030 con 0x55d5edac3da0 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ab3fff640 1 -- 172.21.15.33:0/2996171600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac004f040 con 0x55d5edac3da0 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 -- 172.21.15.33:0/2996171600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 msgr2=0x55d5edabc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edabc860 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009bf0 tx=0x7f5ac0031970 comp rx=0 tx=0).stop 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 -- 172.21.15.33:0/2996171600 shutdown_connections 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edac3da0 0x55d5edabc860 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edabec30 0x55d5edacfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/2996171600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edad6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 -- 172.21.15.33:0/2996171600 >> 172.21.15.33:0/2996171600 conn(0x55d5edab98b0 msgr2=0x55d5edad6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.811+0000 7f5ad33f6740 1 -- 172.21.15.33:0/2996171600 shutdown_connections 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 -- 172.21.15.33:0/2996171600 wait complete. 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 -- start start 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 Processor -- start 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edaccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 0x55d5edacd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edac3da0 0x55d5edb38a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81a8aa0 con 0x55d5edabec30 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81a9f10 con 0x55d5edac3da0 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81a9170 con 0x55d5edabe240 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ad33f6740 10 client.0 _ll_register_callbacks cb 0x7f5ac0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 0x55d5edacd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 0x55d5edacd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42882/0 (socket says 172.21.15.33:42882) 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 -- 172.21.15.33:0/1842432905 learned_addr learned my addr 172.21.15.33:0/1842432905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edaccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acebaf640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edac3da0 0x55d5edb38a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 -- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 msgr2=0x55d5edaccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edaccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 -- 172.21.15.33:0/1842432905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edac3da0 msgr2=0x55d5edb38a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edac3da0 0x55d5edb38a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.812+0000 7f5acdbad640 1 -- 172.21.15.33:0/1842432905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5edb38ff0 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5acebaf640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edac3da0 0x55d5edb38a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5acdbad640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 0x55d5edacd140 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8099640 tx=0x7f5ab80998a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c388aeaff6e9e7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1842432905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab809ba90 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1842432905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ab809a7e0 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac0058030 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1842432905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ab809b390 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5edb3a370 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1842432905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac0058030 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1842432905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ab809a250 con 0x55d5edabec30 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 20 client.15738 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edb3a8c0 0x55d5edb3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 4 client.15738 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.813+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5edac8480 con 0x55d5edb3a8c0 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edb3a8c0 0x55d5edb3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ace3ae640 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edb3a8c0 0x55d5edb3cce0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f7dbaf4f13cb9c server_cookie=5869d654b746d4dc in_seq=0 out_seq=0 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ab1ffb640 10 client.15738 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ab1ffb640 1 -- 172.21.15.33:0/1842432905 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d5edac8480 con 0x55d5edb3a8c0 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ab1ffb640 10 client.15738 handle_command_reply: tid=0 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 client.15738 shutdown 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 msgr2=0x55d5edacd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 0x55d5edacd140 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8099640 tx=0x7f5ab80998a0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 shutdown_connections 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d5edb3a8c0 0x55d5edb3cce0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5edac3da0 0x55d5edb38a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ab1ffb640 0 client.15738 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5edabec30 0x55d5edacd140 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 --2- 172.21.15.33:0/1842432905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5edabe240 0x55d5edaccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 >> 172.21.15.33:0/1842432905 conn(0x55d5edab98b0 msgr2=0x55d5edab8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 shutdown_connections 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.814+0000 7f5ad33f6740 1 -- 172.21.15.33:0/1842432905 wait complete. 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad33f6740 20 client.15738 trim_cache size 0 max 16384 2024-04-24T23:13:15.825 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "d71f3e4d-d9ac-46f4-a9de-96fce6d37f5a": { 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d71f3e4d-d9ac-46f4-a9de-96fce6d37f5a", 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:13:15.826 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ab40a2c20 msgr2=0x7f5ab40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ab40a2c20 0x7f5ab40a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55d5edabf490 tx=0x7f5ac002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 msgr2=0x7f5ac8197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 0x7f5ac8197f80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800d950 tx=0x7f5ab800de10 comp rx=0 tx=0).stop 2024-04-24T23:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 shutdown_connections 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ab40a2c20 0x7f5ab40a50e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac8102230 0x7f5ac819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8101870 0x7f5ac8197f80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 --2- 172.21.15.33:0/2066941855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac8100680 0x7f5ac8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 >> 172.21.15.33:0/2066941855 conn(0x7f5ac80fbbf0 msgr2=0x7f5ac80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 shutdown_connections 2024-04-24T23:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:15.815+0000 7f5ad0612640 1 -- 172.21.15.33:0/2066941855 wait complete. 2024-04-24T23:13:15.829 INFO:tasks.cephfs.filesystem:scrub status for tag:d71f3e4d-d9ac-46f4-a9de-96fce6d37f5a - {'path': '/', 'tag': 'd71f3e4d-d9ac-46f4-a9de-96fce6d37f5a', 'options': 'recursive,force'} 2024-04-24T23:13:17.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:16 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:13:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:16 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:16 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:13:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:16 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:17.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:16 smithi038 ceph-mon[32450]: pgmap v389: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-24T23:13:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:16 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:13:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:16 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:16 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:13:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:16 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:16 smithi117 ceph-mon[32529]: pgmap v389: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-24T23:13:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:16 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:13:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:16 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:16 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:13:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:16 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:16 smithi033 ceph-mon[29201]: pgmap v389: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-24T23:13:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:17 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:13:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:17 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:13:18.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:17 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:13:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:18 smithi038 ceph-mon[32450]: pgmap v390: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:19.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:18 smithi117 ceph-mon[32529]: pgmap v390: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:19.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:18 smithi033 ceph-mon[29201]: pgmap v390: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:21.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:20 smithi038 ceph-mon[32450]: pgmap v391: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T23:13:21.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:20 smithi117 ceph-mon[32529]: pgmap v391: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T23:13:21.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:20 smithi033 ceph-mon[29201]: pgmap v391: 129 pgs: 129 active+clean; 656 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T23:13:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:22 smithi038 ceph-mon[32450]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:13:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:22 smithi117 ceph-mon[32529]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:13:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:22 smithi033 ceph-mon[29201]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:13:25.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:24 smithi038 ceph-mon[32450]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:25.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:24 smithi117 ceph-mon[32529]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:25.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:24 smithi033 ceph-mon[29201]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:26 smithi117 ceph-mon[32529]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T23:13:27.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:26 smithi033 ceph-mon[29201]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T23:13:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:26 smithi038 ceph-mon[32450]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T23:13:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:28 smithi117 ceph-mon[32529]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:13:29.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:28 smithi033 ceph-mon[29201]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:13:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:28 smithi038 ceph-mon[32450]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:13:31.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:30 smithi117 ceph-mon[32529]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:31.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:30 smithi033 ceph-mon[29201]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:31.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:30 smithi038 ceph-mon[32450]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:33.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:32 smithi117 ceph-mon[32529]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:13:33.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:32 smithi033 ceph-mon[29201]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:13:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:32 smithi038 ceph-mon[32450]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:13:35.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:34 smithi033 ceph-mon[29201]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:35.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:34 smithi038 ceph-mon[32450]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:34 smithi117 ceph-mon[32529]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:36 smithi033 ceph-mon[29201]: pgmap v399: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:36 smithi038 ceph-mon[32450]: pgmap v399: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:36 smithi117 ceph-mon[32529]: pgmap v399: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:39.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:38 smithi033 ceph-mon[29201]: pgmap v400: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:39.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:38 smithi038 ceph-mon[32450]: pgmap v400: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:38 smithi117 ceph-mon[32529]: pgmap v400: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:40 smithi033 ceph-mon[29201]: pgmap v401: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:41.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:40 smithi038 ceph-mon[32450]: pgmap v401: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:40 smithi117 ceph-mon[32529]: pgmap v401: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:43.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:42 smithi033 ceph-mon[29201]: pgmap v402: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:42 smithi038 ceph-mon[32450]: pgmap v402: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:42 smithi117 ceph-mon[32529]: pgmap v402: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:13:45.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:44 smithi033 ceph-mon[29201]: pgmap v403: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:45.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:44 smithi038 ceph-mon[32450]: pgmap v403: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:44 smithi117 ceph-mon[32529]: pgmap v403: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:45.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:13:45.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:13:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 -- 172.21.15.33:0/1425189591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c101870 msgr2=0x7f7f6c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1425189591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c101870 0x7f7f6c101cf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009900 tx=0x7f7f6002a410 comp rx=0 tx=0).stop 2024-04-24T23:13:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 -- 172.21.15.33:0/1425189591 shutdown_connections 2024-04-24T23:13:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1425189591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c102230 0x7f7f6c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1425189591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c101870 0x7f7f6c101cf0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1425189591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 0x7f7f6c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 -- 172.21.15.33:0/1425189591 >> 172.21.15.33:0/1425189591 conn(0x7f7f6c0fbbf0 msgr2=0x7f7f6c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 -- 172.21.15.33:0/1425189591 shutdown_connections 2024-04-24T23:13:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.938+0000 7f7f7089a640 1 -- 172.21.15.33:0/1425189591 wait complete. 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 Processor -- start 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 -- start start 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 0x7f7f6c104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 0x7f7f6c103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10c300 con 0x7f7f6c100680 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10c180 con 0x7f7f6c101870 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f7089a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10c480 con 0x7f7f6c102230 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 0x7f7f6c104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 0x7f7f6c104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45812/0 (socket says 172.21.15.33:45812) 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f6a59c640 1 -- 172.21.15.33:0/1160735111 learned_addr learned my addr 172.21.15.33:0/1160735111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.939+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 0x7f7f6c103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6a59c640 1 -- 172.21.15.33:0/1160735111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 msgr2=0x7f7f6c103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 0x7f7f6c103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6a59c640 1 -- 172.21.15.33:0/1160735111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 msgr2=0x7f7f6c102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6a59c640 1 -- 172.21.15.33:0/1160735111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1036d0 con 0x7f7f6c100680 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 0x7f7f6c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 0x7f7f6c104610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00d9c0 tx=0x7f7f5c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f5b7fe640 1 -- 172.21.15.33:0/1160735111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c017070 con 0x7f7f6c100680 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1039c0 con 0x7f7f6c100680 2024-04-24T23:13:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.940+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1a65a0 con 0x7f7f6c100680 2024-04-24T23:13:45.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.941+0000 7f7f5b7fe640 1 -- 172.21.15.33:0/1160735111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c009b40 con 0x7f7f6c100680 2024-04-24T23:13:45.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.941+0000 7f7f5b7fe640 1 -- 172.21.15.33:0/1160735111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c017070 con 0x7f7f6c100680 2024-04-24T23:13:45.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.943+0000 7f7f735a4740 1 Processor -- start 2024-04-24T23:13:45.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 -- start start 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b5b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b60730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b63230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618a4abf6b0 con 0x5618a4b5b240 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618a4a67880 con 0x5618a4b60da0 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f735a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618a4abb670 con 0x5618a4b5bc30 2024-04-24T23:13:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f6ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b5b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b63230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f6ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b5b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45820/0 (socket says 172.21.15.33:45820) 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b63230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43406/0 (socket says 172.21.15.33:43406) 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f6ad9d640 1 -- 172.21.15.33:0/3510990876 learned_addr learned my addr 172.21.15.33:0/3510990876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b60730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 -- 172.21.15.33:0/3510990876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 msgr2=0x5618a4b60730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b60730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 -- 172.21.15.33:0/3510990876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 msgr2=0x5618a4b5b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b5b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.944+0000 7f7f69d9b640 1 -- 172.21.15.33:0/3510990876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a489d7d0 con 0x5618a4b60da0 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.945+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b5b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.945+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b60730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.945+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b63230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f60031440 tx=0x7f7f60031760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c53cb0ef14824f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.945+0000 7f7f58ff9640 1 -- 172.21.15.33:0/3510990876 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60040070 con 0x5618a4b60da0 2024-04-24T23:13:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.945+0000 7f7f58ff9640 1 -- 172.21.15.33:0/3510990876 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60031a10 con 0x5618a4b60da0 2024-04-24T23:13:45.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.945+0000 7f7f58ff9640 1 -- 172.21.15.33:0/3510990876 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035040 con 0x5618a4b60da0 2024-04-24T23:13:45.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 -- 172.21.15.33:0/3510990876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 msgr2=0x5618a4b63230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b63230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f60031440 tx=0x7f7f60031760 comp rx=0 tx=0).stop 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 -- 172.21.15.33:0/3510990876 shutdown_connections 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b63230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b60730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 --2- 172.21.15.33:0/3510990876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b5b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.946+0000 7f7f735a4740 1 -- 172.21.15.33:0/3510990876 >> 172.21.15.33:0/3510990876 conn(0x5618a4b568b0 msgr2=0x5618a4b56cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.947+0000 7f7f5b7fe640 1 -- 172.21.15.33:0/1160735111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7f5c01a070 con 0x7f7f6c100680 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.947+0000 7f7f735a4740 1 -- 172.21.15.33:0/3510990876 shutdown_connections 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.947+0000 7f7f735a4740 1 -- 172.21.15.33:0/3510990876 wait complete. 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.947+0000 7f7f735a4740 1 -- start start 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f735a4740 1 Processor -- start 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f5b7fe640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f540a2db0 0x7f7f540a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f540a2db0 0x7f7f540a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f5b7fe640 1 -- 172.21.15.33:0/1160735111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f5c0cebf0 con 0x7f7f6c100680 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.948+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b6cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b655b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45828/0 (socket says 172.21.15.33:45828) 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f6a59c640 1 -- 172.21.15.33:0/1566475631 learned_addr learned my addr 172.21.15.33:0/1566475631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618a4b63cf0 con 0x5618a4b5b240 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618a4a7af50 con 0x5618a4b60da0 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618a4b63e70 con 0x5618a4b5bc30 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f540a2db0 0x7f7f540a5270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f640096a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f735a4740 10 client.0 _ll_register_callbacks cb 0x5618a4ab6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 -- 172.21.15.33:0/1566475631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 msgr2=0x5618a4b6cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b6cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 -- 172.21.15.33:0/1566475631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 msgr2=0x5618a4b655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a47d33f0 con 0x5618a4b5bc30 2024-04-24T23:13:45.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b65af0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f600314e0 tx=0x7f7f6003d700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa973ecb54de1c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:45.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:13:45.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.949+0000 7f7f51ffb640 1 -- 172.21.15.33:0/1566475631 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6004c030 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618a4accbf0 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f51ffb640 1 -- 172.21.15.33:0/1566475631 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f6003dbc0 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f51ffb640 1 -- 172.21.15.33:0/1566475631 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f60059050 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f735a4740 10 client.25345 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618a47930f0 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f51ffb640 1 -- 172.21.15.33:0/1566475631 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5618a4accbf0 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.950+0000 7f7f51ffb640 1 -- 172.21.15.33:0/1566475631 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f600534c0 con 0x5618a4b5bc30 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f735a4740 10 client.25345 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:45.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f735a4740 10 client.25345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f735a4740 20 client.25345 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f735a4740 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4b70ff0 0x5618a4b733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f735a4740 4 client.25345 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618a4ac6f40 con 0x5618a4b70ff0 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4b70ff0 0x5618a4b733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f51ffb640 10 client.25345 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.951+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4b70ff0 0x5618a4b733e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b56310ee220576 server_cookie=a8b863302010f621 in_seq=0 out_seq=0 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f51ffb640 1 -- 172.21.15.33:0/1566475631 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7f5c0baae0 con 0x5618a4b70ff0 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f51ffb640 10 client.25345 handle_command_reply: tid=0 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 client.25345 shutdown 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 msgr2=0x5618a4b65af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b65af0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f600314e0 tx=0x7f7f6003d700 comp rx=0 tx=0).stop 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 shutdown_connections 2024-04-24T23:13:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4b70ff0 0x5618a4b733e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b60da0 0x5618a4b6cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b5bc30 0x5618a4b65af0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 --2- 172.21.15.33:0/1566475631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 >> 172.21.15.33:0/1566475631 conn(0x5618a4b568b0 msgr2=0x5618a4b55af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f51ffb640 0 client.25345 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 shutdown_connections 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.952+0000 7f7f735a4740 1 -- 172.21.15.33:0/1566475631 wait complete. 2024-04-24T23:13:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.953+0000 7f7f735a4740 20 client.25345 trim_cache size 0 max 16384 2024-04-24T23:13:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 Processor -- start 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 -- start start 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5bc30 0x5618a4b6cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b3240 con 0x5618a4b5b240 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b0b10 con 0x5618a4b5bc30 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.987+0000 7f7f735a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ae960 con 0x5618a4b60da0 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b73000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45840/0 (socket says 172.21.15.33:45840) 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 -- 172.21.15.33:0/4171730226 learned_addr learned my addr 172.21.15.33:0/4171730226 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5bc30 0x5618a4b6cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 -- 172.21.15.33:0/4171730226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 msgr2=0x5618a4b59830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b59830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 -- 172.21.15.33:0/4171730226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5bc30 msgr2=0x5618a4b6cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5bc30 0x5618a4b6cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 -- 172.21.15.33:0/4171730226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a4b563f0 con 0x5618a4b5b240 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5bc30 0x5618a4b6cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b73000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005c60 tx=0x7f7f5c0972a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ac8c32bba1a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f53fff640 1 -- 172.21.15.33:0/4171730226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c021040 con 0x5618a4b5b240 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f53fff640 1 -- 172.21.15.33:0/4171730226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c0b5030 con 0x5618a4b5b240 2024-04-24T23:13:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.988+0000 7f7f53fff640 1 -- 172.21.15.33:0/4171730226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c09a3d0 con 0x5618a4b5b240 2024-04-24T23:13:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 -- 172.21.15.33:0/4171730226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 msgr2=0x5618a4b73000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b73000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005c60 tx=0x7f7f5c0972a0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 -- 172.21.15.33:0/4171730226 shutdown_connections 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5bc30 0x5618a4b6cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4171730226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5b240 0x5618a4b73000 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 -- 172.21.15.33:0/4171730226 >> 172.21.15.33:0/4171730226 conn(0x5618a4b568b0 msgr2=0x5618a4b5cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 -- 172.21.15.33:0/4171730226 shutdown_connections 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.989+0000 7f7f735a4740 1 -- 172.21.15.33:0/4171730226 wait complete. 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 -- start start 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 Processor -- start 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5b240 0x5618a4b6a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 0x5618a4b6a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b6b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f69d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 0x5618a4b6a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1aefd0 con 0x5618a4b5bc30 2024-04-24T23:13:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b0440 con 0x5618a4b5b240 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1af6a0 con 0x5618a4b60da0 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f69d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 0x5618a4b6a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45848/0 (socket says 172.21.15.33:45848) 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f69d9b640 1 -- 172.21.15.33:0/4112540061 learned_addr learned my addr 172.21.15.33:0/4112540061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.990+0000 7f7f735a4740 10 client.0 _ll_register_callbacks cb 0x7f7f64009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5b240 0x5618a4b6a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f6ad9d640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b6b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f69d9b640 1 -- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 msgr2=0x5618a4b6b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b6b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f69d9b640 1 -- 172.21.15.33:0/4112540061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5b240 msgr2=0x5618a4b6a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5b240 0x5618a4b6a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f69d9b640 1 -- 172.21.15.33:0/4112540061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a4b57620 con 0x5618a4b5bc30 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f69d9b640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 0x5618a4b6a8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f60053a80 tx=0x7f7f600058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6b8e670af99408 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f51ffb640 1 -- 172.21.15.33:0/4112540061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6004c040 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f51ffb640 1 -- 172.21.15.33:0/4112540061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f60031dd0 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c0b5030 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f51ffb640 1 -- 172.21.15.33:0/4112540061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f60040070 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f735a4740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.991+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618a4bd74d0 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f51ffb640 1 -- 172.21.15.33:0/4112540061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c0b5030 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f51ffb640 1 -- 172.21.15.33:0/4112540061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f60053110 con 0x5618a4b5bc30 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f735a4740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f735a4740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f735a4740 20 client.15756 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4bd7df0 0x5618a4bda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f735a4740 4 client.15756 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618a4a446f0 con 0x5618a4bd7df0 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.992+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4bd7df0 0x5618a4bda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f51ffb640 10 client.15756 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f6a59c640 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4bd7df0 0x5618a4bda210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ee7e10ff76f564 server_cookie=909b6cead5ac1cce in_seq=0 out_seq=0 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f51ffb640 1 -- 172.21.15.33:0/4112540061 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5618a4a446f0 con 0x5618a4bd7df0 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f51ffb640 10 client.15756 handle_command_reply: tid=0 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 client.15756 shutdown 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 msgr2=0x5618a4b6a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 0x5618a4b6a8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f60053a80 tx=0x7f7f600058d0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 shutdown_connections 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618a4bd7df0 0x5618a4bda210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f51ffb640 0 client.15756 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618a4b60da0 0x5618a4b6b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618a4b5bc30 0x5618a4b6a8f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 --2- 172.21.15.33:0/4112540061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618a4b5b240 0x5618a4b6a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 >> 172.21.15.33:0/4112540061 conn(0x5618a4b568b0 msgr2=0x5618a4b55260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 shutdown_connections 2024-04-24T23:13:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.993+0000 7f7f735a4740 1 -- 172.21.15.33:0/4112540061 wait complete. 2024-04-24T23:13:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f735a4740 20 client.15756 trim_cache size 0 max 16384 2024-04-24T23:13:46.001 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:13:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:13:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:13:46.001 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f540a2db0 msgr2=0x7f7f540a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f540a2db0 0x7f7f540a5270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f640096a0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 msgr2=0x7f7f6c104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 0x7f7f6c104610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00d9c0 tx=0x7f7f5c00de80 comp rx=0 tx=0).stop 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 shutdown_connections 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f540a2db0 0x7f7f540a5270 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 0x7f7f6c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 --2- 172.21.15.33:0/1160735111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c100680 0x7f7f6c104610 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 >> 172.21.15.33:0/1160735111 conn(0x7f7f6c0fbbf0 msgr2=0x7f7f6c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 shutdown_connections 2024-04-24T23:13:46.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:45.994+0000 7f7f7089a640 1 -- 172.21.15.33:0/1160735111 wait complete. 2024-04-24T23:13:46.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:13:46.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:13:46.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/181899542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8101870 msgr2=0x7fd6d8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/181899542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8101870 0x7fd6d8101cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009900 tx=0x7fd6c802a410 comp rx=0 tx=0).stop 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/181899542 shutdown_connections 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/181899542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d8102230 0x7fd6d8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/181899542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8101870 0x7fd6d8101cf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/181899542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8100680 0x7fd6d8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/181899542 >> 172.21.15.33:0/181899542 conn(0x7fd6d80fbbf0 msgr2=0x7fd6d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/181899542 shutdown_connections 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/181899542 wait complete. 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.113+0000 7fd6dfd4f640 1 Processor -- start 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 -- start start 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8100680 0x7fd6d8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 0x7fd6d8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d8102230 0x7fd6d819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d810c300 con 0x7fd6d8101870 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d810c180 con 0x7fd6d8100680 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dfd4f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d810c480 con 0x7fd6d8102230 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 0x7fd6d8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 0x7fd6d8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45876/0 (socket says 172.21.15.33:45876) 2024-04-24T23:13:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 -- 172.21.15.33:0/2613852231 learned_addr learned my addr 172.21.15.33:0/2613852231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6ddaeb640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8100680 0x7fd6d8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6de2ec640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d8102230 0x7fd6d819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 -- 172.21.15.33:0/2613852231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d8102230 msgr2=0x7fd6d819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d8102230 0x7fd6d819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 -- 172.21.15.33:0/2613852231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8100680 msgr2=0x7fd6d8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8100680 0x7fd6d8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 -- 172.21.15.33:0/2613852231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d819f6b0 con 0x7fd6d8101870 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.114+0000 7fd6dd2ea640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 0x7fd6d8197fd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009230 tx=0x7fd6c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.115+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c803f030 con 0x7fd6d8101870 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.115+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c8003ce0 con 0x7fd6d8101870 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.115+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8035070 con 0x7fd6d8101870 2024-04-24T23:13:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.115+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d819f940 con 0x7fd6d8101870 2024-04-24T23:13:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.115+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d819fe80 con 0x7fd6d8101870 2024-04-24T23:13:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.116+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d81a0110 con 0x7fd6d8101870 2024-04-24T23:13:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.119+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd6c8042030 con 0x7fd6d8101870 2024-04-24T23:13:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.119+0000 7fd6ceffd640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6c40a2d20 0x7fd6c40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.120+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd6c80eff80 con 0x7fd6d8101870 2024-04-24T23:13:46.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.126+0000 7fd6ddaeb640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6c40a2d20 0x7fd6c40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.126+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6c8049070 con 0x7fd6d8101870 2024-04-24T23:13:46.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.126+0000 7fd6ddaeb640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6c40a2d20 0x7fd6c40a51e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40095f0 tx=0x7fd6d4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.353+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6d81a02e0 con 0x7fd6d8101870 2024-04-24T23:13:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.354+0000 7fd6ceffd640 1 -- 172.21.15.33:0/2613852231 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7fd6c80bc990 con 0x7fd6d8101870 2024-04-24T23:13:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:13:46.356 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:12:35.176325+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:13:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-24T23:13:46.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6c40a2d20 msgr2=0x7fd6c40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6c40a2d20 0x7fd6c40a51e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40095f0 tx=0x7fd6d4005f90 comp rx=0 tx=0).stop 2024-04-24T23:13:46.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 msgr2=0x7fd6d8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 0x7fd6d8197fd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009230 tx=0x7fd6c80042f0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 shutdown_connections 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6c40a2d20 0x7fd6c40a51e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d8102230 0x7fd6d819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8101870 0x7fd6d8197fd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 --2- 172.21.15.33:0/2613852231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6d8100680 0x7fd6d8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 >> 172.21.15.33:0/2613852231 conn(0x7fd6d80fbbf0 msgr2=0x7fd6d80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.356+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 shutdown_connections 2024-04-24T23:13:46.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.357+0000 7fd6dfd4f640 1 -- 172.21.15.33:0/2613852231 wait complete. 2024-04-24T23:13:46.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:13:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 -- 172.21.15.33:0/4045054621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0100680 msgr2=0x7fcec0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 --2- 172.21.15.33:0/4045054621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0100680 0x7fcec0100aa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009900 tx=0x7fceb002a410 comp rx=0 tx=0).stop 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 -- 172.21.15.33:0/4045054621 shutdown_connections 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 --2- 172.21.15.33:0/4045054621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec0102230 0x7fcec0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 --2- 172.21.15.33:0/4045054621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 0x7fcec0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 --2- 172.21.15.33:0/4045054621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0100680 0x7fcec0100aa0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 -- 172.21.15.33:0/4045054621 >> 172.21.15.33:0/4045054621 conn(0x7fcec00fbbf0 msgr2=0x7fcec00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 -- 172.21.15.33:0/4045054621 shutdown_connections 2024-04-24T23:13:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.475+0000 7fcec48ba640 1 -- 172.21.15.33:0/4045054621 wait complete. 2024-04-24T23:13:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.476+0000 7fcec48ba640 1 Processor -- start 2024-04-24T23:13:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.476+0000 7fcec48ba640 1 -- start start 2024-04-24T23:13:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.476+0000 7fcec48ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec0100680 0x7fcec010b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.476+0000 7fcec48ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 0x7fcec0109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.476+0000 7fcec48ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0102230 0x7fcec0109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcec48ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010d420 con 0x7fcec0102230 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcec48ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010d2a0 con 0x7fcec0101870 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcec48ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010d5a0 con 0x7fcec0100680 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0102230 0x7fcec0109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec0100680 0x7fcec010b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0102230 0x7fcec0109b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45898/0 (socket says 172.21.15.33:45898) 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 0x7fcec0109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebed9d640 1 -- 172.21.15.33:0/230914334 learned_addr learned my addr 172.21.15.33:0/230914334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 -- 172.21.15.33:0/230914334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec0100680 msgr2=0x7fcec010b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec0100680 0x7fcec010b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 -- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0102230 msgr2=0x7fcec0109b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0102230 0x7fcec0109b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 -- 172.21.15.33:0/230914334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec010a410 con 0x7fcec0101870 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 0x7fcec0109640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fceb400e9f0 tx=0x7fceb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.477+0000 7fceaf7fe640 1 -- 172.21.15.33:0/230914334 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb400ccc0 con 0x7fcec0101870 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.478+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec01a1c20 con 0x7fcec0101870 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.478+0000 7fceaf7fe640 1 -- 172.21.15.33:0/230914334 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb4010910 con 0x7fcec0101870 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.478+0000 7fceaf7fe640 1 -- 172.21.15.33:0/230914334 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4018960 con 0x7fcec0101870 2024-04-24T23:13:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.478+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec01a2170 con 0x7fcec0101870 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fceaf7fe640 1 -- 172.21.15.33:0/230914334 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fceb4010ac0 con 0x7fcec0101870 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 Processor -- start 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 -- start start 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 0x5557867def30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555786735670 con 0x5557867da430 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557866dd970 con 0x5557867d9a40 2024-04-24T23:13:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.481+0000 7fcec75ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555786731630 con 0x5557867df5a0 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 0x5557867def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fceaf7fe640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcea80a2c20 0x7fcea80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 0x5557867def30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45910/0 (socket says 172.21.15.33:45910) 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 -- 172.21.15.33:0/2757633177 learned_addr learned my addr 172.21.15.33:0/2757633177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebed9d640 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebed9d640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcea80a2c20 0x7fcea80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fceaf7fe640 1 -- 172.21.15.33:0/230914334 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fceb40cf8d0 con 0x7fcec0101870 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 -- 172.21.15.33:0/2757633177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 msgr2=0x5557867e1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867e1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 -- 172.21.15.33:0/2757633177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 msgr2=0x5557867d9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867d9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebe59c640 1 -- 172.21.15.33:0/2757633177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555786701400 con 0x5557867da430 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.482+0000 7fcebed9d640 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.483+0000 7fcebe59c640 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 0x5557867def30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fceb0031610 tx=0x7fceb0032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d36fbdd14c78626 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.483+0000 7fcebed9d640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcea80a2c20 0x7fcea80a50e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5557867dac90 tx=0x7fceb8004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.483+0000 7fceacff9640 1 -- 172.21.15.33:0/2757633177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0040070 con 0x5557867da430 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.483+0000 7fceacff9640 1 -- 172.21.15.33:0/2757633177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb0032e20 con 0x5557867da430 2024-04-24T23:13:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.483+0000 7fceacff9640 1 -- 172.21.15.33:0/2757633177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0035040 con 0x5557867da430 2024-04-24T23:13:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 -- 172.21.15.33:0/2757633177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 msgr2=0x5557867def30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 0x5557867def30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fceb0031610 tx=0x7fceb0032c30 comp rx=0 tx=0).stop 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 -- 172.21.15.33:0/2757633177 shutdown_connections 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867e1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867da430 0x5557867def30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 --2- 172.21.15.33:0/2757633177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 -- 172.21.15.33:0/2757633177 >> 172.21.15.33:0/2757633177 conn(0x5557867d50b0 msgr2=0x5557867d54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 -- 172.21.15.33:0/2757633177 shutdown_connections 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 -- 172.21.15.33:0/2757633177 wait complete. 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.484+0000 7fcec75ca740 1 -- start start 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 Processor -- start 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 0x5557867e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867da430 0x5557867e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557867e24f0 con 0x5557867d9a40 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557866f1050 con 0x5557867da430 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557867e2670 con 0x5557867df5a0 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 0x5557867e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 0x5557867e3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45924/0 (socket says 172.21.15.33:45924) 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 -- 172.21.15.33:0/4013041906 learned_addr learned my addr 172.21.15.33:0/4013041906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcec75ca740 10 client.0 _ll_register_callbacks cb 0x555786448be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867da430 0x5557867e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebed9d640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 -- 172.21.15.33:0/4013041906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 msgr2=0x5557867eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 -- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867da430 msgr2=0x5557867e40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867da430 0x5557867e40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.485+0000 7fcebe59c640 1 -- 172.21.15.33:0/4013041906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557866c2dc0 con 0x5557867d9a40 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcebe59c640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 0x5557867e3b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fceb002a930 tx=0x7fceb0005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b096906a1795b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcea5ffb640 1 -- 172.21.15.33:0/4013041906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0040050 con 0x5557867d9a40 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcea5ffb640 1 -- 172.21.15.33:0/4013041906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fceb0031770 con 0x5557867d9a40 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb0032e20 con 0x5557867d9a40 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcea5ffb640 1 -- 172.21.15.33:0/4013041906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fceb0059050 con 0x5557867d9a40 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcec75ca740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557864493f0 con 0x5557867d9a40 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.486+0000 7fcea5ffb640 1 -- 172.21.15.33:0/4013041906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb0031970 con 0x5557867d9a40 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcea5ffb640 1 -- 172.21.15.33:0/4013041906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fceb0004780 con 0x5557867d9a40 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcec75ca740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcec75ca740 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcec75ca740 20 client.15774 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcec75ca740 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5557867ef5d0 0x5557867f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcec75ca740 4 client.15774 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fceb0032e20 con 0x5557867ef5d0 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5557867ef5d0 0x5557867f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcea5ffb640 10 client.15774 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.487+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5557867ef5d0 0x5557867f19c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ece653275725050e server_cookie=231fdc5874f7a269 in_seq=0 out_seq=0 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcea5ffb640 1 -- 172.21.15.33:0/4013041906 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fceb0032e20 con 0x5557867ef5d0 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcea5ffb640 10 client.15774 handle_command_reply: tid=0 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 client.15774 shutdown 2024-04-24T23:13:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 msgr2=0x5557867e3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 0x5557867e3b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fceb002a930 tx=0x7fceb0005770 comp rx=0 tx=0).stop 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 shutdown_connections 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5557867ef5d0 0x5557867f19c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867df5a0 0x5557867eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867da430 0x5557867e40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 --2- 172.21.15.33:0/4013041906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867d9a40 0x5557867e3b80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 >> 172.21.15.33:0/4013041906 conn(0x5557867d50b0 msgr2=0x5557867cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcea5ffb640 0 client.15774 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 shutdown_connections 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.488+0000 7fcec75ca740 1 -- 172.21.15.33:0/4013041906 wait complete. 2024-04-24T23:13:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.489+0000 7fcec75ca740 20 client.15774 trim_cache size 0 max 16384 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.515+0000 7fcec75ca740 1 Processor -- start 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.515+0000 7fcec75ca740 1 -- start start 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcec75ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01aab10 con 0x5557867df5a0 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcec75ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a83e0 con 0x5557867d9a40 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcec75ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a6230 con 0x5557867da430 2024-04-24T23:13:46.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45936/0 (socket says 172.21.15.33:45936) 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 -- 172.21.15.33:0/736326960 learned_addr learned my addr 172.21.15.33:0/736326960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebe59c640 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 -- 172.21.15.33:0/736326960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 msgr2=0x5557867eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 -- 172.21.15.33:0/736326960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 msgr2=0x5557867cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.516+0000 7fcebed9d640 1 -- 172.21.15.33:0/736326960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557865137d0 con 0x5557867df5a0 2024-04-24T23:13:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.517+0000 7fcebed9d640 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e2ef0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fceb8013400 tx=0x7fceb8009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3ce02bba50c75e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.517+0000 7fcea7fff640 1 -- 172.21.15.33:0/736326960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8004d20 con 0x5557867df5a0 2024-04-24T23:13:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.517+0000 7fcea7fff640 1 -- 172.21.15.33:0/736326960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb8026070 con 0x5557867df5a0 2024-04-24T23:13:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.517+0000 7fcea7fff640 1 -- 172.21.15.33:0/736326960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb80055c0 con 0x5557867df5a0 2024-04-24T23:13:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 -- 172.21.15.33:0/736326960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 msgr2=0x5557867e2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e2ef0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fceb8013400 tx=0x7fceb8009cf0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 -- 172.21.15.33:0/736326960 shutdown_connections 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e2ef0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 --2- 172.21.15.33:0/736326960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 -- 172.21.15.33:0/736326960 >> 172.21.15.33:0/736326960 conn(0x5557867d50b0 msgr2=0x5557867d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 -- 172.21.15.33:0/736326960 shutdown_connections 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 -- 172.21.15.33:0/736326960 wait complete. 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 -- start start 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.518+0000 7fcec75ca740 1 Processor -- start 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a68a0 con 0x5557867df5a0 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867e82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43502/0 (socket says 172.21.15.33:43502) 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 -- 172.21.15.33:0/1207856667 learned_addr learned my addr 172.21.15.33:0/1207856667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a7d10 con 0x5557867d9a40 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a6f70 con 0x5557867da430 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebed9d640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcec75ca740 10 client.0 _ll_register_callbacks cb 0x7fceb8009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 -- 172.21.15.33:0/1207856667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 msgr2=0x5557867e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 -- 172.21.15.33:0/1207856667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 msgr2=0x5557867e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.519+0000 7fcebe59c640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcebe59c640 1 -- 172.21.15.33:0/1207856667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557867e9da0 con 0x5557867d9a40 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcebed9d640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:46.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcebe59c640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867e82d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5557867d5f50 tx=0x7fceb0004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fe5d96b36ec9ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcea5ffb640 1 -- 172.21.15.33:0/1207856667 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0062060 con 0x5557867d9a40 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb8026070 con 0x5557867d9a40 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcea5ffb640 1 -- 172.21.15.33:0/1207856667 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fceb0031980 con 0x5557867d9a40 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcea5ffb640 1 -- 172.21.15.33:0/1207856667 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fceb0069050 con 0x5557867d9a40 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcec75ca740 10 client.25334 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.520+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557867eb060 con 0x5557867d9a40 2024-04-24T23:13:46.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcea5ffb640 1 -- 172.21.15.33:0/1207856667 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb8026070 con 0x5557867d9a40 2024-04-24T23:13:46.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcea5ffb640 1 -- 172.21.15.33:0/1207856667 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fceb00051a0 con 0x5557867d9a40 2024-04-24T23:13:46.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcec75ca740 10 client.25334 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:46.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcec75ca740 10 client.25334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:13:46.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcec75ca740 20 client.25334 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:46.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcec75ca740 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555786855ce0 0x555786858100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcec75ca740 4 client.25334 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555786742bb0 con 0x555786855ce0 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.521+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555786855ce0 0x555786858100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcea5ffb640 10 client.25334 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcebdd9b640 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555786855ce0 0x555786858100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c30dc6366e7604d server_cookie=5c92b6195599156d in_seq=0 out_seq=0 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcea5ffb640 1 -- 172.21.15.33:0/1207856667 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555786742bb0 con 0x555786855ce0 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcea5ffb640 10 client.25334 handle_command_reply: tid=0 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 client.25334 shutdown 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 msgr2=0x5557867e82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867e82d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5557867d5f50 tx=0x7fceb0004a30 comp rx=0 tx=0).stop 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 shutdown_connections 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555786855ce0 0x555786858100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557867df5a0 0x5557867e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcea5ffb640 0 client.25334 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557867da430 0x5557867e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 --2- 172.21.15.33:0/1207856667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557867d9a40 0x5557867e82d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 >> 172.21.15.33:0/1207856667 conn(0x5557867d50b0 msgr2=0x5557867dacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 shutdown_connections 2024-04-24T23:13:46.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.522+0000 7fcec75ca740 1 -- 172.21.15.33:0/1207856667 wait complete. 2024-04-24T23:13:46.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.523+0000 7fcec75ca740 20 client.25334 trim_cache size 0 max 16384 2024-04-24T23:13:46.529 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:13:46.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.523+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcea80a2c20 msgr2=0x7fcea80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.523+0000 7fcec48ba640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcea80a2c20 0x7fcea80a50e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5557867dac90 tx=0x7fceb8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.523+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 msgr2=0x7fcec0109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.523+0000 7fcec48ba640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 0x7fcec0109640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fceb400e9f0 tx=0x7fceb400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 shutdown_connections 2024-04-24T23:13:46.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcea80a2c20 0x7fcea80a50e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0102230 0x7fcec0109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec0101870 0x7fcec0109640 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 --2- 172.21.15.33:0/230914334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcec0100680 0x7fcec010b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 >> 172.21.15.33:0/230914334 conn(0x7fcec00fbbf0 msgr2=0x7fcec0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 shutdown_connections 2024-04-24T23:13:46.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.524+0000 7fcec48ba640 1 -- 172.21.15.33:0/230914334 wait complete. 2024-04-24T23:13:46.541 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:13:46.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 -- 172.21.15.33:0/3447155913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c103340 msgr2=0x7fa85c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 --2- 172.21.15.33:0/3447155913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c103340 0x7fa85c109bd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa854009ca0 tx=0x7fa85402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 -- 172.21.15.33:0/3447155913 shutdown_connections 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 --2- 172.21.15.33:0/3447155913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c103340 0x7fa85c109bd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 --2- 172.21.15.33:0/3447155913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c102980 0x7fa85c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 --2- 172.21.15.33:0/3447155913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa85c101790 0x7fa85c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 -- 172.21.15.33:0/3447155913 >> 172.21.15.33:0/3447155913 conn(0x7fa85c0fcd20 msgr2=0x7fa85c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 -- 172.21.15.33:0/3447155913 shutdown_connections 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.695+0000 7fa86245d640 1 -- 172.21.15.33:0/3447155913 wait complete. 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 Processor -- start 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 -- start start 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 0x7fa85c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa85c102980 0x7fa85c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c103340 0x7fa85c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10d3f0 con 0x7fa85c101790 2024-04-24T23:13:46.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10d270 con 0x7fa85c102980 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.696+0000 7fa86245d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10d570 con 0x7fa85c103340 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 0x7fa85c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 0x7fa85c0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45958/0 (socket says 172.21.15.33:45958) 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 -- 172.21.15.33:0/936079636 learned_addr learned my addr 172.21.15.33:0/936079636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa85c102980 0x7fa85c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa8609fa640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c103340 0x7fa85c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 -- 172.21.15.33:0/936079636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c103340 msgr2=0x7fa85c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c103340 0x7fa85c074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 -- 172.21.15.33:0/936079636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa85c102980 msgr2=0x7fa85c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa85c102980 0x7fa85c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 -- 172.21.15.33:0/936079636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c074900 con 0x7fa85c101790 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa8609fa640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c103340 0x7fa85c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa85bfff640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 0x7fa85c0754f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e9c0 tx=0x7fa85000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa8597fa640 1 -- 172.21.15.33:0/936079636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000cde0 con 0x7fa85c101790 2024-04-24T23:13:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1a1b80 con 0x7fa85c101790 2024-04-24T23:13:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.697+0000 7fa8597fa640 1 -- 172.21.15.33:0/936079636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa850010910 con 0x7fa85c101790 2024-04-24T23:13:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.698+0000 7fa8597fa640 1 -- 172.21.15.33:0/936079636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500189d0 con 0x7fa85c101790 2024-04-24T23:13:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.698+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c1a2120 con 0x7fa85c101790 2024-04-24T23:13:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.700+0000 7fa86516c740 1 Processor -- start 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.700+0000 7fa86516c740 1 -- start start 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa86516c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa86516c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa86516c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa86516c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56148594f670 con 0x5614859f0d90 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa86516c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5614858f7970 con 0x5614859ebc20 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa86516c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56148594b630 con 0x5614859eb230 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45966/0 (socket says 172.21.15.33:45966) 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 -- 172.21.15.33:0/2992711271 learned_addr learned my addr 172.21.15.33:0/2992711271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa8597fa640 1 -- 172.21.15.33:0/936079636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa850010ac0 con 0x7fa85c101790 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa8609fa640 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 -- 172.21.15.33:0/2992711271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 msgr2=0x5614859eb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859eb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 -- 172.21.15.33:0/2992711271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 msgr2=0x5614859f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 -- 172.21.15.33:0/2992711271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148591b400 con 0x5614859f0d90 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa8609fa640 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.701+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859f3220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00bc80 tx=0x7fa84c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46cccccb443950e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.702+0000 7fa8427fc640 1 -- 172.21.15.33:0/2992711271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c015070 con 0x5614859f0d90 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.702+0000 7fa8427fc640 1 -- 172.21.15.33:0/2992711271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c01a940 con 0x5614859f0d90 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.702+0000 7fa8427fc640 1 -- 172.21.15.33:0/2992711271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c011a80 con 0x5614859f0d90 2024-04-24T23:13:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.702+0000 7fa8597fa640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8440a2ca0 0x7fa8440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.702+0000 7fa8609fa640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8440a2ca0 0x7fa8440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.702+0000 7fa8597fa640 1 -- 172.21.15.33:0/936079636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8500cfca0 con 0x7fa85c101790 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 -- 172.21.15.33:0/2992711271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 msgr2=0x5614859f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859f3220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00bc80 tx=0x7fa84c00f730 comp rx=0 tx=0).stop 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa8609fa640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8440a2ca0 0x7fa8440a5160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa854009430 tx=0x7fa854005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 -- 172.21.15.33:0/2992711271 shutdown_connections 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859f3220 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 --2- 172.21.15.33:0/2992711271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 -- 172.21.15.33:0/2992711271 >> 172.21.15.33:0/2992711271 conn(0x5614859e68a0 msgr2=0x5614859e6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 -- 172.21.15.33:0/2992711271 shutdown_connections 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 -- 172.21.15.33:0/2992711271 wait complete. 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 -- start start 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.703+0000 7fa86516c740 1 Processor -- start 2024-04-24T23:13:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5614859f3d40 con 0x5614859f0d90 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56148590b050 con 0x5614859ebc20 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5614859f3ec0 con 0x5614859eb230 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa86516c740 10 client.0 _ll_register_callbacks cb 0x561485662be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa8609fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa8609fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859fc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45978/0 (socket says 172.21.15.33:45978) 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa8609fa640 1 -- 172.21.15.33:0/3970705558 learned_addr learned my addr 172.21.15.33:0/3970705558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.704+0000 7fa85bfff640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa8609fa640 1 -- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 msgr2=0x5614859f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa8609fa640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa8609fa640 1 -- 172.21.15.33:0/3970705558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 msgr2=0x5614859f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa8609fa640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa8609fa640 1 -- 172.21.15.33:0/3970705558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614858dcdc0 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa8609fa640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859fc950 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa854031f30 tx=0x7fa8540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af82b0269c06c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa83b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa83b7fe640 1 -- 172.21.15.33:0/3970705558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8540047c0 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa83b7fe640 1 -- 172.21.15.33:0/3970705558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa85402f610 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c01a940 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa83b7fe640 1 -- 172.21.15.33:0/3970705558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8540318c0 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa86516c740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.705+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614856633f0 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa83b7fe640 1 -- 172.21.15.33:0/3970705558 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c01a940 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa83b7fe640 1 -- 172.21.15.33:0/3970705558 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa854043040 con 0x5614859f0d90 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa86516c740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa86516c740 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa86516c740 20 client.15792 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa86516c740 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a012d0 0x561485a036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa86516c740 4 client.15792 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148595cbb0 con 0x561485a012d0 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.706+0000 7fa85bfff640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a012d0 0x561485a036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.707+0000 7fa83b7fe640 10 client.15792 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.707+0000 7fa85bfff640 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a012d0 0x561485a036f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7753c108d9ff2a3f server_cookie=b57f1cca027c77c0 in_seq=0 out_seq=0 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.707+0000 7fa83b7fe640 1 -- 172.21.15.33:0/3970705558 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56148595cbb0 con 0x561485a012d0 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.707+0000 7fa83b7fe640 10 client.15792 handle_command_reply: tid=0 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.707+0000 7fa86516c740 1 client.15792 shutdown 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 msgr2=0x5614859fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859fc950 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa854031f30 tx=0x7fa8540095c0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 shutdown_connections 2024-04-24T23:13:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a012d0 0x561485a036f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859f0d90 0x5614859fc950 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa83b7fe640 0 client.15792 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859ebc20 0x5614859f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 --2- 172.21.15.33:0/3970705558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859eb230 0x5614859f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 >> 172.21.15.33:0/3970705558 conn(0x5614859e68a0 msgr2=0x5614859e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 shutdown_connections 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 1 -- 172.21.15.33:0/3970705558 wait complete. 2024-04-24T23:13:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.708+0000 7fa86516c740 20 client.15792 trim_cache size 0 max 16384 2024-04-24T23:13:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.721+0000 7fa86516c740 1 Processor -- start 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 -- start start 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 0x5614859ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859ebc20 0x5614859fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1aedf0 con 0x5614859eb230 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ac6c0 con 0x5614859f0d90 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa86516c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1aa510 con 0x5614859ebc20 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 0x5614859ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 0x5614859ec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45994/0 (socket says 172.21.15.33:45994) 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 -- 172.21.15.33:0/3016170948 learned_addr learned my addr 172.21.15.33:0/3016170948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa8609fa640 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 -- 172.21.15.33:0/3016170948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859ebc20 msgr2=0x5614859fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859ebc20 0x5614859fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 -- 172.21.15.33:0/3016170948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 msgr2=0x5614859ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859ecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.722+0000 7fa85bfff640 1 -- 172.21.15.33:0/3016170948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148572d7d0 con 0x5614859eb230 2024-04-24T23:13:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.723+0000 7fa85bfff640 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 0x5614859ec6c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c6a0 tx=0x7fa850009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269c033f7ec319c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.723+0000 7fa841ffb640 1 -- 172.21.15.33:0/3016170948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500a0360 con 0x5614859eb230 2024-04-24T23:13:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.723+0000 7fa841ffb640 1 -- 172.21.15.33:0/3016170948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8500a4c20 con 0x5614859eb230 2024-04-24T23:13:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.723+0000 7fa841ffb640 1 -- 172.21.15.33:0/3016170948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850098620 con 0x5614859eb230 2024-04-24T23:13:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.723+0000 7fa86516c740 1 -- 172.21.15.33:0/3016170948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 msgr2=0x5614859ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.723+0000 7fa86516c740 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 0x5614859ec6c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c6a0 tx=0x7fa850009490 comp rx=0 tx=0).stop 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 -- 172.21.15.33:0/3016170948 shutdown_connections 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859ecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859ebc20 0x5614859fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 --2- 172.21.15.33:0/3016170948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859eb230 0x5614859ec6c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 -- 172.21.15.33:0/3016170948 >> 172.21.15.33:0/3016170948 conn(0x5614859e68a0 msgr2=0x5614859e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 -- 172.21.15.33:0/3016170948 shutdown_connections 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 -- 172.21.15.33:0/3016170948 wait complete. 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 -- start start 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 Processor -- start 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 0x5614859f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.724+0000 7fa86516c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859fb080 0x5614859fb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa86516c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1aab80 con 0x5614859ebc20 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa86516c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1abff0 con 0x5614859f0d90 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa86516c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ab250 con 0x5614859fb080 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 0x5614859f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 0x5614859f9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46000/0 (socket says 172.21.15.33:46000) 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 -- 172.21.15.33:0/1341659688 learned_addr learned my addr 172.21.15.33:0/1341659688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa86516c740 10 client.0 _ll_register_callbacks cb 0x7fa850005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa8609fa640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859fb080 0x5614859fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 -- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859fb080 msgr2=0x5614859fb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859fb080 0x5614859fb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 -- 172.21.15.33:0/1341659688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 msgr2=0x5614859fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 -- 172.21.15.33:0/1341659688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614859fba40 con 0x5614859ebc20 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa85bfff640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 0x5614859f9b30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa85009bf40 tx=0x7fa850016e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d0e6c8527f20d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa83b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa83b7fe640 1 -- 172.21.15.33:0/1341659688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500b4040 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa83b7fe640 1 -- 172.21.15.33:0/1341659688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8500104a0 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.725+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8500a4c20 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa83b7fe640 1 -- 172.21.15.33:0/1341659688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8500670d0 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561485a67810 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa83b7fe640 1 -- 172.21.15.33:0/1341659688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8500a4c20 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa83b7fe640 1 -- 172.21.15.33:0/1341659688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa85009b400 con 0x5614859ebc20 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 20 client.15798 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a68000 0x561485a6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 4 client.15798 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614859f51c0 con 0x561485a68000 2024-04-24T23:13:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.726+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a68000 0x561485a6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa83b7fe640 10 client.15798 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa85b7fe640 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a68000 0x561485a6a420 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16127074d0e0b91d server_cookie=6443e8c43bbf43bf in_seq=0 out_seq=0 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa83b7fe640 1 -- 172.21.15.33:0/1341659688 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614859f51c0 con 0x561485a68000 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa83b7fe640 10 client.15798 handle_command_reply: tid=0 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa86516c740 1 client.15798 shutdown 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 msgr2=0x5614859f9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa86516c740 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 0x5614859f9b30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa85009bf40 tx=0x7fa850016e30 comp rx=0 tx=0).stop 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 shutdown_connections 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.727+0000 7fa86516c740 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561485a68000 0x561485a6a420 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614859fb080 0x5614859fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa83b7fe640 0 client.15798 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614859f0d90 0x5614859fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 1 --2- 172.21.15.33:0/1341659688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614859ebc20 0x5614859f9b30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 >> 172.21.15.33:0/1341659688 conn(0x5614859e68a0 msgr2=0x5614859e52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 shutdown_connections 2024-04-24T23:13:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 1 -- 172.21.15.33:0/1341659688 wait complete. 2024-04-24T23:13:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86516c740 20 client.15798 trim_cache size 0 max 16384 2024-04-24T23:13:46.735 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8440a2ca0 msgr2=0x7fa8440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86245d640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8440a2ca0 0x7fa8440a5160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa854009430 tx=0x7fa854005ef0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 msgr2=0x7fa85c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.728+0000 7fa86245d640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 0x7fa85c0754f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e9c0 tx=0x7fa85000ee80 comp rx=0 tx=0).stop 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 shutdown_connections 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8440a2ca0 0x7fa8440a5160 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c103340 0x7fa85c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa85c102980 0x7fa85c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 --2- 172.21.15.33:0/936079636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa85c101790 0x7fa85c0754f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 >> 172.21.15.33:0/936079636 conn(0x7fa85c0fcd20 msgr2=0x7fa85c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 shutdown_connections 2024-04-24T23:13:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.729+0000 7fa86245d640 1 -- 172.21.15.33:0/936079636 wait complete. 2024-04-24T23:13:46.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:13:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 -- 172.21.15.33:0/1086605008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 msgr2=0x7efe58100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/1086605008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe58100e10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009900 tx=0x7efe4c02a410 comp rx=0 tx=0).stop 2024-04-24T23:13:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 -- 172.21.15.33:0/1086605008 shutdown_connections 2024-04-24T23:13:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/1086605008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe58101350 0x7efe581017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/1086605008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe58100e10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/1086605008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe58100050 0x7efe58100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 -- 172.21.15.33:0/1086605008 >> 172.21.15.33:0/1086605008 conn(0x7efe580fbbf0 msgr2=0x7efe580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 -- 172.21.15.33:0/1086605008 shutdown_connections 2024-04-24T23:13:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 -- 172.21.15.33:0/1086605008 wait complete. 2024-04-24T23:13:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.853+0000 7efe5e1d5640 1 Processor -- start 2024-04-24T23:13:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 -- start start 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe58100050 0x7efe58199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe5819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe58101350 0x7efe581a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810f5e0 con 0x7efe581009b0 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810f460 con 0x7efe58100050 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe5e1d5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810f760 con 0x7efe58101350 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe5819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe5819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46016/0 (socket says 172.21.15.33:46016) 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 -- 172.21.15.33:0/742401151 learned_addr learned my addr 172.21.15.33:0/742401151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe577fe640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe58100050 0x7efe58199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 -- 172.21.15.33:0/742401151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe58101350 msgr2=0x7efe581a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe58101350 0x7efe581a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 -- 172.21.15.33:0/742401151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe58100050 msgr2=0x7efe58199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe58100050 0x7efe58199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 -- 172.21.15.33:0/742401151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe581a1920 con 0x7efe581009b0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.854+0000 7efe56ffd640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe5819a250 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009230 tx=0x7efe4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.855+0000 7efe54ff9640 1 -- 172.21.15.33:0/742401151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c03f030 con 0x7efe581009b0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.855+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe581a1bb0 con 0x7efe581009b0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.855+0000 7efe54ff9640 1 -- 172.21.15.33:0/742401151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c003ce0 con 0x7efe581009b0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.855+0000 7efe54ff9640 1 -- 172.21.15.33:0/742401151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c035070 con 0x7efe581009b0 2024-04-24T23:13:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.855+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a20f0 con 0x7efe581009b0 2024-04-24T23:13:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.858+0000 7efe54ff9640 1 -- 172.21.15.33:0/742401151 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7efe4c042030 con 0x7efe581009b0 2024-04-24T23:13:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.858+0000 7efe60fc2740 1 Processor -- start 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.858+0000 7efe60fc2740 1 -- start start 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe54ff9640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe400a2c20 0x7efe400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a906650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a90b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a90e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe60fc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c45a86a670 con 0x55c45a90bd90 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe60fc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c45a812970 con 0x55c45a906c20 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe60fc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c45a866630 con 0x55c45a906230 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe54ff9640 1 -- 172.21.15.33:0/742401151 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe4c0efc50 con 0x7efe581009b0 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a90e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a90e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46018/0 (socket says 172.21.15.33:46018) 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe57fff640 1 -- 172.21.15.33:0/2237246195 learned_addr learned my addr 172.21.15.33:0/2237246195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe577fe640 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a906650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe57fff640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe400a2c20 0x7efe400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe56ffd640 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a90b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe56ffd640 1 -- 172.21.15.33:0/2237246195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 msgr2=0x55c45a906650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe56ffd640 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a906650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe56ffd640 1 -- 172.21.15.33:0/2237246195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 msgr2=0x55c45a90e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe56ffd640 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a90e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.859+0000 7efe56ffd640 1 -- 172.21.15.33:0/2237246195 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45a836400 con 0x55c45a906c20 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.860+0000 7efe57fff640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe400a2c20 0x7efe400a50e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe50009fd0 tx=0x7efe50009bb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.860+0000 7efe56ffd640 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a90b720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0ed210 tx=0x7efe4c0bb8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957a96ad708cbb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.860+0000 7efe3dffb640 1 -- 172.21.15.33:0/2237246195 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c049040 con 0x55c45a906c20 2024-04-24T23:13:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.860+0000 7efe3dffb640 1 -- 172.21.15.33:0/2237246195 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c0bbf60 con 0x55c45a906c20 2024-04-24T23:13:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.860+0000 7efe3dffb640 1 -- 172.21.15.33:0/2237246195 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c03bc10 con 0x55c45a906c20 2024-04-24T23:13:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 -- 172.21.15.33:0/2237246195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 msgr2=0x55c45a90b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a90b720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0ed210 tx=0x7efe4c0bb8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 -- 172.21.15.33:0/2237246195 shutdown_connections 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a90e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a90b720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 --2- 172.21.15.33:0/2237246195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a906650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 -- 172.21.15.33:0/2237246195 >> 172.21.15.33:0/2237246195 conn(0x55c45a9018a0 msgr2=0x55c45a901ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 -- 172.21.15.33:0/2237246195 shutdown_connections 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 -- 172.21.15.33:0/2237246195 wait complete. 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 -- start start 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.861+0000 7efe60fc2740 1 Processor -- start 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 0x55c45a921010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906c20 0x55c45a921550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a90bd90 0x55c45a928590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c45a91fb10 con 0x55c45a906230 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c45a826050 con 0x55c45a90bd90 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c45a91fc90 con 0x55c45a906c20 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe60fc2740 10 client.0 _ll_register_callbacks cb 0x55c45a57dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 0x55c45a921010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 0x55c45a921010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46024/0 (socket says 172.21.15.33:46024) 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 -- 172.21.15.33:0/910774940 learned_addr learned my addr 172.21.15.33:0/910774940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe57fff640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a90bd90 0x55c45a928590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe56ffd640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906c20 0x55c45a921550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 -- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906c20 msgr2=0x55c45a921550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906c20 0x55c45a921550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 -- 172.21.15.33:0/910774940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a90bd90 msgr2=0x55c45a928590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a90bd90 0x55c45a928590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.862+0000 7efe577fe640 1 -- 172.21.15.33:0/910774940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45a7f7dc0 con 0x55c45a906230 2024-04-24T23:13:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe577fe640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 0x55c45a921010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe48012700 tx=0x7efe48012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e78dcb8681df56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe36ffd640 1 -- 172.21.15.33:0/910774940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48004320 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe36ffd640 1 -- 172.21.15.33:0/910774940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe480035f0 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c0bbf60 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe36ffd640 1 -- 172.21.15.33:0/910774940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe48025050 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe60fc2740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c45a57e3f0 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.863+0000 7efe36ffd640 1 -- 172.21.15.33:0/910774940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c0bbf60 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe36ffd640 1 -- 172.21.15.33:0/910774940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efe48003830 con 0x55c45a906230 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe60fc2740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe60fc2740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe60fc2740 20 client.15810 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe60fc2740 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a92cf00 0x55c45a92f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe60fc2740 4 client.15810 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.864+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c45a877bb0 con 0x55c45a92cf00 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.865+0000 7efe56ffd640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a92cf00 0x55c45a92f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.865+0000 7efe56ffd640 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a92cf00 0x55c45a92f390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c55996bbe415e68 server_cookie=bcb1275432586aab in_seq=0 out_seq=0 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.865+0000 7efe36ffd640 10 client.15810 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe36ffd640 1 -- 172.21.15.33:0/910774940 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c45a877bb0 con 0x55c45a92cf00 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe36ffd640 10 client.15810 handle_command_reply: tid=0 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 client.15810 shutdown 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 msgr2=0x55c45a921010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 0x55c45a921010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe48012700 tx=0x7efe48012b00 comp rx=0 tx=0).stop 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 shutdown_connections 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a92cf00 0x55c45a92f390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a90bd90 0x55c45a928590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe36ffd640 0 client.15810 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906c20 0x55c45a921550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 --2- 172.21.15.33:0/910774940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906230 0x55c45a921010 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 >> 172.21.15.33:0/910774940 conn(0x55c45a9018a0 msgr2=0x55c45a900b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 shutdown_connections 2024-04-24T23:13:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 1 -- 172.21.15.33:0/910774940 wait complete. 2024-04-24T23:13:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.866+0000 7efe60fc2740 20 client.15810 trim_cache size 0 max 16384 2024-04-24T23:13:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 Processor -- start 2024-04-24T23:13:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 -- start start 2024-04-24T23:13:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a9002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a928550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a920630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a6700 con 0x55c45a90bd90 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a3fd0 con 0x55c45a906c20 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe60fc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe58073470 con 0x55c45a906230 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a920630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a920630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46040/0 (socket says 172.21.15.33:46040) 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe57fff640 1 -- 172.21.15.33:0/492402666 learned_addr learned my addr 172.21.15.33:0/492402666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe56ffd640 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a928550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.893+0000 7efe577fe640 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a9002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe57fff640 1 -- 172.21.15.33:0/492402666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 msgr2=0x55c45a9002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe57fff640 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a9002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe57fff640 1 -- 172.21.15.33:0/492402666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 msgr2=0x55c45a928550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe57fff640 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a928550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe57fff640 1 -- 172.21.15.33:0/492402666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45a6487d0 con 0x55c45a90bd90 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe56ffd640 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a928550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe57fff640 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a920630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efe50010d80 tx=0x7efe5001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7771c861c071a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe3d7fa640 1 -- 172.21.15.33:0/492402666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50010db0 con 0x55c45a90bd90 2024-04-24T23:13:46.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe3d7fa640 1 -- 172.21.15.33:0/492402666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe500045b0 con 0x55c45a90bd90 2024-04-24T23:13:46.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.894+0000 7efe3d7fa640 1 -- 172.21.15.33:0/492402666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50005650 con 0x55c45a90bd90 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 -- 172.21.15.33:0/492402666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 msgr2=0x55c45a920630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a920630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efe50010d80 tx=0x7efe5001e000 comp rx=0 tx=0).stop 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 -- 172.21.15.33:0/492402666 shutdown_connections 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a90bd90 0x55c45a920630 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906c20 0x55c45a928550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 --2- 172.21.15.33:0/492402666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a906230 0x55c45a9002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 -- 172.21.15.33:0/492402666 >> 172.21.15.33:0/492402666 conn(0x55c45a9018a0 msgr2=0x55c45a907f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 -- 172.21.15.33:0/492402666 shutdown_connections 2024-04-24T23:13:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.895+0000 7efe60fc2740 1 -- 172.21.15.33:0/492402666 wait complete. 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.896+0000 7efe60fc2740 1 -- start start 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.896+0000 7efe60fc2740 1 Processor -- start 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.896+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 0x55c45a925d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.896+0000 7efe60fc2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 0x55c45a9262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 0x55c45a9262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 0x55c45a9262b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46042/0 (socket says 172.21.15.33:46042) 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 0x55c45a925d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 0x55c45a925d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43588/0 (socket says 172.21.15.33:43588) 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 -- 172.21.15.33:0/579292615 learned_addr learned my addr 172.21.15.33:0/579292615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a90bd90 0x55c45a9272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a2490 con 0x55c45a906c20 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a3900 con 0x55c45a906230 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a2b60 con 0x55c45a90bd90 2024-04-24T23:13:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe60fc2740 10 client.0 _ll_register_callbacks cb 0x7efe480096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 -- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a90bd90 msgr2=0x55c45a9272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe57fff640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a90bd90 0x55c45a9272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a90bd90 0x55c45a9272c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 -- 172.21.15.33:0/579292615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 msgr2=0x55c45a925d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 0x55c45a925d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45a9278b0 con 0x55c45a906c20 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe577fe640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 0x55c45a925d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe56ffd640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 0x55c45a9262b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005940 tx=0x7efe4c0ba470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab054e5fa6a642b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.897+0000 7efe36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe36ffd640 1 -- 172.21.15.33:0/579292615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c049040 con 0x55c45a906c20 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe500045b0 con 0x55c45a906c20 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe36ffd640 1 -- 172.21.15.33:0/579292615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe4c03bc10 con 0x55c45a906c20 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe36ffd640 1 -- 172.21.15.33:0/579292615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe4c03b460 con 0x55c45a906c20 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 10 client.15816 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55c45a993520 con 0x55c45a906c20 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe36ffd640 1 -- 172.21.15.33:0/579292615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efe4c0bc1c0 con 0x55c45a906c20 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 10 client.15816 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 20 client.15816 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.898+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a993c00 0x55c45a996020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.899+0000 7efe60fc2740 4 client.15816 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.899+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efe500045b0 con 0x55c45a993c00 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.899+0000 7efe577fe640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a993c00 0x55c45a996020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.899+0000 7efe36ffd640 10 client.15816 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.899+0000 7efe577fe640 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a993c00 0x55c45a996020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd57d781aeafa795 server_cookie=1770f19b9248c325 in_seq=0 out_seq=0 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe36ffd640 1 -- 172.21.15.33:0/579292615 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efe500045b0 con 0x55c45a993c00 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe36ffd640 10 client.15816 handle_command_reply: tid=0 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 client.15816 shutdown 2024-04-24T23:13:46.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 msgr2=0x55c45a9262b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 0x55c45a9262b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005940 tx=0x7efe4c0ba470 comp rx=0 tx=0).stop 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 shutdown_connections 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c45a993c00 0x55c45a996020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45a90bd90 0x55c45a9272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe36ffd640 0 client.15816 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c45a906c20 0x55c45a9262b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 --2- 172.21.15.33:0/579292615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c45a906230 0x55c45a925d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 >> 172.21.15.33:0/579292615 conn(0x55c45a9018a0 msgr2=0x55c45a904620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 shutdown_connections 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 1 -- 172.21.15.33:0/579292615 wait complete. 2024-04-24T23:13:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.900+0000 7efe60fc2740 20 client.15816 trim_cache size 0 max 16384 2024-04-24T23:13:46.908 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:13:46.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe400a2c20 msgr2=0x7efe400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe400a2c20 0x7efe400a50e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe50009fd0 tx=0x7efe50009bb0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 msgr2=0x7efe5819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe5819a250 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009230 tx=0x7efe4c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 shutdown_connections 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe400a2c20 0x7efe400a50e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe58101350 0x7efe581a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe581009b0 0x7efe5819a250 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 --2- 172.21.15.33:0/742401151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe58100050 0x7efe58199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 >> 172.21.15.33:0/742401151 conn(0x7efe580fbbf0 msgr2=0x7efe580fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.901+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 shutdown_connections 2024-04-24T23:13:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:46.902+0000 7efe5e1d5640 1 -- 172.21.15.33:0/742401151 wait complete. 2024-04-24T23:13:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:46 smithi033 ceph-mon[29201]: pgmap v404: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:46 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2613852231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:13:47.228 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:46 smithi117 ceph-mon[32529]: pgmap v404: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:47.228 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:46 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2613852231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:13:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:46 smithi038 ceph-mon[32450]: pgmap v404: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:13:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:46 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2613852231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:13:47.918 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:13:47.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:13:47.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.031+0000 7fd27cbee640 1 -- 172.21.15.33:0/3234700843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278101890 msgr2=0x7fd278101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.031+0000 7fd27cbee640 1 --2- 172.21.15.33:0/3234700843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278101890 0x7fd278101d10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009900 tx=0x7fd26c02a410 comp rx=0 tx=0).stop 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 -- 172.21.15.33:0/3234700843 shutdown_connections 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 --2- 172.21.15.33:0/3234700843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd278102250 0x7fd278108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 --2- 172.21.15.33:0/3234700843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278101890 0x7fd278101d10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 --2- 172.21.15.33:0/3234700843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2781006a0 0x7fd278100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 -- 172.21.15.33:0/3234700843 >> 172.21.15.33:0/3234700843 conn(0x7fd2780fbc30 msgr2=0x7fd2780fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 -- 172.21.15.33:0/3234700843 shutdown_connections 2024-04-24T23:13:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.032+0000 7fd27cbee640 1 -- 172.21.15.33:0/3234700843 wait complete. 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 Processor -- start 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 -- start start 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2781006a0 0x7fd2781969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd278101890 0x7fd278196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 0x7fd27819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810c300 con 0x7fd278102250 2024-04-24T23:13:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810c180 con 0x7fd278101890 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd276d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 0x7fd27819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd276d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 0x7fd27819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46060/0 (socket says 172.21.15.33:46060) 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd276d9d640 1 -- 172.21.15.33:0/2889397939 learned_addr learned my addr 172.21.15.33:0/2889397939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810c480 con 0x7fd2781006a0 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd278101890 0x7fd278196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.033+0000 7fd27659c640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2781006a0 0x7fd2781969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd275d9b640 1 -- 172.21.15.33:0/2889397939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2781006a0 msgr2=0x7fd2781969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2781006a0 0x7fd2781969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd275d9b640 1 -- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 msgr2=0x7fd27819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 0x7fd27819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd275d9b640 1 -- 172.21.15.33:0/2889397939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278069bd0 con 0x7fd278101890 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd276d9d640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 0x7fd27819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd278101890 0x7fd278196ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd26c031440 tx=0x7fd26c031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd2677fe640 1 -- 172.21.15.33:0/2889397939 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c03f070 con 0x7fd278101890 2024-04-24T23:13:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd2677fe640 1 -- 172.21.15.33:0/2889397939 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c003cd0 con 0x7fd278101890 2024-04-24T23:13:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd278069e60 con 0x7fd278101890 2024-04-24T23:13:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd2677fe640 1 -- 172.21.15.33:0/2889397939 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c035070 con 0x7fd278101890 2024-04-24T23:13:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.034+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27806a300 con 0x7fd278101890 2024-04-24T23:13:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.037+0000 7fd2677fe640 1 -- 172.21.15.33:0/2889397939 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd26c043420 con 0x7fd278101890 2024-04-24T23:13:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.037+0000 7fd27f9dd740 1 Processor -- start 2024-04-24T23:13:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 -- start start 2024-04-24T23:13:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de3c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de41280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6de43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd9f2c0 con 0x563d6de3bd90 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd47910 con 0x563d6de418f0 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27f9dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd9b280 con 0x563d6de3c780 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de3c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de3c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46076/0 (socket says 172.21.15.33:46076) 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 -- 172.21.15.33:0/1376333447 learned_addr learned my addr 172.21.15.33:0/1376333447 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd27659c640 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de41280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd275d9b640 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6de43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 -- 172.21.15.33:0/1376333447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 msgr2=0x563d6de41280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de41280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 -- 172.21.15.33:0/1376333447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 msgr2=0x563d6de43d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.038+0000 7fd276d9d640 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6de43d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd276d9d640 1 -- 172.21.15.33:0/1376333447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6dd58fb0 con 0x563d6de3bd90 2024-04-24T23:13:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd27659c640 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de41280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd276d9d640 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de3c1b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd2700117e0 tx=0x7fd270011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c766d7bac422286f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd2677fe640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2600a2c20 0x7fd2600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd27659c640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2600a2c20 0x7fd2600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd264ff9640 1 -- 172.21.15.33:0/1376333447 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27000fb40 con 0x563d6de3bd90 2024-04-24T23:13:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd2677fe640 1 -- 172.21.15.33:0/2889397939 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd26c0ef9e0 con 0x7fd278101890 2024-04-24T23:13:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.039+0000 7fd264ff9640 1 -- 172.21.15.33:0/1376333447 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd27000fce0 con 0x563d6de3bd90 2024-04-24T23:13:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1376333447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 msgr2=0x563d6de3c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27659c640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2600a2c20 0x7fd2600a50e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x563d6de3cb80 tx=0x7fd268005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de3c1b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd2700117e0 tx=0x7fd270011ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1376333447 shutdown_connections 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6de43d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de41280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1376333447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de3c1b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.040+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1376333447 >> 172.21.15.33:0/1376333447 conn(0x563d6de03950 msgr2=0x563d6de37810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1376333447 shutdown_connections 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1376333447 wait complete. 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 -- start start 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 Processor -- start 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de46260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3c780 0x563d6de467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de418f0 0x563d6de4d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d6de448f0 con 0x563d6de3bd90 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd5ae90 con 0x563d6de3c780 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27f9dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d6de44a70 con 0x563d6de418f0 2024-04-24T23:13:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de46260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.041+0000 7fd27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de46260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46092/0 (socket says 172.21.15.33:46092) 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 -- 172.21.15.33:0/2844663495 learned_addr learned my addr 172.21.15.33:0/2844663495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27f9dd740 10 client.0 _ll_register_callbacks cb 0x563d6dd96970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3c780 0x563d6de467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 -- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de418f0 msgr2=0x563d6de4d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de418f0 0x563d6de4d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 -- 172.21.15.33:0/2844663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3c780 msgr2=0x563d6de467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3c780 0x563d6de467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 -- 172.21.15.33:0/2844663495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6db7b7e0 con 0x563d6de3bd90 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3c780 0x563d6de467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd27659c640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de46260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd26801e9d0 tx=0x7fd268004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e39c8044e1630f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd25dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.042+0000 7fd25dffb640 1 -- 172.21.15.33:0/2844663495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268009db0 con 0x563d6de3bd90 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd25dffb640 1 -- 172.21.15.33:0/2844663495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd268018030 con 0x563d6de3bd90 2024-04-24T23:13:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd27000fce0 con 0x563d6de3bd90 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd25dffb640 1 -- 172.21.15.33:0/2844663495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd268004030 con 0x563d6de3bd90 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd27f9dd740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d6dbdbf70 con 0x563d6de3bd90 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd25dffb640 1 -- 172.21.15.33:0/2844663495 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd27000fce0 con 0x563d6de3bd90 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.043+0000 7fd25dffb640 1 -- 172.21.15.33:0/2844663495 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd26802a430 con 0x563d6de3bd90 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd27f9dd740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd27f9dd740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd27f9dd740 20 client.15828 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6de51eb0 0x563d6de54370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd27f9dd740 4 client.15828 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d6dd735a0 con 0x563d6de51eb0 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.044+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6de51eb0 0x563d6de54370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.045+0000 7fd25dffb640 10 client.15828 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.045+0000 7fd275d9b640 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6de51eb0 0x563d6de54370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d250cbb023a10a9 server_cookie=894bf58d8e9ebbc7 in_seq=0 out_seq=0 2024-04-24T23:13:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.045+0000 7fd25dffb640 1 -- 172.21.15.33:0/2844663495 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d6dd735a0 con 0x563d6de51eb0 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.045+0000 7fd25dffb640 10 client.15828 handle_command_reply: tid=0 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.045+0000 7fd27f9dd740 1 client.15828 shutdown 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 msgr2=0x563d6de46260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de46260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd26801e9d0 tx=0x7fd268004730 comp rx=0 tx=0).stop 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 shutdown_connections 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6de51eb0 0x563d6de54370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de418f0 0x563d6de4d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd25dffb640 0 client.15828 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3c780 0x563d6de467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/2844663495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6de46260 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 >> 172.21.15.33:0/2844663495 conn(0x563d6de03950 msgr2=0x563d6de36950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 shutdown_connections 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 1 -- 172.21.15.33:0/2844663495 wait complete. 2024-04-24T23:13:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.046+0000 7fd27f9dd740 20 client.15828 trim_cache size 0 max 16384 2024-04-24T23:13:48.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 Processor -- start 2024-04-24T23:13:48.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 -- start start 2024-04-24T23:13:48.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3bd90 0x563d6de412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de3ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 0x563d6de45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd276d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 0x563d6de45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd276d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 0x563d6de45e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46096/0 (socket says 172.21.15.33:46096) 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd276d9d640 1 -- 172.21.15.33:0/3502721323 learned_addr learned my addr 172.21.15.33:0/3502721323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781acc60 con 0x563d6de418f0 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd275d9b640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de3ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781aa530 con 0x563d6de3bd90 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781a8380 con 0x563d6de3c780 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.082+0000 7fd27659c640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3bd90 0x563d6de412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd275d9b640 1 -- 172.21.15.33:0/3502721323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3bd90 msgr2=0x563d6de412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd275d9b640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3bd90 0x563d6de412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd275d9b640 1 -- 172.21.15.33:0/3502721323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 msgr2=0x563d6de45e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd275d9b640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 0x563d6de45e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd275d9b640 1 -- 172.21.15.33:0/3502721323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6da73170 con 0x563d6de3c780 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd27659c640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3bd90 0x563d6de412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd276d9d640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 0x563d6de45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd275d9b640 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de3ae60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0ed000 tx=0x7fd26c03b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8149089bbeb2fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd25ffff640 1 -- 172.21.15.33:0/3502721323 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c048040 con 0x563d6de3c780 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd25ffff640 1 -- 172.21.15.33:0/3502721323 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c0efe50 con 0x563d6de3c780 2024-04-24T23:13:48.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.083+0000 7fd25ffff640 1 -- 172.21.15.33:0/3502721323 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0ed250 con 0x563d6de3c780 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 msgr2=0x563d6de3ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de3ae60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0ed000 tx=0x7fd26c03b900 comp rx=0 tx=0).stop 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 shutdown_connections 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de418f0 0x563d6de45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6de3ae60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/3502721323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de3bd90 0x563d6de412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 >> 172.21.15.33:0/3502721323 conn(0x563d6de03950 msgr2=0x563d6de3a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 shutdown_connections 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.084+0000 7fd27f9dd740 1 -- 172.21.15.33:0/3502721323 wait complete. 2024-04-24T23:13:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 -- start start 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 Processor -- start 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6deb2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6deb4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6deb7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6deb2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6deb2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46104/0 (socket says 172.21.15.33:46104) 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27659c640 1 -- 172.21.15.33:0/1053873780 learned_addr learned my addr 172.21.15.33:0/1053873780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd275d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6deb4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781a89f0 con 0x563d6de3bd90 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781a9e60 con 0x563d6de418f0 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.085+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2781a90c0 con 0x563d6de3c780 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27659c640 1 -- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 msgr2=0x563d6deb4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27659c640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6deb4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27659c640 1 -- 172.21.15.33:0/1053873780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 msgr2=0x563d6deb7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd276d9d640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6deb7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27f9dd740 10 client.0 _ll_register_callbacks cb 0x7fd268009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27659c640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6deb7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27659c640 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6debbd50 con 0x563d6de3bd90 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd275d9b640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6deb4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd276d9d640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6deb7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd26c0efe50 con 0x563d6de3bd90 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd27659c640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6deb2550 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd268013010 tx=0x7fd26802ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7526a203a2145fec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd25dffb640 10 client.15834 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd25dffb640 1 -- 172.21.15.33:0/1053873780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26803e070 con 0x563d6de3bd90 2024-04-24T23:13:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.086+0000 7fd25dffb640 1 -- 172.21.15.33:0/1053873780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd2680041d0 con 0x563d6de3bd90 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd25dffb640 1 -- 172.21.15.33:0/1053873780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd268004410 con 0x563d6de3bd90 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 10 client.15834 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=25}) -- 0x563d6debd040 con 0x563d6de3bd90 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd25dffb640 1 -- 172.21.15.33:0/1053873780 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd268005040 con 0x563d6de3bd90 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 20 client.15834 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6debd890 0x563d6debfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 4 client.15834 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d6ddac800 con 0x563d6debd890 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.087+0000 7fd275d9b640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6debd890 0x563d6debfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.088+0000 7fd25dffb640 10 client.15834 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.088+0000 7fd275d9b640 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6debd890 0x563d6debfcb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cc3621b6be890f server_cookie=bcc59010a9d726bd in_seq=0 out_seq=0 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd25dffb640 1 -- 172.21.15.33:0/1053873780 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d6ddac800 con 0x563d6debd890 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd25dffb640 10 client.15834 handle_command_reply: tid=0 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd27f9dd740 1 client.15834 shutdown 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 msgr2=0x563d6deb2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6deb2550 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd268013010 tx=0x7fd26802ae10 comp rx=0 tx=0).stop 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 shutdown_connections 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.089+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563d6debd890 0x563d6debfcb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d6de418f0 0x563d6deb7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd25dffb640 0 client.15834 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6de3c780 0x563d6deb4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 1 --2- 172.21.15.33:0/1053873780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d6de3bd90 0x563d6deb2550 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 >> 172.21.15.33:0/1053873780 conn(0x563d6de03950 msgr2=0x563d6de45de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 shutdown_connections 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 1 -- 172.21.15.33:0/1053873780 wait complete. 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.090+0000 7fd27f9dd740 20 client.15834 trim_cache size 0 max 16384 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3a67ab05-a487-4d72-8a1e-167bf68a9864", 2024-04-24T23:13:48.097 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:13:48.098 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:13:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2600a2c20 msgr2=0x7fd2600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2600a2c20 0x7fd2600a50e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x563d6de3cb80 tx=0x7fd268005f90 comp rx=0 tx=0).stop 2024-04-24T23:13:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd278101890 msgr2=0x7fd278196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd278101890 0x7fd278196ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd26c031440 tx=0x7fd26c031b60 comp rx=0 tx=0).stop 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 shutdown_connections 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2600a2c20 0x7fd2600a50e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278102250 0x7fd27819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd278101890 0x7fd278196ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 --2- 172.21.15.33:0/2889397939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2781006a0 0x7fd2781969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 >> 172.21.15.33:0/2889397939 conn(0x7fd2780fbc30 msgr2=0x7fd278104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 shutdown_connections 2024-04-24T23:13:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.091+0000 7fd27cbee640 1 -- 172.21.15.33:0/2889397939 wait complete. 2024-04-24T23:13:48.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:13:48.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:13:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 -- 172.21.15.33:0/1190651941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88103340 msgr2=0x7f9e88109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 --2- 172.21.15.33:0/1190651941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88103340 0x7f9e88109bd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ca0 tx=0x7f9e8002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 -- 172.21.15.33:0/1190651941 shutdown_connections 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 --2- 172.21.15.33:0/1190651941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88103340 0x7f9e88109bd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 --2- 172.21.15.33:0/1190651941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88102980 0x7f9e88102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 --2- 172.21.15.33:0/1190651941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e88101790 0x7f9e88101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 -- 172.21.15.33:0/1190651941 >> 172.21.15.33:0/1190651941 conn(0x7f9e880fcd20 msgr2=0x7f9e880ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.213+0000 7f9e901c1640 1 -- 172.21.15.33:0/1190651941 shutdown_connections 2024-04-24T23:13:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.214+0000 7f9e901c1640 1 -- 172.21.15.33:0/1190651941 wait complete. 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.214+0000 7f9e901c1640 1 Processor -- start 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.214+0000 7f9e901c1640 1 -- start start 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.214+0000 7f9e901c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88101790 0x7f9e88197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.214+0000 7f9e901c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 0x7f9e88197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e901c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e88103340 0x7f9e8819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8d75c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 0x7f9e88197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e901c1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810d410 con 0x7f9e88102980 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8d75c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 0x7f9e88197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46126/0 (socket says 172.21.15.33:46126) 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8d75c640 1 -- 172.21.15.33:0/4225850019 learned_addr learned my addr 172.21.15.33:0/4225850019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810d290 con 0x7f9e88103340 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810d590 con 0x7f9e88101790 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88101790 0x7f9e88197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e88103340 0x7f9e8819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 -- 172.21.15.33:0/4225850019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e88103340 msgr2=0x7f9e8819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e88103340 0x7f9e8819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 -- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 msgr2=0x7f9e88197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 0x7f9e88197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 -- 172.21.15.33:0/4225850019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819f6a0 con 0x7f9e88101790 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 0x7f9e88197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.215+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88101790 0x7f9e88197a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9c0 tx=0x7f9e8400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.216+0000 7f9e7effd640 1 -- 172.21.15.33:0/4225850019 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400cd80 con 0x7f9e88101790 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.216+0000 7f9e7effd640 1 -- 172.21.15.33:0/4225850019 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e84010910 con 0x7f9e88101790 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.216+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819f990 con 0x7f9e88101790 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.216+0000 7f9e7effd640 1 -- 172.21.15.33:0/4225850019 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84018980 con 0x7f9e88101790 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.216+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8819fe90 con 0x7f9e88101790 2024-04-24T23:13:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e92faa740 1 Processor -- start 2024-04-24T23:13:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e7effd640 1 -- 172.21.15.33:0/4225850019 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9e8401f020 con 0x7f9e88101790 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e92faa740 1 -- start start 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad8d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad9db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.219+0000 7f9e92faa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e51acf16b0 con 0x55e51ad8d1c0 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e92faa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e51ac99880 con 0x55e51ad90d10 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e92faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e51aced670 con 0x55e51ad8dbb0 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad8d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8df5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8d75c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad9db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad8d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46142/0 (socket says 172.21.15.33:46142) 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 -- 172.21.15.33:0/3109075985 learned_addr learned my addr 172.21.15.33:0/3109075985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 -- 172.21.15.33:0/3109075985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 msgr2=0x55e51ad906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad906a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 -- 172.21.15.33:0/3109075985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 msgr2=0x55e51ad9db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad9db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 -- 172.21.15.33:0/3109075985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e51aacf7d0 con 0x55e51ad8d1c0 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.220+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad8d5e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9e80035000 tx=0x7f9e80009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3045441165de289 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.221+0000 7f9e7effd640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e740a2c20 0x7f9e740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.221+0000 7f9e73fff640 1 -- 172.21.15.33:0/3109075985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80031cb0 con 0x55e51ad8d1c0 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.221+0000 7f9e73fff640 1 -- 172.21.15.33:0/3109075985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80036050 con 0x55e51ad8d1c0 2024-04-24T23:13:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.221+0000 7f9e73fff640 1 -- 172.21.15.33:0/3109075985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800043a0 con 0x55e51ad8d1c0 2024-04-24T23:13:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.221+0000 7f9e7effd640 1 -- 172.21.15.33:0/4225850019 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e840cf640 con 0x7f9e88101790 2024-04-24T23:13:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.221+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e740a2c20 0x7f9e740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e8df5d640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e740a2c20 0x7f9e740a50e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55e51ad8dfb0 tx=0x7f9e8400efb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 -- 172.21.15.33:0/3109075985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 msgr2=0x55e51ad8d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad8d5e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9e80035000 tx=0x7f9e80009360 comp rx=0 tx=0).stop 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 -- 172.21.15.33:0/3109075985 shutdown_connections 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad9db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 --2- 172.21.15.33:0/3109075985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad8d5e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 -- 172.21.15.33:0/3109075985 >> 172.21.15.33:0/3109075985 conn(0x55e51ad888b0 msgr2=0x55e51ad88cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 -- 172.21.15.33:0/3109075985 shutdown_connections 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 -- 172.21.15.33:0/3109075985 wait complete. 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.222+0000 7f9e92faa740 1 -- start start 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 Processor -- start 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad8d1c0 0x55e51ad97400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 0x55e51ad9c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e51ad91570 con 0x55e51ad90d10 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e51acacf50 con 0x55e51ad8d1c0 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e51ad916f0 con 0x55e51ad8dbb0 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e92faa740 10 client.0 _ll_register_callbacks cb 0x55e51ace8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 0x55e51ad9c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e8d75c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 0x55e51ad9c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46152/0 (socket says 172.21.15.33:46152) 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.223+0000 7f9e8e75e640 1 -- 172.21.15.33:0/756652992 learned_addr learned my addr 172.21.15.33:0/756652992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8d75c640 1 -- 172.21.15.33:0/756652992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad8d1c0 msgr2=0x55e51ad97400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad8d1c0 0x55e51ad97400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8d75c640 1 -- 172.21.15.33:0/756652992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 msgr2=0x55e51ad9c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 0x55e51ad9c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8d75c640 1 -- 172.21.15.33:0/756652992 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e51aa053f0 con 0x55e51ad8dbb0 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 0x55e51ad9c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad97940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009640 tx=0x7f9e7800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd94d93678556a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e717fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e717fa640 1 -- 172.21.15.33:0/756652992 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800fe50 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e717fa640 1 -- 172.21.15.33:0/756652992 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e78005020 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.224+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e80036050 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.225+0000 7f9e717fa640 1 -- 172.21.15.33:0/756652992 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e78005220 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.225+0000 7f9e92faa740 10 client.25417 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.225+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e51a9c50f0 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.225+0000 7f9e717fa640 1 -- 172.21.15.33:0/756652992 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80036050 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e717fa640 1 -- 172.21.15.33:0/756652992 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e7801c470 con 0x55e51ad8dbb0 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e92faa740 10 client.25417 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e92faa740 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e92faa740 20 client.25417 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e92faa740 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51adb43e0 0x55e51adb6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e92faa740 4 client.25417 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e51acfebf0 con 0x55e51adb43e0 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.226+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51adb43e0 0x55e51adb6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e717fa640 10 client.25417 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51adb43e0 0x55e51adb6860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef53b5ef044fabb6 server_cookie=6deb4b382d831f2e in_seq=0 out_seq=0 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e717fa640 1 -- 172.21.15.33:0/756652992 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e51acfebf0 con 0x55e51adb43e0 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e717fa640 10 client.25417 handle_command_reply: tid=0 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e92faa740 1 client.25417 shutdown 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 msgr2=0x55e51ad97940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e92faa740 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad97940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009640 tx=0x7f9e7800f790 comp rx=0 tx=0).stop 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.227+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 shutdown_connections 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51adb43e0 0x55e51adb6860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad90d10 0x55e51ad9c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad97940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e717fa640 0 client.25417 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 --2- 172.21.15.33:0/756652992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad8d1c0 0x55e51ad97400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 >> 172.21.15.33:0/756652992 conn(0x55e51ad888b0 msgr2=0x55e51ad87af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 shutdown_connections 2024-04-24T23:13:48.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 1 -- 172.21.15.33:0/756652992 wait complete. 2024-04-24T23:13:48.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.228+0000 7f9e92faa740 20 client.25417 trim_cache size 0 max 16384 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 Processor -- start 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 -- start start 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad87970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad96f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881aa990 con 0x55e51ad8d1c0 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881a8260 con 0x55e51ad90d10 2024-04-24T23:13:48.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e92faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881a6160 con 0x55e51ad8dbb0 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad87970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad87970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46154/0 (socket says 172.21.15.33:46154) 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.245+0000 7f9e8e75e640 1 -- 172.21.15.33:0/68653537 learned_addr learned my addr 172.21.15.33:0/68653537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad96f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 -- 172.21.15.33:0/68653537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 msgr2=0x55e51ad96a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad96a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 -- 172.21.15.33:0/68653537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 msgr2=0x55e51ad96f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad96f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 -- 172.21.15.33:0/68653537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e51ad89620 con 0x55e51ad8d1c0 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad87970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006010 tx=0x7f9e80000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87a7cc98145f92a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e737fe640 1 -- 172.21.15.33:0/68653537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80059070 con 0x55e51ad8d1c0 2024-04-24T23:13:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e737fe640 1 -- 172.21.15.33:0/68653537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80003970 con 0x55e51ad8d1c0 2024-04-24T23:13:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.246+0000 7f9e737fe640 1 -- 172.21.15.33:0/68653537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80053620 con 0x55e51ad8d1c0 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 -- 172.21.15.33:0/68653537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 msgr2=0x55e51ad87970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad87970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006010 tx=0x7f9e80000f40 comp rx=0 tx=0).stop 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 -- 172.21.15.33:0/68653537 shutdown_connections 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51ad96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51ad96a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 --2- 172.21.15.33:0/68653537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51ad87970 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 -- 172.21.15.33:0/68653537 >> 172.21.15.33:0/68653537 conn(0x55e51ad888b0 msgr2=0x55e51ad87260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 -- 172.21.15.33:0/68653537 shutdown_connections 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 -- 172.21.15.33:0/68653537 wait complete. 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.247+0000 7f9e92faa740 1 -- start start 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 Processor -- start 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51adc5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51adc58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51adc6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881a6720 con 0x55e51ad8d1c0 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881a7b90 con 0x55e51ad90d10 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881a6df0 con 0x55e51ad8dbb0 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e92faa740 10 client.0 _ll_register_callbacks cb 0x7f9e78012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51adc5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8d75c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51adc58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8e75e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51adc5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46170/0 (socket says 172.21.15.33:46170) 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8e75e640 1 -- 172.21.15.33:0/786492540 learned_addr learned my addr 172.21.15.33:0/786492540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8d75c640 1 -- 172.21.15.33:0/786492540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 msgr2=0x55e51adc6840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51adc6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.248+0000 7f9e8d75c640 1 -- 172.21.15.33:0/786492540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 msgr2=0x55e51adc5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51adc5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e8d75c640 1 -- 172.21.15.33:0/786492540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e51adc6e30 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51adc5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e8d75c640 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51adc58a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9e7801c960 tx=0x7f9e7801c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808b935c94c33db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e717fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e717fa640 1 -- 172.21.15.33:0/786492540 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800fe50 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e80003970 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e717fa640 1 -- 172.21.15.33:0/786492540 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e78009cf0 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e717fa640 1 -- 172.21.15.33:0/786492540 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e78004890 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e92faa740 10 client.25423 fetch_fsmap learned FSMap version 25 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e51ae1a480 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.249+0000 7f9e717fa640 1 -- 172.21.15.33:0/786492540 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80003970 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e717fa640 1 -- 172.21.15.33:0/786492540 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e780326e0 con 0x55e51ad8dbb0 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e92faa740 10 client.25423 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e92faa740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e92faa740 20 client.25423 populate_metadata read hostname 'smithi033' 2024-04-24T23:13:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e92faa740 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51ae1add0 0x55e51ae1d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e92faa740 4 client.25423 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e51acf8f40 con 0x55e51ae1add0 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51ae1add0 0x55e51ae1d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e717fa640 10 client.25423 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.250+0000 7f9e8e75e640 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51ae1add0 0x55e51ae1d1f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2e2be44fd06633 server_cookie=8cddc5d2b4584262 in_seq=0 out_seq=0 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e717fa640 1 -- 172.21.15.33:0/786492540 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e51acf8f40 con 0x55e51ae1add0 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e717fa640 10 client.25423 handle_command_reply: tid=0 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 client.25423 shutdown 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 msgr2=0x55e51adc58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51adc58a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9e7801c960 tx=0x7f9e7801c720 comp rx=0 tx=0).stop 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 shutdown_connections 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e51ae1add0 0x55e51ae1d1f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e717fa640 0 client.25423 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:13:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e51ad90d10 0x55e51adc6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e51ad8dbb0 0x55e51adc58a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 --2- 172.21.15.33:0/786492540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51ad8d1c0 0x55e51adc5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 >> 172.21.15.33:0/786492540 conn(0x55e51ad888b0 msgr2=0x55e51ad87af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 shutdown_connections 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 1 -- 172.21.15.33:0/786492540 wait complete. 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e92faa740 20 client.25423 trim_cache size 0 max 16384 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:13:48.260 INFO:teuthology.orchestra.run.smithi033.stdout: "3a67ab05-a487-4d72-8a1e-167bf68a9864": { 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "3a67ab05-a487-4d72-8a1e-167bf68a9864", 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:13:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e740a2c20 msgr2=0x7f9e740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e901c1640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e740a2c20 0x7f9e740a50e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55e51ad8dfb0 tx=0x7f9e8400efb0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88101790 msgr2=0x7f9e88197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.251+0000 7f9e901c1640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88101790 0x7f9e88197a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9c0 tx=0x7f9e8400ee80 comp rx=0 tx=0).stop 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 shutdown_connections 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9e740a2c20 0x7f9e740a50e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e88103340 0x7f9e8819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e88102980 0x7f9e88197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 --2- 172.21.15.33:0/4225850019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e88101790 0x7f9e88197a80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 >> 172.21.15.33:0/4225850019 conn(0x7f9e880fcd20 msgr2=0x7f9e88105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 shutdown_connections 2024-04-24T23:13:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:13:48.252+0000 7f9e901c1640 1 -- 172.21.15.33:0/4225850019 wait complete. 2024-04-24T23:13:48.265 INFO:tasks.cephfs.filesystem:scrub status for tag:3a67ab05-a487-4d72-8a1e-167bf68a9864 - {'path': '/', 'tag': '3a67ab05-a487-4d72-8a1e-167bf68a9864', 'options': 'recursive,force'} 2024-04-24T23:13:49.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:48 smithi033 ceph-mon[29201]: pgmap v405: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:49.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:48 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:13:49.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:48 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:49.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:48 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:13:49.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:48 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:49.223 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:48 smithi038 ceph-mon[32450]: pgmap v405: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:49.223 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:48 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:13:49.223 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:48 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:49.223 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:48 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:13:49.223 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:48 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:48 smithi117 ceph-mon[32529]: pgmap v405: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T23:13:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:48 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:13:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:48 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:49.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:48 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:13:49.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:48 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:13:51.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:50 smithi033 ceph-mon[29201]: pgmap v406: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:50 smithi038 ceph-mon[32450]: pgmap v406: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:51.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:50 smithi117 ceph-mon[32529]: pgmap v406: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:53.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:52 smithi033 ceph-mon[29201]: pgmap v407: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T23:13:53.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:13:53.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:52 smithi038 ceph-mon[32450]: pgmap v407: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T23:13:53.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:13:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:52 smithi117 ceph-mon[32529]: pgmap v407: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T23:13:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:13:54.131 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 94% (11822/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T23:13:54.411 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:13:54.411 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-04-24T23:13:54.412 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.413 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-04-24T23:13:54.413 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.413 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:13:54.413 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-24T23:13:54.413 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:13:54.416 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T23:13:54.443 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:13:54.445 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:13:54.445 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:13:55.045 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:13:55.102 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:13:55.143 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:13:55.145 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:13:55.145 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:13:55.186 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:13:55.190 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:13:55.190 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T23:13:55.247 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-24T23:13:55.302 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-24T23:13:55.302 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-24T23:13:55.324 INFO:tasks.workunit.client.0.smithi033.stderr:+ dbench 1 2024-04-24T23:13:55.333 INFO:tasks.workunit.client.0.smithi033.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-24T23:13:55.333 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-04-24T23:13:55.333 INFO:tasks.workunit.client.0.smithi033.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-24T23:13:55.333 INFO:tasks.workunit.client.0.smithi033.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-24T23:13:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:54 smithi038 ceph-mon[32450]: pgmap v408: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:13:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:54 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:54 smithi117 ceph-mon[32529]: pgmap v408: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:13:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:54 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:55.434 INFO:tasks.workunit.client.0.smithi033.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-24T23:13:55.434 INFO:tasks.workunit.client.0.smithi033.stdout:releasing clients 2024-04-24T23:13:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:54 smithi033 ceph-mon[29201]: pgmap v408: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:13:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:54 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:56.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 3573 38.43 MB/sec warmup 1 sec latency 7.386 ms 2024-04-24T23:13:56.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:56.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:56.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:56.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:13:57.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 6724 26.62 MB/sec warmup 2 sec latency 14.058 ms 2024-04-24T23:13:57.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:57 smithi038 ceph-mon[32450]: pgmap v409: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:13:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:57 smithi117 ceph-mon[32529]: pgmap v409: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:13:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:57 smithi033 ceph-mon[29201]: pgmap v409: 129 pgs: 129 active+clean; 661 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:13:58.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 9018 21.56 MB/sec warmup 3 sec latency 30.697 ms 2024-04-24T23:13:58.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:58 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:13:58.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:58 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:13:58.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:58 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:13:59.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 11532 19.75 MB/sec warmup 4 sec latency 25.051 ms 2024-04-24T23:13:59.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:13:59 smithi038 ceph-mon[32450]: pgmap v410: 129 pgs: 129 active+clean; 2.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 141 KiB/s wr, 27 op/s 2024-04-24T23:13:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:13:59 smithi117 ceph-mon[32529]: pgmap v410: 129 pgs: 129 active+clean; 2.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 141 KiB/s wr, 27 op/s 2024-04-24T23:13:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:13:59 smithi033 ceph-mon[29201]: pgmap v410: 129 pgs: 129 active+clean; 2.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 141 KiB/s wr, 27 op/s 2024-04-24T23:14:00.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 14733 19.49 MB/sec warmup 5 sec latency 8.045 ms 2024-04-24T23:14:01.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 17684 19.14 MB/sec warmup 6 sec latency 23.835 ms 2024-04-24T23:14:01.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:01 smithi038 ceph-mon[32450]: pgmap v411: 129 pgs: 129 active+clean; 2.2 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 159 KiB/s wr, 27 op/s 2024-04-24T23:14:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:01 smithi117 ceph-mon[32529]: pgmap v411: 129 pgs: 129 active+clean; 2.2 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 159 KiB/s wr, 27 op/s 2024-04-24T23:14:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:01 smithi033 ceph-mon[29201]: pgmap v411: 129 pgs: 129 active+clean; 2.2 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 159 KiB/s wr, 27 op/s 2024-04-24T23:14:02.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20201 17.68 MB/sec warmup 7 sec latency 13.624 ms 2024-04-24T23:14:03.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 22298 17.07 MB/sec warmup 8 sec latency 13.357 ms 2024-04-24T23:14:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:03 smithi038 ceph-mon[32450]: pgmap v412: 129 pgs: 129 active+clean; 7.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 744 KiB/s wr, 94 op/s 2024-04-24T23:14:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:03 smithi117 ceph-mon[32529]: pgmap v412: 129 pgs: 129 active+clean; 7.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 744 KiB/s wr, 94 op/s 2024-04-24T23:14:03.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:03 smithi033 ceph-mon[29201]: pgmap v412: 129 pgs: 129 active+clean; 7.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 744 KiB/s wr, 94 op/s 2024-04-24T23:14:04.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 25382 17.21 MB/sec warmup 9 sec latency 16.760 ms 2024-04-24T23:14:05.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 28667 17.37 MB/sec warmup 10 sec latency 13.009 ms 2024-04-24T23:14:05.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:05 smithi038 ceph-mon[32450]: pgmap v413: 129 pgs: 129 active+clean; 8.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 786 KiB/s wr, 96 op/s 2024-04-24T23:14:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:05 smithi117 ceph-mon[32529]: pgmap v413: 129 pgs: 129 active+clean; 8.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 786 KiB/s wr, 96 op/s 2024-04-24T23:14:05.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:05 smithi033 ceph-mon[29201]: pgmap v413: 129 pgs: 129 active+clean; 8.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 786 KiB/s wr, 96 op/s 2024-04-24T23:14:06.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 31293 16.84 MB/sec warmup 11 sec latency 23.592 ms 2024-04-24T23:14:07.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 33668 16.53 MB/sec warmup 12 sec latency 13.456 ms 2024-04-24T23:14:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:07 smithi038 ceph-mon[32450]: pgmap v414: 129 pgs: 129 active+clean; 13 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-24T23:14:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:07 smithi038 ceph-mon[32450]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-24T23:14:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:07 smithi117 ceph-mon[32529]: pgmap v414: 129 pgs: 129 active+clean; 13 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-24T23:14:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:07 smithi117 ceph-mon[32529]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-24T23:14:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:07 smithi033 ceph-mon[29201]: pgmap v414: 129 pgs: 129 active+clean; 13 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-24T23:14:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:07 smithi033 ceph-mon[29201]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-24T23:14:08.434 INFO:tasks.workunit.client.0.smithi033.stdout: 1 36452 16.39 MB/sec warmup 13 sec latency 13.134 ms 2024-04-24T23:14:09.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 39494 16.52 MB/sec warmup 14 sec latency 21.592 ms 2024-04-24T23:14:09.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:09 smithi038 ceph-mon[32450]: pgmap v415: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-24T23:14:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:09 smithi117 ceph-mon[32529]: pgmap v415: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-24T23:14:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:09 smithi033 ceph-mon[29201]: pgmap v415: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-24T23:14:10.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 42672 16.60 MB/sec warmup 15 sec latency 14.574 ms 2024-04-24T23:14:11.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:10 smithi038 ceph-mon[32450]: pgmap v416: 129 pgs: 129 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-04-24T23:14:11.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:10 smithi117 ceph-mon[32529]: pgmap v416: 129 pgs: 129 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-04-24T23:14:11.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:10 smithi033 ceph-mon[29201]: pgmap v416: 129 pgs: 129 active+clean; 14 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-04-24T23:14:11.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44999 16.09 MB/sec warmup 16 sec latency 19.367 ms 2024-04-24T23:14:12.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 47379 15.98 MB/sec warmup 17 sec latency 15.666 ms 2024-04-24T23:14:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:12 smithi038 ceph-mon[32450]: pgmap v417: 129 pgs: 129 active+clean; 20 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.9 MiB/s wr, 240 op/s 2024-04-24T23:14:13.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:12 smithi117 ceph-mon[32529]: pgmap v417: 129 pgs: 129 active+clean; 20 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.9 MiB/s wr, 240 op/s 2024-04-24T23:14:13.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:12 smithi033 ceph-mon[29201]: pgmap v417: 129 pgs: 129 active+clean; 20 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.9 MiB/s wr, 240 op/s 2024-04-24T23:14:13.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 50410 16.05 MB/sec warmup 18 sec latency 14.046 ms 2024-04-24T23:14:14.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 53507 16.18 MB/sec warmup 19 sec latency 16.655 ms 2024-04-24T23:14:15.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:14 smithi038 ceph-mon[32450]: pgmap v418: 129 pgs: 129 active+clean; 21 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-24T23:14:15.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:14 smithi117 ceph-mon[32529]: pgmap v418: 129 pgs: 129 active+clean; 21 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-24T23:14:15.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:14 smithi033 ceph-mon[29201]: pgmap v418: 129 pgs: 129 active+clean; 21 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-24T23:14:15.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 55828 15.79 MB/sec warmup 20 sec latency 78.838 ms 2024-04-24T23:14:16.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 57445 15.57 MB/sec warmup 21 sec latency 165.901 ms 2024-04-24T23:14:17.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:16 smithi038 ceph-mon[32450]: pgmap v419: 129 pgs: 129 active+clean; 22 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-24T23:14:17.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:16 smithi117 ceph-mon[32529]: pgmap v419: 129 pgs: 129 active+clean; 22 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-24T23:14:17.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:16 smithi033 ceph-mon[29201]: pgmap v419: 129 pgs: 129 active+clean; 22 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-24T23:14:17.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59157 15.18 MB/sec warmup 22 sec latency 185.654 ms 2024-04-24T23:14:18.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:14:18.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 -- 172.21.15.33:0/258729484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad01006a0 msgr2=0x7f8ad0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 --2- 172.21.15.33:0/258729484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad01006a0 0x7f8ad0100ac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009900 tx=0x7f8ac002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 -- 172.21.15.33:0/258729484 shutdown_connections 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 --2- 172.21.15.33:0/258729484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad0102250 0x7f8ad0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 --2- 172.21.15.33:0/258729484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0101890 0x7f8ad0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 --2- 172.21.15.33:0/258729484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad01006a0 0x7f8ad0100ac0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 -- 172.21.15.33:0/258729484 >> 172.21.15.33:0/258729484 conn(0x7f8ad00fbc30 msgr2=0x7f8ad00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 -- 172.21.15.33:0/258729484 shutdown_connections 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 -- 172.21.15.33:0/258729484 wait complete. 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.380+0000 7f8ad5702640 1 Processor -- start 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 -- start start 2024-04-24T23:14:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad01006a0 0x7f8ad0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 0x7f8ad019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102250 0x7f8ad01a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010b220 con 0x7f8ad0101890 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010b0a0 con 0x7f8ad0102250 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ad5702640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010b3a0 con 0x7f8ad01006a0 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 0x7f8ad019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 0x7f8ad019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40916/0 (socket says 172.21.15.33:40916) 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 -- 172.21.15.33:0/732609945 learned_addr learned my addr 172.21.15.33:0/732609945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8aceffd640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad01006a0 0x7f8ad0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102250 0x7f8ad01a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 -- 172.21.15.33:0/732609945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad01006a0 msgr2=0x7f8ad0199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad01006a0 0x7f8ad0199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 -- 172.21.15.33:0/732609945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102250 msgr2=0x7f8ad01a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102250 0x7f8ad01a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 -- 172.21.15.33:0/732609945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad01a1960 con 0x7f8ad0101890 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.381+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 0x7f8ad019a280 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400e9f0 tx=0x7f8ac400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.382+0000 7f8ab7fff640 1 -- 172.21.15.33:0/732609945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400cd80 con 0x7f8ad0101890 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.382+0000 7f8ab7fff640 1 -- 172.21.15.33:0/732609945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4010910 con 0x7f8ad0101890 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.382+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad01a1c50 con 0x7f8ad0101890 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.382+0000 7f8ab7fff640 1 -- 172.21.15.33:0/732609945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4018990 con 0x7f8ad0101890 2024-04-24T23:14:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.382+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad01a21a0 con 0x7f8ad0101890 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.384+0000 7f8ab7fff640 1 -- 172.21.15.33:0/732609945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8ac4010ac0 con 0x7f8ad0101890 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.384+0000 7f8ab7fff640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ab80a2c10 0x7f8ab80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.384+0000 7f8ad8405740 1 Processor -- start 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 -- start start 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8aceffd640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ab80a2c10 0x7f8ab80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ab7fff640 1 -- 172.21.15.33:0/732609945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ac40cf740 con 0x7f8ad0101890 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0a240 0x5631bbe0a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0ac30 0x5631bbe0f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe12230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631bbd6e6b0 con 0x5631bbe0fda0 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5631bbd16880 con 0x5631bbe0a240 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ad8405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631bbd6a670 con 0x5631bbe0ac30 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe12230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe12230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40924/0 (socket says 172.21.15.33:40924) 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 -- 172.21.15.33:0/2085037603 learned_addr learned my addr 172.21.15.33:0/2085037603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0ac30 0x5631bbe0f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8aceffd640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0a240 0x5631bbe0a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8aceffd640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ab80a2c10 0x7f8ab80a50d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009230 tx=0x7f8ac00315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 -- 172.21.15.33:0/2085037603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0ac30 msgr2=0x5631bbe0f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0ac30 0x5631bbe0f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 -- 172.21.15.33:0/2085037603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0a240 msgr2=0x5631bbe0a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0a240 0x5631bbe0a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 -- 172.21.15.33:0/2085037603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631bbb4c7d0 con 0x5631bbe0fda0 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8aceffd640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0a240 0x5631bbe0a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0ac30 0x5631bbe0f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.385+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe12230 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009fc0 tx=0x7f8ac8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5568e95b5fbea528 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.386+0000 7f8ab57fa640 1 -- 172.21.15.33:0/2085037603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004d20 con 0x5631bbe0fda0 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.386+0000 7f8ab57fa640 1 -- 172.21.15.33:0/2085037603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8003e50 con 0x5631bbe0fda0 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.386+0000 7f8ab57fa640 1 -- 172.21.15.33:0/2085037603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80174a0 con 0x5631bbe0fda0 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 -- 172.21.15.33:0/2085037603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 msgr2=0x5631bbe12230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe12230 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009fc0 tx=0x7f8ac8009500 comp rx=0 tx=0).stop 2024-04-24T23:14:18.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 -- 172.21.15.33:0/2085037603 shutdown_connections 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe12230 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0ac30 0x5631bbe0f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 --2- 172.21.15.33:0/2085037603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0a240 0x5631bbe0a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 -- 172.21.15.33:0/2085037603 >> 172.21.15.33:0/2085037603 conn(0x5631bbe058b0 msgr2=0x5631bbe05cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 -- 172.21.15.33:0/2085037603 shutdown_connections 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 -- 172.21.15.33:0/2085037603 wait complete. 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 -- start start 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.387+0000 7f8ad8405740 1 Processor -- start 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0ac30 0x5631bbe14b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631bbe12d50 con 0x5631bbe0fda0 2024-04-24T23:14:18.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5631bbd29f50 con 0x5631bbe0ac30 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631bbe12ed0 con 0x5631bbe0a240 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ad8405740 10 client.0 _ll_register_callbacks cb 0x5631bbd65000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe1bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40926/0 (socket says 172.21.15.33:40926) 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe14630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57712/0 (socket says 172.21.15.33:57712) 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8acf7fe640 1 -- 172.21.15.33:0/3252015073 learned_addr learned my addr 172.21.15.33:0/3252015073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 -- 172.21.15.33:0/3252015073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0ac30 msgr2=0x5631bbe14b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0ac30 0x5631bbe14b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0ac30 0x5631bbe14b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 -- 172.21.15.33:0/3252015073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 msgr2=0x5631bbe1bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe1bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8aceffd640 1 -- 172.21.15.33:0/3252015073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631bba823f0 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.388+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0ac30 0x5631bbe14b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8aceffd640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe14630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0006010 tx=0x7f8ac0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791ef6bbdb374ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ab2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/3252015073 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0035040 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/3252015073 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ac00037f0 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8003e50 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/3252015073 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ac005a050 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ad8405740 10 client.25435 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631bba420f0 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.389+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/3252015073 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8003e50 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/3252015073 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f8ac003ca00 con 0x5631bbe0a240 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ad8405740 10 client.25435 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ad8405740 10 client.25435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ad8405740 20 client.25435 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ad8405740 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe22950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ad8405740 4 client.25435 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631bbd7bbf0 con 0x5631bbe204c0 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe22950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ab2ff5640 10 client.25435 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.390+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe22950 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597d4610f912564e server_cookie=e6dd6af789c150fd in_seq=0 out_seq=0 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/3252015073 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631bbd7bbf0 con 0x5631bbe204c0 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ab2ff5640 10 client.25435 handle_command_reply: tid=0 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 client.25435 shutdown 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 msgr2=0x5631bbe14630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe14630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0006010 tx=0x7f8ac0009260 comp rx=0 tx=0).stop 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 shutdown_connections 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe22950 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0fda0 0x5631bbe1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ab2ff5640 0 client.25435 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0ac30 0x5631bbe14b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 --2- 172.21.15.33:0/3252015073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe14630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 >> 172.21.15.33:0/3252015073 conn(0x5631bbe058b0 msgr2=0x5631bbe04af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 shutdown_connections 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 1 -- 172.21.15.33:0/3252015073 wait complete. 2024-04-24T23:14:18.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.391+0000 7f8ad8405740 20 client.25435 trim_cache size 0 max 16384 2024-04-24T23:14:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.412+0000 7f8ad8405740 1 Processor -- start 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 -- start start 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe05c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe14950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01aef40 con 0x5631bbe0ac30 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ac810 con 0x5631bbe0fda0 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ad8405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01aa660 con 0x5631bbe0a240 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ace7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe14950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ace7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe1bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40936/0 (socket says 172.21.15.33:40936) 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ace7fc640 1 -- 172.21.15.33:0/1038603769 learned_addr learned my addr 172.21.15.33:0/1038603769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8aceffd640 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe05c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8acf7fe640 1 -- 172.21.15.33:0/1038603769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 msgr2=0x5631bbe05c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe05c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8acf7fe640 1 -- 172.21.15.33:0/1038603769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 msgr2=0x5631bbe1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe1bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8acf7fe640 1 -- 172.21.15.33:0/1038603769 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631bbe05450 con 0x5631bbe0fda0 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.413+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.414+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe14950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8006010 tx=0x7f8ac8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44127f75a49344e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.414+0000 7f8ab4ff9640 1 -- 172.21.15.33:0/1038603769 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8003970 con 0x5631bbe0fda0 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.414+0000 7f8ab4ff9640 1 -- 172.21.15.33:0/1038603769 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8017480 con 0x5631bbe0fda0 2024-04-24T23:14:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.414+0000 7f8ab4ff9640 1 -- 172.21.15.33:0/1038603769 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80236b0 con 0x5631bbe0fda0 2024-04-24T23:14:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- 172.21.15.33:0/1038603769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 msgr2=0x5631bbe14950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe14950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8006010 tx=0x7f8ac8009bd0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- 172.21.15.33:0/1038603769 shutdown_connections 2024-04-24T23:14:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe14950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- 172.21.15.33:0/1038603769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe05c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- 172.21.15.33:0/1038603769 >> 172.21.15.33:0/1038603769 conn(0x5631bbe058b0 msgr2=0x5631bbe14240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- 172.21.15.33:0/1038603769 shutdown_connections 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- 172.21.15.33:0/1038603769 wait complete. 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- start start 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 Processor -- start 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe18d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe1a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01aacd0 con 0x5631bbe0ac30 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ac140 con 0x5631bbe0fda0 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ab3a0 con 0x5631bbe0a240 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.415+0000 7f8ad8405740 10 client.0 _ll_register_callbacks cb 0x7f8ac0040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe18d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8ace7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe18d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57728/0 (socket says 172.21.15.33:57728) 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 -- 172.21.15.33:0/4288546803 learned_addr learned my addr 172.21.15.33:0/4288546803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8acf7fe640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe1a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 -- 172.21.15.33:0/4288546803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 msgr2=0x5631bbe1a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe1a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 -- 172.21.15.33:0/4288546803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 msgr2=0x5631bbe192a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe192a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 -- 172.21.15.33:0/4288546803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631bbe1a830 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8aceffd640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe18d60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ac002a930 tx=0x7f8ac0048ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd078c1f20e5c0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8ab2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/4288546803 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0062070 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/4288546803 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ac003c640 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.416+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8017480 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/4288546803 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ac006e050 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 10 client.25441 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631bbe1bb40 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/4288546803 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8017480 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/4288546803 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f8ac005d070 con 0x5631bbe0a240 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 10 client.25441 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 10 client.25441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 20 client.25441 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe88ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 4 client.25441 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631bbd75f40 con 0x5631bbe204c0 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.417+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe88ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ab2ff5640 10 client.25441 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ace7fc640 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe88ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef0a44bf5a5c6ba server_cookie=bede04fe48be7e0a in_seq=0 out_seq=0 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ab2ff5640 1 -- 172.21.15.33:0/4288546803 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5631bbd75f40 con 0x5631bbe204c0 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ab2ff5640 10 client.25441 handle_command_reply: tid=0 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 client.25441 shutdown 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 msgr2=0x5631bbe18d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe18d60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ac002a930 tx=0x7f8ac0048ae0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 shutdown_connections 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5631bbe204c0 0x5631bbe88ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631bbe0fda0 0x5631bbe1a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631bbe0ac30 0x5631bbe192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 --2- 172.21.15.33:0/4288546803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631bbe0a240 0x5631bbe18d60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ab2ff5640 0 client.25441 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 >> 172.21.15.33:0/4288546803 conn(0x5631bbe058b0 msgr2=0x5631bbe04260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 shutdown_connections 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 1 -- 172.21.15.33:0/4288546803 wait complete. 2024-04-24T23:14:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.418+0000 7f8ad8405740 20 client.25441 trim_cache size 0 max 16384 2024-04-24T23:14:18.427 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:14:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:14:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:14:18.427 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ab80a2c10 msgr2=0x7f8ab80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ab80a2c10 0x7f8ab80a50d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009230 tx=0x7f8ac00315b0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 msgr2=0x7f8ad019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 0x7f8ad019a280 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400e9f0 tx=0x7f8ac400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 shutdown_connections 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ab80a2c10 0x7f8ab80a50d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102250 0x7f8ad01a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad0101890 0x7f8ad019a280 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 --2- 172.21.15.33:0/732609945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad01006a0 0x7f8ad0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 >> 172.21.15.33:0/732609945 conn(0x7f8ad00fbc30 msgr2=0x7f8ad0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 shutdown_connections 2024-04-24T23:14:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.419+0000 7f8ad5702640 1 -- 172.21.15.33:0/732609945 wait complete. 2024-04-24T23:14:18.432 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:14:18.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:18.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 61307 15.08 MB/sec warmup 23 sec latency 13.770 ms 2024-04-24T23:14:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 -- 172.21.15.33:0/756113054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c1034b0 msgr2=0x7f733c109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 --2- 172.21.15.33:0/756113054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c1034b0 0x7f733c109d40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7334009cd0 tx=0x7f733402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 -- 172.21.15.33:0/756113054 shutdown_connections 2024-04-24T23:14:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 --2- 172.21.15.33:0/756113054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c1034b0 0x7f733c109d40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 --2- 172.21.15.33:0/756113054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f733c102af0 0x7f733c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 --2- 172.21.15.33:0/756113054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0ffa80 0x7f733c0ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 -- 172.21.15.33:0/756113054 >> 172.21.15.33:0/756113054 conn(0x7f733c0fb3b0 msgr2=0x7f733c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 -- 172.21.15.33:0/756113054 shutdown_connections 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 -- 172.21.15.33:0/756113054 wait complete. 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.536+0000 7f7341f7e640 1 Processor -- start 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 -- start start 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 0x7f733c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102af0 0x7f733c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f733c1034b0 0x7f733c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f733c10d420 con 0x7f733c0ffa80 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f733c10d2a0 con 0x7f733c1034b0 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f7341f7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f733c10d5a0 con 0x7f733c102af0 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 0x7f733c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 0x7f733c197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40968/0 (socket says 172.21.15.33:40968) 2024-04-24T23:14:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 -- 172.21.15.33:0/4210509713 learned_addr learned my addr 172.21.15.33:0/4210509713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733bfff640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f733c1034b0 0x7f733c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733affd640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102af0 0x7f733c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 -- 172.21.15.33:0/4210509713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102af0 msgr2=0x7f733c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102af0 0x7f733c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 -- 172.21.15.33:0/4210509713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f733c1034b0 msgr2=0x7f733c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f733c1034b0 0x7f733c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 -- 172.21.15.33:0/4210509713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c19f680 con 0x7f733c0ffa80 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.537+0000 7f733b7fe640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 0x7f733c197a60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f732c00e9c0 tx=0x7f732c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.538+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c00cd80 con 0x7f733c0ffa80 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.538+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c0035f0 con 0x7f733c0ffa80 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.538+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c19f970 con 0x7f733c0ffa80 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.538+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c010690 con 0x7f733c0ffa80 2024-04-24T23:14:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.538+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c19fe70 con 0x7f733c0ffa80 2024-04-24T23:14:18.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.539+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c108bc0 con 0x7f733c0ffa80 2024-04-24T23:14:18.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.540+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f732c0037a0 con 0x7f733c0ffa80 2024-04-24T23:14:18.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.541+0000 7f7338ff9640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73240a2b90 0x7f73240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.541+0000 7f733affd640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73240a2b90 0x7f73240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.541+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f732c0cf780 con 0x7f733c0ffa80 2024-04-24T23:14:18.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.542+0000 7f733affd640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73240a2b90 0x7f73240a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f733c198f70 tx=0x7f7330005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.546+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f732c09c2f0 con 0x7f733c0ffa80 2024-04-24T23:14:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.800+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f733c02d170 con 0x7f733c0ffa80 2024-04-24T23:14:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.801+0000 7f7338ff9640 1 -- 172.21.15.33:0/4210509713 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11860 (secure 0 0 0) 0x7f732c09bbf0 con 0x7f733c0ffa80 2024-04-24T23:14:18.803 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:14:18.803 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:13:57.199404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:14:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 26 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73240a2b90 msgr2=0x7f73240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73240a2b90 0x7f73240a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f733c198f70 tx=0x7f7330005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 msgr2=0x7f733c197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 0x7f733c197a60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f732c00e9c0 tx=0x7f732c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 shutdown_connections 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f73240a2b90 0x7f73240a5050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f733c1034b0 0x7f733c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102af0 0x7f733c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 --2- 172.21.15.33:0/4210509713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f733c0ffa80 0x7f733c197a60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 >> 172.21.15.33:0/4210509713 conn(0x7f733c0fb3b0 msgr2=0x7f733c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 shutdown_connections 2024-04-24T23:14:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.803+0000 7f7341f7e640 1 -- 172.21.15.33:0/4210509713 wait complete. 2024-04-24T23:14:18.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:14:18.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 -- 172.21.15.33:0/3956854550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 msgr2=0x7f9850100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 --2- 172.21.15.33:0/3956854550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f9850100ab0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9840009900 tx=0x7f984002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 -- 172.21.15.33:0/3956854550 shutdown_connections 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 --2- 172.21.15.33:0/3956854550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850102240 0x7f9850108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 --2- 172.21.15.33:0/3956854550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9850101880 0x7f9850101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 --2- 172.21.15.33:0/3956854550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f9850100ab0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 -- 172.21.15.33:0/3956854550 >> 172.21.15.33:0/3956854550 conn(0x7f98500fbc00 msgr2=0x7f98500fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 -- 172.21.15.33:0/3956854550 shutdown_connections 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.966+0000 7f98551b0640 1 -- 172.21.15.33:0/3956854550 wait complete. 2024-04-24T23:14:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 Processor -- start 2024-04-24T23:14:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 -- start start 2024-04-24T23:14:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f985006a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850101880 0x7f985019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9850102240 0x7f985019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f985010c2e0 con 0x7f9850100690 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f985010c160 con 0x7f9850101880 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f98551b0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f985010c460 con 0x7f9850102240 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f984ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f985006a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f984ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f985006a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40988/0 (socket says 172.21.15.33:40988) 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f984e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850101880 0x7f985019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f984ed9d640 1 -- 172.21.15.33:0/1809031545 learned_addr learned my addr 172.21.15.33:0/1809031545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.967+0000 7f984f59e640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9850102240 0x7f985019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984e59c640 1 -- 172.21.15.33:0/1809031545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9850102240 msgr2=0x7f985019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984e59c640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9850102240 0x7f985019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984e59c640 1 -- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 msgr2=0x7f985006a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984e59c640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f985006a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984e59c640 1 -- 172.21.15.33:0/1809031545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985019f700 con 0x7f9850101880 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984ed9d640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f985006a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f984e59c640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850101880 0x7f985019c7d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f984400c970 tx=0x7f984400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f9837fff640 1 -- 172.21.15.33:0/1809031545 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844007bb0 con 0x7f9850101880 2024-04-24T23:14:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985019f990 con 0x7f9850101880 2024-04-24T23:14:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f9837fff640 1 -- 172.21.15.33:0/1809031545 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844007d50 con 0x7f9850101880 2024-04-24T23:14:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f9837fff640 1 -- 172.21.15.33:0/1809031545 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844018820 con 0x7f9850101880 2024-04-24T23:14:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.968+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985019ff30 con 0x7f9850101880 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 Processor -- start 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9837fff640 1 -- 172.21.15.33:0/1809031545 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f98440189c0 con 0x7f9850101880 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 -- start start 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508121650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508126720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x563508129220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563508085670 con 0x563508126d90 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56350802d970 con 0x563508121c20 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9857fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563508081630 con 0x563508121230 2024-04-24T23:14:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x563508129220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x563508129220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40998/0 (socket says 172.21.15.33:40998) 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 -- 172.21.15.33:0/1932755322 learned_addr learned my addr 172.21.15.33:0/1932755322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984ed9d640 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508126720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984f59e640 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508121650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 -- 172.21.15.33:0/1932755322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 msgr2=0x563508121650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508121650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 -- 172.21.15.33:0/1932755322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 msgr2=0x563508126720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508126720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984e59c640 1 -- 172.21.15.33:0/1932755322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563508051400 con 0x563508126d90 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f984ed9d640 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508126720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.971+0000 7f9837fff640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f98380a2b90 0x7f98380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.972+0000 7f984f59e640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f98380a2b90 0x7f98380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.972+0000 7f984e59c640 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x563508129220 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f98440d7580 tx=0x7f98440d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7225bfac51f26c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.972+0000 7f9837fff640 1 -- 172.21.15.33:0/1809031545 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f98440cf3f0 con 0x7f9850101880 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.972+0000 7f98357fa640 1 -- 172.21.15.33:0/1932755322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844097b60 con 0x563508126d90 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.972+0000 7f98357fa640 1 -- 172.21.15.33:0/1932755322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844084e50 con 0x563508126d90 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.972+0000 7f98357fa640 1 -- 172.21.15.33:0/1932755322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844018cb0 con 0x563508126d90 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f984f59e640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f98380a2b90 0x7f98380a5050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x563508122480 tx=0x7f9848009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- 172.21.15.33:0/1932755322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 msgr2=0x563508129220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x563508129220 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f98440d7580 tx=0x7f98440d7670 comp rx=0 tx=0).stop 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- 172.21.15.33:0/1932755322 shutdown_connections 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x563508129220 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508126720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- 172.21.15.33:0/1932755322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508121650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- 172.21.15.33:0/1932755322 >> 172.21.15.33:0/1932755322 conn(0x56350811c8a0 msgr2=0x56350811cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- 172.21.15.33:0/1932755322 shutdown_connections 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- 172.21.15.33:0/1932755322 wait complete. 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- start start 2024-04-24T23:14:18.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 Processor -- start 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121230 0x56350812b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121c20 0x56350812b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563508129ce0 con 0x563508126d90 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563508041050 con 0x563508121230 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563508129e60 con 0x563508121c20 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f9857fa0740 10 client.0 _ll_register_callbacks cb 0x563507d98be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f984f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.973+0000 7f984f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081328d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41000/0 (socket says 172.21.15.33:41000) 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 -- 172.21.15.33:0/3156576249 learned_addr learned my addr 172.21.15.33:0/3156576249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984e59c640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121c20 0x56350812b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984ed9d640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121230 0x56350812b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 -- 172.21.15.33:0/3156576249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121c20 msgr2=0x56350812b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121c20 0x56350812b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 -- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121230 msgr2=0x56350812b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121230 0x56350812b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 -- 172.21.15.33:0/3156576249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563508012dc0 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984e59c640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121c20 0x56350812b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f984f59e640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081328d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9848003fd0 tx=0x7f9848004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22ff744ad68defa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9832ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9832ff5640 1 -- 172.21.15.33:0/3156576249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984800f8f0 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9832ff5640 1 -- 172.21.15.33:0/3156576249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f984800fa90 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9844084e50 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9832ff5640 1 -- 172.21.15.33:0/3156576249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f984800fc90 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9857fa0740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.974+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563507d993f0 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9832ff5640 1 -- 172.21.15.33:0/3156576249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844084e50 con 0x563508126d90 2024-04-24T23:14:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9832ff5640 1 -- 172.21.15.33:0/3156576249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9848026070 con 0x563508126d90 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9857fa0740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9857fa0740 10 client.15888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9857fa0740 20 client.15888 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563508137290 0x563508139720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9857fa0740 4 client.15888 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563508092bb0 con 0x563508137290 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f984ed9d640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563508137290 0x563508139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f984ed9d640 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563508137290 0x563508139720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ceda25d4ec6e533d server_cookie=3c3391f64a9a15ce in_seq=0 out_seq=0 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.975+0000 7f9832ff5640 10 client.15888 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9832ff5640 1 -- 172.21.15.33:0/3156576249 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563508092bb0 con 0x563508137290 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9832ff5640 10 client.15888 handle_command_reply: tid=0 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 client.15888 shutdown 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 msgr2=0x5635081328d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081328d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9848003fd0 tx=0x7f9848004ab0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 shutdown_connections 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563508137290 0x563508139720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081328d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121c20 0x56350812b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9832ff5640 0 client.15888 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3156576249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121230 0x56350812b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.976+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 >> 172.21.15.33:0/3156576249 conn(0x56350811c8a0 msgr2=0x56350811bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.977+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 shutdown_connections 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.977+0000 7f9857fa0740 1 -- 172.21.15.33:0/3156576249 wait complete. 2024-04-24T23:14:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.977+0000 7f9857fa0740 20 client.15888 trim_cache size 0 max 16384 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.991+0000 7f9857fa0740 1 Processor -- start 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.991+0000 7f9857fa0740 1 -- start start 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x5635081262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 0x5635081224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508126d90 0x5635081229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9857fa0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98501acc50 con 0x563508121c20 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9857fa0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98501aa520 con 0x563508126d90 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9857fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98501a8370 con 0x563508121230 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 0x5635081224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 0x5635081224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41016/0 (socket says 172.21.15.33:41016) 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 -- 172.21.15.33:0/841853251 learned_addr learned my addr 172.21.15.33:0/841853251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984f59e640 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508126d90 0x5635081229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984ed9d640 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x5635081262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 -- 172.21.15.33:0/841853251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 msgr2=0x5635081262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x5635081262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 -- 172.21.15.33:0/841853251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508126d90 msgr2=0x5635081229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508126d90 0x5635081229f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 -- 172.21.15.33:0/841853251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563507e637d0 con 0x563508121c20 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f984e59c640 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 0x5635081224b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f984409b5c0 tx=0x7f984400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fa91be1b03ad67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9834ff9640 1 -- 172.21.15.33:0/841853251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984409b220 con 0x563508121c20 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9834ff9640 1 -- 172.21.15.33:0/841853251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844084e50 con 0x563508121c20 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.992+0000 7f9834ff9640 1 -- 172.21.15.33:0/841853251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984409b400 con 0x563508121c20 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 -- 172.21.15.33:0/841853251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 msgr2=0x5635081224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 0x5635081224b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f984409b5c0 tx=0x7f984400fec0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 -- 172.21.15.33:0/841853251 shutdown_connections 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508126d90 0x5635081229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508121c20 0x5635081224b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 --2- 172.21.15.33:0/841853251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x5635081262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 -- 172.21.15.33:0/841853251 >> 172.21.15.33:0/841853251 conn(0x56350811c8a0 msgr2=0x56350811bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 -- 172.21.15.33:0/841853251 shutdown_connections 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 -- 172.21.15.33:0/841853251 wait complete. 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.993+0000 7f9857fa0740 1 -- start start 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 Processor -- start 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508199000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508199540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081a0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98501a89e0 con 0x563508126d90 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98501a9e50 con 0x563508121c20 2024-04-24T23:14:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98501a90b0 con 0x563508121230 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9857fa0740 10 client.0 _ll_register_callbacks cb 0x7f9848009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55484/0 (socket says 172.21.15.33:55484) 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57812/0 (socket says 172.21.15.33:57812) 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984e59c640 1 -- 172.21.15.33:0/3020946765 learned_addr learned my addr 172.21.15.33:0/3020946765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 -- 172.21.15.33:0/3020946765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 msgr2=0x563508199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 -- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 msgr2=0x563508199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 -- 172.21.15.33:0/3020946765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635081a0b00 con 0x563508126d90 2024-04-24T23:14:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984ed9d640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984e59c640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f984f59e640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081a0510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9848028820 tx=0x7f9848011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a1dfa5243c6768 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.994+0000 7f9832ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9832ff5640 1 -- 172.21.15.33:0/3020946765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848028440 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9832ff5640 1 -- 172.21.15.33:0/3020946765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9848011030 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9844084e50 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9832ff5640 1 -- 172.21.15.33:0/3020946765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9848043050 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5635081a1e70 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9832ff5640 1 -- 172.21.15.33:0/3020946765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844084e50 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9832ff5640 1 -- 172.21.15.33:0/3020946765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f984803e030 con 0x563508126d90 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 10 client.15894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 20 client.15894 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5635081a28c0 0x5635081a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 4 client.15894 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56350808cf00 con 0x5635081a28c0 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.995+0000 7f984ed9d640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5635081a28c0 0x5635081a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f984ed9d640 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5635081a28c0 0x5635081a4ce0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=267c7e00d477586b server_cookie=833295474f04cda4 in_seq=0 out_seq=0 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9832ff5640 10 client.15894 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9832ff5640 1 -- 172.21.15.33:0/3020946765 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56350808cf00 con 0x5635081a28c0 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9832ff5640 10 client.15894 handle_command_reply: tid=0 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 client.15894 shutdown 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 msgr2=0x5635081a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081a0510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9848028820 tx=0x7f9848011f50 comp rx=0 tx=0).stop 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 shutdown_connections 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5635081a28c0 0x5635081a4ce0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563508126d90 0x5635081a0510 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563508121c20 0x563508199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 --2- 172.21.15.33:0/3020946765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563508121230 0x563508199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9832ff5640 0 client.15894 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 >> 172.21.15.33:0/3020946765 conn(0x56350811c8a0 msgr2=0x56350811f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 shutdown_connections 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 1 -- 172.21.15.33:0/3020946765 wait complete. 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.996+0000 7f9857fa0740 20 client.15894 trim_cache size 0 max 16384 2024-04-24T23:14:19.005 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f98380a2b90 msgr2=0x7f98380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f98380a2b90 0x7f98380a5050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x563508122480 tx=0x7f9848009500 comp rx=0 tx=0).stop 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850101880 msgr2=0x7f985019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850101880 0x7f985019c7d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f984400c970 tx=0x7f984400ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 shutdown_connections 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f98380a2b90 0x7f98380a5050 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9850102240 0x7f985019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9850101880 0x7f985019c7d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 --2- 172.21.15.33:0/1809031545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9850100690 0x7f985006a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 >> 172.21.15.33:0/1809031545 conn(0x7f98500fbc00 msgr2=0x7f9850104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 shutdown_connections 2024-04-24T23:14:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:18.997+0000 7f98551b0640 1 -- 172.21.15.33:0/1809031545 wait complete. 2024-04-24T23:14:19.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:14:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 -- 172.21.15.33:0/2250308702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 msgr2=0x7fceac101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 --2- 172.21.15.33:0/2250308702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac101cf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009900 tx=0x7fce9c02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 -- 172.21.15.33:0/2250308702 shutdown_connections 2024-04-24T23:14:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 --2- 172.21.15.33:0/2250308702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceac102230 0x7fceac108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 --2- 172.21.15.33:0/2250308702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac101cf0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 --2- 172.21.15.33:0/2250308702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac100680 0x7fceac100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 -- 172.21.15.33:0/2250308702 >> 172.21.15.33:0/2250308702 conn(0x7fceac0fbbf0 msgr2=0x7fceac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 -- 172.21.15.33:0/2250308702 shutdown_connections 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.118+0000 7fceb367b640 1 -- 172.21.15.33:0/2250308702 wait complete. 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 Processor -- start 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 -- start start 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceac100680 0x7fceac069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac102230 0x7fceac19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fceac10c300 con 0x7fceac101870 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fceac10c180 con 0x7fceac102230 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb367b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fceac10c480 con 0x7fceac100680 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41044/0 (socket says 172.21.15.33:41044) 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 -- 172.21.15.33:0/503633247 learned_addr learned my addr 172.21.15.33:0/503633247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb1c18640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac102230 0x7fceac19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb1417640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceac100680 0x7fceac069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 -- 172.21.15.33:0/503633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceac100680 msgr2=0x7fceac069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceac100680 0x7fceac069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 -- 172.21.15.33:0/503633247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac102230 msgr2=0x7fceac19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac102230 0x7fceac19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.119+0000 7fceb0c16640 1 -- 172.21.15.33:0/503633247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac19f710 con 0x7fceac101870 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fceb1c18640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac102230 0x7fceac19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fceb0c16640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac19c810 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009230 tx=0x7fce9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fcea27fc640 1 -- 172.21.15.33:0/503633247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c03f030 con 0x7fceac101870 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fcea27fc640 1 -- 172.21.15.33:0/503633247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c003ce0 con 0x7fceac101870 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fcea27fc640 1 -- 172.21.15.33:0/503633247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c035070 con 0x7fceac101870 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac19f9a0 con 0x7fceac101870 2024-04-24T23:14:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.120+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac19fe40 con 0x7fceac101870 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.122+0000 7fcea27fc640 1 -- 172.21.15.33:0/503633247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fce9c042030 con 0x7fceac101870 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.123+0000 7fcea27fc640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce980a2c10 0x7fce980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.123+0000 7fceb6381740 1 Processor -- start 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.123+0000 7fcea27fc640 1 -- 172.21.15.33:0/503633247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fce9c0ef770 con 0x7fceac101870 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.123+0000 7fceb6381740 1 -- start start 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.123+0000 7fceb1417640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce980a2c10 0x7fce980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb6381740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2709a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb6381740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd270eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb6381740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2711630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb6381740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbd2669670 con 0x55dbd270f1a0 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb6381740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dbd2611970 con 0x55dbd270a030 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb6381740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dbd2665630 con 0x55dbd2709640 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd270eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd270eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55508/0 (socket says 172.21.15.33:55508) 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 -- 172.21.15.33:0/2944480258 learned_addr learned my addr 172.21.15.33:0/2944480258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb1c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2711630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb1417640 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2709a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb1417640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce980a2c10 0x7fce980a50d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcea80095f0 tx=0x7fcea8011810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 -- 172.21.15.33:0/2944480258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 msgr2=0x55dbd2709a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2709a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 -- 172.21.15.33:0/2944480258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 msgr2=0x55dbd2711630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2711630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 -- 172.21.15.33:0/2944480258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd2635400 con 0x55dbd270a030 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fceb0c16640 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd270eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0bbc80 tx=0x7fce9c0045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9ce411ce0fe4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fce977fe640 1 -- 172.21.15.33:0/2944480258 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c049040 con 0x55dbd270a030 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fce977fe640 1 -- 172.21.15.33:0/2944480258 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c0c48b0 con 0x55dbd270a030 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.124+0000 7fce977fe640 1 -- 172.21.15.33:0/2944480258 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c03bc10 con 0x55dbd270a030 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 -- 172.21.15.33:0/2944480258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 msgr2=0x55dbd270eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd270eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0bbc80 tx=0x7fce9c0045a0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 -- 172.21.15.33:0/2944480258 shutdown_connections 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2711630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd270eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 --2- 172.21.15.33:0/2944480258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2709a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 -- 172.21.15.33:0/2944480258 >> 172.21.15.33:0/2944480258 conn(0x55dbd27008a0 msgr2=0x55dbd2700ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 -- 172.21.15.33:0/2944480258 shutdown_connections 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 -- 172.21.15.33:0/2944480258 wait complete. 2024-04-24T23:14:19.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.125+0000 7fceb6381740 1 -- start start 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 Processor -- start 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd27134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd2713a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd271aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbd2711f10 con 0x55dbd270f1a0 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dbd2625050 con 0x55dbd270a030 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dbd2712090 con 0x55dbd2709640 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb6381740 10 client.0 _ll_register_callbacks cb 0x55dbd237cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb1c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd271aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb1417640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd27134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb1c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd271aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41058/0 (socket says 172.21.15.33:41058) 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb1c18640 1 -- 172.21.15.33:0/3718037275 learned_addr learned my addr 172.21.15.33:0/3718037275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd2713a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb0c16640 1 -- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 msgr2=0x55dbd27134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb0c16640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd27134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb0c16640 1 -- 172.21.15.33:0/3718037275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 msgr2=0x55dbd271aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb0c16640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd271aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb0c16640 1 -- 172.21.15.33:0/3718037275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd25f6dc0 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb1417640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd27134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.126+0000 7fceb1c18640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd271aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb0c16640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd2713a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0b8100 tx=0x7fce9c005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fba0d62a0a352e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fce94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fce94ff9640 1 -- 172.21.15.33:0/3718037275 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0d8020 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fce94ff9640 1 -- 172.21.15.33:0/3718037275 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fce9c0b8360 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9c0c48b0 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fce94ff9640 1 -- 172.21.15.33:0/3718037275 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce9c049070 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 10 client.25445 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dbd237d3f0 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fce94ff9640 1 -- 172.21.15.33:0/3718037275 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c0c48b0 con 0x55dbd270a030 2024-04-24T23:14:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fce94ff9640 1 -- 172.21.15.33:0/3718037275 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fce9c0c4420 con 0x55dbd270a030 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 10 client.25445 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 10 client.25445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 20 client.25445 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd271efd0 0x55dbd2721400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 4 client.25445 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.127+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbd2676bb0 con 0x55dbd271efd0 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.128+0000 7fceb1417640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd271efd0 0x55dbd2721400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.128+0000 7fceb1417640 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd271efd0 0x55dbd2721400 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=849e90f1fa040f88 server_cookie=5317d5a3b738093d in_seq=0 out_seq=0 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.128+0000 7fce94ff9640 10 client.25445 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fce94ff9640 1 -- 172.21.15.33:0/3718037275 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbd2676bb0 con 0x55dbd271efd0 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fce94ff9640 10 client.25445 handle_command_reply: tid=0 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 client.25445 shutdown 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 msgr2=0x55dbd2713a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd2713a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0b8100 tx=0x7fce9c005b70 comp rx=0 tx=0).stop 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 shutdown_connections 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd271efd0 0x55dbd2721400 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd271aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fce94ff9640 0 client.25445 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd2713a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 --2- 172.21.15.33:0/3718037275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd27134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 >> 172.21.15.33:0/3718037275 conn(0x55dbd27008a0 msgr2=0x55dbd26ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 shutdown_connections 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 1 -- 172.21.15.33:0/3718037275 wait complete. 2024-04-24T23:14:19.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.129+0000 7fceb6381740 20 client.25445 trim_cache size 0 max 16384 2024-04-24T23:14:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 Processor -- start 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 -- start start 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd26ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd271aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2703780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1aaa60 con 0x55dbd270f1a0 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1a8330 con 0x55dbd270a030 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb6381740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1a6180 con 0x55dbd2709640 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2703780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2703780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41068/0 (socket says 172.21.15.33:41068) 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1417640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd26ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 -- 172.21.15.33:0/90740836 learned_addr learned my addr 172.21.15.33:0/90740836 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb0c16640 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd271aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 -- 172.21.15.33:0/90740836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 msgr2=0x55dbd26ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd26ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 -- 172.21.15.33:0/90740836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 msgr2=0x55dbd271aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd271aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1c18640 1 -- 172.21.15.33:0/90740836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd24477d0 con 0x55dbd270f1a0 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.143+0000 7fceb1417640 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd26ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb1c18640 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2703780 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcea400fcb0 tx=0x7fcea4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0f9acee2bc12e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fce96ffd640 1 -- 172.21.15.33:0/90740836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4022070 con 0x55dbd270f1a0 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fce96ffd640 1 -- 172.21.15.33:0/90740836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea4003e50 con 0x55dbd270f1a0 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fce96ffd640 1 -- 172.21.15.33:0/90740836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4004830 con 0x55dbd270f1a0 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb6381740 1 -- 172.21.15.33:0/90740836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 msgr2=0x55dbd2703780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb6381740 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2703780 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcea400fcb0 tx=0x7fcea4009500 comp rx=0 tx=0).stop 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb6381740 1 -- 172.21.15.33:0/90740836 shutdown_connections 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb6381740 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270f1a0 0x55dbd2703780 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb6381740 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270a030 0x55dbd271aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.144+0000 7fceb6381740 1 --2- 172.21.15.33:0/90740836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd26ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- 172.21.15.33:0/90740836 >> 172.21.15.33:0/90740836 conn(0x55dbd27008a0 msgr2=0x55dbd26ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- 172.21.15.33:0/90740836 shutdown_connections 2024-04-24T23:14:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- 172.21.15.33:0/90740836 wait complete. 2024-04-24T23:14:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- start start 2024-04-24T23:14:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 Processor -- start 2024-04-24T23:14:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2717b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 0x55dbd2718050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270f1a0 0x55dbd2719010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1a67f0 con 0x55dbd270a030 2024-04-24T23:14:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1a7c60 con 0x55dbd270f1a0 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1a6ec0 con 0x55dbd2709640 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb6381740 10 client.0 _ll_register_callbacks cb 0x7fcea4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 0x55dbd2718050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb1417640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2717b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb0c16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 0x55dbd2718050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41082/0 (socket says 172.21.15.33:41082) 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb0c16640 1 -- 172.21.15.33:0/1222731130 learned_addr learned my addr 172.21.15.33:0/1222731130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.145+0000 7fceb1c18640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270f1a0 0x55dbd2719010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb1417640 1 -- 172.21.15.33:0/1222731130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270f1a0 msgr2=0x55dbd2719010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb1417640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270f1a0 0x55dbd2719010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb1417640 1 -- 172.21.15.33:0/1222731130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 msgr2=0x55dbd2718050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb1417640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 0x55dbd2718050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb1417640 1 -- 172.21.15.33:0/1222731130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd2719600 con 0x55dbd2709640 2024-04-24T23:14:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb0c16640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 0x55dbd2718050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb1417640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2717b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55dbd2721c40 tx=0x7fcea8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3c021d32718bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fce94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fce94ff9640 1 -- 172.21.15.33:0/1222731130 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8017040 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea4003e50 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fce94ff9640 1 -- 172.21.15.33:0/1222731130 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcea802b430 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fce94ff9640 1 -- 172.21.15.33:0/1222731130 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcea803a050 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb6381740 10 client.25480 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dbd271a9d0 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.146+0000 7fce94ff9640 1 -- 172.21.15.33:0/1222731130 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea4003e50 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fce94ff9640 1 -- 172.21.15.33:0/1222731130 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fcea801db50 con 0x55dbd2709640 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb6381740 10 client.25480 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb6381740 10 client.25480 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb6381740 20 client.25480 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb6381740 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd27859a0 0x55dbd2787dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb6381740 4 client.25480 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbd2670f00 con 0x55dbd27859a0 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb0c16640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd27859a0 0x55dbd2787dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fce94ff9640 10 client.25480 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.147+0000 7fceb0c16640 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd27859a0 0x55dbd2787dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd434f68271b43a1 server_cookie=ab32be35f4035a68 in_seq=0 out_seq=0 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fce94ff9640 1 -- 172.21.15.33:0/1222731130 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbd2670f00 con 0x55dbd27859a0 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fce94ff9640 10 client.25480 handle_command_reply: tid=0 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 client.25480 shutdown 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 msgr2=0x55dbd2717b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2717b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55dbd2721c40 tx=0x7fcea8004730 comp rx=0 tx=0).stop 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 shutdown_connections 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dbd27859a0 0x55dbd2787dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbd270f1a0 0x55dbd2719010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbd270a030 0x55dbd2718050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 --2- 172.21.15.33:0/1222731130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbd2709640 0x55dbd2717b10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fce94ff9640 0 client.25480 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 >> 172.21.15.33:0/1222731130 conn(0x55dbd27008a0 msgr2=0x55dbd26ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 shutdown_connections 2024-04-24T23:14:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 1 -- 172.21.15.33:0/1222731130 wait complete. 2024-04-24T23:14:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb6381740 20 client.25480 trim_cache size 0 max 16384 2024-04-24T23:14:19.156 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce980a2c10 msgr2=0x7fce980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce980a2c10 0x7fce980a50d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcea80095f0 tx=0x7fcea8011810 comp rx=0 tx=0).stop 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 msgr2=0x7fceac19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac19c810 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009230 tx=0x7fce9c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 shutdown_connections 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fce980a2c10 0x7fce980a50d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fceac102230 0x7fceac19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceac101870 0x7fceac19c810 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 --2- 172.21.15.33:0/503633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fceac100680 0x7fceac069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.148+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 >> 172.21.15.33:0/503633247 conn(0x7fceac0fbbf0 msgr2=0x7fceac104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.149+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 shutdown_connections 2024-04-24T23:14:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.149+0000 7fceb367b640 1 -- 172.21.15.33:0/503633247 wait complete. 2024-04-24T23:14:19.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:18 smithi033 ceph-mon[29201]: pgmap v420: 129 pgs: 129 active+clean; 30 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-04-24T23:14:19.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:18 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4210509713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:19.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:14:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:18 smithi038 ceph-mon[32450]: pgmap v420: 129 pgs: 129 active+clean; 30 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-04-24T23:14:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:18 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4210509713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:19.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:18 smithi117 ceph-mon[32529]: pgmap v420: 129 pgs: 129 active+clean; 30 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-04-24T23:14:19.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:18 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4210509713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.268+0000 7f164891d640 1 -- 172.21.15.33:0/3110606498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16441021f0 msgr2=0x7f16441045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.268+0000 7f164891d640 1 --2- 172.21.15.33:0/3110606498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16441021f0 0x7f16441045e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1634009900 tx=0x7f163402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- 172.21.15.33:0/3110606498 shutdown_connections 2024-04-24T23:14:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 --2- 172.21.15.33:0/3110606498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1644107450 0x7f16441098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 --2- 172.21.15.33:0/3110606498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1644104b20 0x7f1644106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 --2- 172.21.15.33:0/3110606498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16441021f0 0x7f16441045e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- 172.21.15.33:0/3110606498 >> 172.21.15.33:0/3110606498 conn(0x7f16440fbbe0 msgr2=0x7f16440fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- 172.21.15.33:0/3110606498 shutdown_connections 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- 172.21.15.33:0/3110606498 wait complete. 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 Processor -- start 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- start start 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16441021f0 0x7f1644101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 0x7f16441001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1644107450 0x7f16441006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f164410d3f0 con 0x7f1644104b20 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f164410d270 con 0x7f1644107450 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.269+0000 7f164891d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f164410d570 con 0x7f16441021f0 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 0x7f16441001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 0x7f16441001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41112/0 (socket says 172.21.15.33:41112) 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 -- 172.21.15.33:0/115038177 learned_addr learned my addr 172.21.15.33:0/115038177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f1642d9d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1644107450 0x7f16441006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f164259c640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16441021f0 0x7f1644101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 -- 172.21.15.33:0/115038177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16441021f0 msgr2=0x7f1644101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16441021f0 0x7f1644101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 -- 172.21.15.33:0/115038177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1644107450 msgr2=0x7f16441006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1644107450 0x7f16441006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 -- 172.21.15.33:0/115038177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1644100f70 con 0x7f1644104b20 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163bfff640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 0x7f16441001a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f162c00e9c0 tx=0x7f162c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163b7fe640 1 -- 172.21.15.33:0/115038177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00cd80 con 0x7f1644104b20 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16441a1be0 con 0x7f1644104b20 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163b7fe640 1 -- 172.21.15.33:0/115038177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162c005500 con 0x7f1644104b20 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f163b7fe640 1 -- 172.21.15.33:0/115038177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c010690 con 0x7f1644104b20 2024-04-24T23:14:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.270+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16441a2180 con 0x7f1644104b20 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f163b7fe640 1 -- 172.21.15.33:0/115038177 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f162c0046c0 con 0x7f1644104b20 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 Processor -- start 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 -- start start 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52af0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52af5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af5d90 0x555c52af8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c52a54670 con 0x555c52af0230 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555c529fc970 con 0x555c52af5d90 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.273+0000 7f164b621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c52a50630 con 0x555c52af0c20 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f163b7fe640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f16300a2b90 0x7f16300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52af5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52af5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57902/0 (socket says 172.21.15.33:57902) 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 -- 172.21.15.33:0/1454010146 learned_addr learned my addr 172.21.15.33:0/1454010146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f163bfff640 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af5d90 0x555c52af8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f1642d9d640 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52af0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f1642d9d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f16300a2b90 0x7f16300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 -- 172.21.15.33:0/1454010146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af5d90 msgr2=0x555c52af8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af5d90 0x555c52af8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 -- 172.21.15.33:0/1454010146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 msgr2=0x555c52af0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52af0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f163b7fe640 1 -- 172.21.15.33:0/115038177 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f162c0cf860 con 0x7f1644104b20 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 -- 172.21.15.33:0/1454010146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c52a20400 con 0x555c52af0c20 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f164259c640 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52af5720 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f163402a930 tx=0x7f1634032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ea445672b01d091 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f1642d9d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f16300a2b90 0x7f16300a5050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x555c52af1480 tx=0x7f163c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f1638ff9640 1 -- 172.21.15.33:0/1454010146 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163403f070 con 0x555c52af0c20 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f1638ff9640 1 -- 172.21.15.33:0/1454010146 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16340318b0 con 0x555c52af0c20 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.274+0000 7f1638ff9640 1 -- 172.21.15.33:0/1454010146 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634035070 con 0x555c52af0c20 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 -- 172.21.15.33:0/1454010146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 msgr2=0x555c52af5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52af5720 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f163402a930 tx=0x7f1634032cb0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 -- 172.21.15.33:0/1454010146 shutdown_connections 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af5d90 0x555c52af8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52af5720 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 --2- 172.21.15.33:0/1454010146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52af0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 -- 172.21.15.33:0/1454010146 >> 172.21.15.33:0/1454010146 conn(0x555c52aeb8a0 msgr2=0x555c52aebca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 -- 172.21.15.33:0/1454010146 shutdown_connections 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.275+0000 7f164b621740 1 -- 172.21.15.33:0/1454010146 wait complete. 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 -- start start 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 Processor -- start 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52afa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 0x555c52afa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52b018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c52af8ce0 con 0x555c52af0c20 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555c52a10050 con 0x555c52af0230 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c52af8e60 con 0x555c52af5d90 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164b621740 10 client.0 _ll_register_callbacks cb 0x555c52767be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 0x555c52afa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 0x555c52afa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41128/0 (socket says 172.21.15.33:41128) 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 -- 172.21.15.33:0/616594283 learned_addr learned my addr 172.21.15.33:0/616594283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f164259c640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52afa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f1642d9d640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52b018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 -- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 msgr2=0x555c52b018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52b018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 -- 172.21.15.33:0/616594283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 msgr2=0x555c52afa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52afa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 -- 172.21.15.33:0/616594283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c529e1dc0 con 0x555c52af0c20 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f163bfff640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 0x555c52afa870 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f162c09bcc0 tx=0x7f162c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd7009419733f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.276+0000 7f1629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f1629ffb640 1 -- 172.21.15.33:0/616594283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0a4710 con 0x555c52af0c20 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f1629ffb640 1 -- 172.21.15.33:0/616594283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f162c07aba0 con 0x555c52af0c20 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16340318b0 con 0x555c52af0c20 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f1629ffb640 1 -- 172.21.15.33:0/616594283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f162c09c430 con 0x555c52af0c20 2024-04-24T23:14:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 10 client.15924 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c527683f0 con 0x555c52af0c20 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f1629ffb640 1 -- 172.21.15.33:0/616594283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16340318b0 con 0x555c52af0c20 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f1629ffb640 1 -- 172.21.15.33:0/616594283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f162c09b180 con 0x555c52af0c20 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 20 client.15924 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b06210 0x555c52b08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 4 client.15924 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c52a61bb0 con 0x555c52b06210 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.277+0000 7f164259c640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b06210 0x555c52b08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.278+0000 7f164259c640 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b06210 0x555c52b08660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a936210da0484d server_cookie=ca91d4042da8b56e in_seq=0 out_seq=0 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.278+0000 7f1629ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f1629ffb640 1 -- 172.21.15.33:0/616594283 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c52a61bb0 con 0x555c52b06210 2024-04-24T23:14:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f1629ffb640 10 client.15924 handle_command_reply: tid=0 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 client.15924 shutdown 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 msgr2=0x555c52afa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 0x555c52afa870 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f162c09bcc0 tx=0x7f162c01d000 comp rx=0 tx=0).stop 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 shutdown_connections 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b06210 0x555c52b08660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52b018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0c20 0x555c52afa870 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 --2- 172.21.15.33:0/616594283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52afa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f1629ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 >> 172.21.15.33:0/616594283 conn(0x555c52aeb8a0 msgr2=0x555c52aeab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 shutdown_connections 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 1 -- 172.21.15.33:0/616594283 wait complete. 2024-04-24T23:14:19.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.279+0000 7f164b621740 20 client.15924 trim_cache size 0 max 16384 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 Processor -- start 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 -- start start 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52aee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0c20 0x555c52b01870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52aeecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16441aeeb0 con 0x555c52af0230 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ac780 con 0x555c52af0c20 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164b621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16441aa5d0 con 0x555c52af5d90 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52aee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52aee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41138/0 (socket says 172.21.15.33:41138) 2024-04-24T23:14:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 -- 172.21.15.33:0/2773165873 learned_addr learned my addr 172.21.15.33:0/2773165873 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f163bfff640 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0c20 0x555c52b01870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 -- 172.21.15.33:0/2773165873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 msgr2=0x555c52aeecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52aeecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 -- 172.21.15.33:0/2773165873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0c20 msgr2=0x555c52b01870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0c20 0x555c52b01870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.294+0000 7f164259c640 1 -- 172.21.15.33:0/2773165873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c528327d0 con 0x555c52af0230 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.295+0000 7f164259c640 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52aee7b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f16340314e0 tx=0x7f163402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497fcbe42447ede5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.295+0000 7f162bfff640 1 -- 172.21.15.33:0/2773165873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163403f040 con 0x555c52af0230 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.295+0000 7f162bfff640 1 -- 172.21.15.33:0/2773165873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163404d430 con 0x555c52af0230 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.295+0000 7f162bfff640 1 -- 172.21.15.33:0/2773165873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634035040 con 0x555c52af0230 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.295+0000 7f164b621740 1 -- 172.21.15.33:0/2773165873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 msgr2=0x555c52aee7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.295+0000 7f164b621740 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52aee7b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f16340314e0 tx=0x7f163402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- 172.21.15.33:0/2773165873 shutdown_connections 2024-04-24T23:14:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af5d90 0x555c52aeecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0c20 0x555c52b01870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 --2- 172.21.15.33:0/2773165873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af0230 0x555c52aee7b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- 172.21.15.33:0/2773165873 >> 172.21.15.33:0/2773165873 conn(0x555c52aeb8a0 msgr2=0x555c52aeab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- 172.21.15.33:0/2773165873 shutdown_connections 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- 172.21.15.33:0/2773165873 wait complete. 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- start start 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 Processor -- start 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52aff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52aff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 0x555c52b00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16441aac40 con 0x555c52af5d90 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ac0b0 con 0x555c52af0230 2024-04-24T23:14:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ab310 con 0x555c52af0c20 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164b621740 10 client.0 _ll_register_callbacks cb 0x7f162c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f1642d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 0x555c52b00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f164259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52aff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f1642d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 0x555c52b00720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41152/0 (socket says 172.21.15.33:41152) 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f163bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52aff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.296+0000 7f1642d9d640 1 -- 172.21.15.33:0/1819591022 learned_addr learned my addr 172.21.15.33:0/1819591022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1642d9d640 1 -- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 msgr2=0x555c52aff710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1642d9d640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52aff710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1642d9d640 1 -- 172.21.15.33:0/1819591022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 msgr2=0x555c52aff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1642d9d640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52aff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1642d9d640 1 -- 172.21.15.33:0/1819591022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c52b00d10 con 0x555c52af5d90 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f164259c640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52aff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f163bfff640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52aff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1642d9d640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 0x555c52b00720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f163c009270 tx=0x7f163c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5449b2a813baf99e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1629ffb640 1 -- 172.21.15.33:0/1819591022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c003e00 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163404d430 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1629ffb640 1 -- 172.21.15.33:0/1819591022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f163c004480 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f1629ffb640 1 -- 172.21.15.33:0/1819591022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f163c036050 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f164b621740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.297+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c52b6ca70 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f1629ffb640 1 -- 172.21.15.33:0/1819591022 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163404d430 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f1629ffb640 1 -- 172.21.15.33:0/1819591022 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f163c01d040 con 0x555c52af5d90 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164b621740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164b621740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164b621740 20 client.15930 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164b621740 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b6d4d0 0x555c52b6f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164b621740 4 client.15930 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c52a5bf00 con 0x555c52b6d4d0 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164259c640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b6d4d0 0x555c52b6f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f164259c640 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b6d4d0 0x555c52b6f8f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ac59251d7199a server_cookie=fb91d4a9d7fe4b53 in_seq=0 out_seq=0 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.298+0000 7f1629ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f1629ffb640 1 -- 172.21.15.33:0/1819591022 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c52a5bf00 con 0x555c52b6d4d0 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f1629ffb640 10 client.15930 handle_command_reply: tid=0 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 client.15930 shutdown 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 msgr2=0x555c52b00720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 0x555c52b00720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f163c009270 tx=0x7f163c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 shutdown_connections 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555c52b6d4d0 0x555c52b6f8f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c52af5d90 0x555c52b00720 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f1629ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c52af0c20 0x555c52aff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 --2- 172.21.15.33:0/1819591022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c52af0230 0x555c52aff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 >> 172.21.15.33:0/1819591022 conn(0x555c52aeb8a0 msgr2=0x555c52af16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 shutdown_connections 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 1 -- 172.21.15.33:0/1819591022 wait complete. 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164b621740 20 client.15930 trim_cache size 0 max 16384 2024-04-24T23:14:19.309 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:14:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f16300a2b90 msgr2=0x7f16300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f16300a2b90 0x7f16300a5050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x555c52af1480 tx=0x7f163c009500 comp rx=0 tx=0).stop 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 msgr2=0x7f16441001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 0x7f16441001a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f162c00e9c0 tx=0x7f162c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 shutdown_connections 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f16300a2b90 0x7f16300a5050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1644107450 0x7f16441006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.299+0000 7f164891d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1644104b20 0x7f16441001a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.300+0000 7f164891d640 1 --2- 172.21.15.33:0/115038177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16441021f0 0x7f1644101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.300+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 >> 172.21.15.33:0/115038177 conn(0x7f16440fbbe0 msgr2=0x7f1644102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.300+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 shutdown_connections 2024-04-24T23:14:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:19.300+0000 7f164891d640 1 -- 172.21.15.33:0/115038177 wait complete. 2024-04-24T23:14:19.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 64320 15.21 MB/sec warmup 24 sec latency 11.562 ms 2024-04-24T23:14:20.313 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:14:20.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:20.314 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:14:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 -- 172.21.15.33:0/667476566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 msgr2=0x7ff164100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 --2- 172.21.15.33:0/667476566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff164100ac0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff154009900 tx=0x7ff15402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 -- 172.21.15.33:0/667476566 shutdown_connections 2024-04-24T23:14:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 --2- 172.21.15.33:0/667476566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff164102250 0x7ff164108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 --2- 172.21.15.33:0/667476566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff164101890 0x7ff164101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 --2- 172.21.15.33:0/667476566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff164100ac0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 -- 172.21.15.33:0/667476566 >> 172.21.15.33:0/667476566 conn(0x7ff1640fbc30 msgr2=0x7ff1640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 -- 172.21.15.33:0/667476566 shutdown_connections 2024-04-24T23:14:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 -- 172.21.15.33:0/667476566 wait complete. 2024-04-24T23:14:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.429+0000 7ff168a3f640 1 Processor -- start 2024-04-24T23:14:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 -- start start 2024-04-24T23:14:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff16406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff164101890 0x7ff16419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff164102250 0x7ff16419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410c320 con 0x7ff1641006a0 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410c1a0 con 0x7ff164102250 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff168a3f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410c4a0 con 0x7ff164101890 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff16406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff16406a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41164/0 (socket says 172.21.15.33:41164) 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 -- 172.21.15.33:0/2574518758 learned_addr learned my addr 172.21.15.33:0/2574518758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff161d9b640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff164101890 0x7ff16419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff164102250 0x7ff16419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 -- 172.21.15.33:0/2574518758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff164101890 msgr2=0x7ff16419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff164101890 0x7ff16419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 -- 172.21.15.33:0/2574518758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff164102250 msgr2=0x7ff16419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff164102250 0x7ff16419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 -- 172.21.15.33:0/2574518758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16419f740 con 0x7ff1641006a0 2024-04-24T23:14:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.430+0000 7ff16259c640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff16406a8b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff154031610 tx=0x7ff154004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.431+0000 7ff1537fe640 1 -- 172.21.15.33:0/2574518758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15403f030 con 0x7ff1641006a0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.431+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16419f970 con 0x7ff1641006a0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.431+0000 7ff1537fe640 1 -- 172.21.15.33:0/2574518758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff154003ad0 con 0x7ff1641006a0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.431+0000 7ff1537fe640 1 -- 172.21.15.33:0/2574518758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154035070 con 0x7ff1641006a0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.431+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16419feb0 con 0x7ff1641006a0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff1537fe640 1 -- 172.21.15.33:0/2574518758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff154042030 con 0x7ff1641006a0 2024-04-24T23:14:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 Processor -- start 2024-04-24T23:14:20.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 -- start start 2024-04-24T23:14:20.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 67227 15.19 MB/sec warmup 25 sec latency 14.519 ms 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee194a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee194f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1951d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ee18ad2c0 con 0x556ee194f8f0 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ee1855910 con 0x556ee194a780 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.433+0000 7ff16b82f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ee18a9280 con 0x556ee1949d90 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff161d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1951d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee194f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff1537fe640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff14c0a2b90 0x7ff14c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff161d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1951d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41178/0 (socket says 172.21.15.33:41178) 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee194a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff161d9b640 1 -- 172.21.15.33:0/1840149328 learned_addr learned my addr 172.21.15.33:0/1840149328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff14c0a2b90 0x7ff14c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff1537fe640 1 -- 172.21.15.33:0/2574518758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff1540ef960 con 0x7ff1641006a0 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 -- 172.21.15.33:0/1840149328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 msgr2=0x556ee194a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee194a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 -- 172.21.15.33:0/1840149328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 msgr2=0x556ee1951d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1951d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 -- 172.21.15.33:0/1840149328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ee1866fb0 con 0x556ee194a780 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff14c0a2b90 0x7ff14c0a5050 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x556ee194afe0 tx=0x7ff15c004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.434+0000 7ff16259c640 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee194f280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1540ed120 tx=0x7ff1540b82b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ff46429ebfb274 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff150ff9640 1 -- 172.21.15.33:0/1840149328 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154049070 con 0x556ee194a780 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff150ff9640 1 -- 172.21.15.33:0/1840149328 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15403be40 con 0x556ee194a780 2024-04-24T23:14:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff150ff9640 1 -- 172.21.15.33:0/1840149328 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15403b460 con 0x556ee194a780 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 -- 172.21.15.33:0/1840149328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 msgr2=0x556ee194f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee194f280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1540ed120 tx=0x7ff1540b82b0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 -- 172.21.15.33:0/1840149328 shutdown_connections 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1951d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee194f280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 --2- 172.21.15.33:0/1840149328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee194a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.435+0000 7ff16b82f740 1 -- 172.21.15.33:0/1840149328 >> 172.21.15.33:0/1840149328 conn(0x556ee1911950 msgr2=0x556ee1945810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 -- 172.21.15.33:0/1840149328 shutdown_connections 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 -- 172.21.15.33:0/1840149328 wait complete. 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 -- start start 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 Processor -- start 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 0x556ee1953e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee1954360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194f8f0 0x556ee195b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ee1952660 con 0x556ee1949d90 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ee1868e90 con 0x556ee194a780 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ee19527e0 con 0x556ee194f8f0 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16b82f740 10 client.0 _ll_register_callbacks cb 0x556ee18a4970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 0x556ee1953e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 0x556ee1953e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41192/0 (socket says 172.21.15.33:41192) 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 -- 172.21.15.33:0/2918866732 learned_addr learned my addr 172.21.15.33:0/2918866732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff161d9b640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee1954360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194f8f0 0x556ee195b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 -- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194f8f0 msgr2=0x556ee195b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194f8f0 0x556ee195b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 -- 172.21.15.33:0/2918866732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 msgr2=0x556ee1954360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee1954360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.436+0000 7ff16259c640 1 -- 172.21.15.33:0/2918866732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ee16897e0 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16259c640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 0x556ee1953e20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff1540ed440 tx=0x7ff154003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0c83af8104a692 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff149ffb640 1 -- 172.21.15.33:0/2918866732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154049070 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff149ffb640 1 -- 172.21.15.33:0/2918866732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff1540b8550 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ee18815a0 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff149ffb640 1 -- 172.21.15.33:0/2918866732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff1540c01e0 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ee16e9f70 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff149ffb640 1 -- 172.21.15.33:0/2918866732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556ee18815a0 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff149ffb640 1 -- 172.21.15.33:0/2918866732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff1540c46a0 con 0x556ee1949d90 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 20 client.15942 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee1961ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 4 client.15942 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.437+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ee18ba800 con 0x556ee195f680 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.438+0000 7ff161d9b640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee1961ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.438+0000 7ff161d9b640 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee1961ad0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583a4e56d6f28b3f server_cookie=110e107ca080cca2 in_seq=0 out_seq=0 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.438+0000 7ff149ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.438+0000 7ff149ffb640 1 -- 172.21.15.33:0/2918866732 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ee18ba800 con 0x556ee195f680 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.438+0000 7ff149ffb640 10 client.15942 handle_command_reply: tid=0 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 client.15942 shutdown 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 msgr2=0x556ee1953e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 0x556ee1953e20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff1540ed440 tx=0x7ff154003f20 comp rx=0 tx=0).stop 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 shutdown_connections 2024-04-24T23:14:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee1961ad0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff149ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194f8f0 0x556ee195b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee1954360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2918866732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee1949d90 0x556ee1953e20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 >> 172.21.15.33:0/2918866732 conn(0x556ee1911950 msgr2=0x556ee1944950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 shutdown_connections 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 1 -- 172.21.15.33:0/2918866732 wait complete. 2024-04-24T23:14:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.439+0000 7ff16b82f740 20 client.15942 trim_cache size 0 max 16384 2024-04-24T23:14:20.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 Processor -- start 2024-04-24T23:14:20.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 -- start start 2024-04-24T23:14:20.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee1949d90 0x556ee19447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 0x556ee1944d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee195b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641aaae0 con 0x556ee194f8f0 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641a83b0 con 0x556ee1949d90 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16b82f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641a6200 con 0x556ee194a780 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee195b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff161d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 0x556ee1944d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff16259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee1949d90 0x556ee19447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee195b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41200/0 (socket says 172.21.15.33:41200) 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff161d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 0x556ee1944d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57964/0 (socket says 172.21.15.33:57964) 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 -- 172.21.15.33:0/307399120 learned_addr learned my addr 172.21.15.33:0/307399120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 -- 172.21.15.33:0/307399120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 msgr2=0x556ee1944d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 0x556ee1944d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 -- 172.21.15.33:0/307399120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee1949d90 msgr2=0x556ee19447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee1949d90 0x556ee19447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff162d9d640 1 -- 172.21.15.33:0/307399120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ee1581170 con 0x556ee194f8f0 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.457+0000 7ff161d9b640 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 0x556ee1944d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.458+0000 7ff162d9d640 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee195b330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff15c013ef0 tx=0x7ff15c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0286e14bc07a505 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.458+0000 7ff14bfff640 1 -- 172.21.15.33:0/307399120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c003e00 con 0x556ee194f8f0 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.458+0000 7ff14bfff640 1 -- 172.21.15.33:0/307399120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c00ecb0 con 0x556ee194f8f0 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.458+0000 7ff14bfff640 1 -- 172.21.15.33:0/307399120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0223f0 con 0x556ee194f8f0 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.458+0000 7ff16b82f740 1 -- 172.21.15.33:0/307399120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 msgr2=0x556ee195b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee195b330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff15c013ef0 tx=0x7ff15c005bd0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- 172.21.15.33:0/307399120 shutdown_connections 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee195b330 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee194a780 0x556ee1944d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- 172.21.15.33:0/307399120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee1949d90 0x556ee19447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- 172.21.15.33:0/307399120 >> 172.21.15.33:0/307399120 conn(0x556ee1911950 msgr2=0x556ee19440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- 172.21.15.33:0/307399120 shutdown_connections 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- 172.21.15.33:0/307399120 wait complete. 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- start start 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 Processor -- start 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee195ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee195b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1959180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641a6870 con 0x556ee194f8f0 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641a7ce0 con 0x556ee194a780 2024-04-24T23:14:20.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641a6f40 con 0x556ee1949d90 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16b82f740 10 client.0 _ll_register_callbacks cb 0x7ff1580137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1959180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1959180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41210/0 (socket says 172.21.15.33:41210) 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff161d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee195b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff162d9d640 1 -- 172.21.15.33:0/2908690071 learned_addr learned my addr 172.21.15.33:0/2908690071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.459+0000 7ff16259c640 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee195ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff162d9d640 1 -- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 msgr2=0x556ee195ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee195ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff162d9d640 1 -- 172.21.15.33:0/2908690071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 msgr2=0x556ee195b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee195b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff162d9d640 1 -- 172.21.15.33:0/2908690071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ee1959770 con 0x556ee194f8f0 2024-04-24T23:14:20.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff162d9d640 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1959180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00a010 tx=0x7ff15c0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643ea4190815e94c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff149ffb640 1 -- 172.21.15.33:0/2908690071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c032070 con 0x556ee194f8f0 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15c00ecb0 con 0x556ee194f8f0 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff149ffb640 1 -- 172.21.15.33:0/2908690071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff15c003e00 con 0x556ee194f8f0 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff149ffb640 1 -- 172.21.15.33:0/2908690071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff15c041050 con 0x556ee194f8f0 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff16b82f740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:20.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ee19c65c0 con 0x556ee194f8f0 2024-04-24T23:14:20.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff149ffb640 1 -- 172.21.15.33:0/2908690071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c00ecb0 con 0x556ee194f8f0 2024-04-24T23:14:20.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.460+0000 7ff149ffb640 1 -- 172.21.15.33:0/2908690071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff15c02a070 con 0x556ee194f8f0 2024-04-24T23:14:20.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16b82f740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:20.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16b82f740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:20.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16b82f740 20 client.15948 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:20.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee19c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16b82f740 4 client.15948 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:14:20.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556ee18b4b50 con 0x556ee195f680 2024-04-24T23:14:20.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16259c640 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee19c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff16259c640 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee19c9010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a645df2fd3b067 server_cookie=5e3dfb7375983127 in_seq=0 out_seq=0 2024-04-24T23:14:20.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.461+0000 7ff149ffb640 10 client.15948 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff149ffb640 1 -- 172.21.15.33:0/2908690071 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556ee18b4b50 con 0x556ee195f680 2024-04-24T23:14:20.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff149ffb640 10 client.15948 handle_command_reply: tid=0 2024-04-24T23:14:20.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 client.15948 shutdown 2024-04-24T23:14:20.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 msgr2=0x556ee1959180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1959180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00a010 tx=0x7ff15c0109e0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 shutdown_connections 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ee195f680 0x556ee19c9010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ee194f8f0 0x556ee1959180 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ee194a780 0x556ee195b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 --2- 172.21.15.33:0/2908690071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ee1949d90 0x556ee195ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 >> 172.21.15.33:0/2908690071 conn(0x556ee1911950 msgr2=0x556ee1944950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff149ffb640 0 client.15948 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 shutdown_connections 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 1 -- 172.21.15.33:0/2908690071 wait complete. 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.462+0000 7ff16b82f740 20 client.15948 trim_cache size 0 max 16384 2024-04-24T23:14:20.473 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:14:20.474 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:14:20.474 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "01af613d-45c3-47ad-ae07-3556239f6e94", 2024-04-24T23:14:20.474 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:14:20.474 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:14:20.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff14c0a2b90 msgr2=0x7ff14c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff14c0a2b90 0x7ff14c0a5050 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x556ee194afe0 tx=0x7ff15c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 msgr2=0x7ff16406a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff16406a8b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff154031610 tx=0x7ff154004270 comp rx=0 tx=0).stop 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 shutdown_connections 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff14c0a2b90 0x7ff14c0a5050 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff164102250 0x7ff16419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff164101890 0x7ff16419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 --2- 172.21.15.33:0/2574518758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1641006a0 0x7ff16406a8b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 >> 172.21.15.33:0/2574518758 conn(0x7ff1640fbc30 msgr2=0x7ff164104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 shutdown_connections 2024-04-24T23:14:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.463+0000 7ff168a3f640 1 -- 172.21.15.33:0/2574518758 wait complete. 2024-04-24T23:14:20.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:14:20.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:14:20.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 -- 172.21.15.33:0/2766045780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 msgr2=0x7f29841045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 --2- 172.21.15.33:0/2766045780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f29841045f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2980009900 tx=0x7f298002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 -- 172.21.15.33:0/2766045780 shutdown_connections 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 --2- 172.21.15.33:0/2766045780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2984107460 0x7f29841098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 --2- 172.21.15.33:0/2766045780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 0x7f2984106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 --2- 172.21.15.33:0/2766045780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f29841045f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 -- 172.21.15.33:0/2766045780 >> 172.21.15.33:0/2766045780 conn(0x7f29840fbbf0 msgr2=0x7f29840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 -- 172.21.15.33:0/2766045780 shutdown_connections 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.585+0000 7f298c7a2640 1 -- 172.21.15.33:0/2766045780 wait complete. 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 Processor -- start 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 -- start start 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f2984197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 0x7f2984198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2984107460 0x7f298419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f298410d450 con 0x7f2984102200 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f298410d2d0 con 0x7f2984104b30 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298c7a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f298410d5d0 con 0x7f2984107460 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298a53e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f2984197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298a53e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f2984197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41228/0 (socket says 172.21.15.33:41228) 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298a53e640 1 -- 172.21.15.33:0/578704040 learned_addr learned my addr 172.21.15.33:0/578704040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f2989d3d640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 0x7f2984198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.586+0000 7f298ad3f640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2984107460 0x7f298419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298a53e640 1 -- 172.21.15.33:0/578704040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2984107460 msgr2=0x7f298419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298a53e640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2984107460 0x7f298419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298a53e640 1 -- 172.21.15.33:0/578704040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 msgr2=0x7f2984198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298a53e640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 0x7f2984198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298a53e640 1 -- 172.21.15.33:0/578704040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298419f750 con 0x7f2984102200 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f2989d3d640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 0x7f2984198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298a53e640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f2984197b30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2980031440 tx=0x7f2980032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f297b7fe640 1 -- 172.21.15.33:0/578704040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298003f030 con 0x7f2984102200 2024-04-24T23:14:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298419f9e0 con 0x7f2984102200 2024-04-24T23:14:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f297b7fe640 1 -- 172.21.15.33:0/578704040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2980003ce0 con 0x7f2984102200 2024-04-24T23:14:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f297b7fe640 1 -- 172.21.15.33:0/578704040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980035070 con 0x7f2984102200 2024-04-24T23:14:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.587+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298419ff20 con 0x7f2984102200 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.589+0000 7f297b7fe640 1 -- 172.21.15.33:0/578704040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2980042030 con 0x7f2984102200 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 Processor -- start 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 -- start start 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855408a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x56285540db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855410640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5628553686b0 con 0x562855408650 2024-04-24T23:14:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562855310880 con 0x56285540e1b0 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298f594740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562855364670 con 0x562855409040 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f297b7fe640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29700a2b90 0x7f29700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298ad3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855408a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298ad3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855408a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41238/0 (socket says 172.21.15.33:41238) 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298ad3f640 1 -- 172.21.15.33:0/2237146470 learned_addr learned my addr 172.21.15.33:0/2237146470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f2989d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855410640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298ad3f640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29700a2b90 0x7f29700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f297b7fe640 1 -- 172.21.15.33:0/578704040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f29800ef960 con 0x7f2984102200 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.590+0000 7f298a53e640 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x56285540db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2989d3d640 1 -- 172.21.15.33:0/2237146470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 msgr2=0x56285540db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2989d3d640 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x56285540db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2989d3d640 1 -- 172.21.15.33:0/2237146470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 msgr2=0x562855408a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2989d3d640 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855408a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2989d3d640 1 -- 172.21.15.33:0/2237146470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628551467d0 con 0x56285540e1b0 2024-04-24T23:14:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f298ad3f640 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855408a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f298ad3f640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29700a2b90 0x7f29700a5050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5628554098a0 tx=0x7f297c0096a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2989d3d640 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855410640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2984069430 tx=0x7f29740118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0714d65fb5f5a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2978ff9640 1 -- 172.21.15.33:0/2237146470 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974004030 con 0x56285540e1b0 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2978ff9640 1 -- 172.21.15.33:0/2237146470 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2974005500 con 0x56285540e1b0 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.591+0000 7f2978ff9640 1 -- 172.21.15.33:0/2237146470 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974004cc0 con 0x56285540e1b0 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 -- 172.21.15.33:0/2237146470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 msgr2=0x562855410640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855410640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2984069430 tx=0x7f29740118d0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 -- 172.21.15.33:0/2237146470 shutdown_connections 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855410640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x56285540db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 --2- 172.21.15.33:0/2237146470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855408a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 -- 172.21.15.33:0/2237146470 >> 172.21.15.33:0/2237146470 conn(0x5628553ff8b0 msgr2=0x5628553ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 -- 172.21.15.33:0/2237146470 shutdown_connections 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 -- 172.21.15.33:0/2237146470 wait complete. 2024-04-24T23:14:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 -- start start 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.592+0000 7f298f594740 1 Processor -- start 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855412a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x562855412f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855419fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562855411160 con 0x562855408650 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562855323f50 con 0x56285540e1b0 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5628554112e0 con 0x562855409040 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 10 client.0 _ll_register_callbacks cb 0x56285535f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855412a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855412a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41244/0 (socket says 172.21.15.33:41244) 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 -- 172.21.15.33:0/1515646750 learned_addr learned my addr 172.21.15.33:0/1515646750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298ad3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855419fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x562855412f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 -- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 msgr2=0x562855412f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x562855412f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 -- 172.21.15.33:0/1515646750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 msgr2=0x562855419fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855419fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 -- 172.21.15.33:0/1515646750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285507c3f0 con 0x562855408650 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298a53e640 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855412a20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2980000c00 tx=0x7f29800bba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6e041683e82499 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f296dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f296dffb640 1 -- 172.21.15.33:0/1515646750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980049040 con 0x562855408650 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f296dffb640 1 -- 172.21.15.33:0/1515646750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2980003800 con 0x562855408650 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2974005500 con 0x562855408650 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f296dffb640 1 -- 172.21.15.33:0/1515646750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f29800bbe80 con 0x562855408650 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.593+0000 7f298f594740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56285503c0f0 con 0x562855408650 2024-04-24T23:14:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f296dffb640 1 -- 172.21.15.33:0/1515646750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2974005500 con 0x562855408650 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f296dffb640 1 -- 172.21.15.33:0/1515646750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f29800d1070 con 0x562855408650 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 20 client.15960 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56285541e8b0 0x562855420d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 4 client.15960 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562855375bf0 con 0x56285541e8b0 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56285541e8b0 0x562855420d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56285541e8b0 0x562855420d20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8775a7ba5d0872 server_cookie=9fb1510de5be0765 in_seq=0 out_seq=0 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.594+0000 7f296dffb640 10 client.15960 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f296dffb640 1 -- 172.21.15.33:0/1515646750 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562855375bf0 con 0x56285541e8b0 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f296dffb640 10 client.15960 handle_command_reply: tid=0 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 client.15960 shutdown 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 msgr2=0x562855412a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855412a20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2980000c00 tx=0x7f29800bba30 comp rx=0 tx=0).stop 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 shutdown_connections 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56285541e8b0 0x562855420d20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56285540e1b0 0x562855419fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562855409040 0x562855412f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 --2- 172.21.15.33:0/1515646750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855412a20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 >> 172.21.15.33:0/1515646750 conn(0x5628553ff8b0 msgr2=0x5628553feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.595+0000 7f296dffb640 0 client.15960 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.596+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 shutdown_connections 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.596+0000 7f298f594740 1 -- 172.21.15.33:0/1515646750 wait complete. 2024-04-24T23:14:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.596+0000 7f298f594740 20 client.15960 trim_cache size 0 max 16384 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 Processor -- start 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 -- start start 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855408650 0x562855412630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 0x562855419f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855412b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29841aaa40 con 0x562855409040 2024-04-24T23:14:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29841a8310 con 0x562855408650 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f298f594740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29841a6160 con 0x56285540e1b0 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f2989d3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 0x562855419f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f2989d3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 0x562855419f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41258/0 (socket says 172.21.15.33:41258) 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.613+0000 7f2989d3d640 1 -- 172.21.15.33:0/1242035231 learned_addr learned my addr 172.21.15.33:0/1242035231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f298a53e640 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855408650 0x562855412630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f298ad3f640 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855412b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f2989d3d640 1 -- 172.21.15.33:0/1242035231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 msgr2=0x562855412b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855412b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f2989d3d640 1 -- 172.21.15.33:0/1242035231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855408650 msgr2=0x562855412630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855408650 0x562855412630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f2989d3d640 1 -- 172.21.15.33:0/1242035231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628553ff450 con 0x562855409040 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f298ad3f640 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855412b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 0x562855419f60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2974009230 tx=0x7f2974005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eb567151457181 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f296ffff640 1 -- 172.21.15.33:0/1242035231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297402cbd0 con 0x562855409040 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f296ffff640 1 -- 172.21.15.33:0/1242035231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297402cd70 con 0x562855409040 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.614+0000 7f296ffff640 1 -- 172.21.15.33:0/1242035231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297400fa10 con 0x562855409040 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- 172.21.15.33:0/1242035231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 msgr2=0x562855419f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 0x562855419f60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2974009230 tx=0x7f2974005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- 172.21.15.33:0/1242035231 shutdown_connections 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855412b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855409040 0x562855419f60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- 172.21.15.33:0/1242035231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855408650 0x562855412630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- 172.21.15.33:0/1242035231 >> 172.21.15.33:0/1242035231 conn(0x5628553ff8b0 msgr2=0x56285540a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- 172.21.15.33:0/1242035231 shutdown_connections 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- 172.21.15.33:0/1242035231 wait complete. 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- start start 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 Processor -- start 2024-04-24T23:14:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855419cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855409040 0x562855416770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855416cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29841a67d0 con 0x562855408650 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29841a7c40 con 0x562855409040 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.615+0000 7f298f594740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29841a6ea0 con 0x56285540e1b0 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298f594740 10 client.0 _ll_register_callbacks cb 0x7f29800bbd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855419cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855419cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41274/0 (socket says 172.21.15.33:41274) 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 -- 172.21.15.33:0/1305885386 learned_addr learned my addr 172.21.15.33:0/1305885386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855409040 0x562855416770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298ad3f640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855416cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 -- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 msgr2=0x562855416cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855416cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 -- 172.21.15.33:0/1305885386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855409040 msgr2=0x562855416770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855409040 0x562855416770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 -- 172.21.15.33:0/1305885386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562855417350 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298ad3f640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855416cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298a53e640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855419cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f298003b650 tx=0x7f29800092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209d1150cd43d785 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f296dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f296dffb640 1 -- 172.21.15.33:0/1305885386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980049040 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f296dffb640 1 -- 172.21.15.33:0/1305885386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f29800b84f0 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f297402cd70 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f296dffb640 1 -- 172.21.15.33:0/1305885386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f29800bb7e0 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298f594740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.616+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562855484040 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f296dffb640 1 -- 172.21.15.33:0/1305885386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297402cd70 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f296dffb640 1 -- 172.21.15.33:0/1305885386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f29800d1070 con 0x562855408650 2024-04-24T23:14:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 20 client.15966 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628554848b0 0x562855486cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 4 client.15966 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56285536ff40 con 0x5628554848b0 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628554848b0 0x562855486cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f296dffb640 10 client.15966 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f2989d3d640 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628554848b0 0x562855486cd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169453856d9dcde7 server_cookie=586c6745082f8a3e in_seq=0 out_seq=0 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f296dffb640 1 -- 172.21.15.33:0/1305885386 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56285536ff40 con 0x5628554848b0 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f296dffb640 10 client.15966 handle_command_reply: tid=0 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.617+0000 7f298f594740 1 client.15966 shutdown 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 msgr2=0x562855419cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855419cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f298003b650 tx=0x7f29800092d0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 shutdown_connections 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628554848b0 0x562855486cd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56285540e1b0 0x562855416cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562855409040 0x562855416770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f296dffb640 0 client.15966 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 --2- 172.21.15.33:0/1305885386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562855408650 0x562855419cd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 >> 172.21.15.33:0/1305885386 conn(0x5628553ff8b0 msgr2=0x562855412630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 shutdown_connections 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 1 -- 172.21.15.33:0/1305885386 wait complete. 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298f594740 20 client.15966 trim_cache size 0 max 16384 2024-04-24T23:14:20.626 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: "01af613d-45c3-47ad-ae07-3556239f6e94": { 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "01af613d-45c3-47ad-ae07-3556239f6e94", 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:14:20.627 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29700a2b90 msgr2=0x7f29700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29700a2b90 0x7f29700a5050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5628554098a0 tx=0x7f297c0096a0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 msgr2=0x7f2984197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f2984197b30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2980031440 tx=0x7f2980032ce0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 shutdown_connections 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29700a2b90 0x7f29700a5050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2984107460 0x7f298419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2984104b30 0x7f2984198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 --2- 172.21.15.33:0/578704040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2984102200 0x7f2984197b30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 >> 172.21.15.33:0/578704040 conn(0x7f29840fbbf0 msgr2=0x7f2984102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 shutdown_connections 2024-04-24T23:14:20.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:20.618+0000 7f298c7a2640 1 -- 172.21.15.33:0/578704040 wait complete. 2024-04-24T23:14:20.631 INFO:tasks.cephfs.filesystem:scrub status for tag:01af613d-45c3-47ad-ae07-3556239f6e94 - {'path': '/', 'tag': '01af613d-45c3-47ad-ae07-3556239f6e94', 'options': 'recursive,force'} 2024-04-24T23:14:21.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:20 smithi038 ceph-mon[32450]: pgmap v421: 129 pgs: 129 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 3.2 MiB/s wr, 221 op/s 2024-04-24T23:14:21.188 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:20 smithi117 ceph-mon[32529]: pgmap v421: 129 pgs: 129 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 3.2 MiB/s wr, 221 op/s 2024-04-24T23:14:21.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:20 smithi033 ceph-mon[29201]: pgmap v421: 129 pgs: 129 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 3.2 MiB/s wr, 221 op/s 2024-04-24T23:14:21.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 69266 14.99 MB/sec warmup 26 sec latency 17.273 ms 2024-04-24T23:14:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:21 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:14:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:21 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:21 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:14:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:21 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:14:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:21 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:21 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:21 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:14:22.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:21 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:21 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:14:22.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:21 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:14:22.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:21 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:21 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:21 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:14:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:21 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:21 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:14:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:21 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:14:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:21 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:21 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:22.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71830 14.96 MB/sec warmup 27 sec latency 14.945 ms 2024-04-24T23:14:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:22 smithi038 ceph-mon[32450]: pgmap v422: 129 pgs: 129 active+clean; 43 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 372 op/s 2024-04-24T23:14:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:22 smithi117 ceph-mon[32529]: pgmap v422: 129 pgs: 129 active+clean; 43 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 372 op/s 2024-04-24T23:14:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:22 smithi033 ceph-mon[29201]: pgmap v422: 129 pgs: 129 active+clean; 43 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 372 op/s 2024-04-24T23:14:23.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 74916 15.09 MB/sec warmup 28 sec latency 16.693 ms 2024-04-24T23:14:24.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 77884 15.07 MB/sec warmup 29 sec latency 13.501 ms 2024-04-24T23:14:25.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:24 smithi038 ceph-mon[32450]: pgmap v423: 129 pgs: 129 active+clean; 43 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 292 op/s 2024-04-24T23:14:25.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:24 smithi117 ceph-mon[32529]: pgmap v423: 129 pgs: 129 active+clean; 43 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 292 op/s 2024-04-24T23:14:25.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:24 smithi033 ceph-mon[29201]: pgmap v423: 129 pgs: 129 active+clean; 43 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 292 op/s 2024-04-24T23:14:25.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 79790 14.90 MB/sec warmup 30 sec latency 27.862 ms 2024-04-24T23:14:26.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 82380 14.88 MB/sec warmup 31 sec latency 13.062 ms 2024-04-24T23:14:27.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:26 smithi038 ceph-mon[32450]: pgmap v424: 129 pgs: 129 active+clean; 46 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 328 op/s 2024-04-24T23:14:27.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:26 smithi117 ceph-mon[32529]: pgmap v424: 129 pgs: 129 active+clean; 46 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 328 op/s 2024-04-24T23:14:27.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:26 smithi033 ceph-mon[29201]: pgmap v424: 129 pgs: 129 active+clean; 46 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 328 op/s 2024-04-24T23:14:27.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 85421 14.99 MB/sec warmup 32 sec latency 13.509 ms 2024-04-24T23:14:28.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:27 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:14:28.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:27 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:14:28.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:27 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:14:28.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87780 14.79 MB/sec warmup 33 sec latency 32.385 ms 2024-04-24T23:14:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:28 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:14:29.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:28 smithi117 ceph-mon[32529]: pgmap v425: 129 pgs: 129 active+clean; 50 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 383 op/s 2024-04-24T23:14:29.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:28 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:14:29.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:28 smithi033 ceph-mon[29201]: pgmap v425: 129 pgs: 129 active+clean; 50 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 383 op/s 2024-04-24T23:14:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:28 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:14:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:28 smithi038 ceph-mon[32450]: pgmap v425: 129 pgs: 129 active+clean; 50 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 383 op/s 2024-04-24T23:14:29.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89944 14.74 MB/sec warmup 34 sec latency 14.351 ms 2024-04-24T23:14:30.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 92638 14.81 MB/sec warmup 35 sec latency 98.123 ms 2024-04-24T23:14:31.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:30 smithi033 ceph-mon[29201]: pgmap v426: 129 pgs: 129 active+clean; 50 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 331 op/s 2024-04-24T23:14:31.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:30 smithi038 ceph-mon[32450]: pgmap v426: 129 pgs: 129 active+clean; 50 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 331 op/s 2024-04-24T23:14:31.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:30 smithi117 ceph-mon[32529]: pgmap v426: 129 pgs: 129 active+clean; 50 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 331 op/s 2024-04-24T23:14:31.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 95249 14.74 MB/sec warmup 36 sec latency 13.927 ms 2024-04-24T23:14:32.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 97054 14.58 MB/sec warmup 37 sec latency 22.980 ms 2024-04-24T23:14:33.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:32 smithi033 ceph-mon[29201]: pgmap v427: 129 pgs: 129 active+clean; 54 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-04-24T23:14:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:32 smithi038 ceph-mon[32450]: pgmap v427: 129 pgs: 129 active+clean; 54 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-04-24T23:14:33.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:32 smithi117 ceph-mon[32529]: pgmap v427: 129 pgs: 129 active+clean; 54 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-04-24T23:14:33.435 INFO:tasks.workunit.client.0.smithi033.stdout: 1 99902 14.64 MB/sec warmup 38 sec latency 8.814 ms 2024-04-24T23:14:34.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 102747 14.64 MB/sec warmup 39 sec latency 19.609 ms 2024-04-24T23:14:35.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:34 smithi033 ceph-mon[29201]: pgmap v428: 129 pgs: 129 active+clean; 55 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-04-24T23:14:35.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:34 smithi038 ceph-mon[32450]: pgmap v428: 129 pgs: 129 active+clean; 55 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-04-24T23:14:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:34 smithi117 ceph-mon[32529]: pgmap v428: 129 pgs: 129 active+clean; 55 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-04-24T23:14:35.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 104614 14.52 MB/sec warmup 40 sec latency 19.636 ms 2024-04-24T23:14:36.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 107202 14.51 MB/sec warmup 41 sec latency 17.743 ms 2024-04-24T23:14:37.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:36 smithi033 ceph-mon[29201]: pgmap v429: 129 pgs: 129 active+clean; 59 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-04-24T23:14:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:36 smithi038 ceph-mon[32450]: pgmap v429: 129 pgs: 129 active+clean; 59 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-04-24T23:14:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:36 smithi117 ceph-mon[32529]: pgmap v429: 129 pgs: 129 active+clean; 59 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-04-24T23:14:37.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 110289 14.61 MB/sec warmup 42 sec latency 14.217 ms 2024-04-24T23:14:38.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 112918 14.57 MB/sec warmup 43 sec latency 23.108 ms 2024-04-24T23:14:39.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:38 smithi033 ceph-mon[29201]: pgmap v430: 129 pgs: 129 active+clean; 61 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-04-24T23:14:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:38 smithi038 ceph-mon[32450]: pgmap v430: 129 pgs: 129 active+clean; 61 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-04-24T23:14:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:38 smithi117 ceph-mon[32529]: pgmap v430: 129 pgs: 129 active+clean; 61 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-04-24T23:14:39.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 114765 14.43 MB/sec warmup 44 sec latency 24.302 ms 2024-04-24T23:14:40.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 117698 14.49 MB/sec warmup 45 sec latency 8.780 ms 2024-04-24T23:14:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:40 smithi033 ceph-mon[29201]: pgmap v431: 129 pgs: 129 active+clean; 61 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-24T23:14:41.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:40 smithi038 ceph-mon[32450]: pgmap v431: 129 pgs: 129 active+clean; 61 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-24T23:14:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:40 smithi117 ceph-mon[32529]: pgmap v431: 129 pgs: 129 active+clean; 61 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-24T23:14:41.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 120724 14.56 MB/sec warmup 46 sec latency 17.573 ms 2024-04-24T23:14:42.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 123074 14.43 MB/sec warmup 47 sec latency 16.215 ms 2024-04-24T23:14:43.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:42 smithi033 ceph-mon[29201]: pgmap v432: 129 pgs: 129 active+clean; 65 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-24T23:14:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:42 smithi038 ceph-mon[32450]: pgmap v432: 129 pgs: 129 active+clean; 65 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-24T23:14:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:42 smithi117 ceph-mon[32529]: pgmap v432: 129 pgs: 129 active+clean; 65 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-24T23:14:43.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 125242 14.42 MB/sec warmup 48 sec latency 12.272 ms 2024-04-24T23:14:44.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 128119 14.48 MB/sec warmup 49 sec latency 12.624 ms 2024-04-24T23:14:45.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:44 smithi033 ceph-mon[29201]: pgmap v433: 129 pgs: 129 active+clean; 69 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-04-24T23:14:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:44 smithi038 ceph-mon[32450]: pgmap v433: 129 pgs: 129 active+clean; 69 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-04-24T23:14:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:44 smithi117 ceph-mon[32529]: pgmap v433: 129 pgs: 129 active+clean; 69 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-04-24T23:14:45.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 131077 14.48 MB/sec warmup 50 sec latency 12.269 ms 2024-04-24T23:14:46.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 132880 14.39 MB/sec warmup 51 sec latency 18.006 ms 2024-04-24T23:14:47.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:46 smithi033 ceph-mon[29201]: pgmap v434: 129 pgs: 129 active+clean; 70 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-24T23:14:47.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:46 smithi038 ceph-mon[32450]: pgmap v434: 129 pgs: 129 active+clean; 70 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-24T23:14:47.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:46 smithi117 ceph-mon[32529]: pgmap v434: 129 pgs: 129 active+clean; 70 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-24T23:14:47.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 135487 14.38 MB/sec warmup 52 sec latency 12.618 ms 2024-04-24T23:14:48.436 INFO:tasks.workunit.client.0.smithi033.stdout: 1 138487 14.44 MB/sec warmup 53 sec latency 15.040 ms 2024-04-24T23:14:49.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:48 smithi033 ceph-mon[29201]: pgmap v435: 129 pgs: 129 active+clean; 70 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-24T23:14:49.228 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:48 smithi117 ceph-mon[32529]: pgmap v435: 129 pgs: 129 active+clean; 70 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-24T23:14:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:48 smithi038 ceph-mon[32450]: pgmap v435: 129 pgs: 129 active+clean; 70 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-24T23:14:49.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 140775 14.33 MB/sec warmup 54 sec latency 15.202 ms 2024-04-24T23:14:50.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 142778 14.30 MB/sec warmup 55 sec latency 34.951 ms 2024-04-24T23:14:50.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:14:50.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 -- 172.21.15.33:0/492514476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810eb00 msgr2=0x7fa108110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 --2- 172.21.15.33:0/492514476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810eb00 0x7fa108110fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa100009be0 tx=0x7fa10002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 -- 172.21.15.33:0/492514476 shutdown_connections 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 --2- 172.21.15.33:0/492514476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810eb00 0x7fa108110fc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 --2- 172.21.15.33:0/492514476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10810c180 0x7fa10810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 --2- 172.21.15.33:0/492514476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa108109850 0x7fa10810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 -- 172.21.15.33:0/492514476 >> 172.21.15.33:0/492514476 conn(0x7fa10806e5d0 msgr2=0x7fa108070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 -- 172.21.15.33:0/492514476 shutdown_connections 2024-04-24T23:14:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.750+0000 7fa10d293640 1 -- 172.21.15.33:0/492514476 wait complete. 2024-04-24T23:14:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 Processor -- start 2024-04-24T23:14:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 -- start start 2024-04-24T23:14:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 0x7fa1081a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810c180 0x7fa1081a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10810eb00 0x7fa1081ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa108113a00 con 0x7fa108109850 2024-04-24T23:14:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa108113880 con 0x7fa10810eb00 2024-04-24T23:14:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa10d293640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa108113b80 con 0x7fa10810c180 2024-04-24T23:14:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810c180 0x7fa1081a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.751+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 0x7fa1081a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 0x7fa1081a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41168/0 (socket says 172.21.15.33:41168) 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 -- 172.21.15.33:0/3844137527 learned_addr learned my addr 172.21.15.33:0/3844137527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa1077fe640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10810eb00 0x7fa1081ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 -- 172.21.15.33:0/3844137527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810c180 msgr2=0x7fa1081a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810c180 0x7fa1081a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 -- 172.21.15.33:0/3844137527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10810eb00 msgr2=0x7fa1081ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10810eb00 0x7fa1081ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 -- 172.21.15.33:0/3844137527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa108119270 con 0x7fa108109850 2024-04-24T23:14:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.752+0000 7fa106ffd640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 0x7fa1081a68f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00e9f0 tx=0x7fa0fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.753+0000 7fa0effff640 1 -- 172.21.15.33:0/3844137527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00ccc0 con 0x7fa108109850 2024-04-24T23:14:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.753+0000 7fa0effff640 1 -- 172.21.15.33:0/3844137527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc010910 con 0x7fa108109850 2024-04-24T23:14:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.753+0000 7fa0effff640 1 -- 172.21.15.33:0/3844137527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc018960 con 0x7fa108109850 2024-04-24T23:14:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.753+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa108119560 con 0x7fa108109850 2024-04-24T23:14:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.753+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa108119a60 con 0x7fa108109850 2024-04-24T23:14:50.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa0effff640 1 -- 172.21.15.33:0/3844137527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa0fc010ac0 con 0x7fa108109850 2024-04-24T23:14:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 Processor -- start 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 -- start start 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 0x555ad57ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57abc30 0x555ad57b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ad570f6b0 con 0x555ad57ab240 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555ad56b7880 con 0x555ad57abc30 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.757+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ad570b670 con 0x555ad57b0da0 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 0x555ad57ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57abc30 0x555ad57b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa0effff640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0f00a2d60 0x7fa0f00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 0x555ad57ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41184/0 (socket says 172.21.15.33:41184) 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1077fe640 1 -- 172.21.15.33:0/1127555125 learned_addr learned my addr 172.21.15.33:0/1127555125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1077fe640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0f00a2d60 0x7fa0f00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa0effff640 1 -- 172.21.15.33:0/3844137527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0fc0cf710 con 0x7fa108109850 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1067fc640 1 -- 172.21.15.33:0/1127555125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57abc30 msgr2=0x555ad57b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57abc30 0x555ad57b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1067fc640 1 -- 172.21.15.33:0/1127555125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 msgr2=0x555ad57ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 0x555ad57ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1067fc640 1 -- 172.21.15.33:0/1127555125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad54ed7d0 con 0x555ad57b0da0 2024-04-24T23:14:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.758+0000 7fa1077fe640 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 0x555ad57ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.759+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b3230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009c20 tx=0x7fa0f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3014cfde0c6364ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.760+0000 7fa1077fe640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0f00a2d60 0x7fa0f00a5220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa100004af0 tx=0x7fa100009280 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.760+0000 7fa0ed7fa640 1 -- 172.21.15.33:0/1127555125 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004050 con 0x555ad57b0da0 2024-04-24T23:14:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.761+0000 7fa0ed7fa640 1 -- 172.21.15.33:0/1127555125 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8005500 con 0x555ad57b0da0 2024-04-24T23:14:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.761+0000 7fa0ed7fa640 1 -- 172.21.15.33:0/1127555125 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004db0 con 0x555ad57b0da0 2024-04-24T23:14:50.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1127555125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 msgr2=0x555ad57b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b3230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009c20 tx=0x7fa0f8011810 comp rx=0 tx=0).stop 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1127555125 shutdown_connections 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b3230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57abc30 0x555ad57b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1127555125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57ab240 0x555ad57ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.762+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1127555125 >> 172.21.15.33:0/1127555125 conn(0x555ad57a68b0 msgr2=0x555ad57a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.763+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1127555125 shutdown_connections 2024-04-24T23:14:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.763+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1127555125 wait complete. 2024-04-24T23:14:50.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.763+0000 7fa10ff9f740 1 -- start start 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.763+0000 7fa10ff9f740 1 Processor -- start 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ad57b3d50 con 0x555ad57abc30 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555ad56caf50 con 0x555ad57ab240 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ad57b3ed0 con 0x555ad57b0da0 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49486/0 (socket says 172.21.15.33:49486) 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.764+0000 7fa106ffd640 1 -- 172.21.15.33:0/491878364 learned_addr learned my addr 172.21.15.33:0/491878364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa10ff9f740 10 client.0 _ll_register_callbacks cb 0x555ad5706000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa1077fe640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa106ffd640 1 -- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 msgr2=0x555ad57bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa106ffd640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa106ffd640 1 -- 172.21.15.33:0/491878364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 msgr2=0x555ad57b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa106ffd640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.765+0000 7fa106ffd640 1 -- 172.21.15.33:0/491878364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad54233f0 con 0x555ad57ab240 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa1077fe640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa1067fc640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa106ffd640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b56f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc097f50 tx=0x7fa0fc09b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff665466b0c8f2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa0eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa0eaff5640 1 -- 172.21.15.33:0/491878364 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc026040 con 0x555ad57ab240 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa0eaff5640 1 -- 172.21.15.33:0/491878364 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0fc09b1d0 con 0x555ad57ab240 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.766+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f8005500 con 0x555ad57ab240 2024-04-24T23:14:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa0eaff5640 1 -- 172.21.15.33:0/491878364 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa0fc0a47f0 con 0x555ad57ab240 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 10 client.25517 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ad53e30f0 con 0x555ad57ab240 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa0eaff5640 1 -- 172.21.15.33:0/491878364 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc09bb80 con 0x555ad57ab240 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa0eaff5640 1 -- 172.21.15.33:0/491878364 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa0fc0978d0 con 0x555ad57ab240 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 10 client.25517 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 20 client.25517 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad57c1490 0x555ad57c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 4 client.25517 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.767+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0f8005500 con 0x555ad57c1490 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.768+0000 7fa1067fc640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad57c1490 0x555ad57c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.768+0000 7fa0eaff5640 10 client.25517 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.769+0000 7fa1067fc640 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad57c1490 0x555ad57c38e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed141ae15194ef01 server_cookie=bfd86f0235613033 in_seq=0 out_seq=0 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa0eaff5640 1 -- 172.21.15.33:0/491878364 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa0f8005500 con 0x555ad57c1490 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa0eaff5640 10 client.25517 handle_command_reply: tid=0 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 client.25517 shutdown 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 msgr2=0x555ad57b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b56f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc097f50 tx=0x7fa0fc09b780 comp rx=0 tx=0).stop 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 shutdown_connections 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad57c1490 0x555ad57c38e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa0eaff5640 0 client.25517 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/491878364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b56f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.770+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 >> 172.21.15.33:0/491878364 conn(0x555ad57a68b0 msgr2=0x555ad57a5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.771+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 shutdown_connections 2024-04-24T23:14:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.771+0000 7fa10ff9f740 1 -- 172.21.15.33:0/491878364 wait complete. 2024-04-24T23:14:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.771+0000 7fa10ff9f740 20 client.25517 trim_cache size 0 max 16384 2024-04-24T23:14:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 Processor -- start 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 -- start start 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081c4360 con 0x555ad57abc30 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081c1c30 con 0x555ad57ab240 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa108121720 con 0x555ad57b0da0 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57bcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41194/0 (socket says 172.21.15.33:41194) 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa1067fc640 1 -- 172.21.15.33:0/380864613 learned_addr learned my addr 172.21.15.33:0/380864613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa1077fe640 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.795+0000 7fa106ffd640 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa1067fc640 1 -- 172.21.15.33:0/380864613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 msgr2=0x555ad57b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa1067fc640 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa1067fc640 1 -- 172.21.15.33:0/380864613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 msgr2=0x555ad57ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa1067fc640 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa1067fc640 1 -- 172.21.15.33:0/380864613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad57ad780 con 0x555ad57abc30 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa1067fc640 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57bcc30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009b40 tx=0x7fa0f80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff0d2839756d8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa0ecff9640 1 -- 172.21.15.33:0/380864613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004360 con 0x555ad57abc30 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa0ecff9640 1 -- 172.21.15.33:0/380864613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8033070 con 0x555ad57abc30 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.796+0000 7fa0ecff9640 1 -- 172.21.15.33:0/380864613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8003850 con 0x555ad57abc30 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 -- 172.21.15.33:0/380864613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 msgr2=0x555ad57bcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57bcc30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009b40 tx=0x7fa0f80096c0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 -- 172.21.15.33:0/380864613 shutdown_connections 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57bcc30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/380864613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 -- 172.21.15.33:0/380864613 >> 172.21.15.33:0/380864613 conn(0x555ad57a68b0 msgr2=0x555ad57b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 -- 172.21.15.33:0/380864613 shutdown_connections 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 -- 172.21.15.33:0/380864613 wait complete. 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.797+0000 7fa10ff9f740 1 -- start start 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 Processor -- start 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081c00f0 con 0x555ad57abc30 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081c1560 con 0x555ad57ab240 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081c07c0 con 0x555ad57b0da0 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa10ff9f740 10 client.0 _ll_register_callbacks cb 0x7fa0fc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:50.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57ba380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41198/0 (socket says 172.21.15.33:41198) 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa1067fc640 1 -- 172.21.15.33:0/1910485554 learned_addr learned my addr 172.21.15.33:0/1910485554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa1077fe640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 -- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 msgr2=0x555ad57bb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 -- 172.21.15.33:0/1910485554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 msgr2=0x555ad57ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 -- 172.21.15.33:0/1910485554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad57bb980 con 0x555ad57ab240 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:50.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa106ffd640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b9e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c6a0 tx=0x7fa0fc0cfa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c37f2c2a98fcea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa0eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.798+0000 7fa0eaff5640 1 -- 172.21.15.33:0/1910485554 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0b9060 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f8033070 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa0eaff5640 1 -- 172.21.15.33:0/1910485554 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0fc09b1e0 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa0eaff5640 1 -- 172.21.15.33:0/1910485554 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa0fc0cfe60 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 10 client.25523 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ad57bcd00 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa0eaff5640 1 -- 172.21.15.33:0/1910485554 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8033070 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa0eaff5640 1 -- 172.21.15.33:0/1910485554 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa0fc0ba370 con 0x555ad57ab240 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 10 client.25523 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 20 client.25523 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad5827d40 0x555ad582a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 4 client.25523 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:14:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ad571cbf0 con 0x555ad5827d40 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.799+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad5827d40 0x555ad582a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa1067fc640 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad5827d40 0x555ad582a160 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c699dfd8261250b server_cookie=c3ba05a7c4adb22c in_seq=0 out_seq=0 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa0eaff5640 10 client.25523 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa0eaff5640 1 -- 172.21.15.33:0/1910485554 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ad571cbf0 con 0x555ad5827d40 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa0eaff5640 10 client.25523 handle_command_reply: tid=0 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 client.25523 shutdown 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 msgr2=0x555ad57b9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b9e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c6a0 tx=0x7fa0fc0cfa80 comp rx=0 tx=0).stop 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 shutdown_connections 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555ad5827d40 0x555ad582a160 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ad57b0da0 0x555ad57bb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ad57abc30 0x555ad57ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa0eaff5640 0 client.25523 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 --2- 172.21.15.33:0/1910485554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ad57ab240 0x555ad57b9e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 >> 172.21.15.33:0/1910485554 conn(0x555ad57a68b0 msgr2=0x555ad57a9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 shutdown_connections 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 1 -- 172.21.15.33:0/1910485554 wait complete. 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.800+0000 7fa10ff9f740 20 client.25523 trim_cache size 0 max 16384 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:14:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:14:50.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0f00a2d60 msgr2=0x7fa0f00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0f00a2d60 0x7fa0f00a5220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa100004af0 tx=0x7fa100009280 comp rx=0 tx=0).stop 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 msgr2=0x7fa1081a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 0x7fa1081a68f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00e9f0 tx=0x7fa0fc00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 shutdown_connections 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa0f00a2d60 0x7fa0f00a5220 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10810eb00 0x7fa1081ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10810c180 0x7fa1081a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 --2- 172.21.15.33:0/3844137527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109850 0x7fa1081a68f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 >> 172.21.15.33:0/3844137527 conn(0x7fa10806e5d0 msgr2=0x7fa10810ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 shutdown_connections 2024-04-24T23:14:50.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.801+0000 7fa10d293640 1 -- 172.21.15.33:0/3844137527 wait complete. 2024-04-24T23:14:50.818 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:14:50.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 -- 172.21.15.33:0/2741063439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc1018b0 msgr2=0x7f40cc101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 --2- 172.21.15.33:0/2741063439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc1018b0 0x7f40cc101d30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009900 tx=0x7f40c002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 -- 172.21.15.33:0/2741063439 shutdown_connections 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 --2- 172.21.15.33:0/2741063439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40cc102270 0x7f40cc108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 --2- 172.21.15.33:0/2741063439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc1018b0 0x7f40cc101d30 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 --2- 172.21.15.33:0/2741063439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 0x7f40cc100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 -- 172.21.15.33:0/2741063439 >> 172.21.15.33:0/2741063439 conn(0x7f40cc0fbc30 msgr2=0x7f40cc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 -- 172.21.15.33:0/2741063439 shutdown_connections 2024-04-24T23:14:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.929+0000 7f40d224d640 1 -- 172.21.15.33:0/2741063439 wait complete. 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 Processor -- start 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 -- start start 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 0x7f40cc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40cc1018b0 0x7f40cc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 0x7f40cc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10c340 con 0x7f40cc102270 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10c1c0 con 0x7f40cc1018b0 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40d224d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10c4c0 con 0x7f40cc1006c0 2024-04-24T23:14:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 0x7f40cc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 0x7f40cc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 0x7f40cc19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41218/0 (socket says 172.21.15.33:41218) 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40cc1018b0 0x7f40cc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 -- 172.21.15.33:0/3588364140 learned_addr learned my addr 172.21.15.33:0/3588364140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 -- 172.21.15.33:0/3588364140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 msgr2=0x7f40cc197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 0x7f40cc197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 -- 172.21.15.33:0/3588364140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40cc1018b0 msgr2=0x7f40cc1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.930+0000 7f40cbfff640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40cc1018b0 0x7f40cc1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40cbfff640 1 -- 172.21.15.33:0/3588364140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc19f780 con 0x7f40cc102270 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40cb7fe640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 0x7f40cc197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40cbfff640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 0x7f40cc19f110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f40c400e9b0 tx=0x7f40c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c400cd80 con 0x7f40cc102270 2024-04-24T23:14:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc19fa70 con 0x7f40cc102270 2024-04-24T23:14:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c40035f0 con 0x7f40cc102270 2024-04-24T23:14:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4010670 con 0x7f40cc102270 2024-04-24T23:14:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.931+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc19ff90 con 0x7f40cc102270 2024-04-24T23:14:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.932+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b8005290 con 0x7f40cc102270 2024-04-24T23:14:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.933+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f40c4004020 con 0x7f40cc102270 2024-04-24T23:14:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.934+0000 7f40c8ff9640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40b40a2c10 0x7f40b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.934+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40c40cf5e0 con 0x7f40cc102270 2024-04-24T23:14:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.934+0000 7f40cb7fe640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40b40a2c10 0x7f40b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.934+0000 7f40cb7fe640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40b40a2c10 0x7f40b40a50d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f40cc101710 tx=0x7f40bc009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:50.939+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f40c409c150 con 0x7f40cc102270 2024-04-24T23:14:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.189+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40b8005580 con 0x7f40cc102270 2024-04-24T23:14:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.190+0000 7f40c8ff9640 1 -- 172.21.15.33:0/3588364140 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11860 (secure 0 0 0) 0x7f40c409ba50 con 0x7f40cc102270 2024-04-24T23:14:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:14:51.193 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:13:57.199404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:14:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 26 2024-04-24T23:14:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.192+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40b40a2c10 msgr2=0x7f40b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.192+0000 7f40d224d640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40b40a2c10 0x7f40b40a50d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f40cc101710 tx=0x7f40bc009500 comp rx=0 tx=0).stop 2024-04-24T23:14:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.192+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 msgr2=0x7f40cc19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.192+0000 7f40d224d640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 0x7f40cc19f110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f40c400e9b0 tx=0x7f40c400ee70 comp rx=0 tx=0).stop 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 shutdown_connections 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f40b40a2c10 0x7f40b40a50d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102270 0x7f40cc19f110 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40cc1018b0 0x7f40cc1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 --2- 172.21.15.33:0/3588364140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40cc1006c0 0x7f40cc197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 >> 172.21.15.33:0/3588364140 conn(0x7f40cc0fbc30 msgr2=0x7f40cc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 shutdown_connections 2024-04-24T23:14:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.193+0000 7f40d224d640 1 -- 172.21.15.33:0/3588364140 wait complete. 2024-04-24T23:14:51.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:50 smithi033 ceph-mon[29201]: pgmap v436: 129 pgs: 129 active+clean; 72 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-24T23:14:51.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:14:51.222 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:50 smithi038 ceph-mon[32450]: pgmap v436: 129 pgs: 129 active+clean; 72 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-24T23:14:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 -- 172.21.15.33:0/149427322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102250 msgr2=0x7f02b0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 --2- 172.21.15.33:0/149427322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102250 0x7f02b0108ae0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009ca0 tx=0x7f02a802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 -- 172.21.15.33:0/149427322 shutdown_connections 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 --2- 172.21.15.33:0/149427322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102250 0x7f02b0108ae0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 --2- 172.21.15.33:0/149427322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b0101890 0x7f02b0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 --2- 172.21.15.33:0/149427322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b01006a0 0x7f02b0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 -- 172.21.15.33:0/149427322 >> 172.21.15.33:0/149427322 conn(0x7f02b00fbc30 msgr2=0x7f02b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 -- 172.21.15.33:0/149427322 shutdown_connections 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.313+0000 7f02b5d77640 1 -- 172.21.15.33:0/149427322 wait complete. 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 Processor -- start 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 -- start start 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 0x7f02b0196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b0101890 0x7f02b0196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b0102250 0x7f02b019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010c300 con 0x7f02b01006a0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010c180 con 0x7f02b0101890 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02b5d77640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010c480 con 0x7f02b0102250 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 0x7f02b0196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 0x7f02b0196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41236/0 (socket says 172.21.15.33:41236) 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02af7fe640 1 -- 172.21.15.33:0/4260828928 learned_addr learned my addr 172.21.15.33:0/4260828928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02aeffd640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b0101890 0x7f02b0196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.314+0000 7f02affff640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b0102250 0x7f02b019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02af7fe640 1 -- 172.21.15.33:0/4260828928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b0102250 msgr2=0x7f02b019ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02af7fe640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b0102250 0x7f02b019ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02af7fe640 1 -- 172.21.15.33:0/4260828928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b0101890 msgr2=0x7f02b0196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02af7fe640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b0101890 0x7f02b0196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02af7fe640 1 -- 172.21.15.33:0/4260828928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b019e540 con 0x7f02b01006a0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02af7fe640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 0x7f02b0196920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9c0 tx=0x7f02a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02acff9640 1 -- 172.21.15.33:0/4260828928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000cd80 con 0x7f02b01006a0 2024-04-24T23:14:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b019e830 con 0x7f02b01006a0 2024-04-24T23:14:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02acff9640 1 -- 172.21.15.33:0/4260828928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a00035f0 con 0x7f02b01006a0 2024-04-24T23:14:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02acff9640 1 -- 172.21.15.33:0/4260828928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0010690 con 0x7f02b01006a0 2024-04-24T23:14:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.315+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b019ed80 con 0x7f02b01006a0 2024-04-24T23:14:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02acff9640 1 -- 172.21.15.33:0/4260828928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f02a00037a0 con 0x7f02b01006a0 2024-04-24T23:14:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 Processor -- start 2024-04-24T23:14:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 -- start start 2024-04-24T23:14:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3d440 0x557ad0c3d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c42930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c42fa0 0x557ad0c45430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ad0b9f670 con 0x557ad0c3de30 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ad0b47970 con 0x557ad0c42fa0 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02b8b66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ad0b9b630 con 0x557ad0c3d440 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c42930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c42930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41246/0 (socket says 172.21.15.33:41246) 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 -- 172.21.15.33:0/2897385802 learned_addr learned my addr 172.21.15.33:0/2897385802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02aeffd640 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c42fa0 0x557ad0c45430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02affff640 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3d440 0x557ad0c3d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02acff9640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02980a2b90 0x7f02980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 -- 172.21.15.33:0/2897385802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3d440 msgr2=0x557ad0c3d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3d440 0x557ad0c3d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 -- 172.21.15.33:0/2897385802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c42fa0 msgr2=0x557ad0c45430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c42fa0 0x557ad0c45430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02af7fe640 1 -- 172.21.15.33:0/2897385802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad0b6b400 con 0x557ad0c3de30 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02aeffd640 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c42fa0 0x557ad0c45430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.318+0000 7f02affff640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02980a2b90 0x7f02980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.319+0000 7f02acff9640 1 -- 172.21.15.33:0/4260828928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f02a00cf7b0 con 0x7f02b01006a0 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.319+0000 7f02af7fe640 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c42930 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f02a0077790 tx=0x7f02a0066900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229be4e2c730b7c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.319+0000 7f0295ffb640 1 -- 172.21.15.33:0/2897385802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a009ba50 con 0x557ad0c3de30 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.319+0000 7f0295ffb640 1 -- 172.21.15.33:0/2897385802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a0066970 con 0x557ad0c3de30 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.319+0000 7f0295ffb640 1 -- 172.21.15.33:0/2897385802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a009b5e0 con 0x557ad0c3de30 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.319+0000 7f02affff640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02980a2b90 0x7f02980a5050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005e30 tx=0x7f02a80095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 -- 172.21.15.33:0/2897385802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 msgr2=0x557ad0c42930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c42930 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f02a0077790 tx=0x7f02a0066900 comp rx=0 tx=0).stop 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 -- 172.21.15.33:0/2897385802 shutdown_connections 2024-04-24T23:14:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c42fa0 0x557ad0c45430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c42930 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 --2- 172.21.15.33:0/2897385802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3d440 0x557ad0c3d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 -- 172.21.15.33:0/2897385802 >> 172.21.15.33:0/2897385802 conn(0x557ad0c38ab0 msgr2=0x557ad0c38eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 -- 172.21.15.33:0/2897385802 shutdown_connections 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 -- 172.21.15.33:0/2897385802 wait complete. 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 -- start start 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.320+0000 7f02b8b66740 1 Processor -- start 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ad0c45ef0 con 0x557ad0c3de30 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557ad0b5b050 con 0x557ad0c3d440 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ad0c46070 con 0x557ad0c42fa0 2024-04-24T23:14:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02b8b66740 10 client.0 _ll_register_callbacks cb 0x557ad08b2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49582/0 (socket says 172.21.15.33:49582) 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02af7fe640 1 -- 172.21.15.33:0/1953979949 learned_addr learned my addr 172.21.15.33:0/1953979949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c47a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41254/0 (socket says 172.21.15.33:41254) 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02affff640 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 -- 172.21.15.33:0/1953979949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 msgr2=0x557ad0c4ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 -- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 msgr2=0x557ad0c474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.321+0000 7f02aeffd640 1 -- 172.21.15.33:0/1953979949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad0b2cdc0 con 0x557ad0c3de30 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02aeffd640 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c47a10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f02b0197e30 tx=0x7f02a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a0ad998f4b150d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f028effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f028effd640 1 -- 172.21.15.33:0/1953979949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4004020 con 0x557ad0c3de30 2024-04-24T23:14:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557ad0bacbb0 con 0x557ad0c3de30 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f028effd640 1 -- 172.21.15.33:0/1953979949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f02a40041c0 con 0x557ad0c3de30 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f028effd640 1 -- 172.21.15.33:0/1953979949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02a40043c0 con 0x557ad0c3de30 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 10 client.16002 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ad08b33f0 con 0x557ad0c3de30 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f028effd640 1 -- 172.21.15.33:0/1953979949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557ad0bacbb0 con 0x557ad0c3de30 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f028effd640 1 -- 172.21.15.33:0/1953979949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f02a4010700 con 0x557ad0c3de30 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 10 client.16002 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 10 client.16002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 20 client.16002 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0c53380 0x557ad0c55800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 4 client.16002 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.322+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ad0ba6f00 con 0x557ad0c53380 2024-04-24T23:14:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.323+0000 7f02af7fe640 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0c53380 0x557ad0c55800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.323+0000 7f02af7fe640 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0c53380 0x557ad0c55800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e944adfb930b2f5 server_cookie=6b9066167bfcbc95 in_seq=0 out_seq=0 2024-04-24T23:14:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.323+0000 7f028effd640 10 client.16002 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f028effd640 1 -- 172.21.15.33:0/1953979949 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ad0ba6f00 con 0x557ad0c53380 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f028effd640 10 client.16002 handle_command_reply: tid=0 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 client.16002 shutdown 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 msgr2=0x557ad0c47a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c47a10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f02b0197e30 tx=0x7f02a40118d0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 shutdown_connections 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0c53380 0x557ad0c55800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c47a10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1953979949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 >> 172.21.15.33:0/1953979949 conn(0x557ad0c38ab0 msgr2=0x557ad0c39af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.324+0000 7f028effd640 0 client.16002 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.325+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 shutdown_connections 2024-04-24T23:14:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.325+0000 7f02b8b66740 1 -- 172.21.15.33:0/1953979949 wait complete. 2024-04-24T23:14:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.325+0000 7f02b8b66740 20 client.16002 trim_cache size 0 max 16384 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 Processor -- start 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 -- start start 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c3c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3de30 0x557ad0c3c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01aaab0 con 0x557ad0c42fa0 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01a8380 con 0x557ad0c3d440 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02b8b66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01a61d0 con 0x557ad0c3de30 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c3c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c3c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49586/0 (socket says 172.21.15.33:49586) 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 -- 172.21.15.33:0/796128822 learned_addr learned my addr 172.21.15.33:0/796128822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41260/0 (socket says 172.21.15.33:41260) 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02aeffd640 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3de30 0x557ad0c3c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 -- 172.21.15.33:0/796128822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3de30 msgr2=0x557ad0c3c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3de30 0x557ad0c3c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 -- 172.21.15.33:0/796128822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c42fa0 msgr2=0x557ad0c4ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02af7fe640 1 -- 172.21.15.33:0/796128822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad097d7d0 con 0x557ad0c3d440 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.340+0000 7f02aeffd640 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3de30 0x557ad0c3c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.341+0000 7f02af7fe640 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c3c070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02a00980e0 tx=0x7f02a00097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ae2b187c68c207 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.341+0000 7f02957fa640 1 -- 172.21.15.33:0/796128822 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a006e070 con 0x557ad0c3d440 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.341+0000 7f02957fa640 1 -- 172.21.15.33:0/796128822 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a009c380 con 0x557ad0c3d440 2024-04-24T23:14:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.341+0000 7f02957fa640 1 -- 172.21.15.33:0/796128822 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a009b7c0 con 0x557ad0c3d440 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- 172.21.15.33:0/796128822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 msgr2=0x557ad0c3c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c3c070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02a00980e0 tx=0x7f02a00097b0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- 172.21.15.33:0/796128822 shutdown_connections 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c42fa0 0x557ad0c4ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c3de30 0x557ad0c3c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- 172.21.15.33:0/796128822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c3c070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- 172.21.15.33:0/796128822 >> 172.21.15.33:0/796128822 conn(0x557ad0c38ab0 msgr2=0x557ad0c3b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- 172.21.15.33:0/796128822 shutdown_connections 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- 172.21.15.33:0/796128822 wait complete. 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- start start 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 Processor -- start 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c4c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c4cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01a6840 con 0x557ad0c3de30 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01a7cb0 con 0x557ad0c3d440 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01a6f10 con 0x557ad0c42fa0 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.342+0000 7f02b8b66740 10 client.0 _ll_register_callbacks cb 0x7f02a400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c4c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c4cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c4c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49594/0 (socket says 172.21.15.33:49594) 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c4cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41270/0 (socket says 172.21.15.33:41270) 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 -- 172.21.15.33:0/1170059935 learned_addr learned my addr 172.21.15.33:0/1170059935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02affff640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 -- 172.21.15.33:0/1170059935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 msgr2=0x557ad0c4dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 -- 172.21.15.33:0/1170059935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 msgr2=0x557ad0c4cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c4cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 -- 172.21.15.33:0/1170059935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad0c4e190 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02affff640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02af7fe640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c4c650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f02a001b040 tx=0x7f02a00cfb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85a90878226de24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f028effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f028effd640 1 -- 172.21.15.33:0/1170059935 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a006e070 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a009c380 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f028effd640 1 -- 172.21.15.33:0/1170059935 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f02a0099d20 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f028effd640 1 -- 172.21.15.33:0/1170059935 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02a009b7c0 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02b8b66740 10 client.25547 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.343+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ad0cb9e80 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f028effd640 1 -- 172.21.15.33:0/1170059935 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a009c380 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f028effd640 1 -- 172.21.15.33:0/1170059935 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f02a00d5180 con 0x557ad0c3d440 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02b8b66740 10 client.25547 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02b8b66740 10 client.25547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02b8b66740 20 client.25547 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0cba300 0x557ad0cbc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02b8b66740 4 client.25547 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ad0b24820 con 0x557ad0cba300 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f02aeffd640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0cba300 0x557ad0cbc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.344+0000 7f028effd640 10 client.25547 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02aeffd640 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0cba300 0x557ad0cbc6f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0cb2133ccb89bb1 server_cookie=df24fd6fee6507c1 in_seq=0 out_seq=0 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f028effd640 1 -- 172.21.15.33:0/1170059935 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ad0b24820 con 0x557ad0cba300 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f028effd640 10 client.25547 handle_command_reply: tid=0 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 client.25547 shutdown 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 msgr2=0x557ad0c4c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c4c650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f02a001b040 tx=0x7f02a00cfb20 comp rx=0 tx=0).stop 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 shutdown_connections 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557ad0cba300 0x557ad0cbc6f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad0c42fa0 0x557ad0c4dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ad0c3de30 0x557ad0c4cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f028effd640 0 client.25547 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 --2- 172.21.15.33:0/1170059935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557ad0c3d440 0x557ad0c4c650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 >> 172.21.15.33:0/1170059935 conn(0x557ad0c38ab0 msgr2=0x557ad0c3c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 shutdown_connections 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 1 -- 172.21.15.33:0/1170059935 wait complete. 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.345+0000 7f02b8b66740 20 client.25547 trim_cache size 0 max 16384 2024-04-24T23:14:51.353 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:14:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02980a2b90 msgr2=0x7f02980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02980a2b90 0x7f02980a5050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005e30 tx=0x7f02a80095c0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 msgr2=0x7f02b0196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 0x7f02b0196920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9c0 tx=0x7f02a000ee80 comp rx=0 tx=0).stop 2024-04-24T23:14:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 shutdown_connections 2024-04-24T23:14:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02980a2b90 0x7f02980a5050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b0102250 0x7f02b019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b0101890 0x7f02b0196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 --2- 172.21.15.33:0/4260828928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b01006a0 0x7f02b0196920 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 >> 172.21.15.33:0/4260828928 conn(0x7f02b00fbc30 msgr2=0x7f02b0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 shutdown_connections 2024-04-24T23:14:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.346+0000 7f02b5d77640 1 -- 172.21.15.33:0/4260828928 wait complete. 2024-04-24T23:14:51.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:14:51.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:50 smithi117 ceph-mon[32529]: pgmap v436: 129 pgs: 129 active+clean; 72 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-24T23:14:51.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 145798 14.38 MB/sec warmup 56 sec latency 15.625 ms 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 -- 172.21.15.33:0/40794978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 msgr2=0x7f9da0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 --2- 172.21.15.33:0/40794978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0100aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009900 tx=0x7f9d9c02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 -- 172.21.15.33:0/40794978 shutdown_connections 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 --2- 172.21.15.33:0/40794978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da0102230 0x7f9da0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 --2- 172.21.15.33:0/40794978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0101870 0x7f9da0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 --2- 172.21.15.33:0/40794978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0100aa0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.467+0000 7f9da73e9640 1 -- 172.21.15.33:0/40794978 >> 172.21.15.33:0/40794978 conn(0x7f9da00fbbf0 msgr2=0x7f9da00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 -- 172.21.15.33:0/40794978 shutdown_connections 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 -- 172.21.15.33:0/40794978 wait complete. 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 Processor -- start 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 -- start start 2024-04-24T23:14:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da0101870 0x7f9da019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0102230 0x7f9da01a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010c300 con 0x7f9da0100680 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010c180 con 0x7f9da0102230 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da73e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010c480 con 0x7f9da0101870 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da5986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0102230 0x7f9da01a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41294/0 (socket says 172.21.15.33:41294) 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.468+0000 7f9da5185640 1 -- 172.21.15.33:0/1720608370 learned_addr learned my addr 172.21.15.33:0/1720608370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da4984640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da0101870 0x7f9da019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5986640 1 -- 172.21.15.33:0/1720608370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da0101870 msgr2=0x7f9da019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5986640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da0101870 0x7f9da019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5986640 1 -- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 msgr2=0x7f9da0069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5986640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5986640 1 -- 172.21.15.33:0/1720608370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da01a1960 con 0x7f9da0102230 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5185640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da5986640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0102230 0x7f9da01a1170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009fd0 tx=0x7f9d9800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9d967fc640 1 -- 172.21.15.33:0/1720608370 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800ce20 con 0x7f9da0102230 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9d967fc640 1 -- 172.21.15.33:0/1720608370 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d98005020 con 0x7f9da0102230 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da01a1bf0 con 0x7f9da0102230 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9d967fc640 1 -- 172.21.15.33:0/1720608370 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98010720 con 0x7f9da0102230 2024-04-24T23:14:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.469+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da01a2140 con 0x7f9da0102230 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9d967fc640 1 -- 172.21.15.33:0/1720608370 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9d98005540 con 0x7f9da0102230 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 Processor -- start 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 -- start start 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 0x56531feec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 0x56531fef1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fef4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56531fe50670 con 0x56531fef1d90 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56531fdf8970 con 0x56531feecc20 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56531fe4c630 con 0x56531feec230 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9da4984640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 0x56531feec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9da5185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 0x56531fef1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9d967fc640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9d8c0a2bc0 0x7f9d8c0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9da5185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 0x56531fef1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49624/0 (socket says 172.21.15.33:49624) 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9da4984640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 0x56531feec650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:50314/0 (socket says 172.21.15.33:50314) 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.472+0000 7f9da5185640 1 -- 172.21.15.33:0/405499259 learned_addr learned my addr 172.21.15.33:0/405499259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da4984640 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fef4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5986640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9d8c0a2bc0 0x7f9d8c0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5185640 1 -- 172.21.15.33:0/405499259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 msgr2=0x56531feec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5185640 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 0x56531feec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9d967fc640 1 -- 172.21.15.33:0/1720608370 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d980cfa40 con 0x7f9da0102230 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5185640 1 -- 172.21.15.33:0/405499259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 msgr2=0x56531fef4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5185640 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fef4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5185640 1 -- 172.21.15.33:0/405499259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531fe1c400 con 0x56531feecc20 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da4984640 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 0x56531feec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5185640 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 0x56531fef1720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c031440 tx=0x7f9d9c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221b7bf3ec60152b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9da5986640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9d8c0a2bc0 0x7f9d8c0a5080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d98017000 tx=0x7f9d980054b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9d8b7fe640 1 -- 172.21.15.33:0/405499259 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c035040 con 0x56531feecc20 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9d8b7fe640 1 -- 172.21.15.33:0/405499259 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c0319b0 con 0x56531feecc20 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.473+0000 7f9d8b7fe640 1 -- 172.21.15.33:0/405499259 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0049e0 con 0x56531feecc20 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 -- 172.21.15.33:0/405499259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 msgr2=0x56531fef1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 0x56531fef1720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c031440 tx=0x7f9d9c005de0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 -- 172.21.15.33:0/405499259 shutdown_connections 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fef4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feecc20 0x56531fef1720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/405499259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feec230 0x56531feec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 -- 172.21.15.33:0/405499259 >> 172.21.15.33:0/405499259 conn(0x56531fee78a0 msgr2=0x56531fee7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 -- 172.21.15.33:0/405499259 shutdown_connections 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 -- 172.21.15.33:0/405499259 wait complete. 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 -- start start 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.474+0000 7f9daa1cb740 1 Processor -- start 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 0x56531fef6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fef6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531fef1d90 0x56531fefd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56531fef4ce0 con 0x56531feec230 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56531fe0c050 con 0x56531fef1d90 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56531fef4e60 con 0x56531feecc20 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9daa1cb740 10 client.0 _ll_register_callbacks cb 0x56531fb63be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 0x56531fef6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 0x56531fef6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41298/0 (socket says 172.21.15.33:41298) 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 -- 172.21.15.33:0/1006129625 learned_addr learned my addr 172.21.15.33:0/1006129625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531fef1d90 0x56531fefd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da5185640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fef6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 -- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 msgr2=0x56531fef6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fef6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 -- 172.21.15.33:0/1006129625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531fef1d90 msgr2=0x56531fefd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531fef1d90 0x56531fefd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 -- 172.21.15.33:0/1006129625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531fddddc0 con 0x56531feec230 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9da4984640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 0x56531fef6330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9d90003720 tx=0x7f9d90005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74e8b8c2e210bed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.475+0000 7f9d88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1006129625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90011b50 con 0x56531feec230 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1006129625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d90011cf0 con 0x56531feec230 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9c0319b0 con 0x56531feec230 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1006129625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d9002b050 con 0x56531feec230 2024-04-24T23:14:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 10 client.16020 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56531fb643f0 con 0x56531feec230 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1006129625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c0319b0 con 0x56531feec230 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1006129625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9d900203c0 con 0x56531feec230 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 10 client.16020 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 10 client.16020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 20 client.16020 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff02170 0x56531ff04610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 4 client.16020 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56531fe5dbb0 con 0x56531ff02170 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.476+0000 7f9da5185640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff02170 0x56531ff04610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.477+0000 7f9da5185640 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff02170 0x56531ff04610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea7e942d8e6de052 server_cookie=b2a3ef8b86fa6541 in_seq=0 out_seq=0 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.477+0000 7f9d88ff9640 10 client.16020 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.477+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1006129625 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56531fe5dbb0 con 0x56531ff02170 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.477+0000 7f9d88ff9640 10 client.16020 handle_command_reply: tid=0 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 client.16020 shutdown 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 msgr2=0x56531fef6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 0x56531fef6330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9d90003720 tx=0x7f9d90005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 shutdown_connections 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff02170 0x56531ff04610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531fef1d90 0x56531fefd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fef6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1006129625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feec230 0x56531fef6330 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 >> 172.21.15.33:0/1006129625 conn(0x56531fee78a0 msgr2=0x56531fee6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9d88ff9640 0 client.16020 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 shutdown_connections 2024-04-24T23:14:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1006129625 wait complete. 2024-04-24T23:14:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.478+0000 7f9daa1cb740 20 client.16020 trim_cache size 0 max 16384 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 Processor -- start 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 -- start start 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 0x56531feea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531fef1d90 0x56531fefd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531feeb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01aef10 con 0x56531feecc20 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01ac7e0 con 0x56531feeacf0 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01aa630 con 0x56531fef1d90 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9da4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 0x56531feea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9da4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 0x56531feea7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41308/0 (socket says 172.21.15.33:41308) 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9da4984640 1 -- 172.21.15.33:0/3677539196 learned_addr learned my addr 172.21.15.33:0/3677539196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.493+0000 7f9da4984640 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531feeb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da5185640 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531fef1d90 0x56531fefd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da4984640 1 -- 172.21.15.33:0/3677539196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531fef1d90 msgr2=0x56531fefd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da4984640 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531fef1d90 0x56531fefd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da4984640 1 -- 172.21.15.33:0/3677539196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 msgr2=0x56531feeb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da4984640 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531feeb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da4984640 1 -- 172.21.15.33:0/3677539196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531fc2e7d0 con 0x56531feecc20 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9da4984640 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 0x56531feea7b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d900114c0 tx=0x7f9d90014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f053e86fe8364907 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9d8affd640 1 -- 172.21.15.33:0/3677539196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90020a00 con 0x56531feecc20 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9d8affd640 1 -- 172.21.15.33:0/3677539196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90011e00 con 0x56531feecc20 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.494+0000 7f9d8affd640 1 -- 172.21.15.33:0/3677539196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004c70 con 0x56531feecc20 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 -- 172.21.15.33:0/3677539196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 msgr2=0x56531feea7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 0x56531feea7b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d900114c0 tx=0x7f9d90014040 comp rx=0 tx=0).stop 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 -- 172.21.15.33:0/3677539196 shutdown_connections 2024-04-24T23:14:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531feeb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531fef1d90 0x56531fefd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/3677539196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531feecc20 0x56531feea7b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 -- 172.21.15.33:0/3677539196 >> 172.21.15.33:0/3677539196 conn(0x56531fee78a0 msgr2=0x56531fee6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 -- 172.21.15.33:0/3677539196 shutdown_connections 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 -- 172.21.15.33:0/3677539196 wait complete. 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 -- start start 2024-04-24T23:14:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.495+0000 7f9daa1cb740 1 Processor -- start 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531fefb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fefbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fefcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01aaca0 con 0x56531fef1d90 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01ac110 con 0x56531feeacf0 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01ab370 con 0x56531feecc20 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9daa1cb740 10 client.0 _ll_register_callbacks cb 0x7f9d900045d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da4984640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531fefb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fefcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fefbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fefcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41312/0 (socket says 172.21.15.33:41312) 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da4984640 1 -- 172.21.15.33:0/1742956749 learned_addr learned my addr 172.21.15.33:0/1742956749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 -- 172.21.15.33:0/1742956749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 msgr2=0x56531fefb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531fefb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 -- 172.21.15.33:0/1742956749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 msgr2=0x56531fefcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fefcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 -- 172.21.15.33:0/1742956749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531fefd310 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da4984640 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fefcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9da5185640 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fefbd10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x56531fee8770 tx=0x7f9d9c031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef9c79e955a181ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.496+0000 7f9d88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1742956749 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c035040 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1742956749 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d9c004040 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90011e00 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1742956749 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c060050 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 10 client.25588 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56531ff68ed0 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1742956749 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90011e00 con 0x56531feecc20 2024-04-24T23:14:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1742956749 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9d9c0035f0 con 0x56531feecc20 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 10 client.25588 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 10 client.25588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 20 client.25588 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff69550 0x56531ff6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 4 client.25588 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.497+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56531fe57f00 con 0x56531ff69550 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9da4984640 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff69550 0x56531ff6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9d88ff9640 10 client.25588 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9da4984640 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff69550 0x56531ff6b940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=179b55adcfbf282a server_cookie=a87a45e6c51fe222 in_seq=0 out_seq=0 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9d88ff9640 1 -- 172.21.15.33:0/1742956749 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56531fe57f00 con 0x56531ff69550 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9d88ff9640 10 client.25588 handle_command_reply: tid=0 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 client.25588 shutdown 2024-04-24T23:14:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 msgr2=0x56531fefbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fefbd10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x56531fee8770 tx=0x7f9d9c031b60 comp rx=0 tx=0).stop 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 shutdown_connections 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56531ff69550 0x56531ff6b940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56531fef1d90 0x56531fefcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9d88ff9640 0 client.25588 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56531feecc20 0x56531fefbd10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 --2- 172.21.15.33:0/1742956749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56531feeacf0 0x56531fefb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 >> 172.21.15.33:0/1742956749 conn(0x56531fee78a0 msgr2=0x56531fee6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 shutdown_connections 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 1 -- 172.21.15.33:0/1742956749 wait complete. 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.498+0000 7f9daa1cb740 20 client.25588 trim_cache size 0 max 16384 2024-04-24T23:14:51.506 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:14:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9d8c0a2bc0 msgr2=0x7f9d8c0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9d8c0a2bc0 0x7f9d8c0a5080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d98017000 tx=0x7f9d980054b0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0102230 msgr2=0x7f9da01a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0102230 0x7f9da01a1170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009fd0 tx=0x7f9d9800ef10 comp rx=0 tx=0).stop 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 shutdown_connections 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9d8c0a2bc0 0x7f9d8c0a5080 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9da0102230 0x7f9da01a1170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da0101870 0x7f9da019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 --2- 172.21.15.33:0/1720608370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da0100680 0x7f9da0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 >> 172.21.15.33:0/1720608370 conn(0x7f9da00fbbf0 msgr2=0x7f9da0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 shutdown_connections 2024-04-24T23:14:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.499+0000 7f9da73e9640 1 -- 172.21.15.33:0/1720608370 wait complete. 2024-04-24T23:14:51.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 -- 172.21.15.33:0/1256752058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8101870 msgr2=0x7f95c8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 --2- 172.21.15.33:0/1256752058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8101870 0x7f95c8101cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009900 tx=0x7f95b002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 -- 172.21.15.33:0/1256752058 shutdown_connections 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 --2- 172.21.15.33:0/1256752058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95c8102230 0x7f95c8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 --2- 172.21.15.33:0/1256752058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8101870 0x7f95c8101cf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 --2- 172.21.15.33:0/1256752058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95c8100680 0x7f95c8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 -- 172.21.15.33:0/1256752058 >> 172.21.15.33:0/1256752058 conn(0x7f95c80fbbf0 msgr2=0x7f95c80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 -- 172.21.15.33:0/1256752058 shutdown_connections 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.621+0000 7f95cce72640 1 -- 172.21.15.33:0/1256752058 wait complete. 2024-04-24T23:14:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.622+0000 7f95cce72640 1 Processor -- start 2024-04-24T23:14:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.622+0000 7f95cce72640 1 -- start start 2024-04-24T23:14:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.622+0000 7f95cce72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 0x7f95c8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.622+0000 7f95cce72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95c8101870 0x7f95c8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.622+0000 7f95cce72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95c8102230 0x7f95c819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95cce72640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95c810c300 con 0x7f95c8100680 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95cce72640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f95c810c180 con 0x7f95c8102230 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95cce72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95c810c480 con 0x7f95c8101870 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 0x7f95c8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 0x7f95c8197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41320/0 (socket says 172.21.15.33:41320) 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 -- 172.21.15.33:0/2465972370 learned_addr learned my addr 172.21.15.33:0/2465972370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95c8102230 0x7f95c819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95c8101870 0x7f95c8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 -- 172.21.15.33:0/2465972370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95c8101870 msgr2=0x7f95c8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95c8101870 0x7f95c8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 -- 172.21.15.33:0/2465972370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95c8102230 msgr2=0x7f95c819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95c8102230 0x7f95c819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 -- 172.21.15.33:0/2465972370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c819f6f0 con 0x7f95c8100680 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95c659c640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 0x7f95c8197ad0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f95b800e9c0 tx=0x7f95b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95bffff640 1 -- 172.21.15.33:0/2465972370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b800cd80 con 0x7f95c8100680 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c819f9e0 con 0x7f95c8100680 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95bffff640 1 -- 172.21.15.33:0/2465972370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b80035f0 con 0x7f95c8100680 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95bffff640 1 -- 172.21.15.33:0/2465972370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8010690 con 0x7f95c8100680 2024-04-24T23:14:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.623+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c819ff00 con 0x7f95c8100680 2024-04-24T23:14:51.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.625+0000 7f95bffff640 1 -- 172.21.15.33:0/2465972370 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f95b80037a0 con 0x7f95c8100680 2024-04-24T23:14:51.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 Processor -- start 2024-04-24T23:14:51.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 -- start start 2024-04-24T23:14:51.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f97650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943f9c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943f9f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a943efb670 con 0x55a943f9cd90 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a943ea3970 con 0x55a943f97230 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a943ef7630 con 0x55a943f97c20 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95bffff640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95b40a2b90 0x7f95b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95bdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943f9f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95c659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943f9c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95c6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f97650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95bdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943f9f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41328/0 (socket says 172.21.15.33:41328) 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.626+0000 7f95bdd9b640 1 -- 172.21.15.33:0/3116958150 learned_addr learned my addr 172.21.15.33:0/3116958150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95b40a2b90 0x7f95b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bffff640 1 -- 172.21.15.33:0/2465972370 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f95b80cf7d0 con 0x7f95c8100680 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bdd9b640 1 -- 172.21.15.33:0/3116958150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 msgr2=0x55a943f9c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943f9c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bdd9b640 1 -- 172.21.15.33:0/3116958150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 msgr2=0x55a943f97650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f97650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bdd9b640 1 -- 172.21.15.33:0/3116958150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a943ec7400 con 0x55a943f9cd90 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95b40a2b90 0x7f95b40a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55a943f98480 tx=0x7f95c0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943f9f220 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f95b002a930 tx=0x7f95b0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d3f4037b253b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bcd99640 1 -- 172.21.15.33:0/3116958150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b003f070 con 0x55a943f9cd90 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bcd99640 1 -- 172.21.15.33:0/3116958150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b00318b0 con 0x55a943f9cd90 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.627+0000 7f95bcd99640 1 -- 172.21.15.33:0/3116958150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0035070 con 0x55a943f9cd90 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 -- 172.21.15.33:0/3116958150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 msgr2=0x55a943f9f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943f9f220 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f95b002a930 tx=0x7f95b0032cb0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 -- 172.21.15.33:0/3116958150 shutdown_connections 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943f9f220 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943f9c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/3116958150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f97650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 -- 172.21.15.33:0/3116958150 >> 172.21.15.33:0/3116958150 conn(0x55a943f928a0 msgr2=0x55a943f92ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 -- 172.21.15.33:0/3116958150 shutdown_connections 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 -- 172.21.15.33:0/3116958150 wait complete. 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.628+0000 7f95cfb7e740 1 -- start start 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 Processor -- start 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 0x55a943fa1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97c20 0x55a943fa1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943fa88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a943f9fce0 con 0x55a943f97230 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a943eb7050 con 0x55a943f97c20 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a943f9fe60 con 0x55a943f9cd90 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 0x55a943fa1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95cfb7e740 10 client.0 _ll_register_callbacks cb 0x55a943c0ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 0x55a943fa1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41344/0 (socket says 172.21.15.33:41344) 2024-04-24T23:14:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 -- 172.21.15.33:0/4005282229 learned_addr learned my addr 172.21.15.33:0/4005282229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97c20 0x55a943fa1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943fa88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 -- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 msgr2=0x55a943fa88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943fa88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 -- 172.21.15.33:0/4005282229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97c20 msgr2=0x55a943fa1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97c20 0x55a943fa1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 -- 172.21.15.33:0/4005282229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a943e88dc0 con 0x55a943f97230 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97c20 0x55a943fa1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95c659c640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 0x55a943fa1330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f95b809b9a0 tx=0x7f95b806ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939b8b01c31a9788 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95adffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.629+0000 7f95adffb640 1 -- 172.21.15.33:0/4005282229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b809c3a0 con 0x55a943f97230 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95adffb640 1 -- 172.21.15.33:0/4005282229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f95b80a4480 con 0x55a943f97230 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b00318b0 con 0x55a943f97230 2024-04-24T23:14:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95adffb640 1 -- 172.21.15.33:0/4005282229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95b80a4910 con 0x55a943f97230 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 10 client.16038 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a943c0f3f0 con 0x55a943f97230 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95adffb640 1 -- 172.21.15.33:0/4005282229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b00318b0 con 0x55a943f97230 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95adffb640 1 -- 172.21.15.33:0/4005282229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f95b80b4070 con 0x55a943f97230 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 10 client.16038 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 20 client.16038 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a943fad190 0x55a943faf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 4 client.16038 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a943f08bb0 con 0x55a943fad190 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.630+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a943fad190 0x55a943faf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.631+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a943fad190 0x55a943faf650 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e111e3d33aa84e server_cookie=36b295bb323ef61b in_seq=0 out_seq=0 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.631+0000 7f95adffb640 10 client.16038 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:51.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95adffb640 1 -- 172.21.15.33:0/4005282229 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a943f08bb0 con 0x55a943fad190 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95adffb640 10 client.16038 handle_command_reply: tid=0 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 client.16038 shutdown 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 msgr2=0x55a943fa1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 0x55a943fa1330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f95b809b9a0 tx=0x7f95b806ac20 comp rx=0 tx=0).stop 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 shutdown_connections 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a943fad190 0x55a943faf650 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943fa88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97c20 0x55a943fa1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95adffb640 0 client.16038 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/4005282229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97230 0x55a943fa1330 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 >> 172.21.15.33:0/4005282229 conn(0x55a943f928a0 msgr2=0x55a943f91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 shutdown_connections 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 1 -- 172.21.15.33:0/4005282229 wait complete. 2024-04-24T23:14:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.632+0000 7f95cfb7e740 20 client.16038 trim_cache size 0 max 16384 2024-04-24T23:14:51.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 Processor -- start 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 -- start start 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f98640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97c20 0x55a943fa8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943f98b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81acc60 con 0x55a943f97c20 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81aa530 con 0x55a943f97230 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.645+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81a8380 con 0x55a943f9cd90 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f98640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97c20 0x55a943fa8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f98640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49692/0 (socket says 172.21.15.33:49692) 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95c659c640 1 -- 172.21.15.33:0/2272729182 learned_addr learned my addr 172.21.15.33:0/2272729182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943f98b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 -- 172.21.15.33:0/2272729182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 msgr2=0x55a943f98b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943f98b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 -- 172.21.15.33:0/2272729182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 msgr2=0x55a943f98640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f98640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 -- 172.21.15.33:0/2272729182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a943cd97d0 con 0x55a943f97c20 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943f98b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95c659c640 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f98640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97c20 0x55a943fa8870 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f95b0000c00 tx=0x7f95b0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba173f353013720a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95affff640 1 -- 172.21.15.33:0/2272729182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0055070 con 0x55a943f97c20 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95affff640 1 -- 172.21.15.33:0/2272729182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b004f110 con 0x55a943f97c20 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.646+0000 7f95affff640 1 -- 172.21.15.33:0/2272729182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b003f040 con 0x55a943f97c20 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- 172.21.15.33:0/2272729182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97c20 msgr2=0x55a943fa8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97c20 0x55a943fa8870 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f95b0000c00 tx=0x7f95b0005750 comp rx=0 tx=0).stop 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- 172.21.15.33:0/2272729182 shutdown_connections 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f9cd90 0x55a943f98b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f97c20 0x55a943fa8870 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/2272729182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943f98640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- 172.21.15.33:0/2272729182 >> 172.21.15.33:0/2272729182 conn(0x55a943f928a0 msgr2=0x55a943f91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- 172.21.15.33:0/2272729182 shutdown_connections 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- 172.21.15.33:0/2272729182 wait complete. 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- start start 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 Processor -- start 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943fa5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943fa64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943fa74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81a89f0 con 0x55a943f9cd90 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81a9e60 con 0x55a943f97230 2024-04-24T23:14:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81a90c0 con 0x55a943f97c20 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.647+0000 7f95cfb7e740 10 client.0 _ll_register_callbacks cb 0x7f95b809bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943fa74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943fa74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41354/0 (socket says 172.21.15.33:41354) 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 -- 172.21.15.33:0/534057289 learned_addr learned my addr 172.21.15.33:0/534057289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c659c640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943fa5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95bdd9b640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943fa64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 -- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 msgr2=0x55a943fa64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943fa64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 -- 172.21.15.33:0/534057289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 msgr2=0x55a943fa5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943fa5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 -- 172.21.15.33:0/534057289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a943fa7ad0 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943fa74e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f95c0003ae0 tx=0x7f95c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b473b2935f890d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95adffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95adffb640 1 -- 172.21.15.33:0/534057289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c002e060 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b004f110 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95adffb640 1 -- 172.21.15.33:0/534057289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f95c0020450 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95adffb640 1 -- 172.21.15.33:0/534057289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95c0035050 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95cfb7e740 10 client.16044 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.648+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a944013790 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95adffb640 1 -- 172.21.15.33:0/534057289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b004f110 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95adffb640 1 -- 172.21.15.33:0/534057289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f95c0028070 con 0x55a943f9cd90 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 10 client.16044 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 20 client.16044 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9440141f0 0x55a944016610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 4 client.16044 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a943f02f00 con 0x55a9440141f0 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95c659c640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9440141f0 0x55a944016610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95c659c640 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9440141f0 0x55a944016610 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b8d6226901f3c4 server_cookie=fd0f803d151e5ffe in_seq=0 out_seq=0 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95adffb640 10 client.16044 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95adffb640 1 -- 172.21.15.33:0/534057289 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a943f02f00 con 0x55a9440141f0 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95adffb640 10 client.16044 handle_command_reply: tid=0 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.649+0000 7f95cfb7e740 1 client.16044 shutdown 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 msgr2=0x55a943fa74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943fa74e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f95c0003ae0 tx=0x7f95c0004a40 comp rx=0 tx=0).stop 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 shutdown_connections 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9440141f0 0x55a944016610 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a943f9cd90 0x55a943fa74e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a943f97c20 0x55a943fa64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 --2- 172.21.15.33:0/534057289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a943f97230 0x55a943fa5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95adffb640 0 client.16044 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 >> 172.21.15.33:0/534057289 conn(0x55a943f928a0 msgr2=0x55a943f957b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 shutdown_connections 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 1 -- 172.21.15.33:0/534057289 wait complete. 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cfb7e740 20 client.16044 trim_cache size 0 max 16384 2024-04-24T23:14:51.659 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95b40a2b90 msgr2=0x7f95b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95b40a2b90 0x7f95b40a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55a943f98480 tx=0x7f95c0009500 comp rx=0 tx=0).stop 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 msgr2=0x7f95c8197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 0x7f95c8197ad0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f95b800e9c0 tx=0x7f95b800ee80 comp rx=0 tx=0).stop 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 shutdown_connections 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f95b40a2b90 0x7f95b40a5050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95c8102230 0x7f95c819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95c8101870 0x7f95c8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 --2- 172.21.15.33:0/2465972370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8100680 0x7f95c8197ad0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.650+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 >> 172.21.15.33:0/2465972370 conn(0x7f95c80fbbf0 msgr2=0x7f95c80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.651+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 shutdown_connections 2024-04-24T23:14:51.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:51.651+0000 7f95cce72640 1 -- 172.21.15.33:0/2465972370 wait complete. 2024-04-24T23:14:52.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:51 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3588364140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:52.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:51 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3588364140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:52.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:51 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3588364140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:52.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 147898 14.25 MB/sec warmup 57 sec latency 120.227 ms 2024-04-24T23:14:52.664 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:14:52.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:52.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:14:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 -- 172.21.15.33:0/3391541375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 msgr2=0x7fb27c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 --2- 172.21.15.33:0/3391541375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c104650 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009ca0 tx=0x7fb26c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 -- 172.21.15.33:0/3391541375 shutdown_connections 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 --2- 172.21.15.33:0/3391541375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb27c1074c0 0x7fb27c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 --2- 172.21.15.33:0/3391541375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c104b90 0x7fb27c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 --2- 172.21.15.33:0/3391541375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c104650 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 -- 172.21.15.33:0/3391541375 >> 172.21.15.33:0/3391541375 conn(0x7fb27c0fbbf0 msgr2=0x7fb27c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 -- 172.21.15.33:0/3391541375 shutdown_connections 2024-04-24T23:14:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 -- 172.21.15.33:0/3391541375 wait complete. 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.781+0000 7fb283389640 1 Processor -- start 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 -- start start 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb27c104b90 0x7fb27c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c1074c0 0x7fb27c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c10d420 con 0x7fb27c102260 2024-04-24T23:14:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c10d2a0 con 0x7fb27c104b90 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb283389640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c10d5a0 con 0x7fb27c1074c0 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281125640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281125640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41370/0 (socket says 172.21.15.33:41370) 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281125640 1 -- 172.21.15.33:0/2277580740 learned_addr learned my addr 172.21.15.33:0/2277580740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c1074c0 0x7fb27c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb280924640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb27c104b90 0x7fb27c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 -- 172.21.15.33:0/2277580740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb27c104b90 msgr2=0x7fb27c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb27c104b90 0x7fb27c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 -- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 msgr2=0x7fb27c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 -- 172.21.15.33:0/2277580740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c1a19d0 con 0x7fb27c1074c0 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281125640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb281926640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c1074c0 0x7fb27c1a1330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb27400e970 tx=0x7fb27400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.782+0000 7fb27a7fc640 1 -- 172.21.15.33:0/2277580740 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27400ccc0 con 0x7fb27c1074c0 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.783+0000 7fb27a7fc640 1 -- 172.21.15.33:0/2277580740 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2740035f0 con 0x7fb27c1074c0 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.783+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c1a1cc0 con 0x7fb27c1074c0 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.783+0000 7fb27a7fc640 1 -- 172.21.15.33:0/2277580740 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27400ce60 con 0x7fb27c1074c0 2024-04-24T23:14:52.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.783+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c1a21c0 con 0x7fb27c1074c0 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.785+0000 7fb27a7fc640 1 -- 172.21.15.33:0/2277580740 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb2740037a0 con 0x7fb27c1074c0 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.786+0000 7fb286173740 1 Processor -- start 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.786+0000 7fb27a7fc640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2640a2b90 0x7fb2640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.786+0000 7fb286173740 1 -- start start 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.786+0000 7fb286173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.786+0000 7fb286173740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 0x5584890e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb286173740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb286173740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5584890382c0 con 0x5584890e3100 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb286173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558488fe0910 con 0x5584890dd5a0 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb286173740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558489034280 con 0x5584890ddf90 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb280924640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 0x5584890e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb280924640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 0x5584890e2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:50444/0 (socket says 172.21.15.33:50444) 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb280924640 1 -- 172.21.15.33:0/1646157946 learned_addr learned my addr 172.21.15.33:0/1646157946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.787+0000 7fb280924640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2640a2b90 0x7fb2640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb280924640 1 -- 172.21.15.33:0/1646157946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 msgr2=0x5584890dd9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb280924640 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890dd9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb280924640 1 -- 172.21.15.33:0/1646157946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 msgr2=0x5584890e5590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:14:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb281926640 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb280924640 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb280924640 1 -- 172.21.15.33:0/1646157946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558488ff1fb0 con 0x5584890ddf90 2024-04-24T23:14:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb281926640 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.788+0000 7fb27a7fc640 1 -- 172.21.15.33:0/2277580740 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2740cf680 con 0x7fb27c1074c0 2024-04-24T23:14:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.789+0000 7fb280924640 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 0x5584890e2a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb270011a60 tx=0x7fb270011f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d634f4cb04b6e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.789+0000 7fb280924640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2640a2b90 0x7fb2640a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb27000f850 tx=0x7fb270018000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.789+0000 7fb2637fe640 1 -- 172.21.15.33:0/1646157946 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270011300 con 0x5584890ddf90 2024-04-24T23:14:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.790+0000 7fb2637fe640 1 -- 172.21.15.33:0/1646157946 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27000e850 con 0x5584890ddf90 2024-04-24T23:14:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.790+0000 7fb2637fe640 1 -- 172.21.15.33:0/1646157946 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700116b0 con 0x5584890ddf90 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 -- 172.21.15.33:0/1646157946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 msgr2=0x5584890e2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 0x5584890e2a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb270011a60 tx=0x7fb270011f20 comp rx=0 tx=0).stop 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 -- 172.21.15.33:0/1646157946 shutdown_connections 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ddf90 0x5584890e2a90 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb270011a60 tx=0x7fb270011f20 comp rx=0 tx=0).stop 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 --2- 172.21.15.33:0/1646157946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.791+0000 7fb286173740 1 -- 172.21.15.33:0/1646157946 >> 172.21.15.33:0/1646157946 conn(0x55848909c950 msgr2=0x5584890d9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.792+0000 7fb286173740 1 -- 172.21.15.33:0/1646157946 shutdown_connections 2024-04-24T23:14:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.792+0000 7fb286173740 1 -- 172.21.15.33:0/1646157946 wait complete. 2024-04-24T23:14:52.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.792+0000 7fb286173740 1 -- start start 2024-04-24T23:14:52.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.792+0000 7fb286173740 1 Processor -- start 2024-04-24T23:14:52.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.793+0000 7fb286173740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890dd5a0 0x5584890e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.793+0000 7fb286173740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.793+0000 7fb286173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x5584890ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb286173740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5584890e6050 con 0x5584890e3100 2024-04-24T23:14:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb286173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558488ff3e90 con 0x5584890ed220 2024-04-24T23:14:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb286173740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5584890e61d0 con 0x5584890dd5a0 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x5584890ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x5584890ef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49752/0 (socket says 172.21.15.33:49752) 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb286173740 10 client.0 _ll_register_callbacks cb 0x55848902f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 -- 172.21.15.33:0/3813436007 learned_addr learned my addr 172.21.15.33:0/3813436007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb280924640 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 -- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890dd5a0 msgr2=0x5584890e7d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890dd5a0 0x5584890e7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 -- 172.21.15.33:0/3813436007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 msgr2=0x5584890e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.794+0000 7fb281926640 1 -- 172.21.15.33:0/3813436007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558488e147e0 con 0x5584890ed220 2024-04-24T23:14:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.796+0000 7fb281926640 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x5584890ef610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb274098490 tx=0x7fb274099d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59554b2ef8222ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.796+0000 7fb260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:14:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.796+0000 7fb260ff9640 1 -- 172.21.15.33:0/3813436007 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740b3020 con 0x5584890ed220 2024-04-24T23:14:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.796+0000 7fb260ff9640 1 -- 172.21.15.33:0/3813436007 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb27409afa0 con 0x5584890ed220 2024-04-24T23:14:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.796+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb27000e850 con 0x5584890ed220 2024-04-24T23:14:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb260ff9640 1 -- 172.21.15.33:0/3813436007 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb27409ff00 con 0x5584890ed220 2024-04-24T23:14:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb286173740 10 client.25592 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558488e74f70 con 0x5584890ed220 2024-04-24T23:14:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb260ff9640 1 -- 172.21.15.33:0/3813436007 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27000e850 con 0x5584890ed220 2024-04-24T23:14:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb260ff9640 1 -- 172.21.15.33:0/3813436007 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb274097a10 con 0x5584890ed220 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb286173740 10 client.25592 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb286173740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb286173740 20 client.25592 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.797+0000 7fb286173740 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5584890f3e90 0x5584890f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.798+0000 7fb286173740 4 client.25592 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.798+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55848900c5a0 con 0x5584890f3e90 2024-04-24T23:14:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.798+0000 7fb281125640 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5584890f3e90 0x5584890f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.798+0000 7fb260ff9640 10 client.25592 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.799+0000 7fb281125640 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5584890f3e90 0x5584890f6330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985b97bb69213686 server_cookie=3b6c7ee0324a4ea6 in_seq=0 out_seq=0 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb260ff9640 1 -- 172.21.15.33:0/3813436007 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55848900c5a0 con 0x5584890f3e90 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb260ff9640 10 client.25592 handle_command_reply: tid=0 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 client.25592 shutdown 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 msgr2=0x5584890ef610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x5584890ef610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb274098490 tx=0x7fb274099d50 comp rx=0 tx=0).stop 2024-04-24T23:14:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 shutdown_connections 2024-04-24T23:14:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5584890f3e90 0x5584890f6330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x5584890ef610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.800+0000 7fb286173740 1 --2- 172.21.15.33:0/3813436007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890dd5a0 0x5584890e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.801+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 >> 172.21.15.33:0/3813436007 conn(0x55848909c950 msgr2=0x5584890cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.801+0000 7fb260ff9640 0 client.25592 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.801+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 shutdown_connections 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.801+0000 7fb286173740 1 -- 172.21.15.33:0/3813436007 wait complete. 2024-04-24T23:14:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.801+0000 7fb286173740 20 client.25592 trim_cache size 0 max 16384 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.838+0000 7fb286173740 1 Processor -- start 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.838+0000 7fb286173740 1 -- start start 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb286173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890df710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb286173740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb286173740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ed220 0x5584890d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb286173740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1aab60 con 0x5584890e3100 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb286173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1a8430 con 0x5584890dd5a0 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb286173740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1a6280 con 0x5584890ed220 2024-04-24T23:14:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890dfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41390/0 (socket says 172.21.15.33:41390) 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 -- 172.21.15.33:0/3515525221 learned_addr learned my addr 172.21.15.33:0/3515525221 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb281125640 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb281926640 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ed220 0x5584890d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 -- 172.21.15.33:0/3515525221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ed220 msgr2=0x5584890d8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ed220 0x5584890d8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 -- 172.21.15.33:0/3515525221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 msgr2=0x5584890df710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890df710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 -- 172.21.15.33:0/3515525221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558488d0c170 con 0x5584890e3100 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb281125640 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.839+0000 7fb280924640 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890dfc50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb270011a60 tx=0x7fb270009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4127917a14e2c4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.840+0000 7fb262ffd640 1 -- 172.21.15.33:0/3515525221 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270005630 con 0x5584890e3100 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.840+0000 7fb262ffd640 1 -- 172.21.15.33:0/3515525221 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270011100 con 0x5584890e3100 2024-04-24T23:14:52.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.840+0000 7fb262ffd640 1 -- 172.21.15.33:0/3515525221 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700035f0 con 0x5584890e3100 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 -- 172.21.15.33:0/3515525221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 msgr2=0x5584890dfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890dfc50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb270011a60 tx=0x7fb270009bd0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 -- 172.21.15.33:0/3515525221 shutdown_connections 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890ed220 0x5584890d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890e3100 0x5584890dfc50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 --2- 172.21.15.33:0/3515525221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890dd5a0 0x5584890df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 -- 172.21.15.33:0/3515525221 >> 172.21.15.33:0/3515525221 conn(0x55848909c950 msgr2=0x5584890e7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 -- 172.21.15.33:0/3515525221 shutdown_connections 2024-04-24T23:14:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 -- 172.21.15.33:0/3515525221 wait complete. 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 -- start start 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 Processor -- start 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 0x558489104af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.841+0000 7fb286173740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890e3100 0x558489105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb286173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x558489106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb286173740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1a68f0 con 0x5584890dd5a0 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb286173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1a7d60 con 0x5584890ed220 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb286173740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1a6fc0 con 0x5584890e3100 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb286173740 10 client.0 _ll_register_callbacks cb 0x7fb274009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 0x558489104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 0x558489104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41392/0 (socket says 172.21.15.33:41392) 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 -- 172.21.15.33:0/3693663033 learned_addr learned my addr 172.21.15.33:0/3693663033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb280924640 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890e3100 0x558489105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 -- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890e3100 msgr2=0x558489105030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890e3100 0x558489105030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 -- 172.21.15.33:0/3693663033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 msgr2=0x558489106040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x558489106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 -- 172.21.15.33:0/3693663033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558489106630 con 0x5584890dd5a0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb281125640 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 0x558489104af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb26c02a900 tx=0x7fb26c031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9d65183c3e3351 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.842+0000 7fb260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb260ff9640 1 -- 172.21.15.33:0/3693663033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c04b070 con 0x5584890dd5a0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb260ff9640 1 -- 172.21.15.33:0/3693663033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb26c045390 con 0x5584890dd5a0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb270011100 con 0x5584890dd5a0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb260ff9640 1 -- 172.21.15.33:0/3693663033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb26c02f3d0 con 0x5584890dd5a0 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb286173740 10 client.16059 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584891079e0 con 0x5584890dd5a0 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.843+0000 7fb260ff9640 1 -- 172.21.15.33:0/3693663033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270011100 con 0x5584890dd5a0 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb260ff9640 1 -- 172.21.15.33:0/3693663033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb26c045070 con 0x5584890dd5a0 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb286173740 10 client.16059 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb286173740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb286173740 20 client.16059 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb286173740 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55848915a720 0x55848915cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb286173740 4 client.16059 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.844+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558489045800 con 0x55848915a720 2024-04-24T23:14:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.845+0000 7fb280924640 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55848915a720 0x55848915cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.846+0000 7fb260ff9640 10 client.16059 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.846+0000 7fb280924640 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55848915a720 0x55848915cb40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c186e0a1f5f2ea2e server_cookie=181f4dc37dd7df68 in_seq=0 out_seq=0 2024-04-24T23:14:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb260ff9640 1 -- 172.21.15.33:0/3693663033 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558489045800 con 0x55848915a720 2024-04-24T23:14:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb260ff9640 10 client.16059 handle_command_reply: tid=0 2024-04-24T23:14:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 client.16059 shutdown 2024-04-24T23:14:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 msgr2=0x558489104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 0x558489104af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb26c02a900 tx=0x7fb26c031880 comp rx=0 tx=0).stop 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 shutdown_connections 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55848915a720 0x55848915cb40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584890ed220 0x558489106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584890e3100 0x558489105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 --2- 172.21.15.33:0/3693663033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584890dd5a0 0x558489104af0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.848+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 >> 172.21.15.33:0/3693663033 conn(0x55848909c950 msgr2=0x5584890e7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.849+0000 7fb260ff9640 0 client.16059 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.849+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 shutdown_connections 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.849+0000 7fb286173740 1 -- 172.21.15.33:0/3693663033 wait complete. 2024-04-24T23:14:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.849+0000 7fb286173740 20 client.16059 trim_cache size 0 max 16384 2024-04-24T23:14:52.854 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:14:52.854 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:14:52.854 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "862e0c86-5cb5-4a40-802e-ca0d095a4214", 2024-04-24T23:14:52.854 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:14:52.854 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:14:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.851+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2640a2b90 msgr2=0x7fb2640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.851+0000 7fb283389640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2640a2b90 0x7fb2640a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb27000f850 tx=0x7fb270018000 comp rx=0 tx=0).stop 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.851+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c1074c0 msgr2=0x7fb27c1a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.851+0000 7fb283389640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c1074c0 0x7fb27c1a1330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb27400e970 tx=0x7fb27400ee30 comp rx=0 tx=0).stop 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 shutdown_connections 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2640a2b90 0x7fb2640a5050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c1074c0 0x7fb27c1a1330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb27c104b90 0x7fb27c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 --2- 172.21.15.33:0/2277580740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c102260 0x7fb27c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 >> 172.21.15.33:0/2277580740 conn(0x7fb27c0fbbf0 msgr2=0x7fb27c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 shutdown_connections 2024-04-24T23:14:52.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:52.852+0000 7fb283389640 1 -- 172.21.15.33:0/2277580740 wait complete. 2024-04-24T23:14:52.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:14:52.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:14:53.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 -- 172.21.15.33:0/460938140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 msgr2=0x7f3a70101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 --2- 172.21.15.33:0/460938140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70101cf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009900 tx=0x7f3a6002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:53.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 -- 172.21.15.33:0/460938140 shutdown_connections 2024-04-24T23:14:53.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 --2- 172.21.15.33:0/460938140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70102230 0x7f3a70108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 --2- 172.21.15.33:0/460938140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70101cf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 --2- 172.21.15.33:0/460938140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a70100680 0x7f3a70100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 -- 172.21.15.33:0/460938140 >> 172.21.15.33:0/460938140 conn(0x7f3a700fbbf0 msgr2=0x7f3a700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 -- 172.21.15.33:0/460938140 shutdown_connections 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.003+0000 7f3a77be3640 1 -- 172.21.15.33:0/460938140 wait complete. 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 Processor -- start 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 -- start start 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70100680 0x7f3a70197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a70102230 0x7f3a7019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010c2e0 con 0x7f3a70101870 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010c160 con 0x7f3a70100680 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a77be3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010c460 con 0x7f3a70102230 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41414/0 (socket says 172.21.15.33:41414) 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 -- 172.21.15.33:0/495265808 learned_addr learned my addr 172.21.15.33:0/495265808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7597f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70100680 0x7f3a70197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a76180640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a70102230 0x7f3a7019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 -- 172.21.15.33:0/495265808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a70102230 msgr2=0x7f3a7019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a70102230 0x7f3a7019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 -- 172.21.15.33:0/495265808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70100680 msgr2=0x7f3a70197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70100680 0x7f3a70197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7517e640 1 -- 172.21.15.33:0/495265808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019f6c0 con 0x7f3a70101870 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.004+0000 7f3a7597f640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70100680 0x7f3a70197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:53.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.005+0000 7f3a7517e640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70197fe0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009230 tx=0x7f3a600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.005+0000 7f3a66ffd640 1 -- 172.21.15.33:0/495265808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6003f030 con 0x7f3a70101870 2024-04-24T23:14:53.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.005+0000 7f3a66ffd640 1 -- 172.21.15.33:0/495265808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a60003ce0 con 0x7f3a70101870 2024-04-24T23:14:53.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.005+0000 7f3a66ffd640 1 -- 172.21.15.33:0/495265808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60035070 con 0x7f3a70101870 2024-04-24T23:14:53.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.005+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7019f950 con 0x7f3a70101870 2024-04-24T23:14:53.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.005+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7019fe90 con 0x7f3a70101870 2024-04-24T23:14:53.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.007+0000 7f3a66ffd640 1 -- 172.21.15.33:0/495265808 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a60042030 con 0x7f3a70101870 2024-04-24T23:14:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.007+0000 7f3a7a9cd740 1 Processor -- start 2024-04-24T23:14:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 -- start start 2024-04-24T23:14:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21ceda0 0x5650e21d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5650e212d6b0 con 0x5650e21c9c30 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5650e20d5880 con 0x5650e21ceda0 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650e2129670 con 0x5650e21c9240 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a76180640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7517e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21ceda0 0x5650e21d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7597f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a66ffd640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a5c0a2c10 0x7f3a5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a76180640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:50500/0 (socket says 172.21.15.33:50500) 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a76180640 1 -- 172.21.15.33:0/2188426017 learned_addr learned my addr 172.21.15.33:0/2188426017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.008+0000 7f3a7597f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41422/0 (socket says 172.21.15.33:41422) 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a7517e640 1 -- 172.21.15.33:0/2188426017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 msgr2=0x5650e21c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a7517e640 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a7517e640 1 -- 172.21.15.33:0/2188426017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 msgr2=0x5650e21ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a7517e640 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a7517e640 1 -- 172.21.15.33:0/2188426017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e1f0b7d0 con 0x5650e21ceda0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a76180640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a5c0a2c10 0x7f3a5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a66ffd640 1 -- 172.21.15.33:0/495265808 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a600ef7a0 con 0x7f3a70101870 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a7517e640 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21ceda0 0x5650e21d1230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a600ecf60 tx=0x7f3a6008aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2497a8cbaf379628 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a5bfff640 1 -- 172.21.15.33:0/2188426017 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600bb680 con 0x5650e21ceda0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a5bfff640 1 -- 172.21.15.33:0/2188426017 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6008aab0 con 0x5650e21ceda0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.009+0000 7f3a5bfff640 1 -- 172.21.15.33:0/2188426017 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600044e0 con 0x5650e21ceda0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/2188426017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21ceda0 msgr2=0x5650e21d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a76180640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a5c0a2c10 0x7f3a5c0a50d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3a68013a40 tx=0x7f3a68009c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21ceda0 0x5650e21d1230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a600ecf60 tx=0x7f3a6008aca0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/2188426017 shutdown_connections 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21ceda0 0x5650e21d1230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/2188426017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/2188426017 >> 172.21.15.33:0/2188426017 conn(0x5650e21c48b0 msgr2=0x5650e21c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/2188426017 shutdown_connections 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/2188426017 wait complete. 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 -- start start 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.010+0000 7f3a7a9cd740 1 Processor -- start 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9c30 0x5650e21d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5650e21d1b10 con 0x5650e21ceda0 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5650e20e8f50 con 0x5650e21c9240 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650e21d1c90 con 0x5650e21c9c30 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7a9cd740 10 client.0 _ll_register_callbacks cb 0x5650e2124000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41424/0 (socket says 172.21.15.33:41424) 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 -- 172.21.15.33:0/3024649751 learned_addr learned my addr 172.21.15.33:0/3024649751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7517e640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9c30 0x5650e21d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a7597f640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 -- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9c30 msgr2=0x5650e21d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9c30 0x5650e21d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 -- 172.21.15.33:0/3024649751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 msgr2=0x5650e21d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 -- 172.21.15.33:0/3024649751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e1e413f0 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a76180640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da870 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800fd60 tx=0x7f3a68009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5592bfc4d50b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.011+0000 7f3a597fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a597fa640 1 -- 172.21.15.33:0/3024649751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68010c40 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a597fa640 1 -- 172.21.15.33:0/3024649751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a68010de0 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6008aab0 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a597fa640 1 -- 172.21.15.33:0/3024649751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a6800fe50 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 10 client.16071 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650e1e010f0 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a597fa640 1 -- 172.21.15.33:0/3024649751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6008aab0 con 0x5650e21ceda0 2024-04-24T23:14:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a597fa640 1 -- 172.21.15.33:0/3024649751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f3a68026070 con 0x5650e21ceda0 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 20 client.16071 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e21df1f0 0x5650e21e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 4 client.16071 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.012+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e213abf0 con 0x5650e21df1f0 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.013+0000 7f3a7597f640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e21df1f0 0x5650e21e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.013+0000 7f3a7597f640 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e21df1f0 0x5650e21e1680 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bace5e861a08f330 server_cookie=641eb2c8265db4c1 in_seq=0 out_seq=0 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.013+0000 7f3a597fa640 10 client.16071 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.013+0000 7f3a597fa640 1 -- 172.21.15.33:0/3024649751 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650e213abf0 con 0x5650e21df1f0 2024-04-24T23:14:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.013+0000 7f3a597fa640 10 client.16071 handle_command_reply: tid=0 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 client.16071 shutdown 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 msgr2=0x5650e21da870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da870 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800fd60 tx=0x7f3a68009b30 comp rx=0 tx=0).stop 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 shutdown_connections 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e21df1f0 0x5650e21e1680 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da870 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9c30 0x5650e21d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a597fa640 0 client.16071 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/3024649751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 >> 172.21.15.33:0/3024649751 conn(0x5650e21c48b0 msgr2=0x5650e21c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 shutdown_connections 2024-04-24T23:14:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/3024649751 wait complete. 2024-04-24T23:14:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.014+0000 7f3a7a9cd740 20 client.16071 trim_cache size 0 max 16384 2024-04-24T23:14:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 Processor -- start 2024-04-24T23:14:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 -- start start 2024-04-24T23:14:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9c30 0x5650e21cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701aaa70 con 0x5650e21ceda0 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701a8340 con 0x5650e21c9c30 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701a6190 con 0x5650e21c9240 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7597f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a76180640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a76180640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41428/0 (socket says 172.21.15.33:41428) 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7517e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9c30 0x5650e21cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a76180640 1 -- 172.21.15.33:0/783114401 learned_addr learned my addr 172.21.15.33:0/783114401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.033+0000 7f3a7597f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:50522/0 (socket says 172.21.15.33:50522) 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a7597f640 1 -- 172.21.15.33:0/783114401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9c30 msgr2=0x5650e21cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a7597f640 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9c30 0x5650e21cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a7597f640 1 -- 172.21.15.33:0/783114401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 msgr2=0x5650e21da830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a7597f640 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a7597f640 1 -- 172.21.15.33:0/783114401 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e21c4450 con 0x5650e21c9240 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a76180640 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a7597f640 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21cb090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x5650e21e1ec0 tx=0x7f3a6c0158c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e1f2d31e95f64a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a5b7fe640 1 -- 172.21.15.33:0/783114401 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c026030 con 0x5650e21c9240 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a5b7fe640 1 -- 172.21.15.33:0/783114401 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6c005020 con 0x5650e21c9240 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.034+0000 7f3a5b7fe640 1 -- 172.21.15.33:0/783114401 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0139f0 con 0x5650e21c9240 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/783114401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 msgr2=0x5650e21cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21cb090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x5650e21e1ec0 tx=0x7f3a6c0158c0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/783114401 shutdown_connections 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21ceda0 0x5650e21da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9c30 0x5650e21cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/783114401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21c9240 0x5650e21cb090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/783114401 >> 172.21.15.33:0/783114401 conn(0x5650e21c48b0 msgr2=0x5650e21c4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/783114401 shutdown_connections 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/783114401 wait complete. 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 -- start start 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.035+0000 7f3a7a9cd740 1 Processor -- start 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21ceda0 0x5650e21d9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701a6800 con 0x5650e21c9c30 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701a7c70 con 0x5650e21c9240 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701a6ed0 con 0x5650e21ceda0 2024-04-24T23:14:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7a9cd740 10 client.0 _ll_register_callbacks cb 0x7f3a68005ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7517e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7517e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21d8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41440/0 (socket says 172.21.15.33:41440) 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7517e640 1 -- 172.21.15.33:0/353936443 learned_addr learned my addr 172.21.15.33:0/353936443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a76180640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21ceda0 0x5650e21d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 -- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21ceda0 msgr2=0x5650e21d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21ceda0 0x5650e21d9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 -- 172.21.15.33:0/353936443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 msgr2=0x5650e21d8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21d8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 -- 172.21.15.33:0/353936443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e21d9610 con 0x5650e21c9240 2024-04-24T23:14:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7517e640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a76180640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21ceda0 0x5650e21d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a7597f640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d7b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5650e21ce640 tx=0x7f3a6c0152e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cb92b0d8eb0d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.036+0000 7f3a597fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a597fa640 1 -- 172.21.15.33:0/353936443 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c02f070 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6c005020 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a597fa640 1 -- 172.21.15.33:0/353936443 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a6c0051d0 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a597fa640 1 -- 172.21.15.33:0/353936443 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a6c026030 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 10 client.25613 fetch_fsmap learned FSMap version 26 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650e21da890 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a597fa640 1 -- 172.21.15.33:0/353936443 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6c005020 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a597fa640 1 -- 172.21.15.33:0/353936443 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f3a6c004b30 con 0x5650e21c9240 2024-04-24T23:14:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 10 client.25613 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 20 client.25613 populate_metadata read hostname 'smithi033' 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e2245c00 0x5650e2248020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 4 client.25613 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650e2134f40 con 0x5650e2245c00 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.037+0000 7f3a7517e640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e2245c00 0x5650e2248020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a597fa640 10 client.25613 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7517e640 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e2245c00 0x5650e2248020 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19843f420c996ae7 server_cookie=888dccd108ce7f07 in_seq=0 out_seq=0 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a597fa640 1 -- 172.21.15.33:0/353936443 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650e2134f40 con 0x5650e2245c00 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a597fa640 10 client.25613 handle_command_reply: tid=0 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 client.25613 shutdown 2024-04-24T23:14:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 msgr2=0x5650e21d7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d7b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5650e21ce640 tx=0x7f3a6c0152e0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 shutdown_connections 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5650e2245c00 0x5650e2248020 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e21ceda0 0x5650e21d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650e21c9c30 0x5650e21d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 --2- 172.21.15.33:0/353936443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5650e21c9240 0x5650e21d7b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a597fa640 0 client.25613 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 >> 172.21.15.33:0/353936443 conn(0x5650e21c48b0 msgr2=0x5650e21d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 shutdown_connections 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 1 -- 172.21.15.33:0/353936443 wait complete. 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.038+0000 7f3a7a9cd740 20 client.25613 trim_cache size 0 max 16384 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "862e0c86-5cb5-4a40-802e-ca0d095a4214": { 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "862e0c86-5cb5-4a40-802e-ca0d095a4214", 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:14:53.049 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:14:53.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a5c0a2c10 msgr2=0x7f3a5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a5c0a2c10 0x7f3a5c0a50d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3a68013a40 tx=0x7f3a68009c60 comp rx=0 tx=0).stop 2024-04-24T23:14:53.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 msgr2=0x7f3a70197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70197fe0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009230 tx=0x7f3a600042f0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 shutdown_connections 2024-04-24T23:14:53.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a5c0a2c10 0x7f3a5c0a50d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a70102230 0x7f3a7019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a70101870 0x7f3a70197fe0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 --2- 172.21.15.33:0/495265808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a70100680 0x7f3a70197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 >> 172.21.15.33:0/495265808 conn(0x7f3a700fbbf0 msgr2=0x7f3a700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 shutdown_connections 2024-04-24T23:14:53.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:14:53.039+0000 7f3a77be3640 1 -- 172.21.15.33:0/495265808 wait complete. 2024-04-24T23:14:53.052 INFO:tasks.cephfs.filesystem:scrub status for tag:862e0c86-5cb5-4a40-802e-ca0d095a4214 - {'path': '/', 'tag': '862e0c86-5cb5-4a40-802e-ca0d095a4214', 'options': 'recursive,force'} 2024-04-24T23:14:53.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:52 smithi033 ceph-mon[29201]: pgmap v437: 129 pgs: 129 active+clean; 77 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-04-24T23:14:53.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:52 smithi038 ceph-mon[32450]: pgmap v437: 129 pgs: 129 active+clean; 77 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-04-24T23:14:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:52 smithi117 ceph-mon[32529]: pgmap v437: 129 pgs: 129 active+clean; 77 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-04-24T23:14:53.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 149740 14.22 MB/sec warmup 58 sec latency 23.403 ms 2024-04-24T23:14:54.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:53 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:14:54.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:53 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:53 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:14:54.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:53 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:14:54.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:53 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:53 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:53 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:14:54.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:53 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:53 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:14:54.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:53 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:14:54.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:53 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:53 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 152764 14.29 MB/sec warmup 59 sec latency 10.441 ms 2024-04-24T23:14:54.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:53 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:14:54.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:53 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:53 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:14:54.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:53 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:14:54.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:53 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:54.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:53 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:54 smithi038 ceph-mon[32450]: pgmap v438: 129 pgs: 129 active+clean; 80 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-24T23:14:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:54 smithi117 ceph-mon[32529]: pgmap v438: 129 pgs: 129 active+clean; 80 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-24T23:14:55.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 155017 14.18 MB/sec warmup 60 sec latency 36.722 ms 2024-04-24T23:14:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:54 smithi033 ceph-mon[29201]: pgmap v438: 129 pgs: 129 active+clean; 80 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-24T23:14:56.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 157165 14.17 MB/sec warmup 61 sec latency 11.975 ms 2024-04-24T23:14:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:56 smithi038 ceph-mon[32450]: pgmap v439: 129 pgs: 129 active+clean; 80 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-04-24T23:14:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:56 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:56 smithi117 ceph-mon[32529]: pgmap v439: 129 pgs: 129 active+clean; 80 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-04-24T23:14:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:56 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:57.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 159849 14.21 MB/sec warmup 62 sec latency 89.416 ms 2024-04-24T23:14:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:56 smithi033 ceph-mon[29201]: pgmap v439: 129 pgs: 129 active+clean; 80 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-04-24T23:14:57.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:56 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:58.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:58.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:58.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:57 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:14:58.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:58.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:58.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:57 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:14:58.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 162041 14.10 MB/sec warmup 63 sec latency 37.632 ms 2024-04-24T23:14:58.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:58.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:58.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:57 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:14:59.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 164107 14.08 MB/sec warmup 64 sec latency 12.729 ms 2024-04-24T23:14:59.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:14:59 smithi033 ceph-mon[29201]: pgmap v440: 129 pgs: 129 active+clean; 82 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-04-24T23:14:59.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:14:59 smithi038 ceph-mon[32450]: pgmap v440: 129 pgs: 129 active+clean; 82 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-04-24T23:14:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:14:59 smithi117 ceph-mon[32529]: pgmap v440: 129 pgs: 129 active+clean; 82 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-04-24T23:15:00.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167035 14.15 MB/sec warmup 65 sec latency 14.919 ms 2024-04-24T23:15:01.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 169630 14.12 MB/sec warmup 66 sec latency 16.507 ms 2024-04-24T23:15:01.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:01 smithi033 ceph-mon[29201]: pgmap v441: 129 pgs: 129 active+clean; 85 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-04-24T23:15:01.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:01 smithi038 ceph-mon[32450]: pgmap v441: 129 pgs: 129 active+clean; 85 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-04-24T23:15:01.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:01 smithi117 ceph-mon[32529]: pgmap v441: 129 pgs: 129 active+clean; 85 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-04-24T23:15:02.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 171085 14.02 MB/sec warmup 67 sec latency 49.603 ms 2024-04-24T23:15:03.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 173978 14.07 MB/sec warmup 68 sec latency 19.411 ms 2024-04-24T23:15:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:03 smithi033 ceph-mon[29201]: pgmap v442: 129 pgs: 129 active+clean; 84 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 286 op/s 2024-04-24T23:15:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:15:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:15:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:03 smithi038 ceph-mon[32450]: pgmap v442: 129 pgs: 129 active+clean; 84 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 286 op/s 2024-04-24T23:15:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:15:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:15:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:03 smithi117 ceph-mon[32529]: pgmap v442: 129 pgs: 129 active+clean; 84 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 286 op/s 2024-04-24T23:15:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:15:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:15:04.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 176191 13.98 MB/sec warmup 69 sec latency 27.073 ms 2024-04-24T23:15:05.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 178105 13.96 MB/sec warmup 70 sec latency 19.596 ms 2024-04-24T23:15:05.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:05 smithi038 ceph-mon[32450]: pgmap v443: 129 pgs: 129 active+clean; 88 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-04-24T23:15:05.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:05 smithi117 ceph-mon[32529]: pgmap v443: 129 pgs: 129 active+clean; 88 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-04-24T23:15:05.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:05 smithi033 ceph-mon[29201]: pgmap v443: 129 pgs: 129 active+clean; 88 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-04-24T23:15:06.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 181072 14.01 MB/sec warmup 71 sec latency 17.431 ms 2024-04-24T23:15:07.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 182552 13.91 MB/sec warmup 72 sec latency 144.043 ms 2024-04-24T23:15:07.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:07 smithi038 ceph-mon[32450]: pgmap v444: 129 pgs: 129 active+clean; 88 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-04-24T23:15:07.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:07 smithi117 ceph-mon[32529]: pgmap v444: 129 pgs: 129 active+clean; 88 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-04-24T23:15:07.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:07 smithi033 ceph-mon[29201]: pgmap v444: 129 pgs: 129 active+clean; 88 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-04-24T23:15:08.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 184930 13.91 MB/sec warmup 73 sec latency 57.864 ms 2024-04-24T23:15:09.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 187642 13.90 MB/sec warmup 74 sec latency 16.606 ms 2024-04-24T23:15:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:09 smithi038 ceph-mon[32450]: pgmap v445: 129 pgs: 129 active+clean; 89 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-04-24T23:15:09.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:09 smithi117 ceph-mon[32529]: pgmap v445: 129 pgs: 129 active+clean; 89 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-04-24T23:15:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:09 smithi033 ceph-mon[29201]: pgmap v445: 129 pgs: 129 active+clean; 89 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-04-24T23:15:10.437 INFO:tasks.workunit.client.0.smithi033.stdout: 1 189494 13.86 MB/sec warmup 75 sec latency 23.926 ms 2024-04-24T23:15:11.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 192166 13.86 MB/sec warmup 76 sec latency 9.199 ms 2024-04-24T23:15:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:11 smithi038 ceph-mon[32450]: pgmap v446: 129 pgs: 129 active+clean; 90 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 230 op/s 2024-04-24T23:15:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:11 smithi117 ceph-mon[32529]: pgmap v446: 129 pgs: 129 active+clean; 90 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 230 op/s 2024-04-24T23:15:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:11 smithi033 ceph-mon[29201]: pgmap v446: 129 pgs: 129 active+clean; 90 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 230 op/s 2024-04-24T23:15:12.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 195058 13.91 MB/sec warmup 77 sec latency 49.758 ms 2024-04-24T23:15:13.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 196304 13.78 MB/sec warmup 78 sec latency 88.868 ms 2024-04-24T23:15:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:13 smithi038 ceph-mon[32450]: pgmap v447: 129 pgs: 129 active+clean; 93 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-24T23:15:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:13 smithi117 ceph-mon[32529]: pgmap v447: 129 pgs: 129 active+clean; 93 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-24T23:15:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:13 smithi033 ceph-mon[29201]: pgmap v447: 129 pgs: 129 active+clean; 93 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-24T23:15:14.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 199234 13.82 MB/sec warmup 79 sec latency 10.751 ms 2024-04-24T23:15:15.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 202324 13.87 MB/sec warmup 80 sec latency 14.956 ms 2024-04-24T23:15:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:15 smithi038 ceph-mon[32450]: pgmap v448: 129 pgs: 129 active+clean; 92 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-04-24T23:15:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:15 smithi117 ceph-mon[32529]: pgmap v448: 129 pgs: 129 active+clean; 92 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-04-24T23:15:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:15 smithi033 ceph-mon[29201]: pgmap v448: 129 pgs: 129 active+clean; 92 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-04-24T23:15:16.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 204583 13.80 MB/sec warmup 81 sec latency 12.495 ms 2024-04-24T23:15:17.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 206556 13.79 MB/sec warmup 82 sec latency 12.611 ms 2024-04-24T23:15:17.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:17 smithi038 ceph-mon[32450]: pgmap v449: 129 pgs: 129 active+clean; 96 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-04-24T23:15:17.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:17 smithi117 ceph-mon[32529]: pgmap v449: 129 pgs: 129 active+clean; 96 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-04-24T23:15:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:17 smithi033 ceph-mon[29201]: pgmap v449: 129 pgs: 129 active+clean; 96 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-04-24T23:15:18.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 209066 13.79 MB/sec warmup 83 sec latency 58.215 ms 2024-04-24T23:15:19.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 211054 13.74 MB/sec warmup 84 sec latency 15.364 ms 2024-04-24T23:15:19.478 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:19 smithi117 ceph-mon[32529]: pgmap v450: 129 pgs: 129 active+clean; 96 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 271 op/s 2024-04-24T23:15:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:19 smithi038 ceph-mon[32450]: pgmap v450: 129 pgs: 129 active+clean; 96 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 271 op/s 2024-04-24T23:15:19.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:19 smithi033 ceph-mon[29201]: pgmap v450: 129 pgs: 129 active+clean; 96 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 271 op/s 2024-04-24T23:15:20.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213495 13.75 MB/sec warmup 85 sec latency 12.985 ms 2024-04-24T23:15:21.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 216431 13.79 MB/sec warmup 86 sec latency 7.456 ms 2024-04-24T23:15:21.473 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:21 smithi038 ceph-mon[32450]: pgmap v451: 129 pgs: 129 active+clean; 96 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2024-04-24T23:15:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:21 smithi117 ceph-mon[32529]: pgmap v451: 129 pgs: 129 active+clean; 96 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2024-04-24T23:15:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:21 smithi033 ceph-mon[29201]: pgmap v451: 129 pgs: 129 active+clean; 96 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2024-04-24T23:15:22.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 218117 13.71 MB/sec warmup 87 sec latency 17.393 ms 2024-04-24T23:15:23.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:23.053 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 -- 172.21.15.33:0/1508022640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101870 msgr2=0x7fa824101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 --2- 172.21.15.33:0/1508022640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101870 0x7fa824101cf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009900 tx=0x7fa80c02a410 comp rx=0 tx=0).stop 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 -- 172.21.15.33:0/1508022640 shutdown_connections 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 --2- 172.21.15.33:0/1508022640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824102230 0x7fa824108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 --2- 172.21.15.33:0/1508022640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101870 0x7fa824101cf0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 --2- 172.21.15.33:0/1508022640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa824100680 0x7fa824100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 -- 172.21.15.33:0/1508022640 >> 172.21.15.33:0/1508022640 conn(0x7fa8240fbbf0 msgr2=0x7fa8240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 -- 172.21.15.33:0/1508022640 shutdown_connections 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.158+0000 7fa82a76a640 1 -- 172.21.15.33:0/1508022640 wait complete. 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 Processor -- start 2024-04-24T23:15:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 -- start start 2024-04-24T23:15:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa824100680 0x7fa824197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824101870 0x7fa824198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 0x7fa82419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410c2e0 con 0x7fa824102230 2024-04-24T23:15:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410c160 con 0x7fa824101870 2024-04-24T23:15:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa82a76a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410c460 con 0x7fa824100680 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa828d07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 0x7fa82419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.159+0000 7fa823fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa824100680 0x7fa824197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824101870 0x7fa824198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 0x7fa82419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48258/0 (socket says 172.21.15.33:48258) 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 -- 172.21.15.33:0/2043836243 learned_addr learned my addr 172.21.15.33:0/2043836243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 -- 172.21.15.33:0/2043836243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa824100680 msgr2=0x7fa824197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa824100680 0x7fa824197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 -- 172.21.15.33:0/2043836243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824101870 msgr2=0x7fa824198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824101870 0x7fa824198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 -- 172.21.15.33:0/2043836243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419f6f0 con 0x7fa824102230 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa8237fe640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824101870 0x7fa824198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa828d07640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 0x7fa82419f080 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00eb10 tx=0x7fa81c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa8217fa640 1 -- 172.21.15.33:0/2043836243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004b20 con 0x7fa824102230 2024-04-24T23:15:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa8217fa640 1 -- 172.21.15.33:0/2043836243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81c004cc0 con 0x7fa824102230 2024-04-24T23:15:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa8217fa640 1 -- 172.21.15.33:0/2043836243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c010760 con 0x7fa824102230 2024-04-24T23:15:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419f9e0 con 0x7fa824102230 2024-04-24T23:15:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.160+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82419ff00 con 0x7fa824102230 2024-04-24T23:15:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.162+0000 7fa8217fa640 1 -- 172.21.15.33:0/2043836243 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa81c005020 con 0x7fa824102230 2024-04-24T23:15:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 Processor -- start 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 -- start start 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218d650 0x55e92218da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e922192b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e922195640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9220ed6b0 con 0x55e92218d650 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e922095880 con 0x55e92218e040 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa82d47e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9220e9670 con 0x55e9221931b0 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218d650 0x55e92218da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e922192b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e922192b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46604/0 (socket says 172.21.15.33:46604) 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218d650 0x55e92218da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48262/0 (socket says 172.21.15.33:48262) 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 -- 172.21.15.33:0/3117954396 learned_addr learned my addr 172.21.15.33:0/3117954396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa8237fe640 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e922195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa8217fa640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8100a2c10 0x7fa8100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 -- 172.21.15.33:0/3117954396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 msgr2=0x55e922195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e922195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 -- 172.21.15.33:0/3117954396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218d650 msgr2=0x55e92218da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218d650 0x55e92218da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa823fff640 1 -- 172.21.15.33:0/3117954396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e921ecb7d0 con 0x55e92218e040 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa828d07640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8100a2c10 0x7fa8100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.163+0000 7fa8217fa640 1 -- 172.21.15.33:0/2043836243 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa81c0cf900 con 0x7fa824102230 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa823fff640 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e922192b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa814009ca0 tx=0x7fa814011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755ce46075bbb69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3117954396 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814004220 con 0x55e92218e040 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa828d07640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8100a2c10 0x7fa8100a50d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0987a0 tx=0x7fa81c066a50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3117954396 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8140043c0 con 0x55e92218e040 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3117954396 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814019890 con 0x55e92218e040 2024-04-24T23:15:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa82d47e740 1 -- 172.21.15.33:0/3117954396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 msgr2=0x55e922192b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.164+0000 7fa82d47e740 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e922192b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa814009ca0 tx=0x7fa814011810 comp rx=0 tx=0).stop 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- 172.21.15.33:0/3117954396 shutdown_connections 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e922195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e922192b40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 --2- 172.21.15.33:0/3117954396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218d650 0x55e92218da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- 172.21.15.33:0/3117954396 >> 172.21.15.33:0/3117954396 conn(0x55e9221848b0 msgr2=0x55e922184cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- 172.21.15.33:0/3117954396 shutdown_connections 2024-04-24T23:15:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- 172.21.15.33:0/3117954396 wait complete. 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- start start 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 Processor -- start 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e9221979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218e040 0x55e922197f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e92219ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e922196160 con 0x55e92218e040 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e9220a8f50 con 0x55e92218d650 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9221962e0 con 0x55e9221931b0 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.165+0000 7fa82d47e740 10 client.0 _ll_register_callbacks cb 0x55e9220e4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e9221979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218e040 0x55e922197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e9221979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46614/0 (socket says 172.21.15.33:46614) 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8237fe640 1 -- 172.21.15.33:0/860122412 learned_addr learned my addr 172.21.15.33:0/860122412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8237fe640 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e92219ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 -- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 msgr2=0x55e92219ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e92219ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 -- 172.21.15.33:0/860122412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 msgr2=0x55e9221979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e9221979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 -- 172.21.15.33:0/860122412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e921e013f0 con 0x55e92218e040 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa823fff640 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218e040 0x55e922197f30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55e922185820 tx=0x7fa814003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f359c207469459 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8037fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8037fe640 1 -- 172.21.15.33:0/860122412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814003880 con 0x55e92218e040 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8037fe640 1 -- 172.21.15.33:0/860122412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa814003a20 con 0x55e92218e040 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8140043c0 con 0x55e92218e040 2024-04-24T23:15:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa8037fe640 1 -- 172.21.15.33:0/860122412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa814033050 con 0x55e92218e040 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa82d47e740 10 client.16089 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.166+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e921dc10f0 con 0x55e92218e040 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa8037fe640 1 -- 172.21.15.33:0/860122412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8140043c0 con 0x55e92218e040 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa8037fe640 1 -- 172.21.15.33:0/860122412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa814004b00 con 0x55e92218e040 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa82d47e740 10 client.16089 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa82d47e740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa82d47e740 20 client.16089 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa82d47e740 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9221a3630 0x55e9221a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa82d47e740 4 client.16089 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9220fabf0 con 0x55e9221a3630 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa8237fe640 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9221a3630 0x55e9221a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa8237fe640 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9221a3630 0x55e9221a5a80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd83894c974bed70 server_cookie=8e732e0e1de22d3 in_seq=0 out_seq=0 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.167+0000 7fa8037fe640 10 client.16089 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa8037fe640 1 -- 172.21.15.33:0/860122412 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9220fabf0 con 0x55e9221a3630 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa8037fe640 10 client.16089 handle_command_reply: tid=0 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 client.16089 shutdown 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218e040 msgr2=0x55e922197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218e040 0x55e922197f30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55e922185820 tx=0x7fa814003d40 comp rx=0 tx=0).stop 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 shutdown_connections 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9221a3630 0x55e9221a5a80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9221931b0 0x55e92219ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92218e040 0x55e922197f30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa8037fe640 0 client.16089 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 --2- 172.21.15.33:0/860122412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e9221979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 >> 172.21.15.33:0/860122412 conn(0x55e9221848b0 msgr2=0x55e922183af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.168+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 shutdown_connections 2024-04-24T23:15:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.169+0000 7fa82d47e740 1 -- 172.21.15.33:0/860122412 wait complete. 2024-04-24T23:15:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.169+0000 7fa82d47e740 20 client.16089 trim_cache size 0 max 16384 2024-04-24T23:15:23.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.186+0000 7fa82d47e740 1 Processor -- start 2024-04-24T23:15:23.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.186+0000 7fa82d47e740 1 -- start start 2024-04-24T23:15:23.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.186+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e92218f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.186+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 0x55e92218f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.186+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.186+0000 7fa82d47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241aaa60 con 0x55e9221931b0 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa82d47e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241a8330 con 0x55e92218d650 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa82d47e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241a6180 con 0x55e92218e040 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 0x55e92218f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 0x55e92218f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33546/0 (socket says 172.21.15.33:33546) 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 -- 172.21.15.33:0/1599369360 learned_addr learned my addr 172.21.15.33:0/1599369360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa8237fe640 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e92218f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 -- 172.21.15.33:0/1599369360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 msgr2=0x55e92218f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e92218f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 -- 172.21.15.33:0/1599369360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 msgr2=0x55e92219ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 -- 172.21.15.33:0/1599369360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e922196040 con 0x55e92218e040 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa8237fe640 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa823fff640 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 0x55e92218f900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa814005f90 tx=0x7fa814005fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cb6058c917ce2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa819ffb640 1 -- 172.21.15.33:0/1599369360 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814019790 con 0x55e92218e040 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa819ffb640 1 -- 172.21.15.33:0/1599369360 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81403c070 con 0x55e92218e040 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.187+0000 7fa819ffb640 1 -- 172.21.15.33:0/1599369360 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814003ae0 con 0x55e92218e040 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 -- 172.21.15.33:0/1599369360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 msgr2=0x55e92218f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 0x55e92218f900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa814005f90 tx=0x7fa814005fc0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 -- 172.21.15.33:0/1599369360 shutdown_connections 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218e040 0x55e92218f900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 --2- 172.21.15.33:0/1599369360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218d650 0x55e92218f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 -- 172.21.15.33:0/1599369360 >> 172.21.15.33:0/1599369360 conn(0x55e9221848b0 msgr2=0x55e922184c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 -- 172.21.15.33:0/1599369360 shutdown_connections 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 -- 172.21.15.33:0/1599369360 wait complete. 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.188+0000 7fa82d47e740 1 -- start start 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 Processor -- start 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218d650 0x55e92219c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e92219c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241a67f0 con 0x55e9221931b0 2024-04-24T23:15:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241a7c60 con 0x55e92218e040 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241a6ec0 con 0x55e92218d650 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa82d47e740 10 client.0 _ll_register_callbacks cb 0x7fa814005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218d650 0x55e92219c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa823fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e92219c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa823fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e92219c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46620/0 (socket says 172.21.15.33:46620) 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa823fff640 1 -- 172.21.15.33:0/33524084 learned_addr learned my addr 172.21.15.33:0/33524084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 -- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218d650 msgr2=0x55e92219c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218d650 0x55e92219c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 -- 172.21.15.33:0/33524084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 msgr2=0x55e92219c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e92219c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa8237fe640 1 -- 172.21.15.33:0/33524084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92219dd50 con 0x55e9221931b0 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.189+0000 7fa823fff640 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e92219c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8237fe640 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219d760 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006010 tx=0x7fa80c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cdfd900e4f6352 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8037fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8037fe640 1 -- 172.21.15.33:0/33524084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c051070 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8037fe640 1 -- 172.21.15.33:0/33524084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa80c005040 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81403c070 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8037fe640 1 -- 172.21.15.33:0/33524084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa80c060050 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 10 client.16095 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e92219f0d0 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8037fe640 1 -- 172.21.15.33:0/33524084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81403c070 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa8037fe640 1 -- 172.21.15.33:0/33524084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa80c04dce0 con 0x55e9221931b0 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 10 client.16095 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:23.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 20 client.16095 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e92220a1d0 0x55e92220c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 4 client.16095 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.190+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9220f4f40 con 0x55e92220a1d0 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa823fff640 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e92220a1d0 0x55e92220c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa823fff640 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e92220a1d0 0x55e92220c5f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93870dd2411f1938 server_cookie=98164fda61524c35 in_seq=0 out_seq=0 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa8037fe640 10 client.16095 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa8037fe640 1 -- 172.21.15.33:0/33524084 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9220f4f40 con 0x55e92220a1d0 2024-04-24T23:15:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa8037fe640 10 client.16095 handle_command_reply: tid=0 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 client.16095 shutdown 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 msgr2=0x55e92219d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219d760 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006010 tx=0x7fa80c005dd0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 shutdown_connections 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e92220a1d0 0x55e92220c5f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9221931b0 0x55e92219d760 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e92218e040 0x55e92219c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 --2- 172.21.15.33:0/33524084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e92218d650 0x55e92219c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa8037fe640 0 client.16095 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 >> 172.21.15.33:0/33524084 conn(0x55e9221848b0 msgr2=0x55e9221979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 shutdown_connections 2024-04-24T23:15:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 1 -- 172.21.15.33:0/33524084 wait complete. 2024-04-24T23:15:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.191+0000 7fa82d47e740 20 client.16095 trim_cache size 0 max 16384 2024-04-24T23:15:23.203 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:15:23.203 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:15:23.204 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:15:23.204 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:15:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8100a2c10 msgr2=0x7fa8100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8100a2c10 0x7fa8100a50d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0987a0 tx=0x7fa81c066a50 comp rx=0 tx=0).stop 2024-04-24T23:15:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 msgr2=0x7fa82419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 0x7fa82419f080 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00eb10 tx=0x7fa81c00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 shutdown_connections 2024-04-24T23:15:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8100a2c10 0x7fa8100a50d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824102230 0x7fa82419f080 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa824101870 0x7fa824198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 --2- 172.21.15.33:0/2043836243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa824100680 0x7fa824197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 >> 172.21.15.33:0/2043836243 conn(0x7fa8240fbbf0 msgr2=0x7fa8240fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 shutdown_connections 2024-04-24T23:15:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.192+0000 7fa82a76a640 1 -- 172.21.15.33:0/2043836243 wait complete. 2024-04-24T23:15:23.207 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:15:23.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:15:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 -- 172.21.15.33:0/1708897423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4218101890 msgr2=0x7f4218101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 --2- 172.21.15.33:0/1708897423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4218101890 0x7f4218101d10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4208009900 tx=0x7f420802a410 comp rx=0 tx=0).stop 2024-04-24T23:15:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 -- 172.21.15.33:0/1708897423 shutdown_connections 2024-04-24T23:15:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 --2- 172.21.15.33:0/1708897423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4218102250 0x7f4218108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 --2- 172.21.15.33:0/1708897423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4218101890 0x7f4218101d10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 --2- 172.21.15.33:0/1708897423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42181006a0 0x7f4218100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 -- 172.21.15.33:0/1708897423 >> 172.21.15.33:0/1708897423 conn(0x7f42180fbc30 msgr2=0x7f42180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 -- 172.21.15.33:0/1708897423 shutdown_connections 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.309+0000 7f421fce1640 1 -- 172.21.15.33:0/1708897423 wait complete. 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 Processor -- start 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 -- start start 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 0x7f421810b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4218101890 0x7f4218109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4218102250 0x7f4218109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f421810d440 con 0x7f42181006a0 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f421810d2c0 con 0x7f4218102250 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421fce1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f421810d5c0 con 0x7f4218101890 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 0x7f421810b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 0x7f421810b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48312/0 (socket says 172.21.15.33:48312) 2024-04-24T23:15:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 -- 172.21.15.33:0/3419907316 learned_addr learned my addr 172.21.15.33:0/3419907316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421e27e640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4218102250 0x7f4218109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421d27c640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4218101890 0x7f4218109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 -- 172.21.15.33:0/3419907316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4218101890 msgr2=0x7f4218109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4218101890 0x7f4218109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 -- 172.21.15.33:0/3419907316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4218102250 msgr2=0x7f4218109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4218102250 0x7f4218109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.310+0000 7f421da7d640 1 -- 172.21.15.33:0/3419907316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421810a430 con 0x7f42181006a0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.311+0000 7f421da7d640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 0x7f421810b020 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9c0 tx=0x7f421400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.311+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421400cde0 con 0x7f42181006a0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.311+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4214010910 con 0x7f42181006a0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.311+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42140189c0 con 0x7f42181006a0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.311+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42181a1bc0 con 0x7f42181006a0 2024-04-24T23:15:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.311+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42181a2130 con 0x7f42181006a0 2024-04-24T23:15:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.312+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4218107a40 con 0x7f42181006a0 2024-04-24T23:15:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.313+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4214010ac0 con 0x7f42181006a0 2024-04-24T23:15:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.313+0000 7f420effd640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42040a2c10 0x7f42040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.314+0000 7f421d27c640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42040a2c10 0x7f42040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.314+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f42140cf8a0 con 0x7f42181006a0 2024-04-24T23:15:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.314+0000 7f421d27c640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42040a2c10 0x7f42040a50d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42080314e0 tx=0x7f4208000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.318+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4214053740 con 0x7f42181006a0 2024-04-24T23:15:23.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 220371 13.71 MB/sec warmup 88 sec latency 116.199 ms 2024-04-24T23:15:23.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:23 smithi033 ceph-mon[29201]: pgmap v452: 129 pgs: 129 active+clean; 101 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2024-04-24T23:15:23.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.543+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4218062340 con 0x7f42181006a0 2024-04-24T23:15:23.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.543+0000 7f420effd640 1 -- 172.21.15.33:0/3419907316 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11860 (secure 0 0 0) 0x7f421409c470 con 0x7f42181006a0 2024-04-24T23:15:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:15:23.546 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:13:57.199404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:15:23.546 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 26 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42040a2c10 msgr2=0x7f42040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42040a2c10 0x7f42040a50d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42080314e0 tx=0x7f4208000bc0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 msgr2=0x7f421810b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 0x7f421810b020 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9c0 tx=0x7f421400ee80 comp rx=0 tx=0).stop 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 shutdown_connections 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42040a2c10 0x7f42040a50d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4218102250 0x7f4218109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4218101890 0x7f4218109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 --2- 172.21.15.33:0/3419907316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42181006a0 0x7f421810b020 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 >> 172.21.15.33:0/3419907316 conn(0x7f42180fbc30 msgr2=0x7f4218104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 shutdown_connections 2024-04-24T23:15:23.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.546+0000 7f421fce1640 1 -- 172.21.15.33:0/3419907316 wait complete. 2024-04-24T23:15:23.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:15:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 -- 172.21.15.33:0/2490604183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01029a0 msgr2=0x7f31b0102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 --2- 172.21.15.33:0/2490604183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01029a0 0x7f31b0102e20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009900 tx=0x7f31a002a410 comp rx=0 tx=0).stop 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 -- 172.21.15.33:0/2490604183 shutdown_connections 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 --2- 172.21.15.33:0/2490604183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31b0103360 0x7f31b0109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 --2- 172.21.15.33:0/2490604183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01029a0 0x7f31b0102e20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 --2- 172.21.15.33:0/2490604183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31b01017b0 0x7f31b0101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.663+0000 7f31b785d640 1 -- 172.21.15.33:0/2490604183 >> 172.21.15.33:0/2490604183 conn(0x7f31b00fcd20 msgr2=0x7f31b00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 -- 172.21.15.33:0/2490604183 shutdown_connections 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 -- 172.21.15.33:0/2490604183 wait complete. 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 Processor -- start 2024-04-24T23:15:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 -- start start 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 0x7f31b0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31b01029a0 0x7f31b0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31b0103360 0x7f31b019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31b010d430 con 0x7f31b01017b0 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31b010d2b0 con 0x7f31b01029a0 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.664+0000 7f31b785d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31b010d5b0 con 0x7f31b0103360 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 0x7f31b0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 0x7f31b0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48336/0 (socket says 172.21.15.33:48336) 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 -- 172.21.15.33:0/142436892 learned_addr learned my addr 172.21.15.33:0/142436892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b4df8640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31b01029a0 0x7f31b0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31b0103360 0x7f31b019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 -- 172.21.15.33:0/142436892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31b0103360 msgr2=0x7f31b019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31b0103360 0x7f31b019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 -- 172.21.15.33:0/142436892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31b01029a0 msgr2=0x7f31b0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31b01029a0 0x7f31b0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 -- 172.21.15.33:0/142436892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b019f6d0 con 0x7f31b01017b0 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b55f9640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 0x7f31b0197ab0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00e9c0 tx=0x7f31ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31a67fc640 1 -- 172.21.15.33:0/142436892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00cd80 con 0x7f31b01017b0 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b019f9c0 con 0x7f31b01017b0 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31a67fc640 1 -- 172.21.15.33:0/142436892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac0035f0 con 0x7f31b01017b0 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31a67fc640 1 -- 172.21.15.33:0/142436892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac010690 con 0x7f31b01017b0 2024-04-24T23:15:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.665+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b019ff30 con 0x7f31b01017b0 2024-04-24T23:15:23.672 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:23 smithi038 ceph-mon[32450]: pgmap v452: 129 pgs: 129 active+clean; 101 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2024-04-24T23:15:23.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:23 smithi117 ceph-mon[32529]: pgmap v452: 129 pgs: 129 active+clean; 101 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2024-04-24T23:15:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 Processor -- start 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31a67fc640 1 -- 172.21.15.33:0/142436892 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f31ac0037a0 con 0x7f31b01017b0 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 -- start start 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a3ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ffc20 0x55b93a404720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a407220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b93a363670 con 0x55b93a3ff230 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b93a30b970 con 0x55b93a3ffc20 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31ba56a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b93a35f630 con 0x55b93a404d90 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a3ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a3ff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48342/0 (socket says 172.21.15.33:48342) 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 -- 172.21.15.33:0/1806446474 learned_addr learned my addr 172.21.15.33:0/1806446474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b55f9640 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ffc20 0x55b93a404720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b4df8640 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a407220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 -- 172.21.15.33:0/1806446474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 msgr2=0x55b93a407220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a407220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 -- 172.21.15.33:0/1806446474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ffc20 msgr2=0x55b93a404720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ffc20 0x55b93a404720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.668+0000 7f31b5dfa640 1 -- 172.21.15.33:0/1806446474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93a32f400 con 0x55b93a3ff230 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31a67fc640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f319c0a2b90 0x7f319c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a3ff650 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009fd0 tx=0x7f31a80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7c12a9396a805e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31b55f9640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f319c0a2b90 0x7f319c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f319b7fe640 1 -- 172.21.15.33:0/1806446474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8004d20 con 0x55b93a3ff230 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31a67fc640 1 -- 172.21.15.33:0/142436892 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f31ac0cf7a0 con 0x7f31b01017b0 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f319b7fe640 1 -- 172.21.15.33:0/1806446474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8004050 con 0x55b93a3ff230 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f319b7fe640 1 -- 172.21.15.33:0/1806446474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8005260 con 0x55b93a3ff230 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31ba56a740 1 -- 172.21.15.33:0/1806446474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 msgr2=0x55b93a3ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31b55f9640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f319c0a2b90 0x7f319c0a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0cd280 tx=0x7f31ac0d5550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.669+0000 7f31ba56a740 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a3ff650 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009fd0 tx=0x7f31a80118d0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- 172.21.15.33:0/1806446474 shutdown_connections 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a407220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ffc20 0x55b93a404720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 --2- 172.21.15.33:0/1806446474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a3ff650 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- 172.21.15.33:0/1806446474 >> 172.21.15.33:0/1806446474 conn(0x55b93a3fa8a0 msgr2=0x55b93a3faca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- 172.21.15.33:0/1806446474 shutdown_connections 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- 172.21.15.33:0/1806446474 wait complete. 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- start start 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 Processor -- start 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a409310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a409850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a404d90 0x55b93a410890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b93a407d40 con 0x55b93a3ff230 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b93a31f050 con 0x55b93a404d90 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b93a407ec0 con 0x55b93a3ffc20 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.670+0000 7f31ba56a740 10 client.0 _ll_register_callbacks cb 0x55b93a076be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a409310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a409310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48352/0 (socket says 172.21.15.33:48352) 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a409850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b5dfa640 1 -- 172.21.15.33:0/3705728846 learned_addr learned my addr 172.21.15.33:0/3705728846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a404d90 0x55b93a410890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 -- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a404d90 msgr2=0x55b93a410890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a404d90 0x55b93a410890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 -- 172.21.15.33:0/3705728846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 msgr2=0x55b93a409310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a409310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 -- 172.21.15.33:0/3705728846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93a2f0dc0 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a404d90 0x55b93a410890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31b4df8640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a409850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31a0031440 tx=0x7f31a0032ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d22d0379cd1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f3198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f3198ff9640 1 -- 172.21.15.33:0/3705728846 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0035040 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f3198ff9640 1 -- 172.21.15.33:0/3705728846 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f31a0031970 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a8004050 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f3198ff9640 1 -- 172.21.15.33:0/3705728846 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f31a004d050 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31ba56a740 10 client.25675 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.671+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b93a0773f0 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f3198ff9640 1 -- 172.21.15.33:0/3705728846 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8004050 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f3198ff9640 1 -- 172.21.15.33:0/3705728846 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f31a0004780 con 0x55b93a3ffc20 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31ba56a740 10 client.25675 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31ba56a740 10 client.25675 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31ba56a740 20 client.25675 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a415150 0x55b93a4175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31ba56a740 4 client.25675 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93a370bb0 con 0x55b93a415150 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a415150 0x55b93a4175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a415150 0x55b93a4175d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f70c035e8f36245c server_cookie=37e4667b08774d2f in_seq=0 out_seq=0 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.672+0000 7f3198ff9640 10 client.25675 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f3198ff9640 1 -- 172.21.15.33:0/3705728846 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b93a370bb0 con 0x55b93a415150 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f3198ff9640 10 client.25675 handle_command_reply: tid=0 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 client.25675 shutdown 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 msgr2=0x55b93a409850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a409850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31a0031440 tx=0x7f31a0032ed0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 shutdown_connections 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a415150 0x55b93a4175d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a404d90 0x55b93a410890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a409850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3705728846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ff230 0x55b93a409310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f3198ff9640 0 client.25675 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 >> 172.21.15.33:0/3705728846 conn(0x55b93a3fa8a0 msgr2=0x55b93a3f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 shutdown_connections 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 1 -- 172.21.15.33:0/3705728846 wait complete. 2024-04-24T23:15:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.673+0000 7f31ba56a740 20 client.25675 trim_cache size 0 max 16384 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 Processor -- start 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 -- start start 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a3fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a3fe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a404d90 0x55b93a410850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01accb0 con 0x55b93a404d90 2024-04-24T23:15:23.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01aa580 con 0x55b93a3ff230 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31ba56a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01a83d0 con 0x55b93a3ffc20 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31b4df8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a3fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a3fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31b4df8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a3fe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33604/0 (socket says 172.21.15.33:33604) 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a3fdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46686/0 (socket says 172.21.15.33:46686) 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31b4df8640 1 -- 172.21.15.33:0/3349158630 learned_addr learned my addr 172.21.15.33:0/3349158630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.688+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a404d90 0x55b93a410850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b5dfa640 1 -- 172.21.15.33:0/3349158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 msgr2=0x55b93a3fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a3fe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b5dfa640 1 -- 172.21.15.33:0/3349158630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a404d90 msgr2=0x55b93a410850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a404d90 0x55b93a410850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b5dfa640 1 -- 172.21.15.33:0/3349158630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93a1417d0 con 0x55b93a3ff230 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b4df8640 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a3fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a3fdf20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31a8014040 tx=0x7f31a8004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5481ad77c941e9f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f319affd640 1 -- 172.21.15.33:0/3349158630 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8009be0 con 0x55b93a3ff230 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f319affd640 1 -- 172.21.15.33:0/3349158630 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8004530 con 0x55b93a3ff230 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.689+0000 7f319affd640 1 -- 172.21.15.33:0/3349158630 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800f4e0 con 0x55b93a3ff230 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- 172.21.15.33:0/3349158630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 msgr2=0x55b93a3fdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a3fdf20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31a8014040 tx=0x7f31a8004870 comp rx=0 tx=0).stop 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- 172.21.15.33:0/3349158630 shutdown_connections 2024-04-24T23:15:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a404d90 0x55b93a410850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a3ffc20 0x55b93a3fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- 172.21.15.33:0/3349158630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a3fdf20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- 172.21.15.33:0/3349158630 >> 172.21.15.33:0/3349158630 conn(0x55b93a3fa8a0 msgr2=0x55b93a3fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- 172.21.15.33:0/3349158630 shutdown_connections 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- 172.21.15.33:0/3349158630 wait complete. 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- start start 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 Processor -- start 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a40e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 0x55b93a40e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a40f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01a8a40 con 0x55b93a3ffc20 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01a9eb0 con 0x55b93a3ff230 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.690+0000 7f31ba56a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01a9110 con 0x55b93a404d90 2024-04-24T23:15:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31ba56a740 10 client.0 _ll_register_callbacks cb 0x7f31a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 0x55b93a40e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b5dfa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a40e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 0x55b93a40e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48378/0 (socket says 172.21.15.33:48378) 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 -- 172.21.15.33:0/2940648667 learned_addr learned my addr 172.21.15.33:0/2940648667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a40f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 -- 172.21.15.33:0/2940648667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 msgr2=0x55b93a40f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a40f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 -- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 msgr2=0x55b93a40e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a40e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 -- 172.21.15.33:0/2940648667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93a40fc20 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31b4df8640 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 0x55b93a40e690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f31a00314e0 tx=0x7f31a00049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cce431edffab059 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f3198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f3198ff9640 1 -- 172.21.15.33:0/2940648667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0056060 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a8004530 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f3198ff9640 1 -- 172.21.15.33:0/2940648667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f31a0031980 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f3198ff9640 1 -- 172.21.15.33:0/2940648667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f31a0035040 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31ba56a740 10 client.16119 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.691+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b93a484130 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f3198ff9640 1 -- 172.21.15.33:0/2940648667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8004530 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f3198ff9640 1 -- 172.21.15.33:0/2940648667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f31a00052a0 con 0x55b93a3ffc20 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31ba56a740 10 client.16119 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31ba56a740 10 client.16119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31ba56a740 20 client.16119 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31ba56a740 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a484ad0 0x55b93a486ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31ba56a740 4 client.16119 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b93a36af00 con 0x55b93a484ad0 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.692+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a484ad0 0x55b93a486ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f3198ff9640 10 client.16119 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31b5dfa640 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a484ad0 0x55b93a486ef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cff7adefdb9f072 server_cookie=4255affb05d39a32 in_seq=0 out_seq=0 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f3198ff9640 1 -- 172.21.15.33:0/2940648667 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b93a36af00 con 0x55b93a484ad0 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f3198ff9640 10 client.16119 handle_command_reply: tid=0 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 client.16119 shutdown 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 msgr2=0x55b93a40e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 0x55b93a40e690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f31a00314e0 tx=0x7f31a00049b0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 shutdown_connections 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b93a484ad0 0x55b93a486ef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b93a404d90 0x55b93a40f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b93a3ffc20 0x55b93a40e690 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f3198ff9640 0 client.16119 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:23.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 --2- 172.21.15.33:0/2940648667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b93a3ff230 0x55b93a40e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 >> 172.21.15.33:0/2940648667 conn(0x55b93a3fa8a0 msgr2=0x55b93a3fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 shutdown_connections 2024-04-24T23:15:23.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 1 -- 172.21.15.33:0/2940648667 wait complete. 2024-04-24T23:15:23.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.693+0000 7f31ba56a740 20 client.16119 trim_cache size 0 max 16384 2024-04-24T23:15:23.706 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:15:23.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f319c0a2b90 msgr2=0x7f319c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f319c0a2b90 0x7f319c0a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0cd280 tx=0x7f31ac0d5550 comp rx=0 tx=0).stop 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 msgr2=0x7f31b0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 0x7f31b0197ab0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00e9c0 tx=0x7f31ac00ee80 comp rx=0 tx=0).stop 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 shutdown_connections 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f319c0a2b90 0x7f319c0a5050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31b0103360 0x7f31b019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31b01029a0 0x7f31b0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 --2- 172.21.15.33:0/142436892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b01017b0 0x7f31b0197ab0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 >> 172.21.15.33:0/142436892 conn(0x7f31b00fcd20 msgr2=0x7f31b0105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 shutdown_connections 2024-04-24T23:15:23.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.694+0000 7f31b785d640 1 -- 172.21.15.33:0/142436892 wait complete. 2024-04-24T23:15:23.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:15:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 -- 172.21.15.33:0/1970808740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 msgr2=0x7f6890101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 --2- 172.21.15.33:0/1970808740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f6890101d10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6884009900 tx=0x7f688402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 -- 172.21.15.33:0/1970808740 shutdown_connections 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 --2- 172.21.15.33:0/1970808740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6890102250 0x7f6890108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 --2- 172.21.15.33:0/1970808740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f6890101d10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 --2- 172.21.15.33:0/1970808740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68901006a0 0x7f6890100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 -- 172.21.15.33:0/1970808740 >> 172.21.15.33:0/1970808740 conn(0x7f68900fbc30 msgr2=0x7f68900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 -- 172.21.15.33:0/1970808740 shutdown_connections 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 -- 172.21.15.33:0/1970808740 wait complete. 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.819+0000 7f68960d4640 1 Processor -- start 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 -- start start 2024-04-24T23:15:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68901006a0 0x7f689006a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f689019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6890102250 0x7f689019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f689010c320 con 0x7f6890101890 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f689010c1a0 con 0x7f68901006a0 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f68960d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f689010c4a0 con 0x7f6890102250 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f689019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68901006a0 0x7f689006a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f689019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48386/0 (socket says 172.21.15.33:48386) 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 -- 172.21.15.33:0/3662901661 learned_addr learned my addr 172.21.15.33:0/3662901661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688ffff640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6890102250 0x7f689019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 -- 172.21.15.33:0/3662901661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6890102250 msgr2=0x7f689019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6890102250 0x7f689019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 -- 172.21.15.33:0/3662901661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68901006a0 msgr2=0x7f689006a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68901006a0 0x7f689006a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 -- 172.21.15.33:0/3662901661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689019f740 con 0x7f6890101890 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.820+0000 7f688effd640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f689019c810 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6884009230 tx=0x7f68840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.821+0000 7f688cff9640 1 -- 172.21.15.33:0/3662901661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688403f030 con 0x7f6890101890 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.821+0000 7f688cff9640 1 -- 172.21.15.33:0/3662901661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6884003ce0 con 0x7f6890101890 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.821+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689019f970 con 0x7f6890101890 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.821+0000 7f688cff9640 1 -- 172.21.15.33:0/3662901661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884035070 con 0x7f6890101890 2024-04-24T23:15:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.821+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689019feb0 con 0x7f6890101890 2024-04-24T23:15:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f688cff9640 1 -- 172.21.15.33:0/3662901661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6884042030 con 0x7f6890101890 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 Processor -- start 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 -- start start 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 0x560d52b61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61c20 0x560d52b66720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b66d90 0x560d52b69220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560d52ac5670 con 0x560d52b61c20 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d52a6d970 con 0x560d52b66d90 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.823+0000 7f6898de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560d52ac1630 con 0x560d52b61230 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 0x560d52b61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61c20 0x560d52b66720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 0x560d52b61650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33638/0 (socket says 172.21.15.33:33638) 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688ffff640 1 -- 172.21.15.33:0/3142949774 learned_addr learned my addr 172.21.15.33:0/3142949774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688effd640 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b66d90 0x560d52b69220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688cff9640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68780a2c10 0x7f68780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688ffff640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68780a2c10 0x7f68780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 -- 172.21.15.33:0/3142949774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 msgr2=0x560d52b61650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 0x560d52b61650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 -- 172.21.15.33:0/3142949774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b66d90 msgr2=0x560d52b69220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b66d90 0x560d52b69220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 -- 172.21.15.33:0/3142949774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d52a91400 con 0x560d52b61c20 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688ffff640 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 0x560d52b61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688cff9640 1 -- 172.21.15.33:0/3662901661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68840ef800 con 0x7f6890101890 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688f7fe640 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61c20 0x560d52b66720 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6880005fd0 tx=0x7f68800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d700a783fbec652f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f688ffff640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68780a2c10 0x7f68780a50d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x560d52b62480 tx=0x7f6888004730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f6875ffb640 1 -- 172.21.15.33:0/3142949774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880004d20 con 0x560d52b61c20 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f6875ffb640 1 -- 172.21.15.33:0/3142949774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880003e50 con 0x560d52b61c20 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.824+0000 7f6875ffb640 1 -- 172.21.15.33:0/3142949774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000f7a0 con 0x560d52b61c20 2024-04-24T23:15:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 -- 172.21.15.33:0/3142949774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61c20 msgr2=0x560d52b66720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61c20 0x560d52b66720 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6880005fd0 tx=0x7f68800118d0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 -- 172.21.15.33:0/3142949774 shutdown_connections 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b66d90 0x560d52b69220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61c20 0x560d52b66720 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 --2- 172.21.15.33:0/3142949774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61230 0x560d52b61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 -- 172.21.15.33:0/3142949774 >> 172.21.15.33:0/3142949774 conn(0x560d52b5c8a0 msgr2=0x560d52b5cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 -- 172.21.15.33:0/3142949774 shutdown_connections 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 -- 172.21.15.33:0/3142949774 wait complete. 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.825+0000 7f6898de3740 1 -- start start 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 Processor -- start 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61230 0x560d52b6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61c20 0x560d52b6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 0x560d52b728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560d52b69d40 con 0x560d52b66d90 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d52a81050 con 0x560d52b61230 2024-04-24T23:15:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560d52b69ec0 con 0x560d52b61c20 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f6898de3740 10 client.0 _ll_register_callbacks cb 0x560d527d8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 0x560d52b728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 0x560d52b728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48394/0 (socket says 172.21.15.33:48394) 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 -- 172.21.15.33:0/2394156988 learned_addr learned my addr 172.21.15.33:0/2394156988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688f7fe640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61230 0x560d52b6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688effd640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61c20 0x560d52b6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 -- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61c20 msgr2=0x560d52b6b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61c20 0x560d52b6b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 -- 172.21.15.33:0/2394156988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61230 msgr2=0x560d52b6b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61230 0x560d52b6b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 -- 172.21.15.33:0/2394156988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d52a52dc0 con 0x560d52b66d90 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688f7fe640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61230 0x560d52b6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.826+0000 7f688ffff640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 0x560d52b728b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f68880042e0 tx=0x7f68880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b949261bfdb36be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f686effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f686effd640 1 -- 172.21.15.33:0/2394156988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888025030 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f686effd640 1 -- 172.21.15.33:0/2394156988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6888004310 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880003e50 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f686effd640 1 -- 172.21.15.33:0/2394156988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6888032050 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 10 client.16131 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d527d93f0 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f686effd640 1 -- 172.21.15.33:0/2394156988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880003e50 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f686effd640 1 -- 172.21.15.33:0/2394156988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f688801f040 con 0x560d52b66d90 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 10 client.16131 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 10 client.16131 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 20 client.16131 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52b77250 0x560d52b79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 4 client.16131 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.827+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d52ad2bb0 con 0x560d52b77250 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.828+0000 7f688f7fe640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52b77250 0x560d52b79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.828+0000 7f686effd640 10 client.16131 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.828+0000 7f688f7fe640 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52b77250 0x560d52b79710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f77353018ae9a0b server_cookie=de749ce6fa030472 in_seq=0 out_seq=0 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f686effd640 1 -- 172.21.15.33:0/2394156988 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d52ad2bb0 con 0x560d52b77250 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f686effd640 10 client.16131 handle_command_reply: tid=0 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 client.16131 shutdown 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 msgr2=0x560d52b728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 0x560d52b728b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f68880042e0 tx=0x7f68880046c0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 shutdown_connections 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52b77250 0x560d52b79710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b66d90 0x560d52b728b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b61c20 0x560d52b6b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f686effd640 0 client.16131 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 --2- 172.21.15.33:0/2394156988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61230 0x560d52b6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 >> 172.21.15.33:0/2394156988 conn(0x560d52b5c8a0 msgr2=0x560d52b5bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 shutdown_connections 2024-04-24T23:15:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 1 -- 172.21.15.33:0/2394156988 wait complete. 2024-04-24T23:15:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.829+0000 7f6898de3740 20 client.16131 trim_cache size 0 max 16384 2024-04-24T23:15:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 Processor -- start 2024-04-24T23:15:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 -- start start 2024-04-24T23:15:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b72870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68901acc60 con 0x560d52b61230 2024-04-24T23:15:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68901aa530 con 0x560d52b61c20 2024-04-24T23:15:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f6898de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68901a8380 con 0x560d52b66d90 2024-04-24T23:15:23.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48398/0 (socket says 172.21.15.33:48398) 2024-04-24T23:15:23.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 -- 172.21.15.33:0/1604550566 learned_addr learned my addr 172.21.15.33:0/1604550566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688effd640 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b72870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688ffff640 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 -- 172.21.15.33:0/1604550566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 msgr2=0x560d52b5f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b5f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 -- 172.21.15.33:0/1604550566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 msgr2=0x560d52b72870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b72870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.843+0000 7f688f7fe640 1 -- 172.21.15.33:0/1604550566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d528a37d0 con 0x560d52b61230 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.844+0000 7f688f7fe640 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b662d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6880009ef0 tx=0x7f6880000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f720f90acefc6dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.844+0000 7f68757fa640 1 -- 172.21.15.33:0/1604550566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688001d040 con 0x560d52b61230 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.844+0000 7f68757fa640 1 -- 172.21.15.33:0/1604550566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688002b730 con 0x560d52b61230 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.844+0000 7f68757fa640 1 -- 172.21.15.33:0/1604550566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000f7a0 con 0x560d52b61230 2024-04-24T23:15:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.844+0000 7f6898de3740 1 -- 172.21.15.33:0/1604550566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 msgr2=0x560d52b662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.844+0000 7f6898de3740 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b662d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6880009ef0 tx=0x7f6880000f40 comp rx=0 tx=0).stop 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- 172.21.15.33:0/1604550566 shutdown_connections 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b72870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 --2- 172.21.15.33:0/1604550566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b662d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- 172.21.15.33:0/1604550566 >> 172.21.15.33:0/1604550566 conn(0x560d52b5c8a0 msgr2=0x560d52b5bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- 172.21.15.33:0/1604550566 shutdown_connections 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- 172.21.15.33:0/1604550566 wait complete. 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- start start 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 Processor -- start 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b71b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68901a89f0 con 0x560d52b61230 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68901a9e60 con 0x560d52b61c20 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68901a90c0 con 0x560d52b66d90 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f6898de3740 10 client.0 _ll_register_callbacks cb 0x7f6888014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f688effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f688ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b71b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48414/0 (socket says 172.21.15.33:48414) 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.845+0000 7f688f7fe640 1 -- 172.21.15.33:0/3961247705 learned_addr learned my addr 172.21.15.33:0/3961247705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688ffff640 1 -- 172.21.15.33:0/3961247705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 msgr2=0x560d52b70af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688ffff640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b70af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688ffff640 1 -- 172.21.15.33:0/3961247705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 msgr2=0x560d52b705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688ffff640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688ffff640 1 -- 172.21.15.33:0/3961247705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d52b720f0 con 0x560d52b66d90 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688effd640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b70af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688f7fe640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f688ffff640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b71b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6888021e00 tx=0x7f68880045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0185c23be3484c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f686effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f686effd640 1 -- 172.21.15.33:0/3961247705 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688803b060 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688002b730 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f686effd640 1 -- 172.21.15.33:0/3961247705 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6888003bd0 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f686effd640 1 -- 172.21.15.33:0/3961247705 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6888025030 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f6898de3740 10 client.25699 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.846+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d52bddef0 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f686effd640 1 -- 172.21.15.33:0/3961247705 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688002b730 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f686effd640 1 -- 172.21.15.33:0/3961247705 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6888036530 con 0x560d52b66d90 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f6898de3740 10 client.25699 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f6898de3740 10 client.25699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f6898de3740 20 client.25699 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f6898de3740 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52bde890 0x560d52be0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f6898de3740 4 client.25699 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d52accf00 con 0x560d52bde890 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f688f7fe640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52bde890 0x560d52be0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f688f7fe640 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52bde890 0x560d52be0c80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99ff8e7ac9ca8a10 server_cookie=bf1240d692779455 in_seq=0 out_seq=0 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f686effd640 10 client.25699 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f686effd640 1 -- 172.21.15.33:0/3961247705 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d52accf00 con 0x560d52bde890 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.847+0000 7f686effd640 10 client.25699 handle_command_reply: tid=0 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 client.25699 shutdown 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 msgr2=0x560d52b71b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b71b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6888021e00 tx=0x7f68880045c0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 shutdown_connections 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x560d52bde890 0x560d52be0c80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d52b66d90 0x560d52b71b00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f686effd640 0 client.25699 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d52b61c20 0x560d52b70af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 --2- 172.21.15.33:0/3961247705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d52b61230 0x560d52b705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 >> 172.21.15.33:0/3961247705 conn(0x560d52b5c8a0 msgr2=0x560d52b5bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 shutdown_connections 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 1 -- 172.21.15.33:0/3961247705 wait complete. 2024-04-24T23:15:23.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f6898de3740 20 client.25699 trim_cache size 0 max 16384 2024-04-24T23:15:23.859 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68780a2c10 msgr2=0x7f68780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68780a2c10 0x7f68780a50d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x560d52b62480 tx=0x7f6888004730 comp rx=0 tx=0).stop 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 msgr2=0x7f689019c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f689019c810 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6884009230 tx=0x7f68840042f0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 shutdown_connections 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68780a2c10 0x7f68780a50d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6890102250 0x7f689019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6890101890 0x7f689019c810 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 --2- 172.21.15.33:0/3662901661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68901006a0 0x7f689006a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.848+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 >> 172.21.15.33:0/3662901661 conn(0x7f68900fbc30 msgr2=0x7f6890104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.849+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 shutdown_connections 2024-04-24T23:15:23.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.849+0000 7f68960d4640 1 -- 172.21.15.33:0/3662901661 wait complete. 2024-04-24T23:15:23.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 -- 172.21.15.33:0/2327491504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41074e0 msgr2=0x7f2ca41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/2327491504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41074e0 0x7f2ca41098d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009ca0 tx=0x7f2c9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 -- 172.21.15.33:0/2327491504 shutdown_connections 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/2327491504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41074e0 0x7f2ca41098d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/2327491504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/2327491504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca4102280 0x7f2ca4104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 -- 172.21.15.33:0/2327491504 >> 172.21.15.33:0/2327491504 conn(0x7f2ca40fbc10 msgr2=0x7f2ca40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 -- 172.21.15.33:0/2327491504 shutdown_connections 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.974+0000 7f2cabcf5640 1 -- 172.21.15.33:0/2327491504 wait complete. 2024-04-24T23:15:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 Processor -- start 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 -- start start 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca4102280 0x7f2ca4101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca41074e0 0x7f2ca4100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca410d440 con 0x7f2ca4104bb0 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca410d2c0 con 0x7f2ca41074e0 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2cabcf5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca410d5c0 con 0x7f2ca4102280 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2caa292640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca41074e0 0x7f2ca4100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48426/0 (socket says 172.21.15.33:48426) 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 -- 172.21.15.33:0/3262685872 learned_addr learned my addr 172.21.15.33:0/3262685872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca4102280 0x7f2ca4101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 -- 172.21.15.33:0/3262685872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca4102280 msgr2=0x7f2ca4101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca4102280 0x7f2ca4101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 -- 172.21.15.33:0/3262685872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca41074e0 msgr2=0x7f2ca4100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca41074e0 0x7f2ca4100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2ca9290640 1 -- 172.21.15.33:0/3262685872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca4101000 con 0x7f2ca4104bb0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.975+0000 7f2caa292640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca41074e0 0x7f2ca4100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.976+0000 7f2ca9290640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4100230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400d740 tx=0x7f2c9400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.976+0000 7f2c9affd640 1 -- 172.21.15.33:0/3262685872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94017070 con 0x7f2ca4104bb0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.976+0000 7f2c9affd640 1 -- 172.21.15.33:0/3262685872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c9400de50 con 0x7f2ca4104bb0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.976+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca41a1be0 con 0x7f2ca4104bb0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.976+0000 7f2c9affd640 1 -- 172.21.15.33:0/3262685872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94004e00 con 0x7f2ca4104bb0 2024-04-24T23:15:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.976+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca41a20e0 con 0x7f2ca4104bb0 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.978+0000 7f2c9affd640 1 -- 172.21.15.33:0/3262685872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2c94004fa0 con 0x7f2ca4104bb0 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 Processor -- start 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 -- start start 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c84650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c89720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 0x562591c8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562591be8670 con 0x562591c89d90 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562591b90970 con 0x562591c84230 2024-04-24T23:15:23.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caea01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562591be4630 con 0x562591c84c20 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 0x562591c8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caa292640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c84650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 0x562591c8c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48428/0 (socket says 172.21.15.33:48428) 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 -- 172.21.15.33:0/1425511434 learned_addr learned my addr 172.21.15.33:0/1425511434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c89720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 -- 172.21.15.33:0/1425511434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 msgr2=0x562591c89720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c89720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2c9affd640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c900a2b90 0x7f2c900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 -- 172.21.15.33:0/1425511434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 msgr2=0x562591c84650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c84650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2ca9290640 1 -- 172.21.15.33:0/1425511434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562591bb4400 con 0x562591c89d90 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.979+0000 7f2caa292640 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2caa292640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c900a2b90 0x7f2c900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2c9affd640 1 -- 172.21.15.33:0/3262685872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2c940cecf0 con 0x7f2ca4104bb0 2024-04-24T23:15:23.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 0x562591c8c220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ada60 tx=0x7f2c940b0780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c877705c1108ecb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2c8ffff640 1 -- 172.21.15.33:0/1425511434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94020030 con 0x562591c89d90 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2c8ffff640 1 -- 172.21.15.33:0/1425511434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c940a5120 con 0x562591c89d90 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2c8ffff640 1 -- 172.21.15.33:0/1425511434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940a5510 con 0x562591c89d90 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.980+0000 7f2caa292640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c900a2b90 0x7f2c900a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005e30 tx=0x7f2c9c0095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 -- 172.21.15.33:0/1425511434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 msgr2=0x562591c8c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 0x562591c8c220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ada60 tx=0x7f2c940b0780 comp rx=0 tx=0).stop 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 -- 172.21.15.33:0/1425511434 shutdown_connections 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c89d90 0x562591c8c220 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c89720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 --2- 172.21.15.33:0/1425511434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 -- 172.21.15.33:0/1425511434 >> 172.21.15.33:0/1425511434 conn(0x562591c7f8a0 msgr2=0x562591c7fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 -- 172.21.15.33:0/1425511434 shutdown_connections 2024-04-24T23:15:23.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 -- 172.21.15.33:0/1425511434 wait complete. 2024-04-24T23:15:23.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 -- start start 2024-04-24T23:15:23.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.981+0000 7f2caea01740 1 Processor -- start 2024-04-24T23:15:23.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 0x562591c8e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c95970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562591c8cd40 con 0x562591c84c20 2024-04-24T23:15:23.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562591ba4050 con 0x562591c84230 2024-04-24T23:15:23.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562591c8cec0 con 0x562591c89d90 2024-04-24T23:15:23.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caea01740 10 client.0 _ll_register_callbacks cb 0x5625918fbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:23.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9290640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 0x562591c8e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9290640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 0x562591c8e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48436/0 (socket says 172.21.15.33:48436) 2024-04-24T23:15:23.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9290640 1 -- 172.21.15.33:0/1197017383 learned_addr learned my addr 172.21.15.33:0/1197017383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:23.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2caa292640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c95970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 msgr2=0x562591c95970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c95970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1197017383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 msgr2=0x562591c8e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 0x562591c8e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1197017383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562591b75dc0 con 0x562591c84230 2024-04-24T23:15:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 0x562591c8e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c8e3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x562591c85020 tx=0x7f2ca0012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f3e9953309c162 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2c8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.982+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1197017383 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000f500 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1197017383 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ca000f6a0 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562591bf5bb0 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1197017383 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2ca000f8a0 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 10 client.25685 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625918fc3f0 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1197017383 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562591bf5bb0 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1197017383 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f2ca0010bf0 con 0x562591c84230 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 10 client.25685 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 20 client.25685 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591c9a240 0x562591c9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 4 client.25685 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562591beff00 con 0x562591c9a240 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.983+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591c9a240 0x562591c9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591c9a240 0x562591c9c6e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2d4295d82ad099 server_cookie=f32fd453c57e27de in_seq=0 out_seq=0 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2c8d7fa640 10 client.25685 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1197017383 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562591beff00 con 0x562591c9a240 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2c8d7fa640 10 client.25685 handle_command_reply: tid=0 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2caea01740 1 client.25685 shutdown 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 msgr2=0x562591c8e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2caea01740 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c8e3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x562591c85020 tx=0x7f2ca0012aa0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 shutdown_connections 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.984+0000 7f2caea01740 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591c9a240 0x562591c9c6e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c95970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84c20 0x562591c8e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2c8d7fa640 0 client.25685 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 1 --2- 172.21.15.33:0/1197017383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84230 0x562591c8e3f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 >> 172.21.15.33:0/1197017383 conn(0x562591c7f8a0 msgr2=0x562591c7eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 shutdown_connections 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 1 -- 172.21.15.33:0/1197017383 wait complete. 2024-04-24T23:15:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.985+0000 7f2caea01740 20 client.25685 trim_cache size 0 max 16384 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 Processor -- start 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 -- start start 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c89650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84c20 0x562591c82c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41aefb0 con 0x562591c84230 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41ac880 con 0x562591c84c20 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caea01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41aa6d0 con 0x562591c89d90 2024-04-24T23:15:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c89650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c89650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48452/0 (socket says 172.21.15.33:48452) 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 -- 172.21.15.33:0/3959014621 learned_addr learned my addr 172.21.15.33:0/3959014621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9290640 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84c20 0x562591c82c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caa292640 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 -- 172.21.15.33:0/3959014621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 msgr2=0x562591c831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 -- 172.21.15.33:0/3959014621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84c20 msgr2=0x562591c82c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84c20 0x562591c82c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2ca9a91640 1 -- 172.21.15.33:0/3959014621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625919c67d0 con 0x562591c84230 2024-04-24T23:15:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.998+0000 7f2caa292640 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.999+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c89650 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00058c0 tx=0x7f2ca000f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f07f30609a9fb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.999+0000 7f2c8f7fe640 1 -- 172.21.15.33:0/3959014621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000fb00 con 0x562591c84230 2024-04-24T23:15:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.999+0000 7f2c8f7fe640 1 -- 172.21.15.33:0/3959014621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca0005510 con 0x562591c84230 2024-04-24T23:15:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.999+0000 7f2c8f7fe640 1 -- 172.21.15.33:0/3959014621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca001cb40 con 0x562591c84230 2024-04-24T23:15:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.999+0000 7f2caea01740 1 -- 172.21.15.33:0/3959014621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 msgr2=0x562591c89650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:23.999+0000 7f2caea01740 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c89650 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00058c0 tx=0x7f2ca000f7c0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- 172.21.15.33:0/3959014621 shutdown_connections 2024-04-24T23:15:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c89d90 0x562591c831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c84c20 0x562591c82c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 --2- 172.21.15.33:0/3959014621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c89650 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- 172.21.15.33:0/3959014621 >> 172.21.15.33:0/3959014621 conn(0x562591c7f8a0 msgr2=0x562591c82580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- 172.21.15.33:0/3959014621 shutdown_connections 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- 172.21.15.33:0/3959014621 wait complete. 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- start start 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 Processor -- start 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c89d90 0x562591c93a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41aad40 con 0x562591c84230 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41ac1b0 con 0x562591c89d90 2024-04-24T23:15:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41ab410 con 0x562591c84c20 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caea01740 10 client.0 _ll_register_callbacks cb 0x7f2ca0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2ca9a91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2ca9a91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48464/0 (socket says 172.21.15.33:48464) 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1641411345 learned_addr learned my addr 172.21.15.33:0/1641411345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.000+0000 7f2caa292640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c89d90 0x562591c93a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 msgr2=0x562591c92ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c92ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1641411345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c89d90 msgr2=0x562591c93a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c89d90 0x562591c93a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9a91640 1 -- 172.21.15.33:0/1641411345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562591c94070 con 0x562591c84230 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2ca9a91640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c92580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ca001c4a0 tx=0x7f2ca000fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82cef93bb99cd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2c8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1641411345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0037060 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca0005510 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1641411345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ca002f070 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1641411345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2ca003e050 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2caea01740 10 client.16155 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562591d00820 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.001+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1641411345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca0005510 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1641411345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f2ca00038d0 con 0x562591c84230 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 10 client.16155 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 20 client.16155 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591d010b0 0x562591d034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 4 client.16155 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562591b6d820 con 0x562591d010b0 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591d010b0 0x562591d034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2ca9290640 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591d010b0 0x562591d034a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369449aaa89087b7 server_cookie=19f277e793727baf in_seq=0 out_seq=0 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2c8d7fa640 10 client.16155 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2c8d7fa640 1 -- 172.21.15.33:0/1641411345 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562591b6d820 con 0x562591d010b0 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2c8d7fa640 10 client.16155 handle_command_reply: tid=0 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.002+0000 7f2caea01740 1 client.16155 shutdown 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 msgr2=0x562591c92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c92580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ca001c4a0 tx=0x7f2ca000fd30 comp rx=0 tx=0).stop 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 shutdown_connections 2024-04-24T23:15:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562591d010b0 0x562591d034a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562591c89d90 0x562591c93a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562591c84c20 0x562591c92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 --2- 172.21.15.33:0/1641411345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562591c84230 0x562591c92580 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2c8d7fa640 0 client.16155 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 >> 172.21.15.33:0/1641411345 conn(0x562591c7f8a0 msgr2=0x562591c82e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 shutdown_connections 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 1 -- 172.21.15.33:0/1641411345 wait complete. 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2caea01740 20 client.16155 trim_cache size 0 max 16384 2024-04-24T23:15:24.012 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c900a2b90 msgr2=0x7f2c900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c900a2b90 0x7f2c900a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005e30 tx=0x7f2c9c0095c0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 msgr2=0x7f2ca4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4100230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400d740 tx=0x7f2c9400dc00 comp rx=0 tx=0).stop 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 shutdown_connections 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2c900a2b90 0x7f2c900a5050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ca41074e0 0x7f2ca4100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca4104bb0 0x7f2ca4100230 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 --2- 172.21.15.33:0/3262685872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca4102280 0x7f2ca4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 >> 172.21.15.33:0/3262685872 conn(0x7f2ca40fbc10 msgr2=0x7f2ca4102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 shutdown_connections 2024-04-24T23:15:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:24.003+0000 7f2cabcf5640 1 -- 172.21.15.33:0/3262685872 wait complete. 2024-04-24T23:15:24.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 222986 13.70 MB/sec warmup 89 sec latency 19.742 ms 2024-04-24T23:15:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:24 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3419907316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:24 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3419907316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:24 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3419907316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:25.016 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:15:25.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:15:25.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 -- 172.21.15.33:0/1222665629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0107460 msgr2=0x7f04a01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1222665629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0107460 0x7f04a01098b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0498009ca0 tx=0x7f049802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 -- 172.21.15.33:0/1222665629 shutdown_connections 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1222665629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0107460 0x7f04a01098b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1222665629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 0x7f04a0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1222665629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0102200 0x7f04a01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 -- 172.21.15.33:0/1222665629 >> 172.21.15.33:0/1222665629 conn(0x7f04a00fbbf0 msgr2=0x7f04a00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 -- 172.21.15.33:0/1222665629 shutdown_connections 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.124+0000 7f04a83c1640 1 -- 172.21.15.33:0/1222665629 wait complete. 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 Processor -- start 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 -- start start 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0102200 0x7f04a0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 0x7f04a0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0107460 0x7f04a019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d420 con 0x7f04a0102200 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d2a0 con 0x7f04a0104b30 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a83c1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d5a0 con 0x7f04a0107460 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a695e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0107460 0x7f04a019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a595c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 0x7f04a0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a615d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0102200 0x7f04a0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a695e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0107460 0x7f04a019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33756/0 (socket says 172.21.15.33:33756) 2024-04-24T23:15:25.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a695e640 1 -- 172.21.15.33:0/1106315135 learned_addr learned my addr 172.21.15.33:0/1106315135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a595c640 1 -- 172.21.15.33:0/1106315135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0107460 msgr2=0x7f04a019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a595c640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0107460 0x7f04a019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a595c640 1 -- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0102200 msgr2=0x7f04a0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a595c640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0102200 0x7f04a0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.125+0000 7f04a595c640 1 -- 172.21.15.33:0/1106315135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a019f680 con 0x7f04a0104b30 2024-04-24T23:15:25.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.126+0000 7f04a595c640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 0x7f04a0197fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f049000e9c0 tx=0x7f049000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.126+0000 7f04977fe640 1 -- 172.21.15.33:0/1106315135 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049000ccc0 con 0x7f04a0104b30 2024-04-24T23:15:25.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.126+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a019f970 con 0x7f04a0104b30 2024-04-24T23:15:25.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.126+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a019ff10 con 0x7f04a0104b30 2024-04-24T23:15:25.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.126+0000 7f04977fe640 1 -- 172.21.15.33:0/1106315135 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0490010910 con 0x7f04a0104b30 2024-04-24T23:15:25.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.126+0000 7f04977fe640 1 -- 172.21.15.33:0/1106315135 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490018880 con 0x7f04a0104b30 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.128+0000 7f04977fe640 1 -- 172.21.15.33:0/1106315135 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0490010ac0 con 0x7f04a0104b30 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 Processor -- start 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 -- start start 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588130252c0 con 0x5588130c1d90 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558812fcd910 con 0x5588130c2780 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558813021280 con 0x5588130c78f0 2024-04-24T23:15:25.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04a615d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04a695e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04a595c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04a615d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46802/0 (socket says 172.21.15.33:46802) 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.129+0000 7f04a615d640 1 -- 172.21.15.33:0/1742730720 learned_addr learned my addr 172.21.15.33:0/1742730720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04977fe640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f048c0a2b90 0x7f048c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 -- 172.21.15.33:0/1742730720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 msgr2=0x5588130c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 -- 172.21.15.33:0/1742730720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 msgr2=0x5588130c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 -- 172.21.15.33:0/1742730720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558812fdefb0 con 0x5588130c1d90 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04977fe640 1 -- 172.21.15.33:0/1106315135 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04900cf670 con 0x7f04a0104b30 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f048c0a2b90 0x7f048c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a615d640 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f04a695e640 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130c21b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0498005e30 tx=0x7f04980093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5409673a9a8c2335 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.130+0000 7f0494bfe640 1 -- 172.21.15.33:0/1742730720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498042040 con 0x5588130c1d90 2024-04-24T23:15:25.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04a695e640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f048c0a2b90 0x7f048c0a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f049804d000 tx=0x7f049804e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f0494bfe640 1 -- 172.21.15.33:0/1742730720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498044070 con 0x5588130c1d90 2024-04-24T23:15:25.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f0494bfe640 1 -- 172.21.15.33:0/1742730720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498048430 con 0x5588130c1d90 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1742730720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 msgr2=0x5588130c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130c21b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0498005e30 tx=0x7f04980093b0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1742730720 shutdown_connections 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1742730720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130c21b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.131+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1742730720 >> 172.21.15.33:0/1742730720 conn(0x558813089950 msgr2=0x5588130bd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1742730720 shutdown_connections 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1742730720 wait complete. 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 -- start start 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 Processor -- start 2024-04-24T23:15:25.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588130ca940 con 0x5588130c1d90 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558812fe0e90 con 0x5588130c2780 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588130caac0 con 0x5588130c78f0 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04ab1a9740 10 client.0 _ll_register_callbacks cb 0x55881301c970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04a615d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04a615d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48494/0 (socket says 172.21.15.33:48494) 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04a595c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04a615d640 1 -- 172.21.15.33:0/4070651857 learned_addr learned my addr 172.21.15.33:0/4070651857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.132+0000 7f04a695e640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a595c640 1 -- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 msgr2=0x5588130d7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a595c640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a595c640 1 -- 172.21.15.33:0/4070651857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 msgr2=0x5588130d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a595c640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a595c640 1 -- 172.21.15.33:0/4070651857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558812e017e0 con 0x5588130c2780 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a615d640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a695e640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04a595c640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d0c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f049009b890 tx=0x7f0490009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c08275e2e80c50ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f0489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f0489ffb640 1 -- 172.21.15.33:0/4070651857 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490010db0 con 0x5588130c2780 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f0489ffb640 1 -- 172.21.15.33:0/4070651857 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04900d1b30 con 0x5588130c2780 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0498044070 con 0x5588130c2780 2024-04-24T23:15:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f0489ffb640 1 -- 172.21.15.33:0/4070651857 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f049009bb90 con 0x5588130c2780 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 10 client.25703 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558812e61f70 con 0x5588130c2780 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f0489ffb640 1 -- 172.21.15.33:0/4070651857 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498044070 con 0x5588130c2780 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f0489ffb640 1 -- 172.21.15.33:0/4070651857 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f049009b070 con 0x5588130c2780 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 10 client.25703 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 20 client.25703 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588130dc180 0x5588130de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 4 client.25703 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.133+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558812ff95a0 con 0x5588130dc180 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.134+0000 7f04a615d640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588130dc180 0x5588130de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.134+0000 7f0489ffb640 10 client.25703 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.134+0000 7f04a615d640 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588130dc180 0x5588130de5e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf685aba5ac24ae server_cookie=b18a9941be7fd662 in_seq=0 out_seq=0 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f0489ffb640 1 -- 172.21.15.33:0/4070651857 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558812ff95a0 con 0x5588130dc180 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f0489ffb640 10 client.25703 handle_command_reply: tid=0 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 client.25703 shutdown 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 msgr2=0x5588130d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d0c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f049009b890 tx=0x7f0490009700 comp rx=0 tx=0).stop 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 shutdown_connections 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588130dc180 0x5588130de5e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d0c00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f0489ffb640 0 client.25703 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/4070651857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 >> 172.21.15.33:0/4070651857 conn(0x558813089950 msgr2=0x5588130bc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 shutdown_connections 2024-04-24T23:15:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 1 -- 172.21.15.33:0/4070651857 wait complete. 2024-04-24T23:15:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.135+0000 7f04ab1a9740 20 client.25703 trim_cache size 0 max 16384 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 Processor -- start 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 -- start start 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c1d90 0x5588130c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 0x5588130d7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c78f0 0x5588130c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0484012490 con 0x5588130c2780 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f048400fd60 con 0x5588130c78f0 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f048400dbb0 con 0x5588130c1d90 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04a595c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 0x5588130d7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04a695e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c78f0 0x5588130c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04a595c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 0x5588130d7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48510/0 (socket says 172.21.15.33:48510) 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04a595c640 1 -- 172.21.15.33:0/1948739933 learned_addr learned my addr 172.21.15.33:0/1948739933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.154+0000 7f04a615d640 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c1d90 0x5588130c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a595c640 1 -- 172.21.15.33:0/1948739933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c1d90 msgr2=0x5588130c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a595c640 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c1d90 0x5588130c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a595c640 1 -- 172.21.15.33:0/1948739933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c78f0 msgr2=0x5588130c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a595c640 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c78f0 0x5588130c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a595c640 1 -- 172.21.15.33:0/1948739933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558812cf9170 con 0x5588130c2780 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a615d640 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c1d90 0x5588130c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f04a595c640 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 0x5588130d7c00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0490099e40 tx=0x7f04900cf8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575bbed4b32d88c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f048bfff640 1 -- 172.21.15.33:0/1948739933 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900b9070 con 0x5588130c2780 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f048bfff640 1 -- 172.21.15.33:0/1948739933 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04900a4750 con 0x5588130c2780 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.155+0000 7f048bfff640 1 -- 172.21.15.33:0/1948739933 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049009acb0 con 0x5588130c2780 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1948739933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 msgr2=0x5588130d7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 0x5588130d7c00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0490099e40 tx=0x7f04900cf8f0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1948739933 shutdown_connections 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c78f0 0x5588130c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c2780 0x5588130d7c00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/1948739933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c1d90 0x5588130c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1948739933 >> 172.21.15.33:0/1948739933 conn(0x558813089950 msgr2=0x5588130d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1948739933 shutdown_connections 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- 172.21.15.33:0/1948739933 wait complete. 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- start start 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 Processor -- start 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f048400e220 con 0x5588130c1d90 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f048400f690 con 0x5588130c2780 2024-04-24T23:15:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f048400e8f0 con 0x5588130c78f0 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04ab1a9740 10 client.0 _ll_register_callbacks cb 0x7f049000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04a615d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04a615d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48514/0 (socket says 172.21.15.33:48514) 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04a615d640 1 -- 172.21.15.33:0/3612017035 learned_addr learned my addr 172.21.15.33:0/3612017035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04a595c640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.156+0000 7f04a695e640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a615d640 1 -- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 msgr2=0x5588130d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a615d640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a615d640 1 -- 172.21.15.33:0/3612017035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 msgr2=0x5588130d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a615d640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a615d640 1 -- 172.21.15.33:0/3612017035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588130d6730 con 0x5588130c1d90 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a595c640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a695e640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04a615d640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d4cd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5588130dee90 tx=0x7f049c012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b2ddd9a13c720b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f0489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f0489ffb640 1 -- 172.21.15.33:0/3612017035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c020480 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f0489ffb640 1 -- 172.21.15.33:0/3612017035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f049c020620 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04900a4750 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f0489ffb640 1 -- 172.21.15.33:0/3612017035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f049c031050 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04ab1a9740 10 client.16173 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5588130d7b40 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.157+0000 7f0489ffb640 1 -- 172.21.15.33:0/3612017035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04900a4750 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f0489ffb640 1 -- 172.21.15.33:0/3612017035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f049c029420 con 0x5588130c1d90 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04ab1a9740 10 client.16173 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04ab1a9740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04ab1a9740 20 client.16173 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558813142bc0 0x558813144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04ab1a9740 4 client.16173 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:15:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558813032800 con 0x558813142bc0 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04a595c640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558813142bc0 0x558813144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f04a595c640 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558813142bc0 0x558813144fe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305aed5381bb0989 server_cookie=b4a7ab2c851206dc in_seq=0 out_seq=0 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.158+0000 7f0489ffb640 10 client.16173 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f0489ffb640 1 -- 172.21.15.33:0/3612017035 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558813032800 con 0x558813142bc0 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f0489ffb640 10 client.16173 handle_command_reply: tid=0 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 client.16173 shutdown 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 msgr2=0x5588130d4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d4cd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5588130dee90 tx=0x7f049c012ad0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 shutdown_connections 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558813142bc0 0x558813144fe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588130c78f0 0x5588130d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f0489ffb640 0 client.16173 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588130c2780 0x5588130d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 --2- 172.21.15.33:0/3612017035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588130c1d90 0x5588130d4cd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.159+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 >> 172.21.15.33:0/3612017035 conn(0x558813089950 msgr2=0x5588130c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 shutdown_connections 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04ab1a9740 1 -- 172.21.15.33:0/3612017035 wait complete. 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04ab1a9740 20 client.16173 trim_cache size 0 max 16384 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "e35803ef-f2d9-40bb-a418-5f483cf8c8b6", 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:15:25.168 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:15:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f048c0a2b90 msgr2=0x7f048c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f048c0a2b90 0x7f048c0a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f049804d000 tx=0x7f049804e000 comp rx=0 tx=0).stop 2024-04-24T23:15:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 msgr2=0x7f04a0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 0x7f04a0197fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f049000e9c0 tx=0x7f049000ee80 comp rx=0 tx=0).stop 2024-04-24T23:15:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 shutdown_connections 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f048c0a2b90 0x7f048c0a5050 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04a0107460 0x7f04a019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04a0104b30 0x7f04a0197fa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 --2- 172.21.15.33:0/1106315135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a0102200 0x7f04a0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 >> 172.21.15.33:0/1106315135 conn(0x7f04a00fbbf0 msgr2=0x7f04a0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 shutdown_connections 2024-04-24T23:15:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.160+0000 7f04a83c1640 1 -- 172.21.15.33:0/1106315135 wait complete. 2024-04-24T23:15:25.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:25.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 -- 172.21.15.33:0/325425625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec102980 msgr2=0x7ff8ec102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/325425625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec102980 0x7ff8ec102e00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009900 tx=0x7ff8dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 -- 172.21.15.33:0/325425625 shutdown_connections 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/325425625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec103340 0x7ff8ec109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/325425625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec102980 0x7ff8ec102e00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/325425625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8ec101790 0x7ff8ec101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 -- 172.21.15.33:0/325425625 >> 172.21.15.33:0/325425625 conn(0x7ff8ec0fcd20 msgr2=0x7ff8ec0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 -- 172.21.15.33:0/325425625 shutdown_connections 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.285+0000 7ff8f3c58640 1 -- 172.21.15.33:0/325425625 wait complete. 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 Processor -- start 2024-04-24T23:15:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 -- start start 2024-04-24T23:15:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec101790 0x7ff8ec0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8ec102980 0x7ff8ec073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 0x7ff8ec074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec10d410 con 0x7ff8ec103340 2024-04-24T23:15:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec10d290 con 0x7ff8ec102980 2024-04-24T23:15:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f3c58640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec10d590 con 0x7ff8ec101790 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f21f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 0x7ff8ec074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f21f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 0x7ff8ec074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48524/0 (socket says 172.21.15.33:48524) 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f21f5640 1 -- 172.21.15.33:0/3163328762 learned_addr learned my addr 172.21.15.33:0/3163328762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8ec102980 0x7ff8ec073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.286+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec101790 0x7ff8ec0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f21f5640 1 -- 172.21.15.33:0/3163328762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec101790 msgr2=0x7ff8ec0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f21f5640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec101790 0x7ff8ec0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f21f5640 1 -- 172.21.15.33:0/3163328762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8ec102980 msgr2=0x7ff8ec073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f21f5640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8ec102980 0x7ff8ec073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f21f5640 1 -- 172.21.15.33:0/3163328762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec074900 con 0x7ff8ec103340 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f21f5640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 0x7ff8ec074070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e970 tx=0x7ff8e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8e2ffd640 1 -- 172.21.15.33:0/3163328762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400ccc0 con 0x7ff8ec103340 2024-04-24T23:15:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec1a1ba0 con 0x7ff8ec103340 2024-04-24T23:15:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8e2ffd640 1 -- 172.21.15.33:0/3163328762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e400ce60 con 0x7ff8ec103340 2024-04-24T23:15:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8e2ffd640 1 -- 172.21.15.33:0/3163328762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4010450 con 0x7ff8ec103340 2024-04-24T23:15:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.287+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1a20c0 con 0x7ff8ec103340 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.289+0000 7ff8e2ffd640 1 -- 172.21.15.33:0/3163328762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff8e4005020 con 0x7ff8ec103340 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 Processor -- start 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 -- start start 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2240 0x5582394c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394c7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c7da0 0x5582394ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582394266b0 con 0x5582394c2240 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5582393ce880 con 0x5582394c2c30 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f6966740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558239422670 con 0x5582394c7da0 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8e2ffd640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8d80a2b90 0x7ff8d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f19f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2240 0x5582394c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f19f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46836/0 (socket says 172.21.15.33:46836) 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f19f4640 1 -- 172.21.15.33:0/1337863067 learned_addr learned my addr 172.21.15.33:0/1337863067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f21f5640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8d80a2b90 0x7ff8d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.290+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c7da0 0x5582394ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8e2ffd640 1 -- 172.21.15.33:0/3163328762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff8e40cf820 con 0x7ff8ec103340 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f19f4640 1 -- 172.21.15.33:0/1337863067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c7da0 msgr2=0x5582394ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c7da0 0x5582394ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f19f4640 1 -- 172.21.15.33:0/1337863067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2240 msgr2=0x5582394c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2240 0x5582394c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f19f4640 1 -- 172.21.15.33:0/1337863067 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582392047d0 con 0x5582394c2c30 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f21f5640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8d80a2b90 0x7ff8d80a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400d010 tx=0x7ff8e40978e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394c7730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8011470 tx=0x7ff8e8011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bba6d96c9f90547 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8d7fff640 1 -- 172.21.15.33:0/1337863067 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8003970 con 0x5582394c2c30 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8d7fff640 1 -- 172.21.15.33:0/1337863067 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8004020 con 0x5582394c2c30 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.291+0000 7ff8d7fff640 1 -- 172.21.15.33:0/1337863067 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800f8d0 con 0x5582394c2c30 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 -- 172.21.15.33:0/1337863067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 msgr2=0x5582394c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394c7730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8011470 tx=0x7ff8e8011930 comp rx=0 tx=0).stop 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 -- 172.21.15.33:0/1337863067 shutdown_connections 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c7da0 0x5582394ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394c7730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 --2- 172.21.15.33:0/1337863067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2240 0x5582394c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 -- 172.21.15.33:0/1337863067 >> 172.21.15.33:0/1337863067 conn(0x5582394bd8b0 msgr2=0x5582394bdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 -- 172.21.15.33:0/1337863067 shutdown_connections 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 -- 172.21.15.33:0/1337863067 wait complete. 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 -- start start 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.292+0000 7ff8f6966740 1 Processor -- start 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582394cad50 con 0x5582394c7da0 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5582393e1f50 con 0x5582394c2c30 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5582394caed0 con 0x5582394c2240 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f6966740 10 client.0 _ll_register_callbacks cb 0x55823941d000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48552/0 (socket says 172.21.15.33:48552) 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 -- 172.21.15.33:0/557783951 learned_addr learned my addr 172.21.15.33:0/557783951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 -- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 msgr2=0x5582394cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 -- 172.21.15.33:0/557783951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 msgr2=0x5582394ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 -- 172.21.15.33:0/557783951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823913a3f0 con 0x5582394c7da0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d3b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc02a930 tx=0x7ff8dc041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94975ea66932f397 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.293+0000 7ff8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8d57fa640 1 -- 172.21.15.33:0/557783951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc035040 con 0x5582394c7da0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8d57fa640 1 -- 172.21.15.33:0/557783951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff8dc005030 con 0x5582394c7da0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e8004020 con 0x5582394c7da0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8d57fa640 1 -- 172.21.15.33:0/557783951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc053050 con 0x5582394c7da0 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 10 client.16185 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:25.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582390fa0f0 con 0x5582394c7da0 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8d57fa640 1 -- 172.21.15.33:0/557783951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8004020 con 0x5582394c7da0 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8d57fa640 1 -- 172.21.15.33:0/557783951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff8dc0035f0 con 0x5582394c7da0 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 10 client.16185 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 20 client.16185 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5582394d8380 0x5582394da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 4 client.16185 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558239433bf0 con 0x5582394d8380 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.294+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5582394d8380 0x5582394da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.295+0000 7ff8d57fa640 10 client.16185 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.295+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5582394d8380 0x5582394da840 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a65ce1e19f2c53 server_cookie=21512e2b2c17fee7 in_seq=0 out_seq=0 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8d57fa640 1 -- 172.21.15.33:0/557783951 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558239433bf0 con 0x5582394d8380 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8d57fa640 10 client.16185 handle_command_reply: tid=0 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 client.16185 shutdown 2024-04-24T23:15:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 msgr2=0x5582394d3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d3b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc02a930 tx=0x7ff8dc041040 comp rx=0 tx=0).stop 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 shutdown_connections 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5582394d8380 0x5582394da840 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d3b50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2c30 0x5582394ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 --2- 172.21.15.33:0/557783951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8d57fa640 0 client.16185 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 >> 172.21.15.33:0/557783951 conn(0x5582394bd8b0 msgr2=0x5582394bcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 shutdown_connections 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 1 -- 172.21.15.33:0/557783951 wait complete. 2024-04-24T23:15:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.296+0000 7ff8f6966740 20 client.16185 trim_cache size 0 max 16384 2024-04-24T23:15:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.314+0000 7ff8f6966740 1 Processor -- start 2024-04-24T23:15:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.314+0000 7ff8f6966740 1 -- start start 2024-04-24T23:15:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 0x5582394d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c7da0 0x5582394c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f6966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1aee80 con 0x5582394c2c30 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f6966740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ac750 con 0x5582394c7da0 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f6966740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1aa5a0 con 0x5582394c2240 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 0x5582394d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 0x5582394d3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48568/0 (socket says 172.21.15.33:48568) 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c7da0 0x5582394c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 -- 172.21.15.33:0/2695023913 learned_addr learned my addr 172.21.15.33:0/2695023913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 -- 172.21.15.33:0/2695023913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 msgr2=0x5582394cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 -- 172.21.15.33:0/2695023913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c7da0 msgr2=0x5582394c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c7da0 0x5582394c1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 -- 172.21.15.33:0/2695023913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582394c4630 con 0x5582394c2c30 2024-04-24T23:15:25.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c7da0 0x5582394c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 0x5582394d3b10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8012fd0 tx=0x7ff8e8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61347903f350d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8d77fe640 1 -- 172.21.15.33:0/2695023913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8003970 con 0x5582394c2c30 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8d77fe640 1 -- 172.21.15.33:0/2695023913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8003ff0 con 0x5582394c2c30 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.315+0000 7ff8d77fe640 1 -- 172.21.15.33:0/2695023913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004cf0 con 0x5582394c2c30 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 -- 172.21.15.33:0/2695023913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 msgr2=0x5582394d3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 0x5582394d3b10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8012fd0 tx=0x7ff8e8009d20 comp rx=0 tx=0).stop 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 -- 172.21.15.33:0/2695023913 shutdown_connections 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c7da0 0x5582394c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c2c30 0x5582394d3b10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 --2- 172.21.15.33:0/2695023913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2240 0x5582394cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 -- 172.21.15.33:0/2695023913 >> 172.21.15.33:0/2695023913 conn(0x5582394bd8b0 msgr2=0x5582394c0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 -- 172.21.15.33:0/2695023913 shutdown_connections 2024-04-24T23:15:25.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 -- 172.21.15.33:0/2695023913 wait complete. 2024-04-24T23:15:25.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 -- start start 2024-04-24T23:15:25.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.316+0000 7ff8f6966740 1 Processor -- start 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2240 0x5582394d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2c30 0x5582394d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1aac10 con 0x5582394c7da0 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ac080 con 0x5582394c2240 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ab2e0 con 0x5582394c2c30 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f6966740 10 client.0 _ll_register_callbacks cb 0x7ff8dc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:25.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2240 0x5582394d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48580/0 (socket says 172.21.15.33:48580) 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 -- 172.21.15.33:0/3948939465 learned_addr learned my addr 172.21.15.33:0/3948939465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2c30 0x5582394d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 -- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2c30 msgr2=0x5582394d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2c30 0x5582394d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 -- 172.21.15.33:0/3948939465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2240 msgr2=0x5582394d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2240 0x5582394d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.317+0000 7ff8f11f3640 1 -- 172.21.15.33:0/3948939465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582394d3430 con 0x5582394c7da0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f11f3640 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d2e40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc031ba0 tx=0x7ff8dc005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7774c24548be9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8d57fa640 1 -- 172.21.15.33:0/3948939465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc035040 con 0x5582394c7da0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e8003ff0 con 0x5582394c7da0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8d57fa640 1 -- 172.21.15.33:0/3948939465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff8dc003b60 con 0x5582394c7da0 2024-04-24T23:15:25.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8d57fa640 1 -- 172.21.15.33:0/3948939465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc066050 con 0x5582394c7da0 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 10 client.16191 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55823953f030 con 0x5582394c7da0 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8d57fa640 1 -- 172.21.15.33:0/3948939465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8003ff0 con 0x5582394c7da0 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8d57fa640 1 -- 172.21.15.33:0/3948939465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff8dc04b400 con 0x5582394c7da0 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 10 client.16191 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 20 client.16191 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55823953fa30 0x558239541e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 4 client.16191 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55823942df40 con 0x55823953fa30 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.318+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55823953fa30 0x558239541e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8d57fa640 10 client.16191 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f19f4640 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55823953fa30 0x558239541e50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a065ea3e84566349 server_cookie=a37664f5d5e58453 in_seq=0 out_seq=0 2024-04-24T23:15:25.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8d57fa640 1 -- 172.21.15.33:0/3948939465 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55823942df40 con 0x55823953fa30 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8d57fa640 10 client.16191 handle_command_reply: tid=0 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 client.16191 shutdown 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 msgr2=0x5582394d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d2e40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc031ba0 tx=0x7ff8dc005d70 comp rx=0 tx=0).stop 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 shutdown_connections 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55823953fa30 0x558239541e50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582394c7da0 0x5582394d2e40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582394c2c30 0x5582394d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 --2- 172.21.15.33:0/3948939465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582394c2240 0x5582394d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8d57fa640 0 client.16191 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 >> 172.21.15.33:0/3948939465 conn(0x5582394bd8b0 msgr2=0x5582394bcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 shutdown_connections 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 1 -- 172.21.15.33:0/3948939465 wait complete. 2024-04-24T23:15:25.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.319+0000 7ff8f6966740 20 client.16191 trim_cache size 0 max 16384 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: "e35803ef-f2d9-40bb-a418-5f483cf8c8b6": { 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "e35803ef-f2d9-40bb-a418-5f483cf8c8b6", 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:15:25.328 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:15:25.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8d80a2b90 msgr2=0x7ff8d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8d80a2b90 0x7ff8d80a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400d010 tx=0x7ff8e40978e0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 msgr2=0x7ff8ec074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:25.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 0x7ff8ec074070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e970 tx=0x7ff8e400ed70 comp rx=0 tx=0).stop 2024-04-24T23:15:25.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 shutdown_connections 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff8d80a2b90 0x7ff8d80a5050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8ec103340 0x7ff8ec074070 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff8ec102980 0x7ff8ec073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 --2- 172.21.15.33:0/3163328762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec101790 0x7ff8ec0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 >> 172.21.15.33:0/3163328762 conn(0x7ff8ec0fcd20 msgr2=0x7ff8ec105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 shutdown_connections 2024-04-24T23:15:25.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:25.320+0000 7ff8f3c58640 1 -- 172.21.15.33:0/3163328762 wait complete. 2024-04-24T23:15:25.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:25 smithi033 ceph-mon[29201]: pgmap v453: 129 pgs: 129 active+clean; 100 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-04-24T23:15:25.332 INFO:tasks.cephfs.filesystem:scrub status for tag:e35803ef-f2d9-40bb-a418-5f483cf8c8b6 - {'path': '/', 'tag': 'e35803ef-f2d9-40bb-a418-5f483cf8c8b6', 'options': 'recursive,force'} 2024-04-24T23:15:25.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 224961 13.68 MB/sec warmup 90 sec latency 24.223 ms 2024-04-24T23:15:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:25 smithi038 ceph-mon[32450]: pgmap v453: 129 pgs: 129 active+clean; 100 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-04-24T23:15:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:25 smithi117 ceph-mon[32529]: pgmap v453: 129 pgs: 129 active+clean; 100 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-04-24T23:15:26.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 227737 13.68 MB/sec warmup 91 sec latency 16.548 ms 2024-04-24T23:15:26.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:26 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:15:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:26 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:26 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:15:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:26 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:15:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:26 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:26 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:26 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:15:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:26 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:26 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:15:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:26 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:15:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:26 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:26 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:26 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:15:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:26 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:26 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:15:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:26 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:15:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:26 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:26.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:26 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:27.438 INFO:tasks.workunit.client.0.smithi033.stdout: 1 230759 13.72 MB/sec warmup 92 sec latency 17.456 ms 2024-04-24T23:15:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:27 smithi038 ceph-mon[32450]: pgmap v454: 129 pgs: 129 active+clean; 103 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 297 op/s 2024-04-24T23:15:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:27 smithi117 ceph-mon[32529]: pgmap v454: 129 pgs: 129 active+clean; 103 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 297 op/s 2024-04-24T23:15:27.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:27 smithi033 ceph-mon[29201]: pgmap v454: 129 pgs: 129 active+clean; 103 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 297 op/s 2024-04-24T23:15:28.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 232599 13.65 MB/sec warmup 93 sec latency 65.125 ms 2024-04-24T23:15:29.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 234870 13.65 MB/sec warmup 94 sec latency 15.615 ms 2024-04-24T23:15:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:29 smithi038 ceph-mon[32450]: pgmap v455: 129 pgs: 129 active+clean; 103 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-24T23:15:29.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:29 smithi117 ceph-mon[32529]: pgmap v455: 129 pgs: 129 active+clean; 103 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-24T23:15:29.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:29 smithi033 ceph-mon[29201]: pgmap v455: 129 pgs: 129 active+clean; 103 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-24T23:15:30.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 237905 13.70 MB/sec warmup 95 sec latency 16.346 ms 2024-04-24T23:15:31.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 240085 13.64 MB/sec warmup 96 sec latency 24.755 ms 2024-04-24T23:15:31.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:31 smithi033 ceph-mon[29201]: pgmap v456: 129 pgs: 129 active+clean; 104 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 215 op/s 2024-04-24T23:15:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:31 smithi038 ceph-mon[32450]: pgmap v456: 129 pgs: 129 active+clean; 104 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 215 op/s 2024-04-24T23:15:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:31 smithi117 ceph-mon[32529]: pgmap v456: 129 pgs: 129 active+clean; 104 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 215 op/s 2024-04-24T23:15:32.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 242153 13.63 MB/sec warmup 97 sec latency 14.270 ms 2024-04-24T23:15:32.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:32 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:15:32.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:32 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:15:32.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:32 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:15:33.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 244916 13.66 MB/sec warmup 98 sec latency 91.055 ms 2024-04-24T23:15:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:33 smithi038 ceph-mon[32450]: pgmap v457: 129 pgs: 129 active+clean; 108 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-04-24T23:15:33.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:15:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:33 smithi117 ceph-mon[32529]: pgmap v457: 129 pgs: 129 active+clean; 108 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-04-24T23:15:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:15:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:33 smithi033 ceph-mon[29201]: pgmap v457: 129 pgs: 129 active+clean; 108 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-04-24T23:15:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:15:34.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 247176 13.61 MB/sec warmup 99 sec latency 14.223 ms 2024-04-24T23:15:35.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 249277 13.60 MB/sec warmup 100 sec latency 17.525 ms 2024-04-24T23:15:35.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:35 smithi038 ceph-mon[32450]: pgmap v458: 129 pgs: 129 active+clean; 109 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-24T23:15:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:35 smithi117 ceph-mon[32529]: pgmap v458: 129 pgs: 129 active+clean; 109 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-24T23:15:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:35 smithi033 ceph-mon[29201]: pgmap v458: 129 pgs: 129 active+clean; 109 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-24T23:15:36.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 252251 13.65 MB/sec warmup 101 sec latency 11.578 ms 2024-04-24T23:15:37.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 254674 13.62 MB/sec warmup 102 sec latency 43.535 ms 2024-04-24T23:15:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:37 smithi038 ceph-mon[32450]: pgmap v459: 129 pgs: 129 active+clean; 110 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 257 op/s 2024-04-24T23:15:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:37 smithi117 ceph-mon[32529]: pgmap v459: 129 pgs: 129 active+clean; 110 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 257 op/s 2024-04-24T23:15:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:37 smithi033 ceph-mon[29201]: pgmap v459: 129 pgs: 129 active+clean; 110 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 257 op/s 2024-04-24T23:15:38.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 256243 13.58 MB/sec warmup 103 sec latency 61.219 ms 2024-04-24T23:15:39.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 259207 13.62 MB/sec warmup 104 sec latency 23.707 ms 2024-04-24T23:15:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:39 smithi038 ceph-mon[32450]: pgmap v460: 129 pgs: 129 active+clean; 114 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 263 op/s 2024-04-24T23:15:39.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:39 smithi117 ceph-mon[32529]: pgmap v460: 129 pgs: 129 active+clean; 114 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 263 op/s 2024-04-24T23:15:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:39 smithi033 ceph-mon[29201]: pgmap v460: 129 pgs: 129 active+clean; 114 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 263 op/s 2024-04-24T23:15:40.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 261846 13.60 MB/sec warmup 105 sec latency 17.263 ms 2024-04-24T23:15:41.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 263897 13.60 MB/sec warmup 106 sec latency 15.367 ms 2024-04-24T23:15:41.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:41 smithi038 ceph-mon[32450]: pgmap v461: 129 pgs: 129 active+clean; 115 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-24T23:15:41.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:41 smithi117 ceph-mon[32529]: pgmap v461: 129 pgs: 129 active+clean; 115 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-24T23:15:41.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:41 smithi033 ceph-mon[29201]: pgmap v461: 129 pgs: 129 active+clean; 115 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-24T23:15:42.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 266667 13.60 MB/sec warmup 107 sec latency 11.286 ms 2024-04-24T23:15:43.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 269423 13.61 MB/sec warmup 108 sec latency 44.158 ms 2024-04-24T23:15:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:43 smithi038 ceph-mon[32450]: pgmap v462: 129 pgs: 129 active+clean; 115 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-24T23:15:43.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:43 smithi117 ceph-mon[32529]: pgmap v462: 129 pgs: 129 active+clean; 115 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-24T23:15:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:43 smithi033 ceph-mon[29201]: pgmap v462: 129 pgs: 129 active+clean; 115 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-24T23:15:44.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 271307 13.57 MB/sec warmup 109 sec latency 30.063 ms 2024-04-24T23:15:45.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 273886 13.58 MB/sec warmup 110 sec latency 12.776 ms 2024-04-24T23:15:45.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:45 smithi038 ceph-mon[32450]: pgmap v463: 129 pgs: 129 active+clean; 117 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-04-24T23:15:45.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:45 smithi117 ceph-mon[32529]: pgmap v463: 129 pgs: 129 active+clean; 117 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-04-24T23:15:45.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:45 smithi033 ceph-mon[29201]: pgmap v463: 129 pgs: 129 active+clean; 117 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-04-24T23:15:46.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 276853 13.61 MB/sec warmup 111 sec latency 15.768 ms 2024-04-24T23:15:47.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 279078 13.57 MB/sec warmup 112 sec latency 86.554 ms 2024-04-24T23:15:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:47 smithi038 ceph-mon[32450]: pgmap v464: 129 pgs: 129 active+clean; 120 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 309 op/s 2024-04-24T23:15:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:47 smithi117 ceph-mon[32529]: pgmap v464: 129 pgs: 129 active+clean; 120 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 309 op/s 2024-04-24T23:15:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:47 smithi033 ceph-mon[29201]: pgmap v464: 129 pgs: 129 active+clean; 120 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 309 op/s 2024-04-24T23:15:48.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 280941 13.56 MB/sec warmup 113 sec latency 53.575 ms 2024-04-24T23:15:49.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 283767 13.59 MB/sec warmup 114 sec latency 29.868 ms 2024-04-24T23:15:49.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:49 smithi038 ceph-mon[32450]: pgmap v465: 129 pgs: 129 active+clean; 120 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 283 op/s 2024-04-24T23:15:49.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:49 smithi117 ceph-mon[32529]: pgmap v465: 129 pgs: 129 active+clean; 120 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 283 op/s 2024-04-24T23:15:49.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:49 smithi033 ceph-mon[29201]: pgmap v465: 129 pgs: 129 active+clean; 120 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 283 op/s 2024-04-24T23:15:50.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 286170 13.55 MB/sec warmup 115 sec latency 18.602 ms 2024-04-24T23:15:51.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 288256 13.54 MB/sec warmup 116 sec latency 13.348 ms 2024-04-24T23:15:51.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:51 smithi038 ceph-mon[32450]: pgmap v466: 129 pgs: 129 active+clean; 120 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-04-24T23:15:51.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:51 smithi117 ceph-mon[32529]: pgmap v466: 129 pgs: 129 active+clean; 120 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-04-24T23:15:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:51 smithi033 ceph-mon[29201]: pgmap v466: 129 pgs: 129 active+clean; 120 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-04-24T23:15:52.439 INFO:tasks.workunit.client.0.smithi033.stdout: 1 291151 13.58 MB/sec warmup 117 sec latency 11.759 ms 2024-04-24T23:15:53.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 292966 13.51 MB/sec warmup 118 sec latency 56.938 ms 2024-04-24T23:15:53.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:53 smithi038 ceph-mon[32450]: pgmap v467: 129 pgs: 129 active+clean; 124 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 291 op/s 2024-04-24T23:15:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:53 smithi117 ceph-mon[32529]: pgmap v467: 129 pgs: 129 active+clean; 124 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 291 op/s 2024-04-24T23:15:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:53 smithi033 ceph-mon[29201]: pgmap v467: 129 pgs: 129 active+clean; 124 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 291 op/s 2024-04-24T23:15:54.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 295100 13.52 MB/sec warmup 119 sec latency 31.880 ms 2024-04-24T23:15:55.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:55.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 -- 172.21.15.33:0/603573836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910075720 msgr2=0x7f7910073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 --2- 172.21.15.33:0/603573836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910075720 0x7f7910073b70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7900009900 tx=0x7f790002a480 comp rx=0 tx=0).stop 2024-04-24T23:15:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 -- 172.21.15.33:0/603573836 shutdown_connections 2024-04-24T23:15:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 --2- 172.21.15.33:0/603573836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7910109620 0x7f791010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 --2- 172.21.15.33:0/603573836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910074140 0x7f79100745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 --2- 172.21.15.33:0/603573836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910075720 0x7f7910073b70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 -- 172.21.15.33:0/603573836 >> 172.21.15.33:0/603573836 conn(0x7f79100fcd20 msgr2=0x7f79100ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 -- 172.21.15.33:0/603573836 shutdown_connections 2024-04-24T23:15:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.443+0000 7f7914a1b640 1 -- 172.21.15.33:0/603573836 wait complete. 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 Processor -- start 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 -- start start 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910074140 0x7f7910199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7910075720 0x7f791019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 0x7f79101a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f791010f630 con 0x7f7910109620 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f791010f4b0 con 0x7f7910075720 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f7914a1b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f791010f7b0 con 0x7f7910074140 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 0x7f79101a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 0x7f79101a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59142/0 (socket says 172.21.15.33:59142) 2024-04-24T23:15:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 -- 172.21.15.33:0/2139759070 learned_addr learned my addr 172.21.15.33:0/2139759070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790dd9b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7910075720 0x7f791019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790e59c640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910074140 0x7f7910199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 -- 172.21.15.33:0/2139759070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910074140 msgr2=0x7f7910199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910074140 0x7f7910199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 -- 172.21.15.33:0/2139759070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7910075720 msgr2=0x7f791019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7910075720 0x7f791019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.444+0000 7f790ed9d640 1 -- 172.21.15.33:0/2139759070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a1900 con 0x7f7910109620 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.445+0000 7f790ed9d640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 0x7f79101a1260 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f790800eb10 tx=0x7f790800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.445+0000 7f78ff7fe640 1 -- 172.21.15.33:0/2139759070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908004b20 con 0x7f7910109620 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.445+0000 7f78ff7fe640 1 -- 172.21.15.33:0/2139759070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7908004cc0 con 0x7f7910109620 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.445+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a1bf0 con 0x7f7910109620 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.445+0000 7f78ff7fe640 1 -- 172.21.15.33:0/2139759070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908010760 con 0x7f7910109620 2024-04-24T23:15:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.445+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a2110 con 0x7f7910109620 2024-04-24T23:15:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.447+0000 7f78ff7fe640 1 -- 172.21.15.33:0/2139759070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7908005020 con 0x7f7910109620 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.447+0000 7f79177fd740 1 Processor -- start 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 -- start start 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f78ff7fe640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78f80a2c10 0x7f78f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 0x55fd819ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819ed440 0x55fd819f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fd819486b0 con 0x55fd819f25b0 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fd818f0880 con 0x55fd819ed440 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f79177fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fd81944670 con 0x55fd819eca50 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f790dd9b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78f80a2c10 0x7f78f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f790e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 0x55fd819ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f78ff7fe640 1 -- 172.21.15.33:0/2139759070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f79080cf900 con 0x7f7910109620 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f790e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 0x55fd819ece70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:43360/0 (socket says 172.21.15.33:43360) 2024-04-24T23:15:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f790e59c640 1 -- 172.21.15.33:0/201083332 learned_addr learned my addr 172.21.15.33:0/201083332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f790ed9d640 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.448+0000 7f790dd9b640 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819ed440 0x55fd819f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790e59c640 1 -- 172.21.15.33:0/201083332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819ed440 msgr2=0x55fd819f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790e59c640 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819ed440 0x55fd819f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790e59c640 1 -- 172.21.15.33:0/201083332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 msgr2=0x55fd819f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790e59c640 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790e59c640 1 -- 172.21.15.33:0/201083332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd817267d0 con 0x55fd819eca50 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790e59c640 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 0x55fd819ece70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79000314e0 tx=0x7f7900004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d9ee3797e8fd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f790dd9b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78f80a2c10 0x7f78f80a50d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55fd819ed840 tx=0x7f7904009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f78fcff9640 1 -- 172.21.15.33:0/201083332 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900035040 con 0x55fd819eca50 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f78fcff9640 1 -- 172.21.15.33:0/201083332 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7900004430 con 0x55fd819eca50 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.449+0000 7f78fcff9640 1 -- 172.21.15.33:0/201083332 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000316b0 con 0x55fd819eca50 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 -- 172.21.15.33:0/201083332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 msgr2=0x55fd819ece70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 0x55fd819ece70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79000314e0 tx=0x7f7900004040 comp rx=0 tx=0).stop 2024-04-24T23:15:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 -- 172.21.15.33:0/201083332 shutdown_connections 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819ed440 0x55fd819f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 --2- 172.21.15.33:0/201083332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819eca50 0x55fd819ece70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 -- 172.21.15.33:0/201083332 >> 172.21.15.33:0/201083332 conn(0x55fd819df8b0 msgr2=0x55fd819dfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 -- 172.21.15.33:0/201083332 shutdown_connections 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 -- 172.21.15.33:0/201083332 wait complete. 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 -- start start 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.450+0000 7f79177fd740 1 Processor -- start 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fd819f5500 con 0x55fd819f25b0 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fd81903f50 con 0x55fd819eca50 2024-04-24T23:15:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fd819f5680 con 0x55fd819ed440 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 10 client.0 _ll_register_callbacks cb 0x55fd8193f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819fe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59160/0 (socket says 172.21.15.33:59160) 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 -- 172.21.15.33:0/232584331 learned_addr learned my addr 172.21.15.33:0/232584331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790dd9b640 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 -- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 msgr2=0x55fd819f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 -- 172.21.15.33:0/232584331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 msgr2=0x55fd819f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 -- 172.21.15.33:0/232584331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8165c3f0 con 0x55fd819f25b0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f790ed9d640 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819fe3c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f79080cd070 tx=0x7f790809ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64acd36f8a53f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f78f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f78f5ffb640 1 -- 172.21.15.33:0/232584331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790809bdd0 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f78f5ffb640 1 -- 172.21.15.33:0/232584331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f79080a45b0 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7900004430 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.451+0000 7f78f5ffb640 1 -- 172.21.15.33:0/232584331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f79080a0180 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 10 client.16203 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd8161c0f0 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f78f5ffb640 1 -- 172.21.15.33:0/232584331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7900004430 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f78f5ffb640 1 -- 172.21.15.33:0/232584331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f79080b1020 con 0x55fd819f25b0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 20 client.16203 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a02dd0 0x55fd81a05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 4 client.16203 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd81955bf0 con 0x55fd81a02dd0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.452+0000 7f790e59c640 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a02dd0 0x55fd81a05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f790e59c640 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a02dd0 0x55fd81a05270 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23be3ba4ad845b5 server_cookie=1511153954cce662 in_seq=0 out_seq=0 2024-04-24T23:15:55.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f78f5ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f78f5ffb640 1 -- 172.21.15.33:0/232584331 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd81955bf0 con 0x55fd81a02dd0 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f78f5ffb640 10 client.16203 handle_command_reply: tid=0 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 client.16203 shutdown 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 msgr2=0x55fd819fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819fe3c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f79080cd070 tx=0x7f790809ba00 comp rx=0 tx=0).stop 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 shutdown_connections 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a02dd0 0x55fd81a05270 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819fe3c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 --2- 172.21.15.33:0/232584331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.453+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 >> 172.21.15.33:0/232584331 conn(0x55fd819df8b0 msgr2=0x55fd819deaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.454+0000 7f78f5ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:55.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.454+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 shutdown_connections 2024-04-24T23:15:55.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.454+0000 7f79177fd740 1 -- 172.21.15.33:0/232584331 wait complete. 2024-04-24T23:15:55.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.454+0000 7f79177fd740 20 client.16203 trim_cache size 0 max 16384 2024-04-24T23:15:55.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.473+0000 7f79177fd740 1 Processor -- start 2024-04-24T23:15:55.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.473+0000 7f79177fd740 1 -- start start 2024-04-24T23:15:55.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f79177fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f79177fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f79177fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f79177fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79101aaa90 con 0x55fd819f25b0 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f79177fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79101a8360 con 0x55fd819eca50 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f79177fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79101a61b0 con 0x55fd819ed440 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819e2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59168/0 (socket says 172.21.15.33:59168) 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819e2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33326/0 (socket says 172.21.15.33:33326) 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 -- 172.21.15.33:0/1216020061 learned_addr learned my addr 172.21.15.33:0/1216020061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790dd9b640 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 -- 172.21.15.33:0/1216020061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 msgr2=0x55fd819fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 -- 172.21.15.33:0/1216020061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 msgr2=0x55fd819e2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819e2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 -- 172.21.15.33:0/1216020061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd819d1090 con 0x55fd819f25b0 2024-04-24T23:15:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790e59c640 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f790ed9d640 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819e2da0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f790809bd60 tx=0x7f790809b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8880b40ed159cf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f78f7fff640 1 -- 172.21.15.33:0/1216020061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080a9070 con 0x55fd819f25b0 2024-04-24T23:15:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f78f7fff640 1 -- 172.21.15.33:0/1216020061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79080a4a40 con 0x55fd819f25b0 2024-04-24T23:15:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.474+0000 7f78f7fff640 1 -- 172.21.15.33:0/1216020061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790809bdd0 con 0x55fd819f25b0 2024-04-24T23:15:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 -- 172.21.15.33:0/1216020061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 msgr2=0x55fd819e2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819e2da0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f790809bd60 tx=0x7f790809b7f0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 -- 172.21.15.33:0/1216020061 shutdown_connections 2024-04-24T23:15:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819f25b0 0x55fd819e2da0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819ed440 0x55fd819fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 --2- 172.21.15.33:0/1216020061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 -- 172.21.15.33:0/1216020061 >> 172.21.15.33:0/1216020061 conn(0x55fd819df8b0 msgr2=0x55fd819f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 -- 172.21.15.33:0/1216020061 shutdown_connections 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 -- 172.21.15.33:0/1216020061 wait complete. 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.475+0000 7f79177fd740 1 -- start start 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 Processor -- start 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 0x55fd819fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819f25b0 0x55fd819fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79101a6820 con 0x55fd819ed440 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79101a7c90 con 0x55fd819eca50 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79101a6ef0 con 0x55fd819f25b0 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f79177fd740 10 client.0 _ll_register_callbacks cb 0x7f7908004eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 0x55fd819fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 0x55fd819fc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59172/0 (socket says 172.21.15.33:59172) 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 -- 172.21.15.33:0/2549237164 learned_addr learned my addr 172.21.15.33:0/2549237164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790e59c640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790ed9d640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819f25b0 0x55fd819fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 -- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819f25b0 msgr2=0x55fd819fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819f25b0 0x55fd819fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 -- 172.21.15.33:0/2549237164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 msgr2=0x55fd819fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 -- 172.21.15.33:0/2549237164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd819fdd80 con 0x55fd819ed440 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790ed9d640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819f25b0 0x55fd819fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790e59c640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f790dd9b640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 0x55fd819fc7d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7904009f60 tx=0x7f7904005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba04b0ca2bf92c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f78f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.476+0000 7f78f5ffb640 1 -- 172.21.15.33:0/2549237164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790402e060 con 0x55fd819ed440 2024-04-24T23:15:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79080a4a40 con 0x55fd819ed440 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f78f5ffb640 1 -- 172.21.15.33:0/2549237164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f790401b070 con 0x55fd819ed440 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f78f5ffb640 1 -- 172.21.15.33:0/2549237164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7904003d00 con 0x55fd819ed440 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 10 client.16209 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd81a69b50 con 0x55fd819ed440 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f78f5ffb640 1 -- 172.21.15.33:0/2549237164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79080a4a40 con 0x55fd819ed440 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f78f5ffb640 1 -- 172.21.15.33:0/2549237164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f790401e040 con 0x55fd819ed440 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 10 client.16209 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 20 client.16209 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a6a5e0 0x55fd81a6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 4 client.16209 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fd8194ff40 con 0x55fd81a6a5e0 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.477+0000 7f790e59c640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a6a5e0 0x55fd81a6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f790e59c640 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a6a5e0 0x55fd81a6ca00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e707d2cf49269bf3 server_cookie=521f00f21a7376c9 in_seq=0 out_seq=0 2024-04-24T23:15:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f78f5ffb640 10 client.16209 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f78f5ffb640 1 -- 172.21.15.33:0/2549237164 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fd8194ff40 con 0x55fd81a6a5e0 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f78f5ffb640 10 client.16209 handle_command_reply: tid=0 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 client.16209 shutdown 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 msgr2=0x55fd819fc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 0x55fd819fc7d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7904009f60 tx=0x7f7904005f90 comp rx=0 tx=0).stop 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 shutdown_connections 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fd81a6a5e0 0x55fd81a6ca00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd819f25b0 0x55fd819fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd819ed440 0x55fd819fc7d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 --2- 172.21.15.33:0/2549237164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fd819eca50 0x55fd819fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 >> 172.21.15.33:0/2549237164 conn(0x55fd819df8b0 msgr2=0x55fd819de260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f78f5ffb640 0 client.16209 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 shutdown_connections 2024-04-24T23:15:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 1 -- 172.21.15.33:0/2549237164 wait complete. 2024-04-24T23:15:55.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.478+0000 7f79177fd740 20 client.16209 trim_cache size 0 max 16384 2024-04-24T23:15:55.487 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:15:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:15:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:15:55.487 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78f80a2c10 msgr2=0x7f78f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78f80a2c10 0x7f78f80a50d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55fd819ed840 tx=0x7f7904009530 comp rx=0 tx=0).stop 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 msgr2=0x7f79101a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 0x7f79101a1260 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f790800eb10 tx=0x7f790800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 shutdown_connections 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78f80a2c10 0x7f78f80a50d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7910109620 0x7f79101a1260 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7910075720 0x7f791019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 --2- 172.21.15.33:0/2139759070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910074140 0x7f7910199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 >> 172.21.15.33:0/2139759070 conn(0x7f79100fcd20 msgr2=0x7f79101055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 shutdown_connections 2024-04-24T23:15:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.479+0000 7f7914a1b640 1 -- 172.21.15.33:0/2139759070 wait complete. 2024-04-24T23:15:55.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:15:55.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/406473444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681020a0 msgr2=0x7f6b68104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/406473444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681020a0 0x7f6b68104490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009860 tx=0x7f6b5802a340 comp rx=0 tx=0).stop 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/406473444 shutdown_connections 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/406473444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68107300 0x7f6b681096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/406473444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b681049d0 0x7f6b68106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/406473444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681020a0 0x7f6b68104490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/406473444 >> 172.21.15.33:0/406473444 conn(0x7f6b680fba10 msgr2=0x7f6b680fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/406473444 shutdown_connections 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/406473444 wait complete. 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.600+0000 7f6b6d6c7640 1 Processor -- start 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 -- start start 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b681020a0 0x7f6b681978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 0x7f6b68197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b68107300 0x7f6b6819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6810d250 con 0x7f6b681020a0 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6810d0d0 con 0x7f6b681049d0 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6d6c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6810d3d0 con 0x7f6b68107300 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b681020a0 0x7f6b681978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 0x7f6b68197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 0x7f6b68197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33360/0 (socket says 172.21.15.33:33360) 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 -- 172.21.15.33:0/83473311 learned_addr learned my addr 172.21.15.33:0/83473311 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b6cec6640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b68107300 0x7f6b6819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 -- 172.21.15.33:0/83473311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b68107300 msgr2=0x7f6b6819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b68107300 0x7f6b6819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 -- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b681020a0 msgr2=0x7f6b681978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b681020a0 0x7f6b681978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b677fe640 1 -- 172.21.15.33:0/83473311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6819f500 con 0x7f6b681049d0 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.601+0000 7f6b67fff640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b681020a0 0x7f6b681978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.602+0000 7f6b677fe640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 0x7f6b68197e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e9c0 tx=0x7f6b5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.602+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00cd80 con 0x7f6b681049d0 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.602+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6819f7f0 con 0x7f6b681049d0 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.602+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5c005500 con 0x7f6b681049d0 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.602+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c010690 con 0x7f6b681049d0 2024-04-24T23:15:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.602+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6819fd40 con 0x7f6b681049d0 2024-04-24T23:15:55.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.603+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b680fd500 con 0x7f6b681049d0 2024-04-24T23:15:55.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.604+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6b5c0046c0 con 0x7f6b681049d0 2024-04-24T23:15:55.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.605+0000 7f6b657fa640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b500a2b90 0x7f6b500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.605+0000 7f6b67fff640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b500a2b90 0x7f6b500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.605+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6b5c0cf9b0 con 0x7f6b681049d0 2024-04-24T23:15:55.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.606+0000 7f6b67fff640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b500a2b90 0x7f6b500a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6b5802a930 tx=0x7f6b58030570 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.611+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6b5c053740 con 0x7f6b681049d0 2024-04-24T23:15:55.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:55 smithi038 ceph-mon[32450]: pgmap v468: 129 pgs: 129 active+clean; 124 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-04-24T23:15:55.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:55 smithi117 ceph-mon[32529]: pgmap v468: 129 pgs: 129 active+clean; 124 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-04-24T23:15:55.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:55 smithi033 ceph-mon[29201]: pgmap v468: 129 pgs: 129 active+clean; 124 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-04-24T23:15:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.850+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b6802d3c0 con 0x7f6b681049d0 2024-04-24T23:15:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.851+0000 7f6b657fa640 1 -- 172.21.15.33:0/83473311 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11860 (secure 0 0 0) 0x7f6b5c09c520 con 0x7f6b681049d0 2024-04-24T23:15:55.853 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:15:55.853 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:13:57.199404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:15:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 26 2024-04-24T23:15:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b500a2b90 msgr2=0x7f6b500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b500a2b90 0x7f6b500a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6b5802a930 tx=0x7f6b58030570 comp rx=0 tx=0).stop 2024-04-24T23:15:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 msgr2=0x7f6b68197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 0x7f6b68197e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e9c0 tx=0x7f6b5c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:15:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 shutdown_connections 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6b500a2b90 0x7f6b500a5050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b68107300 0x7f6b6819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b681049d0 0x7f6b68197e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 --2- 172.21.15.33:0/83473311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b681020a0 0x7f6b681978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 >> 172.21.15.33:0/83473311 conn(0x7f6b680fba10 msgr2=0x7f6b681029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 shutdown_connections 2024-04-24T23:15:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.853+0000 7f6b6d6c7640 1 -- 172.21.15.33:0/83473311 wait complete. 2024-04-24T23:15:55.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:15:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 -- 172.21.15.33:0/596084678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 msgr2=0x7fd684074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/596084678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684074580 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd674009900 tx=0x7fd67402a480 comp rx=0 tx=0).stop 2024-04-24T23:15:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 -- 172.21.15.33:0/596084678 shutdown_connections 2024-04-24T23:15:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/596084678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684109640 0x7fd68410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/596084678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684074580 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/596084678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd684075700 0x7fd684073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 -- 172.21.15.33:0/596084678 >> 172.21.15.33:0/596084678 conn(0x7fd6840fcd20 msgr2=0x7fd6840ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 -- 172.21.15.33:0/596084678 shutdown_connections 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 -- 172.21.15.33:0/596084678 wait complete. 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.981+0000 7fd68bc2a640 1 Processor -- start 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 -- start start 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684075700 0x7fd68419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd684109640 0x7fd6841a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410f610 con 0x7fd684074120 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410f490 con 0x7fd684075700 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68bc2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410f790 con 0x7fd684109640 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54706/0 (socket says 172.21.15.33:54706) 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 -- 172.21.15.33:0/636225396 learned_addr learned my addr 172.21.15.33:0/636225396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6891c5640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684075700 0x7fd68419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd684109640 0x7fd6841a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 -- 172.21.15.33:0/636225396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd684109640 msgr2=0x7fd6841a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd684109640 0x7fd6841a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 -- 172.21.15.33:0/636225396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684075700 msgr2=0x7fd68419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684075700 0x7fd68419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 -- 172.21.15.33:0/636225396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6841a1890 con 0x7fd684074120 2024-04-24T23:15:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6891c5640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684075700 0x7fd68419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.982+0000 7fd6899c6640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684199c80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd68000e9f0 tx=0x7fd68000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.983+0000 7fd67affd640 1 -- 172.21.15.33:0/636225396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68000cd80 con 0x7fd684074120 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.983+0000 7fd67affd640 1 -- 172.21.15.33:0/636225396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680005500 con 0x7fd684074120 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.983+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6841a1b80 con 0x7fd684074120 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.983+0000 7fd67affd640 1 -- 172.21.15.33:0/636225396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680010690 con 0x7fd684074120 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.983+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841a2080 con 0x7fd684074120 2024-04-24T23:15:55.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.985+0000 7fd67affd640 1 -- 172.21.15.33:0/636225396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd6800046c0 con 0x7fd684074120 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.987+0000 7fd68ea12740 1 Processor -- start 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.987+0000 7fd68ea12740 1 -- start start 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.987+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e7a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.987+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.987+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7fda0 0x559248e82230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.987+0000 7fd68ea12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559248dcd670 con 0x559248e7a240 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68ea12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559248d75970 con 0x559248e7fda0 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68ea12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559248dc9630 con 0x559248e7ac30 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68a1c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e7a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e7f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:43310/0 (socket says 172.21.15.33:43310) 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd6899c6640 1 -- 172.21.15.33:0/1932707255 learned_addr learned my addr 172.21.15.33:0/1932707255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd67affd640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6700a2b90 0x7fd6700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd6891c5640 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7fda0 0x559248e82230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68a1c7640 1 -- 172.21.15.33:0/1932707255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 msgr2=0x559248e7f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e7f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68a1c7640 1 -- 172.21.15.33:0/1932707255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7fda0 msgr2=0x559248e82230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7fda0 0x559248e82230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd68a1c7640 1 -- 172.21.15.33:0/1932707255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559248d99400 con 0x559248e7a240 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.988+0000 7fd67affd640 1 -- 172.21.15.33:0/636225396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd6800cf880 con 0x7fd684074120 2024-04-24T23:15:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.989+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6700a2b90 0x7fd6700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.989+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e7a660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009f00 tx=0x7fd67c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e814983d9fdf3a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.989+0000 7fd66ffff640 1 -- 172.21.15.33:0/1932707255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c020070 con 0x559248e7a240 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.989+0000 7fd66ffff640 1 -- 172.21.15.33:0/1932707255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c0045d0 con 0x559248e7a240 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.989+0000 7fd66ffff640 1 -- 172.21.15.33:0/1932707255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c01b420 con 0x559248e7a240 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 -- 172.21.15.33:0/1932707255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 msgr2=0x559248e7a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e7a660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009f00 tx=0x7fd67c011810 comp rx=0 tx=0).stop 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6700a2b90 0x7fd6700a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd67c01d000 tx=0x7fd67c01ba70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 -- 172.21.15.33:0/1932707255 shutdown_connections 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7fda0 0x559248e82230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e7f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1932707255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e7a660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.990+0000 7fd68ea12740 1 -- 172.21.15.33:0/1932707255 >> 172.21.15.33:0/1932707255 conn(0x559248e648a0 msgr2=0x559248e64ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.991+0000 7fd68ea12740 1 -- 172.21.15.33:0/1932707255 shutdown_connections 2024-04-24T23:15:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.992+0000 7fd68ea12740 1 -- 172.21.15.33:0/1932707255 wait complete. 2024-04-24T23:15:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 -- start start 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 Processor -- start 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 0x559248e8cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e8d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e94050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559248e82d50 con 0x559248e7fda0 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559248d89050 con 0x559248e7a240 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559248e82ed0 con 0x559248e7ac30 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.993+0000 7fd68ea12740 10 client.0 _ll_register_callbacks cb 0x559248ae0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 0x559248e8cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 0x559248e8cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36890/0 (socket says 172.21.15.33:36890) 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 -- 172.21.15.33:0/1169259189 learned_addr learned my addr 172.21.15.33:0/1169259189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e94050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6891c5640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e8d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 -- 172.21.15.33:0/1169259189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 msgr2=0x559248e8d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e8d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 -- 172.21.15.33:0/1169259189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 msgr2=0x559248e94050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e94050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.994+0000 7fd6899c6640 1 -- 172.21.15.33:0/1169259189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559248d5adc0 con 0x559248e7a240 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.995+0000 7fd6899c6640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 0x559248e8cad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd680098120 tx=0x7fd6800056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646b64c9fba02fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.995+0000 7fd66d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.996+0000 7fd66d7fa640 1 -- 172.21.15.33:0/1169259189 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68009b890 con 0x559248e7a240 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.996+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c0045d0 con 0x559248e7a240 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.996+0000 7fd66d7fa640 1 -- 172.21.15.33:0/1169259189 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd68009b1a0 con 0x559248e7a240 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.996+0000 7fd66d7fa640 1 -- 172.21.15.33:0/1169259189 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd680099e90 con 0x559248e7a240 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.996+0000 7fd68ea12740 10 client.25763 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.996+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559248ae13f0 con 0x559248e7a240 2024-04-24T23:15:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd66d7fa640 1 -- 172.21.15.33:0/1169259189 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c0045d0 con 0x559248e7a240 2024-04-24T23:15:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd66d7fa640 1 -- 172.21.15.33:0/1169259189 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd68009ac10 con 0x559248e7a240 2024-04-24T23:15:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd68ea12740 10 client.25763 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd68ea12740 10 client.25763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:15:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd68ea12740 20 client.25763 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248e98d60 0x559248e9b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd68ea12740 4 client.25763 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.997+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559248ddabb0 con 0x559248e98d60 2024-04-24T23:15:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.998+0000 7fd6891c5640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248e98d60 0x559248e9b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.999+0000 7fd6891c5640 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248e98d60 0x559248e9b1f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e37bac058aca4591 server_cookie=6cda9d84d5ce4f59 in_seq=0 out_seq=0 2024-04-24T23:15:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:55.999+0000 7fd66d7fa640 10 client.25763 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.001+0000 7fd66d7fa640 1 -- 172.21.15.33:0/1169259189 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559248ddabb0 con 0x559248e98d60 2024-04-24T23:15:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.001+0000 7fd66d7fa640 10 client.25763 handle_command_reply: tid=0 2024-04-24T23:15:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 client.25763 shutdown 2024-04-24T23:15:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 msgr2=0x559248e8cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 0x559248e8cad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd680098120 tx=0x7fd6800056b0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 shutdown_connections 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248e98d60 0x559248e9b1f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e94050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7ac30 0x559248e8d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 --2- 172.21.15.33:0/1169259189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7a240 0x559248e8cad0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd66d7fa640 0 client.25763 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.002+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 >> 172.21.15.33:0/1169259189 conn(0x559248e648a0 msgr2=0x559248e63b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.003+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 shutdown_connections 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.003+0000 7fd68ea12740 1 -- 172.21.15.33:0/1169259189 wait complete. 2024-04-24T23:15:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.003+0000 7fd68ea12740 20 client.25763 trim_cache size 0 max 16384 2024-04-24T23:15:56.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 Processor -- start 2024-04-24T23:15:56.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 -- start start 2024-04-24T23:15:56.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e67810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7fda0 0x559248e67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841a2250 con 0x559248e7a240 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6840711a0 con 0x559248e7ac30 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68ea12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd68406eff0 con 0x559248e7fda0 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e67810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e67810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54722/0 (socket says 172.21.15.33:54722) 2024-04-24T23:15:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 -- 172.21.15.33:0/4079721627 learned_addr learned my addr 172.21.15.33:0/4079721627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6891c5640 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd68a1c7640 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7fda0 0x559248e67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 -- 172.21.15.33:0/4079721627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7fda0 msgr2=0x559248e67d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7fda0 0x559248e67d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 -- 172.21.15.33:0/4079721627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 msgr2=0x559248e94010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e94010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.022+0000 7fd6899c6640 1 -- 172.21.15.33:0/4079721627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559248bab7d0 con 0x559248e7a240 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.023+0000 7fd6899c6640 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e67810 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c6a0 tx=0x7fd6800d5300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b338b179d148f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.023+0000 7fd66f7fe640 1 -- 172.21.15.33:0/4079721627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800a8030 con 0x559248e7a240 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.023+0000 7fd66f7fe640 1 -- 172.21.15.33:0/4079721627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68009c450 con 0x559248e7a240 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.023+0000 7fd66f7fe640 1 -- 172.21.15.33:0/4079721627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68009b6b0 con 0x559248e7a240 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.023+0000 7fd68ea12740 1 -- 172.21.15.33:0/4079721627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 msgr2=0x559248e67810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e67810 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c6a0 tx=0x7fd6800d5300 comp rx=0 tx=0).stop 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 -- 172.21.15.33:0/4079721627 shutdown_connections 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7fda0 0x559248e67d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e94010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 --2- 172.21.15.33:0/4079721627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7a240 0x559248e67810 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 -- 172.21.15.33:0/4079721627 >> 172.21.15.33:0/4079721627 conn(0x559248e648a0 msgr2=0x559248e63b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 -- 172.21.15.33:0/4079721627 shutdown_connections 2024-04-24T23:15:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 -- 172.21.15.33:0/4079721627 wait complete. 2024-04-24T23:15:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 -- start start 2024-04-24T23:15:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.024+0000 7fd68ea12740 1 Processor -- start 2024-04-24T23:15:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7a240 0x559248e91460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e92960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd68406f660 con 0x559248e7fda0 2024-04-24T23:15:56.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd684070ad0 con 0x559248e7ac30 2024-04-24T23:15:56.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd68406fd30 con 0x559248e7a240 2024-04-24T23:15:56.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 10 client.0 _ll_register_callbacks cb 0x7fd680009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:56.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68a1c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e92960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7a240 0x559248e91460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6891c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68a1c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e92960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54728/0 (socket says 172.21.15.33:54728) 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68a1c7640 1 -- 172.21.15.33:0/221608467 learned_addr learned my addr 172.21.15.33:0/221608467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 -- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 msgr2=0x559248e919a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e919a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 -- 172.21.15.33:0/221608467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 msgr2=0x559248e92960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e92960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 -- 172.21.15.33:0/221608467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559248e92f50 con 0x559248e7a240 2024-04-24T23:15:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd6899c6640 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7a240 0x559248e91460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c6a0 tx=0x7fd68009b3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c8b04b363aa0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd66d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:15:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd66d7fa640 1 -- 172.21.15.33:0/221608467 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800a8070 con 0x559248e7a240 2024-04-24T23:15:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68009c450 con 0x559248e7a240 2024-04-24T23:15:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.025+0000 7fd66d7fa640 1 -- 172.21.15.33:0/221608467 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd6800a1290 con 0x559248e7a240 2024-04-24T23:15:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd66d7fa640 1 -- 172.21.15.33:0/221608467 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd68009b6b0 con 0x559248e7a240 2024-04-24T23:15:56.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 10 client.25795 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:56.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559248e94250 con 0x559248e7a240 2024-04-24T23:15:56.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd66d7fa640 1 -- 172.21.15.33:0/221608467 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68009c450 con 0x559248e7a240 2024-04-24T23:15:56.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd66d7fa640 1 -- 172.21.15.33:0/221608467 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd6800862d0 con 0x559248e7a240 2024-04-24T23:15:56.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 10 client.25795 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:56.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 10 client.25795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:15:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 20 client.25795 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248eff680 0x559248f01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 4 client.25795 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559248dd4f00 con 0x559248eff680 2024-04-24T23:15:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.026+0000 7fd6891c5640 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248eff680 0x559248f01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd66d7fa640 10 client.25795 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd6891c5640 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248eff680 0x559248f01a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18ec543fa36d75b8 server_cookie=ac8dc6b96daf1c4f in_seq=0 out_seq=0 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd66d7fa640 1 -- 172.21.15.33:0/221608467 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559248dd4f00 con 0x559248eff680 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd66d7fa640 10 client.25795 handle_command_reply: tid=0 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 client.25795 shutdown 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7a240 msgr2=0x559248e91460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7a240 0x559248e91460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c6a0 tx=0x7fd68009b3d0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 shutdown_connections 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559248eff680 0x559248f01a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559248e7fda0 0x559248e92960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559248e7ac30 0x559248e919a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 --2- 172.21.15.33:0/221608467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559248e7a240 0x559248e91460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd66d7fa640 0 client.25795 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 >> 172.21.15.33:0/221608467 conn(0x559248e648a0 msgr2=0x559248e67810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 shutdown_connections 2024-04-24T23:15:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 1 -- 172.21.15.33:0/221608467 wait complete. 2024-04-24T23:15:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.027+0000 7fd68ea12740 20 client.25795 trim_cache size 0 max 16384 2024-04-24T23:15:56.045 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:15:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6700a2b90 msgr2=0x7fd6700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6700a2b90 0x7fd6700a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd67c01d000 tx=0x7fd67c01ba70 comp rx=0 tx=0).stop 2024-04-24T23:15:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 msgr2=0x7fd684199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684199c80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd68000e9f0 tx=0x7fd68000edf0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 shutdown_connections 2024-04-24T23:15:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd6700a2b90 0x7fd6700a5050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd684109640 0x7fd6841a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd684075700 0x7fd68419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 --2- 172.21.15.33:0/636225396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd684074120 0x7fd684199c80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 >> 172.21.15.33:0/636225396 conn(0x7fd6840fcd20 msgr2=0x7fd6841055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 shutdown_connections 2024-04-24T23:15:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.028+0000 7fd68bc2a640 1 -- 172.21.15.33:0/636225396 wait complete. 2024-04-24T23:15:56.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 -- 172.21.15.33:0/4037421920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88100680 msgr2=0x7f9f88100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/4037421920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88100680 0x7f9f88100aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009860 tx=0x7f9f7802a340 comp rx=0 tx=0).stop 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 -- 172.21.15.33:0/4037421920 shutdown_connections 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/4037421920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f88108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/4037421920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88101870 0x7f9f88101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/4037421920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88100680 0x7f9f88100aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 -- 172.21.15.33:0/4037421920 >> 172.21.15.33:0/4037421920 conn(0x7f9f880fbbf0 msgr2=0x7f9f880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 -- 172.21.15.33:0/4037421920 shutdown_connections 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.159+0000 7f9f8cf34640 1 -- 172.21.15.33:0/4037421920 wait complete. 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 Processor -- start 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 -- start start 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88100680 0x7f9f88197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 0x7f9f88198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f8819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810c2d0 con 0x7f9f88102230 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810c150 con 0x7f9f88101870 2024-04-24T23:15:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8cf34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810c450 con 0x7f9f88100680 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f86d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f8819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 0x7f9f88198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f86d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f8819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54748/0 (socket says 172.21.15.33:54748) 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 0x7f9f88198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36932/0 (socket says 172.21.15.33:36932) 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f86d9d640 1 -- 172.21.15.33:0/1452704794 learned_addr learned my addr 172.21.15.33:0/1452704794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88100680 0x7f9f88197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 -- 172.21.15.33:0/1452704794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88100680 msgr2=0x7f9f88197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88100680 0x7f9f88197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 -- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 msgr2=0x7f9f8819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f8819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f85d9b640 1 -- 172.21.15.33:0/1452704794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8819f710 con 0x7f9f88101870 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f8659c640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88100680 0x7f9f88197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.160+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f8819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.161+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 0x7f9f88198030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000ec80 tx=0x7f9f8000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.161+0000 7f9f7f7fe640 1 -- 172.21.15.33:0/1452704794 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80003b70 con 0x7f9f88101870 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.161+0000 7f9f7f7fe640 1 -- 172.21.15.33:0/1452704794 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f80003d10 con 0x7f9f88101870 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.161+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8819fa00 con 0x7f9f88101870 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.161+0000 7f9f7f7fe640 1 -- 172.21.15.33:0/1452704794 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80010930 con 0x7f9f88101870 2024-04-24T23:15:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.161+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8819ff50 con 0x7f9f88101870 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.163+0000 7f9f8fc43740 1 Processor -- start 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 -- start start 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f7f7fe640 1 -- 172.21.15.33:0/1452704794 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f80003ec0 con 0x7f9f88101870 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624026a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x55862402bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x55862402e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558623f86670 con 0x558624027030 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558623f2e970 con 0x55862402c1a0 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558623f82630 con 0x558624026640 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x55862402bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x55862402bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54760/0 (socket says 172.21.15.33:54760) 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 -- 172.21.15.33:0/4246939762 learned_addr learned my addr 172.21.15.33:0/4246939762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x55862402e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624026a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 -- 172.21.15.33:0/4246939762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 msgr2=0x558624026a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624026a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 -- 172.21.15.33:0/4246939762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 msgr2=0x55862402e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x55862402e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 -- 172.21.15.33:0/4246939762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558623f52400 con 0x558624027030 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.164+0000 7f9f8659c640 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x55862402bb30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009860 tx=0x7f9f780095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bc284b9f04a854 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.165+0000 7f9f7f7fe640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f740a2b90 0x7f9f740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.165+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f740a2b90 0x7f9f740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.165+0000 7f9f7f7fe640 1 -- 172.21.15.33:0/1452704794 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9f800cfb10 con 0x7f9f88101870 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.165+0000 7f9f7cff9640 1 -- 172.21.15.33:0/4246939762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78037040 con 0x558624027030 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.165+0000 7f9f7cff9640 1 -- 172.21.15.33:0/4246939762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78030850 con 0x558624027030 2024-04-24T23:15:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.165+0000 7f9f7cff9640 1 -- 172.21.15.33:0/4246939762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004900 con 0x558624027030 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4246939762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 msgr2=0x55862402bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x55862402bb30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009860 tx=0x7f9f780095a0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f740a2b90 0x7f9f740a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x558624027890 tx=0x7f9f70009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4246939762 shutdown_connections 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x55862402e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x55862402bb30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4246939762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624026a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4246939762 >> 172.21.15.33:0/4246939762 conn(0x55862401d8a0 msgr2=0x55862401dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4246939762 shutdown_connections 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4246939762 wait complete. 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 -- start start 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.166+0000 7f9f8fc43740 1 Processor -- start 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624030740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624030c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x558624037cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55862402f0f0 con 0x558624027030 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558623f42050 con 0x55862402c1a0 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55862402f270 con 0x558624026640 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 10 client.0 _ll_register_callbacks cb 0x558623c99be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624030c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624030740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624030c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54762/0 (socket says 172.21.15.33:54762) 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 -- 172.21.15.33:0/2451661541 learned_addr learned my addr 172.21.15.33:0/2451661541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x558624037cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 -- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 msgr2=0x558624030740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624030740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 -- 172.21.15.33:0/2451661541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 msgr2=0x558624037cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x558624037cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 -- 172.21.15.33:0/2451661541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558623f13dc0 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8659c640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624030740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624030c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9f800989b0 tx=0x7f9f80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f80bc147bad667 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f6dffb640 1 -- 172.21.15.33:0/2451661541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800d5020 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f6dffb640 1 -- 172.21.15.33:0/2451661541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9f800a49c0 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78030850 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.167+0000 7f9f6dffb640 1 -- 172.21.15.33:0/2451661541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f8009bb20 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 10 client.16245 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558623c9a3f0 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f6dffb640 1 -- 172.21.15.33:0/2451661541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78030850 con 0x558624027030 2024-04-24T23:15:56.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f6dffb640 1 -- 172.21.15.33:0/2451661541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9f8009b430 con 0x558624027030 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 10 client.16245 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 20 client.16245 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55862403c560 0x55862403ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 4 client.16245 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558623f93bb0 con 0x55862403c560 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8659c640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55862403c560 0x55862403ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f6dffb640 10 client.16245 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.168+0000 7f9f8659c640 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55862403c560 0x55862403ea00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ddacbdc3792bde server_cookie=6f8fc64ed20ed00 in_seq=0 out_seq=0 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f6dffb640 1 -- 172.21.15.33:0/2451661541 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558623f93bb0 con 0x55862403c560 2024-04-24T23:15:56.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f6dffb640 10 client.16245 handle_command_reply: tid=0 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 client.16245 shutdown 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 msgr2=0x558624030c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624030c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9f800989b0 tx=0x7f9f80009840 comp rx=0 tx=0).stop 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 shutdown_connections 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55862403c560 0x55862403ea00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55862402c1a0 0x558624037cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624030c80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/2451661541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558624026640 0x558624030740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 >> 172.21.15.33:0/2451661541 conn(0x55862401d8a0 msgr2=0x55862401cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f6dffb640 0 client.16245 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 shutdown_connections 2024-04-24T23:15:56.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 1 -- 172.21.15.33:0/2451661541 wait complete. 2024-04-24T23:15:56.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.169+0000 7f9f8fc43740 20 client.16245 trim_cache size 0 max 16384 2024-04-24T23:15:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 Processor -- start 2024-04-24T23:15:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 -- start start 2024-04-24T23:15:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 0x55862403e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624027030 0x558624037c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x5586240207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881aaac0 con 0x558624026640 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881a8390 con 0x558624027030 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881a61c0 con 0x55862402c1a0 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 0x55862403e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 0x55862403e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54764/0 (socket says 172.21.15.33:54764) 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 -- 172.21.15.33:0/95398625 learned_addr learned my addr 172.21.15.33:0/95398625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624027030 0x558624037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x5586240207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 -- 172.21.15.33:0/95398625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 msgr2=0x5586240207b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x5586240207b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 -- 172.21.15.33:0/95398625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624027030 msgr2=0x558624037c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624027030 0x558624037c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f8659c640 1 -- 172.21.15.33:0/95398625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558623d647d0 con 0x558624026640 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624027030 0x558624037c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.183+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x5586240207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.184+0000 7f9f8659c640 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 0x55862403e570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009830 tx=0x7f9f7802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c5afdbcc22bd7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.184+0000 7f9f6ffff640 1 -- 172.21.15.33:0/95398625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78037040 con 0x558624026640 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.184+0000 7f9f6ffff640 1 -- 172.21.15.33:0/95398625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78030830 con 0x558624026640 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.184+0000 7f9f6ffff640 1 -- 172.21.15.33:0/95398625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78003c00 con 0x558624026640 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.184+0000 7f9f8fc43740 1 -- 172.21.15.33:0/95398625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 msgr2=0x55862403e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.184+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 0x55862403e570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009830 tx=0x7f9f7802a340 comp rx=0 tx=0).stop 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- 172.21.15.33:0/95398625 shutdown_connections 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x5586240207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624027030 0x558624037c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/95398625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624026640 0x55862403e570 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- 172.21.15.33:0/95398625 >> 172.21.15.33:0/95398625 conn(0x55862401d8a0 msgr2=0x55862401cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- 172.21.15.33:0/95398625 shutdown_connections 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- 172.21.15.33:0/95398625 wait complete. 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- start start 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 Processor -- start 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624026640 0x558624035970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624035eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x558624036ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881a6850 con 0x558624027030 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881a7cc0 con 0x558624026640 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881a6f20 con 0x55862402c1a0 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8fc43740 10 client.0 _ll_register_callbacks cb 0x7f9f8000ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624035eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624035eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54774/0 (socket says 172.21.15.33:54774) 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f85d9b640 1 -- 172.21.15.33:0/4251245071 learned_addr learned my addr 172.21.15.33:0/4251245071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f8659c640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624026640 0x558624035970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.185+0000 7f9f86d9d640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x558624036ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f85d9b640 1 -- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 msgr2=0x558624036ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x558624036ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f85d9b640 1 -- 172.21.15.33:0/4251245071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624026640 msgr2=0x558624035970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624026640 0x558624035970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f85d9b640 1 -- 172.21.15.33:0/4251245071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586240374b0 con 0x558624027030 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f85d9b640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624035eb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9f80098440 tx=0x7f9f800d3570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57afd63a53a532 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f6dffb640 1 -- 172.21.15.33:0/4251245071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800d5070 con 0x558624027030 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f6dffb640 1 -- 172.21.15.33:0/4251245071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9f80096b50 con 0x558624027030 2024-04-24T23:15:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78030830 con 0x558624027030 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f6dffb640 1 -- 172.21.15.33:0/4251245071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f8009b940 con 0x558624027030 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f8fc43740 10 client.16251 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586240a30c0 con 0x558624027030 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.186+0000 7f9f6dffb640 1 -- 172.21.15.33:0/4251245071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78030830 con 0x558624027030 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f6dffb640 1 -- 172.21.15.33:0/4251245071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9f8009aea0 con 0x558624027030 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8fc43740 10 client.16251 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8fc43740 10 client.16251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8fc43740 20 client.16251 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5586240a3950 0x5586240a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8fc43740 4 client.16251 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558623f8df00 con 0x5586240a3950 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8659c640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5586240a3950 0x5586240a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f8659c640 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5586240a3950 0x5586240a5d70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb425b5b15f10839 server_cookie=747562b53b7328ab in_seq=0 out_seq=0 2024-04-24T23:15:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.187+0000 7f9f6dffb640 10 client.16251 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f6dffb640 1 -- 172.21.15.33:0/4251245071 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558623f8df00 con 0x5586240a3950 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f6dffb640 10 client.16251 handle_command_reply: tid=0 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 client.16251 shutdown 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 msgr2=0x558624035eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624035eb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9f80098440 tx=0x7f9f800d3570 comp rx=0 tx=0).stop 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 shutdown_connections 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5586240a3950 0x5586240a5d70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55862402c1a0 0x558624036ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558624027030 0x558624035eb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f6dffb640 0 client.16251 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 --2- 172.21.15.33:0/4251245071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558624026640 0x558624035970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 >> 172.21.15.33:0/4251245071 conn(0x55862401d8a0 msgr2=0x5586240207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 shutdown_connections 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 1 -- 172.21.15.33:0/4251245071 wait complete. 2024-04-24T23:15:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8fc43740 20 client.16251 trim_cache size 0 max 16384 2024-04-24T23:15:56.198 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f740a2b90 msgr2=0x7f9f740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f740a2b90 0x7f9f740a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x558624027890 tx=0x7f9f70009260 comp rx=0 tx=0).stop 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 msgr2=0x7f9f88198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.188+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 0x7f9f88198030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000ec80 tx=0x7f9f8000c6a0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 shutdown_connections 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9f740a2b90 0x7f9f740a5050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f88102230 0x7f9f8819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88101870 0x7f9f88198030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 --2- 172.21.15.33:0/1452704794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f88100680 0x7f9f88197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 >> 172.21.15.33:0/1452704794 conn(0x7f9f880fbbf0 msgr2=0x7f9f88068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 shutdown_connections 2024-04-24T23:15:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.189+0000 7f9f8cf34640 1 -- 172.21.15.33:0/1452704794 wait complete. 2024-04-24T23:15:56.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:15:56.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 -- 172.21.15.33:0/4233820175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0102230 msgr2=0x7f48a0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 --2- 172.21.15.33:0/4233820175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0102230 0x7f48a0108ac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ca0 tx=0x7f489802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 -- 172.21.15.33:0/4233820175 shutdown_connections 2024-04-24T23:15:56.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 --2- 172.21.15.33:0/4233820175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0102230 0x7f48a0108ac0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 --2- 172.21.15.33:0/4233820175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48a0101870 0x7f48a0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 --2- 172.21.15.33:0/4233820175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48a0100680 0x7f48a0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 -- 172.21.15.33:0/4233820175 >> 172.21.15.33:0/4233820175 conn(0x7f48a00fbbf0 msgr2=0x7f48a00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 -- 172.21.15.33:0/4233820175 shutdown_connections 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 -- 172.21.15.33:0/4233820175 wait complete. 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.319+0000 7f48a51a2640 1 Processor -- start 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 -- start start 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 0x7f48a0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48a0101870 0x7f48a0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48a0102230 0x7f48a019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48a010c300 con 0x7f48a0100680 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48a010c180 con 0x7f48a0102230 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f48a51a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48a010c480 con 0x7f48a0101870 2024-04-24T23:15:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 0x7f48a0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 0x7f48a0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54792/0 (socket says 172.21.15.33:54792) 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 -- 172.21.15.33:0/1912664349 learned_addr learned my addr 172.21.15.33:0/1912664349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489f59e640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48a0102230 0x7f48a019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f4897fff640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48a0101870 0x7f48a0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 -- 172.21.15.33:0/1912664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48a0101870 msgr2=0x7f48a0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48a0101870 0x7f48a0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 -- 172.21.15.33:0/1912664349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48a0102230 msgr2=0x7f48a019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48a0102230 0x7f48a019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 -- 172.21.15.33:0/1912664349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a019f6b0 con 0x7f48a0100680 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.320+0000 7f489ed9d640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 0x7f48a0197a90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f489000e9c0 tx=0x7f489000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.321+0000 7f489cd99640 1 -- 172.21.15.33:0/1912664349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489000cd80 con 0x7f48a0100680 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.321+0000 7f489cd99640 1 -- 172.21.15.33:0/1912664349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48900035f0 con 0x7f48a0100680 2024-04-24T23:15:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.321+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a019f9a0 con 0x7f48a0100680 2024-04-24T23:15:56.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.321+0000 7f489cd99640 1 -- 172.21.15.33:0/1912664349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890010690 con 0x7f48a0100680 2024-04-24T23:15:56.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.321+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a019fef0 con 0x7f48a0100680 2024-04-24T23:15:56.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f489cd99640 1 -- 172.21.15.33:0/1912664349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4890010830 con 0x7f48a0100680 2024-04-24T23:15:56.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 Processor -- start 2024-04-24T23:15:56.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 -- start start 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fb3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3c20 0x5608e8fb8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fbb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5608e8f17670 con 0x5608e8fb3c20 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5608e8ebf970 con 0x5608e8fb3230 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.323+0000 7f48a7f91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5608e8f13630 con 0x5608e8fb8d90 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fb3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3c20 0x5608e8fb8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fb3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36994/0 (socket says 172.21.15.33:36994) 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 -- 172.21.15.33:0/668492214 learned_addr learned my addr 172.21.15.33:0/668492214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f4897fff640 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fbb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 -- 172.21.15.33:0/668492214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 msgr2=0x5608e8fbb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fbb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 -- 172.21.15.33:0/668492214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3c20 msgr2=0x5608e8fb8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3c20 0x5608e8fb8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 -- 172.21.15.33:0/668492214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e8ee3400 con 0x5608e8fb3230 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489cd99640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f488c0a2b90 0x7f488c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489ed9d640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f488c0a2b90 0x7f488c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489f59e640 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fb3650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ca0 tx=0x7f4898004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a237fe4f8dde75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f489cd99640 1 -- 172.21.15.33:0/1912664349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f48900cf830 con 0x7f48a0100680 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.324+0000 7f48957fa640 1 -- 172.21.15.33:0/668492214 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48980098c0 con 0x5608e8fb3230 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.325+0000 7f48957fa640 1 -- 172.21.15.33:0/668492214 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4898036040 con 0x5608e8fb3230 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.325+0000 7f48957fa640 1 -- 172.21.15.33:0/668492214 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898004250 con 0x5608e8fb3230 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.325+0000 7f48a7f91740 1 -- 172.21.15.33:0/668492214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 msgr2=0x5608e8fb3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.325+0000 7f48a7f91740 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fb3650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ca0 tx=0x7f4898004a40 comp rx=0 tx=0).stop 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.325+0000 7f489ed9d640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f488c0a2b90 0x7f488c0a5050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48900cd310 tx=0x7f4890095040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- 172.21.15.33:0/668492214 shutdown_connections 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fbb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3c20 0x5608e8fb8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 --2- 172.21.15.33:0/668492214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fb3650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- 172.21.15.33:0/668492214 >> 172.21.15.33:0/668492214 conn(0x5608e8fae8a0 msgr2=0x5608e8faeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- 172.21.15.33:0/668492214 shutdown_connections 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- 172.21.15.33:0/668492214 wait complete. 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- start start 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 Processor -- start 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb8d90 0x5608e8fc4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5608e8fbbb00 con 0x5608e8fb3230 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5608e8ed3050 con 0x5608e8fb8d90 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5608e8fbbc80 con 0x5608e8fb3c20 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.326+0000 7f48a7f91740 10 client.0 _ll_register_callbacks cb 0x5608e8c2abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fbd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54810/0 (socket says 172.21.15.33:54810) 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 -- 172.21.15.33:0/1043726288 learned_addr learned my addr 172.21.15.33:0/1043726288 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f4897fff640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb8d90 0x5608e8fc4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 -- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 msgr2=0x5608e8fbd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fbd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 -- 172.21.15.33:0/1043726288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb8d90 msgr2=0x5608e8fc4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb8d90 0x5608e8fc4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 -- 172.21.15.33:0/1043726288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e8ea4dc0 con 0x5608e8fb3230 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f4897fff640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fbd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f489f59e640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fbd0a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f489802f470 tx=0x7f4898005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cae688ce9f21e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48867fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48867fc640 1 -- 172.21.15.33:0/1043726288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489802f610 con 0x5608e8fb3230 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48867fc640 1 -- 172.21.15.33:0/1043726288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f489802f7b0 con 0x5608e8fb3230 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4898036040 con 0x5608e8fb3230 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48867fc640 1 -- 172.21.15.33:0/1043726288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f489805f050 con 0x5608e8fb3230 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48a7f91740 10 client.16263 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.327+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608e8c2b3f0 con 0x5608e8fb3230 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48867fc640 1 -- 172.21.15.33:0/1043726288 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4898036040 con 0x5608e8fb3230 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48867fc640 1 -- 172.21.15.33:0/1043726288 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4898030af0 con 0x5608e8fb3230 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48a7f91740 10 client.16263 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48a7f91740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48a7f91740 20 client.16263 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e8fc8b60 0x5608e8fcaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48a7f91740 4 client.16263 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e8f24bb0 con 0x5608e8fc8b60 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f4897fff640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e8fc8b60 0x5608e8fcaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f48867fc640 10 client.16263 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.328+0000 7f4897fff640 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e8fc8b60 0x5608e8fcaf50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d0efb6651066b4 server_cookie=dd64d6bc990c3d38 in_seq=0 out_seq=0 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48867fc640 1 -- 172.21.15.33:0/1043726288 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608e8f24bb0 con 0x5608e8fc8b60 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48867fc640 10 client.16263 handle_command_reply: tid=0 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 client.16263 shutdown 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 msgr2=0x5608e8fbd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fbd0a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f489802f470 tx=0x7f4898005f70 comp rx=0 tx=0).stop 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 shutdown_connections 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e8fc8b60 0x5608e8fcaf50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb8d90 0x5608e8fc4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fbd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1043726288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fbd0a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48867fc640 0 client.16263 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 >> 172.21.15.33:0/1043726288 conn(0x5608e8fae8a0 msgr2=0x5608e8fadb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 shutdown_connections 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 1 -- 172.21.15.33:0/1043726288 wait complete. 2024-04-24T23:15:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.329+0000 7f48a7f91740 20 client.16263 trim_cache size 0 max 16384 2024-04-24T23:15:56.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.346+0000 7f48a7f91740 1 Processor -- start 2024-04-24T23:15:56.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 -- start start 2024-04-24T23:15:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fb1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 0x5608e8fc45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fb1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01accb0 con 0x5608e8fb3230 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01aa580 con 0x5608e8fb3c20 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f48a7f91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01a83d0 con 0x5608e8fb8d90 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f4897fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 0x5608e8fc45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f4897fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 0x5608e8fc45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37002/0 (socket says 172.21.15.33:37002) 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f4897fff640 1 -- 172.21.15.33:0/3464915009 learned_addr learned my addr 172.21.15.33:0/3464915009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f489f59e640 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fb1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.347+0000 7f489f59e640 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fb1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4897fff640 1 -- 172.21.15.33:0/3464915009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 msgr2=0x5608e8fb1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4897fff640 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fb1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4897fff640 1 -- 172.21.15.33:0/3464915009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 msgr2=0x5608e8fb1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4897fff640 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fb1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4897fff640 1 -- 172.21.15.33:0/3464915009 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e8cf57d0 con 0x5608e8fb3c20 2024-04-24T23:15:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f489f59e640 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fb1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4897fff640 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 0x5608e8fc45e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4888012600 tx=0x7f4888005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30305212a831988 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4894ff9640 1 -- 172.21.15.33:0/3464915009 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48880203d0 con 0x5608e8fb3c20 2024-04-24T23:15:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4894ff9640 1 -- 172.21.15.33:0/3464915009 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4888010430 con 0x5608e8fb3c20 2024-04-24T23:15:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.348+0000 7f4894ff9640 1 -- 172.21.15.33:0/3464915009 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488801e680 con 0x5608e8fb3c20 2024-04-24T23:15:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 -- 172.21.15.33:0/3464915009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 msgr2=0x5608e8fc45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 0x5608e8fc45e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4888012600 tx=0x7f4888005f90 comp rx=0 tx=0).stop 2024-04-24T23:15:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 -- 172.21.15.33:0/3464915009 shutdown_connections 2024-04-24T23:15:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb8d90 0x5608e8fb1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3c20 0x5608e8fc45e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 --2- 172.21.15.33:0/3464915009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb3230 0x5608e8fb1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 -- 172.21.15.33:0/3464915009 >> 172.21.15.33:0/3464915009 conn(0x5608e8fae8a0 msgr2=0x5608e8fbccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 -- 172.21.15.33:0/3464915009 shutdown_connections 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.349+0000 7f48a7f91740 1 -- 172.21.15.33:0/3464915009 wait complete. 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 -- start start 2024-04-24T23:15:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 Processor -- start 2024-04-24T23:15:56.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fc1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fc1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb8d90 0x5608e8fc2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01a8a40 con 0x5608e8fb8d90 2024-04-24T23:15:56.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01a9eb0 con 0x5608e8fb3230 2024-04-24T23:15:56.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01a9110 con 0x5608e8fb3c20 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f4897fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fc1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f4897fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fc1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:43432/0 (socket says 172.21.15.33:43432) 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f4897fff640 1 -- 172.21.15.33:0/1045900747 learned_addr learned my addr 172.21.15.33:0/1045900747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f489f59e640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fc1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f48a7f91740 10 client.0 _ll_register_callbacks cb 0x7f48900d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.350+0000 7f489f59e640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb8d90 0x5608e8fc2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f4897fff640 1 -- 172.21.15.33:0/1045900747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 msgr2=0x5608e8fc1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f4897fff640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fc1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f4897fff640 1 -- 172.21.15.33:0/1045900747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb8d90 msgr2=0x5608e8fc2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:56.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f4897fff640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb8d90 0x5608e8fc2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f4897fff640 1 -- 172.21.15.33:0/1045900747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e8fc3270 con 0x5608e8fb3c20 2024-04-24T23:15:56.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f489f59e640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb8d90 0x5608e8fc2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f4897fff640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fc1c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48880058c0 tx=0x7f48880106f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cba33485611ae0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48867fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:15:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48867fc640 1 -- 172.21.15.33:0/1045900747 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888004050 con 0x5608e8fb3c20 2024-04-24T23:15:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4888010430 con 0x5608e8fb3c20 2024-04-24T23:15:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48867fc640 1 -- 172.21.15.33:0/1045900747 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f48880041f0 con 0x5608e8fb3c20 2024-04-24T23:15:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48867fc640 1 -- 172.21.15.33:0/1045900747 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4888037050 con 0x5608e8fb3c20 2024-04-24T23:15:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48a7f91740 10 client.25828 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.351+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608e8fc45f0 con 0x5608e8fb3c20 2024-04-24T23:15:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48867fc640 1 -- 172.21.15.33:0/1045900747 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4888010430 con 0x5608e8fb3c20 2024-04-24T23:15:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48867fc640 1 -- 172.21.15.33:0/1045900747 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4888004b70 con 0x5608e8fb3c20 2024-04-24T23:15:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48a7f91740 10 client.25828 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48a7f91740 10 client.25828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48a7f91740 20 client.25828 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e902f560 0x5608e9031980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48a7f91740 4 client.25828 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608e8f1ef00 con 0x5608e902f560 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f489f59e640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e902f560 0x5608e9031980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f48867fc640 10 client.25828 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.352+0000 7f489f59e640 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e902f560 0x5608e9031980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f138b4a6bed976d8 server_cookie=16bbb4a61fb382f4 in_seq=0 out_seq=0 2024-04-24T23:15:56.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48867fc640 1 -- 172.21.15.33:0/1045900747 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608e8f1ef00 con 0x5608e902f560 2024-04-24T23:15:56.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48867fc640 10 client.25828 handle_command_reply: tid=0 2024-04-24T23:15:56.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 client.25828 shutdown 2024-04-24T23:15:56.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 msgr2=0x5608e8fc1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fc1c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48880058c0 tx=0x7f48880106f0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 shutdown_connections 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5608e902f560 0x5608e9031980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608e8fb8d90 0x5608e8fc2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5608e8fb3c20 0x5608e8fc1c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 --2- 172.21.15.33:0/1045900747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5608e8fb3230 0x5608e8fc1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48867fc640 0 client.25828 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 >> 172.21.15.33:0/1045900747 conn(0x5608e8fae8a0 msgr2=0x5608e8fb46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 shutdown_connections 2024-04-24T23:15:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 1 -- 172.21.15.33:0/1045900747 wait complete. 2024-04-24T23:15:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.353+0000 7f48a7f91740 20 client.25828 trim_cache size 0 max 16384 2024-04-24T23:15:56.370 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:15:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f488c0a2b90 msgr2=0x7f488c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f488c0a2b90 0x7f488c0a5050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48900cd310 tx=0x7f4890095040 comp rx=0 tx=0).stop 2024-04-24T23:15:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 msgr2=0x7f48a0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 0x7f48a0197a90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f489000e9c0 tx=0x7f489000ee80 comp rx=0 tx=0).stop 2024-04-24T23:15:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 shutdown_connections 2024-04-24T23:15:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f488c0a2b90 0x7f488c0a5050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48a0102230 0x7f48a019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48a0101870 0x7f48a0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 --2- 172.21.15.33:0/1912664349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48a0100680 0x7f48a0197a90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 >> 172.21.15.33:0/1912664349 conn(0x7f48a00fbbf0 msgr2=0x7f48a00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 shutdown_connections 2024-04-24T23:15:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:56.354+0000 7f48a51a2640 1 -- 172.21.15.33:0/1912664349 wait complete. 2024-04-24T23:15:56.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 300472 8.08 MB/sec execute 1 sec latency 23.657 ms 2024-04-24T23:15:56.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:56 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/83473311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:56.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:56 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/83473311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:56.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:56 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/83473311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:57.375 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:15:57.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:15:57.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:15:57.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 302466 10.12 MB/sec execute 2 sec latency 33.761 ms 2024-04-24T23:15:57.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2671715884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 msgr2=0x7fa1e81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2671715884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e81045f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009900 tx=0x7fa1e402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2671715884 shutdown_connections 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2671715884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8107460 0x7fa1e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2671715884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8104b30 0x7fa1e8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2671715884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e81045f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.493+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2671715884 >> 172.21.15.33:0/2671715884 conn(0x7fa1e80fbbf0 msgr2=0x7fa1e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2671715884 shutdown_connections 2024-04-24T23:15:57.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2671715884 wait complete. 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 Processor -- start 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 -- start start 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8104b30 0x7fa1e8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8107460 0x7fa1e819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d450 con 0x7fa1e8104b30 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d2d0 con 0x7fa1e8102200 2024-04-24T23:15:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.494+0000 7fa1ef7b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d5d0 con 0x7fa1e8107460 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ed553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8104b30 0x7fa1e8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ed553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e8197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37042/0 (socket says 172.21.15.33:37042) 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ed553640 1 -- 172.21.15.33:0/2167515784 learned_addr learned my addr 172.21.15.33:0/2167515784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1edd54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8107460 0x7fa1e819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 -- 172.21.15.33:0/2167515784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8107460 msgr2=0x7fa1e819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8107460 0x7fa1e819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 -- 172.21.15.33:0/2167515784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 msgr2=0x7fa1e8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 -- 172.21.15.33:0/2167515784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e819f740 con 0x7fa1e8104b30 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8104b30 0x7fa1e8198060 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9c0 tx=0x7fa1d800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1de7fc640 1 -- 172.21.15.33:0/2167515784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800ccc0 con 0x7fa1e8104b30 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e819fa30 con 0x7fa1e8104b30 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1de7fc640 1 -- 172.21.15.33:0/2167515784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8010910 con 0x7fa1e8104b30 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1de7fc640 1 -- 172.21.15.33:0/2167515784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80188e0 con 0x7fa1e8104b30 2024-04-24T23:15:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.495+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e819ffd0 con 0x7fa1e8104b30 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1de7fc640 1 -- 172.21.15.33:0/2167515784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa1d8018a80 con 0x7fa1e8104b30 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 Processor -- start 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 -- start start 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 0x562d2fe221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe22780 0x562d2fe27280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe29d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562d2fd852c0 con 0x562d2fe21d90 2024-04-24T23:15:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d2fd2d910 con 0x562d2fe22780 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.498+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d2fd81280 con 0x562d2fe278f0 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1edd54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 0x562d2fe221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1edd54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 0x562d2fe221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54856/0 (socket says 172.21.15.33:54856) 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1edd54640 1 -- 172.21.15.33:0/1660334010 learned_addr learned my addr 172.21.15.33:0/1660334010 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe22780 0x562d2fe27280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe29d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1de7fc640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1d40a2b90 0x7fa1d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ed553640 1 -- 172.21.15.33:0/1660334010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 msgr2=0x562d2fe29d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe29d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ed553640 1 -- 172.21.15.33:0/1660334010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 msgr2=0x562d2fe221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 0x562d2fe221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1ed553640 1 -- 172.21.15.33:0/1660334010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2fd3efb0 con 0x562d2fe22780 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1edd54640 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 0x562d2fe221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1edd54640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1d40a2b90 0x7fa1d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.499+0000 7fa1de7fc640 1 -- 172.21.15.33:0/2167515784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1d80cf6b0 con 0x7fa1e8104b30 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.500+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe22780 0x562d2fe27280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006010 tx=0x7fa1e4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fca249a412ca76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.500+0000 7fa1edd54640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1d40a2b90 0x7fa1d40a5050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562d2fe22fe0 tx=0x7fa1e0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.500+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1660334010 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4040070 con 0x562d2fe22780 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.500+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1660334010 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e4005500 con 0x562d2fe22780 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.500+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1660334010 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4035040 con 0x562d2fe22780 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1660334010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe22780 msgr2=0x562d2fe27280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe22780 0x562d2fe27280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006010 tx=0x7fa1e4004040 comp rx=0 tx=0).stop 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1660334010 shutdown_connections 2024-04-24T23:15:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe29d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe22780 0x562d2fe27280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1660334010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe21d90 0x562d2fe221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1660334010 >> 172.21.15.33:0/1660334010 conn(0x562d2fde9950 msgr2=0x562d2fe1d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1660334010 shutdown_connections 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1660334010 wait complete. 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- start start 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 Processor -- start 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe2c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.501+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562d2fe2a840 con 0x562d2fe22780 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d2fd40e90 con 0x562d2fe21d90 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1ecd52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1ecd52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe2c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54864/0 (socket says 172.21.15.33:54864) 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1ecd52640 1 -- 172.21.15.33:0/1764836848 learned_addr learned my addr 172.21.15.33:0/1764836848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d2fe2a9c0 con 0x562d2fe278f0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe2c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1f25a2740 10 client.0 _ll_register_callbacks cb 0x562d2fd7c970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 -- 172.21.15.33:0/1764836848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 msgr2=0x562d2fe2c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe2c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 -- 172.21.15.33:0/1764836848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 msgr2=0x562d2fe2c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe2c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 -- 172.21.15.33:0/1764836848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2fb617e0 con 0x562d2fe278f0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe2c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1edd54640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe336e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005fd0 tx=0x7fa1e0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5fdde10c40e778 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1764836848 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0004b20 con 0x562d2fe278f0 2024-04-24T23:15:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e4005500 con 0x562d2fe278f0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.502+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1764836848 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1e0004cc0 con 0x562d2fe278f0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.503+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1764836848 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1e0033050 con 0x562d2fe278f0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.503+0000 7fa1f25a2740 10 client.25840 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.503+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d2fbc1f70 con 0x562d2fe278f0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.503+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1764836848 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e4005500 con 0x562d2fe278f0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.503+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1764836848 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa1e0028400 con 0x562d2fe278f0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1f25a2740 10 client.25840 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1f25a2740 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1f25a2740 20 client.25840 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fe3c030 0x562d2fe3e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1f25a2740 4 client.25840 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d2fd595a0 con 0x562d2fe3c030 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fe3c030 0x562d2fe3e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1d0ff9640 10 client.25840 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.504+0000 7fa1ed553640 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fe3c030 0x562d2fe3e490 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a570ba7882167c5 server_cookie=86aee5a9ee16b6d6 in_seq=0 out_seq=0 2024-04-24T23:15:57.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1764836848 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d2fd595a0 con 0x562d2fe3c030 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1d0ff9640 10 client.25840 handle_command_reply: tid=0 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 client.25840 shutdown 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 msgr2=0x562d2fe336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe336e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005fd0 tx=0x7fa1e0004ab0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 shutdown_connections 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fe3c030 0x562d2fe3e490 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe336e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/1764836848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe2c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1d0ff9640 0 client.25840 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 >> 172.21.15.33:0/1764836848 conn(0x562d2fde9950 msgr2=0x562d2fe1c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 shutdown_connections 2024-04-24T23:15:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 1 -- 172.21.15.33:0/1764836848 wait complete. 2024-04-24T23:15:57.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.505+0000 7fa1f25a2740 20 client.25840 trim_cache size 0 max 16384 2024-04-24T23:15:57.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.525+0000 7fa1f25a2740 1 Processor -- start 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.525+0000 7fa1f25a2740 1 -- start start 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe21d90 0x562d2fe20e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe278f0 0x562d2fe42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81aaaf0 con 0x562d2fe22780 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81a83c0 con 0x562d2fe278f0 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81a6210 con 0x562d2fe21d90 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1ecd52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe278f0 0x562d2fe42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1ecd52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54870/0 (socket says 172.21.15.33:54870) 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1ecd52640 1 -- 172.21.15.33:0/3529381304 learned_addr learned my addr 172.21.15.33:0/3529381304 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1ed553640 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe21d90 0x562d2fe20e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 -- 172.21.15.33:0/3529381304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe21d90 msgr2=0x562d2fe20e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe21d90 0x562d2fe20e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 -- 172.21.15.33:0/3529381304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 msgr2=0x562d2fe336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 -- 172.21.15.33:0/3529381304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2fe24260 con 0x562d2fe278f0 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.526+0000 7fa1edd54640 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe278f0 0x562d2fe42d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00048a0 tx=0x7fa1e0003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f06111a52834bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3529381304 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0009500 con 0x562d2fe278f0 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3529381304 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0011cd0 con 0x562d2fe278f0 2024-04-24T23:15:57.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3529381304 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0005280 con 0x562d2fe278f0 2024-04-24T23:15:57.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3529381304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe278f0 msgr2=0x562d2fe42d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe278f0 0x562d2fe42d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00048a0 tx=0x7fa1e0003c20 comp rx=0 tx=0).stop 2024-04-24T23:15:57.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3529381304 shutdown_connections 2024-04-24T23:15:57.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe278f0 0x562d2fe42d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3529381304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe21d90 0x562d2fe20e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3529381304 >> 172.21.15.33:0/3529381304 conn(0x562d2fde9950 msgr2=0x562d2fe20720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3529381304 shutdown_connections 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.527+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3529381304 wait complete. 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 -- start start 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 Processor -- start 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe31810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe31d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81a6880 con 0x562d2fe22780 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81a7cf0 con 0x562d2fe21d90 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81a6f50 con 0x562d2fe278f0 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1f25a2740 10 client.0 _ll_register_callbacks cb 0x7fa1e0003ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1ed553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1ecd52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe31810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1ed553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe33200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37092/0 (socket says 172.21.15.33:37092) 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1ed553640 1 -- 172.21.15.33:0/3869751136 learned_addr learned my addr 172.21.15.33:0/3869751136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.528+0000 7fa1edd54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe31d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1edd54640 1 -- 172.21.15.33:0/3869751136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 msgr2=0x562d2fe33200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1edd54640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe33200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1edd54640 1 -- 172.21.15.33:0/3869751136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 msgr2=0x562d2fe31810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1edd54640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe31810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1edd54640 1 -- 172.21.15.33:0/3869751136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2fa59170 con 0x562d2fe278f0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1ed553640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe33200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1ecd52640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe31810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1edd54640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe31d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0000f80 tx=0x7fa1e001dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ce31c95115df5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/3869751136 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0028650 con 0x562d2fe278f0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/3869751136 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1e0028c70 con 0x562d2fe278f0 2024-04-24T23:15:57.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e0011cd0 con 0x562d2fe278f0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/3869751136 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1e0004ab0 con 0x562d2fe278f0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1f25a2740 10 client.25846 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d2fea3130 con 0x562d2fe278f0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.529+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/3869751136 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0011cd0 con 0x562d2fe278f0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/3869751136 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa1e00374c0 con 0x562d2fe278f0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1f25a2740 10 client.25846 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1f25a2740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1f25a2740 20 client.25846 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fea3a00 0x562d2fea5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1f25a2740 4 client.25846 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d2fe2bf50 con 0x562d2fea3a00 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1ed553640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fea3a00 0x562d2fea5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1d0ff9640 10 client.25846 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.530+0000 7fa1ed553640 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fea3a00 0x562d2fea5e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55064606c9d1c28 server_cookie=515c323db25d5854 in_seq=0 out_seq=0 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/3869751136 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d2fe2bf50 con 0x562d2fea3a00 2024-04-24T23:15:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1d0ff9640 10 client.25846 handle_command_reply: tid=0 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 client.25846 shutdown 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 msgr2=0x562d2fe31d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe31d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0000f80 tx=0x7fa1e001dc00 comp rx=0 tx=0).stop 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 shutdown_connections 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d2fea3a00 0x562d2fea5e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d2fe278f0 0x562d2fe31d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d2fe22780 0x562d2fe31810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 --2- 172.21.15.33:0/3869751136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d2fe21d90 0x562d2fe33200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1d0ff9640 0 client.25846 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.531+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 >> 172.21.15.33:0/3869751136 conn(0x562d2fde9950 msgr2=0x562d2fe1c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 shutdown_connections 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1f25a2740 1 -- 172.21.15.33:0/3869751136 wait complete. 2024-04-24T23:15:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1f25a2740 20 client.25846 trim_cache size 0 max 16384 2024-04-24T23:15:57.539 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:15:57.539 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:15:57.539 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "36b514a8-079b-4005-8c5d-ce3acb506e23", 2024-04-24T23:15:57.539 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:15:57.539 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1d40a2b90 msgr2=0x7fa1d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1d40a2b90 0x7fa1d40a5050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562d2fe22fe0 tx=0x7fa1e0004a40 comp rx=0 tx=0).stop 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8104b30 msgr2=0x7fa1e8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8104b30 0x7fa1e8198060 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9c0 tx=0x7fa1d800edc0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 shutdown_connections 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1d40a2b90 0x7fa1d40a5050 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8107460 0x7fa1e819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1e8104b30 0x7fa1e8198060 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 --2- 172.21.15.33:0/2167515784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1e8102200 0x7fa1e8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 >> 172.21.15.33:0/2167515784 conn(0x7fa1e80fbbf0 msgr2=0x7fa1e8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 shutdown_connections 2024-04-24T23:15:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.532+0000 7fa1ef7b7640 1 -- 172.21.15.33:0/2167515784 wait complete. 2024-04-24T23:15:57.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:57 smithi033 ceph-mon[29201]: pgmap v469: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 289 op/s 2024-04-24T23:15:57.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:57.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:57 smithi038 ceph-mon[32450]: pgmap v469: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 289 op/s 2024-04-24T23:15:57.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:57 smithi117 ceph-mon[32529]: pgmap v469: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 289 op/s 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 -- 172.21.15.33:0/1601906807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8100680 msgr2=0x7f39f8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 --2- 172.21.15.33:0/1601906807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8100680 0x7f39f8100aa0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f39e800d910 tx=0x7f39e802c380 comp rx=0 tx=0).stop 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 -- 172.21.15.33:0/1601906807 shutdown_connections 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 --2- 172.21.15.33:0/1601906807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8102230 0x7f39f8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 --2- 172.21.15.33:0/1601906807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8101870 0x7f39f8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 --2- 172.21.15.33:0/1601906807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8100680 0x7f39f8100aa0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 -- 172.21.15.33:0/1601906807 >> 172.21.15.33:0/1601906807 conn(0x7f39f80fbbf0 msgr2=0x7f39f80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 -- 172.21.15.33:0/1601906807 shutdown_connections 2024-04-24T23:15:57.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.683+0000 7f39fe161640 1 -- 172.21.15.33:0/1601906807 wait complete. 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 Processor -- start 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 -- start start 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8100680 0x7f39f819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8101870 0x7f39f819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8102230 0x7f39f81a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39f810b250 con 0x7f39f8101870 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39f810b0d0 con 0x7f39f8100680 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39fe161640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39f810b3d0 con 0x7f39f8102230 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8101870 0x7f39f819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8101870 0x7f39f819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54900/0 (socket says 172.21.15.33:54900) 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8102230 0x7f39f81a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f6ffd640 1 -- 172.21.15.33:0/3641213319 learned_addr learned my addr 172.21.15.33:0/3641213319 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f77fe640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8100680 0x7f39f819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f7fff640 1 -- 172.21.15.33:0/3641213319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8100680 msgr2=0x7f39f819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f7fff640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8100680 0x7f39f819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f7fff640 1 -- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8101870 msgr2=0x7f39f819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f7fff640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8101870 0x7f39f819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.684+0000 7f39f7fff640 1 -- 172.21.15.33:0/3641213319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f81a5d20 con 0x7f39f8102230 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.685+0000 7f39f7fff640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8102230 0x7f39f81a56b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009fd0 tx=0x7f39f000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.685+0000 7f39f4ff9640 1 -- 172.21.15.33:0/3641213319 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f000ce20 con 0x7f39f8102230 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.685+0000 7f39f4ff9640 1 -- 172.21.15.33:0/3641213319 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f0005020 con 0x7f39f8102230 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.685+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f81a5fb0 con 0x7f39f8102230 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.685+0000 7f39f4ff9640 1 -- 172.21.15.33:0/3641213319 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0010720 con 0x7f39f8102230 2024-04-24T23:15:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.685+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f81a6480 con 0x7f39f8102230 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.687+0000 7f39f4ff9640 1 -- 172.21.15.33:0/3641213319 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f39f0005540 con 0x7f39f8102230 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 Processor -- start 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 -- start start 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f4ff9640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39e00a2b90 0x7f39e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79cd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7c30 0x55dc79cdc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79cdf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc79c3b6b0 con 0x55dc79cdcda0 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dc79be3880 con 0x55dc79cd7240 2024-04-24T23:15:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc79c37670 con 0x55dc79cd7c30 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79cd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79cdf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f4ff9640 1 -- 172.21.15.33:0/3641213319 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f39f00cfae0 con 0x7f39f8102230 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39e00a2b90 0x7f39e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79cd7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37112/0 (socket says 172.21.15.33:37112) 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 -- 172.21.15.33:0/2814332013 learned_addr learned my addr 172.21.15.33:0/2814332013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7c30 0x55dc79cdc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 -- 172.21.15.33:0/2814332013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7c30 msgr2=0x55dc79cdc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7c30 0x55dc79cdc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 -- 172.21.15.33:0/2814332013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 msgr2=0x55dc79cdf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79cdf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f77fe640 1 -- 172.21.15.33:0/2814332013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc79a197d0 con 0x55dc79cd7240 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.688+0000 7f39f7fff640 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79cdf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.689+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39e00a2b90 0x7f39e00a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dc79cd8030 tx=0x7f39ec009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.689+0000 7f39f77fe640 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79cd7660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f39e80049e0 tx=0x7f39e8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c23bcae0be34b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.689+0000 7f39ddffb640 1 -- 172.21.15.33:0/2814332013 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8042020 con 0x55dc79cd7240 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.689+0000 7f39ddffb640 1 -- 172.21.15.33:0/2814332013 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8030070 con 0x55dc79cd7240 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.689+0000 7f39ddffb640 1 -- 172.21.15.33:0/2814332013 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e803ee60 con 0x55dc79cd7240 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2814332013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 msgr2=0x55dc79cd7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79cd7660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f39e80049e0 tx=0x7f39e8005d50 comp rx=0 tx=0).stop 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2814332013 shutdown_connections 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79cdf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7c30 0x55dc79cdc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2814332013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79cd7660 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2814332013 >> 172.21.15.33:0/2814332013 conn(0x55dc79cd28b0 msgr2=0x55dc79cd2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2814332013 shutdown_connections 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2814332013 wait complete. 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- start start 2024-04-24T23:15:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 Processor -- start 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79ce15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7c30 0x55dc79ce1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc79cdfcf0 con 0x55dc79cd7c30 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dc79bf6f50 con 0x55dc79cd7240 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.690+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc79cdfe70 con 0x55dc79cdcda0 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f3a00e6b740 10 client.0 _ll_register_callbacks cb 0x55dc79c32000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7c30 0x55dc79ce1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79ce15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79ce15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37114/0 (socket says 172.21.15.33:37114) 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7c30 0x55dc79ce1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54932/0 (socket says 172.21.15.33:54932) 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 -- 172.21.15.33:0/1578680841 learned_addr learned my addr 172.21.15.33:0/1578680841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f7fff640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 -- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 msgr2=0x55dc79ce8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 -- 172.21.15.33:0/1578680841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7c30 msgr2=0x55dc79ce1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7c30 0x55dc79ce1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 -- 172.21.15.33:0/1578680841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7994f3f0 con 0x55dc79cd7240 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f7fff640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39f77fe640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79ce15d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39e800d910 tx=0x7f39e800d190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb636ebf6c6dc571 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:15:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f39d6ffd640 1 -- 172.21.15.33:0/1578680841 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8042030 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.691+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e8030070 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f39d6ffd640 1 -- 172.21.15.33:0/1578680841 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f39e803e660 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f39d6ffd640 1 -- 172.21.15.33:0/1578680841 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39e803e860 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 10 client.25835 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc7990f0f0 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f39d6ffd640 1 -- 172.21.15.33:0/1578680841 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8030070 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f39d6ffd640 1 -- 172.21.15.33:0/1578680841 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f39e8003ca0 con 0x55dc79cd7240 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 10 client.25835 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 10 client.25835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 20 client.25835 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79ced040 0x55dc79cef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 4 client.25835 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.692+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc79c48bf0 con 0x55dc79ced040 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.693+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79ced040 0x55dc79cef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.693+0000 7f39d6ffd640 10 client.25835 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.693+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79ced040 0x55dc79cef430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f0c9357c92d9c5 server_cookie=5baf95455a255cf in_seq=0 out_seq=0 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f39d6ffd640 1 -- 172.21.15.33:0/1578680841 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc79c48bf0 con 0x55dc79ced040 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f39d6ffd640 10 client.25835 handle_command_reply: tid=0 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 client.25835 shutdown 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 msgr2=0x55dc79ce15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79ce15d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39e800d910 tx=0x7f39e800d190 comp rx=0 tx=0).stop 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 shutdown_connections 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79ced040 0x55dc79cef430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7c30 0x55dc79ce1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f39d6ffd640 0 client.25835 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/1578680841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7240 0x55dc79ce15d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 >> 172.21.15.33:0/1578680841 conn(0x55dc79cd28b0 msgr2=0x55dc79cd1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 shutdown_connections 2024-04-24T23:15:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 1 -- 172.21.15.33:0/1578680841 wait complete. 2024-04-24T23:15:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.694+0000 7f3a00e6b740 20 client.25835 trim_cache size 0 max 16384 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 Processor -- start 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 -- start start 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 0x55dc79ce0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79ce0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81b3270 con 0x55dc79cdcda0 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81b0b40 con 0x55dc79cd7c30 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.713+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81ae990 con 0x55dc79cd7240 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 0x55dc79ce0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 0x55dc79ce0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:43540/0 (socket says 172.21.15.33:43540) 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 -- 172.21.15.33:0/2898148361 learned_addr learned my addr 172.21.15.33:0/2898148361 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f7fff640 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79ce0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 -- 172.21.15.33:0/2898148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 msgr2=0x55dc79ce8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 -- 172.21.15.33:0/2898148361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 msgr2=0x55dc79ce0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79ce0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 -- 172.21.15.33:0/2898148361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc79cd23f0 con 0x55dc79cd7240 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39f77fe640 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 0x55dc79ce0210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39e802c8a0 tx=0x7f39e8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2aa71c83da9f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39dd7fa640 1 -- 172.21.15.33:0/2898148361 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8042030 con 0x55dc79cd7240 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39dd7fa640 1 -- 172.21.15.33:0/2898148361 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8030040 con 0x55dc79cd7240 2024-04-24T23:15:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.714+0000 7f39dd7fa640 1 -- 172.21.15.33:0/2898148361 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8051780 con 0x55dc79cd7240 2024-04-24T23:15:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2898148361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 msgr2=0x55dc79ce0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 0x55dc79ce0210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39e802c8a0 tx=0x7f39e8005d50 comp rx=0 tx=0).stop 2024-04-24T23:15:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2898148361 shutdown_connections 2024-04-24T23:15:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cdcda0 0x55dc79ce0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/2898148361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cd7240 0x55dc79ce0210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2898148361 >> 172.21.15.33:0/2898148361 conn(0x55dc79cd28b0 msgr2=0x55dc79cd1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2898148361 shutdown_connections 2024-04-24T23:15:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.715+0000 7f3a00e6b740 1 -- 172.21.15.33:0/2898148361 wait complete. 2024-04-24T23:15:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 -- start start 2024-04-24T23:15:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 Processor -- start 2024-04-24T23:15:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 0x55dc79ce6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81af000 con 0x55dc79cd7240 2024-04-24T23:15:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81b0470 con 0x55dc79cd7c30 2024-04-24T23:15:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81af6d0 con 0x55dc79cdcda0 2024-04-24T23:15:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f3a00e6b740 10 client.0 _ll_register_callbacks cb 0x7f39f0005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 0x55dc79ce6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f39f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 0x55dc79ce6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54948/0 (socket says 172.21.15.33:54948) 2024-04-24T23:15:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f39f77fe640 1 -- 172.21.15.33:0/367660910 learned_addr learned my addr 172.21.15.33:0/367660910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.716+0000 7f39f7fff640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39f77fe640 1 -- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 msgr2=0x55dc79ce7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39f77fe640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39f77fe640 1 -- 172.21.15.33:0/367660910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 msgr2=0x55dc79ce6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39f77fe640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39f77fe640 1 -- 172.21.15.33:0/367660910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc79cd2200 con 0x55dc79cd7240 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39f77fe640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 0x55dc79ce6130 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f39e802c3c0 tx=0x7f39e803e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a9afc890031c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39d6ffd640 1 -- 172.21.15.33:0/367660910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8042050 con 0x55dc79cd7240 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39d6ffd640 1 -- 172.21.15.33:0/367660910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f39e8034af0 con 0x55dc79cd7240 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e8030040 con 0x55dc79cd7240 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39d6ffd640 1 -- 172.21.15.33:0/367660910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39e806f050 con 0x55dc79cd7240 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f3a00e6b740 10 client.16305 fetch_fsmap learned FSMap version 26 2024-04-24T23:15:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc79ce8d60 con 0x55dc79cd7240 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.717+0000 7f39d6ffd640 1 -- 172.21.15.33:0/367660910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8030040 con 0x55dc79cd7240 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f39d6ffd640 1 -- 172.21.15.33:0/367660910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f39e8047420 con 0x55dc79cd7240 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 10 client.16305 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 20 client.16305 populate_metadata read hostname 'smithi033' 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79d53b50 0x55dc79d55f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 4 client.16305 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc79c42f40 con 0x55dc79d53b50 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79d53b50 0x55dc79d55f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f39d6ffd640 10 client.16305 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f39f6ffd640 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79d53b50 0x55dc79d55f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a7e0d28c9578cb server_cookie=bf6812a295a19da5 in_seq=0 out_seq=0 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f39d6ffd640 1 -- 172.21.15.33:0/367660910 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc79c42f40 con 0x55dc79d53b50 2024-04-24T23:15:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f39d6ffd640 10 client.16305 handle_command_reply: tid=0 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.718+0000 7f3a00e6b740 1 client.16305 shutdown 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 msgr2=0x55dc79ce6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 0x55dc79ce6130 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f39e802c3c0 tx=0x7f39e803e710 comp rx=0 tx=0).stop 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 shutdown_connections 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55dc79d53b50 0x55dc79d55f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc79cdcda0 0x55dc79ce7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc79cd7c30 0x55dc79ce6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 --2- 172.21.15.33:0/367660910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc79cd7240 0x55dc79ce6130 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 >> 172.21.15.33:0/367660910 conn(0x55dc79cd28b0 msgr2=0x55dc79cd5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f39d6ffd640 0 client.16305 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 shutdown_connections 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 1 -- 172.21.15.33:0/367660910 wait complete. 2024-04-24T23:15:57.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.719+0000 7f3a00e6b740 20 client.16305 trim_cache size 0 max 16384 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: "36b514a8-079b-4005-8c5d-ce3acb506e23": { 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "36b514a8-079b-4005-8c5d-ce3acb506e23", 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:15:57.735 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:15:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39e00a2b90 msgr2=0x7f39e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39e00a2b90 0x7f39e00a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dc79cd8030 tx=0x7f39ec009530 comp rx=0 tx=0).stop 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8102230 msgr2=0x7f39f81a56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8102230 0x7f39f81a56b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009fd0 tx=0x7f39f000ef10 comp rx=0 tx=0).stop 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 shutdown_connections 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f39e00a2b90 0x7f39e00a5050 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f8102230 0x7f39f81a56b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f8101870 0x7f39f819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 --2- 172.21.15.33:0/3641213319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f8100680 0x7f39f819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 >> 172.21.15.33:0/3641213319 conn(0x7f39f80fbbf0 msgr2=0x7f39f8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 shutdown_connections 2024-04-24T23:15:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:15:57.720+0000 7f39fe161640 1 -- 172.21.15.33:0/3641213319 wait complete. 2024-04-24T23:15:57.737 INFO:tasks.cephfs.filesystem:scrub status for tag:36b514a8-079b-4005-8c5d-ce3acb506e23 - {'path': '/', 'tag': '36b514a8-079b-4005-8c5d-ce3acb506e23', 'options': 'recursive,force'} 2024-04-24T23:15:58.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 305288 12.75 MB/sec execute 3 sec latency 13.096 ms 2024-04-24T23:15:58.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:15:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:15:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:15:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:15:58.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:58.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:58.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:15:58.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:58.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:15:58.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:15:59.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 306596 10.92 MB/sec execute 4 sec latency 61.543 ms 2024-04-24T23:15:59.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:15:59 smithi038 ceph-mon[32450]: pgmap v470: 129 pgs: 129 active+clean; 128 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 238 op/s 2024-04-24T23:15:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:15:59 smithi117 ceph-mon[32529]: pgmap v470: 129 pgs: 129 active+clean; 128 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 238 op/s 2024-04-24T23:15:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:15:59 smithi033 ceph-mon[29201]: pgmap v470: 129 pgs: 129 active+clean; 128 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 238 op/s 2024-04-24T23:16:00.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 309182 11.54 MB/sec execute 5 sec latency 14.787 ms 2024-04-24T23:16:01.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 312139 12.52 MB/sec execute 6 sec latency 25.342 ms 2024-04-24T23:16:01.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:01 smithi038 ceph-mon[32450]: pgmap v471: 129 pgs: 129 active+clean; 128 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 224 op/s 2024-04-24T23:16:01.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:01 smithi117 ceph-mon[32529]: pgmap v471: 129 pgs: 129 active+clean; 128 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 224 op/s 2024-04-24T23:16:01.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:01 smithi033 ceph-mon[29201]: pgmap v471: 129 pgs: 129 active+clean; 128 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 224 op/s 2024-04-24T23:16:02.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 314452 11.94 MB/sec execute 7 sec latency 24.138 ms 2024-04-24T23:16:03.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 316449 12.03 MB/sec execute 8 sec latency 24.214 ms 2024-04-24T23:16:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:03 smithi038 ceph-mon[32450]: pgmap v472: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2024-04-24T23:16:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:03 smithi117 ceph-mon[32529]: pgmap v472: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2024-04-24T23:16:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:03 smithi033 ceph-mon[29201]: pgmap v472: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2024-04-24T23:16:04.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 318877 12.12 MB/sec execute 9 sec latency 116.801 ms 2024-04-24T23:16:05.440 INFO:tasks.workunit.client.0.smithi033.stdout: 1 320886 12.02 MB/sec execute 10 sec latency 14.996 ms 2024-04-24T23:16:05.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:05 smithi038 ceph-mon[32450]: pgmap v473: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-04-24T23:16:05.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:05 smithi117 ceph-mon[32529]: pgmap v473: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-04-24T23:16:05.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:05 smithi033 ceph-mon[29201]: pgmap v473: 129 pgs: 129 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-04-24T23:16:06.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 323461 12.20 MB/sec execute 11 sec latency 12.218 ms 2024-04-24T23:16:07.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 326421 12.65 MB/sec execute 12 sec latency 12.925 ms 2024-04-24T23:16:07.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:07 smithi038 ceph-mon[32450]: pgmap v474: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-04-24T23:16:07.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:07 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:16:07.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:07 smithi117 ceph-mon[32529]: pgmap v474: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-04-24T23:16:07.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:07 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:16:07.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:07 smithi033 ceph-mon[29201]: pgmap v474: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-04-24T23:16:07.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:07 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:16:08.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 328706 12.32 MB/sec execute 13 sec latency 24.923 ms 2024-04-24T23:16:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:08 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:16:08.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:08 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:16:08.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:08 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:16:09.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 330410 12.31 MB/sec execute 14 sec latency 43.322 ms 2024-04-24T23:16:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:09 smithi038 ceph-mon[32450]: pgmap v475: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-04-24T23:16:09.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:09 smithi117 ceph-mon[32529]: pgmap v475: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-04-24T23:16:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:09 smithi033 ceph-mon[29201]: pgmap v475: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-04-24T23:16:10.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 333107 12.38 MB/sec execute 15 sec latency 12.872 ms 2024-04-24T23:16:11.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 335123 12.30 MB/sec execute 16 sec latency 13.499 ms 2024-04-24T23:16:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:11 smithi038 ceph-mon[32450]: pgmap v476: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 219 op/s 2024-04-24T23:16:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:11 smithi117 ceph-mon[32529]: pgmap v476: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 219 op/s 2024-04-24T23:16:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:11 smithi033 ceph-mon[29201]: pgmap v476: 129 pgs: 129 active+clean; 135 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 219 op/s 2024-04-24T23:16:12.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 337643 12.40 MB/sec execute 17 sec latency 37.168 ms 2024-04-24T23:16:13.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 340608 12.68 MB/sec execute 18 sec latency 13.388 ms 2024-04-24T23:16:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:13 smithi038 ceph-mon[32450]: pgmap v477: 129 pgs: 129 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-04-24T23:16:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:13 smithi117 ceph-mon[32529]: pgmap v477: 129 pgs: 129 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-04-24T23:16:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:13 smithi033 ceph-mon[29201]: pgmap v477: 129 pgs: 129 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-04-24T23:16:14.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 341756 12.20 MB/sec execute 19 sec latency 57.251 ms 2024-04-24T23:16:15.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 344542 12.44 MB/sec execute 20 sec latency 10.276 ms 2024-04-24T23:16:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:15 smithi038 ceph-mon[32450]: pgmap v478: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-24T23:16:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:15 smithi117 ceph-mon[32529]: pgmap v478: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-24T23:16:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:15 smithi033 ceph-mon[29201]: pgmap v478: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-24T23:16:16.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 347435 12.54 MB/sec execute 21 sec latency 12.593 ms 2024-04-24T23:16:17.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 349278 12.42 MB/sec execute 22 sec latency 15.817 ms 2024-04-24T23:16:17.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:17 smithi038 ceph-mon[32450]: pgmap v479: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-04-24T23:16:17.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:17 smithi117 ceph-mon[32529]: pgmap v479: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-04-24T23:16:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:17 smithi033 ceph-mon[29201]: pgmap v479: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-04-24T23:16:18.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 351843 12.49 MB/sec execute 23 sec latency 19.030 ms 2024-04-24T23:16:19.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 354430 12.56 MB/sec execute 24 sec latency 55.555 ms 2024-04-24T23:16:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:19 smithi038 ceph-mon[32450]: pgmap v480: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-04-24T23:16:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:19 smithi117 ceph-mon[32529]: pgmap v480: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-04-24T23:16:19.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:19 smithi033 ceph-mon[29201]: pgmap v480: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-04-24T23:16:20.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 356395 12.46 MB/sec execute 25 sec latency 20.042 ms 2024-04-24T23:16:21.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 358971 12.52 MB/sec execute 26 sec latency 13.413 ms 2024-04-24T23:16:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:21 smithi038 ceph-mon[32450]: pgmap v481: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-04-24T23:16:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:21 smithi117 ceph-mon[32529]: pgmap v481: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-04-24T23:16:21.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:21 smithi033 ceph-mon[29201]: pgmap v481: 129 pgs: 129 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-04-24T23:16:22.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 361933 12.71 MB/sec execute 27 sec latency 45.302 ms 2024-04-24T23:16:23.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 364393 12.59 MB/sec execute 28 sec latency 19.127 ms 2024-04-24T23:16:23.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:23 smithi038 ceph-mon[32450]: pgmap v482: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-04-24T23:16:23.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:23 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:16:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:23 smithi117 ceph-mon[32529]: pgmap v482: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-04-24T23:16:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:23 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:16:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:23 smithi033 ceph-mon[29201]: pgmap v482: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-04-24T23:16:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:23 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:16:24.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 366077 12.55 MB/sec execute 29 sec latency 81.563 ms 2024-04-24T23:16:25.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 368992 12.71 MB/sec execute 30 sec latency 21.076 ms 2024-04-24T23:16:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:25 smithi038 ceph-mon[32450]: pgmap v483: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-04-24T23:16:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:25 smithi117 ceph-mon[32529]: pgmap v483: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-04-24T23:16:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:25 smithi033 ceph-mon[29201]: pgmap v483: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 269 op/s 2024-04-24T23:16:26.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 371262 12.57 MB/sec execute 31 sec latency 28.542 ms 2024-04-24T23:16:27.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 373315 12.58 MB/sec execute 32 sec latency 20.675 ms 2024-04-24T23:16:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:27 smithi038 ceph-mon[32450]: pgmap v484: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 288 op/s 2024-04-24T23:16:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:27 smithi117 ceph-mon[32529]: pgmap v484: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 288 op/s 2024-04-24T23:16:27.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:27 smithi033 ceph-mon[29201]: pgmap v484: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 288 op/s 2024-04-24T23:16:27.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:16:27.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:16:27.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 -- 172.21.15.33:0/1677381485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 msgr2=0x7fb93c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1677381485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c101cf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009900 tx=0x7fb92c02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:27.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 -- 172.21.15.33:0/1677381485 shutdown_connections 2024-04-24T23:16:27.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1677381485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb93c102230 0x7fb93c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1677381485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c101cf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1677381485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c100680 0x7fb93c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 -- 172.21.15.33:0/1677381485 >> 172.21.15.33:0/1677381485 conn(0x7fb93c0fbbf0 msgr2=0x7fb93c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 -- 172.21.15.33:0/1677381485 shutdown_connections 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.849+0000 7fb9437c8640 1 -- 172.21.15.33:0/1677381485 wait complete. 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 Processor -- start 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 -- start start 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb93c100680 0x7fb93c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c102230 0x7fb93c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c10c300 con 0x7fb93c101870 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c10c180 con 0x7fb93c102230 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb9437c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c10c480 con 0x7fb93c100680 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56212/0 (socket says 172.21.15.33:56212) 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 -- 172.21.15.33:0/1946044641 learned_addr learned my addr 172.21.15.33:0/1946044641 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb941d65640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c102230 0x7fb93c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb941564640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb93c100680 0x7fb93c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 -- 172.21.15.33:0/1946044641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb93c100680 msgr2=0x7fb93c069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb93c100680 0x7fb93c069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 -- 172.21.15.33:0/1946044641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c102230 msgr2=0x7fb93c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c102230 0x7fb93c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.850+0000 7fb940d63640 1 -- 172.21.15.33:0/1946044641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c19f710 con 0x7fb93c101870 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.851+0000 7fb940d63640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c19c810 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009230 tx=0x7fb92c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.851+0000 7fb9327fc640 1 -- 172.21.15.33:0/1946044641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03f030 con 0x7fb93c101870 2024-04-24T23:16:27.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.851+0000 7fb9327fc640 1 -- 172.21.15.33:0/1946044641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c003ce0 con 0x7fb93c101870 2024-04-24T23:16:27.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.851+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c19f9a0 con 0x7fb93c101870 2024-04-24T23:16:27.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.851+0000 7fb9327fc640 1 -- 172.21.15.33:0/1946044641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c035070 con 0x7fb93c101870 2024-04-24T23:16:27.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.851+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c19fe90 con 0x7fb93c101870 2024-04-24T23:16:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.853+0000 7fb9327fc640 1 -- 172.21.15.33:0/1946044641 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb92c042030 con 0x7fb93c101870 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.853+0000 7fb9465b4740 1 Processor -- start 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.853+0000 7fb9465b4740 1 -- start start 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x55625007da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x556250082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5562500831b0 0x556250085640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9327fc640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9280a2b90 0x7fb9280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9465b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55624ffdd6b0 con 0x55625007e040 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9465b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55624ff85880 con 0x5562500831b0 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9465b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55624ffd9670 con 0x55625007d650 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x556250082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x556250082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56216/0 (socket says 172.21.15.33:56216) 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 -- 172.21.15.33:0/4166173758 learned_addr learned my addr 172.21.15.33:0/4166173758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb940d63640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9280a2b90 0x7fb9280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941d65640 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5562500831b0 0x556250085640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb9327fc640 1 -- 172.21.15.33:0/1946044641 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb92c0ef840 con 0x7fb93c101870 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941d65640 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x55625007da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 -- 172.21.15.33:0/4166173758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 msgr2=0x55625007da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x55625007da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 -- 172.21.15.33:0/4166173758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5562500831b0 msgr2=0x556250085640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5562500831b0 0x556250085640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 -- 172.21.15.33:0/4166173758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624fdbb7d0 con 0x55625007e040 2024-04-24T23:16:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb940d63640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9280a2b90 0x7fb9280a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0bbac0 tx=0x7fb92c049bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.854+0000 7fb941564640 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x556250082b40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb938009ca0 tx=0x7fb938011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b4650818165f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9277fe640 1 -- 172.21.15.33:0/4166173758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938009e50 con 0x55625007e040 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9277fe640 1 -- 172.21.15.33:0/4166173758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9380035f0 con 0x55625007e040 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9277fe640 1 -- 172.21.15.33:0/4166173758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938019650 con 0x55625007e040 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 -- 172.21.15.33:0/4166173758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 msgr2=0x556250082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x556250082b40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb938009ca0 tx=0x7fb938011810 comp rx=0 tx=0).stop 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 -- 172.21.15.33:0/4166173758 shutdown_connections 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5562500831b0 0x556250085640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x556250082b40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4166173758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x55625007da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 -- 172.21.15.33:0/4166173758 >> 172.21.15.33:0/4166173758 conn(0x5562500748b0 msgr2=0x556250074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 -- 172.21.15.33:0/4166173758 shutdown_connections 2024-04-24T23:16:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.855+0000 7fb9465b4740 1 -- 172.21.15.33:0/4166173758 wait complete. 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 -- start start 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 Processor -- start 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x556250087a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007e040 0x556250087fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x55625008f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556250086160 con 0x5562500831b0 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55624ff98f50 con 0x55625007e040 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5562500862e0 con 0x55625007d650 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb9465b4740 10 client.0 _ll_register_callbacks cb 0x55624ffd4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb941d65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x55625008f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb941564640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007e040 0x556250087fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.856+0000 7fb941d65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x55625008f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56228/0 (socket says 172.21.15.33:56228) 2024-04-24T23:16:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 -- 172.21.15.33:0/3675764226 learned_addr learned my addr 172.21.15.33:0/3675764226 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x556250087a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 -- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 msgr2=0x556250087a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x556250087a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 -- 172.21.15.33:0/3675764226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007e040 msgr2=0x556250087fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007e040 0x556250087fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 -- 172.21.15.33:0/3675764226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624fcf13f0 con 0x5562500831b0 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941564640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007e040 0x556250087fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb941d65640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x55625008f000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb934011f50 tx=0x7fb934009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e36fa7a1e25990 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb924ff9640 1 -- 172.21.15.33:0/3675764226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934004b20 con 0x5562500831b0 2024-04-24T23:16:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb924ff9640 1 -- 172.21.15.33:0/3675764226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb934004cc0 con 0x5562500831b0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9380035f0 con 0x5562500831b0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb924ff9640 1 -- 172.21.15.33:0/3675764226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb93402b050 con 0x5562500831b0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb9465b4740 10 client.16317 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.857+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55624fcb10f0 con 0x5562500831b0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb924ff9640 1 -- 172.21.15.33:0/3675764226 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9380035f0 con 0x5562500831b0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb924ff9640 1 -- 172.21.15.33:0/3675764226 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb93400e3c0 con 0x5562500831b0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb9465b4740 10 client.16317 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb9465b4740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb9465b4740 20 client.16317 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x556250095da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb9465b4740 4 client.16317 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55624ffeabf0 con 0x5562500938e0 2024-04-24T23:16:27.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb941564640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x556250095da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb924ff9640 10 client.16317 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.858+0000 7fb941564640 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x556250095da0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522ac947a55f200c server_cookie=b1c1bacb0498a334 in_seq=0 out_seq=0 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb924ff9640 1 -- 172.21.15.33:0/3675764226 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55624ffeabf0 con 0x5562500938e0 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb924ff9640 10 client.16317 handle_command_reply: tid=0 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 client.16317 shutdown 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 msgr2=0x55625008f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x55625008f000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb934011f50 tx=0x7fb934009c30 comp rx=0 tx=0).stop 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 shutdown_connections 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x556250095da0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x55625008f000 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb924ff9640 0 client.16317 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007e040 0x556250087fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3675764226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007d650 0x556250087a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 >> 172.21.15.33:0/3675764226 conn(0x5562500748b0 msgr2=0x556250073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 shutdown_connections 2024-04-24T23:16:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.859+0000 7fb9465b4740 1 -- 172.21.15.33:0/3675764226 wait complete. 2024-04-24T23:16:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.860+0000 7fb9465b4740 20 client.16317 trim_cache size 0 max 16384 2024-04-24T23:16:27.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 Processor -- start 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 -- start start 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x556250077860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 0x55625008efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x556250077da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1aab20 con 0x5562500831b0 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1a83f0 con 0x55625007d650 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.880+0000 7fb9465b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1a6240 con 0x55625007e040 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 0x55625008efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941d65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x556250077da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 0x55625008efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:50892/0 (socket says 172.21.15.33:50892) 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 -- 172.21.15.33:0/4152012507 learned_addr learned my addr 172.21.15.33:0/4152012507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941d65640 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x556250077860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 -- 172.21.15.33:0/4152012507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 msgr2=0x556250077860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x556250077860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 -- 172.21.15.33:0/4152012507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 msgr2=0x556250077da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x556250077da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 -- 172.21.15.33:0/4152012507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556250066090 con 0x55625007e040 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb941564640 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 0x55625008efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb938009230 tx=0x7fb938005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348c9ae5a3e8f00a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb926ffd640 1 -- 172.21.15.33:0/4152012507 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93802b070 con 0x55625007e040 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb926ffd640 1 -- 172.21.15.33:0/4152012507 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb938005590 con 0x55625007e040 2024-04-24T23:16:27.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.881+0000 7fb926ffd640 1 -- 172.21.15.33:0/4152012507 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938025cb0 con 0x55625007e040 2024-04-24T23:16:27.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 -- 172.21.15.33:0/4152012507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 msgr2=0x55625008efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 0x55625008efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb938009230 tx=0x7fb938005850 comp rx=0 tx=0).stop 2024-04-24T23:16:27.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 -- 172.21.15.33:0/4152012507 shutdown_connections 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562500831b0 0x556250077da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55625007e040 0x55625008efc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 --2- 172.21.15.33:0/4152012507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x556250077860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 -- 172.21.15.33:0/4152012507 >> 172.21.15.33:0/4152012507 conn(0x5562500748b0 msgr2=0x556250074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 -- 172.21.15.33:0/4152012507 shutdown_connections 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 -- 172.21.15.33:0/4152012507 wait complete. 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.882+0000 7fb9465b4740 1 -- start start 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 Processor -- start 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x55625008c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x55625008cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5562500831b0 0x55625008db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1a68b0 con 0x55625007e040 2024-04-24T23:16:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1a7d20 con 0x55625007d650 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1a6f80 con 0x5562500831b0 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb9465b4740 10 client.0 _ll_register_callbacks cb 0x7fb9340115c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x55625008cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x55625008cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56252/0 (socket says 172.21.15.33:56252) 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941d65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x55625008c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 -- 172.21.15.33:0/3957225434 learned_addr learned my addr 172.21.15.33:0/3957225434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941d65640 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5562500831b0 0x55625008db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 -- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5562500831b0 msgr2=0x55625008db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5562500831b0 0x55625008db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 -- 172.21.15.33:0/3957225434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 msgr2=0x55625008c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x55625008c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.883+0000 7fb941564640 1 -- 172.21.15.33:0/3957225434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55625008e160 con 0x55625007e040 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb941564640 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x55625008cb60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb938009ef0 tx=0x7fb938005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d0caebfa3228c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb924ff9640 1 -- 172.21.15.33:0/3957225434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93802b040 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb924ff9640 1 -- 172.21.15.33:0/3957225434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb938016040 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb938005590 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb924ff9640 1 -- 172.21.15.33:0/3957225434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb938040050 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 10 client.16323 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562500f9ee0 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb924ff9640 1 -- 172.21.15.33:0/3957225434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb938005590 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb924ff9640 1 -- 172.21.15.33:0/3957225434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb938033420 con 0x55625007e040 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 10 client.16323 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 20 client.16323 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x5562500fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 4 client.16323 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:16:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.884+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55624ffe4f40 con 0x5562500938e0 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb941d65640 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x5562500fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb924ff9640 10 client.16323 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb941d65640 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x5562500fc6a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb4ff837f138e41 server_cookie=c703cc8f78c9ec9c in_seq=0 out_seq=0 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb924ff9640 1 -- 172.21.15.33:0/3957225434 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55624ffe4f40 con 0x5562500938e0 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb924ff9640 10 client.16323 handle_command_reply: tid=0 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 client.16323 shutdown 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 msgr2=0x55625008cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x55625008cb60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb938009ef0 tx=0x7fb938005dc0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 shutdown_connections 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5562500938e0 0x5562500fc6a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5562500831b0 0x55625008db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55625007e040 0x55625008cb60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb924ff9640 0 client.16323 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 --2- 172.21.15.33:0/3957225434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55625007d650 0x55625008c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 >> 172.21.15.33:0/3957225434 conn(0x5562500748b0 msgr2=0x556250073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 shutdown_connections 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.885+0000 7fb9465b4740 1 -- 172.21.15.33:0/3957225434 wait complete. 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9465b4740 20 client.16323 trim_cache size 0 max 16384 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:16:27.894 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9280a2b90 msgr2=0x7fb9280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9280a2b90 0x7fb9280a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0bbac0 tx=0x7fb92c049bd0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 msgr2=0x7fb93c19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c19c810 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009230 tx=0x7fb92c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 shutdown_connections 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9280a2b90 0x7fb9280a5050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c102230 0x7fb93c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c101870 0x7fb93c19c810 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 --2- 172.21.15.33:0/1946044641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb93c100680 0x7fb93c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 >> 172.21.15.33:0/1946044641 conn(0x7fb93c0fbbf0 msgr2=0x7fb93c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 shutdown_connections 2024-04-24T23:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:27.886+0000 7fb9437c8640 1 -- 172.21.15.33:0/1946044641 wait complete. 2024-04-24T23:16:27.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:16:27.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:16:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 -- 172.21.15.33:0/4274319354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2981004c0 msgr2=0x7fd2981008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/4274319354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2981004c0 0x7fd2981008e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 -- 172.21.15.33:0/4274319354 shutdown_connections 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/4274319354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd298102070 0x7fd298108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/4274319354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2981016b0 0x7fd298101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/4274319354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2981004c0 0x7fd2981008e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 -- 172.21.15.33:0/4274319354 >> 172.21.15.33:0/4274319354 conn(0x7fd2980fba10 msgr2=0x7fd2980fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 -- 172.21.15.33:0/4274319354 shutdown_connections 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.003+0000 7fd29dd9e640 1 -- 172.21.15.33:0/4274319354 wait complete. 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 Processor -- start 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 -- start start 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2981004c0 0x7fd298104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2981016b0 0x7fd298102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 0x7fd298102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd29810c120 con 0x7fd298102070 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd29810bfa0 con 0x7fd2981016b0 2024-04-24T23:16:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29dd9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd29810c2a0 con 0x7fd2981004c0 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 0x7fd298102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 0x7fd298102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56260/0 (socket says 172.21.15.33:56260) 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 -- 172.21.15.33:0/3481532704 learned_addr learned my addr 172.21.15.33:0/3481532704 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd297fff640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2981016b0 0x7fd298102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29cd9c640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2981004c0 0x7fd298104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 -- 172.21.15.33:0/3481532704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2981004c0 msgr2=0x7fd298104450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2981004c0 0x7fd298104450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 -- 172.21.15.33:0/3481532704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2981016b0 msgr2=0x7fd298102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2981016b0 0x7fd298102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.004+0000 7fd29d59d640 1 -- 172.21.15.33:0/3481532704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd298103860 con 0x7fd298102070 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.005+0000 7fd29d59d640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 0x7fd298102fd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd29000eaa0 tx=0x7fd29000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.005+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29000cdb0 con 0x7fd298102070 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.005+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2981a1a30 con 0x7fd298102070 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.005+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290005020 con 0x7fd298102070 2024-04-24T23:16:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.005+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900116b0 con 0x7fd298102070 2024-04-24T23:16:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.005+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981a1f30 con 0x7fd298102070 2024-04-24T23:16:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.006+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2981a21c0 con 0x7fd298102070 2024-04-24T23:16:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.007+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd290005560 con 0x7fd298102070 2024-04-24T23:16:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.008+0000 7fd295ffb640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2880a2b90 0x7fd2880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.008+0000 7fd29cd9c640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2880a2b90 0x7fd2880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.008+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd2900cf610 con 0x7fd298102070 2024-04-24T23:16:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.008+0000 7fd29cd9c640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2880a2b90 0x7fd2880a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005ab0 tx=0x7fd28c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.012+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd290016030 con 0x7fd298102070 2024-04-24T23:16:28.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.243+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2981a2390 con 0x7fd298102070 2024-04-24T23:16:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.244+0000 7fd295ffb640 1 -- 172.21.15.33:0/3481532704 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fd29009bb90 con 0x7fd298102070 2024-04-24T23:16:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:16:28.246 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:16:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:16:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2880a2b90 msgr2=0x7fd2880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2880a2b90 0x7fd2880a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005ab0 tx=0x7fd28c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 msgr2=0x7fd298102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 0x7fd298102fd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd29000eaa0 tx=0x7fd29000ef60 comp rx=0 tx=0).stop 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 shutdown_connections 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd2880a2b90 0x7fd2880a5050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd298102070 0x7fd298102fd0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2981016b0 0x7fd298102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 --2- 172.21.15.33:0/3481532704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2981004c0 0x7fd298104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 >> 172.21.15.33:0/3481532704 conn(0x7fd2980fba10 msgr2=0x7fd2980fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 shutdown_connections 2024-04-24T23:16:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.246+0000 7fd29dd9e640 1 -- 172.21.15.33:0/3481532704 wait complete. 2024-04-24T23:16:28.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:16:28.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 -- 172.21.15.33:0/933752050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e80104bd0 msgr2=0x7f2e80106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 --2- 172.21.15.33:0/933752050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e80104bd0 0x7f2e80106fc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009900 tx=0x7f2e7002a410 comp rx=0 tx=0).stop 2024-04-24T23:16:28.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 -- 172.21.15.33:0/933752050 shutdown_connections 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 --2- 172.21.15.33:0/933752050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80107500 0x7f2e801098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 --2- 172.21.15.33:0/933752050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e80104bd0 0x7f2e80106fc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 --2- 172.21.15.33:0/933752050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e801022a0 0x7f2e80104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 -- 172.21.15.33:0/933752050 >> 172.21.15.33:0/933752050 conn(0x7f2e800fbc30 msgr2=0x7f2e800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 -- 172.21.15.33:0/933752050 shutdown_connections 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 -- 172.21.15.33:0/933752050 wait complete. 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 Processor -- start 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.367+0000 7f2e87909640 1 -- start start 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 0x7f2e80199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80104bd0 0x7f2e8019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e80107500 0x7f2e801a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8010d3f0 con 0x7f2e801022a0 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8010d270 con 0x7f2e80107500 2024-04-24T23:16:28.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8010d570 con 0x7f2e80104bd0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 0x7f2e80199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 0x7f2e80199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56282/0 (socket says 172.21.15.33:56282) 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 -- 172.21.15.33:0/3097988942 learned_addr learned my addr 172.21.15.33:0/3097988942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e85ea6640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e80107500 0x7f2e801a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80104bd0 0x7f2e8019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 -- 172.21.15.33:0/3097988942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80104bd0 msgr2=0x7f2e8019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80104bd0 0x7f2e8019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 -- 172.21.15.33:0/3097988942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e80107500 msgr2=0x7f2e801a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e80107500 0x7f2e801a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 -- 172.21.15.33:0/3097988942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e801a1960 con 0x7f2e801022a0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80104bd0 0x7f2e8019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e856a5640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 0x7f2e80199d40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00e9f0 tx=0x7f2e7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e767fc640 1 -- 172.21.15.33:0/3097988942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00cd80 con 0x7f2e801022a0 2024-04-24T23:16:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e767fc640 1 -- 172.21.15.33:0/3097988942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7c010910 con 0x7f2e801022a0 2024-04-24T23:16:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e801a1c50 con 0x7f2e801022a0 2024-04-24T23:16:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e767fc640 1 -- 172.21.15.33:0/3097988942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c018990 con 0x7f2e801022a0 2024-04-24T23:16:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.368+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801a21a0 con 0x7f2e801022a0 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e767fc640 1 -- 172.21.15.33:0/3097988942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2e7c010ac0 con 0x7f2e801022a0 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 Processor -- start 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 -- start start 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e5624e670 con 0x558e562ea230 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558e561f6970 con 0x558e562eac20 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e8a614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e5624a630 con 0x558e562efd90 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e85ea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e85ea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56290/0 (socket says 172.21.15.33:56290) 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e84ea4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.371+0000 7f2e85ea6640 1 -- 172.21.15.33:0/1083524724 learned_addr learned my addr 172.21.15.33:0/1083524724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e767fc640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2e6c0a2b90 0x7f2e6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e856a5640 1 -- 172.21.15.33:0/1083524724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 msgr2=0x558e562f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e856a5640 1 -- 172.21.15.33:0/1083524724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 msgr2=0x558e562ea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562ea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e856a5640 1 -- 172.21.15.33:0/1083524724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5621a400 con 0x558e562eac20 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e85ea6640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2e6c0a2b90 0x7f2e6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e767fc640 1 -- 172.21.15.33:0/3097988942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e7c0cf7f0 con 0x7f2e801022a0 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562ef720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0ccea0 tx=0x7f2e7c0669a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb743c34aa6c5a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e85ea6640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2e6c0a2b90 0x7f2e6c0a5050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fd0 tx=0x7f2e78004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e6b7fe640 1 -- 172.21.15.33:0/1083524724 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c09b7a0 con 0x558e562eac20 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e6b7fe640 1 -- 172.21.15.33:0/1083524724 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7c09c360 con 0x558e562eac20 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.372+0000 7f2e6b7fe640 1 -- 172.21.15.33:0/1083524724 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c09b5c0 con 0x558e562eac20 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 -- 172.21.15.33:0/1083524724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 msgr2=0x558e562ef720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562ef720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0ccea0 tx=0x7f2e7c0669a0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 -- 172.21.15.33:0/1083524724 shutdown_connections 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562ef720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1083524724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 -- 172.21.15.33:0/1083524724 >> 172.21.15.33:0/1083524724 conn(0x558e562e58a0 msgr2=0x558e562e5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 -- 172.21.15.33:0/1083524724 shutdown_connections 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.373+0000 7f2e8a614740 1 -- 172.21.15.33:0/1083524724 wait complete. 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 -- start start 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 Processor -- start 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 0x558e562f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562efd90 0x558e562fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e562f2ce0 con 0x558e562efd90 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558e5620a050 con 0x558e562eac20 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e562f2e60 con 0x558e562ea230 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e8a614740 10 client.0 _ll_register_callbacks cb 0x558e55f61be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 0x558e562f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e84ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 0x558e562f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:50942/0 (socket says 172.21.15.33:50942) 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e856a5640 1 -- 172.21.15.33:0/1397308604 learned_addr learned my addr 172.21.15.33:0/1397308604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.374+0000 7f2e85ea6640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562efd90 0x558e562fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e856a5640 1 -- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 msgr2=0x558e562f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e856a5640 1 -- 172.21.15.33:0/1397308604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562efd90 msgr2=0x558e562fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562efd90 0x558e562fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e856a5640 1 -- 172.21.15.33:0/1397308604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e561dbdc0 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 0x558e562f4220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0b9740 tx=0x7f2e7c018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b486c319a4cd383c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1397308604 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c09b5c0 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1397308604 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e7c0d5170 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c09c360 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1397308604 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c09b7a0 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e8a614740 10 client.25897 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e55f623f0 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1397308604 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7c09c360 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1397308604 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2e7c0a4440 con 0x558e562ea230 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e8a614740 10 client.25897 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e8a614740 10 client.25897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.375+0000 7f2e8a614740 20 client.25897 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.376+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e562ffc90 0x558e56302080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.376+0000 7f2e8a614740 4 client.25897 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.376+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e5625bbb0 con 0x558e562ffc90 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.376+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e562ffc90 0x558e56302080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.376+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e562ffc90 0x558e56302080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e10a1a6b38741ce6 server_cookie=a98d3960c731b280 in_seq=0 out_seq=0 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.376+0000 7f2e68ff9640 10 client.25897 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1397308604 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e5625bbb0 con 0x558e562ffc90 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e68ff9640 10 client.25897 handle_command_reply: tid=0 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 client.25897 shutdown 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 msgr2=0x558e562f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 0x558e562f4220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0b9740 tx=0x7f2e7c018420 comp rx=0 tx=0).stop 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 shutdown_connections 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e562ffc90 0x558e56302080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562efd90 0x558e562fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1397308604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562ea230 0x558e562f4220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e68ff9640 0 client.25897 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:16:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 >> 172.21.15.33:0/1397308604 conn(0x558e562e58a0 msgr2=0x558e562e4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 shutdown_connections 2024-04-24T23:16:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 1 -- 172.21.15.33:0/1397308604 wait complete. 2024-04-24T23:16:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.377+0000 7f2e8a614740 20 client.25897 trim_cache size 0 max 16384 2024-04-24T23:16:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.390+0000 7f2e8a614740 1 Processor -- start 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.390+0000 7f2e8a614740 1 -- start start 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562ea230 0x558e562e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 0x558e562fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e8a614740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801aef60 con 0x558e562eac20 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e8a614740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801ac830 con 0x558e562ea230 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e8a614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801aa680 con 0x558e562efd90 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 0x558e562fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562ea230 0x558e562e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e85ea6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 0x558e562fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56306/0 (socket says 172.21.15.33:56306) 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562ea230 0x558e562e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36398/0 (socket says 172.21.15.33:36398) 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 -- 172.21.15.33:0/1330695880 learned_addr learned my addr 172.21.15.33:0/1330695880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 -- 172.21.15.33:0/1330695880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 msgr2=0x558e562e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562e8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 -- 172.21.15.33:0/1330695880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562ea230 msgr2=0x558e562e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562ea230 0x558e562e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 -- 172.21.15.33:0/1330695880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5602c7d0 con 0x558e562eac20 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 0x558e562fb760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006010 tx=0x7f2e700058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c851b7c4b8ac0a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e6affd640 1 -- 172.21.15.33:0/1330695880 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70035040 con 0x558e562eac20 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e6affd640 1 -- 172.21.15.33:0/1330695880 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e70005520 con 0x558e562eac20 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.391+0000 7f2e6affd640 1 -- 172.21.15.33:0/1330695880 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70044530 con 0x558e562eac20 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 -- 172.21.15.33:0/1330695880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 msgr2=0x558e562fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 0x558e562fb760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006010 tx=0x7f2e700058d0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 -- 172.21.15.33:0/1330695880 shutdown_connections 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562eac20 0x558e562fb760 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1330695880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562ea230 0x558e562e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 -- 172.21.15.33:0/1330695880 >> 172.21.15.33:0/1330695880 conn(0x558e562e58a0 msgr2=0x558e562e4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 -- 172.21.15.33:0/1330695880 shutdown_connections 2024-04-24T23:16:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 -- 172.21.15.33:0/1330695880 wait complete. 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 -- start start 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.392+0000 7f2e8a614740 1 Processor -- start 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801aacf0 con 0x558e562ea230 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801ac160 con 0x558e562eac20 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801ab3c0 con 0x558e562efd90 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e8a614740 10 client.0 _ll_register_callbacks cb 0x7f2e78009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56322/0 (socket says 172.21.15.33:56322) 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 -- 172.21.15.33:0/1229496163 learned_addr learned my addr 172.21.15.33:0/1229496163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e85ea6640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 -- 172.21.15.33:0/1229496163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 msgr2=0x558e562fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562fa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 -- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 msgr2=0x558e562f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 -- 172.21.15.33:0/1229496163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e562fa6c0 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e856a5640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562f8bd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0cfdf0 tx=0x7f2e7c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b980ceb48980c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.393+0000 7f2e68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1229496163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c09b5c0 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1229496163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e7c0d5170 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e70005520 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1229496163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c06f320 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 10 client.16347 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e562fb9c0 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1229496163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7c0b1080 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1229496163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2e7c0b9300 con 0x558e562ea230 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 10 client.16347 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:16:28.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 20 client.16347 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e56366620 0x558e56368a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 4 client.16347 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e56255f00 con 0x558e56366620 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.394+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e56366620 0x558e56368a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e68ff9640 10 client.16347 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e84ea4640 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e56366620 0x558e56368a40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1aeec6d878c6a15c server_cookie=564f5e2890215056 in_seq=0 out_seq=0 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e68ff9640 1 -- 172.21.15.33:0/1229496163 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e56255f00 con 0x558e56366620 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e68ff9640 10 client.16347 handle_command_reply: tid=0 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 client.16347 shutdown 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 msgr2=0x558e562f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562f8bd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0cfdf0 tx=0x7f2e7c009700 comp rx=0 tx=0).stop 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 shutdown_connections 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558e56366620 0x558e56368a40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e562efd90 0x558e562fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e562eac20 0x558e562f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e68ff9640 0 client.16347 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 --2- 172.21.15.33:0/1229496163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e562ea230 0x558e562f8bd0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 >> 172.21.15.33:0/1229496163 conn(0x558e562e58a0 msgr2=0x558e562eb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 shutdown_connections 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 1 -- 172.21.15.33:0/1229496163 wait complete. 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.395+0000 7f2e8a614740 20 client.16347 trim_cache size 0 max 16384 2024-04-24T23:16:28.404 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2e6c0a2b90 msgr2=0x7f2e6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2e6c0a2b90 0x7f2e6c0a5050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fd0 tx=0x7f2e78004a40 comp rx=0 tx=0).stop 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 msgr2=0x7f2e80199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 0x7f2e80199d40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00e9f0 tx=0x7f2e7c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 shutdown_connections 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2e6c0a2b90 0x7f2e6c0a5050 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e80107500 0x7f2e801a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e80104bd0 0x7f2e8019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 --2- 172.21.15.33:0/3097988942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e801022a0 0x7f2e80199d40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 >> 172.21.15.33:0/3097988942 conn(0x7f2e800fbc30 msgr2=0x7f2e80101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 shutdown_connections 2024-04-24T23:16:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.396+0000 7f2e87909640 1 -- 172.21.15.33:0/3097988942 wait complete. 2024-04-24T23:16:28.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:16:28.441 INFO:tasks.workunit.client.0.smithi033.stdout: 1 376345 12.76 MB/sec execute 33 sec latency 41.728 ms 2024-04-24T23:16:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 -- 172.21.15.33:0/971052090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 msgr2=0x7f7db8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/971052090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8101cf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009900 tx=0x7f7da002a410 comp rx=0 tx=0).stop 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 -- 172.21.15.33:0/971052090 shutdown_connections 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/971052090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db8102230 0x7f7db8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/971052090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8101cf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/971052090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7db8100680 0x7f7db8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 -- 172.21.15.33:0/971052090 >> 172.21.15.33:0/971052090 conn(0x7f7db80fbbf0 msgr2=0x7f7db80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 -- 172.21.15.33:0/971052090 shutdown_connections 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.511+0000 7f7dbe56a640 1 -- 172.21.15.33:0/971052090 wait complete. 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 Processor -- start 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 -- start start 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7db8100680 0x7f7db81969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db8102230 0x7f7db819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810c300 con 0x7f7db8101870 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810c180 con 0x7f7db8102230 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbe56a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810c480 con 0x7f7db8100680 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56348/0 (socket says 172.21.15.33:56348) 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 -- 172.21.15.33:0/1763942552 learned_addr learned my addr 172.21.15.33:0/1763942552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db7fff640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7db8100680 0x7f7db81969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db8102230 0x7f7db819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 -- 172.21.15.33:0/1763942552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7db8100680 msgr2=0x7f7db81969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7db8100680 0x7f7db81969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 -- 172.21.15.33:0/1763942552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db8102230 msgr2=0x7f7db819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.512+0000 7f7db77fe640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db8102230 0x7f7db819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7db77fe640 1 -- 172.21.15.33:0/1763942552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db819e610 con 0x7f7db8101870 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7db77fe640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8196f30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009230 tx=0x7f7da0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7db57fa640 1 -- 172.21.15.33:0/1763942552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0046070 con 0x7f7db8101870 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db80688d0 con 0x7f7db8101870 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7db57fa640 1 -- 172.21.15.33:0/1763942552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da0003ad0 con 0x7f7db8101870 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7db57fa640 1 -- 172.21.15.33:0/1763942552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0035050 con 0x7f7db8101870 2024-04-24T23:16:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.513+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db8068dc0 con 0x7f7db8101870 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7db57fa640 1 -- 172.21.15.33:0/1763942552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7da0049070 con 0x7f7db8101870 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 Processor -- start 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 -- start start 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56415f231670 con 0x56415f2d1640 2024-04-24T23:16:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56415f1d9970 con 0x56415f2d71a0 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dc1270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56415f22d630 con 0x56415f2d2030 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dbcb07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dbcb07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56360/0 (socket says 172.21.15.33:56360) 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7dbcb07640 1 -- 172.21.15.33:0/234965948 learned_addr learned my addr 172.21.15.33:0/234965948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7db7fff640 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.516+0000 7f7db77fe640 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 -- 172.21.15.33:0/234965948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 msgr2=0x56415f2d6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2d6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7db57fa640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7da40a2b90 0x7f7da40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 -- 172.21.15.33:0/234965948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 msgr2=0x56415f2d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2d9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 -- 172.21.15.33:0/234965948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56415f1fd400 con 0x56415f2d1640 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7db57fa640 1 -- 172.21.15.33:0/1763942552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7da00362a0 con 0x7f7db8101870 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d1a60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009fd0 tx=0x7f7da80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0dbf6d30f1deb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7da40a2b90 0x7f7da40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dae7fc640 1 -- 172.21.15.33:0/234965948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da801e070 con 0x56415f2d1640 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dae7fc640 1 -- 172.21.15.33:0/234965948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da8004630 con 0x56415f2d1640 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.517+0000 7f7dae7fc640 1 -- 172.21.15.33:0/234965948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8019630 con 0x56415f2d1640 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 -- 172.21.15.33:0/234965948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 msgr2=0x56415f2d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d1a60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009fd0 tx=0x7f7da80118d0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7da40a2b90 0x7f7da40a5050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7da801b000 tx=0x7f7da8019d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 -- 172.21.15.33:0/234965948 shutdown_connections 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 --2- 172.21.15.33:0/234965948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d1a60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 -- 172.21.15.33:0/234965948 >> 172.21.15.33:0/234965948 conn(0x56415f2cccb0 msgr2=0x56415f2cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 -- 172.21.15.33:0/234965948 shutdown_connections 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 -- 172.21.15.33:0/234965948 wait complete. 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 -- start start 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.518+0000 7f7dc1270740 1 Processor -- start 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d1640 0x56415f2db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2dbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 0x56415f2e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56415f2da150 con 0x56415f2d71a0 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56415f1ed050 con 0x56415f2d1640 2024-04-24T23:16:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56415f2da2d0 con 0x56415f2d2030 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dc1270740 10 client.0 _ll_register_callbacks cb 0x56415ef44be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 0x56415f2e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7db7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d1640 0x56415f2db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2dbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 0x56415f2e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56370/0 (socket says 172.21.15.33:56370) 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 -- 172.21.15.33:0/713374279 learned_addr learned my addr 172.21.15.33:0/713374279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 -- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 msgr2=0x56415f2dbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2dbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 -- 172.21.15.33:0/713374279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d1640 msgr2=0x56415f2db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d1640 0x56415f2db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 -- 172.21.15.33:0/713374279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56415f1bedc0 con 0x56415f2d71a0 2024-04-24T23:16:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7db77fe640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2dbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7db7fff640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d1640 0x56415f2db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 0x56415f2e2ce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7da8006010 tx=0x7f7da8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caf85987c1afcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7d977fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.519+0000 7f7d977fe640 1 -- 172.21.15.33:0/713374279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8034020 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7d977fe640 1 -- 172.21.15.33:0/713374279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7da8019630 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da8004630 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7d977fe640 1 -- 172.21.15.33:0/713374279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7da803f050 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 10 client.16359 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56415ef453f0 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7d977fe640 1 -- 172.21.15.33:0/713374279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da8004630 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7d977fe640 1 -- 172.21.15.33:0/713374279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7da800e950 con 0x56415f2d71a0 2024-04-24T23:16:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 10 client.16359 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 20 client.16359 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f2e7460 0x56415f2e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 4 client.16359 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56415f23ebb0 con 0x56415f2e7460 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.520+0000 7f7db7fff640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f2e7460 0x56415f2e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.521+0000 7f7db7fff640 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f2e7460 0x56415f2e98f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f038201b28aae158 server_cookie=7f5b75151d5ba399 in_seq=0 out_seq=0 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.521+0000 7f7d977fe640 10 client.16359 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.521+0000 7f7d977fe640 1 -- 172.21.15.33:0/713374279 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56415f23ebb0 con 0x56415f2e7460 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.521+0000 7f7d977fe640 10 client.16359 handle_command_reply: tid=0 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.521+0000 7f7dc1270740 1 client.16359 shutdown 2024-04-24T23:16:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 msgr2=0x56415f2e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 0x56415f2e2ce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7da8006010 tx=0x7f7da8019000 comp rx=0 tx=0).stop 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 shutdown_connections 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f2e7460 0x56415f2e98f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d71a0 0x56415f2e2ce0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d2030 0x56415f2dbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7d977fe640 0 client.16359 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 --2- 172.21.15.33:0/713374279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d1640 0x56415f2db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 >> 172.21.15.33:0/713374279 conn(0x56415f2cccb0 msgr2=0x56415f2c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 shutdown_connections 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 1 -- 172.21.15.33:0/713374279 wait complete. 2024-04-24T23:16:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.522+0000 7f7dc1270740 20 client.16359 trim_cache size 0 max 16384 2024-04-24T23:16:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 Processor -- start 2024-04-24T23:16:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 -- start start 2024-04-24T23:16:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d2030 0x56415f2e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d71a0 0x56415f2cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81aaac0 con 0x56415f2d1640 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81a8390 con 0x56415f2d2030 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dc1270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81a61e0 con 0x56415f2d71a0 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56382/0 (socket says 172.21.15.33:56382) 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 -- 172.21.15.33:0/2901186017 learned_addr learned my addr 172.21.15.33:0/2901186017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db77fe640 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d2030 0x56415f2e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d71a0 0x56415f2cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 -- 172.21.15.33:0/2901186017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d71a0 msgr2=0x56415f2cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d71a0 0x56415f2cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 -- 172.21.15.33:0/2901186017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d2030 msgr2=0x56415f2e2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d2030 0x56415f2e2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 -- 172.21.15.33:0/2901186017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56415f00f7d0 con 0x56415f2d1640 2024-04-24T23:16:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.536+0000 7f7db7fff640 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d6750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x56415f2d2aa0 tx=0x7f7db0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba90ea9e15f5b9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dadffb640 1 -- 172.21.15.33:0/2901186017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0020310 con 0x56415f2d1640 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dadffb640 1 -- 172.21.15.33:0/2901186017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db0010430 con 0x56415f2d1640 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dadffb640 1 -- 172.21.15.33:0/2901186017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db001e680 con 0x56415f2d1640 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 -- 172.21.15.33:0/2901186017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 msgr2=0x56415f2d6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d6750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x56415f2d2aa0 tx=0x7f7db0005f90 comp rx=0 tx=0).stop 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 -- 172.21.15.33:0/2901186017 shutdown_connections 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d71a0 0x56415f2cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d2030 0x56415f2e2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 --2- 172.21.15.33:0/2901186017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d1640 0x56415f2d6750 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 -- 172.21.15.33:0/2901186017 >> 172.21.15.33:0/2901186017 conn(0x56415f2cccb0 msgr2=0x56415f2c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 -- 172.21.15.33:0/2901186017 shutdown_connections 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.537+0000 7f7dc1270740 1 -- 172.21.15.33:0/2901186017 wait complete. 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 -- start start 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 Processor -- start 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d1640 0x56415f2e05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 0x56415f2e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81a6850 con 0x56415f2d2030 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81a7cc0 con 0x56415f2d71a0 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81a6f20 con 0x56415f2d1640 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dc1270740 10 client.0 _ll_register_callbacks cb 0x7f7da801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:28.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 0x56415f2e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 0x56415f2e0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56390/0 (socket says 172.21.15.33:56390) 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 -- 172.21.15.33:0/3461313082 learned_addr learned my addr 172.21.15.33:0/3461313082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db7fff640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d1640 0x56415f2e05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 -- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d1640 msgr2=0x56415f2e05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d1640 0x56415f2e05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 -- 172.21.15.33:0/3461313082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 msgr2=0x56415f2e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.538+0000 7f7db77fe640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7db77fe640 1 -- 172.21.15.33:0/3461313082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56415f2e20e0 con 0x56415f2d2030 2024-04-24T23:16:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7db7fff640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d1640 0x56415f2e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dbcb07640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7db77fe640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 0x56415f2e0ae0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7da00314e0 tx=0x7f7da00378b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8457429fe85333e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7d977fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7d977fe640 1 -- 172.21.15.33:0/3461313082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da00f7040 con 0x56415f2d2030 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7d977fe640 1 -- 172.21.15.33:0/3461313082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7da00ee920 con 0x56415f2d2030 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db0010430 con 0x56415f2d2030 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7d977fe640 1 -- 172.21.15.33:0/3461313082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7da00d3e60 con 0x56415f2d2030 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 10 client.16365 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56415f34dca0 con 0x56415f2d2030 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7d977fe640 1 -- 172.21.15.33:0/3461313082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db0010430 con 0x56415f2d2030 2024-04-24T23:16:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7d977fe640 1 -- 172.21.15.33:0/3461313082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7da00bd880 con 0x56415f2d2030 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 10 client.16365 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 20 client.16365 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f34e6a0 0x56415f350ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 4 client.16365 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.539+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56415f238f00 con 0x56415f34e6a0 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7db7fff640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f34e6a0 0x56415f350ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7d977fe640 10 client.16365 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7db7fff640 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f34e6a0 0x56415f350ac0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17ae60f494057e1e server_cookie=7b95b9e35768ed46 in_seq=0 out_seq=0 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7d977fe640 1 -- 172.21.15.33:0/3461313082 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56415f238f00 con 0x56415f34e6a0 2024-04-24T23:16:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7d977fe640 10 client.16365 handle_command_reply: tid=0 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 client.16365 shutdown 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 msgr2=0x56415f2e0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 0x56415f2e0ae0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7da00314e0 tx=0x7f7da00378b0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 shutdown_connections 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56415f34e6a0 0x56415f350ac0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56415f2d71a0 0x56415f2e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56415f2d2030 0x56415f2e0ae0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 --2- 172.21.15.33:0/3461313082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56415f2d1640 0x56415f2e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7d977fe640 0 client.16365 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 >> 172.21.15.33:0/3461313082 conn(0x56415f2cccb0 msgr2=0x56415f2d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 shutdown_connections 2024-04-24T23:16:28.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.540+0000 7f7dc1270740 1 -- 172.21.15.33:0/3461313082 wait complete. 2024-04-24T23:16:28.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dc1270740 20 client.16365 trim_cache size 0 max 16384 2024-04-24T23:16:28.552 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7da40a2b90 msgr2=0x7f7da40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7da40a2b90 0x7f7da40a5050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7da801b000 tx=0x7f7da8019d00 comp rx=0 tx=0).stop 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 msgr2=0x7f7db8196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8196f30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009230 tx=0x7f7da0004270 comp rx=0 tx=0).stop 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 shutdown_connections 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7da40a2b90 0x7f7da40a5050 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db8102230 0x7f7db819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101870 0x7f7db8196f30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 --2- 172.21.15.33:0/1763942552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7db8100680 0x7f7db81969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 >> 172.21.15.33:0/1763942552 conn(0x7f7db80fbbf0 msgr2=0x7f7db8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 shutdown_connections 2024-04-24T23:16:28.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.541+0000 7f7dbe56a640 1 -- 172.21.15.33:0/1763942552 wait complete. 2024-04-24T23:16:28.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3481532704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:28.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:16:28.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3481532704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:28.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3481532704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 -- 172.21.15.33:0/649068494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c104b90 msgr2=0x7efd3c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 --2- 172.21.15.33:0/649068494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c104b90 0x7efd3c105010 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009900 tx=0x7efd2c02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 -- 172.21.15.33:0/649068494 shutdown_connections 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 --2- 172.21.15.33:0/649068494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd3c105550 0x7efd3c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 --2- 172.21.15.33:0/649068494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c104b90 0x7efd3c105010 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 --2- 172.21.15.33:0/649068494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c1039a0 0x7efd3c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.700+0000 7efd43ded640 1 -- 172.21.15.33:0/649068494 >> 172.21.15.33:0/649068494 conn(0x7efd3c0fef30 msgr2=0x7efd3c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 -- 172.21.15.33:0/649068494 shutdown_connections 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 -- 172.21.15.33:0/649068494 wait complete. 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 Processor -- start 2024-04-24T23:16:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 -- start start 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 0x7efd3c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c104b90 0x7efd3c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd3c105550 0x7efd3c1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10f620 con 0x7efd3c1039a0 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10f4a0 con 0x7efd3c104b90 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd43ded640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10f7a0 con 0x7efd3c105550 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd41b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 0x7efd3c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd41b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 0x7efd3c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56408/0 (socket says 172.21.15.33:56408) 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd41b89640 1 -- 172.21.15.33:0/528877416 learned_addr learned my addr 172.21.15.33:0/528877416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd41388640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c104b90 0x7efd3c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.701+0000 7efd4238a640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd3c105550 0x7efd3c1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd41b89640 1 -- 172.21.15.33:0/528877416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd3c105550 msgr2=0x7efd3c1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd41b89640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd3c105550 0x7efd3c1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd41b89640 1 -- 172.21.15.33:0/528877416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c104b90 msgr2=0x7efd3c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd41b89640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c104b90 0x7efd3c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd41b89640 1 -- 172.21.15.33:0/528877416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c1a18d0 con 0x7efd3c1039a0 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd41b89640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 0x7efd3c199c90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efd3800e9c0 tx=0x7efd3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd32ffd640 1 -- 172.21.15.33:0/528877416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3800cd80 con 0x7efd3c1039a0 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c1a1bc0 con 0x7efd3c1039a0 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd32ffd640 1 -- 172.21.15.33:0/528877416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd380035f0 con 0x7efd3c1039a0 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd32ffd640 1 -- 172.21.15.33:0/528877416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38010690 con 0x7efd3c1039a0 2024-04-24T23:16:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.702+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c1a20e0 con 0x7efd3c1039a0 2024-04-24T23:16:28.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.705+0000 7efd32ffd640 1 -- 172.21.15.33:0/528877416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efd380037a0 con 0x7efd3c1039a0 2024-04-24T23:16:28.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.705+0000 7efd46bcf740 1 Processor -- start 2024-04-24T23:16:28.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.705+0000 7efd46bcf740 1 -- start start 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 0x55fbe90b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd46bcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbe9015670 con 0x55fbe90b6d90 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd46bcf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8fbd970 con 0x55fbe90b1c20 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd46bcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fbe9011630 con 0x55fbe90b1230 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd41b89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 0x55fbe90b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd41b89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36474/0 (socket says 172.21.15.33:36474) 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 0x55fbe90b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51056/0 (socket says 172.21.15.33:51056) 2024-04-24T23:16:28.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd41388640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd41b89640 1 -- 172.21.15.33:0/1402256120 learned_addr learned my addr 172.21.15.33:0/1402256120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd32ffd640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd280a2b90 0x7efd280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 -- 172.21.15.33:0/1402256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 msgr2=0x55fbe90b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 -- 172.21.15.33:0/1402256120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 msgr2=0x55fbe90b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 -- 172.21.15.33:0/1402256120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe8fe1400 con 0x55fbe90b1230 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd41b89640 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd32ffd640 1 -- 172.21.15.33:0/528877416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd380cf7a0 con 0x7efd3c1039a0 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd280a2b90 0x7efd280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.706+0000 7efd4238a640 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 0x55fbe90b1650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd34009f00 tx=0x7efd34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144c4d8be9bd64f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd27fff640 1 -- 172.21.15.33:0/1402256120 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34004030 con 0x55fbe90b1230 2024-04-24T23:16:28.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd27fff640 1 -- 172.21.15.33:0/1402256120 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd34004830 con 0x55fbe90b1230 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd27fff640 1 -- 172.21.15.33:0/1402256120 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3401c690 con 0x55fbe90b1230 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd4238a640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd280a2b90 0x7efd280a5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efd34020000 tx=0x7efd34021000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 -- 172.21.15.33:0/1402256120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 msgr2=0x55fbe90b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 0x55fbe90b1650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd34009f00 tx=0x7efd34011810 comp rx=0 tx=0).stop 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 -- 172.21.15.33:0/1402256120 shutdown_connections 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1402256120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1230 0x55fbe90b1650 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.707+0000 7efd46bcf740 1 -- 172.21.15.33:0/1402256120 >> 172.21.15.33:0/1402256120 conn(0x55fbe90ac8a0 msgr2=0x55fbe90acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 -- 172.21.15.33:0/1402256120 shutdown_connections 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 -- 172.21.15.33:0/1402256120 wait complete. 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 -- start start 2024-04-24T23:16:28.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 Processor -- start 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 0x55fbe90bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 0x55fbe90bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbe90b9d40 con 0x55fbe90b6d90 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbe8fd1050 con 0x55fbe90b1230 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fbe90b9ec0 con 0x55fbe90b1c20 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.708+0000 7efd46bcf740 10 client.0 _ll_register_callbacks cb 0x55fbe8d28be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd41388640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 0x55fbe90bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd41b89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 0x55fbe90bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56426/0 (socket says 172.21.15.33:56426) 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd41388640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 0x55fbe90bb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51070/0 (socket says 172.21.15.33:51070) 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd41b89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 0x55fbe90bb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36476/0 (socket says 172.21.15.33:36476) 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 -- 172.21.15.33:0/1137496302 learned_addr learned my addr 172.21.15.33:0/1137496302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 -- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 msgr2=0x55fbe90bb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 0x55fbe90bb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 -- 172.21.15.33:0/1137496302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 msgr2=0x55fbe90bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 0x55fbe90bb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 -- 172.21.15.33:0/1137496302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe8fa2dc0 con 0x55fbe90b6d90 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd41388640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 0x55fbe90bb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd41b89640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 0x55fbe90bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd4238a640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90c2890 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd34011d10 tx=0x7efd34003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496301fd60a49da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd257fa640 1 -- 172.21.15.33:0/1137496302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34004bf0 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd34004830 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd257fa640 1 -- 172.21.15.33:0/1137496302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd34004d90 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd257fa640 1 -- 172.21.15.33:0/1137496302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd3403d050 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd46bcf740 10 client.16377 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.709+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbe8d293f0 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd257fa640 1 -- 172.21.15.33:0/1137496302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd34004830 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd257fa640 1 -- 172.21.15.33:0/1137496302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd34005040 con 0x55fbe90b6d90 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd46bcf740 10 client.16377 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd46bcf740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd46bcf740 20 client.16377 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:28.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe90c6ff0 0x55fbe90c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd46bcf740 4 client.16377 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbe9022bb0 con 0x55fbe90c6ff0 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.710+0000 7efd41b89640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe90c6ff0 0x55fbe90c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd41b89640 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe90c6ff0 0x55fbe90c9490 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5a89751a865109 server_cookie=84341d79e22804c5 in_seq=0 out_seq=0 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd257fa640 10 client.16377 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd257fa640 1 -- 172.21.15.33:0/1137496302 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbe9022bb0 con 0x55fbe90c6ff0 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd257fa640 10 client.16377 handle_command_reply: tid=0 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 client.16377 shutdown 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 msgr2=0x55fbe90c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90c2890 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd34011d10 tx=0x7efd34003d50 comp rx=0 tx=0).stop 2024-04-24T23:16:28.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 shutdown_connections 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe90c6ff0 0x55fbe90c9490 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b6d90 0x55fbe90c2890 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b1c20 0x55fbe90bb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd257fa640 0 client.16377 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 --2- 172.21.15.33:0/1137496302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1230 0x55fbe90bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.711+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 >> 172.21.15.33:0/1137496302 conn(0x55fbe90ac8a0 msgr2=0x55fbe90abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.712+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 shutdown_connections 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.712+0000 7efd46bcf740 1 -- 172.21.15.33:0/1137496302 wait complete. 2024-04-24T23:16:28.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.712+0000 7efd46bcf740 20 client.16377 trim_cache size 0 max 16384 2024-04-24T23:16:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 Processor -- start 2024-04-24T23:16:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 -- start start 2024-04-24T23:16:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe90b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1aee30 con 0x55fbe90b1230 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1ac700 con 0x55fbe90b1c20 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd46bcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1aa550 con 0x55fbe90b6d90 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd41b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd41b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56430/0 (socket says 172.21.15.33:56430) 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd41b89640 1 -- 172.21.15.33:0/119026786 learned_addr learned my addr 172.21.15.33:0/119026786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.726+0000 7efd4238a640 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe90b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41388640 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41b89640 1 -- 172.21.15.33:0/119026786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 msgr2=0x55fbe90b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41b89640 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe90b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41b89640 1 -- 172.21.15.33:0/119026786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 msgr2=0x55fbe90c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41b89640 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41b89640 1 -- 172.21.15.33:0/119026786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe8df37d0 con 0x55fbe90b1230 2024-04-24T23:16:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd4238a640 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe90b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd41b89640 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90b26c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efd3809bc00 tx=0x7efd380cf9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5c6f0c5d8ef4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd277fe640 1 -- 172.21.15.33:0/119026786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3809b7b0 con 0x55fbe90b1230 2024-04-24T23:16:28.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd277fe640 1 -- 172.21.15.33:0/119026786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3809c370 con 0x55fbe90b1230 2024-04-24T23:16:28.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.727+0000 7efd277fe640 1 -- 172.21.15.33:0/119026786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3809b5d0 con 0x55fbe90b1230 2024-04-24T23:16:28.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 -- 172.21.15.33:0/119026786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 msgr2=0x55fbe90b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90b26c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efd3809bc00 tx=0x7efd380cf9a0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 -- 172.21.15.33:0/119026786 shutdown_connections 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe90b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 --2- 172.21.15.33:0/119026786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90b26c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 -- 172.21.15.33:0/119026786 >> 172.21.15.33:0/119026786 conn(0x55fbe90ac8a0 msgr2=0x55fbe90bb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 -- 172.21.15.33:0/119026786 shutdown_connections 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 -- 172.21.15.33:0/119026786 wait complete. 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.728+0000 7efd46bcf740 1 -- start start 2024-04-24T23:16:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 Processor -- start 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe912b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1aabc0 con 0x55fbe90b1230 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1ac030 con 0x55fbe90b1c20 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1ab290 con 0x55fbe90b6d90 2024-04-24T23:16:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd46bcf740 10 client.0 _ll_register_callbacks cb 0x7efd3401f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90bf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56440/0 (socket says 172.21.15.33:56440) 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 -- 172.21.15.33:0/2592748631 learned_addr learned my addr 172.21.15.33:0/2592748631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41388640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd4238a640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe912b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 -- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 msgr2=0x55fbe912b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe912b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 -- 172.21.15.33:0/2592748631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 msgr2=0x55fbe90bf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90bf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.729+0000 7efd41b89640 1 -- 172.21.15.33:0/2592748631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe90c29b0 con 0x55fbe90b1230 2024-04-24T23:16:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd41b89640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90bf3a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c6a0 tx=0x7efd38005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1cdb0e84727b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd257fa640 1 -- 172.21.15.33:0/2592748631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3809b5d0 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd257fa640 1 -- 172.21.15.33:0/2592748631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd3809b2c0 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3809c370 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd257fa640 1 -- 172.21.15.33:0/2592748631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd3809b7b0 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 10 client.16383 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbe912c9f0 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd257fa640 1 -- 172.21.15.33:0/2592748631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3809c370 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd257fa640 1 -- 172.21.15.33:0/2592748631 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd3809ab30 con 0x55fbe90b1230 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 10 client.16383 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 20 client.16383 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe912d360 0x55fbe912f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 4 client.16383 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.730+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbe901cf00 con 0x55fbe912d360 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd41388640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe912d360 0x55fbe912f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd257fa640 10 client.16383 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd41388640 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe912d360 0x55fbe912f780 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94982c06d2bbc84 server_cookie=2c3cc61a62f183d6 in_seq=0 out_seq=0 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd257fa640 1 -- 172.21.15.33:0/2592748631 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbe901cf00 con 0x55fbe912d360 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd257fa640 10 client.16383 handle_command_reply: tid=0 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd46bcf740 1 client.16383 shutdown 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 msgr2=0x55fbe90bf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd46bcf740 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90bf3a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c6a0 tx=0x7efd38005ca0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 shutdown_connections 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.731+0000 7efd46bcf740 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbe912d360 0x55fbe912f780 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbe90b6d90 0x55fbe912b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbe90b1c20 0x55fbe90bf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd257fa640 0 client.16383 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 1 --2- 172.21.15.33:0/2592748631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbe90b1230 0x55fbe90bf3a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 >> 172.21.15.33:0/2592748631 conn(0x55fbe90ac8a0 msgr2=0x55fbe90ab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 shutdown_connections 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 1 -- 172.21.15.33:0/2592748631 wait complete. 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd46bcf740 20 client.16383 trim_cache size 0 max 16384 2024-04-24T23:16:28.743 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:16:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd280a2b90 msgr2=0x7efd280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd280a2b90 0x7efd280a5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efd34020000 tx=0x7efd34021000 comp rx=0 tx=0).stop 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 msgr2=0x7efd3c199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 0x7efd3c199c90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efd3800e9c0 tx=0x7efd3800ee80 comp rx=0 tx=0).stop 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 shutdown_connections 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd280a2b90 0x7efd280a5050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd3c105550 0x7efd3c1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c104b90 0x7efd3c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 --2- 172.21.15.33:0/528877416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c1039a0 0x7efd3c199c90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.732+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 >> 172.21.15.33:0/528877416 conn(0x7efd3c0fef30 msgr2=0x7efd3c107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.733+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 shutdown_connections 2024-04-24T23:16:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:28.733+0000 7efd43ded640 1 -- 172.21.15.33:0/528877416 wait complete. 2024-04-24T23:16:29.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 378403 12.59 MB/sec execute 34 sec latency 114.176 ms 2024-04-24T23:16:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:29 smithi038 ceph-mon[32450]: pgmap v485: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:16:29.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:29 smithi117 ceph-mon[32529]: pgmap v485: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:16:29.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:29 smithi033 ceph-mon[29201]: pgmap v485: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:16:29.748 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:16:29.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:16:29.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:16:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 -- 172.21.15.33:0/482341260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4102230 msgr2=0x7fcaa4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/482341260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4102230 0x7fcaa4108ac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ca0 tx=0x7fca9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 -- 172.21.15.33:0/482341260 shutdown_connections 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/482341260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4102230 0x7fcaa4108ac0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/482341260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4101870 0x7fcaa4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/482341260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100680 0x7fcaa4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 -- 172.21.15.33:0/482341260 >> 172.21.15.33:0/482341260 conn(0x7fcaa40fbbf0 msgr2=0x7fcaa40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 -- 172.21.15.33:0/482341260 shutdown_connections 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.854+0000 7fcaab0e1640 1 -- 172.21.15.33:0/482341260 wait complete. 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 Processor -- start 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 -- start start 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4100680 0x7fcaa4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 0x7fcaa4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4102230 0x7fcaa419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410c300 con 0x7fcaa4101870 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410c180 con 0x7fcaa4102230 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaab0e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410c480 con 0x7fcaa4100680 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 0x7fcaa4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4100680 0x7fcaa4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 0x7fcaa4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56464/0 (socket says 172.21.15.33:56464) 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaa3fff640 1 -- 172.21.15.33:0/2935658329 learned_addr learned my addr 172.21.15.33:0/2935658329 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.855+0000 7fcaa967e640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4102230 0x7fcaa419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2935658329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4102230 msgr2=0x7fcaa419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4102230 0x7fcaa419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 msgr2=0x7fcaa4197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 0x7fcaa4197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2935658329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa419f6b0 con 0x7fcaa4100680 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 0x7fcaa4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4100680 0x7fcaa4197a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca9800e9c0 tx=0x7fca9800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa1ffb640 1 -- 172.21.15.33:0/2935658329 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9800ccc0 con 0x7fcaa4100680 2024-04-24T23:16:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa419f9a0 con 0x7fcaa4100680 2024-04-24T23:16:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa1ffb640 1 -- 172.21.15.33:0/2935658329 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca980035f0 con 0x7fcaa4100680 2024-04-24T23:16:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaa1ffb640 1 -- 172.21.15.33:0/2935658329 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9800ce60 con 0x7fcaa4100680 2024-04-24T23:16:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.856+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa419ff40 con 0x7fcaa4100680 2024-04-24T23:16:29.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.858+0000 7fcaa1ffb640 1 -- 172.21.15.33:0/2935658329 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fca980037a0 con 0x7fcaa4100680 2024-04-24T23:16:29.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 Processor -- start 2024-04-24T23:16:29.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 -- start start 2024-04-24T23:16:29.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef0381f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03824690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03827190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ef0377e2c0 con 0x55ef0381f1a0 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ef03726910 con 0x55ef0381fb90 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef0377a280 con 0x55ef03824d00 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa967e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef0381f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa967e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef0381f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56466/0 (socket says 172.21.15.33:56466) 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03824690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa967e640 1 -- 172.21.15.33:0/501188758 learned_addr learned my addr 172.21.15.33:0/501188758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa1ffb640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca940a2b90 0x7fca940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03827190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa967e640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca940a2b90 0x7fca940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.859+0000 7fcaa1ffb640 1 -- 172.21.15.33:0/2935658329 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca980cf840 con 0x7fcaa4100680 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/501188758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 msgr2=0x55ef03827190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03827190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/501188758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 msgr2=0x55ef0381f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef0381f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/501188758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef03737fb0 con 0x55ef0381fb90 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa967e640 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef0381f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03824690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca98098110 tx=0x7fca98098010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2ac197316e41da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fcaa967e640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca940a2b90 0x7fca940a5050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55ef038203f0 tx=0x7fca9c02a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fca8effd640 1 -- 172.21.15.33:0/501188758 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9809c3b0 con 0x55ef0381fb90 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fca8effd640 1 -- 172.21.15.33:0/501188758 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9809bcb0 con 0x55ef0381fb90 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.860+0000 7fca8effd640 1 -- 172.21.15.33:0/501188758 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9809b360 con 0x55ef0381fb90 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- 172.21.15.33:0/501188758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 msgr2=0x55ef03824690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03824690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca98098110 tx=0x7fca98098010 comp rx=0 tx=0).stop 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- 172.21.15.33:0/501188758 shutdown_connections 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03827190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03824690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/501188758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef0381f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- 172.21.15.33:0/501188758 >> 172.21.15.33:0/501188758 conn(0x55ef037e2950 msgr2=0x55ef03816810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- 172.21.15.33:0/501188758 shutdown_connections 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- 172.21.15.33:0/501188758 wait complete. 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- start start 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 Processor -- start 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef03829600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 0x55ef03829b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03830b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ef03827c50 con 0x55ef0381fb90 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ef03739e90 con 0x55ef0381f1a0 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef03827dd0 con 0x55ef03824d00 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.861+0000 7fcaaddf4740 10 client.0 _ll_register_callbacks cb 0x55ef03775970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 0x55ef03829b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef03829600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa967e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03830b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef03829600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36532/0 (socket says 172.21.15.33:36532) 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/3919356961 learned_addr learned my addr 172.21.15.33:0/3919356961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa967e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03830b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51128/0 (socket says 172.21.15.33:51128) 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 0x55ef03829b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56476/0 (socket says 172.21.15.33:56476) 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 -- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 msgr2=0x55ef03830b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03830b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 -- 172.21.15.33:0/3919356961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 msgr2=0x55ef03829600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef03829600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 -- 172.21.15.33:0/3919356961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0355a7e0 con 0x55ef0381fb90 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef03829600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa967e640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03830b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 0x55ef03829b40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fca90009640 tx=0x7fca9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9139fc6b26f01870 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fca83fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fca83fff640 1 -- 172.21.15.33:0/3919356961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9000fe50 con 0x55ef0381fb90 2024-04-24T23:16:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9809bcb0 con 0x55ef0381fb90 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fca83fff640 1 -- 172.21.15.33:0/3919356961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca90005020 con 0x55ef0381fb90 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.862+0000 7fca83fff640 1 -- 172.21.15.33:0/3919356961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca90005220 con 0x55ef0381fb90 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef035baf70 con 0x55ef0381fb90 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fca83fff640 1 -- 172.21.15.33:0/3919356961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9809bcb0 con 0x55ef0381fb90 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fca83fff640 1 -- 172.21.15.33:0/3919356961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca9001c470 con 0x55ef0381fb90 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 20 client.16395 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef03835170 0x55ef038375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 4 client.16395 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef037525a0 con 0x55ef03835170 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.863+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef03835170 0x55ef038375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef03835170 0x55ef038375d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8baee2b8decec58 server_cookie=23b3f683692af2c9 in_seq=0 out_seq=0 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fca83fff640 10 client.16395 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fca83fff640 1 -- 172.21.15.33:0/3919356961 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef037525a0 con 0x55ef03835170 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fca83fff640 10 client.16395 handle_command_reply: tid=0 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 client.16395 shutdown 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 msgr2=0x55ef03829b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 0x55ef03829b40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fca90009640 tx=0x7fca9000f790 comp rx=0 tx=0).stop 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 shutdown_connections 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef03835170 0x55ef038375d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03830b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381fb90 0x55ef03829b40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3919356961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef03829600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fca83fff640 0 client.16395 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.864+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 >> 172.21.15.33:0/3919356961 conn(0x55ef037e2950 msgr2=0x55ef03815950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.865+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 shutdown_connections 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.865+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3919356961 wait complete. 2024-04-24T23:16:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.865+0000 7fcaaddf4740 20 client.16395 trim_cache size 0 max 16384 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.882+0000 7fcaaddf4740 1 Processor -- start 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.882+0000 7fcaaddf4740 1 -- start start 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef038150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03830b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03815600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41aaa20 con 0x55ef0381f1a0 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a82f0 con 0x55ef0381fb90 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a6140 con 0x55ef03824d00 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef038150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef038150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56484/0 (socket says 172.21.15.33:56484) 2024-04-24T23:16:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2628661994 learned_addr learned my addr 172.21.15.33:0/2628661994 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa967e640 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03815600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03830b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2628661994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 msgr2=0x55ef03815600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03815600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2628661994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 msgr2=0x55ef03830b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03830b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/2628661994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef03452170 con 0x55ef0381f1a0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa3fff640 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03830b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef038150c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fca9809b960 tx=0x7fca98098080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687c06462b1155a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fca8e7fc640 1 -- 172.21.15.33:0/2628661994 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98003b10 con 0x55ef0381f1a0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fca8e7fc640 1 -- 172.21.15.33:0/2628661994 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9809c3b0 con 0x55ef0381f1a0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.883+0000 7fca8e7fc640 1 -- 172.21.15.33:0/2628661994 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98004000 con 0x55ef0381f1a0 2024-04-24T23:16:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 -- 172.21.15.33:0/2628661994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 msgr2=0x55ef038150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef038150c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fca9809b960 tx=0x7fca98098080 comp rx=0 tx=0).stop 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 -- 172.21.15.33:0/2628661994 shutdown_connections 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef03824d00 0x55ef03815600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381fb90 0x55ef03830b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/2628661994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef0381f1a0 0x55ef038150c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 -- 172.21.15.33:0/2628661994 >> 172.21.15.33:0/2628661994 conn(0x55ef037e2950 msgr2=0x55ef03821090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 -- 172.21.15.33:0/2628661994 shutdown_connections 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 -- 172.21.15.33:0/2628661994 wait complete. 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 -- start start 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.884+0000 7fcaaddf4740 1 Processor -- start 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef0382e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef0381fb90 0x55ef0382e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 0x55ef0382f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a67b0 con 0x55ef03824d00 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a7c20 con 0x55ef0381f1a0 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a6e80 con 0x55ef0381fb90 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaaddf4740 10 client.0 _ll_register_callbacks cb 0x7fca90012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 0x55ef0382f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef0382e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef0381fb90 0x55ef0382e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa8e7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef0382e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36544/0 (socket says 172.21.15.33:36544) 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 0x55ef0382f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56492/0 (socket says 172.21.15.33:56492) 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa8e7d640 1 -- 172.21.15.33:0/3450039766 learned_addr learned my addr 172.21.15.33:0/3450039766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 -- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef0381fb90 msgr2=0x55ef0382e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef0381fb90 0x55ef0382e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 -- 172.21.15.33:0/3450039766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 msgr2=0x55ef0382e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef0382e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 -- 172.21.15.33:0/3450039766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0382fc50 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef0382e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fcaa967e640 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 0x55ef0382f660 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009bf0 tx=0x7fca9c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08345abf29c8e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fca83fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.885+0000 7fca83fff640 1 -- 172.21.15.33:0/3450039766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c055060 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9809c3b0 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fca83fff640 1 -- 172.21.15.33:0/3450039766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca9c0307c0 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fca83fff640 1 -- 172.21.15.33:0/3450039766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca9c05c050 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 10 client.16401 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef0389b710 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fca83fff640 1 -- 172.21.15.33:0/3450039766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9809c3b0 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fca83fff640 1 -- 172.21.15.33:0/3450039766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca9c04ca40 con 0x55ef03824d00 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 10 client.16401 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 20 client.16401 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef0389c0b0 0x55ef0389e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 4 client.16401 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:16:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef0378b800 con 0x55ef0389c0b0 2024-04-24T23:16:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.886+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef0389c0b0 0x55ef0389e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.887+0000 7fcaa8e7d640 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef0389c0b0 0x55ef0389e4d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d25f6573a33e33 server_cookie=58e5040f63487cc3 in_seq=0 out_seq=0 2024-04-24T23:16:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.887+0000 7fca83fff640 10 client.16401 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.887+0000 7fca83fff640 1 -- 172.21.15.33:0/3450039766 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef0378b800 con 0x55ef0389c0b0 2024-04-24T23:16:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.887+0000 7fca83fff640 10 client.16401 handle_command_reply: tid=0 2024-04-24T23:16:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.887+0000 7fcaaddf4740 1 client.16401 shutdown 2024-04-24T23:16:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 msgr2=0x55ef0382f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 0x55ef0382f660 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009bf0 tx=0x7fca9c0095c0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 shutdown_connections 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ef0389c0b0 0x55ef0389e4d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef03824d00 0x55ef0382f660 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fca83fff640 0 client.16401 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef0381fb90 0x55ef0382e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 --2- 172.21.15.33:0/3450039766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef0381f1a0 0x55ef0382e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 >> 172.21.15.33:0/3450039766 conn(0x55ef037e2950 msgr2=0x55ef038197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 shutdown_connections 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 1 -- 172.21.15.33:0/3450039766 wait complete. 2024-04-24T23:16:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaaddf4740 20 client.16401 trim_cache size 0 max 16384 2024-04-24T23:16:29.897 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:16:29.897 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:16:29.897 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0e10b59e-2c69-46ba-808d-5787a764914c", 2024-04-24T23:16:29.897 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:16:29.897 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca940a2b90 msgr2=0x7fca940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca940a2b90 0x7fca940a5050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55ef038203f0 tx=0x7fca9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4100680 msgr2=0x7fcaa4197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4100680 0x7fcaa4197a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca9800e9c0 tx=0x7fca9800ee80 comp rx=0 tx=0).stop 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.888+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 shutdown_connections 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca940a2b90 0x7fca940a5050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4102230 0x7fcaa419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa4101870 0x7fcaa4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 --2- 172.21.15.33:0/2935658329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa4100680 0x7fcaa4197a90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 >> 172.21.15.33:0/2935658329 conn(0x7fcaa40fbbf0 msgr2=0x7fcaa40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 shutdown_connections 2024-04-24T23:16:29.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:29.889+0000 7fcaab0e1640 1 -- 172.21.15.33:0/2935658329 wait complete. 2024-04-24T23:16:29.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:16:29.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 -- 172.21.15.33:0/480711565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe41001a0 msgr2=0x7fafe4109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 --2- 172.21.15.33:0/480711565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe41001a0 0x7fafe4109d50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009ca0 tx=0x7fafdc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 -- 172.21.15.33:0/480711565 shutdown_connections 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 --2- 172.21.15.33:0/480711565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe41001a0 0x7fafe4109d50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 --2- 172.21.15.33:0/480711565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe40ff7e0 0x7fafe40ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 --2- 172.21.15.33:0/480711565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe4102d70 0x7fafe4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 -- 172.21.15.33:0/480711565 >> 172.21.15.33:0/480711565 conn(0x7fafe40fb380 msgr2=0x7fafe40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 -- 172.21.15.33:0/480711565 shutdown_connections 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 -- 172.21.15.33:0/480711565 wait complete. 2024-04-24T23:16:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 Processor -- start 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.006+0000 7fafec6f5640 1 -- start start 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe40ff7e0 0x7fafe4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 0x7fafe4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe4102d70 0x7fafe419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafe410d420 con 0x7fafe4102d70 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fafe410d2a0 con 0x7fafe40ff7e0 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafe410d5a0 con 0x7fafe41001a0 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 0x7fafe4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafea491640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe40ff7e0 0x7fafe4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafeac92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe4102d70 0x7fafe419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 0x7fafe4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51168/0 (socket says 172.21.15.33:51168) 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 -- 172.21.15.33:0/3051728298 learned_addr learned my addr 172.21.15.33:0/3051728298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 -- 172.21.15.33:0/3051728298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe40ff7e0 msgr2=0x7fafe4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe40ff7e0 0x7fafe4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 -- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe4102d70 msgr2=0x7fafe419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe4102d70 0x7fafe419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 -- 172.21.15.33:0/3051728298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe419f6b0 con 0x7fafe41001a0 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafeac92640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe4102d70 0x7fafe419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafea491640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe40ff7e0 0x7fafe4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafe9c90640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 0x7fafe4197fd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafd400d770 tx=0x7fafd400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafdb7fe640 1 -- 172.21.15.33:0/3051728298 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4017070 con 0x7fafe41001a0 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe419f9a0 con 0x7fafe41001a0 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.007+0000 7fafdb7fe640 1 -- 172.21.15.33:0/3051728298 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd4005500 con 0x7fafe41001a0 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.008+0000 7fafdb7fe640 1 -- 172.21.15.33:0/3051728298 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4004e00 con 0x7fafe41001a0 2024-04-24T23:16:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.008+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe419fef0 con 0x7fafe41001a0 2024-04-24T23:16:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafdb7fe640 1 -- 172.21.15.33:0/3051728298 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fafd4004fa0 con 0x7fafe41001a0 2024-04-24T23:16:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 Processor -- start 2024-04-24T23:16:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 -- start start 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9c6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563aaa9226b0 con 0x563aaa9bec30 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563aaa8ca880 con 0x563aaa9c3da0 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafef400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563aaa91e670 con 0x563aaa9be240 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafea491640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafe9c90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafdb7fe640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafd00a2b90 0x7fafd00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafeac92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafeac92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9be660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51180/0 (socket says 172.21.15.33:51180) 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafeac92640 1 -- 172.21.15.33:0/717978335 learned_addr learned my addr 172.21.15.33:0/717978335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.010+0000 7fafea491640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56516/0 (socket says 172.21.15.33:56516) 2024-04-24T23:16:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafeac92640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafd00a2b90 0x7fafd00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafdb7fe640 1 -- 172.21.15.33:0/3051728298 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fafd40cee00 con 0x7fafe41001a0 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafea491640 1 -- 172.21.15.33:0/717978335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 msgr2=0x563aaa9be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafea491640 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9be660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafea491640 1 -- 172.21.15.33:0/717978335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 msgr2=0x563aaa9c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafea491640 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafea491640 1 -- 172.21.15.33:0/717978335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aaa7007d0 con 0x563aaa9bec30 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafeac92640 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafea491640 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c3730 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009fa0 tx=0x7fafe00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364acfe6fa0408c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafeac92640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafd00a2b90 0x7fafd00a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x563aaa9bf490 tx=0x7fafdc02a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafd8ff9640 1 -- 172.21.15.33:0/717978335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0004310 con 0x563aaa9bec30 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafd8ff9640 1 -- 172.21.15.33:0/717978335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe00044b0 con 0x563aaa9bec30 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.011+0000 7fafd8ff9640 1 -- 172.21.15.33:0/717978335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe00035f0 con 0x563aaa9bec30 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 -- 172.21.15.33:0/717978335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 msgr2=0x563aaa9c3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c3730 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009fa0 tx=0x7fafe00118d0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 -- 172.21.15.33:0/717978335 shutdown_connections 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c3730 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 --2- 172.21.15.33:0/717978335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 -- 172.21.15.33:0/717978335 >> 172.21.15.33:0/717978335 conn(0x563aaa9b98b0 msgr2=0x563aaa9b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 -- 172.21.15.33:0/717978335 shutdown_connections 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 -- 172.21.15.33:0/717978335 wait complete. 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 -- start start 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.012+0000 7fafef400740 1 Processor -- start 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563aaa9c6d50 con 0x563aaa9bec30 2024-04-24T23:16:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563aaa8ddf50 con 0x563aaa9c3da0 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563aaa9c6ed0 con 0x563aaa9be240 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 10 client.0 _ll_register_callbacks cb 0x563aaa919000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51192/0 (socket says 172.21.15.33:51192) 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 -- 172.21.15.33:0/1841448439 learned_addr learned my addr 172.21.15.33:0/1841448439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafeac92640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafe9c90640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 -- 172.21.15.33:0/1841448439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 msgr2=0x563aaa9cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 -- 172.21.15.33:0/1841448439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 msgr2=0x563aaa9c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 -- 172.21.15.33:0/1841448439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aaa6363f0 con 0x563aaa9be240 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafea491640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9c86f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafe001b040 tx=0x7fafe0009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad1430575cae5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafcdffb640 1 -- 172.21.15.33:0/1841448439 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0028070 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe00044b0 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.013+0000 7fafcdffb640 1 -- 172.21.15.33:0/1841448439 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fafe0005000 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafcdffb640 1 -- 172.21.15.33:0/1841448439 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fafe0034050 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 10 client.25966 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aaa5f60f0 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafcdffb640 1 -- 172.21.15.33:0/1841448439 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe00044b0 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafcdffb640 1 -- 172.21.15.33:0/1841448439 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fafe000f4e0 con 0x563aaa9be240 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 10 client.25966 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 20 client.25966 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaa9d4280 0x563aaa9d6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 4 client.25966 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aaa92fbf0 con 0x563aaa9d4280 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.014+0000 7fafe9c90640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaa9d4280 0x563aaa9d6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafcdffb640 10 client.25966 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafe9c90640 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaa9d4280 0x563aaa9d6720 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873acb002ce20441 server_cookie=b48a0753d1a8e0a9 in_seq=0 out_seq=0 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafcdffb640 1 -- 172.21.15.33:0/1841448439 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563aaa92fbf0 con 0x563aaa9d4280 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafcdffb640 10 client.25966 handle_command_reply: tid=0 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 client.25966 shutdown 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 msgr2=0x563aaa9c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9c86f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafe001b040 tx=0x7fafe0009d00 comp rx=0 tx=0).stop 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 shutdown_connections 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaa9d4280 0x563aaa9d6720 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaa9cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 --2- 172.21.15.33:0/1841448439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9c86f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafcdffb640 0 client.25966 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 >> 172.21.15.33:0/1841448439 conn(0x563aaa9b98b0 msgr2=0x563aaa9b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 shutdown_connections 2024-04-24T23:16:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 1 -- 172.21.15.33:0/1841448439 wait complete. 2024-04-24T23:16:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.015+0000 7fafef400740 20 client.25966 trim_cache size 0 max 16384 2024-04-24T23:16:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.033+0000 7fafef400740 1 Processor -- start 2024-04-24T23:16:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.033+0000 7fafef400740 1 -- start start 2024-04-24T23:16:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafef400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafef400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9bec30 0x563aaa9cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafef400740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 0x563aaa9c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafef400740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafe41acc50 con 0x563aaa9c3da0 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafef400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fafe41aa520 con 0x563aaa9bec30 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafef400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafe41a8370 con 0x563aaa9be240 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 0x563aaa9c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 0x563aaa9c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56528/0 (socket says 172.21.15.33:56528) 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 -- 172.21.15.33:0/763020360 learned_addr learned my addr 172.21.15.33:0/763020360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafe9c90640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9bec30 0x563aaa9cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 -- 172.21.15.33:0/763020360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 msgr2=0x563aaa9bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafea491640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 -- 172.21.15.33:0/763020360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9bec30 msgr2=0x563aaa9cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9bec30 0x563aaa9cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 -- 172.21.15.33:0/763020360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aaa9ba620 con 0x563aaa9c3da0 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafe9c90640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9bec30 0x563aaa9cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:30.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafea491640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafeac92640 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 0x563aaa9c7d10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009ca0 tx=0x7fafdc005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da613572de8d4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafcffff640 1 -- 172.21.15.33:0/763020360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0307c0 con 0x563aaa9c3da0 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafcffff640 1 -- 172.21.15.33:0/763020360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafdc036040 con 0x563aaa9c3da0 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.034+0000 7fafcffff640 1 -- 172.21.15.33:0/763020360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc04cdf0 con 0x563aaa9c3da0 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 -- 172.21.15.33:0/763020360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 msgr2=0x563aaa9c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 0x563aaa9c7d10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009ca0 tx=0x7fafdc005df0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 -- 172.21.15.33:0/763020360 shutdown_connections 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9c3da0 0x563aaa9c7d10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9bec30 0x563aaa9cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 --2- 172.21.15.33:0/763020360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 -- 172.21.15.33:0/763020360 >> 172.21.15.33:0/763020360 conn(0x563aaa9b98b0 msgr2=0x563aaa9bc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 -- 172.21.15.33:0/763020360 shutdown_connections 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 -- 172.21.15.33:0/763020360 wait complete. 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 -- start start 2024-04-24T23:16:30.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.035+0000 7fafef400740 1 Processor -- start 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaaa38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafe41a89e0 con 0x563aaa9bec30 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fafe41a9e50 con 0x563aaa9c3da0 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafe41a90b0 con 0x563aaa9be240 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 10 client.0 _ll_register_callbacks cb 0x7fafd409b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafea491640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56538/0 (socket says 172.21.15.33:56538) 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 -- 172.21.15.33:0/817537865 learned_addr learned my addr 172.21.15.33:0/817537865 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafeac92640 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaaa38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 -- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 msgr2=0x563aaa9ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 -- 172.21.15.33:0/817537865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 msgr2=0x563aaaa38850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaaa38850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 -- 172.21.15.33:0/817537865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aaaa38d90 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafe9c90640 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9cd020 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fafd409adf0 tx=0x7fafd4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3880b0082f47f937 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafcdffb640 1 -- 172.21.15.33:0/817537865 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4084070 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafdc036040 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.036+0000 7fafcdffb640 1 -- 172.21.15.33:0/817537865 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fafd40cc620 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafcdffb640 1 -- 172.21.15.33:0/817537865 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fafd4020040 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 10 client.16419 fetch_fsmap learned FSMap version 27 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aaaa39f20 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafcdffb640 1 -- 172.21.15.33:0/817537865 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafdc036040 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafcdffb640 1 -- 172.21.15.33:0/817537865 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fafd40bb0b0 con 0x563aaa9bec30 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 10 client.16419 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 20 client.16419 populate_metadata read hostname 'smithi033' 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaaa3a8b0 0x563aaaa3ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 4 client.16419 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563aaa929f40 con 0x563aaaa3a8b0 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafea491640 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaaa3a8b0 0x563aaaa3ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafea491640 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaaa3a8b0 0x563aaaa3ccd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3024ad4b2f482d5c server_cookie=15f16428de0b9c8 in_seq=0 out_seq=0 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.037+0000 7fafcdffb640 10 client.16419 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafcdffb640 1 -- 172.21.15.33:0/817537865 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563aaa929f40 con 0x563aaaa3a8b0 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafcdffb640 10 client.16419 handle_command_reply: tid=0 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 client.16419 shutdown 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 msgr2=0x563aaa9cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9cd020 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fafd409adf0 tx=0x7fafd4009c30 comp rx=0 tx=0).stop 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 shutdown_connections 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563aaaa3a8b0 0x563aaaa3ccd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563aaa9c3da0 0x563aaaa38850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563aaa9bec30 0x563aaa9cd020 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 --2- 172.21.15.33:0/817537865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563aaa9be240 0x563aaa9ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 >> 172.21.15.33:0/817537865 conn(0x563aaa9b98b0 msgr2=0x563aaa9b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafcdffb640 0 client.16419 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 shutdown_connections 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 1 -- 172.21.15.33:0/817537865 wait complete. 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafef400740 20 client.16419 trim_cache size 0 max 16384 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "0e10b59e-2c69-46ba-808d-5787a764914c": { 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0e10b59e-2c69-46ba-808d-5787a764914c", 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:16:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafd00a2b90 msgr2=0x7fafd00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafec6f5640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafd00a2b90 0x7fafd00a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x563aaa9bf490 tx=0x7fafdc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 msgr2=0x7fafe4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafec6f5640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 0x7fafe4197fd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafd400d770 tx=0x7fafd400dc30 comp rx=0 tx=0).stop 2024-04-24T23:16:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 shutdown_connections 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.038+0000 7fafec6f5640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fafd00a2b90 0x7fafd00a5050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.039+0000 7fafec6f5640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe4102d70 0x7fafe419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.039+0000 7fafec6f5640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafe41001a0 0x7fafe4197fd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.039+0000 7fafec6f5640 1 --2- 172.21.15.33:0/3051728298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe40ff7e0 0x7fafe4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.039+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 >> 172.21.15.33:0/3051728298 conn(0x7fafe40fb380 msgr2=0x7fafe4105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.039+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 shutdown_connections 2024-04-24T23:16:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:16:30.039+0000 7fafec6f5640 1 -- 172.21.15.33:0/3051728298 wait complete. 2024-04-24T23:16:30.051 INFO:tasks.cephfs.filesystem:scrub status for tag:0e10b59e-2c69-46ba-808d-5787a764914c - {'path': '/', 'tag': '0e10b59e-2c69-46ba-808d-5787a764914c', 'options': 'recursive,force'} 2024-04-24T23:16:30.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 380300 12.58 MB/sec execute 35 sec latency 21.787 ms 2024-04-24T23:16:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:30 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:16:30.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:16:30.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:16:30.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:30 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:16:30.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:16:30.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:16:30.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:30 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:16:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:16:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:16:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:30.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:31.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 383333 12.75 MB/sec execute 36 sec latency 11.680 ms 2024-04-24T23:16:31.606 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:31 smithi033 ceph-mon[29201]: pgmap v486: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-04-24T23:16:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:31 smithi038 ceph-mon[32450]: pgmap v486: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-04-24T23:16:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:31 smithi117 ceph-mon[32529]: pgmap v486: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-04-24T23:16:32.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 385498 12.60 MB/sec execute 37 sec latency 22.508 ms 2024-04-24T23:16:33.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 387431 12.60 MB/sec execute 38 sec latency 25.316 ms 2024-04-24T23:16:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:33 smithi038 ceph-mon[32450]: pgmap v487: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 320 op/s 2024-04-24T23:16:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:33 smithi117 ceph-mon[32529]: pgmap v487: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 320 op/s 2024-04-24T23:16:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:33 smithi033 ceph-mon[29201]: pgmap v487: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 320 op/s 2024-04-24T23:16:34.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 390202 12.71 MB/sec execute 39 sec latency 87.296 ms 2024-04-24T23:16:35.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 391810 12.57 MB/sec execute 40 sec latency 28.382 ms 2024-04-24T23:16:35.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:35 smithi038 ceph-mon[32450]: pgmap v488: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-04-24T23:16:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:35 smithi117 ceph-mon[32529]: pgmap v488: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-04-24T23:16:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:35 smithi033 ceph-mon[29201]: pgmap v488: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-04-24T23:16:36.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 394364 12.60 MB/sec execute 41 sec latency 16.024 ms 2024-04-24T23:16:37.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 397143 12.65 MB/sec execute 42 sec latency 22.653 ms 2024-04-24T23:16:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:37 smithi038 ceph-mon[32450]: pgmap v489: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-04-24T23:16:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:37 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:16:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:37 smithi117 ceph-mon[32529]: pgmap v489: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-04-24T23:16:37.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:37 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:16:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:37 smithi033 ceph-mon[29201]: pgmap v489: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-04-24T23:16:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:37 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:16:38.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 398977 12.58 MB/sec execute 43 sec latency 12.931 ms 2024-04-24T23:16:38.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:38 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:16:38.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:38 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:16:38.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:38 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:16:39.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 401436 12.61 MB/sec execute 44 sec latency 17.377 ms 2024-04-24T23:16:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:39 smithi038 ceph-mon[32450]: pgmap v490: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-04-24T23:16:39.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:39 smithi117 ceph-mon[32529]: pgmap v490: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-04-24T23:16:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:39 smithi033 ceph-mon[29201]: pgmap v490: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-04-24T23:16:40.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 403786 12.61 MB/sec execute 45 sec latency 39.507 ms 2024-04-24T23:16:41.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 405810 12.59 MB/sec execute 46 sec latency 12.932 ms 2024-04-24T23:16:41.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:41 smithi038 ceph-mon[32450]: pgmap v491: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-04-24T23:16:41.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:41 smithi117 ceph-mon[32529]: pgmap v491: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-04-24T23:16:41.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:41 smithi033 ceph-mon[29201]: pgmap v491: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-04-24T23:16:42.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 408527 12.62 MB/sec execute 47 sec latency 16.646 ms 2024-04-24T23:16:43.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 411484 12.72 MB/sec execute 48 sec latency 21.217 ms 2024-04-24T23:16:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:43 smithi038 ceph-mon[32450]: pgmap v492: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2024-04-24T23:16:43.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:43 smithi117 ceph-mon[32529]: pgmap v492: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2024-04-24T23:16:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:43 smithi033 ceph-mon[29201]: pgmap v492: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2024-04-24T23:16:44.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 413871 12.64 MB/sec execute 49 sec latency 12.287 ms 2024-04-24T23:16:45.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 415596 12.63 MB/sec execute 50 sec latency 60.729 ms 2024-04-24T23:16:45.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:45 smithi038 ceph-mon[32450]: pgmap v493: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-24T23:16:45.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:45 smithi117 ceph-mon[32529]: pgmap v493: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-24T23:16:45.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:45 smithi033 ceph-mon[29201]: pgmap v493: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-24T23:16:46.442 INFO:tasks.workunit.client.0.smithi033.stdout: 1 418546 12.72 MB/sec execute 51 sec latency 22.691 ms 2024-04-24T23:16:47.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 420347 12.61 MB/sec execute 52 sec latency 16.388 ms 2024-04-24T23:16:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:47 smithi038 ceph-mon[32450]: pgmap v494: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T23:16:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:47 smithi117 ceph-mon[32529]: pgmap v494: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T23:16:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:47 smithi033 ceph-mon[29201]: pgmap v494: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T23:16:48.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 422842 12.64 MB/sec execute 53 sec latency 19.790 ms 2024-04-24T23:16:49.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 425785 12.73 MB/sec execute 54 sec latency 8.648 ms 2024-04-24T23:16:49.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:49 smithi038 ceph-mon[32450]: pgmap v495: 129 pgs: 129 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-04-24T23:16:49.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:49 smithi117 ceph-mon[32529]: pgmap v495: 129 pgs: 129 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-04-24T23:16:49.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:49 smithi033 ceph-mon[29201]: pgmap v495: 129 pgs: 129 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-04-24T23:16:50.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 426840 12.56 MB/sec execute 55 sec latency 71.199 ms 2024-04-24T23:16:51.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 429622 12.64 MB/sec execute 56 sec latency 9.627 ms 2024-04-24T23:16:51.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:51 smithi038 ceph-mon[32450]: pgmap v496: 129 pgs: 129 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 226 op/s 2024-04-24T23:16:51.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:51 smithi117 ceph-mon[32529]: pgmap v496: 129 pgs: 129 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 226 op/s 2024-04-24T23:16:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:51 smithi033 ceph-mon[29201]: pgmap v496: 129 pgs: 129 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 226 op/s 2024-04-24T23:16:52.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 432092 12.62 MB/sec execute 57 sec latency 14.736 ms 2024-04-24T23:16:53.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 433859 12.57 MB/sec execute 58 sec latency 23.373 ms 2024-04-24T23:16:53.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:53 smithi038 ceph-mon[32450]: pgmap v497: 129 pgs: 129 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 316 op/s 2024-04-24T23:16:53.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:53 smithi117 ceph-mon[32529]: pgmap v497: 129 pgs: 129 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 316 op/s 2024-04-24T23:16:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:53 smithi033 ceph-mon[29201]: pgmap v497: 129 pgs: 129 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 316 op/s 2024-04-24T23:16:54.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 436727 12.64 MB/sec execute 59 sec latency 12.434 ms 2024-04-24T23:16:55.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 438769 12.55 MB/sec execute 60 sec latency 103.374 ms 2024-04-24T23:16:55.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:55 smithi038 ceph-mon[32450]: pgmap v498: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-04-24T23:16:55.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:55 smithi117 ceph-mon[32529]: pgmap v498: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-04-24T23:16:55.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:55 smithi033 ceph-mon[29201]: pgmap v498: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-04-24T23:16:56.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 441062 12.58 MB/sec execute 61 sec latency 12.702 ms 2024-04-24T23:16:57.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 443965 12.65 MB/sec execute 62 sec latency 12.911 ms 2024-04-24T23:16:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:57 smithi038 ceph-mon[32450]: pgmap v499: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-04-24T23:16:57.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:57 smithi117 ceph-mon[32529]: pgmap v499: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-04-24T23:16:57.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:57 smithi033 ceph-mon[29201]: pgmap v499: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-04-24T23:16:58.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 446960 12.73 MB/sec execute 63 sec latency 13.071 ms 2024-04-24T23:16:58.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:58 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:58.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:58 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:58.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:58 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:59.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 448792 12.64 MB/sec execute 64 sec latency 21.473 ms 2024-04-24T23:16:59.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:59 smithi038 ceph-mon[32450]: pgmap v500: 129 pgs: 129 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 253 op/s 2024-04-24T23:16:59.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:59.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:59.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:16:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:16:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:59 smithi117 ceph-mon[32529]: pgmap v500: 129 pgs: 129 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 253 op/s 2024-04-24T23:16:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:16:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:16:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:59 smithi033 ceph-mon[29201]: pgmap v500: 129 pgs: 129 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 253 op/s 2024-04-24T23:16:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:59.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:16:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:17:00.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:00.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 -- 172.21.15.33:0/2356504918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c100680 msgr2=0x7f319c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 --2- 172.21.15.33:0/2356504918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c100680 0x7f319c100aa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f318c009900 tx=0x7f318c02a410 comp rx=0 tx=0).stop 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 -- 172.21.15.33:0/2356504918 shutdown_connections 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 --2- 172.21.15.33:0/2356504918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f319c102230 0x7f319c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 --2- 172.21.15.33:0/2356504918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c101870 0x7f319c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 --2- 172.21.15.33:0/2356504918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c100680 0x7f319c100aa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 -- 172.21.15.33:0/2356504918 >> 172.21.15.33:0/2356504918 conn(0x7f319c0fbbf0 msgr2=0x7f319c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 -- 172.21.15.33:0/2356504918 shutdown_connections 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.159+0000 7f31a2363640 1 -- 172.21.15.33:0/2356504918 wait complete. 2024-04-24T23:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 Processor -- start 2024-04-24T23:17:00.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 -- start start 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 0x7f319c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f319c101870 0x7f319c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c102230 0x7f319c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f319c10c340 con 0x7f319c102230 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.160+0000 7f31a2363640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f319c10c1c0 con 0x7f319c101870 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31a2363640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f319c10c4c0 con 0x7f319c100680 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 0x7f319c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31a0900640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c102230 0x7f319c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 0x7f319c197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39062/0 (socket says 172.21.15.33:39062) 2024-04-24T23:17:00.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f319c101870 0x7f319c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 -- 172.21.15.33:0/1090732317 learned_addr learned my addr 172.21.15.33:0/1090732317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 -- 172.21.15.33:0/1090732317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f319c101870 msgr2=0x7f319c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f319c101870 0x7f319c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 -- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c102230 msgr2=0x7f319c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c102230 0x7f319c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 -- 172.21.15.33:0/1090732317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c19f7a0 con 0x7f319c100680 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31a0900640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c102230 0x7f319c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f319bfff640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 0x7f319c197bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f318c031440 tx=0x7f318c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31997fa640 1 -- 172.21.15.33:0/1090732317 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c03f030 con 0x7f319c100680 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c19fa30 con 0x7f319c100680 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31997fa640 1 -- 172.21.15.33:0/1090732317 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c003ce0 con 0x7f319c100680 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31997fa640 1 -- 172.21.15.33:0/1090732317 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c035070 con 0x7f319c100680 2024-04-24T23:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.161+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c19ff70 con 0x7f319c100680 2024-04-24T23:17:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31997fa640 1 -- 172.21.15.33:0/1090732317 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f318c042030 con 0x7f319c100680 2024-04-24T23:17:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 Processor -- start 2024-04-24T23:17:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 -- start start 2024-04-24T23:17:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182348b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348bc30 0x561823490730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823493230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618233ef6b0 con 0x561823490da0 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561823397880 con 0x56182348bc30 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a5079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618233eb670 con 0x56182348b240 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823493230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f31a0900640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182348b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823493230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48284/0 (socket says 172.21.15.33:48284) 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f319b7fe640 1 -- 172.21.15.33:0/4029648665 learned_addr learned my addr 172.21.15.33:0/4029648665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.164+0000 7f319bfff640 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348bc30 0x561823490730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f31997fa640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f31840a2b90 0x7f31840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f319b7fe640 1 -- 172.21.15.33:0/4029648665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 msgr2=0x56182348b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f319b7fe640 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182348b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f319b7fe640 1 -- 172.21.15.33:0/4029648665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348bc30 msgr2=0x561823490730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f319b7fe640 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348bc30 0x561823490730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f319b7fe640 1 -- 172.21.15.33:0/4029648665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618231cd7d0 con 0x561823490da0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f31997fa640 1 -- 172.21.15.33:0/1090732317 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f318c0ef740 con 0x7f319c100680 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f31a0900640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f31840a2b90 0x7f31840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f319b7fe640 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823493230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3190009ca0 tx=0x7f3190011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e546c1f17bcd760 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f31827fc640 1 -- 172.21.15.33:0/4029648665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319001e070 con 0x561823490da0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f31827fc640 1 -- 172.21.15.33:0/4029648665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190019b10 con 0x561823490da0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.165+0000 7f31827fc640 1 -- 172.21.15.33:0/4029648665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319000ea30 con 0x561823490da0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 -- 172.21.15.33:0/4029648665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 msgr2=0x561823493230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823493230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3190009ca0 tx=0x7f3190011810 comp rx=0 tx=0).stop 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a0900640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f31840a2b90 0x7f31840a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x56182348c490 tx=0x7f3194004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 -- 172.21.15.33:0/4029648665 shutdown_connections 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823493230 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348bc30 0x561823490730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 --2- 172.21.15.33:0/4029648665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182348b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 -- 172.21.15.33:0/4029648665 >> 172.21.15.33:0/4029648665 conn(0x5618234868b0 msgr2=0x561823486cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 -- 172.21.15.33:0/4029648665 shutdown_connections 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 -- 172.21.15.33:0/4029648665 wait complete. 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 -- start start 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.166+0000 7f31a5079740 1 Processor -- start 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 0x5618234956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x561823495be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561823493d50 con 0x56182348b240 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618233aaf50 con 0x561823490da0 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561823493ed0 con 0x56182348bc30 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 10 client.0 _ll_register_callbacks cb 0x5618233e6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 0x5618234956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 0x5618234956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48292/0 (socket says 172.21.15.33:48292) 2024-04-24T23:17:00.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 -- 172.21.15.33:0/1252615507 learned_addr learned my addr 172.21.15.33:0/1252615507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a0900640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319b7fe640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x561823495be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 -- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 msgr2=0x561823495be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x561823495be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 -- 172.21.15.33:0/1252615507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 msgr2=0x56182349cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 -- 172.21.15.33:0/1252615507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618231033f0 con 0x56182348b240 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f319bfff640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 0x5618234956a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f318c0bbbf0 tx=0x7f318c0ba170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aae902d0a3e179f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f317b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f317b7fe640 1 -- 172.21.15.33:0/1252615507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0f5020 con 0x56182348b240 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f317b7fe640 1 -- 172.21.15.33:0/1252615507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f318c003800 con 0x56182348b240 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.167+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3190019b10 con 0x56182348b240 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f317b7fe640 1 -- 172.21.15.33:0/1252615507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f318c049070 con 0x56182348b240 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 10 client.16431 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618230c30f0 con 0x56182348b240 2024-04-24T23:17:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f317b7fe640 1 -- 172.21.15.33:0/1252615507 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190019b10 con 0x56182348b240 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f317b7fe640 1 -- 172.21.15.33:0/1252615507 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f318c0b82d0 con 0x56182348b240 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 10 client.16431 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 20 client.16431 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618234a14a0 0x5618234a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 4 client.16431 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618233fcbf0 con 0x5618234a14a0 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.168+0000 7f319b7fe640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618234a14a0 0x5618234a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.169+0000 7f317b7fe640 10 client.16431 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.169+0000 7f319b7fe640 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618234a14a0 0x5618234a3930 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756e1b06c9aa25b9 server_cookie=977d2b445754f63c in_seq=0 out_seq=0 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.169+0000 7f317b7fe640 1 -- 172.21.15.33:0/1252615507 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618233fcbf0 con 0x5618234a14a0 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.169+0000 7f317b7fe640 10 client.16431 handle_command_reply: tid=0 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.169+0000 7f31a5079740 1 client.16431 shutdown 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 msgr2=0x5618234956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 0x5618234956a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f318c0bbbf0 tx=0x7f318c0ba170 comp rx=0 tx=0).stop 2024-04-24T23:17:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 shutdown_connections 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5618234a14a0 0x5618234a3930 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x561823495be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f317b7fe640 0 client.16431 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 --2- 172.21.15.33:0/1252615507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348b240 0x5618234956a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 >> 172.21.15.33:0/1252615507 conn(0x5618234868b0 msgr2=0x561823485af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 shutdown_connections 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 1 -- 172.21.15.33:0/1252615507 wait complete. 2024-04-24T23:17:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.170+0000 7f31a5079740 20 client.16431 trim_cache size 0 max 16384 2024-04-24T23:17:00.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 Processor -- start 2024-04-24T23:17:00.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 -- start start 2024-04-24T23:17:00.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348b240 0x561823495b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x56182349cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823494990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1acd10 con 0x561823490da0 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1aa5e0 con 0x56182348b240 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a5079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1a8430 con 0x56182348bc30 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823494990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823494990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48298/0 (socket says 172.21.15.33:48298) 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 -- 172.21.15.33:0/2619066694 learned_addr learned my addr 172.21.15.33:0/2619066694 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f319b7fe640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x56182349cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f319bfff640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348b240 0x561823495b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 -- 172.21.15.33:0/2619066694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 msgr2=0x56182349cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x56182349cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 -- 172.21.15.33:0/2619066694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348b240 msgr2=0x561823495b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348b240 0x561823495b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f31a0900640 1 -- 172.21.15.33:0/2619066694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561823478090 con 0x561823490da0 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f319b7fe640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x56182349cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.189+0000 7f319bfff640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348b240 0x561823495b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.190+0000 7f31a0900640 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823494990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f31940119e0 tx=0x7f319401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb537d64ff504f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.190+0000 7f3181ffb640 1 -- 172.21.15.33:0/2619066694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004b20 con 0x561823490da0 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.190+0000 7f3181ffb640 1 -- 172.21.15.33:0/2619066694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3194004cc0 con 0x561823490da0 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.190+0000 7f3181ffb640 1 -- 172.21.15.33:0/2619066694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194005840 con 0x561823490da0 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- 172.21.15.33:0/2619066694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 msgr2=0x561823494990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823494990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f31940119e0 tx=0x7f319401a000 comp rx=0 tx=0).stop 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- 172.21.15.33:0/2619066694 shutdown_connections 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561823490da0 0x561823494990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348bc30 0x56182349cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- 172.21.15.33:0/2619066694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56182348b240 0x561823495b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- 172.21.15.33:0/2619066694 >> 172.21.15.33:0/2619066694 conn(0x5618234868b0 msgr2=0x561823485260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- 172.21.15.33:0/2619066694 shutdown_connections 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- 172.21.15.33:0/2619066694 wait complete. 2024-04-24T23:17:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- start start 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 Processor -- start 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182349a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 0x56182349aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1a8aa0 con 0x56182348bc30 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1a9f10 con 0x561823490da0 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1a9170 con 0x56182348b240 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.191+0000 7f31a5079740 10 client.0 _ll_register_callbacks cb 0x7f318c0f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 0x56182349aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182349a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 0x56182349aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48308/0 (socket says 172.21.15.33:48308) 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 -- 172.21.15.33:0/1891746797 learned_addr learned my addr 172.21.15.33:0/1891746797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f31a0900640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 -- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 msgr2=0x56182349a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182349a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 -- 172.21.15.33:0/1891746797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 msgr2=0x56182349ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 -- 172.21.15.33:0/1891746797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182349c070 con 0x56182348bc30 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319bfff640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182349a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f319b7fe640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 0x56182349aa70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3190009f50 tx=0x7f3190009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d916687f380a1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f317b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f317b7fe640 1 -- 172.21.15.33:0/1891746797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319001e040 con 0x56182348bc30 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f317b7fe640 1 -- 172.21.15.33:0/1891746797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3190005240 con 0x56182348bc30 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3194004cc0 con 0x56182348bc30 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f317b7fe640 1 -- 172.21.15.33:0/1891746797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3190005440 con 0x56182348bc30 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f31a5079740 10 client.16437 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.192+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561823510500 con 0x56182348bc30 2024-04-24T23:17:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f317b7fe640 1 -- 172.21.15.33:0/1891746797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3194004cc0 con 0x56182348bc30 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f317b7fe640 1 -- 172.21.15.33:0/1891746797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f319002f580 con 0x56182348bc30 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 10 client.16437 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 20 client.16437 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561823510d60 0x561823513180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 4 client.16437 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618233f6f40 con 0x561823510d60 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f319bfff640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561823510d60 0x561823513180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f319bfff640 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561823510d60 0x561823513180 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5eef08e7c5719a server_cookie=e0fdc8a1670a1a75 in_seq=0 out_seq=0 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f317b7fe640 10 client.16437 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f317b7fe640 1 -- 172.21.15.33:0/1891746797 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5618233f6f40 con 0x561823510d60 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f317b7fe640 10 client.16437 handle_command_reply: tid=0 2024-04-24T23:17:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.193+0000 7f31a5079740 1 client.16437 shutdown 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 msgr2=0x56182349aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 0x56182349aa70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3190009f50 tx=0x7f3190009c00 comp rx=0 tx=0).stop 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 shutdown_connections 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561823510d60 0x561823513180 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561823490da0 0x56182349ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f317b7fe640 0 client.16437 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56182348bc30 0x56182349aa70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 --2- 172.21.15.33:0/1891746797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182348b240 0x56182349a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 >> 172.21.15.33:0/1891746797 conn(0x5618234868b0 msgr2=0x561823485af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 shutdown_connections 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 1 -- 172.21.15.33:0/1891746797 wait complete. 2024-04-24T23:17:00.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a5079740 20 client.16437 trim_cache size 0 max 16384 2024-04-24T23:17:00.206 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:17:00.206 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:17:00.206 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:17:00.206 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f31840a2b90 msgr2=0x7f31840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f31840a2b90 0x7f31840a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x56182348c490 tx=0x7f3194004a40 comp rx=0 tx=0).stop 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 msgr2=0x7f319c197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 0x7f319c197bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f318c031440 tx=0x7f318c032ce0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 shutdown_connections 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f31840a2b90 0x7f31840a5050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c102230 0x7f319c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f319c101870 0x7f319c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 --2- 172.21.15.33:0/1090732317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f319c100680 0x7f319c197bf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.194+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 >> 172.21.15.33:0/1090732317 conn(0x7f319c0fbbf0 msgr2=0x7f319c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.195+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 shutdown_connections 2024-04-24T23:17:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.195+0000 7f31a2363640 1 -- 172.21.15.33:0/1090732317 wait complete. 2024-04-24T23:17:00.208 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:17:00.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:17:00.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.345+0000 7f1574b43640 1 -- 172.21.15.33:0/484911839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568095e70 msgr2=0x7f15680962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.345+0000 7f1574b43640 1 --2- 172.21.15.33:0/484911839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568095e70 0x7f15680962f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b9f0 tx=0x7f155c02c0b0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.346+0000 7f1574b43640 1 -- 172.21.15.33:0/484911839 shutdown_connections 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.346+0000 7f1574b43640 1 --2- 172.21.15.33:0/484911839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1568096830 0x7f156809d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.346+0000 7f1574b43640 1 --2- 172.21.15.33:0/484911839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568095e70 0x7f15680962f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.346+0000 7f1574b43640 1 --2- 172.21.15.33:0/484911839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568094c80 0x7f15680950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.346+0000 7f1574b43640 1 -- 172.21.15.33:0/484911839 >> 172.21.15.33:0/484911839 conn(0x7f15680901d0 msgr2=0x7f1568092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.347+0000 7f1574b43640 1 -- 172.21.15.33:0/484911839 shutdown_connections 2024-04-24T23:17:00.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.347+0000 7f1574b43640 1 -- 172.21.15.33:0/484911839 wait complete. 2024-04-24T23:17:00.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.348+0000 7f1574b43640 1 Processor -- start 2024-04-24T23:17:00.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.349+0000 7f1574b43640 1 -- start start 2024-04-24T23:17:00.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.349+0000 7f1574b43640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 0x7f156812e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f1574b43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1568096830 0x7f156812e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f156f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 0x7f156812e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f156f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 0x7f156812e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39108/0 (socket says 172.21.15.33:39108) 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f1574b43640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568133a50 0x7f1568135e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f156f7fe640 1 -- 172.21.15.33:0/3839116346 learned_addr learned my addr 172.21.15.33:0/3839116346 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15680a3c10 con 0x7f1568133a50 2024-04-24T23:17:00.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15680a3a90 con 0x7f1568096830 2024-04-24T23:17:00.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15680a3d90 con 0x7f1568094c80 2024-04-24T23:17:00.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.350+0000 7f156ffff640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568133a50 0x7f1568135e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156effd640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1568096830 0x7f156812e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156f7fe640 1 -- 172.21.15.33:0/3839116346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1568096830 msgr2=0x7f156812e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156f7fe640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1568096830 0x7f156812e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156f7fe640 1 -- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568133a50 msgr2=0x7f1568135e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156f7fe640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568133a50 0x7f1568135e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156f7fe640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681364b0 con 0x7f1568094c80 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.351+0000 7f156f7fe640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 0x7f156812e4b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f156400e9f0 tx=0x7f156400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.352+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400ccc0 con 0x7f1568094c80 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.352+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15640035f0 con 0x7f1568094c80 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.352+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400ce60 con 0x7f1568094c80 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.352+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15681367a0 con 0x7f1568094c80 2024-04-24T23:17:00.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.352+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1568136c70 con 0x7f1568094c80 2024-04-24T23:17:00.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.354+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15680061c0 con 0x7f1568094c80 2024-04-24T23:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.356+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f15640037a0 con 0x7f1568094c80 2024-04-24T23:17:00.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.357+0000 7f156cff9640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15600a2d60 0x7f15600a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.358+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15640cf600 con 0x7f1568094c80 2024-04-24T23:17:00.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.359+0000 7f156effd640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15600a2d60 0x7f15600a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.360+0000 7f156effd640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15600a2d60 0x7f15600a5220 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f155c0094e0 tx=0x7f155c0095b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.365+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f156409c170 con 0x7f1568094c80 2024-04-24T23:17:00.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 450883 12.65 MB/sec execute 65 sec latency 53.746 ms 2024-04-24T23:17:00.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.625+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f156809c020 con 0x7f1568094c80 2024-04-24T23:17:00.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.626+0000 7f156cff9640 1 -- 172.21.15.33:0/3839116346 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f156409ba70 con 0x7f1568094c80 2024-04-24T23:17:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:17:00.628 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:17:00.629 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:17:00.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15600a2d60 msgr2=0x7f15600a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15600a2d60 0x7f15600a5220 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f155c0094e0 tx=0x7f155c0095b0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 msgr2=0x7f156812e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 0x7f156812e4b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f156400e9f0 tx=0x7f156400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 shutdown_connections 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15600a2d60 0x7f15600a5220 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1568133a50 0x7f1568135e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1568096830 0x7f156812e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 --2- 172.21.15.33:0/3839116346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1568094c80 0x7f156812e4b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 >> 172.21.15.33:0/3839116346 conn(0x7f15680901d0 msgr2=0x7f15680991f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 shutdown_connections 2024-04-24T23:17:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.628+0000 7f1574b43640 1 -- 172.21.15.33:0/3839116346 wait complete. 2024-04-24T23:17:00.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 -- 172.21.15.33:0/2347216713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc103340 msgr2=0x7fbfbc109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 --2- 172.21.15.33:0/2347216713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc103340 0x7fbfbc109bd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009ca0 tx=0x7fbfb402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 -- 172.21.15.33:0/2347216713 shutdown_connections 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 --2- 172.21.15.33:0/2347216713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc103340 0x7fbfbc109bd0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 --2- 172.21.15.33:0/2347216713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 0x7fbfbc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 --2- 172.21.15.33:0/2347216713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc101790 0x7fbfbc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 -- 172.21.15.33:0/2347216713 >> 172.21.15.33:0/2347216713 conn(0x7fbfbc0fcd20 msgr2=0x7fbfbc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 -- 172.21.15.33:0/2347216713 shutdown_connections 2024-04-24T23:17:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 -- 172.21.15.33:0/2347216713 wait complete. 2024-04-24T23:17:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 Processor -- start 2024-04-24T23:17:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.755+0000 7fbfc185d640 1 -- start start 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 0x7fbfbc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 0x7fbfbc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc103340 0x7fbfbc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc10d3f0 con 0x7fbfbc101790 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc10d270 con 0x7fbfbc102980 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc10d570 con 0x7fbfbc103340 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 0x7fbfbc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 0x7fbfbc197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48344/0 (socket says 172.21.15.33:48344) 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfbaffd640 1 -- 172.21.15.33:0/856190099 learned_addr learned my addr 172.21.15.33:0/856190099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 0x7fbfbc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc103340 0x7fbfbc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 -- 172.21.15.33:0/856190099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc103340 msgr2=0x7fbfbc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc103340 0x7fbfbc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 -- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 msgr2=0x7fbfbc197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 0x7fbfbc197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 -- 172.21.15.33:0/856190099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc19f650 con 0x7fbfbc102980 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 0x7fbfbc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 0x7fbfbc197f70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e9f0 tx=0x7fbfb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc085b640 1 -- 172.21.15.33:0/856190099 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000ccc0 con 0x7fbfbc102980 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.756+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc19f940 con 0x7fbfbc102980 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.757+0000 7fbfc085b640 1 -- 172.21.15.33:0/856190099 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0010910 con 0x7fbfbc102980 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.757+0000 7fbfc085b640 1 -- 172.21.15.33:0/856190099 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0018960 con 0x7fbfbc102980 2024-04-24T23:17:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.757+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc19fe90 con 0x7fbfbc102980 2024-04-24T23:17:00.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc085b640 1 -- 172.21.15.33:0/856190099 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbfb0010ac0 con 0x7fbfbc102980 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 Processor -- start 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 -- start start 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc0d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc12720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc15220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0dfb71670 con 0x55b0dfc0d230 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b0dfb19970 con 0x55b0dfc12d90 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfc456b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0dfb6d630 con 0x55b0dfc0dc20 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.759+0000 7fbfbb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc0d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc0d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48356/0 (socket says 172.21.15.33:48356) 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 -- 172.21.15.33:0/3594802913 learned_addr learned my addr 172.21.15.33:0/3594802913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc15220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc12720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfc085b640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbfa40a2b90 0x7fbfa40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 -- 172.21.15.33:0/3594802913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0dc20 msgr2=0x55b0dfc12720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc12720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 -- 172.21.15.33:0/3594802913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 msgr2=0x55b0dfc15220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc15220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 -- 172.21.15.33:0/3594802913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0dfb3d400 con 0x55b0dfc0d230 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc15220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbfa40a2b90 0x7fbfa40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfc085b640 1 -- 172.21.15.33:0/856190099 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbfb00cf7c0 con 0x7fbfbc102980 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.760+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc0d650 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009ca0 tx=0x7fbfb40093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e232f0337640b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.761+0000 7fbfa1ffb640 1 -- 172.21.15.33:0/3594802913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4042040 con 0x55b0dfc0d230 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.761+0000 7fbfa1ffb640 1 -- 172.21.15.33:0/3594802913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb4044070 con 0x55b0dfc0d230 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.761+0000 7fbfa1ffb640 1 -- 172.21.15.33:0/3594802913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4051630 con 0x55b0dfc0d230 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.761+0000 7fbfc456b740 1 -- 172.21.15.33:0/3594802913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 msgr2=0x55b0dfc0d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.761+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbfa40a2b90 0x7fbfa40a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbfb404b000 tx=0x7fbfb404c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.761+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc0d650 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009ca0 tx=0x7fbfb40093b0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- 172.21.15.33:0/3594802913 shutdown_connections 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc15220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc12720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3594802913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc0d650 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- 172.21.15.33:0/3594802913 >> 172.21.15.33:0/3594802913 conn(0x55b0dfc088a0 msgr2=0x55b0dfc08ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- 172.21.15.33:0/3594802913 shutdown_connections 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- 172.21.15.33:0/3594802913 wait complete. 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- start start 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 Processor -- start 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc1b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc1bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0dfc1a150 con 0x55b0dfc0d230 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b0dfb2d050 con 0x55b0dfc0dc20 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.762+0000 7fbfc456b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0dfc1a2d0 con 0x55b0dfc12d90 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc1b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfc456b740 10 client.0 _ll_register_callbacks cb 0x55b0df884be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc1b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48362/0 (socket says 172.21.15.33:48362) 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 -- 172.21.15.33:0/2754336149 learned_addr learned my addr 172.21.15.33:0/2754336149 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc1bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 -- 172.21.15.33:0/2754336149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 msgr2=0x55b0dfc22d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc22d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 -- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 msgr2=0x55b0dfc1bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc1bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 -- 172.21.15.33:0/2754336149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0dfafedc0 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc1b790 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbfac012cf0 tx=0x7fbfac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c0dc2158672e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbf9affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbf9affd640 1 -- 172.21.15.33:0/2754336149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac012110 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbf9affd640 1 -- 172.21.15.33:0/2754336149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbfac004050 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb4044070 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.763+0000 7fbf9affd640 1 -- 172.21.15.33:0/2754336149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfac025050 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 10 client.16455 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0df8853f0 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbf9affd640 1 -- 172.21.15.33:0/2754336149 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb4044070 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbf9affd640 1 -- 172.21.15.33:0/2754336149 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbfac010430 con 0x55b0dfc0d230 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 10 client.16455 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 10 client.16455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 20 client.16455 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc27680 0x55b0dfc29b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 4 client.16455 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0dfb7ebb0 con 0x55b0dfc27680 2024-04-24T23:17:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc27680 0x55b0dfc29b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.764+0000 7fbf9affd640 10 client.16455 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.765+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc27680 0x55b0dfc29b20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63ba2d9e992af9db server_cookie=fe2440a50392f584 in_seq=0 out_seq=0 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbf9affd640 1 -- 172.21.15.33:0/2754336149 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0dfb7ebb0 con 0x55b0dfc27680 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbf9affd640 10 client.16455 handle_command_reply: tid=0 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 client.16455 shutdown 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 msgr2=0x55b0dfc1b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc1b790 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbfac012cf0 tx=0x7fbfac00f790 comp rx=0 tx=0).stop 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 shutdown_connections 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc27680 0x55b0dfc29b20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc22d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbf9affd640 0 client.16455 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 --2- 172.21.15.33:0/2754336149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc1b790 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 >> 172.21.15.33:0/2754336149 conn(0x55b0dfc088a0 msgr2=0x55b0dfc07b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 shutdown_connections 2024-04-24T23:17:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 1 -- 172.21.15.33:0/2754336149 wait complete. 2024-04-24T23:17:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.767+0000 7fbfc456b740 20 client.16455 trim_cache size 0 max 16384 2024-04-24T23:17:00.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.781+0000 7fbfc456b740 1 Processor -- start 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 -- start start 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc2e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1aaa50 con 0x55b0dfc0d230 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a8320 con 0x55b0dfc0dc20 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfc456b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a6170 con 0x55b0dfc12d90 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48372/0 (socket says 172.21.15.33:48372) 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 -- 172.21.15.33:0/4218714905 learned_addr learned my addr 172.21.15.33:0/4218714905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbb7fe640 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc2e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 -- 172.21.15.33:0/4218714905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 msgr2=0x55b0dfc2e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc2e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 -- 172.21.15.33:0/4218714905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 msgr2=0x55b0dfc22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc22cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 -- 172.21.15.33:0/4218714905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0df94f7d0 con 0x55b0dfc0d230 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.782+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc072b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009620 tx=0x7fbfac012830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e3a29377ffdbf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfa17fa640 1 -- 172.21.15.33:0/4218714905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac010ac0 con 0x55b0dfc0d230 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfa17fa640 1 -- 172.21.15.33:0/4218714905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac010c60 con 0x55b0dfc0d230 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfa17fa640 1 -- 172.21.15.33:0/4218714905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003850 con 0x55b0dfc0d230 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 -- 172.21.15.33:0/4218714905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 msgr2=0x55b0dfc072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc072b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009620 tx=0x7fbfac012830 comp rx=0 tx=0).stop 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 -- 172.21.15.33:0/4218714905 shutdown_connections 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc12d90 0x55b0dfc2e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 --2- 172.21.15.33:0/4218714905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0d230 0x55b0dfc072b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 -- 172.21.15.33:0/4218714905 >> 172.21.15.33:0/4218714905 conn(0x55b0dfc088a0 msgr2=0x55b0dfc0ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 -- 172.21.15.33:0/4218714905 shutdown_connections 2024-04-24T23:17:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.783+0000 7fbfc456b740 1 -- 172.21.15.33:0/4218714905 wait complete. 2024-04-24T23:17:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 -- start start 2024-04-24T23:17:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 Processor -- start 2024-04-24T23:17:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0d230 0x55b0dfc20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc20550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc21560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a67e0 con 0x55b0dfc0dc20 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a7c50 con 0x55b0dfc12d90 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a6eb0 con 0x55b0dfc0d230 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfc456b740 10 client.0 _ll_register_callbacks cb 0x7fbfac009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc20550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfbaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0d230 0x55b0dfc20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc20550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48388/0 (socket says 172.21.15.33:48388) 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfbb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc21560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.784+0000 7fbfba7fc640 1 -- 172.21.15.33:0/3811134996 learned_addr learned my addr 172.21.15.33:0/3811134996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfbaffd640 1 -- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 msgr2=0x55b0dfc21560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc21560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfbaffd640 1 -- 172.21.15.33:0/3811134996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 msgr2=0x55b0dfc20550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc20550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfbaffd640 1 -- 172.21.15.33:0/3811134996 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0dfc21b50 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc20550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfbaffd640 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0d230 0x55b0dfc20010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55b0dfc122d0 tx=0x7fbfac00fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5653db545d7b0313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbf9affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbf9affd640 1 -- 172.21.15.33:0/3811134996 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac037060 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbf9affd640 1 -- 172.21.15.33:0/3811134996 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbfac0052e0 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac010c60 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbf9affd640 1 -- 172.21.15.33:0/3811134996 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfac03e050 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfc456b740 10 client.26011 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.785+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0dfc22e50 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbf9affd640 1 -- 172.21.15.33:0/3811134996 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac010c60 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbf9affd640 1 -- 172.21.15.33:0/3811134996 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbfac003c60 con 0x55b0dfc0d230 2024-04-24T23:17:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfc456b740 10 client.26011 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfc456b740 10 client.26011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfc456b740 20 client.26011 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc8e140 0x55b0dfc90560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfc456b740 4 client.26011 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0dfb78f00 con 0x55b0dfc8e140 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc8e140 0x55b0dfc90560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbf9affd640 10 client.26011 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.786+0000 7fbfba7fc640 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc8e140 0x55b0dfc90560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1ac0f096771ff47 server_cookie=7216be089dd3053a in_seq=0 out_seq=0 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbf9affd640 1 -- 172.21.15.33:0/3811134996 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0dfb78f00 con 0x55b0dfc8e140 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbf9affd640 10 client.26011 handle_command_reply: tid=0 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 client.26011 shutdown 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0d230 msgr2=0x55b0dfc20010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0d230 0x55b0dfc20010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55b0dfc122d0 tx=0x7fbfac00fc90 comp rx=0 tx=0).stop 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 shutdown_connections 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b0dfc8e140 0x55b0dfc90560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0dfc12d90 0x55b0dfc21560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0dfc0dc20 0x55b0dfc20550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 --2- 172.21.15.33:0/3811134996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0dfc0d230 0x55b0dfc20010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbf9affd640 0 client.26011 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 >> 172.21.15.33:0/3811134996 conn(0x55b0dfc088a0 msgr2=0x55b0dfc0b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 shutdown_connections 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 1 -- 172.21.15.33:0/3811134996 wait complete. 2024-04-24T23:17:00.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc456b740 20 client.26011 trim_cache size 0 max 16384 2024-04-24T23:17:00.797 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:17:00.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbfa40a2b90 msgr2=0x7fbfa40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbfa40a2b90 0x7fbfa40a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbfb404b000 tx=0x7fbfb404c000 comp rx=0 tx=0).stop 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 msgr2=0x7fbfbc197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 0x7fbfbc197f70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e9f0 tx=0x7fbfb000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 shutdown_connections 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbfa40a2b90 0x7fbfa40a5050 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfbc103340 0x7fbfbc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc102980 0x7fbfbc197f70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.787+0000 7fbfc185d640 1 --2- 172.21.15.33:0/856190099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfbc101790 0x7fbfbc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.788+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 >> 172.21.15.33:0/856190099 conn(0x7fbfbc0fcd20 msgr2=0x7fbfbc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.788+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 shutdown_connections 2024-04-24T23:17:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.788+0000 7fbfc185d640 1 -- 172.21.15.33:0/856190099 wait complete. 2024-04-24T23:17:00.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 -- 172.21.15.33:0/1882823807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00102230 msgr2=0x7f9c00108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 --2- 172.21.15.33:0/1882823807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00102230 0x7f9c00108ac0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009ca0 tx=0x7f9bf802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 -- 172.21.15.33:0/1882823807 shutdown_connections 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 --2- 172.21.15.33:0/1882823807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00102230 0x7f9c00108ac0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 --2- 172.21.15.33:0/1882823807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00101870 0x7f9c00101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 --2- 172.21.15.33:0/1882823807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00100680 0x7f9c00100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 -- 172.21.15.33:0/1882823807 >> 172.21.15.33:0/1882823807 conn(0x7f9c000fbbf0 msgr2=0x7f9c000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 -- 172.21.15.33:0/1882823807 shutdown_connections 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 -- 172.21.15.33:0/1882823807 wait complete. 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 Processor -- start 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.909+0000 7f9c058da640 1 -- start start 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00100680 0x7f9c00199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 0x7f9c0019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00102230 0x7f9c001a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010c300 con 0x7f9c00101870 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010c180 con 0x7f9c00102230 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010c480 con 0x7f9c00100680 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 0x7f9c0019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 0x7f9c0019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48414/0 (socket says 172.21.15.33:48414) 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 -- 172.21.15.33:0/3798170228 learned_addr learned my addr 172.21.15.33:0/3798170228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00102230 0x7f9c001a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00100680 0x7f9c00199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 -- 172.21.15.33:0/3798170228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00100680 msgr2=0x7f9c00199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00100680 0x7f9c00199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 -- 172.21.15.33:0/3798170228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00102230 msgr2=0x7f9c001a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00102230 0x7f9c001a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 -- 172.21.15.33:0/3798170228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c001a18b0 con 0x7f9c00101870 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 0x7f9c0019a1b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e9f0 tx=0x7f9be800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c048d8640 1 -- 172.21.15.33:0/3798170228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be800cd80 con 0x7f9c00101870 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c048d8640 1 -- 172.21.15.33:0/3798170228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be8010910 con 0x7f9c00101870 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.910+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c001a1ba0 con 0x7f9c00101870 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.911+0000 7f9c048d8640 1 -- 172.21.15.33:0/3798170228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8018990 con 0x7f9c00101870 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.911+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c001a2140 con 0x7f9c00101870 2024-04-24T23:17:00.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c048d8640 1 -- 172.21.15.33:0/3798170228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9be8010ac0 con 0x7f9c00101870 2024-04-24T23:17:00.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 Processor -- start 2024-04-24T23:17:00.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 -- start start 2024-04-24T23:17:00.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x561490266650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x56149026b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56149026bd90 0x56149026e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5614901ca670 con 0x561490266230 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561490172970 con 0x56149026bd90 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9c085ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5614901c6630 con 0x561490266c20 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9bff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x561490266650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9bff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x561490266650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48422/0 (socket says 172.21.15.33:48422) 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9bff7fe640 1 -- 172.21.15.33:0/2483064886 learned_addr learned my addr 172.21.15.33:0/2483064886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56149026bd90 0x56149026e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.913+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x56149026b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9c048d8640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9bec0a2c10 0x7f9bec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9bec0a2c10 0x7f9bec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf7fff640 1 -- 172.21.15.33:0/2483064886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 msgr2=0x56149026b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x56149026b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf7fff640 1 -- 172.21.15.33:0/2483064886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 msgr2=0x561490266650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x561490266650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf7fff640 1 -- 172.21.15.33:0/2483064886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561490196400 con 0x56149026bd90 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x56149026b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9c048d8640 1 -- 172.21.15.33:0/3798170228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9be80cf6f0 con 0x7f9c00101870 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56149026bd90 0x56149026e220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9be80cceb0 tx=0x7f9be809bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f644995e4a56936 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9bec0a2c10 0x7f9bec0a50d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x561490267480 tx=0x7f9bf802a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf5ffb640 1 -- 172.21.15.33:0/2483064886 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80213f0 con 0x56149026bd90 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf5ffb640 1 -- 172.21.15.33:0/2483064886 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be809c2c0 con 0x56149026bd90 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.914+0000 7f9bf5ffb640 1 -- 172.21.15.33:0/2483064886 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8098230 con 0x56149026bd90 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 -- 172.21.15.33:0/2483064886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56149026bd90 msgr2=0x56149026e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56149026bd90 0x56149026e220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9be80cceb0 tx=0x7f9be809bac0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 -- 172.21.15.33:0/2483064886 shutdown_connections 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56149026bd90 0x56149026e220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x56149026b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2483064886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x561490266650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 -- 172.21.15.33:0/2483064886 >> 172.21.15.33:0/2483064886 conn(0x5614902618a0 msgr2=0x561490261ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 -- 172.21.15.33:0/2483064886 shutdown_connections 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.915+0000 7f9c085ee740 1 -- 172.21.15.33:0/2483064886 wait complete. 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 -- start start 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 Processor -- start 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x561490270290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x5614902707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 0x561490277810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56149026ec80 con 0x56149026bd90 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561490186050 con 0x561490266230 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56149026ee00 con 0x561490266c20 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9c085ee740 10 client.0 _ll_register_callbacks cb 0x56148feddbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9bff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 0x561490277810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9bfeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x561490270290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9bff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 0x561490277810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48438/0 (socket says 172.21.15.33:48438) 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9bff7fe640 1 -- 172.21.15.33:0/3567292956 learned_addr learned my addr 172.21.15.33:0/3567292956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.916+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x5614902707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bff7fe640 1 -- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 msgr2=0x5614902707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x5614902707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bff7fe640 1 -- 172.21.15.33:0/3567292956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 msgr2=0x561490270290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x561490270290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bff7fe640 1 -- 172.21.15.33:0/3567292956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561490157dc0 con 0x56149026bd90 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x561490270290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x5614902707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 0x561490277810 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009bf0 tx=0x7f9bf80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0b8ec5836c6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bdeffd640 1 -- 172.21.15.33:0/3567292956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf804e070 con 0x56149026bd90 2024-04-24T23:17:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bdeffd640 1 -- 172.21.15.33:0/3567292956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9bf802f610 con 0x56149026bd90 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be809c2c0 con 0x56149026bd90 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bdeffd640 1 -- 172.21.15.33:0/3567292956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9bf805a050 con 0x56149026bd90 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 10 client.16473 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56148fede3f0 con 0x56149026bd90 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bdeffd640 1 -- 172.21.15.33:0/3567292956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be809c2c0 con 0x56149026bd90 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9bdeffd640 1 -- 172.21.15.33:0/3567292956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9bf8030b20 con 0x56149026bd90 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 10 client.16473 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 10 client.16473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 20 client.16473 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.917+0000 7f9c085ee740 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56149027c0b0 0x56149027e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.918+0000 7f9c085ee740 4 client.16473 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.918+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614901d7bb0 con 0x56149027c0b0 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.918+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56149027c0b0 0x56149027e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.918+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56149027c0b0 0x56149027e530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c80a19077dbd7059 server_cookie=9b9cbe50b0b4e194 in_seq=0 out_seq=0 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.918+0000 7f9bdeffd640 10 client.16473 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9bdeffd640 1 -- 172.21.15.33:0/3567292956 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614901d7bb0 con 0x56149027c0b0 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9bdeffd640 10 client.16473 handle_command_reply: tid=0 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 client.16473 shutdown 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 msgr2=0x561490277810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 0x561490277810 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009bf0 tx=0x7f9bf80095c0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 shutdown_connections 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56149027c0b0 0x56149027e530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56149026bd90 0x561490277810 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561490266c20 0x5614902707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 --2- 172.21.15.33:0/3567292956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x561490270290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 >> 172.21.15.33:0/3567292956 conn(0x5614902618a0 msgr2=0x561490260b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9bdeffd640 0 client.16473 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 shutdown_connections 2024-04-24T23:17:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 1 -- 172.21.15.33:0/3567292956 wait complete. 2024-04-24T23:17:00.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.919+0000 7f9c085ee740 20 client.16473 trim_cache size 0 max 16384 2024-04-24T23:17:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 Processor -- start 2024-04-24T23:17:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 -- start start 2024-04-24T23:17:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x5614902609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266c20 0x5614902777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001aee80 con 0x561490266230 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001ac750 con 0x561490266c20 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9c085ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001aa5a0 con 0x56149026bd90 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.933+0000 7f9bfeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x5614902609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x5614902609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48446/0 (socket says 172.21.15.33:48446) 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 -- 172.21.15.33:0/2171981079 learned_addr learned my addr 172.21.15.33:0/2171981079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266c20 0x5614902777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 -- 172.21.15.33:0/2171981079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 msgr2=0x5614902676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 -- 172.21.15.33:0/2171981079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266c20 msgr2=0x5614902777d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266c20 0x5614902777d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 -- 172.21.15.33:0/2171981079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148ffa87d0 con 0x561490266230 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x5614902609c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9c0019ab40 tx=0x7f9bf0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4150229fe2e2ea32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bf57fa640 1 -- 172.21.15.33:0/2171981079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00040f0 con 0x561490266230 2024-04-24T23:17:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bf57fa640 1 -- 172.21.15.33:0/2171981079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf0025020 con 0x561490266230 2024-04-24T23:17:00.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.934+0000 7f9bf57fa640 1 -- 172.21.15.33:0/2171981079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0003a10 con 0x561490266230 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 -- 172.21.15.33:0/2171981079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 msgr2=0x5614902609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x5614902609c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9c0019ab40 tx=0x7f9bf0005f90 comp rx=0 tx=0).stop 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 -- 172.21.15.33:0/2171981079 shutdown_connections 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266c20 0x5614902777d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 --2- 172.21.15.33:0/2171981079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266230 0x5614902609c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 -- 172.21.15.33:0/2171981079 >> 172.21.15.33:0/2171981079 conn(0x5614902618a0 msgr2=0x5614902602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 -- 172.21.15.33:0/2171981079 shutdown_connections 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 -- 172.21.15.33:0/2171981079 wait complete. 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 -- start start 2024-04-24T23:17:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.935+0000 7f9c085ee740 1 Processor -- start 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x5614902745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 0x561490274b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001aac10 con 0x561490266c20 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001ac080 con 0x561490266230 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001ab2e0 con 0x56149026bd90 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9c085ee740 10 client.0 _ll_register_callbacks cb 0x7f9bf8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 0x561490274b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x5614902745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 0x561490274b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48460/0 (socket says 172.21.15.33:48460) 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bf7fff640 1 -- 172.21.15.33:0/225051998 learned_addr learned my addr 172.21.15.33:0/225051998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bff7fe640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 -- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 msgr2=0x5614902e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 -- 172.21.15.33:0/225051998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 msgr2=0x561490274b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 0x561490274b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 -- 172.21.15.33:0/225051998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561490277af0 con 0x561490266230 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 0x561490274b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bfeffd640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x5614902745c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x561490186070 tx=0x7f9bf00218a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6883f789bdd4aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.936+0000 7f9bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9bdeffd640 1 -- 172.21.15.33:0/225051998 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00211a0 con 0x561490266230 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf0025020 con 0x561490266230 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9bdeffd640 1 -- 172.21.15.33:0/225051998 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9bf0021340 con 0x561490266230 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9bdeffd640 1 -- 172.21.15.33:0/225051998 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9bf003a050 con 0x561490266230 2024-04-24T23:17:00.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 10 client.26012 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614902e1c50 con 0x561490266230 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9bdeffd640 1 -- 172.21.15.33:0/225051998 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf0025020 con 0x561490266230 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9bdeffd640 1 -- 172.21.15.33:0/225051998 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9bf0003b40 con 0x561490266230 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 10 client.26012 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 10 client.26012 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 20 client.26012 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5614902e24c0 0x5614902e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 4 client.26012 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561490270080 con 0x5614902e24c0 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.937+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5614902e24c0 0x5614902e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9bdeffd640 10 client.26012 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9bf7fff640 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5614902e24c0 0x5614902e48e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=624510ebbfa1139e server_cookie=84bb449b323f797f in_seq=0 out_seq=0 2024-04-24T23:17:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9bdeffd640 1 -- 172.21.15.33:0/225051998 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561490270080 con 0x5614902e24c0 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9bdeffd640 10 client.26012 handle_command_reply: tid=0 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 client.26012 shutdown 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 msgr2=0x5614902745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x5614902745c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x561490186070 tx=0x7f9bf00218a0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 shutdown_connections 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5614902e24c0 0x5614902e48e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56149026bd90 0x5614902e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561490266c20 0x561490274b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 --2- 172.21.15.33:0/225051998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561490266230 0x5614902745c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9bdeffd640 0 client.26012 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 >> 172.21.15.33:0/225051998 conn(0x5614902618a0 msgr2=0x561490260b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 shutdown_connections 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 1 -- 172.21.15.33:0/225051998 wait complete. 2024-04-24T23:17:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.938+0000 7f9c085ee740 20 client.26012 trim_cache size 0 max 16384 2024-04-24T23:17:00.948 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:17:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9bec0a2c10 msgr2=0x7f9bec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9bec0a2c10 0x7f9bec0a50d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x561490267480 tx=0x7f9bf802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 msgr2=0x7f9c0019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 0x7f9c0019a1b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e9f0 tx=0x7f9be800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 shutdown_connections 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9bec0a2c10 0x7f9bec0a50d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00102230 0x7f9c001a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c00101870 0x7f9c0019a1b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 --2- 172.21.15.33:0/3798170228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00100680 0x7f9c00199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 >> 172.21.15.33:0/3798170228 conn(0x7f9c000fbbf0 msgr2=0x7f9c000fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 shutdown_connections 2024-04-24T23:17:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:00.939+0000 7f9c058da640 1 -- 172.21.15.33:0/3798170228 wait complete. 2024-04-24T23:17:00.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/3971007915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78100680 msgr2=0x7fbe78100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/3971007915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78100680 0x7fbe78100aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009900 tx=0x7fbe6002a3f0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/3971007915 shutdown_connections 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/3971007915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78102230 0x7fbe78108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/3971007915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe78101870 0x7fbe78101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/3971007915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78100680 0x7fbe78100aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/3971007915 >> 172.21.15.33:0/3971007915 conn(0x7fbe780fbbf0 msgr2=0x7fbe780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/3971007915 shutdown_connections 2024-04-24T23:17:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.062+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/3971007915 wait complete. 2024-04-24T23:17:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 Processor -- start 2024-04-24T23:17:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 -- start start 2024-04-24T23:17:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 0x7fbe7810ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe78101870 0x7fbe781094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78102230 0x7fbe78109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7810d420 con 0x7fbe78100680 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7810d2a0 con 0x7fbe78102230 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7e8d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7810d5a0 con 0x7fbe78101870 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 0x7fbe7810ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 0x7fbe7810ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48478/0 (socket says 172.21.15.33:48478) 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe77fff640 1 -- 172.21.15.33:0/1639313575 learned_addr learned my addr 172.21.15.33:0/1639313575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe777fe640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe78101870 0x7fbe781094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.063+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78102230 0x7fbe78109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe77fff640 1 -- 172.21.15.33:0/1639313575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe78101870 msgr2=0x7fbe781094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe77fff640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe78101870 0x7fbe781094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe77fff640 1 -- 172.21.15.33:0/1639313575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78102230 msgr2=0x7fbe78109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe77fff640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78102230 0x7fbe78109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe77fff640 1 -- 172.21.15.33:0/1639313575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7810a290 con 0x7fbe78100680 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe77fff640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 0x7fbe7810ae80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe60031610 tx=0x7fbe60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe757fa640 1 -- 172.21.15.33:0/1639313575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6003f030 con 0x7fbe78100680 2024-04-24T23:17:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe781a1be0 con 0x7fbe78100680 2024-04-24T23:17:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe757fa640 1 -- 172.21.15.33:0/1639313575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe60003ad0 con 0x7fbe78100680 2024-04-24T23:17:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe757fa640 1 -- 172.21.15.33:0/1639313575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60035070 con 0x7fbe78100680 2024-04-24T23:17:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.064+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe781a2180 con 0x7fbe78100680 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.066+0000 7fbe757fa640 1 -- 172.21.15.33:0/1639313575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbe60042030 con 0x7fbe78100680 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 Processor -- start 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 -- start start 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe757fa640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe640a2b90 0x7fbe640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dcb1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c9dc0d670 con 0x561c9dcaed90 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561c9dbb5970 con 0x561c9dca9c20 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe640a2b90 0x7fbe640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe815e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c9dc09630 con 0x561c9dca9230 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe757fa640 1 -- 172.21.15.33:0/1639313575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe600ef960 con 0x7fbe78100680 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe7ce71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dcb1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe7ce71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dcb1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48482/0 (socket says 172.21.15.33:48482) 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe7ce71640 1 -- 172.21.15.33:0/929030225 learned_addr learned my addr 172.21.15.33:0/929030225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.067+0000 7fbe77fff640 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe777fe640 1 -- 172.21.15.33:0/929030225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 msgr2=0x561c9dca9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe777fe640 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe777fe640 1 -- 172.21.15.33:0/929030225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 msgr2=0x561c9dcb1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe777fe640 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dcb1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe777fe640 1 -- 172.21.15.33:0/929030225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9dbd9400 con 0x561c9dca9c20 2024-04-24T23:17:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe640a2b90 0x7fbe640a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbe70013cf0 tx=0x7fbe70009f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe777fe640 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcae720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009bd0 tx=0x7fbe68011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb99343ae03e1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe6e7fc640 1 -- 172.21.15.33:0/929030225 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68004050 con 0x561c9dca9c20 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe6e7fc640 1 -- 172.21.15.33:0/929030225 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe68005500 con 0x561c9dca9c20 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.068+0000 7fbe6e7fc640 1 -- 172.21.15.33:0/929030225 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68004ca0 con 0x561c9dca9c20 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 -- 172.21.15.33:0/929030225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 msgr2=0x561c9dcae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcae720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009bd0 tx=0x7fbe68011440 comp rx=0 tx=0).stop 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 -- 172.21.15.33:0/929030225 shutdown_connections 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dcb1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcae720 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 --2- 172.21.15.33:0/929030225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 -- 172.21.15.33:0/929030225 >> 172.21.15.33:0/929030225 conn(0x561c9dca48a0 msgr2=0x561c9dca4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 -- 172.21.15.33:0/929030225 shutdown_connections 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 -- 172.21.15.33:0/929030225 wait complete. 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 -- start start 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 Processor -- start 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.069+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9230 0x561c9dcb32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 0x561c9dcb37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c9dcb1d40 con 0x561c9dca9c20 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561c9dbc9050 con 0x561c9dca9230 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c9dcb1ec0 con 0x561c9dcaed90 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 10 client.0 _ll_register_callbacks cb 0x561c9d920be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 0x561c9dcb37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 0x561c9dcb37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48486/0 (socket says 172.21.15.33:48486) 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 -- 172.21.15.33:0/383091051 learned_addr learned my addr 172.21.15.33:0/383091051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe77fff640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9230 0x561c9dcb32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 -- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 msgr2=0x561c9dcba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 -- 172.21.15.33:0/383091051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9230 msgr2=0x561c9dcb32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9230 0x561c9dcb32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 -- 172.21.15.33:0/383091051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9db9adc0 con 0x561c9dca9c20 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe77fff640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9230 0x561c9dcb32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe777fe640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 0x561c9dcb37f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbe68003d70 tx=0x7fbe68003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfcc2f04ed38482 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe577fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe577fe640 1 -- 172.21.15.33:0/383091051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68027020 con 0x561c9dca9c20 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe577fe640 1 -- 172.21.15.33:0/383091051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe680049e0 con 0x561c9dca9c20 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.070+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe68005500 con 0x561c9dca9c20 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe577fe640 1 -- 172.21.15.33:0/383091051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe68032050 con 0x561c9dca9c20 2024-04-24T23:17:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 10 client.16491 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c9d9213f0 con 0x561c9dca9c20 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe577fe640 1 -- 172.21.15.33:0/383091051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe68005500 con 0x561c9dca9c20 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe577fe640 1 -- 172.21.15.33:0/383091051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe68018070 con 0x561c9dca9c20 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 10 client.16491 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 20 client.16491 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dcbee90 0x561c9dcc1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 4 client.16491 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c9dc1abb0 con 0x561c9dcbee90 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.071+0000 7fbe77fff640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dcbee90 0x561c9dcc1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe77fff640 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dcbee90 0x561c9dcc1300 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2586542d95e84771 server_cookie=42f5a4458e1257df in_seq=0 out_seq=0 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe577fe640 10 client.16491 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe577fe640 1 -- 172.21.15.33:0/383091051 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c9dc1abb0 con 0x561c9dcbee90 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe577fe640 10 client.16491 handle_command_reply: tid=0 2024-04-24T23:17:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe815e5740 1 client.16491 shutdown 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 msgr2=0x561c9dcb37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.072+0000 7fbe815e5740 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 0x561c9dcb37f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbe68003d70 tx=0x7fbe68003da0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 shutdown_connections 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dcbee90 0x561c9dcc1300 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9c20 0x561c9dcb37f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe577fe640 0 client.16491 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 --2- 172.21.15.33:0/383091051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9230 0x561c9dcb32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 >> 172.21.15.33:0/383091051 conn(0x561c9dca48a0 msgr2=0x561c9dca3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 shutdown_connections 2024-04-24T23:17:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 1 -- 172.21.15.33:0/383091051 wait complete. 2024-04-24T23:17:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.073+0000 7fbe815e5740 20 client.16491 trim_cache size 0 max 16384 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 Processor -- start 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 -- start start 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dca7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781aeed0 con 0x561c9dcaed90 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781ac7a0 con 0x561c9dca9c20 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe815e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781aa5f0 con 0x561c9dca9230 2024-04-24T23:17:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe7ce71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dca7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe7ce71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dca7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48492/0 (socket says 172.21.15.33:48492) 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe7ce71640 1 -- 172.21.15.33:0/3718693609 learned_addr learned my addr 172.21.15.33:0/3718693609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.086+0000 7fbe777fe640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe77fff640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe7ce71640 1 -- 172.21.15.33:0/3718693609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 msgr2=0x561c9dca7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe7ce71640 1 -- 172.21.15.33:0/3718693609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 msgr2=0x561c9dcba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe7ce71640 1 -- 172.21.15.33:0/3718693609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9d9eb7d0 con 0x561c9dcaed90 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe77fff640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe777fe640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dca7ac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009270 tx=0x7fbe70009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632f5bc1e7a6386a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe6dffb640 1 -- 172.21.15.33:0/3718693609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70003970 con 0x561c9dcaed90 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe6dffb640 1 -- 172.21.15.33:0/3718693609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe70004020 con 0x561c9dcaed90 2024-04-24T23:17:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.087+0000 7fbe6dffb640 1 -- 172.21.15.33:0/3718693609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7002b410 con 0x561c9dcaed90 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 -- 172.21.15.33:0/3718693609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 msgr2=0x561c9dca7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dca7ac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009270 tx=0x7fbe70009500 comp rx=0 tx=0).stop 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 -- 172.21.15.33:0/3718693609 shutdown_connections 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dcaed90 0x561c9dca7ac0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 --2- 172.21.15.33:0/3718693609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dca9230 0x561c9dca7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 -- 172.21.15.33:0/3718693609 >> 172.21.15.33:0/3718693609 conn(0x561c9dca48a0 msgr2=0x561c9dca3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 -- 172.21.15.33:0/3718693609 shutdown_connections 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 -- 172.21.15.33:0/3718693609 wait complete. 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 -- start start 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.088+0000 7fbe815e5740 1 Processor -- start 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9230 0x561c9dcb8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcb85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcb9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781aac60 con 0x561c9dca9230 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781ac0d0 con 0x561c9dca9c20 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781ab330 con 0x561c9dcaed90 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 10 client.0 _ll_register_callbacks cb 0x7fbe70013840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9230 0x561c9dcb8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9230 0x561c9dcb8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48494/0 (socket says 172.21.15.33:48494) 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe77fff640 1 -- 172.21.15.33:0/609495941 learned_addr learned my addr 172.21.15.33:0/609495941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe777fe640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcb85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcb9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 -- 172.21.15.33:0/609495941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 msgr2=0x561c9dcb85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcb85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 -- 172.21.15.33:0/609495941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9230 msgr2=0x561c9dcb8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9230 0x561c9dcb8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 -- 172.21.15.33:0/609495941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9dcb9a80 con 0x561c9dcaed90 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe777fe640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcb85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe7ce71640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcb9490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe70011c40 tx=0x7fbe70004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d7444ef42e444a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe577fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe577fe640 1 -- 172.21.15.33:0/609495941 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70039060 con 0x561c9dcaed90 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe577fe640 1 -- 172.21.15.33:0/609495941 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe70003b70 con 0x561c9dcaed90 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.089+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe70004020 con 0x561c9dcaed90 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe577fe640 1 -- 172.21.15.33:0/609495941 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe70040050 con 0x561c9dcaed90 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 10 client.26044 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c9dd25480 con 0x561c9dcaed90 2024-04-24T23:17:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe577fe640 1 -- 172.21.15.33:0/609495941 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe70004020 con 0x561c9dcaed90 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe577fe640 1 -- 172.21.15.33:0/609495941 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe70020050 con 0x561c9dcaed90 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 10 client.26044 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 10 client.26044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 20 client.26044 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dd25e80 0x561c9dd282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 4 client.26044 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c9dc14f00 con 0x561c9dd25e80 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.090+0000 7fbe77fff640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dd25e80 0x561c9dd282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe77fff640 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dd25e80 0x561c9dd282a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8befc22923ab75ef server_cookie=3e4d1b6c77533611 in_seq=0 out_seq=0 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe577fe640 10 client.26044 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe577fe640 1 -- 172.21.15.33:0/609495941 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c9dc14f00 con 0x561c9dd25e80 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe577fe640 10 client.26044 handle_command_reply: tid=0 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 client.26044 shutdown 2024-04-24T23:17:01.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 msgr2=0x561c9dcb9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcb9490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe70011c40 tx=0x7fbe70004ad0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 shutdown_connections 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561c9dd25e80 0x561c9dd282a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9dcaed90 0x561c9dcb9490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561c9dca9c20 0x561c9dcb85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe577fe640 0 client.26044 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 --2- 172.21.15.33:0/609495941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c9dca9230 0x561c9dcb8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 >> 172.21.15.33:0/609495941 conn(0x561c9dca48a0 msgr2=0x561c9dcaa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 shutdown_connections 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 1 -- 172.21.15.33:0/609495941 wait complete. 2024-04-24T23:17:01.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.091+0000 7fbe815e5740 20 client.26044 trim_cache size 0 max 16384 2024-04-24T23:17:01.101 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:17:01.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe640a2b90 msgr2=0x7fbe640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe640a2b90 0x7fbe640a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbe70013cf0 tx=0x7fbe70009f10 comp rx=0 tx=0).stop 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 msgr2=0x7fbe7810ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 0x7fbe7810ae80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe60031610 tx=0x7fbe60004270 comp rx=0 tx=0).stop 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 shutdown_connections 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe640a2b90 0x7fbe640a5050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe78102230 0x7fbe78109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe78101870 0x7fbe781094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 --2- 172.21.15.33:0/1639313575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe78100680 0x7fbe7810ae80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 >> 172.21.15.33:0/1639313575 conn(0x7fbe780fbbf0 msgr2=0x7fbe78104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 shutdown_connections 2024-04-24T23:17:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:01.092+0000 7fbe7e8d4640 1 -- 172.21.15.33:0/1639313575 wait complete. 2024-04-24T23:17:01.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 453516 12.65 MB/sec execute 66 sec latency 13.747 ms 2024-04-24T23:17:01.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:01 smithi038 ceph-mon[32450]: pgmap v501: 129 pgs: 129 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-04-24T23:17:01.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:01 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3839116346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:01.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:01 smithi117 ceph-mon[32529]: pgmap v501: 129 pgs: 129 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-04-24T23:17:01.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:01 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3839116346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:01.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:01 smithi033 ceph-mon[29201]: pgmap v501: 129 pgs: 129 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-04-24T23:17:01.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:01 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3839116346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:02.106 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:17:02.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:17:02.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 -- 172.21.15.33:0/435402932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478102980 msgr2=0x7fc478102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 --2- 172.21.15.33:0/435402932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478102980 0x7fc478102e00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc468009900 tx=0x7fc46802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 -- 172.21.15.33:0/435402932 shutdown_connections 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 --2- 172.21.15.33:0/435402932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc478103340 0x7fc478109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 --2- 172.21.15.33:0/435402932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478102980 0x7fc478102e00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 --2- 172.21.15.33:0/435402932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc478101790 0x7fc478101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 -- 172.21.15.33:0/435402932 >> 172.21.15.33:0/435402932 conn(0x7fc4780fcd20 msgr2=0x7fc4780ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 -- 172.21.15.33:0/435402932 shutdown_connections 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.223+0000 7fc47ed90640 1 -- 172.21.15.33:0/435402932 wait complete. 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 Processor -- start 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 -- start start 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 0x7fc478197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc478102980 0x7fc478198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc478103340 0x7fc47819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc47810d410 con 0x7fc478101790 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc47810d290 con 0x7fc478103340 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47ed90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc47810d590 con 0x7fc478102980 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 0x7fc478197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 0x7fc478197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48504/0 (socket says 172.21.15.33:48504) 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 -- 172.21.15.33:0/2067848129 learned_addr learned my addr 172.21.15.33:0/2067848129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47d32d640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc478103340 0x7fc47819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc46ffff640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc478102980 0x7fc478198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 -- 172.21.15.33:0/2067848129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc478102980 msgr2=0x7fc478198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc478102980 0x7fc478198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 -- 172.21.15.33:0/2067848129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc478103340 msgr2=0x7fc47819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc478103340 0x7fc47819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.224+0000 7fc47cb2c640 1 -- 172.21.15.33:0/2067848129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47819f6e0 con 0x7fc478101790 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.225+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 0x7fc478197ac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e9c0 tx=0x7fc47400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.225+0000 7fc46dffb640 1 -- 172.21.15.33:0/2067848129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47400cd80 con 0x7fc478101790 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.225+0000 7fc46dffb640 1 -- 172.21.15.33:0/2067848129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4740035f0 con 0x7fc478101790 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.225+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47819f9d0 con 0x7fc478101790 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.225+0000 7fc46dffb640 1 -- 172.21.15.33:0/2067848129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474010690 con 0x7fc478101790 2024-04-24T23:17:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.225+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47819ff40 con 0x7fc478101790 2024-04-24T23:17:02.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.228+0000 7fc46dffb640 1 -- 172.21.15.33:0/2067848129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc4740037a0 con 0x7fc478101790 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.228+0000 7fc481b80740 1 Processor -- start 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 -- start start 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 0x5568a7c241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c24780 0x5568a7c29280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c2bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5568a7b872c0 con 0x5568a7c23d90 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5568a7b2f910 con 0x5568a7c298f0 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc481b80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568a7b83280 con 0x5568a7c24780 2024-04-24T23:17:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc46dffb640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4600a2c10 0x7fc4600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 0x5568a7c241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 0x5568a7c241b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48506/0 (socket says 172.21.15.33:48506) 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 -- 172.21.15.33:0/3342022057 learned_addr learned my addr 172.21.15.33:0/3342022057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c2bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc46dffb640 1 -- 172.21.15.33:0/2067848129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4740cf730 con 0x7fc478101790 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4600a2c10 0x7fc4600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c24780 0x5568a7c29280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 -- 172.21.15.33:0/3342022057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c24780 msgr2=0x5568a7c29280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c24780 0x5568a7c29280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 -- 172.21.15.33:0/3342022057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 msgr2=0x5568a7c2bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c2bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.229+0000 7fc47d32d640 1 -- 172.21.15.33:0/3342022057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a7b40fb0 con 0x5568a7c23d90 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc47d32d640 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 0x5568a7c241b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc470011e20 tx=0x7fc47000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14050994dd5c5a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc47d32d640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4600a2c10 0x7fc4600a50d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc47000fc90 tx=0x7fc47001e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc45effd640 1 -- 172.21.15.33:0/3342022057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47001dcb0 con 0x5568a7c23d90 2024-04-24T23:17:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc45effd640 1 -- 172.21.15.33:0/3342022057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc47001de50 con 0x5568a7c23d90 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc45effd640 1 -- 172.21.15.33:0/3342022057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470004280 con 0x5568a7c23d90 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc481b80740 1 -- 172.21.15.33:0/3342022057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 msgr2=0x5568a7c241b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.230+0000 7fc481b80740 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 0x5568a7c241b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc470011e20 tx=0x7fc47000f790 comp rx=0 tx=0).stop 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- 172.21.15.33:0/3342022057 shutdown_connections 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c2bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c24780 0x5568a7c29280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 --2- 172.21.15.33:0/3342022057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c23d90 0x5568a7c241b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- 172.21.15.33:0/3342022057 >> 172.21.15.33:0/3342022057 conn(0x5568a7beb950 msgr2=0x5568a7c1f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- 172.21.15.33:0/3342022057 shutdown_connections 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- 172.21.15.33:0/3342022057 wait complete. 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- start start 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 Processor -- start 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 0x5568a7c2e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5568a7c2c7e0 con 0x5568a7c24780 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5568a7b42e90 con 0x5568a7c298f0 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568a7c2c960 con 0x5568a7c23d90 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.231+0000 7fc481b80740 10 client.0 _ll_register_callbacks cb 0x5568a7b7e970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc47cb2c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 0x5568a7c2e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc47cb2c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 0x5568a7c2e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39322/0 (socket says 172.21.15.33:39322) 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc47d32d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc47cb2c640 1 -- 172.21.15.33:0/3545886473 learned_addr learned my addr 172.21.15.33:0/3545886473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 -- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 msgr2=0x5568a7c2e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 0x5568a7c2e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 -- 172.21.15.33:0/3545886473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 msgr2=0x5568a7c356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 -- 172.21.15.33:0/3545886473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a79637e0 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 0x5568a7c2e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c2e690 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc468031440 tx=0x7fc468005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cef59b39817cbd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc457fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc457fff640 1 -- 172.21.15.33:0/3545886473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468035040 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc47001de50 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.232+0000 7fc457fff640 1 -- 172.21.15.33:0/3545886473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc468004040 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc457fff640 1 -- 172.21.15.33:0/3545886473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc46804f050 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 10 client.16509 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568a79c3f70 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc457fff640 1 -- 172.21.15.33:0/3545886473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc47001de50 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc457fff640 1 -- 172.21.15.33:0/3545886473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc468004810 con 0x5568a7c24780 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 10 client.16509 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 20 client.16509 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7c39b70 0x5568a7c3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 4 client.16509 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a7b5b5a0 con 0x5568a7c39b70 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.233+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7c39b70 0x5568a7c3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7c39b70 0x5568a7c3c010 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf173d8e2144883 server_cookie=f8238cbaeb4e2fb8 in_seq=0 out_seq=0 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc457fff640 10 client.16509 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc457fff640 1 -- 172.21.15.33:0/3545886473 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568a7b5b5a0 con 0x5568a7c39b70 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc457fff640 10 client.16509 handle_command_reply: tid=0 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc481b80740 1 client.16509 shutdown 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 msgr2=0x5568a7c2e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc481b80740 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c2e690 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc468031440 tx=0x7fc468005e70 comp rx=0 tx=0).stop 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 shutdown_connections 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.234+0000 7fc481b80740 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7c39b70 0x5568a7c3c010 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c298f0 0x5568a7c356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c2e690 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 1 --2- 172.21.15.33:0/3545886473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c23d90 0x5568a7c2e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc457fff640 0 client.16509 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 >> 172.21.15.33:0/3545886473 conn(0x5568a7beb950 msgr2=0x5568a7c1e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 shutdown_connections 2024-04-24T23:17:02.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 1 -- 172.21.15.33:0/3545886473 wait complete. 2024-04-24T23:17:02.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.235+0000 7fc481b80740 20 client.16509 trim_cache size 0 max 16384 2024-04-24T23:17:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.259+0000 7fc481b80740 1 Processor -- start 2024-04-24T23:17:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.259+0000 7fc481b80740 1 -- start start 2024-04-24T23:17:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.260+0000 7fc481b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.260+0000 7fc481b80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c35690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.260+0000 7fc481b80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c3bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.260+0000 7fc481b80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781accb0 con 0x5568a7c24780 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.260+0000 7fc481b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781aa580 con 0x5568a7c23d90 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.260+0000 7fc481b80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781a83d0 con 0x5568a7c298f0 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c35690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c35690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48520/0 (socket says 172.21.15.33:48520) 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 -- 172.21.15.33:0/3980001060 learned_addr learned my addr 172.21.15.33:0/3980001060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 -- 172.21.15.33:0/3980001060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 msgr2=0x5568a7c3bf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c3bf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 -- 172.21.15.33:0/3980001060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 msgr2=0x5568a7c3b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:17:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.261+0000 7fc46ffff640 1 -- 172.21.15.33:0/3980001060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a785b170 con 0x5568a7c24780 2024-04-24T23:17:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.262+0000 7fc46ffff640 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c35690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc46802a930 tx=0x7fc46802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f7d65fda5a8f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.262+0000 7fc45e7fc640 1 -- 172.21.15.33:0/3980001060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468035030 con 0x5568a7c24780 2024-04-24T23:17:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.262+0000 7fc45e7fc640 1 -- 172.21.15.33:0/3980001060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc468009520 con 0x5568a7c24780 2024-04-24T23:17:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.262+0000 7fc45e7fc640 1 -- 172.21.15.33:0/3980001060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4680051a0 con 0x5568a7c24780 2024-04-24T23:17:02.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.263+0000 7fc481b80740 1 -- 172.21.15.33:0/3980001060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 msgr2=0x5568a7c35690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.263+0000 7fc481b80740 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c35690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc46802a930 tx=0x7fc46802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:02.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 -- 172.21.15.33:0/3980001060 shutdown_connections 2024-04-24T23:17:02.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c3bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c35690 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 --2- 172.21.15.33:0/3980001060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c3b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 -- 172.21.15.33:0/3980001060 >> 172.21.15.33:0/3980001060 conn(0x5568a7beb950 msgr2=0x5568a7c2e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 -- 172.21.15.33:0/3980001060 shutdown_connections 2024-04-24T23:17:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 -- 172.21.15.33:0/3980001060 wait complete. 2024-04-24T23:17:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.264+0000 7fc481b80740 1 -- start start 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 Processor -- start 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c32560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c32aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c9e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781a8a40 con 0x5568a7c24780 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781a9eb0 con 0x5568a7c23d90 2024-04-24T23:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.265+0000 7fc481b80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781a9110 con 0x5568a7c298f0 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc481b80740 10 client.0 _ll_register_callbacks cb 0x7fc468009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c32aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c9e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c32aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48524/0 (socket says 172.21.15.33:48524) 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc46ffff640 1 -- 172.21.15.33:0/404131981 learned_addr learned my addr 172.21.15.33:0/404131981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c9e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39328/0 (socket says 172.21.15.33:39328) 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c32560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 -- 172.21.15.33:0/404131981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 msgr2=0x5568a7c32560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c32560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 -- 172.21.15.33:0/404131981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 msgr2=0x5568a7c32aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c32aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47d32d640 1 -- 172.21.15.33:0/404131981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a7c9e6d0 con 0x5568a7c298f0 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc46ffff640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c32aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.266+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c32560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.267+0000 7fc47d32d640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c9e190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc4700099b0 tx=0x7fc470011770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8995e000f6f59bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.267+0000 7fc457fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.267+0000 7fc457fff640 1 -- 172.21.15.33:0/404131981 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470033070 con 0x5568a7c298f0 2024-04-24T23:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.267+0000 7fc457fff640 1 -- 172.21.15.33:0/404131981 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc470011240 con 0x5568a7c298f0 2024-04-24T23:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.267+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc468009520 con 0x5568a7c298f0 2024-04-24T23:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.268+0000 7fc457fff640 1 -- 172.21.15.33:0/404131981 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc47003c050 con 0x5568a7c298f0 2024-04-24T23:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.268+0000 7fc481b80740 10 client.26059 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.269+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568a7c9fa70 con 0x5568a7c298f0 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.269+0000 7fc457fff640 1 -- 172.21.15.33:0/404131981 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc468009520 con 0x5568a7c298f0 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.269+0000 7fc457fff640 1 -- 172.21.15.33:0/404131981 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc470036020 con 0x5568a7c298f0 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.269+0000 7fc481b80740 10 client.26059 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.269+0000 7fc481b80740 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.270+0000 7fc481b80740 20 client.26059 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.270+0000 7fc481b80740 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7ca02e0 0x5568a7ca2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.270+0000 7fc481b80740 4 client.26059 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.270+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5568a7b94800 con 0x5568a7ca02e0 2024-04-24T23:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.270+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7ca02e0 0x5568a7ca2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.271+0000 7fc47cb2c640 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7ca02e0 0x5568a7ca2700 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5718c44cdc85b5 server_cookie=e4807a4bcbbbb8d8 in_seq=0 out_seq=0 2024-04-24T23:17:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.271+0000 7fc457fff640 10 client.26059 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.273+0000 7fc457fff640 1 -- 172.21.15.33:0/404131981 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5568a7b94800 con 0x5568a7ca02e0 2024-04-24T23:17:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.273+0000 7fc457fff640 10 client.26059 handle_command_reply: tid=0 2024-04-24T23:17:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.273+0000 7fc481b80740 1 client.26059 shutdown 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 msgr2=0x5568a7c9e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c9e190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc4700099b0 tx=0x7fc470011770 comp rx=0 tx=0).stop 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 shutdown_connections 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5568a7ca02e0 0x5568a7ca2700 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568a7c298f0 0x5568a7c9e190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568a7c24780 0x5568a7c32aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 --2- 172.21.15.33:0/404131981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568a7c23d90 0x5568a7c32560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 >> 172.21.15.33:0/404131981 conn(0x5568a7beb950 msgr2=0x5568a7c2e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc457fff640 0 client.26059 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 shutdown_connections 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 1 -- 172.21.15.33:0/404131981 wait complete. 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.274+0000 7fc481b80740 20 client.26059 trim_cache size 0 max 16384 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:17:02.277 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "911c64a4-d3a9-44b3-a02b-a0005ffe2a9e", 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.276+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4600a2c10 msgr2=0x7fc4600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.276+0000 7fc47ed90640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4600a2c10 0x7fc4600a50d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc47000fc90 tx=0x7fc47001e000 comp rx=0 tx=0).stop 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.276+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 msgr2=0x7fc478197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.276+0000 7fc47ed90640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 0x7fc478197ac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e9c0 tx=0x7fc47400ee80 comp rx=0 tx=0).stop 2024-04-24T23:17:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.277+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 shutdown_connections 2024-04-24T23:17:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.277+0000 7fc47ed90640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4600a2c10 0x7fc4600a50d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.277+0000 7fc47ed90640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc478103340 0x7fc47819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.277+0000 7fc47ed90640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc478102980 0x7fc478198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.277+0000 7fc47ed90640 1 --2- 172.21.15.33:0/2067848129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478101790 0x7fc478197ac0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.277+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 >> 172.21.15.33:0/2067848129 conn(0x7fc4780fcd20 msgr2=0x7fc478105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.279+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 shutdown_connections 2024-04-24T23:17:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.279+0000 7fc47ed90640 1 -- 172.21.15.33:0/2067848129 wait complete. 2024-04-24T23:17:02.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:02.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 -- 172.21.15.33:0/3678501679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c00101890 msgr2=0x7f1c00101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 --2- 172.21.15.33:0/3678501679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c00101890 0x7f1c00101d10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009900 tx=0x7f1bf002a410 comp rx=0 tx=0).stop 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 -- 172.21.15.33:0/3678501679 shutdown_connections 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 --2- 172.21.15.33:0/3678501679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c00102250 0x7f1c00108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 --2- 172.21.15.33:0/3678501679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c00101890 0x7f1c00101d10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 --2- 172.21.15.33:0/3678501679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c001006a0 0x7f1c00100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 -- 172.21.15.33:0/3678501679 >> 172.21.15.33:0/3678501679 conn(0x7f1c000fbc10 msgr2=0x7f1c000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 -- 172.21.15.33:0/3678501679 shutdown_connections 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 -- 172.21.15.33:0/3678501679 wait complete. 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 Processor -- start 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.412+0000 7f1c08509640 1 -- start start 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 0x7f1c00197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c00101890 0x7f1c00198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c00102250 0x7f1c0019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0010c370 con 0x7f1c001006a0 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0010c1f0 con 0x7f1c00101890 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0010c4f0 con 0x7f1c00102250 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 0x7f1c00197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 0x7f1c00197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48540/0 (socket says 172.21.15.33:48540) 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 -- 172.21.15.33:0/2767036614 learned_addr learned my addr 172.21.15.33:0/2767036614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c00101890 0x7f1c00198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c00102250 0x7f1c0019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 -- 172.21.15.33:0/2767036614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c00102250 msgr2=0x7f1c0019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c00102250 0x7f1c0019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 -- 172.21.15.33:0/2767036614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c00101890 msgr2=0x7f1c00198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c00101890 0x7f1c00198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 -- 172.21.15.33:0/2767036614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0019f6f0 con 0x7f1c001006a0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c00102250 0x7f1c0019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c062a5640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 0x7f1c00197ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00d950 tx=0x7f1bfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1bf77fe640 1 -- 172.21.15.33:0/2767036614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc017070 con 0x7f1c001006a0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1bf77fe640 1 -- 172.21.15.33:0/2767036614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc009c30 con 0x7f1c001006a0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0019f9e0 con 0x7f1c001006a0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1bf77fe640 1 -- 172.21.15.33:0/2767036614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00f600 con 0x7f1c001006a0 2024-04-24T23:17:02.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.413+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0019ff00 con 0x7f1c001006a0 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.416+0000 7f1bf77fe640 1 -- 172.21.15.33:0/2767036614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1bfc01a070 con 0x7f1c001006a0 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.416+0000 7f1c0b215740 1 Processor -- start 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.416+0000 7f1c0b215740 1 -- start start 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.416+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff2723a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2728b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 0x55eff272b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c0b215740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eff26836b0 con 0x55eff27291b0 2024-04-24T23:17:02.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c0b215740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eff262b880 con 0x55eff2723650 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c0b215740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eff267f670 con 0x55eff2724040 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c06aa6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff2723a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 0x55eff272b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 0x55eff272b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48548/0 (socket says 172.21.15.33:48548) 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 -- 172.21.15.33:0/639658133 learned_addr learned my addr 172.21.15.33:0/639658133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c06aa6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff2723a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44342/0 (socket says 172.21.15.33:44342) 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c062a5640 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2728b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1bf77fe640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bec0a2c10 0x7f1bec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 -- 172.21.15.33:0/639658133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 msgr2=0x55eff2728b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2728b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 -- 172.21.15.33:0/639658133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 msgr2=0x55eff2723a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff2723a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 -- 172.21.15.33:0/639658133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff24617d0 con 0x55eff27291b0 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff2723a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c062a5640 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2728b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bec0a2c10 0x7f1bec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1bf77fe640 1 -- 172.21.15.33:0/2767036614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1bfc0cee10 con 0x7f1c001006a0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.417+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 0x55eff272b640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0000c00 tx=0x7f1bf0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dd256272120347 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1bf4bfe640 1 -- 172.21.15.33:0/639658133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0035040 con 0x55eff27291b0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bec0a2c10 0x7f1bec0a50d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55eff27248a0 tx=0x7f1bf8004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1bf4bfe640 1 -- 172.21.15.33:0/639658133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf0005520 con 0x55eff27291b0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1bf4bfe640 1 -- 172.21.15.33:0/639658133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0031910 con 0x55eff27291b0 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 -- 172.21.15.33:0/639658133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 msgr2=0x55eff272b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 0x55eff272b640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0000c00 tx=0x7f1bf0004170 comp rx=0 tx=0).stop 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 -- 172.21.15.33:0/639658133 shutdown_connections 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff27291b0 0x55eff272b640 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2728b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 --2- 172.21.15.33:0/639658133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff2723a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 -- 172.21.15.33:0/639658133 >> 172.21.15.33:0/639658133 conn(0x55eff271a8b0 msgr2=0x55eff271acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.418+0000 7f1c0b215740 1 -- 172.21.15.33:0/639658133 shutdown_connections 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 -- 172.21.15.33:0/639658133 wait complete. 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 -- start start 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 Processor -- start 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff272da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2724040 0x55eff272dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff2734ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eff272c100 con 0x55eff2723650 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eff263ef50 con 0x55eff2724040 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eff272c280 con 0x55eff27291b0 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c0b215740 10 client.0 _ll_register_callbacks cb 0x55eff267a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff272da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff272da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48564/0 (socket says 172.21.15.33:48564) 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c05aa4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2724040 0x55eff272dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 -- 172.21.15.33:0/4036604652 learned_addr learned my addr 172.21.15.33:0/4036604652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff2734ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 -- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 msgr2=0x55eff2734ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff2734ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 -- 172.21.15.33:0/4036604652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2724040 msgr2=0x55eff272dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.419+0000 7f1c062a5640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2724040 0x55eff272dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c062a5640 1 -- 172.21.15.33:0/4036604652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff23973f0 con 0x55eff2723650 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff2734ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2724040 0x55eff272dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c062a5640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff272da70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09af80 tx=0x7f1bfc09abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5b700d3cc71435 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1be9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1be9ffb640 1 -- 172.21.15.33:0/4036604652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc021070 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1be9ffb640 1 -- 172.21.15.33:0/4036604652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1bfc0978f0 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf0005520 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1be9ffb640 1 -- 172.21.15.33:0/4036604652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc09b280 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 10 client.16527 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eff23570f0 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1be9ffb640 1 -- 172.21.15.33:0/4036604652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf0005520 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1be9ffb640 1 -- 172.21.15.33:0/4036604652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1bfc0bb1d0 con 0x55eff2723650 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 10 client.16527 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:02.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 20 client.16527 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff2739970 0x55eff273be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 4 client.16527 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.420+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eff2690bf0 con 0x55eff2739970 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.421+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff2739970 0x55eff273be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.421+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff2739970 0x55eff273be00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e993ca22252bd74 server_cookie=ffa55a9e032f25af in_seq=0 out_seq=0 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.421+0000 7f1be9ffb640 10 client.16527 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1be9ffb640 1 -- 172.21.15.33:0/4036604652 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eff2690bf0 con 0x55eff2739970 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1be9ffb640 10 client.16527 handle_command_reply: tid=0 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 client.16527 shutdown 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 msgr2=0x55eff272da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff272da70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09af80 tx=0x7f1bfc09abe0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 shutdown_connections 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff2739970 0x55eff273be00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff2734ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2724040 0x55eff272dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1be9ffb640 0 client.16527 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 --2- 172.21.15.33:0/4036604652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff272da70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 >> 172.21.15.33:0/4036604652 conn(0x55eff271a8b0 msgr2=0x55eff2719af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 shutdown_connections 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 1 -- 172.21.15.33:0/4036604652 wait complete. 2024-04-24T23:17:02.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.422+0000 7f1c0b215740 20 client.16527 trim_cache size 0 max 16384 2024-04-24T23:17:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 Processor -- start 2024-04-24T23:17:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 -- start start 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff272df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 0x55eff2734f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff271ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001aa9e0 con 0x55eff2724040 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a82b0 con 0x55eff2723650 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c0b215740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a6100 con 0x55eff27291b0 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c05aa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 0x55eff2734f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c05aa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 0x55eff2734f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48574/0 (socket says 172.21.15.33:48574) 2024-04-24T23:17:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c05aa4640 1 -- 172.21.15.33:0/2863963686 learned_addr learned my addr 172.21.15.33:0/2863963686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 455473 12.64 MB/sec execute 67 sec latency 25.860 ms 2024-04-24T23:17:02.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c062a5640 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff272df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.441+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff271ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1c05aa4640 1 -- 172.21.15.33:0/2863963686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 msgr2=0x55eff271ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff271ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1c05aa4640 1 -- 172.21.15.33:0/2863963686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 msgr2=0x55eff272df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff272df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1c05aa4640 1 -- 172.21.15.33:0/2863963686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff271a450 con 0x55eff2724040 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 0x55eff2734f80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0043000 tx=0x7f1bf00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5b0d593627050f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1bebfff640 1 -- 172.21.15.33:0/2863963686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0058070 con 0x55eff2724040 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1bebfff640 1 -- 172.21.15.33:0/2863963686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf0031d90 con 0x55eff2724040 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.442+0000 7f1bebfff640 1 -- 172.21.15.33:0/2863963686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0035040 con 0x55eff2724040 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- 172.21.15.33:0/2863963686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 msgr2=0x55eff2734f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 0x55eff2734f80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0043000 tx=0x7f1bf00315b0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- 172.21.15.33:0/2863963686 shutdown_connections 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff27291b0 0x55eff271ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2724040 0x55eff2734f80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- 172.21.15.33:0/2863963686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff2723650 0x55eff272df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- 172.21.15.33:0/2863963686 >> 172.21.15.33:0/2863963686 conn(0x55eff271a8b0 msgr2=0x55eff2719260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- 172.21.15.33:0/2863963686 shutdown_connections 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- 172.21.15.33:0/2863963686 wait complete. 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- start start 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 Processor -- start 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff2732b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2733070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff27291b0 0x55eff2734010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a6770 con 0x55eff2723650 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a7be0 con 0x55eff27291b0 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a6e40 con 0x55eff2724040 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.443+0000 7f1c0b215740 10 client.0 _ll_register_callbacks cb 0x7f1bfc09b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff2732b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff2732b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48576/0 (socket says 172.21.15.33:48576) 2024-04-24T23:17:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 -- 172.21.15.33:0/3013448769 learned_addr learned my addr 172.21.15.33:0/3013448769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c06aa6640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff27291b0 0x55eff2734010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2733070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 -- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 msgr2=0x55eff2733070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2733070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 -- 172.21.15.33:0/3013448769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff27291b0 msgr2=0x55eff2734010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff27291b0 0x55eff2734010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 -- 172.21.15.33:0/3013448769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff2734600 con 0x55eff2723650 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1be9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c062a5640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff2732b30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09a720 tx=0x7f1bfc0bb490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad088dfce52440d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1be9ffb640 1 -- 172.21.15.33:0/3013448769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc021070 con 0x55eff2723650 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1be9ffb640 1 -- 172.21.15.33:0/3013448769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1bfc09a1c0 con 0x55eff2723650 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf0031d90 con 0x55eff2723650 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1be9ffb640 1 -- 172.21.15.33:0/3013448769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc009de0 con 0x55eff2723650 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c0b215740 10 client.16533 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.444+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eff27a03b0 con 0x55eff2723650 2024-04-24T23:17:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1be9ffb640 1 -- 172.21.15.33:0/3013448769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf0031d90 con 0x55eff2723650 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1be9ffb640 1 -- 172.21.15.33:0/3013448769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1bfc096fd0 con 0x55eff2723650 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 10 client.16533 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 20 client.16533 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff27a0d50 0x55eff27a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 4 client.16533 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eff268af40 con 0x55eff27a0d50 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff27a0d50 0x55eff27a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c05aa4640 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff27a0d50 0x55eff27a3170 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e2d804e81e5f24 server_cookie=1876e09ee74ea5b5 in_seq=0 out_seq=0 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1be9ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1be9ffb640 1 -- 172.21.15.33:0/3013448769 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eff268af40 con 0x55eff27a0d50 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1be9ffb640 10 client.16533 handle_command_reply: tid=0 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.445+0000 7f1c0b215740 1 client.16533 shutdown 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 msgr2=0x55eff2732b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff2732b30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09a720 tx=0x7f1bfc0bb490 comp rx=0 tx=0).stop 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 shutdown_connections 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eff27a0d50 0x55eff27a3170 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eff27291b0 0x55eff2734010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eff2724040 0x55eff2733070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1be9ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 --2- 172.21.15.33:0/3013448769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eff2723650 0x55eff2732b30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 >> 172.21.15.33:0/3013448769 conn(0x55eff271a8b0 msgr2=0x55eff272dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 shutdown_connections 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 1 -- 172.21.15.33:0/3013448769 wait complete. 2024-04-24T23:17:02.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c0b215740 20 client.16533 trim_cache size 0 max 16384 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: "911c64a4-d3a9-44b3-a02b-a0005ffe2a9e": { 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "911c64a4-d3a9-44b3-a02b-a0005ffe2a9e", 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:17:02.456 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bec0a2c10 msgr2=0x7f1bec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bec0a2c10 0x7f1bec0a50d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55eff27248a0 tx=0x7f1bf8004a40 comp rx=0 tx=0).stop 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 msgr2=0x7f1c00197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 0x7f1c00197ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00d950 tx=0x7f1bfc00de10 comp rx=0 tx=0).stop 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 shutdown_connections 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bec0a2c10 0x7f1bec0a50d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c00102250 0x7f1c0019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c00101890 0x7f1c00198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 --2- 172.21.15.33:0/2767036614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c001006a0 0x7f1c00197ad0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 >> 172.21.15.33:0/2767036614 conn(0x7f1c000fbc10 msgr2=0x7f1c000fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 shutdown_connections 2024-04-24T23:17:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:02.446+0000 7f1c08509640 1 -- 172.21.15.33:0/2767036614 wait complete. 2024-04-24T23:17:02.459 INFO:tasks.cephfs.filesystem:scrub status for tag:911c64a4-d3a9-44b3-a02b-a0005ffe2a9e - {'path': '/', 'tag': '911c64a4-d3a9-44b3-a02b-a0005ffe2a9e', 'options': 'recursive,force'} 2024-04-24T23:17:03.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458190 12.66 MB/sec execute 68 sec latency 16.453 ms 2024-04-24T23:17:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: pgmap v502: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 305 op/s 2024-04-24T23:17:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:17:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:17:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:17:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:03 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: pgmap v502: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 305 op/s 2024-04-24T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:17:03.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:17:03.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:03 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: pgmap v502: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 305 op/s 2024-04-24T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:17:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:03 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:04.443 INFO:tasks.workunit.client.0.smithi033.stdout: 1 459947 12.86 MB/sec execute 69 sec latency 73.355 ms 2024-04-24T23:17:05.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 462272 12.87 MB/sec execute 70 sec latency 36.829 ms 2024-04-24T23:17:05.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:05 smithi038 ceph-mon[32450]: pgmap v503: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 256 op/s 2024-04-24T23:17:05.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:05 smithi117 ceph-mon[32529]: pgmap v503: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 256 op/s 2024-04-24T23:17:05.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:05 smithi033 ceph-mon[29201]: pgmap v503: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 256 op/s 2024-04-24T23:17:06.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 464911 12.87 MB/sec execute 71 sec latency 13.824 ms 2024-04-24T23:17:07.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 466854 12.85 MB/sec execute 72 sec latency 12.775 ms 2024-04-24T23:17:07.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:07 smithi038 ceph-mon[32450]: pgmap v504: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-04-24T23:17:07.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:07 smithi117 ceph-mon[32529]: pgmap v504: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-04-24T23:17:07.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:07 smithi033 ceph-mon[29201]: pgmap v504: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-04-24T23:17:08.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469550 12.87 MB/sec execute 73 sec latency 19.148 ms 2024-04-24T23:17:09.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 472658 12.93 MB/sec execute 74 sec latency 20.025 ms 2024-04-24T23:17:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:09 smithi038 ceph-mon[32450]: pgmap v505: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-04-24T23:17:09.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:09 smithi117 ceph-mon[32529]: pgmap v505: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-04-24T23:17:09.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:09 smithi033 ceph-mon[29201]: pgmap v505: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-04-24T23:17:10.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 474045 12.85 MB/sec execute 75 sec latency 59.234 ms 2024-04-24T23:17:11.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476707 12.86 MB/sec execute 76 sec latency 11.636 ms 2024-04-24T23:17:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:11 smithi038 ceph-mon[32450]: pgmap v506: 129 pgs: 129 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-04-24T23:17:11.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:11 smithi117 ceph-mon[32529]: pgmap v506: 129 pgs: 129 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-04-24T23:17:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:11 smithi033 ceph-mon[29201]: pgmap v506: 129 pgs: 129 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-04-24T23:17:12.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479897 12.94 MB/sec execute 77 sec latency 12.678 ms 2024-04-24T23:17:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:12 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:17:12.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:12 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:17:12.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:12 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:17:13.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 482642 12.94 MB/sec execute 78 sec latency 16.078 ms 2024-04-24T23:17:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:13 smithi038 ceph-mon[32450]: pgmap v507: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-04-24T23:17:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:17:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:13 smithi117 ceph-mon[32529]: pgmap v507: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-04-24T23:17:13.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:17:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:13 smithi033 ceph-mon[29201]: pgmap v507: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-04-24T23:17:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:17:14.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 484477 12.89 MB/sec execute 79 sec latency 30.521 ms 2024-04-24T23:17:15.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 487147 12.94 MB/sec execute 80 sec latency 152.719 ms 2024-04-24T23:17:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:15 smithi038 ceph-mon[32450]: pgmap v508: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-04-24T23:17:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:15 smithi117 ceph-mon[32529]: pgmap v508: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-04-24T23:17:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:15 smithi033 ceph-mon[29201]: pgmap v508: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-04-24T23:17:16.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489654 12.92 MB/sec execute 81 sec latency 39.890 ms 2024-04-24T23:17:17.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 491560 12.88 MB/sec execute 82 sec latency 19.791 ms 2024-04-24T23:17:17.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:17 smithi038 ceph-mon[32450]: pgmap v509: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:17:17.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:17 smithi117 ceph-mon[32529]: pgmap v509: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:17:17.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:17 smithi033 ceph-mon[29201]: pgmap v509: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:17:18.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 494407 12.93 MB/sec execute 83 sec latency 8.117 ms 2024-04-24T23:17:19.444 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497393 12.98 MB/sec execute 84 sec latency 14.187 ms 2024-04-24T23:17:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:19 smithi038 ceph-mon[32450]: pgmap v510: 129 pgs: 129 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 3.2 MiB/s wr, 239 op/s 2024-04-24T23:17:19.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:19 smithi117 ceph-mon[32529]: pgmap v510: 129 pgs: 129 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 3.2 MiB/s wr, 239 op/s 2024-04-24T23:17:19.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:19 smithi033 ceph-mon[29201]: pgmap v510: 129 pgs: 129 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 3.2 MiB/s wr, 239 op/s 2024-04-24T23:17:20.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 498991 12.91 MB/sec execute 85 sec latency 43.690 ms 2024-04-24T23:17:21.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 501548 12.92 MB/sec execute 86 sec latency 68.920 ms 2024-04-24T23:17:21.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:21 smithi038 ceph-mon[32450]: pgmap v511: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.7 MiB/s wr, 207 op/s 2024-04-24T23:17:21.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:21 smithi117 ceph-mon[32529]: pgmap v511: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.7 MiB/s wr, 207 op/s 2024-04-24T23:17:21.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:21 smithi033 ceph-mon[29201]: pgmap v511: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.7 MiB/s wr, 207 op/s 2024-04-24T23:17:22.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 504583 12.98 MB/sec execute 87 sec latency 42.048 ms 2024-04-24T23:17:23.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 507072 12.94 MB/sec execute 88 sec latency 13.203 ms 2024-04-24T23:17:23.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:23 smithi038 ceph-mon[32450]: pgmap v512: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-04-24T23:17:23.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:23 smithi117 ceph-mon[32529]: pgmap v512: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-04-24T23:17:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:23 smithi033 ceph-mon[29201]: pgmap v512: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-04-24T23:17:24.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 509142 12.94 MB/sec execute 89 sec latency 19.502 ms 2024-04-24T23:17:25.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 512192 12.99 MB/sec execute 90 sec latency 7.014 ms 2024-04-24T23:17:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:25 smithi038 ceph-mon[32450]: pgmap v513: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-04-24T23:17:25.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:25 smithi117 ceph-mon[32529]: pgmap v513: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-04-24T23:17:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:25 smithi033 ceph-mon[29201]: pgmap v513: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-04-24T23:17:26.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 514603 12.98 MB/sec execute 91 sec latency 92.104 ms 2024-04-24T23:17:27.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516819 12.97 MB/sec execute 92 sec latency 13.351 ms 2024-04-24T23:17:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:27 smithi038 ceph-mon[32450]: pgmap v514: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-04-24T23:17:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:27 smithi117 ceph-mon[32529]: pgmap v514: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-04-24T23:17:27.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:27 smithi033 ceph-mon[29201]: pgmap v514: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-04-24T23:17:28.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 519410 12.98 MB/sec execute 93 sec latency 73.492 ms 2024-04-24T23:17:29.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522439 13.04 MB/sec execute 94 sec latency 77.468 ms 2024-04-24T23:17:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:29 smithi038 ceph-mon[32450]: pgmap v515: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-04-24T23:17:29.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:29 smithi117 ceph-mon[32529]: pgmap v515: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-04-24T23:17:29.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:29 smithi033 ceph-mon[29201]: pgmap v515: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-04-24T23:17:30.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 524930 13.02 MB/sec execute 95 sec latency 12.400 ms 2024-04-24T23:17:31.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 526802 12.99 MB/sec execute 96 sec latency 81.346 ms 2024-04-24T23:17:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:31 smithi038 ceph-mon[32450]: pgmap v516: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-04-24T23:17:31.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:31 smithi117 ceph-mon[32529]: pgmap v516: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-04-24T23:17:31.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:31 smithi033 ceph-mon[29201]: pgmap v516: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-04-24T23:17:32.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529771 13.04 MB/sec execute 97 sec latency 17.088 ms 2024-04-24T23:17:32.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:32.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 -- 172.21.15.33:0/2656818599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508101790 msgr2=0x7fb508101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 --2- 172.21.15.33:0/2656818599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508101790 0x7fb508101bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009900 tx=0x7fb4fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:17:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 -- 172.21.15.33:0/2656818599 shutdown_connections 2024-04-24T23:17:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 --2- 172.21.15.33:0/2656818599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508103340 0x7fb508109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 --2- 172.21.15.33:0/2656818599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 0x7fb508102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 --2- 172.21.15.33:0/2656818599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508101790 0x7fb508101bb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 -- 172.21.15.33:0/2656818599 >> 172.21.15.33:0/2656818599 conn(0x7fb5080fcd20 msgr2=0x7fb5080ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 -- 172.21.15.33:0/2656818599 shutdown_connections 2024-04-24T23:17:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.566+0000 7fb50f03f640 1 -- 172.21.15.33:0/2656818599 wait complete. 2024-04-24T23:17:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 Processor -- start 2024-04-24T23:17:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 -- start start 2024-04-24T23:17:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508101790 0x7fb5080754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 0x7fb508073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 0x7fb508074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb50810d3e0 con 0x7fb508103340 2024-04-24T23:17:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb50810d260 con 0x7fb508101790 2024-04-24T23:17:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50f03f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb50810d560 con 0x7fb508102980 2024-04-24T23:17:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 0x7fb508074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508101790 0x7fb5080754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 0x7fb508074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39702/0 (socket says 172.21.15.33:39702) 2024-04-24T23:17:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 -- 172.21.15.33:0/1001391400 learned_addr learned my addr 172.21.15.33:0/1001391400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50cddb640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508101790 0x7fb5080754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38568/0 (socket says 172.21.15.33:38568) 2024-04-24T23:17:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb507fff640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 0x7fb508073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 -- 172.21.15.33:0/1001391400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 msgr2=0x7fb508073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 0x7fb508073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 -- 172.21.15.33:0/1001391400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508101790 msgr2=0x7fb5080754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508101790 0x7fb5080754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb50d5dc640 1 -- 172.21.15.33:0/1001391400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb508074900 con 0x7fb508103340 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.567+0000 7fb507fff640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 0x7fb508073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.568+0000 7fb50d5dc640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 0x7fb508074070 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb50000eb10 tx=0x7fb50000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.568+0000 7fb505ffb640 1 -- 172.21.15.33:0/1001391400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500004b20 con 0x7fb508103340 2024-04-24T23:17:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.568+0000 7fb505ffb640 1 -- 172.21.15.33:0/1001391400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500004cc0 con 0x7fb508103340 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.568+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5081a1bd0 con 0x7fb508103340 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.568+0000 7fb505ffb640 1 -- 172.21.15.33:0/1001391400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500010760 con 0x7fb508103340 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.568+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081a2120 con 0x7fb508103340 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.570+0000 7fb505ffb640 1 -- 172.21.15.33:0/1001391400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb500005020 con 0x7fb508103340 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.570+0000 7fb511e2f740 1 Processor -- start 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 -- start start 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3904d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c39052730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c39055230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562c38fb16b0 con 0x562c3904dc30 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c38f59880 con 0x562c3904d240 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb511e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562c38fad670 con 0x562c39052da0 2024-04-24T23:17:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c39052730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3904d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c39052730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39706/0 (socket says 172.21.15.33:39706) 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb50cddb640 1 -- 172.21.15.33:0/621376956 learned_addr learned my addr 172.21.15.33:0/621376956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb505ffb640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb4f80a2c10 0x7fb4f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c39055230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb50d5dc640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb4f80a2c10 0x7fb4f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 -- 172.21.15.33:0/621376956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 msgr2=0x562c39055230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c39055230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 -- 172.21.15.33:0/621376956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 msgr2=0x562c39052730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c39052730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 -- 172.21.15.33:0/621376956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c38d8f7d0 con 0x562c3904d240 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb50cddb640 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c39052730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb505ffb640 1 -- 172.21.15.33:0/1001391400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5000cf9f0 con 0x7fb508103340 2024-04-24T23:17:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.571+0000 7fb507fff640 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3904d660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005fd0 tx=0x7fb4f400b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2e96334253d0dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb4f2ffd640 1 -- 172.21.15.33:0/621376956 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004020 con 0x562c3904d240 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb50d5dc640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb4f80a2c10 0x7fb4f80a50d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb500098320 tx=0x7fb50006bd70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb4f2ffd640 1 -- 172.21.15.33:0/621376956 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f400ee10 con 0x562c3904d240 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb4f2ffd640 1 -- 172.21.15.33:0/621376956 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4003830 con 0x562c3904d240 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb511e2f740 1 -- 172.21.15.33:0/621376956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 msgr2=0x562c3904d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb511e2f740 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3904d660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005fd0 tx=0x7fb4f400b460 comp rx=0 tx=0).stop 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb511e2f740 1 -- 172.21.15.33:0/621376956 shutdown_connections 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.572+0000 7fb511e2f740 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c39055230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c39052730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 --2- 172.21.15.33:0/621376956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3904d660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 -- 172.21.15.33:0/621376956 >> 172.21.15.33:0/621376956 conn(0x562c390488b0 msgr2=0x562c39048cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 -- 172.21.15.33:0/621376956 shutdown_connections 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 -- 172.21.15.33:0/621376956 wait complete. 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 -- start start 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 Processor -- start 2024-04-24T23:17:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904d240 0x562c39057660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c39057ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562c39055d50 con 0x562c3904d240 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.573+0000 7fb511e2f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c38f6cf50 con 0x562c3904dc30 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb511e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562c39055ed0 con 0x562c39052da0 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb511e2f740 10 client.0 _ll_register_callbacks cb 0x562c38fa8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904d240 0x562c39057660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904d240 0x562c39057660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39716/0 (socket says 172.21.15.33:39716) 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb507fff640 1 -- 172.21.15.33:0/3193084343 learned_addr learned my addr 172.21.15.33:0/3193084343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c39057ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb507fff640 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 -- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 msgr2=0x562c3905ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 -- 172.21.15.33:0/3193084343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904d240 msgr2=0x562c39057660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904d240 0x562c39057660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 -- 172.21.15.33:0/3193084343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c38cc53f0 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb50cddb640 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c39057ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0314e0 tx=0x7fb4fc0318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee04cc1341fa2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.574+0000 7fb4e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3193084343 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc035040 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3193084343 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb4fc004040 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f400ee10 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3193084343 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc04e050 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 10 client.26075 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c38c850f0 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3193084343 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f400ee10 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3193084343 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb4fc03d4e0 con 0x562c3904dc30 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 10 client.26075 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 20 client.26075 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c39063440 0x562c39065890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 4 client.26075 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c38fbebf0 con 0x562c39063440 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.575+0000 7fb507fff640 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c39063440 0x562c39065890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.576+0000 7fb4e7fff640 10 client.26075 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.576+0000 7fb507fff640 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c39063440 0x562c39065890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fcf77359c00874 server_cookie=50803d65e47e4aa8 in_seq=0 out_seq=0 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.576+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3193084343 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c38fbebf0 con 0x562c39063440 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.576+0000 7fb4e7fff640 10 client.26075 handle_command_reply: tid=0 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 client.26075 shutdown 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 msgr2=0x562c39057ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c39057ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0314e0 tx=0x7fb4fc0318e0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 shutdown_connections 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c39063440 0x562c39065890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb4e7fff640 0 client.26075 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c39057ba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 --2- 172.21.15.33:0/3193084343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904d240 0x562c39057660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 >> 172.21.15.33:0/3193084343 conn(0x562c390488b0 msgr2=0x562c39047af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 shutdown_connections 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 1 -- 172.21.15.33:0/3193084343 wait complete. 2024-04-24T23:17:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.577+0000 7fb511e2f740 20 client.26075 trim_cache size 0 max 16384 2024-04-24T23:17:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.597+0000 7fb511e2f740 1 Processor -- start 2024-04-24T23:17:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.597+0000 7fb511e2f740 1 -- start start 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.597+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 0x562c39043980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.597+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c3905eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.597+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c39052da0 0x562c3904b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb511e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081aeea0 con 0x562c39052da0 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb511e2f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081ac770 con 0x562c3904dc30 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb511e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081aa5c0 con 0x562c3904d240 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 0x562c39043980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c3905eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 0x562c39043980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52066/0 (socket says 172.21.15.33:52066) 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 -- 172.21.15.33:0/274034528 learned_addr learned my addr 172.21.15.33:0/274034528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c39052da0 0x562c3904b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 -- 172.21.15.33:0/274034528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 msgr2=0x562c3905eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c3905eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 -- 172.21.15.33:0/274034528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c39052da0 msgr2=0x562c3904b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c39052da0 0x562c3904b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 -- 172.21.15.33:0/274034528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3903a090 con 0x562c3904d240 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb507fff640 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 0x562c39043980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005fd0 tx=0x7fb4f4005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10316524f5097df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb4f27fc640 1 -- 172.21.15.33:0/274034528 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004020 con 0x562c3904d240 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb4f27fc640 1 -- 172.21.15.33:0/274034528 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f400ee10 con 0x562c3904d240 2024-04-24T23:17:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.598+0000 7fb4f27fc640 1 -- 172.21.15.33:0/274034528 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4005300 con 0x562c3904d240 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 -- 172.21.15.33:0/274034528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 msgr2=0x562c39043980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 0x562c39043980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005fd0 tx=0x7fb4f4005d20 comp rx=0 tx=0).stop 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 -- 172.21.15.33:0/274034528 shutdown_connections 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c39052da0 0x562c3904b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904dc30 0x562c3905eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 --2- 172.21.15.33:0/274034528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c3904d240 0x562c39043980 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 -- 172.21.15.33:0/274034528 >> 172.21.15.33:0/274034528 conn(0x562c390488b0 msgr2=0x562c3904f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 -- 172.21.15.33:0/274034528 shutdown_connections 2024-04-24T23:17:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 -- 172.21.15.33:0/274034528 wait complete. 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.599+0000 7fb511e2f740 1 -- start start 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 Processor -- start 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3905c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c3905c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081aac30 con 0x562c3904dc30 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081ac0a0 con 0x562c3904d240 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081ab300 con 0x562c39052da0 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb511e2f740 10 client.0 _ll_register_callbacks cb 0x7fb4fc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c3905c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb507fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3905c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c3905c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39734/0 (socket says 172.21.15.33:39734) 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 -- 172.21.15.33:0/2145625363 learned_addr learned my addr 172.21.15.33:0/2145625363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb507fff640 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 -- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 msgr2=0x562c3905d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 -- 172.21.15.33:0/2145625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 msgr2=0x562c3905c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3905c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.600+0000 7fb50cddb640 1 -- 172.21.15.33:0/2145625363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3905ddd0 con 0x562c3904dc30 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb50cddb640 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c3905c7d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc02a930 tx=0x7fb4fc032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9e764b86593122 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb4e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb4e7fff640 1 -- 172.21.15.33:0/2145625363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc057060 con 0x562c3904dc30 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb4e7fff640 1 -- 172.21.15.33:0/2145625363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb4fc03dcd0 con 0x562c3904dc30 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f400ee10 con 0x562c3904dc30 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb4e7fff640 1 -- 172.21.15.33:0/2145625363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc05e050 con 0x562c3904dc30 2024-04-24T23:17:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 10 client.16551 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c390c9a70 con 0x562c3904dc30 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb4e7fff640 1 -- 172.21.15.33:0/2145625363 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f400ee10 con 0x562c3904dc30 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb4e7fff640 1 -- 172.21.15.33:0/2145625363 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb4fc0038f0 con 0x562c3904dc30 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 10 client.16551 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 20 client.16551 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c390ca440 0x562c390cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 4 client.16551 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.601+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c38fb8f40 con 0x562c390ca440 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb507fff640 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c390ca440 0x562c390cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb507fff640 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c390ca440 0x562c390cc860 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b325a06509f6d125 server_cookie=58cfebed91d61cd9 in_seq=0 out_seq=0 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb4e7fff640 10 client.16551 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb4e7fff640 1 -- 172.21.15.33:0/2145625363 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562c38fb8f40 con 0x562c390ca440 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb4e7fff640 10 client.16551 handle_command_reply: tid=0 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 client.16551 shutdown 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 msgr2=0x562c3905c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c3905c7d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc02a930 tx=0x7fb4fc032b90 comp rx=0 tx=0).stop 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 shutdown_connections 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562c390ca440 0x562c390cc860 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c39052da0 0x562c3905d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c3904dc30 0x562c3905c7d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 --2- 172.21.15.33:0/2145625363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c3904d240 0x562c3905c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 >> 172.21.15.33:0/2145625363 conn(0x562c390488b0 msgr2=0x562c39047260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.602+0000 7fb4e7fff640 0 client.16551 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 shutdown_connections 2024-04-24T23:17:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb511e2f740 1 -- 172.21.15.33:0/2145625363 wait complete. 2024-04-24T23:17:32.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb511e2f740 20 client.16551 trim_cache size 0 max 16384 2024-04-24T23:17:32.612 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:17:32.612 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:17:32.612 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:17:32.612 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb4f80a2c10 msgr2=0x7fb4f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb4f80a2c10 0x7fb4f80a50d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb500098320 tx=0x7fb50006bd70 comp rx=0 tx=0).stop 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 msgr2=0x7fb508074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 0x7fb508074070 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb50000eb10 tx=0x7fb50000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 shutdown_connections 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb4f80a2c10 0x7fb4f80a50d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb508103340 0x7fb508074070 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb508102980 0x7fb508073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 --2- 172.21.15.33:0/1001391400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb508101790 0x7fb5080754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.603+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 >> 172.21.15.33:0/1001391400 conn(0x7fb5080fcd20 msgr2=0x7fb508105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.604+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 shutdown_connections 2024-04-24T23:17:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.604+0000 7fb50f03f640 1 -- 172.21.15.33:0/1001391400 wait complete. 2024-04-24T23:17:32.617 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:17:32.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 -- 172.21.15.33:0/1501616788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398104bb0 msgr2=0x7f9398105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 --2- 172.21.15.33:0/1501616788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398104bb0 0x7f9398105030 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9388009900 tx=0x7f938802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 -- 172.21.15.33:0/1501616788 shutdown_connections 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 --2- 172.21.15.33:0/1501616788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398105570 0x7f939810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 --2- 172.21.15.33:0/1501616788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398104bb0 0x7f9398105030 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 --2- 172.21.15.33:0/1501616788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93981039c0 0x7f9398103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 -- 172.21.15.33:0/1501616788 >> 172.21.15.33:0/1501616788 conn(0x7f93980fef30 msgr2=0x7f9398101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 -- 172.21.15.33:0/1501616788 shutdown_connections 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 -- 172.21.15.33:0/1501616788 wait complete. 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 Processor -- start 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.726+0000 7f939f73b640 1 -- start start 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981039c0 0x7f9398199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9398104bb0 0x7f939819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 0x7f93981a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f939810f640 con 0x7f9398105570 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f939810f4c0 con 0x7f93981039c0 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f939810f7c0 con 0x7f9398104bb0 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 0x7f93981a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 0x7f93981a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39756/0 (socket says 172.21.15.33:39756) 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 -- 172.21.15.33:0/760547841 learned_addr learned my addr 172.21.15.33:0/760547841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939d4d7640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981039c0 0x7f9398199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939ccd6640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9398104bb0 0x7f939819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 -- 172.21.15.33:0/760547841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9398104bb0 msgr2=0x7f939819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9398104bb0 0x7f939819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 -- 172.21.15.33:0/760547841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981039c0 msgr2=0x7f9398199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981039c0 0x7f9398199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 -- 172.21.15.33:0/760547841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93981a1950 con 0x7f9398105570 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939d4d7640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981039c0 0x7f9398199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939dcd8640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 0x7f93981a12b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f939000efd0 tx=0x7f939000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390003b70 con 0x7f9398105570 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9390003d10 con 0x7f9398105570 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93981a1c40 con 0x7f9398105570 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900109f0 con 0x7f9398105570 2024-04-24T23:17:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.727+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a2140 con 0x7f9398105570 2024-04-24T23:17:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.728+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939810ad60 con 0x7f9398105570 2024-04-24T23:17:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.729+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9390004020 con 0x7f9398105570 2024-04-24T23:17:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.730+0000 7f938e7fc640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93840a2b90 0x7f93840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.730+0000 7f939d4d7640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93840a2b90 0x7f93840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.730+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f93900cf880 con 0x7f9398105570 2024-04-24T23:17:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.731+0000 7f939d4d7640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93840a2b90 0x7f93840a5050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f939819b210 tx=0x7f9394005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.735+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f939009c450 con 0x7f9398105570 2024-04-24T23:17:32.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.980+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f939802d170 con 0x7f9398105570 2024-04-24T23:17:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.981+0000 7f938e7fc640 1 -- 172.21.15.33:0/760547841 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f939009bd50 con 0x7f9398105570 2024-04-24T23:17:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:17:32.983 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:17:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.983+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93840a2b90 msgr2=0x7f93840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.983+0000 7f939f73b640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93840a2b90 0x7f93840a5050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f939819b210 tx=0x7f9394005f90 comp rx=0 tx=0).stop 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.983+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 msgr2=0x7f93981a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.983+0000 7f939f73b640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 0x7f93981a12b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f939000efd0 tx=0x7f939000c6a0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 shutdown_connections 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93840a2b90 0x7f93840a5050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9398105570 0x7f93981a12b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9398104bb0 0x7f939819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 --2- 172.21.15.33:0/760547841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981039c0 0x7f9398199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 >> 172.21.15.33:0/760547841 conn(0x7f93980fef30 msgr2=0x7f9398107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 shutdown_connections 2024-04-24T23:17:32.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:32.984+0000 7f939f73b640 1 -- 172.21.15.33:0/760547841 wait complete. 2024-04-24T23:17:32.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:17:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 -- 172.21.15.33:0/429998870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c102250 msgr2=0x7fab0c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 --2- 172.21.15.33:0/429998870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c102250 0x7fab0c108ae0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fab04009ca0 tx=0x7fab0402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 -- 172.21.15.33:0/429998870 shutdown_connections 2024-04-24T23:17:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 --2- 172.21.15.33:0/429998870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c102250 0x7fab0c108ae0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 --2- 172.21.15.33:0/429998870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0c101890 0x7fab0c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 --2- 172.21.15.33:0/429998870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab0c1006a0 0x7fab0c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 -- 172.21.15.33:0/429998870 >> 172.21.15.33:0/429998870 conn(0x7fab0c0fbc30 msgr2=0x7fab0c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 -- 172.21.15.33:0/429998870 shutdown_connections 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 -- 172.21.15.33:0/429998870 wait complete. 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 Processor -- start 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.107+0000 7fab11b42640 1 -- start start 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab0c1006a0 0x7fab0c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 0x7fab0c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0c102250 0x7fab0c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10c300 con 0x7fab0c101890 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10c180 con 0x7fab0c102250 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10c480 con 0x7fab0c1006a0 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 0x7fab0c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 0x7fab0c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39782/0 (socket says 172.21.15.33:39782) 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 -- 172.21.15.33:0/3280253909 learned_addr learned my addr 172.21.15.33:0/3280253909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab0c1006a0 0x7fab0c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0bfff640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0c102250 0x7fab0c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 -- 172.21.15.33:0/3280253909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab0c1006a0 msgr2=0x7fab0c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab0c1006a0 0x7fab0c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 -- 172.21.15.33:0/3280253909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0c102250 msgr2=0x7fab0c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0c102250 0x7fab0c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 -- 172.21.15.33:0/3280253909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19f6c0 con 0x7fab0c101890 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab0affd640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 0x7fab0c197fe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fab0000ea60 tx=0x7fab0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab08ff9640 1 -- 172.21.15.33:0/3280253909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0000cde0 con 0x7fab0c101890 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab08ff9640 1 -- 172.21.15.33:0/3280253909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab000035f0 con 0x7fab0c101890 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19f9b0 con 0x7fab0c101890 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.108+0000 7fab08ff9640 1 -- 172.21.15.33:0/3280253909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00010670 con 0x7fab0c101890 2024-04-24T23:17:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.109+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c19ff00 con 0x7fab0c101890 2024-04-24T23:17:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab08ff9640 1 -- 172.21.15.33:0/3280253909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab000037a0 con 0x7fab0c101890 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 Processor -- start 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 -- start start 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a8a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a92220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618429ee670 con 0x561842a8a230 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561842996970 con 0x561842a8ac20 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab14855740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618429ea630 con 0x561842a8fd90 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a8a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab08ff9640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faaf40a2b90 0x7faaf40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a8f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38664/0 (socket says 172.21.15.33:38664) 2024-04-24T23:17:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab0b7fe640 1 -- 172.21.15.33:0/3788976042 learned_addr learned my addr 172.21.15.33:0/3788976042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.111+0000 7fab0affd640 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a92220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0bfff640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faaf40a2b90 0x7faaf40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0b7fe640 1 -- 172.21.15.33:0/3788976042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 msgr2=0x561842a92220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a92220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0b7fe640 1 -- 172.21.15.33:0/3788976042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 msgr2=0x561842a8a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a8a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab08ff9640 1 -- 172.21.15.33:0/3280253909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab000cf5d0 con 0x7fab0c101890 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0b7fe640 1 -- 172.21.15.33:0/3788976042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618429ba400 con 0x561842a8ac20 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a8f720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b7d0 tx=0x7faafc004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f26dac6832f848b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7fab0bfff640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faaf40a2b90 0x7faaf40a5050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab0402f540 tx=0x7fab0402a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7faaf1ffb640 1 -- 172.21.15.33:0/3788976042 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc003dd0 con 0x561842a8ac20 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7faaf1ffb640 1 -- 172.21.15.33:0/3788976042 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faafc015070 con 0x561842a8ac20 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.112+0000 7faaf1ffb640 1 -- 172.21.15.33:0/3788976042 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc011730 con 0x561842a8ac20 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 -- 172.21.15.33:0/3788976042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 msgr2=0x561842a8f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a8f720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b7d0 tx=0x7faafc004610 comp rx=0 tx=0).stop 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 -- 172.21.15.33:0/3788976042 shutdown_connections 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a92220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a8f720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 --2- 172.21.15.33:0/3788976042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a8a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 -- 172.21.15.33:0/3788976042 >> 172.21.15.33:0/3788976042 conn(0x561842a858a0 msgr2=0x561842a85ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 -- 172.21.15.33:0/3788976042 shutdown_connections 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 -- 172.21.15.33:0/3788976042 wait complete. 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.113+0000 7fab14855740 1 -- start start 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 Processor -- start 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a94390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a9b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561842a92d40 con 0x561842a8a230 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618429aa050 con 0x561842a8ac20 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561842a92ec0 con 0x561842a8fd90 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab14855740 10 client.0 _ll_register_callbacks cb 0x561842701be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a94390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a94390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39804/0 (socket says 172.21.15.33:39804) 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 -- 172.21.15.33:0/1964918754 learned_addr learned my addr 172.21.15.33:0/1964918754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0affd640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0bfff640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a9b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 -- 172.21.15.33:0/1964918754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 msgr2=0x561842a9b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a9b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 -- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 msgr2=0x561842a948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 -- 172.21.15.33:0/1964918754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56184297bdc0 con 0x561842a8a230 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0bfff640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a9b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a94390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faafc000c00 tx=0x7faafc003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a8d51a6fddeaf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.114+0000 7faaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7faaeaffd640 1 -- 172.21.15.33:0/1964918754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc01a880 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7faaeaffd640 1 -- 172.21.15.33:0/1964918754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faafc01aa20 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faafc015070 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7faaeaffd640 1 -- 172.21.15.33:0/1964918754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faafc030050 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 10 client.16569 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618427023f0 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7faaeaffd640 1 -- 172.21.15.33:0/1964918754 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faafc015070 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7faaeaffd640 1 -- 172.21.15.33:0/1964918754 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faafc027470 con 0x561842a8a230 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 10 client.16569 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 20 client.16569 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842a9fea0 0x561842aa2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 4 client.16569 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618429fbbb0 con 0x561842a9fea0 2024-04-24T23:17:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.115+0000 7fab0affd640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842a9fea0 0x561842aa2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.116+0000 7fab0affd640 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842a9fea0 0x561842aa2290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40043af6ab667704 server_cookie=cd084068fd5bbc27 in_seq=0 out_seq=0 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.116+0000 7faaeaffd640 10 client.16569 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.116+0000 7faaeaffd640 1 -- 172.21.15.33:0/1964918754 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618429fbbb0 con 0x561842a9fea0 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.116+0000 7faaeaffd640 10 client.16569 handle_command_reply: tid=0 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 client.16569 shutdown 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 msgr2=0x561842a94390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a94390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faafc000c00 tx=0x7faafc003e60 comp rx=0 tx=0).stop 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 shutdown_connections 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842a9fea0 0x561842aa2290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842a9b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8ac20 0x561842a948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7faaeaffd640 0 client.16569 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 --2- 172.21.15.33:0/1964918754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8a230 0x561842a94390 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 >> 172.21.15.33:0/1964918754 conn(0x561842a858a0 msgr2=0x561842a84b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 shutdown_connections 2024-04-24T23:17:33.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 1 -- 172.21.15.33:0/1964918754 wait complete. 2024-04-24T23:17:33.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.117+0000 7fab14855740 20 client.16569 trim_cache size 0 max 16384 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 Processor -- start 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 -- start start 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a9b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842aa6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1acce0 con 0x561842a8ac20 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1aa5b0 con 0x561842a8a230 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab14855740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1a8400 con 0x561842a8fd90 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a9b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a9b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39812/0 (socket says 172.21.15.33:39812) 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0affd640 1 -- 172.21.15.33:0/239097190 learned_addr learned my addr 172.21.15.33:0/239097190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0bfff640 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842aa6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0bfff640 1 -- 172.21.15.33:0/239097190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 msgr2=0x561842a842b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0bfff640 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a842b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0bfff640 1 -- 172.21.15.33:0/239097190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 msgr2=0x561842a9b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0bfff640 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a9b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0bfff640 1 -- 172.21.15.33:0/239097190 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618427cc7d0 con 0x561842a8fd90 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0affd640 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a9b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.133+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.134+0000 7fab0bfff640 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842aa6ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab04006010 tx=0x7fab04005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9558315b4f9f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.134+0000 7faaf17fa640 1 -- 172.21.15.33:0/239097190 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04004710 con 0x561842a8fd90 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.134+0000 7faaf17fa640 1 -- 172.21.15.33:0/239097190 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab04036070 con 0x561842a8fd90 2024-04-24T23:17:33.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.134+0000 7faaf17fa640 1 -- 172.21.15.33:0/239097190 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040308b0 con 0x561842a8fd90 2024-04-24T23:17:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 -- 172.21.15.33:0/239097190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 msgr2=0x561842aa6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842aa6ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab04006010 tx=0x7fab04005df0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 -- 172.21.15.33:0/239097190 shutdown_connections 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842aa6ae0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a9b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 --2- 172.21.15.33:0/239097190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 -- 172.21.15.33:0/239097190 >> 172.21.15.33:0/239097190 conn(0x561842a858a0 msgr2=0x561842a8bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 -- 172.21.15.33:0/239097190 shutdown_connections 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 -- 172.21.15.33:0/239097190 wait complete. 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 -- start start 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.135+0000 7fab14855740 1 Processor -- start 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a98890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a98dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842b04350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1a8a70 con 0x561842a8ac20 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1a9ee0 con 0x561842a8a230 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1a9140 con 0x561842a8fd90 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 10 client.0 _ll_register_callbacks cb 0x7fab04009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a98890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a98dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a98dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39824/0 (socket says 172.21.15.33:39824) 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a98890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38680/0 (socket says 172.21.15.33:38680) 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 -- 172.21.15.33:0/3286233579 learned_addr learned my addr 172.21.15.33:0/3286233579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0bfff640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842b04350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 -- 172.21.15.33:0/3286233579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 msgr2=0x561842b04350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842b04350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 -- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 msgr2=0x561842a98890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a98890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 -- 172.21.15.33:0/3286233579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561842b04910 con 0x561842a8ac20 2024-04-24T23:17:33.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a98890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab0affd640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a98dd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fab000ccd10 tx=0x7fab0001ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fee7b2683f659bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7faaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7faaeaffd640 1 -- 172.21.15.33:0/3286233579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000a4710 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7faaeaffd640 1 -- 172.21.15.33:0/3286233579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab00003b30 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.136+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab04036070 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7faaeaffd640 1 -- 172.21.15.33:0/3286233579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab000cf2e0 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 10 client.16575 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561842b05ce0 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7faaeaffd640 1 -- 172.21.15.33:0/3286233579 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab04036070 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7faaeaffd640 1 -- 172.21.15.33:0/3286233579 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab000ba250 con 0x561842a8ac20 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 10 client.16575 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 10 client.16575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 20 client.16575 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842b066e0 0x561842b08b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 4 client.16575 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618429f5f00 con 0x561842b066e0 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842b066e0 0x561842b08b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7faaeaffd640 10 client.16575 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.137+0000 7fab0b7fe640 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842b066e0 0x561842b08b00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5256c23232a85d4 server_cookie=be895f2c5f6c18ff in_seq=0 out_seq=0 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.138+0000 7faaeaffd640 1 -- 172.21.15.33:0/3286233579 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618429f5f00 con 0x561842b066e0 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.138+0000 7faaeaffd640 10 client.16575 handle_command_reply: tid=0 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.138+0000 7fab14855740 1 client.16575 shutdown 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.138+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 msgr2=0x561842a98dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.138+0000 7fab14855740 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a98dd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fab000ccd10 tx=0x7fab0001ffc0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 shutdown_connections 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561842b066e0 0x561842b08b00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561842a8fd90 0x561842b04350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561842a8ac20 0x561842a98dd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 --2- 172.21.15.33:0/3286233579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561842a8a230 0x561842a98890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7faaeaffd640 0 client.16575 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:17:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 >> 172.21.15.33:0/3286233579 conn(0x561842a858a0 msgr2=0x561842a8b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 shutdown_connections 2024-04-24T23:17:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 1 -- 172.21.15.33:0/3286233579 wait complete. 2024-04-24T23:17:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.139+0000 7fab14855740 20 client.16575 trim_cache size 0 max 16384 2024-04-24T23:17:33.146 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faaf40a2b90 msgr2=0x7faaf40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faaf40a2b90 0x7faaf40a5050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab0402f540 tx=0x7fab0402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 msgr2=0x7fab0c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 0x7fab0c197fe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fab0000ea60 tx=0x7fab0000ef20 comp rx=0 tx=0).stop 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 shutdown_connections 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7faaf40a2b90 0x7faaf40a5050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0c102250 0x7fab0c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c101890 0x7fab0c197fe0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 --2- 172.21.15.33:0/3280253909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab0c1006a0 0x7fab0c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 >> 172.21.15.33:0/3280253909 conn(0x7fab0c0fbc30 msgr2=0x7fab0c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 shutdown_connections 2024-04-24T23:17:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.140+0000 7fab11b42640 1 -- 172.21.15.33:0/3280253909 wait complete. 2024-04-24T23:17:33.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.261+0000 7f8271812640 1 -- 172.21.15.33:0/3526876013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c0ffa50 msgr2=0x7f826c0ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.261+0000 7f8271812640 1 --2- 172.21.15.33:0/3526876013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c0ffa50 0x7f826c0ffe70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f825c009930 tx=0x7f825c02a420 comp rx=0 tx=0).stop 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.261+0000 7f8271812640 1 -- 172.21.15.33:0/3526876013 shutdown_connections 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.261+0000 7f8271812640 1 --2- 172.21.15.33:0/3526876013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c1033e0 0x7f826c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.261+0000 7f8271812640 1 --2- 172.21.15.33:0/3526876013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f826c102a20 0x7f826c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.261+0000 7f8271812640 1 --2- 172.21.15.33:0/3526876013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c0ffa50 0x7f826c0ffe70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.262+0000 7f8271812640 1 -- 172.21.15.33:0/3526876013 >> 172.21.15.33:0/3526876013 conn(0x7f826c0fb380 msgr2=0x7f826c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.262+0000 7f8271812640 1 -- 172.21.15.33:0/3526876013 shutdown_connections 2024-04-24T23:17:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.262+0000 7f8271812640 1 -- 172.21.15.33:0/3526876013 wait complete. 2024-04-24T23:17:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.262+0000 7f8271812640 1 Processor -- start 2024-04-24T23:17:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.262+0000 7f8271812640 1 -- start start 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.262+0000 7f8271812640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c0ffa50 0x7f826c199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 0x7f826c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f826c1033e0 0x7f826c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f826c10d400 con 0x7f826c102a20 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f826c10d280 con 0x7f826c1033e0 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f826c10d580 con 0x7f826c0ffa50 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 0x7f826c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 0x7f826c19a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39832/0 (socket says 172.21.15.33:39832) 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 -- 172.21.15.33:0/2313989508 learned_addr learned my addr 172.21.15.33:0/2313989508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826b7fe640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f826c1033e0 0x7f826c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826affd640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c0ffa50 0x7f826c199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 -- 172.21.15.33:0/2313989508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c0ffa50 msgr2=0x7f826c199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c0ffa50 0x7f826c199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 -- 172.21.15.33:0/2313989508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f826c1033e0 msgr2=0x7f826c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f826c1033e0 0x7f826c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 -- 172.21.15.33:0/2313989508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c1a1940 con 0x7f826c102a20 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826affd640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c0ffa50 0x7f826c199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f826a7fc640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 0x7f826c19a2c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f826000e9f0 tx=0x7f826000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8270810640 1 -- 172.21.15.33:0/2313989508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826000cd80 con 0x7f826c102a20 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c1a1c30 con 0x7f826c102a20 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8270810640 1 -- 172.21.15.33:0/2313989508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8260010910 con 0x7f826c102a20 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8270810640 1 -- 172.21.15.33:0/2313989508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82600188f0 con 0x7f826c102a20 2024-04-24T23:17:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.263+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1a2150 con 0x7f826c102a20 2024-04-24T23:17:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f8270810640 1 -- 172.21.15.33:0/2313989508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8260010ac0 con 0x7f826c102a20 2024-04-24T23:17:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 Processor -- start 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 -- start start 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b55a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b55f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55fd90 0x56326b562220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56326b4be670 con 0x56326b55a230 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56326b466970 con 0x56326b55fd90 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.266+0000 7f82745fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56326b4ba630 con 0x56326b55ac20 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b55a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b55a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39848/0 (socket says 172.21.15.33:39848) 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 -- 172.21.15.33:0/842559133 learned_addr learned my addr 172.21.15.33:0/842559133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826a7fc640 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55fd90 0x56326b562220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826affd640 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b55f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f8270810640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f82540a2b90 0x7f82540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 -- 172.21.15.33:0/842559133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 msgr2=0x56326b55f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b55f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 -- 172.21.15.33:0/842559133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55fd90 msgr2=0x56326b562220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55fd90 0x56326b562220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 -- 172.21.15.33:0/842559133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326b48a400 con 0x56326b55a230 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f8270810640 1 -- 172.21.15.33:0/2313989508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f82600cf7f0 con 0x7f826c102a20 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f82540a2b90 0x7f82540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f826b7fe640 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b55a650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8264009fd0 tx=0x7f82640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53b2643a7b167b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.267+0000 7f8251ffb640 1 -- 172.21.15.33:0/842559133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264004250 con 0x56326b55a230 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f8251ffb640 1 -- 172.21.15.33:0/842559133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8264003db0 con 0x56326b55a230 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f8251ffb640 1 -- 172.21.15.33:0/842559133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826401bc20 con 0x56326b55a230 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f826b7fe640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f82540a2b90 0x7f82540a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8264020000 tx=0x7f8264021000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 -- 172.21.15.33:0/842559133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 msgr2=0x56326b55a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b55a650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8264009fd0 tx=0x7f82640118d0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 -- 172.21.15.33:0/842559133 shutdown_connections 2024-04-24T23:17:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55fd90 0x56326b562220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b55f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 --2- 172.21.15.33:0/842559133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b55a650 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 -- 172.21.15.33:0/842559133 >> 172.21.15.33:0/842559133 conn(0x56326b5558a0 msgr2=0x56326b555ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 -- 172.21.15.33:0/842559133 shutdown_connections 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.268+0000 7f82745fb740 1 -- 172.21.15.33:0/842559133 wait complete. 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 -- start start 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 Processor -- start 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 0x56326b5643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b5648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b56b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56326b562d40 con 0x56326b55fd90 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56326b47a050 con 0x56326b55a230 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56326b562ec0 con 0x56326b55ac20 2024-04-24T23:17:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f82745fb740 10 client.0 _ll_register_callbacks cb 0x56326b1d1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f826b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b56b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f826affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 0x56326b5643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f826b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b56b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39850/0 (socket says 172.21.15.33:39850) 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f826affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 0x56326b5643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38708/0 (socket says 172.21.15.33:38708) 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f826b7fe640 1 -- 172.21.15.33:0/721476489 learned_addr learned my addr 172.21.15.33:0/721476489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.269+0000 7f826a7fc640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b5648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826b7fe640 1 -- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 msgr2=0x56326b5648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826b7fe640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b5648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826b7fe640 1 -- 172.21.15.33:0/721476489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 msgr2=0x56326b5643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826b7fe640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 0x56326b5643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826b7fe640 1 -- 172.21.15.33:0/721476489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56326b44bdc0 con 0x56326b55fd90 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826affd640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 0x56326b5643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f826b7fe640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b56b8b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8264004410 tx=0x7f8264004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5ee1928cdc7ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f824affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f824affd640 1 -- 172.21.15.33:0/721476489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264034030 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f824affd640 1 -- 172.21.15.33:0/721476489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f826401b9a0 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8264003db0 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f824affd640 1 -- 172.21.15.33:0/721476489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f826403e050 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f82745fb740 10 client.16587 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56326b1d23f0 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.270+0000 7f824affd640 1 -- 172.21.15.33:0/721476489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8264003db0 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f824affd640 1 -- 172.21.15.33:0/721476489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f826402f920 con 0x56326b55fd90 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f82745fb740 10 client.16587 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f82745fb740 10 client.16587 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f82745fb740 20 client.16587 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f82745fb740 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b572440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f82745fb740 4 client.16587 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56326b4cbbb0 con 0x56326b56ffb0 2024-04-24T23:17:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f826affd640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b572440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f826affd640 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b572440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2004bd73ae534da server_cookie=1a398d9c881a614a in_seq=0 out_seq=0 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.271+0000 7f824affd640 10 client.16587 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f824affd640 1 -- 172.21.15.33:0/721476489 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56326b4cbbb0 con 0x56326b56ffb0 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f824affd640 10 client.16587 handle_command_reply: tid=0 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 client.16587 shutdown 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 msgr2=0x56326b56b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b56b8b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8264004410 tx=0x7f8264004a00 comp rx=0 tx=0).stop 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 shutdown_connections 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b572440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f824affd640 0 client.16587 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b56b8b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55ac20 0x56326b5648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 --2- 172.21.15.33:0/721476489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55a230 0x56326b5643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 >> 172.21.15.33:0/721476489 conn(0x56326b5558a0 msgr2=0x56326b554b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 shutdown_connections 2024-04-24T23:17:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.272+0000 7f82745fb740 1 -- 172.21.15.33:0/721476489 wait complete. 2024-04-24T23:17:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.273+0000 7f82745fb740 20 client.16587 trim_cache size 0 max 16384 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.286+0000 7f82745fb740 1 Processor -- start 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 -- start start 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 0x56326b554b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b56b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b55b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1aeec0 con 0x56326b55fd90 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ac790 con 0x56326b55ac20 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f82745fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1aa5e0 con 0x56326b55a230 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 0x56326b554b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b56b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 0x56326b554b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52212/0 (socket says 172.21.15.33:52212) 2024-04-24T23:17:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 -- 172.21.15.33:0/2895907118 learned_addr learned my addr 172.21.15.33:0/2895907118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826a7fc640 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b56b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38710/0 (socket says 172.21.15.33:38710) 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826b7fe640 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b55b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 -- 172.21.15.33:0/2895907118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 msgr2=0x56326b56b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b56b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 -- 172.21.15.33:0/2895907118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 msgr2=0x56326b55b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b55b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826affd640 1 -- 172.21.15.33:0/2895907118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326b29c7d0 con 0x56326b55a230 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.287+0000 7f826a7fc640 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b56b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.288+0000 7f826affd640 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 0x56326b554b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f825c005ab0 tx=0x7f825c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd171aeffb796886 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.288+0000 7f82517fa640 1 -- 172.21.15.33:0/2895907118 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c035040 con 0x56326b55a230 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.288+0000 7f82517fa640 1 -- 172.21.15.33:0/2895907118 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f825c046dc0 con 0x56326b55a230 2024-04-24T23:17:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.288+0000 7f82517fa640 1 -- 172.21.15.33:0/2895907118 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c005330 con 0x56326b55a230 2024-04-24T23:17:33.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 -- 172.21.15.33:0/2895907118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 msgr2=0x56326b554b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 0x56326b554b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f825c005ab0 tx=0x7f825c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 -- 172.21.15.33:0/2895907118 shutdown_connections 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55fd90 0x56326b55b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b56b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 --2- 172.21.15.33:0/2895907118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55a230 0x56326b554b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 -- 172.21.15.33:0/2895907118 >> 172.21.15.33:0/2895907118 conn(0x56326b5558a0 msgr2=0x56326b555c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 -- 172.21.15.33:0/2895907118 shutdown_connections 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 -- 172.21.15.33:0/2895907118 wait complete. 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 -- start start 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.289+0000 7f82745fb740 1 Processor -- start 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b568ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b569020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55fd90 0x56326b569f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1aac50 con 0x56326b55a230 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ac0c0 con 0x56326b55ac20 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ab320 con 0x56326b55fd90 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f82745fb740 10 client.0 _ll_register_callbacks cb 0x7f82600d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b568ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b568ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39882/0 (socket says 172.21.15.33:39882) 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826affd640 1 -- 172.21.15.33:0/4125012911 learned_addr learned my addr 172.21.15.33:0/4125012911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b569020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826b7fe640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55fd90 0x56326b569f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 -- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55fd90 msgr2=0x56326b569f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55fd90 0x56326b569f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 -- 172.21.15.33:0/4125012911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 msgr2=0x56326b568ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b568ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 -- 172.21.15.33:0/4125012911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56326b56a540 con 0x56326b55ac20 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826affd640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b568ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f826a7fc640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b569020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82600ccfb0 tx=0x7f8260099d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee4c368fe209f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f824affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.290+0000 7f824affd640 1 -- 172.21.15.33:0/4125012911 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82600a0070 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c046dc0 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f824affd640 1 -- 172.21.15.33:0/4125012911 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8260010430 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f824affd640 1 -- 172.21.15.33:0/4125012911 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82600a4930 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 10 client.26123 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56326b56b900 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f824affd640 1 -- 172.21.15.33:0/4125012911 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f825c046dc0 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f824affd640 1 -- 172.21.15.33:0/4125012911 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f826009ac30 con 0x56326b55ac20 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 10 client.26123 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 10 client.26123 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 20 client.26123 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b5d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 4 client.26123 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56326b4c5f00 con 0x56326b56ffb0 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.291+0000 7f826affd640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b5d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f824affd640 10 client.26123 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f826affd640 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b5d8900 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e22004c6bfb6aef6 server_cookie=d4fa51f6a8503fb4 in_seq=0 out_seq=0 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f824affd640 1 -- 172.21.15.33:0/4125012911 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56326b4c5f00 con 0x56326b56ffb0 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f824affd640 10 client.26123 handle_command_reply: tid=0 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 client.26123 shutdown 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 msgr2=0x56326b569020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b569020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82600ccfb0 tx=0x7f8260099d80 comp rx=0 tx=0).stop 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 shutdown_connections 2024-04-24T23:17:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56326b56ffb0 0x56326b5d8900 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56326b55fd90 0x56326b569f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f824affd640 0 client.26123 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326b55ac20 0x56326b569020 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 --2- 172.21.15.33:0/4125012911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56326b55a230 0x56326b568ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 >> 172.21.15.33:0/4125012911 conn(0x56326b5558a0 msgr2=0x56326b554b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 shutdown_connections 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 1 -- 172.21.15.33:0/4125012911 wait complete. 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.292+0000 7f82745fb740 20 client.26123 trim_cache size 0 max 16384 2024-04-24T23:17:33.300 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:17:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f82540a2b90 msgr2=0x7f82540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f82540a2b90 0x7f82540a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8264020000 tx=0x7f8264021000 comp rx=0 tx=0).stop 2024-04-24T23:17:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 msgr2=0x7f826c19a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 0x7f826c19a2c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f826000e9f0 tx=0x7f826000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 shutdown_connections 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f82540a2b90 0x7f82540a5050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f826c1033e0 0x7f826c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826c102a20 0x7f826c19a2c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 --2- 172.21.15.33:0/2313989508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c0ffa50 0x7f826c199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 >> 172.21.15.33:0/2313989508 conn(0x7f826c0fb380 msgr2=0x7f826c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 shutdown_connections 2024-04-24T23:17:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.293+0000 7f8271812640 1 -- 172.21.15.33:0/2313989508 wait complete. 2024-04-24T23:17:33.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:17:33.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 -- 172.21.15.33:0/3398964546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 msgr2=0x7fc5e0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 --2- 172.21.15.33:0/3398964546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0100aa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009900 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 -- 172.21.15.33:0/3398964546 shutdown_connections 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 --2- 172.21.15.33:0/3398964546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5e0102230 0x7fc5e0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 --2- 172.21.15.33:0/3398964546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e0101870 0x7fc5e0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 --2- 172.21.15.33:0/3398964546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0100aa0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.417+0000 7fc5e8027640 1 -- 172.21.15.33:0/3398964546 >> 172.21.15.33:0/3398964546 conn(0x7fc5e00fbbf0 msgr2=0x7fc5e00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 -- 172.21.15.33:0/3398964546 shutdown_connections 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 -- 172.21.15.33:0/3398964546 wait complete. 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 Processor -- start 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 -- start start 2024-04-24T23:17:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5e0101870 0x7fc5e0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e0102230 0x7fc5e019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e010c300 con 0x7fc5e0100680 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e010c180 con 0x7fc5e0102230 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e8027640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e010c480 con 0x7fc5e0101870 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39904/0 (socket says 172.21.15.33:39904) 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/1284552333 learned_addr learned my addr 172.21.15.33:0/1284552333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e0102230 0x7fc5e019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.418+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5e0101870 0x7fc5e0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/1284552333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5e0101870 msgr2=0x7fc5e0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5e0101870 0x7fc5e0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/1284552333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e0102230 msgr2=0x7fc5e019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e0102230 0x7fc5e019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/1284552333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e019f720 con 0x7fc5e0100680 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0197b00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc031610 tx=0x7fc5dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5d6ffd640 1 -- 172.21.15.33:0/1284552333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc03f030 con 0x7fc5e0100680 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e019f9b0 con 0x7fc5e0100680 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5d6ffd640 1 -- 172.21.15.33:0/1284552333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc003ad0 con 0x7fc5e0100680 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5d6ffd640 1 -- 172.21.15.33:0/1284552333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc035070 con 0x7fc5e0100680 2024-04-24T23:17:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.419+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e019fef0 con 0x7fc5e0100680 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 Processor -- start 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 -- start start 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce35a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56258cd95670 con 0x56258ce35640 2024-04-24T23:17:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56258cd3d970 con 0x56258ce3b1a0 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.422+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56258cd91630 con 0x56258ce36030 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5d6ffd640 1 -- 172.21.15.33:0/1284552333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc5dc042030 con 0x7fc5e0100680 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e65c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce35a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce3d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38754/0 (socket says 172.21.15.33:38754) 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e65c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce35a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39916/0 (socket says 172.21.15.33:39916) 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 -- 172.21.15.33:0/374558867 learned_addr learned my addr 172.21.15.33:0/374558867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 -- 172.21.15.33:0/374558867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 msgr2=0x56258ce3ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 -- 172.21.15.33:0/374558867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 msgr2=0x56258ce35a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce35a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 -- 172.21.15.33:0/374558867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258cd61400 con 0x56258ce3b1a0 2024-04-24T23:17:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce3d630 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009c90 tx=0x7fc5d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9014d17fcd3b6e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.423+0000 7fc5cbfff640 1 -- 172.21.15.33:0/374558867 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001e070 con 0x56258ce3b1a0 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5cbfff640 1 -- 172.21.15.33:0/374558867 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d0019b10 con 0x56258ce3b1a0 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5cbfff640 1 -- 172.21.15.33:0/374558867 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000ea30 con 0x56258ce3b1a0 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5d6ffd640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5cc0a2b90 0x7fc5cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5cc0a2b90 0x7fc5cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5d6ffd640 1 -- 172.21.15.33:0/1284552333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc5dc0ef960 con 0x7fc5e0100680 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 -- 172.21.15.33:0/374558867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 msgr2=0x56258ce3d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5cc0a2b90 0x7fc5cc0a5050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x56258ce36890 tx=0x7fc5d8009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce3d630 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009c90 tx=0x7fc5d0011810 comp rx=0 tx=0).stop 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 -- 172.21.15.33:0/374558867 shutdown_connections 2024-04-24T23:17:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce3d630 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/374558867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce35a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 -- 172.21.15.33:0/374558867 >> 172.21.15.33:0/374558867 conn(0x56258ce2c8a0 msgr2=0x56258ce2cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 -- 172.21.15.33:0/374558867 shutdown_connections 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.424+0000 7fc5eae0d740 1 -- 172.21.15.33:0/374558867 wait complete. 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 -- start start 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 Processor -- start 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce3f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce46ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56258ce3e150 con 0x56258ce35640 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56258cd51050 con 0x56258ce3b1a0 2024-04-24T23:17:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56258ce3e2d0 con 0x56258ce36030 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5eae0d740 10 client.0 _ll_register_callbacks cb 0x56258caa8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce3f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5e65c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce46ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce3f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39922/0 (socket says 172.21.15.33:39922) 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/1818724671 learned_addr learned my addr 172.21.15.33:0/1818724671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.425+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e65c4640 1 -- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 msgr2=0x56258ce3fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e65c4640 1 -- 172.21.15.33:0/1818724671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 msgr2=0x56258ce3f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce3f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e65c4640 1 -- 172.21.15.33:0/1818724671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258cd22dc0 con 0x56258ce3b1a0 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce46ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005fd0 tx=0x7fc5d8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81478198f638410 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5c97fa640 1 -- 172.21.15.33:0/1818724671 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800f870 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0019b10 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5c97fa640 1 -- 172.21.15.33:0/1818724671 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc5d800fa10 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5c97fa640 1 -- 172.21.15.33:0/1818724671 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5d800fc10 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5eae0d740 10 client.26135 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56258caa93f0 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.426+0000 7fc5c97fa640 1 -- 172.21.15.33:0/1818724671 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d0019b10 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5c97fa640 1 -- 172.21.15.33:0/1818724671 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc5d8026070 con 0x56258ce3b1a0 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5eae0d740 10 client.26135 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5eae0d740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5eae0d740 20 client.26135 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ce4b620 0x56258ce4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5eae0d740 4 client.26135 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258cda2bb0 con 0x56258ce4b620 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ce4b620 0x56258ce4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5c97fa640 10 client.26135 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.427+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ce4b620 0x56258ce4dae0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8717dd69c335531c server_cookie=7166785f37392506 in_seq=0 out_seq=0 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5c97fa640 1 -- 172.21.15.33:0/1818724671 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56258cda2bb0 con 0x56258ce4b620 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5c97fa640 10 client.26135 handle_command_reply: tid=0 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 client.26135 shutdown 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 msgr2=0x56258ce46ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce46ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005fd0 tx=0x7fc5d8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 shutdown_connections 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ce4b620 0x56258ce4dae0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce3b1a0 0x56258ce46ce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce3fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/1818724671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce35640 0x56258ce3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 >> 172.21.15.33:0/1818724671 conn(0x56258ce2c8a0 msgr2=0x56258ce2bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5c97fa640 0 client.26135 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 shutdown_connections 2024-04-24T23:17:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.428+0000 7fc5eae0d740 1 -- 172.21.15.33:0/1818724671 wait complete. 2024-04-24T23:17:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.429+0000 7fc5eae0d740 20 client.26135 trim_cache size 0 max 16384 2024-04-24T23:17:33.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532709 13.05 MB/sec execute 98 sec latency 13.184 ms 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 Processor -- start 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 -- start start 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 0x56258ce46ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce3b1a0 0x56258ce36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01aaaf0 con 0x56258ce36030 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a83c0 con 0x56258ce35640 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a6210 con 0x56258ce3b1a0 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 0x56258ce46ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 0x56258ce46ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39938/0 (socket says 172.21.15.33:39938) 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 -- 172.21.15.33:0/3084938201 learned_addr learned my addr 172.21.15.33:0/3084938201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce3b1a0 0x56258ce36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 -- 172.21.15.33:0/3084938201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce3b1a0 msgr2=0x56258ce36f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce3b1a0 0x56258ce36f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 -- 172.21.15.33:0/3084938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 msgr2=0x56258ce36a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce36a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e55c2640 1 -- 172.21.15.33:0/3084938201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258cb737d0 con 0x56258ce36030 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.445+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce36a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.446+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 0x56258ce46ca0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009c90 tx=0x7fc5d0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69c3360b7754c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.446+0000 7fc5cb7fe640 1 -- 172.21.15.33:0/3084938201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0028070 con 0x56258ce36030 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.446+0000 7fc5cb7fe640 1 -- 172.21.15.33:0/3084938201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d001f040 con 0x56258ce36030 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.446+0000 7fc5cb7fe640 1 -- 172.21.15.33:0/3084938201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001e050 con 0x56258ce36030 2024-04-24T23:17:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- 172.21.15.33:0/3084938201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 msgr2=0x56258ce46ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 0x56258ce46ca0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009c90 tx=0x7fc5d0005850 comp rx=0 tx=0).stop 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- 172.21.15.33:0/3084938201 shutdown_connections 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce3b1a0 0x56258ce36f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce36030 0x56258ce46ca0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/3084938201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce36a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- 172.21.15.33:0/3084938201 >> 172.21.15.33:0/3084938201 conn(0x56258ce2c8a0 msgr2=0x56258ce2bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- 172.21.15.33:0/3084938201 shutdown_connections 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- 172.21.15.33:0/3084938201 wait complete. 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- start start 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 Processor -- start 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce44220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 0x56258ce45230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a6880 con 0x56258ce3b1a0 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a7cf0 con 0x56258ce35640 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a6f50 con 0x56258ce36030 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.447+0000 7fc5eae0d740 10 client.0 _ll_register_callbacks cb 0x7fc5d8009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e65c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 0x56258ce45230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e55c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce44220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce43ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38786/0 (socket says 172.21.15.33:38786) 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e65c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 0x56258ce45230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39942/0 (socket says 172.21.15.33:39942) 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/2258197621 learned_addr learned my addr 172.21.15.33:0/2258197621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e55c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce44220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52262/0 (socket says 172.21.15.33:52262) 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 msgr2=0x56258ce44220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce44220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/2258197621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 msgr2=0x56258ce45230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 0x56258ce45230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 -- 172.21.15.33:0/2258197621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258ce45820 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce44220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e65c4640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 0x56258ce45230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5e5dc3640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce43ce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0bbbe0 tx=0x7fc5dc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e521a87574f28fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5c97fa640 1 -- 172.21.15.33:0/2258197621 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc049040 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5c97fa640 1 -- 172.21.15.33:0/2258197621 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc5dc003c80 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d001f040 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5c97fa640 1 -- 172.21.15.33:0/2258197621 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5dc0c4aa0 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5eae0d740 10 client.26141 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.448+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56258ce46b90 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5c97fa640 1 -- 172.21.15.33:0/2258197621 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d001f040 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5c97fa640 1 -- 172.21.15.33:0/2258197621 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc5dc0bc140 con 0x56258ce35640 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5eae0d740 10 client.26141 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5eae0d740 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5eae0d740 20 client.26141 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ceb1f10 0x56258ceb4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5eae0d740 4 client.26141 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56258cd9cf00 con 0x56258ceb1f10 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ceb1f10 0x56258ceb4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5c97fa640 10 client.26141 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.449+0000 7fc5e55c2640 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ceb1f10 0x56258ceb4330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde45369bab33135 server_cookie=2c1fa08ff04eca2f in_seq=0 out_seq=0 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5c97fa640 1 -- 172.21.15.33:0/2258197621 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56258cd9cf00 con 0x56258ceb1f10 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5c97fa640 10 client.26141 handle_command_reply: tid=0 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 client.26141 shutdown 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 msgr2=0x56258ce43ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce43ce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0bbbe0 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 shutdown_connections 2024-04-24T23:17:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56258ceb1f10 0x56258ceb4330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56258ce3b1a0 0x56258ce45230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56258ce36030 0x56258ce44220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5c97fa640 0 client.26141 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 --2- 172.21.15.33:0/2258197621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56258ce35640 0x56258ce43ce0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 >> 172.21.15.33:0/2258197621 conn(0x56258ce2c8a0 msgr2=0x56258ce2b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 shutdown_connections 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 1 -- 172.21.15.33:0/2258197621 wait complete. 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.450+0000 7fc5eae0d740 20 client.26141 trim_cache size 0 max 16384 2024-04-24T23:17:33.459 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:17:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5cc0a2b90 msgr2=0x7fc5cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5cc0a2b90 0x7fc5cc0a5050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x56258ce36890 tx=0x7fc5d8009500 comp rx=0 tx=0).stop 2024-04-24T23:17:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 msgr2=0x7fc5e0197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0197b00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc031610 tx=0x7fc5dc004270 comp rx=0 tx=0).stop 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 shutdown_connections 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5cc0a2b90 0x7fc5cc0a5050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e0102230 0x7fc5e019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5e0101870 0x7fc5e0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 --2- 172.21.15.33:0/1284552333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e0100680 0x7fc5e0197b00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 >> 172.21.15.33:0/1284552333 conn(0x7fc5e00fbbf0 msgr2=0x7fc5e00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 shutdown_connections 2024-04-24T23:17:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:33.451+0000 7fc5e8027640 1 -- 172.21.15.33:0/1284552333 wait complete. 2024-04-24T23:17:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:33 smithi038 ceph-mon[32450]: pgmap v517: 129 pgs: 129 active+clean; 186 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-04-24T23:17:33.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:33 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/760547841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:33 smithi117 ceph-mon[32529]: pgmap v517: 129 pgs: 129 active+clean; 186 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-04-24T23:17:33.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:33 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/760547841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:33.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:33 smithi033 ceph-mon[29201]: pgmap v517: 129 pgs: 129 active+clean; 186 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-04-24T23:17:33.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:33 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/760547841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:34.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 535234 13.04 MB/sec execute 99 sec latency 11.878 ms 2024-04-24T23:17:34.465 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:17:34.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:17:34.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:17:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 -- 172.21.15.33:0/787518660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8102230 msgr2=0x7f2be8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 --2- 172.21.15.33:0/787518660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8102230 0x7f2be8108ac0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009be0 tx=0x7f2be002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 -- 172.21.15.33:0/787518660 shutdown_connections 2024-04-24T23:17:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 --2- 172.21.15.33:0/787518660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8102230 0x7f2be8108ac0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 --2- 172.21.15.33:0/787518660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8101870 0x7f2be8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 --2- 172.21.15.33:0/787518660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8100680 0x7f2be8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 -- 172.21.15.33:0/787518660 >> 172.21.15.33:0/787518660 conn(0x7f2be80fbbf0 msgr2=0x7f2be80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 -- 172.21.15.33:0/787518660 shutdown_connections 2024-04-24T23:17:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.577+0000 7f2bee142640 1 -- 172.21.15.33:0/787518660 wait complete. 2024-04-24T23:17:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 Processor -- start 2024-04-24T23:17:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 -- start start 2024-04-24T23:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 0x7f2be8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8101870 0x7f2be8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8102230 0x7f2be819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810c290 con 0x7f2be8100680 2024-04-24T23:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810c110 con 0x7f2be8102230 2024-04-24T23:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2bee142640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810c410 con 0x7f2be8101870 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 0x7f2be8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 0x7f2be8197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39964/0 (socket says 172.21.15.33:39964) 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2be77fe640 1 -- 172.21.15.33:0/2674532652 learned_addr learned my addr 172.21.15.33:0/2674532652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.578+0000 7f2be7fff640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8102230 0x7f2be819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8101870 0x7f2be8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be77fe640 1 -- 172.21.15.33:0/2674532652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8101870 msgr2=0x7f2be8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be77fe640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8101870 0x7f2be8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be77fe640 1 -- 172.21.15.33:0/2674532652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8102230 msgr2=0x7f2be819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be77fe640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8102230 0x7f2be819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be77fe640 1 -- 172.21.15.33:0/2674532652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be819f690 con 0x7f2be8100680 2024-04-24T23:17:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be7fff640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8102230 0x7f2be819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be77fe640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 0x7f2be8197ae0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800e9c0 tx=0x7f2bd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be4ff9640 1 -- 172.21.15.33:0/2674532652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800cd80 con 0x7f2be8100680 2024-04-24T23:17:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be819f980 con 0x7f2be8100680 2024-04-24T23:17:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be4ff9640 1 -- 172.21.15.33:0/2674532652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd80035f0 con 0x7f2be8100680 2024-04-24T23:17:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2be4ff9640 1 -- 172.21.15.33:0/2674532652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8010690 con 0x7f2be8100680 2024-04-24T23:17:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.579+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be819fed0 con 0x7f2be8100680 2024-04-24T23:17:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be4ff9640 1 -- 172.21.15.33:0/2674532652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2bd80037a0 con 0x7f2be8100680 2024-04-24T23:17:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 Processor -- start 2024-04-24T23:17:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 -- start start 2024-04-24T23:17:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be61780 0x56314be66280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be668f0 0x56314be68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56314bdc42c0 con 0x56314be61780 2024-04-24T23:17:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56314bd6c910 con 0x56314be668f0 2024-04-24T23:17:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56314bdc0280 con 0x56314be60d90 2024-04-24T23:17:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be4ff9640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd00a2b90 0x7f2bd00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be61780 0x56314be66280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be7fff640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd00a2b90 0x7f2bd00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be668f0 0x56314be68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be61780 0x56314be66280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39970/0 (socket says 172.21.15.33:39970) 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be4ff9640 1 -- 172.21.15.33:0/2674532652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2bd80cf7b0 con 0x7f2be8100680 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be77fe640 1 -- 172.21.15.33:0/2664785707 learned_addr learned my addr 172.21.15.33:0/2664785707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.582+0000 7f2be7fff640 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be6ffd640 1 -- 172.21.15.33:0/2664785707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 msgr2=0x56314be611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be6ffd640 1 -- 172.21.15.33:0/2664785707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be61780 msgr2=0x56314be66280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be61780 0x56314be66280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be6ffd640 1 -- 172.21.15.33:0/2664785707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56314bd7dfb0 con 0x56314be668f0 2024-04-24T23:17:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be7fff640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd00a2b90 0x7f2bd00a5050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004ad0 tx=0x7f2be0004a60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be668f0 0x56314be68d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009530 tx=0x7f2bdc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da09c33b05bd1032 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2bcdffb640 1 -- 172.21.15.33:0/2664785707 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004020 con 0x56314be668f0 2024-04-24T23:17:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2bcdffb640 1 -- 172.21.15.33:0/2664785707 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc0041c0 con 0x56314be668f0 2024-04-24T23:17:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.583+0000 7f2bcdffb640 1 -- 172.21.15.33:0/2664785707 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0198b0 con 0x56314be668f0 2024-04-24T23:17:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2664785707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be668f0 msgr2=0x56314be68d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be668f0 0x56314be68d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009530 tx=0x7f2bdc011810 comp rx=0 tx=0).stop 2024-04-24T23:17:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2664785707 shutdown_connections 2024-04-24T23:17:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be668f0 0x56314be68d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be61780 0x56314be66280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2664785707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2664785707 >> 172.21.15.33:0/2664785707 conn(0x56314be28950 msgr2=0x56314be5c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2664785707 shutdown_connections 2024-04-24T23:17:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2664785707 wait complete. 2024-04-24T23:17:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.584+0000 7f2bf0e53740 1 -- start start 2024-04-24T23:17:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 Processor -- start 2024-04-24T23:17:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be72650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56314be698a0 con 0x56314be668f0 2024-04-24T23:17:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56314bd7fe90 con 0x56314be61780 2024-04-24T23:17:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56314be69a20 con 0x56314be60d90 2024-04-24T23:17:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2bf0e53740 10 client.0 _ll_register_callbacks cb 0x56314bdbb970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be72650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52294/0 (socket says 172.21.15.33:52294) 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38820/0 (socket says 172.21.15.33:38820) 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be77fe640 1 -- 172.21.15.33:0/2448088199 learned_addr learned my addr 172.21.15.33:0/2448088199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 -- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 msgr2=0x56314be6b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 -- 172.21.15.33:0/2448088199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 msgr2=0x56314be72650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be72650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be6ffd640 1 -- 172.21.15.33:0/2448088199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56314bba07e0 con 0x56314be61780 2024-04-24T23:17:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.585+0000 7f2be7fff640 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be72650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6b680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc000c00 tx=0x7f2bdc005510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b22d9d7cd8cee56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:17:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/2448088199 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc028070 con 0x56314be61780 2024-04-24T23:17:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bdc0041c0 con 0x56314be61780 2024-04-24T23:17:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/2448088199 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2bdc009db0 con 0x56314be61780 2024-04-24T23:17:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/2448088199 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bdc034050 con 0x56314be61780 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bf0e53740 10 client.26153 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56314bc00f70 con 0x56314be61780 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/2448088199 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc0041c0 con 0x56314be61780 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/2448088199 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2bdc004a00 con 0x56314be61780 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bf0e53740 10 client.26153 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bf0e53740 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.586+0000 7f2bf0e53740 20 client.26153 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.587+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314be768d0 0x56314be78d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.587+0000 7f2bf0e53740 4 client.26153 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.587+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56314bd985a0 con 0x56314be768d0 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.587+0000 7f2be77fe640 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314be768d0 0x56314be78d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.587+0000 7f2bc6ffd640 10 client.26153 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.587+0000 7f2be77fe640 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314be768d0 0x56314be78d40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=615f8ccf8aae8e9b server_cookie=2fae90219c378264 in_seq=0 out_seq=0 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/2448088199 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56314bd985a0 con 0x56314be768d0 2024-04-24T23:17:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bc6ffd640 10 client.26153 handle_command_reply: tid=0 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 client.26153 shutdown 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 msgr2=0x56314be6b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6b680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc000c00 tx=0x7f2bdc005510 comp rx=0 tx=0).stop 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 shutdown_connections 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314be768d0 0x56314be78d40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be72650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6b680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/2448088199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bc6ffd640 0 client.26153 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 >> 172.21.15.33:0/2448088199 conn(0x56314be28950 msgr2=0x56314be5b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 shutdown_connections 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 1 -- 172.21.15.33:0/2448088199 wait complete. 2024-04-24T23:17:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.588+0000 7f2bf0e53740 20 client.26153 trim_cache size 0 max 16384 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.607+0000 7f2bf0e53740 1 Processor -- start 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.607+0000 7f2bf0e53740 1 -- start start 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be60d90 0x56314be5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be61780 0x56314be72610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81accd0 con 0x56314be668f0 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81aa5a0 con 0x56314be60d90 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81a83f0 con 0x56314be61780 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be61780 0x56314be72610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39998/0 (socket says 172.21.15.33:39998) 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be7fff640 1 -- 172.21.15.33:0/167538877 learned_addr learned my addr 172.21.15.33:0/167538877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be77fe640 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be60d90 0x56314be5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 -- 172.21.15.33:0/167538877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be60d90 msgr2=0x56314be5f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be60d90 0x56314be5f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 -- 172.21.15.33:0/167538877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 msgr2=0x56314be5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 -- 172.21.15.33:0/167538877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56314ba98170 con 0x56314be61780 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be7fff640 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be61780 0x56314be72610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009b80 tx=0x7f2bdc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3417d22d3f9da2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.608+0000 7f2bcd7fa640 1 -- 172.21.15.33:0/167538877 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc028040 con 0x56314be61780 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bcd7fa640 1 -- 172.21.15.33:0/167538877 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc009870 con 0x56314be61780 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bcd7fa640 1 -- 172.21.15.33:0/167538877 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0048a0 con 0x56314be61780 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 -- 172.21.15.33:0/167538877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be61780 msgr2=0x56314be72610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be61780 0x56314be72610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009b80 tx=0x7f2bdc011810 comp rx=0 tx=0).stop 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 -- 172.21.15.33:0/167538877 shutdown_connections 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be61780 0x56314be72610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/167538877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be60d90 0x56314be5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 -- 172.21.15.33:0/167538877 >> 172.21.15.33:0/167538877 conn(0x56314be28950 msgr2=0x56314be62b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 -- 172.21.15.33:0/167538877 shutdown_connections 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.609+0000 7f2bf0e53740 1 -- 172.21.15.33:0/167538877 wait complete. 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 -- start start 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 Processor -- start 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81a8a60 con 0x56314be668f0 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81a9ed0 con 0x56314be61780 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81a9130 con 0x56314be60d90 2024-04-24T23:17:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2bf0e53740 10 client.0 _ll_register_callbacks cb 0x7f2bdc011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52302/0 (socket says 172.21.15.33:52302) 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 -- 172.21.15.33:0/3392435375 learned_addr learned my addr 172.21.15.33:0/3392435375 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 -- 172.21.15.33:0/3392435375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 msgr2=0x56314be6fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 -- 172.21.15.33:0/3392435375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 msgr2=0x56314be70db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be70db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.610+0000 7f2be77fe640 1 -- 172.21.15.33:0/3392435375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56314be713a0 con 0x56314be60d90 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2be77fe640 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6f860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2bd809b980 tx=0x7f2bd8066da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=255bb4fdba20b365 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/3392435375 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd806a020 con 0x56314be60d90 2024-04-24T23:17:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bdc009870 con 0x56314be60d90 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/3392435375 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2bd8003b30 con 0x56314be60d90 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/3392435375 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bd80a48f0 con 0x56314be60d90 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bf0e53740 10 client.26167 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56314be727b0 con 0x56314be60d90 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.611+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/3392435375 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc009870 con 0x56314be60d90 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/3392435375 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2bd80a4460 con 0x56314be60d90 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bf0e53740 10 client.26167 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bf0e53740 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bf0e53740 20 client.26167 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314bedd4e0 0x56314bedf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bf0e53740 4 client.26167 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56314bdd1800 con 0x56314bedd4e0 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314bedd4e0 0x56314bedf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2bc6ffd640 10 client.26167 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.612+0000 7f2be6ffd640 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314bedd4e0 0x56314bedf900 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662fd50183d507ff server_cookie=908033495f4fcf83 in_seq=0 out_seq=0 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bc6ffd640 1 -- 172.21.15.33:0/3392435375 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56314bdd1800 con 0x56314bedd4e0 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bc6ffd640 10 client.26167 handle_command_reply: tid=0 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 client.26167 shutdown 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 msgr2=0x56314be6f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6f860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2bd809b980 tx=0x7f2bd8066da0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 shutdown_connections 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56314bedd4e0 0x56314bedf900 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56314be668f0 0x56314be70db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56314be61780 0x56314be6fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 --2- 172.21.15.33:0/3392435375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56314be60d90 0x56314be6f860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bc6ffd640 0 client.26167 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 >> 172.21.15.33:0/3392435375 conn(0x56314be28950 msgr2=0x56314be5b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 shutdown_connections 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.613+0000 7f2bf0e53740 1 -- 172.21.15.33:0/3392435375 wait complete. 2024-04-24T23:17:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bf0e53740 20 client.26167 trim_cache size 0 max 16384 2024-04-24T23:17:34.627 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:17:34.627 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:17:34.627 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "685c9e18-5faa-4370-8ca3-62db1feaecd9", 2024-04-24T23:17:34.627 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:17:34.627 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:17:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd00a2b90 msgr2=0x7f2bd00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd00a2b90 0x7f2bd00a5050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004ad0 tx=0x7f2be0004a60 comp rx=0 tx=0).stop 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 msgr2=0x7f2be8197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 0x7f2be8197ae0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800e9c0 tx=0x7f2bd800ee80 comp rx=0 tx=0).stop 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 shutdown_connections 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd00a2b90 0x7f2bd00a5050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be8102230 0x7f2be819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be8101870 0x7f2be8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 --2- 172.21.15.33:0/2674532652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2be8100680 0x7f2be8197ae0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 >> 172.21.15.33:0/2674532652 conn(0x7f2be80fbbf0 msgr2=0x7f2be80fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 shutdown_connections 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.614+0000 7f2bee142640 1 -- 172.21.15.33:0/2674532652 wait complete. 2024-04-24T23:17:34.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:34.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 -- 172.21.15.33:0/1659102427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694107460 msgr2=0x7fe6941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1659102427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694107460 0x7fe6941098b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe684009ca0 tx=0x7fe68402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 -- 172.21.15.33:0/1659102427 shutdown_connections 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1659102427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694107460 0x7fe6941098b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1659102427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694104b30 0x7fe694106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1659102427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102200 0x7fe6941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 -- 172.21.15.33:0/1659102427 >> 172.21.15.33:0/1659102427 conn(0x7fe6940fbbf0 msgr2=0x7fe6940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 -- 172.21.15.33:0/1659102427 shutdown_connections 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 -- 172.21.15.33:0/1659102427 wait complete. 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.739+0000 7fe6998ee640 1 Processor -- start 2024-04-24T23:17:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 -- start start 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694102200 0x7fe694199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694104b30 0x7fe69419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 0x7fe6941a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410d400 con 0x7fe694102200 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410d280 con 0x7fe694107460 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6998ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410d580 con 0x7fe694104b30 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694102200 0x7fe694199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 0x7fe6941a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 0x7fe6941a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38846/0 (socket says 172.21.15.33:38846) 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 -- 172.21.15.33:0/1280411465 learned_addr learned my addr 172.21.15.33:0/1280411465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6927fc640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694104b30 0x7fe69419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 -- 172.21.15.33:0/1280411465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694104b30 msgr2=0x7fe69419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694104b30 0x7fe69419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 -- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694102200 msgr2=0x7fe694199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694102200 0x7fe694199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.740+0000 7fe6937fe640 1 -- 172.21.15.33:0/1280411465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6941a18d0 con 0x7fe694107460 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.741+0000 7fe6937fe640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 0x7fe6941a1230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe684031ae0 tx=0x7fe684031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.741+0000 7fe6988ec640 1 -- 172.21.15.33:0/1280411465 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684003b70 con 0x7fe694107460 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.741+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941a1b60 con 0x7fe694107460 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.741+0000 7fe6988ec640 1 -- 172.21.15.33:0/1280411465 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe684003d10 con 0x7fe694107460 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.741+0000 7fe6988ec640 1 -- 172.21.15.33:0/1280411465 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68402fc40 con 0x7fe694107460 2024-04-24T23:17:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.741+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a2100 con 0x7fe694107460 2024-04-24T23:17:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 Processor -- start 2024-04-24T23:17:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe6988ec640 1 -- 172.21.15.33:0/1280411465 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe684004050 con 0x7fe694107460 2024-04-24T23:17:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 -- start start 2024-04-24T23:17:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee86660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee8b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d86edea6b0 con 0x55d86ee86c30 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d86ed92880 con 0x55d86ee86240 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.743+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d86ede6670 con 0x55d86ee8bda0 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee8b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe6927fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee86660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee8b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40018/0 (socket says 172.21.15.33:40018) 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 -- 172.21.15.33:0/211371044 learned_addr learned my addr 172.21.15.33:0/211371044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe6927fc640 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 -- 172.21.15.33:0/211371044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 msgr2=0x55d86ee8e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 -- 172.21.15.33:0/211371044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 msgr2=0x55d86ee86660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee86660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 -- 172.21.15.33:0/211371044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86ebc87d0 con 0x55d86ee86c30 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe6927fc640 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee86660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe6988ec640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6800a2b90 0x7fe6800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe692ffd640 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee8b730 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009c90 tx=0x7fe68c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d28b008c00db76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe6927fc640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6800a2b90 0x7fe6800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe6988ec640 1 -- 172.21.15.33:0/1280411465 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe6840f04b0 con 0x7fe694107460 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe689ffb640 1 -- 172.21.15.33:0/211371044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c005220 con 0x55d86ee86c30 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe689ffb640 1 -- 172.21.15.33:0/211371044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c004c80 con 0x55d86ee86c30 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.744+0000 7fe689ffb640 1 -- 172.21.15.33:0/211371044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0196d0 con 0x55d86ee86c30 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 -- 172.21.15.33:0/211371044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 msgr2=0x55d86ee8b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe6927fc640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6800a2b90 0x7fe6800a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55d86ee87490 tx=0x7fe67c009770 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee8b730 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009c90 tx=0x7fe68c011810 comp rx=0 tx=0).stop 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 -- 172.21.15.33:0/211371044 shutdown_connections 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee8b730 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/211371044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee86660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 -- 172.21.15.33:0/211371044 >> 172.21.15.33:0/211371044 conn(0x55d86ee818b0 msgr2=0x55d86ee81cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 -- 172.21.15.33:0/211371044 shutdown_connections 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 -- 172.21.15.33:0/211371044 wait complete. 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.745+0000 7fe69c5f9740 1 -- start start 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 Processor -- start 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee90630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee90b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d86ee8ed50 con 0x55d86ee86240 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d86eda5f50 con 0x55d86ee8bda0 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d86ee8eed0 con 0x55d86ee86c30 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe69c5f9740 10 client.0 _ll_register_callbacks cb 0x55d86ede1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee90630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee90630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40020/0 (socket says 172.21.15.33:40020) 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 -- 172.21.15.33:0/56465802 learned_addr learned my addr 172.21.15.33:0/56465802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe6927fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee90b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe6937fe640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 -- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 msgr2=0x55d86ee90b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee90b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 -- 172.21.15.33:0/56465802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 msgr2=0x55d86ee97bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee97bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe692ffd640 1 -- 172.21.15.33:0/56465802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86eafe3f0 con 0x55d86ee86240 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.746+0000 7fe6937fe640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee97bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe692ffd640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee90630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009890 tx=0x7fe68c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d1cd07a292cdd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe672ffd640 1 -- 172.21.15.33:0/56465802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c029070 con 0x55d86ee86240 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe68c004c80 con 0x55d86ee86240 2024-04-24T23:17:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe672ffd640 1 -- 172.21.15.33:0/56465802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe68c023840 con 0x55d86ee86240 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe672ffd640 1 -- 172.21.15.33:0/56465802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe68c035050 con 0x55d86ee86240 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe69c5f9740 10 client.16641 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d86eabe0f0 con 0x55d86ee86240 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.747+0000 7fe672ffd640 1 -- 172.21.15.33:0/56465802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c004c80 con 0x55d86ee86240 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe672ffd640 1 -- 172.21.15.33:0/56465802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe68c019690 con 0x55d86ee86240 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe69c5f9740 10 client.16641 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe69c5f9740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe69c5f9740 20 client.16641 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ee9c820 0x55d86ee9ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe69c5f9740 4 client.16641 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d86edf7bf0 con 0x55d86ee9c820 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe6927fc640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ee9c820 0x55d86ee9ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe672ffd640 10 client.16641 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.748+0000 7fe6927fc640 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ee9c820 0x55d86ee9ecc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c4c753d2e40ed6 server_cookie=985db92580d81d2c in_seq=0 out_seq=0 2024-04-24T23:17:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.749+0000 7fe672ffd640 1 -- 172.21.15.33:0/56465802 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d86edf7bf0 con 0x55d86ee9c820 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.749+0000 7fe672ffd640 10 client.16641 handle_command_reply: tid=0 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.749+0000 7fe69c5f9740 1 client.16641 shutdown 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 msgr2=0x55d86ee90630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee90630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009890 tx=0x7fe68c004970 comp rx=0 tx=0).stop 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 shutdown_connections 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ee9c820 0x55d86ee9ecc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee97bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee90b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe672ffd640 0 client.16641 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/56465802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee90630 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 >> 172.21.15.33:0/56465802 conn(0x55d86ee818b0 msgr2=0x55d86ee80af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 shutdown_connections 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 1 -- 172.21.15.33:0/56465802 wait complete. 2024-04-24T23:17:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.750+0000 7fe69c5f9740 20 client.16641 trim_cache size 0 max 16384 2024-04-24T23:17:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 Processor -- start 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 -- start start 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee80970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee97b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941aef40 con 0x55d86ee86c30 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941ac810 con 0x55d86ee86240 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941aa660 con 0x55d86ee8bda0 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe6927fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee97b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe6927fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee97b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40032/0 (socket says 172.21.15.33:40032) 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee80970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.770+0000 7fe6927fc640 1 -- 172.21.15.33:0/3678130649 learned_addr learned my addr 172.21.15.33:0/3678130649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6937fe640 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6927fc640 1 -- 172.21.15.33:0/3678130649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 msgr2=0x55d86ee8fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6927fc640 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6927fc640 1 -- 172.21.15.33:0/3678130649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 msgr2=0x55d86ee80970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6927fc640 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee80970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6927fc640 1 -- 172.21.15.33:0/3678130649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86ee82620 con 0x55d86ee86c30 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6937fe640 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe692ffd640 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee80970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6927fc640 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee97b70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe69419b190 tx=0x7fe67c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9d1dce15acc03a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6897fa640 1 -- 172.21.15.33:0/3678130649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c011dc0 con 0x55d86ee86c30 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6897fa640 1 -- 172.21.15.33:0/3678130649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c005500 con 0x55d86ee86c30 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.771+0000 7fe6897fa640 1 -- 172.21.15.33:0/3678130649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c02b030 con 0x55d86ee86c30 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 -- 172.21.15.33:0/3678130649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 msgr2=0x55d86ee97b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee97b70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe69419b190 tx=0x7fe67c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 -- 172.21.15.33:0/3678130649 shutdown_connections 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee8bda0 0x55d86ee8fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86c30 0x55d86ee97b70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/3678130649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee86240 0x55d86ee80970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 -- 172.21.15.33:0/3678130649 >> 172.21.15.33:0/3678130649 conn(0x55d86ee818b0 msgr2=0x55d86ee80260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 -- 172.21.15.33:0/3678130649 shutdown_connections 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 -- 172.21.15.33:0/3678130649 wait complete. 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 -- start start 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 Processor -- start 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.772+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee94920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941aacd0 con 0x55d86ee86240 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941ac140 con 0x55d86ee8bda0 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941ab3a0 con 0x55d86ee86c30 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 10 client.0 _ll_register_callbacks cb 0x7fe67c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee97940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40046/0 (socket says 172.21.15.33:40046) 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 -- 172.21.15.33:0/1240580069 learned_addr learned my addr 172.21.15.33:0/1240580069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe6937fe640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee94920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe6927fc640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 -- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 msgr2=0x55d86ee943e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee943e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 -- 172.21.15.33:0/1240580069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 msgr2=0x55d86ee94920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee94920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 -- 172.21.15.33:0/1240580069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86ee94fc0 con 0x55d86ee86240 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe6937fe640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee94920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe692ffd640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee97940 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe68c019ca0 tx=0x7fe68c004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b64ac4dcf47d16d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe672ffd640 1 -- 172.21.15.33:0/1240580069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c015030 con 0x55d86ee86240 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe67c005500 con 0x55d86ee86240 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe672ffd640 1 -- 172.21.15.33:0/1240580069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe68c019840 con 0x55d86ee86240 2024-04-24T23:17:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe672ffd640 1 -- 172.21.15.33:0/1240580069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe68c029040 con 0x55d86ee86240 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 10 client.16647 fetch_fsmap learned FSMap version 27 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.773+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d86ef01f50 con 0x55d86ee86240 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe672ffd640 1 -- 172.21.15.33:0/1240580069 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c005500 con 0x55d86ee86240 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe672ffd640 1 -- 172.21.15.33:0/1240580069 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe68c038070 con 0x55d86ee86240 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe69c5f9740 10 client.16647 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe69c5f9740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe69c5f9740 20 client.16647 populate_metadata read hostname 'smithi033' 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ef02950 0x55d86ef04d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe69c5f9740 4 client.16647 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d86edf1f40 con 0x55d86ef02950 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe6927fc640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ef02950 0x55d86ef04d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe6927fc640 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ef02950 0x55d86ef04d70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7061521a492890a server_cookie=4f496e5cfeb4d5cf in_seq=0 out_seq=0 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.774+0000 7fe672ffd640 10 client.16647 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe672ffd640 1 -- 172.21.15.33:0/1240580069 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d86edf1f40 con 0x55d86ef02950 2024-04-24T23:17:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe672ffd640 10 client.16647 handle_command_reply: tid=0 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 client.16647 shutdown 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 msgr2=0x55d86ee97940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee97940 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe68c019ca0 tx=0x7fe68c004c10 comp rx=0 tx=0).stop 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 shutdown_connections 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d86ef02950 0x55d86ef04d70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d86ee8bda0 0x55d86ee94920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d86ee86c30 0x55d86ee943e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe672ffd640 0 client.16647 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 --2- 172.21.15.33:0/1240580069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d86ee86240 0x55d86ee97940 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 >> 172.21.15.33:0/1240580069 conn(0x55d86ee818b0 msgr2=0x55d86ee80af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 shutdown_connections 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 1 -- 172.21.15.33:0/1240580069 wait complete. 2024-04-24T23:17:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.775+0000 7fe69c5f9740 20 client.16647 trim_cache size 0 max 16384 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: "685c9e18-5faa-4370-8ca3-62db1feaecd9": { 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "685c9e18-5faa-4370-8ca3-62db1feaecd9", 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:17:34.784 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6800a2b90 msgr2=0x7fe6800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6800a2b90 0x7fe6800a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55d86ee87490 tx=0x7fe67c009770 comp rx=0 tx=0).stop 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 msgr2=0x7fe6941a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 0x7fe6941a1230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe684031ae0 tx=0x7fe684031ee0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 shutdown_connections 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6800a2b90 0x7fe6800a5050 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe694107460 0x7fe6941a1230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694104b30 0x7fe69419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 --2- 172.21.15.33:0/1280411465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe694102200 0x7fe694199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 >> 172.21.15.33:0/1280411465 conn(0x7fe6940fbbf0 msgr2=0x7fe694101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 shutdown_connections 2024-04-24T23:17:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:17:34.776+0000 7fe6998ee640 1 -- 172.21.15.33:0/1280411465 wait complete. 2024-04-24T23:17:34.789 INFO:tasks.cephfs.filesystem:scrub status for tag:685c9e18-5faa-4370-8ca3-62db1feaecd9 - {'path': '/', 'tag': '685c9e18-5faa-4370-8ca3-62db1feaecd9', 'options': 'recursive,force'} 2024-04-24T23:17:35.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 537333 13.03 MB/sec execute 100 sec latency 14.329 ms 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: pgmap v518: 129 pgs: 129 active+clean; 186 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: pgmap v518: 129 pgs: 129 active+clean; 186 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-04-24T23:17:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:17:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:17:35.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:17:35.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.677 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: pgmap v518: 129 pgs: 129 active+clean; 186 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-04-24T23:17:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:17:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:17:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:17:35.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:35.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:36.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 539814 13.04 MB/sec execute 101 sec latency 95.744 ms 2024-04-24T23:17:37.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 541485 13.01 MB/sec execute 102 sec latency 12.131 ms 2024-04-24T23:17:37.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:37 smithi038 ceph-mon[32450]: pgmap v519: 129 pgs: 129 active+clean; 192 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-04-24T23:17:37.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:37 smithi117 ceph-mon[32529]: pgmap v519: 129 pgs: 129 active+clean; 192 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-04-24T23:17:37.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:37 smithi033 ceph-mon[29201]: pgmap v519: 129 pgs: 129 active+clean; 192 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-04-24T23:17:38.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 544005 13.02 MB/sec execute 103 sec latency 25.253 ms 2024-04-24T23:17:39.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 546896 13.03 MB/sec execute 104 sec latency 12.650 ms 2024-04-24T23:17:39.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:39 smithi117 ceph-mon[32529]: pgmap v520: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-04-24T23:17:39.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:39 smithi033 ceph-mon[29201]: pgmap v520: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-04-24T23:17:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:39 smithi038 ceph-mon[32450]: pgmap v520: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-04-24T23:17:40.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 548765 13.00 MB/sec execute 105 sec latency 15.897 ms 2024-04-24T23:17:41.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 551053 13.01 MB/sec execute 106 sec latency 97.168 ms 2024-04-24T23:17:41.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:41 smithi117 ceph-mon[32529]: pgmap v521: 129 pgs: 129 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-04-24T23:17:41.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:41 smithi033 ceph-mon[29201]: pgmap v521: 129 pgs: 129 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-04-24T23:17:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:41 smithi038 ceph-mon[32450]: pgmap v521: 129 pgs: 129 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 3.6 MiB/s wr, 204 op/s 2024-04-24T23:17:42.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 553542 13.01 MB/sec execute 107 sec latency 16.448 ms 2024-04-24T23:17:42.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:42 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:17:42.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:42 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:17:42.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:42 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:17:43.445 INFO:tasks.workunit.client.0.smithi033.stdout: 1 555228 12.97 MB/sec execute 108 sec latency 61.452 ms 2024-04-24T23:17:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:43 smithi033 ceph-mon[29201]: pgmap v522: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 288 op/s 2024-04-24T23:17:43.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:17:43.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:43 smithi038 ceph-mon[32450]: pgmap v522: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 288 op/s 2024-04-24T23:17:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:17:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:43 smithi117 ceph-mon[32529]: pgmap v522: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 288 op/s 2024-04-24T23:17:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:17:44.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 558068 13.00 MB/sec execute 109 sec latency 63.517 ms 2024-04-24T23:17:45.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 560694 13.00 MB/sec execute 110 sec latency 13.673 ms 2024-04-24T23:17:45.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:45 smithi033 ceph-mon[29201]: pgmap v523: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 217 op/s 2024-04-24T23:17:45.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:45 smithi038 ceph-mon[32450]: pgmap v523: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 217 op/s 2024-04-24T23:17:45.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:45 smithi117 ceph-mon[32529]: pgmap v523: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 217 op/s 2024-04-24T23:17:46.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 562076 12.95 MB/sec execute 111 sec latency 58.332 ms 2024-04-24T23:17:47.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 564942 12.99 MB/sec execute 112 sec latency 19.179 ms 2024-04-24T23:17:47.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:47 smithi033 ceph-mon[29201]: pgmap v524: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-04-24T23:17:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:47 smithi038 ceph-mon[32450]: pgmap v524: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-04-24T23:17:47.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:47 smithi117 ceph-mon[32529]: pgmap v524: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-04-24T23:17:48.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 567292 12.95 MB/sec execute 113 sec latency 19.765 ms 2024-04-24T23:17:49.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 569334 12.95 MB/sec execute 114 sec latency 24.126 ms 2024-04-24T23:17:49.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:49 smithi033 ceph-mon[29201]: pgmap v525: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 224 op/s 2024-04-24T23:17:49.727 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:49 smithi117 ceph-mon[32529]: pgmap v525: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 224 op/s 2024-04-24T23:17:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:49 smithi038 ceph-mon[32450]: pgmap v525: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 224 op/s 2024-04-24T23:17:50.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572259 12.99 MB/sec execute 115 sec latency 13.059 ms 2024-04-24T23:17:51.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 574265 12.94 MB/sec execute 116 sec latency 55.611 ms 2024-04-24T23:17:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:51 smithi033 ceph-mon[29201]: pgmap v526: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2024-04-24T23:17:51.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:51 smithi038 ceph-mon[32450]: pgmap v526: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2024-04-24T23:17:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:51 smithi117 ceph-mon[32529]: pgmap v526: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 210 op/s 2024-04-24T23:17:52.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576178 12.94 MB/sec execute 117 sec latency 21.518 ms 2024-04-24T23:17:53.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 579206 12.98 MB/sec execute 118 sec latency 14.062 ms 2024-04-24T23:17:53.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:53 smithi033 ceph-mon[29201]: pgmap v527: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-04-24T23:17:53.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:53 smithi038 ceph-mon[32450]: pgmap v527: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-04-24T23:17:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:53 smithi117 ceph-mon[32529]: pgmap v527: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-04-24T23:17:54.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 581354 12.93 MB/sec execute 119 sec latency 12.606 ms 2024-04-24T23:17:55.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 583193 12.93 MB/sec execute 120 sec latency 25.524 ms 2024-04-24T23:17:55.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:55 smithi033 ceph-mon[29201]: pgmap v528: 129 pgs: 129 active+clean; 218 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-04-24T23:17:55.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:55 smithi038 ceph-mon[32450]: pgmap v528: 129 pgs: 129 active+clean; 218 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-04-24T23:17:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:55 smithi117 ceph-mon[32529]: pgmap v528: 129 pgs: 129 active+clean; 218 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-04-24T23:17:56.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 585696 12.93 MB/sec execute 121 sec latency 43.054 ms 2024-04-24T23:17:57.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 587310 12.89 MB/sec execute 122 sec latency 22.072 ms 2024-04-24T23:17:57.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:57 smithi038 ceph-mon[32450]: pgmap v529: 129 pgs: 129 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 298 op/s 2024-04-24T23:17:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:57 smithi117 ceph-mon[32529]: pgmap v529: 129 pgs: 129 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 298 op/s 2024-04-24T23:17:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:57 smithi033 ceph-mon[29201]: pgmap v529: 129 pgs: 129 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 298 op/s 2024-04-24T23:17:58.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 590127 12.92 MB/sec execute 123 sec latency 21.535 ms 2024-04-24T23:17:59.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 592956 12.94 MB/sec execute 124 sec latency 23.719 ms 2024-04-24T23:17:59.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:59 smithi038 ceph-mon[32450]: pgmap v530: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-04-24T23:17:59.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:17:59 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:59 smithi117 ceph-mon[32529]: pgmap v530: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-04-24T23:17:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:17:59 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:59 smithi033 ceph-mon[29201]: pgmap v530: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-04-24T23:17:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:17:59 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:00.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 594804 12.91 MB/sec execute 125 sec latency 13.775 ms 2024-04-24T23:18:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:18:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:18:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:18:01.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 597120 12.92 MB/sec execute 126 sec latency 40.234 ms 2024-04-24T23:18:01.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:01 smithi033 ceph-mon[29201]: pgmap v531: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:18:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:01 smithi038 ceph-mon[32450]: pgmap v531: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:18:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:01 smithi117 ceph-mon[32529]: pgmap v531: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:18:02.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 599472 12.90 MB/sec execute 127 sec latency 68.760 ms 2024-04-24T23:18:03.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 601179 12.87 MB/sec execute 128 sec latency 16.016 ms 2024-04-24T23:18:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:03 smithi038 ceph-mon[32450]: pgmap v532: 129 pgs: 129 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2024-04-24T23:18:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:03 smithi117 ceph-mon[32529]: pgmap v532: 129 pgs: 129 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2024-04-24T23:18:03.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:03 smithi033 ceph-mon[29201]: pgmap v532: 129 pgs: 129 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2024-04-24T23:18:04.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 604143 12.91 MB/sec execute 129 sec latency 12.966 ms 2024-04-24T23:18:04.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:18:04.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:18:04.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 -- 172.21.15.33:0/1630131406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0102230 msgr2=0x7f54b0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 --2- 172.21.15.33:0/1630131406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0102230 0x7f54b0108ac0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009ca0 tx=0x7f54a802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 -- 172.21.15.33:0/1630131406 shutdown_connections 2024-04-24T23:18:04.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 --2- 172.21.15.33:0/1630131406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0102230 0x7f54b0108ac0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 --2- 172.21.15.33:0/1630131406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0101870 0x7f54b0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 --2- 172.21.15.33:0/1630131406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 -- 172.21.15.33:0/1630131406 >> 172.21.15.33:0/1630131406 conn(0x7f54b00fbbf0 msgr2=0x7f54b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:04.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 -- 172.21.15.33:0/1630131406 shutdown_connections 2024-04-24T23:18:04.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.901+0000 7f54b5e38640 1 -- 172.21.15.33:0/1630131406 wait complete. 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 Processor -- start 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 -- start start 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0101870 0x7f54b0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 0x7f54b019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010c2e0 con 0x7f54b0101870 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010c160 con 0x7f54b0100680 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54b5e38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010c460 con 0x7f54b0102230 2024-04-24T23:18:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54affff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 0x7f54b019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44722/0 (socket says 172.21.15.33:44722) 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54af7fe640 1 -- 172.21.15.33:0/3639596740 learned_addr learned my addr 172.21.15.33:0/3639596740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54affff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 0x7f54b019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39332/0 (socket says 172.21.15.33:39332) 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.902+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0101870 0x7f54b0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54af7fe640 1 -- 172.21.15.33:0/3639596740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 msgr2=0x7f54b019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54af7fe640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 0x7f54b019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54af7fe640 1 -- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0101870 msgr2=0x7f54b0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54af7fe640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0101870 0x7f54b0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54af7fe640 1 -- 172.21.15.33:0/3639596740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b019f690 con 0x7f54b0100680 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54affff640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 0x7f54b019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0101870 0x7f54b0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54af7fe640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0197a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e9c0 tx=0x7f54a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54ad7fa640 1 -- 172.21.15.33:0/3639596740 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a000cde0 con 0x7f54b0100680 2024-04-24T23:18:04.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54ad7fa640 1 -- 172.21.15.33:0/3639596740 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a0010910 con 0x7f54b0100680 2024-04-24T23:18:04.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b019f980 con 0x7f54b0100680 2024-04-24T23:18:04.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54ad7fa640 1 -- 172.21.15.33:0/3639596740 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0018a00 con 0x7f54b0100680 2024-04-24T23:18:04.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.903+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b019fed0 con 0x7f54b0100680 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54ad7fa640 1 -- 172.21.15.33:0/3639596740 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f54a001f020 con 0x7f54b0100680 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 Processor -- start 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 -- start start 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7c7240 0x56357f7c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7ccda0 0x56357f7cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56357f72b6b0 con 0x56357f7c7c30 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56357f6d3880 con 0x56357f7c7240 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56357f727670 con 0x56357f7ccda0 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54ad7fa640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f549c0a2b90 0x7f549c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54342/0 (socket says 172.21.15.33:54342) 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54af7fe640 1 -- 172.21.15.33:0/3051902684 learned_addr learned my addr 172.21.15.33:0/3051902684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.906+0000 7f54affff640 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7c7240 0x56357f7c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7ccda0 0x56357f7cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f549c0a2b90 0x7f549c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54ad7fa640 1 -- 172.21.15.33:0/3639596740 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54a00cf5c0 con 0x7f54b0100680 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 -- 172.21.15.33:0/3051902684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7ccda0 msgr2=0x56357f7cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7ccda0 0x56357f7cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 -- 172.21.15.33:0/3051902684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 msgr2=0x56357f7cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 -- 172.21.15.33:0/3051902684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357f5097d0 con 0x56357f7c7240 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7c7240 0x56357f7c7660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f54a8031ae0 tx=0x7f54a8009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ed77d1e6f8d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54affff640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f549c0a2b90 0x7f549c0a5050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f54a8048000 tx=0x7f54a804b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54a5ffb640 1 -- 172.21.15.33:0/3051902684 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8050070 con 0x56357f7c7240 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54a5ffb640 1 -- 172.21.15.33:0/3051902684 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8036040 con 0x56357f7c7240 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.907+0000 7f54a5ffb640 1 -- 172.21.15.33:0/3051902684 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a803b800 con 0x56357f7c7240 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 -- 172.21.15.33:0/3051902684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7c7240 msgr2=0x56357f7c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7c7240 0x56357f7c7660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f54a8031ae0 tx=0x7f54a8009900 comp rx=0 tx=0).stop 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 -- 172.21.15.33:0/3051902684 shutdown_connections 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7ccda0 0x56357f7cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/3051902684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7c7240 0x56357f7c7660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 -- 172.21.15.33:0/3051902684 >> 172.21.15.33:0/3051902684 conn(0x56357f7c28b0 msgr2=0x56357f7c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 -- 172.21.15.33:0/3051902684 shutdown_connections 2024-04-24T23:18:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 -- 172.21.15.33:0/3051902684 wait complete. 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.908+0000 7f54b8c1e740 1 -- start start 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 Processor -- start 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56357f7cfb70 con 0x56357f7c7240 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56357f6e6f50 con 0x56357f7ccda0 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56357f7cfcf0 con 0x56357f7c7c30 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54b8c1e740 10 client.0 _ll_register_callbacks cb 0x56357f722000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54344/0 (socket says 172.21.15.33:54344) 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 -- 172.21.15.33:0/2830543961 learned_addr learned my addr 172.21.15.33:0/2830543961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54affff640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 -- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 msgr2=0x56357f7d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 -- 172.21.15.33:0/2830543961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 msgr2=0x56357f7d8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7d8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.909+0000 7f54af7fe640 1 -- 172.21.15.33:0/2830543961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357f43f3f0 con 0x56357f7c7240 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54af7fe640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d1480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f54a0000c00 tx=0x7f54a0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bb6bcc3258de9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f548effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f548effd640 1 -- 172.21.15.33:0/2830543961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00b4060 con 0x56357f7c7240 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f548effd640 1 -- 172.21.15.33:0/2830543961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54a0098100 con 0x56357f7c7240 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8036040 con 0x56357f7c7240 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f548effd640 1 -- 172.21.15.33:0/2830543961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a009c190 con 0x56357f7c7240 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 10 client.16659 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56357f3ff0f0 con 0x56357f7c7240 2024-04-24T23:18:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f548effd640 1 -- 172.21.15.33:0/2830543961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8036040 con 0x56357f7c7240 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f548effd640 1 -- 172.21.15.33:0/2830543961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54a00a4270 con 0x56357f7c7240 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 10 client.16659 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 20 client.16659 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f7dd230 0x56357f7df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 4 client.16659 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357f738bf0 con 0x56357f7dd230 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.910+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f7dd230 0x56357f7df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.911+0000 7f548effd640 10 client.16659 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.911+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f7dd230 0x56357f7df6d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f75af8d50da8953 server_cookie=d383b5b18a10884 in_seq=0 out_seq=0 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f548effd640 1 -- 172.21.15.33:0/2830543961 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56357f738bf0 con 0x56357f7dd230 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f548effd640 10 client.16659 handle_command_reply: tid=0 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 client.16659 shutdown 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 msgr2=0x56357f7d1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d1480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f54a0000c00 tx=0x7f54a0009490 comp rx=0 tx=0).stop 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 shutdown_connections 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f7dd230 0x56357f7df6d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f548effd640 0 client.16659 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2830543961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d1480 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 >> 172.21.15.33:0/2830543961 conn(0x56357f7c28b0 msgr2=0x56357f7c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 shutdown_connections 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2830543961 wait complete. 2024-04-24T23:18:04.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.912+0000 7f54b8c1e740 20 client.16659 trim_cache size 0 max 16384 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 Processor -- start 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 -- start start 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7240 0x56357f7d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01acc10 con 0x56357f7c7c30 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aa4e0 con 0x56357f7ccda0 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a8330 con 0x56357f7c7240 2024-04-24T23:18:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7d89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54356/0 (socket says 172.21.15.33:54356) 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 -- 172.21.15.33:0/2421072908 learned_addr learned my addr 172.21.15.33:0/2421072908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54af7fe640 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7240 0x56357f7d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54affff640 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 -- 172.21.15.33:0/2421072908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7240 msgr2=0x56357f7d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7240 0x56357f7d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 -- 172.21.15.33:0/2421072908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 msgr2=0x56357f7c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.930+0000 7f54a6ffd640 1 -- 172.21.15.33:0/2421072908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357f7c23f0 con 0x56357f7c7c30 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7d89c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x56357f7c3820 tx=0x7f5498005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5189d4d31b8ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54a57fa640 1 -- 172.21.15.33:0/2421072908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498010800 con 0x56357f7c7c30 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54a57fa640 1 -- 172.21.15.33:0/2421072908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498023020 con 0x56357f7c7c30 2024-04-24T23:18:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54a57fa640 1 -- 172.21.15.33:0/2421072908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549801e420 con 0x56357f7c7c30 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2421072908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 msgr2=0x56357f7d89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7d89c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x56357f7c3820 tx=0x7f5498005f90 comp rx=0 tx=0).stop 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2421072908 shutdown_connections 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f7c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7c30 0x56357f7d89c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.931+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/2421072908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7240 0x56357f7d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2421072908 >> 172.21.15.33:0/2421072908 conn(0x56357f7c28b0 msgr2=0x56357f7d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2421072908 shutdown_connections 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- 172.21.15.33:0/2421072908 wait complete. 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- start start 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 Processor -- start 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f841850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a89a0 con 0x56357f7c7240 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a9e10 con 0x56357f7ccda0 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a9070 con 0x56357f7c7c30 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54b8c1e740 10 client.0 _ll_register_callbacks cb 0x7f54a0018420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54360/0 (socket says 172.21.15.33:54360) 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54af7fe640 1 -- 172.21.15.33:0/276019364 learned_addr learned my addr 172.21.15.33:0/276019364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.932+0000 7f54affff640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f841850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54af7fe640 1 -- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 msgr2=0x56357f7d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54af7fe640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54af7fe640 1 -- 172.21.15.33:0/276019364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 msgr2=0x56357f841850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54af7fe640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f841850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54af7fe640 1 -- 172.21.15.33:0/276019364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357f841d90 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54af7fe640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d58a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f54a00058c0 tx=0x7f54a009b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d1c5270e86b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f548effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f548effd640 1 -- 172.21.15.33:0/276019364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00b4040 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f548effd640 1 -- 172.21.15.33:0/276019364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54a00ba1b0 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5498023020 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f548effd640 1 -- 172.21.15.33:0/276019364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a009ba90 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54b8c1e740 10 client.16665 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.933+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56357f842fd0 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f548effd640 1 -- 172.21.15.33:0/276019364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498023020 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f548effd640 1 -- 172.21.15.33:0/276019364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54a009aee0 con 0x56357f7c7240 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54b8c1e740 10 client.16665 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54b8c1e740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54b8c1e740 20 client.16665 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f8436f0 0x56357f845b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54b8c1e740 4 client.16665 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56357f732f40 con 0x56357f8436f0 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f8436f0 0x56357f845b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f54a6ffd640 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f8436f0 0x56357f845b10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc11e42804f04042 server_cookie=4f16f5279c1ac12 in_seq=0 out_seq=0 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.934+0000 7f548effd640 10 client.16665 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f548effd640 1 -- 172.21.15.33:0/276019364 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56357f732f40 con 0x56357f8436f0 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f548effd640 10 client.16665 handle_command_reply: tid=0 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 client.16665 shutdown 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 msgr2=0x56357f7d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d58a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f54a00058c0 tx=0x7f54a009b520 comp rx=0 tx=0).stop 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 shutdown_connections 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56357f8436f0 0x56357f845b10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56357f7ccda0 0x56357f841850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357f7c7c30 0x56357f7d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f548effd640 0 client.16665 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 --2- 172.21.15.33:0/276019364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56357f7c7240 0x56357f7d58a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 >> 172.21.15.33:0/276019364 conn(0x56357f7c28b0 msgr2=0x56357f7c1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 shutdown_connections 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 1 -- 172.21.15.33:0/276019364 wait complete. 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b8c1e740 20 client.16665 trim_cache size 0 max 16384 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f549c0a2b90 msgr2=0x7f549c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.935+0000 7f54b5e38640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f549c0a2b90 0x7f549c0a5050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f54a8048000 tx=0x7f54a804b000 comp rx=0 tx=0).stop 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 msgr2=0x7f54b0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0197a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e9c0 tx=0x7f54a000ee80 comp rx=0 tx=0).stop 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 shutdown_connections 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f549c0a2b90 0x7f549c0a5050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b0102230 0x7f54b019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b0101870 0x7f54b0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 --2- 172.21.15.33:0/3639596740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54b0100680 0x7f54b0197a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 >> 172.21.15.33:0/3639596740 conn(0x7f54b00fbbf0 msgr2=0x7f54b00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 shutdown_connections 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:04.936+0000 7f54b5e38640 1 -- 172.21.15.33:0/3639596740 wait complete. 2024-04-24T23:18:04.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:18:04.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:18:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 -- 172.21.15.33:0/1857691998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618101350 msgr2=0x7f46181017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 --2- 172.21.15.33:0/1857691998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618101350 0x7f46181017d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4610009ca0 tx=0x7f461002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 -- 172.21.15.33:0/1857691998 shutdown_connections 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 --2- 172.21.15.33:0/1857691998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618101350 0x7f46181017d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 --2- 172.21.15.33:0/1857691998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46181009b0 0x7f4618100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 --2- 172.21.15.33:0/1857691998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4618100050 0x7f4618100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 -- 172.21.15.33:0/1857691998 >> 172.21.15.33:0/1857691998 conn(0x7f46180fbbf0 msgr2=0x7f46180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 -- 172.21.15.33:0/1857691998 shutdown_connections 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.057+0000 7f461ddf9640 1 -- 172.21.15.33:0/1857691998 wait complete. 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 Processor -- start 2024-04-24T23:18:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 -- start start 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 0x7f4618199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46181009b0 0x7f461819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618101350 0x7f46181a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f461810f5d0 con 0x7f4618100050 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f461810f450 con 0x7f46181009b0 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f461ddf9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f461810f750 con 0x7f4618101350 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f46177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 0x7f4618199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f46177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 0x7f4618199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54370/0 (socket says 172.21.15.33:54370) 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f46177fe640 1 -- 172.21.15.33:0/2955509756 learned_addr learned my addr 172.21.15.33:0/2955509756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f460ffff640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46181009b0 0x7f461819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.058+0000 7f4617fff640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618101350 0x7f46181a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46177fe640 1 -- 172.21.15.33:0/2955509756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618101350 msgr2=0x7f46181a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46177fe640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618101350 0x7f46181a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46177fe640 1 -- 172.21.15.33:0/2955509756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46181009b0 msgr2=0x7f461819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46177fe640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46181009b0 0x7f461819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46177fe640 1 -- 172.21.15.33:0/2955509756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46181a1840 con 0x7f4618100050 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46177fe640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 0x7f4618199c30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f460800e9f0 tx=0x7f460800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460800cd80 con 0x7f4618100050 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4608010910 con 0x7f4618100050 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181a1b30 con 0x7f4618100050 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608018990 con 0x7f4618100050 2024-04-24T23:18:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.059+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a20d0 con 0x7f4618100050 2024-04-24T23:18:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.060+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461810af00 con 0x7f4618100050 2024-04-24T23:18:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.061+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4608010ac0 con 0x7f4618100050 2024-04-24T23:18:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.062+0000 7f46157fa640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46040a2c10 0x7f46040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.062+0000 7f460ffff640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46040a2c10 0x7f46040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.062+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46080cf6d0 con 0x7f4618100050 2024-04-24T23:18:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.063+0000 7f460ffff640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46040a2c10 0x7f46040a50d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4600009bd0 tx=0x7f4600005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.067+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f460809bc50 con 0x7f4618100050 2024-04-24T23:18:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.305+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f461802d170 con 0x7f4618100050 2024-04-24T23:18:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.306+0000 7f46157fa640 1 -- 172.21.15.33:0/2955509756 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f460809ff50 con 0x7f4618100050 2024-04-24T23:18:05.308 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:18:05.308 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:18:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46040a2c10 msgr2=0x7f46040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46040a2c10 0x7f46040a50d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4600009bd0 tx=0x7f4600005f90 comp rx=0 tx=0).stop 2024-04-24T23:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 msgr2=0x7f4618199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 0x7f4618199c30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f460800e9f0 tx=0x7f460800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 shutdown_connections 2024-04-24T23:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f46040a2c10 0x7f46040a50d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618101350 0x7f46181a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46181009b0 0x7f461819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 --2- 172.21.15.33:0/2955509756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4618100050 0x7f4618199c30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 >> 172.21.15.33:0/2955509756 conn(0x7f46180fbbf0 msgr2=0x7f46180fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 shutdown_connections 2024-04-24T23:18:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.308+0000 7f461ddf9640 1 -- 172.21.15.33:0/2955509756 wait complete. 2024-04-24T23:18:05.322 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:18:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 -- 172.21.15.33:0/2683889741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974103340 msgr2=0x7f7974109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 --2- 172.21.15.33:0/2683889741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974103340 0x7f7974109bd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f796c009ca0 tx=0x7f796c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 -- 172.21.15.33:0/2683889741 shutdown_connections 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 --2- 172.21.15.33:0/2683889741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974103340 0x7f7974109bd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 --2- 172.21.15.33:0/2683889741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7974102980 0x7f7974102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 --2- 172.21.15.33:0/2683889741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7974101790 0x7f7974101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 -- 172.21.15.33:0/2683889741 >> 172.21.15.33:0/2683889741 conn(0x7f79740fcd20 msgr2=0x7f79740ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 -- 172.21.15.33:0/2683889741 shutdown_connections 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 -- 172.21.15.33:0/2683889741 wait complete. 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.428+0000 7f7979a15640 1 Processor -- start 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 -- start start 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7974101790 0x7f79740754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 0x7f7974073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7974103340 0x7f7974074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d3f0 con 0x7f7974102980 2024-04-24T23:18:05.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d270 con 0x7f7974101790 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7979a15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d570 con 0x7f7974103340 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 0x7f7974073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 0x7f7974073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54374/0 (socket says 172.21.15.33:54374) 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f7972ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7974101790 0x7f79740754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 -- 172.21.15.33:0/4237068208 learned_addr learned my addr 172.21.15.33:0/4237068208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79737fe640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7974103340 0x7f7974074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 -- 172.21.15.33:0/4237068208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7974103340 msgr2=0x7f7974074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7974103340 0x7f7974074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 -- 172.21.15.33:0/4237068208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7974101790 msgr2=0x7f79740754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7974101790 0x7f79740754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.429+0000 7f79727fc640 1 -- 172.21.15.33:0/4237068208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7974074900 con 0x7f7974102980 2024-04-24T23:18:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.430+0000 7f79727fc640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 0x7f7974073b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f796800d950 tx=0x7f796800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.430+0000 7f7978a13640 1 -- 172.21.15.33:0/4237068208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968017030 con 0x7f7974102980 2024-04-24T23:18:05.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.430+0000 7f7978a13640 1 -- 172.21.15.33:0/4237068208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968004020 con 0x7f7974102980 2024-04-24T23:18:05.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.430+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79741a1b80 con 0x7f7974102980 2024-04-24T23:18:05.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.430+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79741a20d0 con 0x7f7974102980 2024-04-24T23:18:05.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.431+0000 7f7978a13640 1 -- 172.21.15.33:0/4237068208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79680047a0 con 0x7f7974102980 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f7978a13640 1 -- 172.21.15.33:0/4237068208 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7968004940 con 0x7f7974102980 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 Processor -- start 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 -- start start 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a356650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a35b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a35e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b2a2ba670 con 0x561b2a35bd90 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561b2a262970 con 0x561b2a356230 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f797c803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b2a2b6630 con 0x561b2a356c20 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f7978a13640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f795c0a2b90 0x7f795c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f7972ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a35b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f79727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a35e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f7972ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a35b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39414/0 (socket says 172.21.15.33:39414) 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.433+0000 7f7972ffd640 1 -- 172.21.15.33:0/1617373183 learned_addr learned my addr 172.21.15.33:0/1617373183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f79737fe640 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a356650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f79737fe640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f795c0a2b90 0x7f795c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7978a13640 1 -- 172.21.15.33:0/4237068208 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f79680ceca0 con 0x7f7974102980 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7972ffd640 1 -- 172.21.15.33:0/1617373183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 msgr2=0x561b2a356650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7972ffd640 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a356650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7972ffd640 1 -- 172.21.15.33:0/1617373183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 msgr2=0x561b2a35e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7972ffd640 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a35e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7972ffd640 1 -- 172.21.15.33:0/1617373183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2a286400 con 0x561b2a356c20 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f79727fc640 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a35e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7972ffd640 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a35b720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7964009ca0 tx=0x7f7964011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08017899115f313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f79737fe640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f795c0a2b90 0x7f795c0a5050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f796c004af0 tx=0x7f796c005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7959ffb640 1 -- 172.21.15.33:0/1617373183 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964004220 con 0x561b2a356c20 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7959ffb640 1 -- 172.21.15.33:0/1617373183 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79640043c0 con 0x561b2a356c20 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.434+0000 7f7959ffb640 1 -- 172.21.15.33:0/1617373183 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964019930 con 0x561b2a356c20 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 -- 172.21.15.33:0/1617373183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 msgr2=0x561b2a35b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a35b720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7964009ca0 tx=0x7f7964011810 comp rx=0 tx=0).stop 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 -- 172.21.15.33:0/1617373183 shutdown_connections 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a35e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a35b720 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 --2- 172.21.15.33:0/1617373183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a356650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 -- 172.21.15.33:0/1617373183 >> 172.21.15.33:0/1617373183 conn(0x561b2a3518a0 msgr2=0x561b2a351ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 -- 172.21.15.33:0/1617373183 shutdown_connections 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 -- 172.21.15.33:0/1617373183 wait complete. 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.435+0000 7f797c803740 1 -- start start 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 Processor -- start 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a360300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a360840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a367880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b2a35ed40 con 0x561b2a35bd90 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561b2a276050 con 0x561b2a356230 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b2a35eec0 con 0x561b2a356c20 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f797c803740 10 client.0 _ll_register_callbacks cb 0x561b29fcdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a367880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f7972ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a360300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a367880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54388/0 (socket says 172.21.15.33:54388) 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79737fe640 1 -- 172.21.15.33:0/488449999 learned_addr learned my addr 172.21.15.33:0/488449999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a360840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 -- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 msgr2=0x561b2a360300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a360300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 -- 172.21.15.33:0/488449999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 msgr2=0x561b2a367880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a367880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 -- 172.21.15.33:0/488449999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2a247dc0 con 0x561b2a356c20 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79737fe640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a367880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f79727fc640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a360840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f79680cc460 tx=0x7f79680996a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0196da52a81c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.436+0000 7f7952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f7952ffd640 1 -- 172.21.15.33:0/488449999 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796809a7f0 con 0x561b2a356c20 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79640043c0 con 0x561b2a356c20 2024-04-24T23:18:05.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f7952ffd640 1 -- 172.21.15.33:0/488449999 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f796809a990 con 0x561b2a356c20 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f7952ffd640 1 -- 172.21.15.33:0/488449999 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79680970f0 con 0x561b2a356c20 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 10 client.26218 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b29fce3f0 con 0x561b2a356c20 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f7952ffd640 1 -- 172.21.15.33:0/488449999 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79640043c0 con 0x561b2a356c20 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f7952ffd640 1 -- 172.21.15.33:0/488449999 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f796807b080 con 0x561b2a356c20 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 10 client.26218 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 10 client.26218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 20 client.26218 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a36c280 0x561b2a36e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 4 client.26218 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2a2c7bb0 con 0x561b2a36c280 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.437+0000 7f7972ffd640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a36c280 0x561b2a36e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.438+0000 7f7972ffd640 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a36c280 0x561b2a36e700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de2513ebafecfa93 server_cookie=da0a372cc1336a27 in_seq=0 out_seq=0 2024-04-24T23:18:05.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.438+0000 7f7952ffd640 10 client.26218 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.438+0000 7f7952ffd640 1 -- 172.21.15.33:0/488449999 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b2a2c7bb0 con 0x561b2a36c280 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.438+0000 7f7952ffd640 10 client.26218 handle_command_reply: tid=0 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.438+0000 7f797c803740 1 client.26218 shutdown 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 msgr2=0x561b2a360840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a360840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f79680cc460 tx=0x7f79680996a0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 shutdown_connections 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a36c280 0x561b2a36e700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a35bd90 0x561b2a367880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a360840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f7952ffd640 0 client.26218 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:05.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 --2- 172.21.15.33:0/488449999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a360300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 >> 172.21.15.33:0/488449999 conn(0x561b2a3518a0 msgr2=0x561b2a350b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 shutdown_connections 2024-04-24T23:18:05.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 1 -- 172.21.15.33:0/488449999 wait complete. 2024-04-24T23:18:05.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.439+0000 7f797c803740 20 client.26218 trim_cache size 0 max 16384 2024-04-24T23:18:05.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 606240 12.87 MB/sec execute 130 sec latency 21.056 ms 2024-04-24T23:18:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 Processor -- start 2024-04-24T23:18:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 -- start start 2024-04-24T23:18:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a3509c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 0x561b2a367840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a35bd90 0x561b2a354870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7960012460 con 0x561b2a356c20 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f796000fd30 con 0x561b2a356230 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f797c803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f796000db80 con 0x561b2a35bd90 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f79727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 0x561b2a367840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f79727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 0x561b2a367840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54396/0 (socket says 172.21.15.33:54396) 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.459+0000 7f79727fc640 1 -- 172.21.15.33:0/627245939 learned_addr learned my addr 172.21.15.33:0/627245939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.460+0000 7f79727fc640 1 -- 172.21.15.33:0/627245939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a35bd90 msgr2=0x561b2a354870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.460+0000 7f79727fc640 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a35bd90 0x561b2a354870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.460+0000 7f79727fc640 1 -- 172.21.15.33:0/627245939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 msgr2=0x561b2a3509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.460+0000 7f79727fc640 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a3509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.460+0000 7f79727fc640 1 -- 172.21.15.33:0/627245939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2a0987d0 con 0x561b2a356c20 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.460+0000 7f79727fc640 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 0x561b2a367840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f796809b0a0 tx=0x7f79680cc490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b4887b86070131 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.461+0000 7f79597fa640 1 -- 172.21.15.33:0/627245939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968097290 con 0x561b2a356c20 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.461+0000 7f79597fa640 1 -- 172.21.15.33:0/627245939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968097a10 con 0x561b2a356c20 2024-04-24T23:18:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.461+0000 7f79597fa640 1 -- 172.21.15.33:0/627245939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796809a7f0 con 0x561b2a356c20 2024-04-24T23:18:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 -- 172.21.15.33:0/627245939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 msgr2=0x561b2a367840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 0x561b2a367840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f796809b0a0 tx=0x7f79680cc490 comp rx=0 tx=0).stop 2024-04-24T23:18:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 -- 172.21.15.33:0/627245939 shutdown_connections 2024-04-24T23:18:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a35bd90 0x561b2a354870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356c20 0x561b2a367840 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 --2- 172.21.15.33:0/627245939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a356230 0x561b2a3509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.462+0000 7f797c803740 1 -- 172.21.15.33:0/627245939 >> 172.21.15.33:0/627245939 conn(0x561b2a3518a0 msgr2=0x561b2a3502b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.463+0000 7f797c803740 1 -- 172.21.15.33:0/627245939 shutdown_connections 2024-04-24T23:18:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.463+0000 7f797c803740 1 -- 172.21.15.33:0/627245939 wait complete. 2024-04-24T23:18:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 -- start start 2024-04-24T23:18:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 Processor -- start 2024-04-24T23:18:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356230 0x561b2a365090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a3655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a35bd90 0x561b2a3665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f796000e1f0 con 0x561b2a356230 2024-04-24T23:18:05.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f796000f660 con 0x561b2a35bd90 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79727fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a3655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79727fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a3655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39438/0 (socket says 172.21.15.33:39438) 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79727fc640 1 -- 172.21.15.33:0/1656482212 learned_addr learned my addr 172.21.15.33:0/1656482212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79737fe640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a35bd90 0x561b2a3665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.464+0000 7f797c803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f796000e8c0 con 0x561b2a356c20 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f7972ffd640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356230 0x561b2a365090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f797c803740 10 client.0 _ll_register_callbacks cb 0x7f796809ad90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79737fe640 1 -- 172.21.15.33:0/1656482212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 msgr2=0x561b2a3655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79737fe640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a3655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79737fe640 1 -- 172.21.15.33:0/1656482212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356230 msgr2=0x561b2a365090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79737fe640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356230 0x561b2a365090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.465+0000 7f79737fe640 1 -- 172.21.15.33:0/1656482212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2a366bd0 con 0x561b2a35bd90 2024-04-24T23:18:05.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f79737fe640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a35bd90 0x561b2a3665e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f796c009ca0 tx=0x7f796c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b45d794af81936e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f7952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:18:05.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f7952ffd640 1 -- 172.21.15.33:0/1656482212 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c0318c0 con 0x561b2a35bd90 2024-04-24T23:18:05.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f7952ffd640 1 -- 172.21.15.33:0/1656482212 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f796c030ba0 con 0x561b2a35bd90 2024-04-24T23:18:05.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7968097a10 con 0x561b2a35bd90 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f7952ffd640 1 -- 172.21.15.33:0/1656482212 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f796c030da0 con 0x561b2a35bd90 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f797c803740 10 client.26216 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.466+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b2a3d29e0 con 0x561b2a35bd90 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f7952ffd640 1 -- 172.21.15.33:0/1656482212 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7968097a10 con 0x561b2a35bd90 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f7952ffd640 1 -- 172.21.15.33:0/1656482212 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f796c048040 con 0x561b2a35bd90 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f797c803740 10 client.26216 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:05.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f797c803740 10 client.26216 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:18:05.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f797c803740 20 client.26216 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:05.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f797c803740 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a3d3400 0x561b2a3d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f797c803740 4 client.26216 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:18:05.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b2a2c1f00 con 0x561b2a3d3400 2024-04-24T23:18:05.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.467+0000 7f7972ffd640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a3d3400 0x561b2a3d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.469+0000 7f7972ffd640 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a3d3400 0x561b2a3d5820 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a43cd00f180e86fe server_cookie=f561bd0bf201974e in_seq=0 out_seq=0 2024-04-24T23:18:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.469+0000 7f7952ffd640 10 client.26216 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f7952ffd640 1 -- 172.21.15.33:0/1656482212 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b2a2c1f00 con 0x561b2a3d3400 2024-04-24T23:18:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f7952ffd640 10 client.26216 handle_command_reply: tid=0 2024-04-24T23:18:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 client.26216 shutdown 2024-04-24T23:18:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a35bd90 msgr2=0x561b2a3665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a35bd90 0x561b2a3665e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f796c009ca0 tx=0x7f796c0095c0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 shutdown_connections 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561b2a3d3400 0x561b2a3d5820 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b2a35bd90 0x561b2a3665e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2a356c20 0x561b2a3655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f7952ffd640 0 client.26216 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 --2- 172.21.15.33:0/1656482212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b2a356230 0x561b2a365090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 >> 172.21.15.33:0/1656482212 conn(0x561b2a3518a0 msgr2=0x561b2a350b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 shutdown_connections 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 1 -- 172.21.15.33:0/1656482212 wait complete. 2024-04-24T23:18:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.470+0000 7f797c803740 20 client.26216 trim_cache size 0 max 16384 2024-04-24T23:18:05.474 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:18:05.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f795c0a2b90 msgr2=0x7f795c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f795c0a2b90 0x7f795c0a5050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f796c004af0 tx=0x7f796c005d80 comp rx=0 tx=0).stop 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 msgr2=0x7f7974073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 0x7f7974073b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f796800d950 tx=0x7f796800de10 comp rx=0 tx=0).stop 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 shutdown_connections 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f795c0a2b90 0x7f795c0a5050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7974103340 0x7f7974074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7974102980 0x7f7974073b30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 --2- 172.21.15.33:0/4237068208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7974101790 0x7f79740754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 >> 172.21.15.33:0/4237068208 conn(0x7f79740fcd20 msgr2=0x7f7974105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 shutdown_connections 2024-04-24T23:18:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.471+0000 7f7979a15640 1 -- 172.21.15.33:0/4237068208 wait complete. 2024-04-24T23:18:05.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 -- 172.21.15.33:0/508738099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0101870 msgr2=0x7f87f0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 --2- 172.21.15.33:0/508738099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0101870 0x7f87f0101cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009900 tx=0x7f87d802a410 comp rx=0 tx=0).stop 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 -- 172.21.15.33:0/508738099 shutdown_connections 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 --2- 172.21.15.33:0/508738099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f0102230 0x7f87f0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 --2- 172.21.15.33:0/508738099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0101870 0x7f87f0101cf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 --2- 172.21.15.33:0/508738099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f0100680 0x7f87f0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 -- 172.21.15.33:0/508738099 >> 172.21.15.33:0/508738099 conn(0x7f87f00fbbf0 msgr2=0x7f87f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 -- 172.21.15.33:0/508738099 shutdown_connections 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 -- 172.21.15.33:0/508738099 wait complete. 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.596+0000 7f87f597e640 1 Processor -- start 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 -- start start 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f0100680 0x7f87f0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f0101870 0x7f87f0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 0x7f87f019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010c2e0 con 0x7f87f0102230 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010c160 con 0x7f87f0101870 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87f597e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010c460 con 0x7f87f0100680 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 0x7f87f019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f0100680 0x7f87f0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 0x7f87f019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54420/0 (socket says 172.21.15.33:54420) 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f0101870 0x7f87f0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 -- 172.21.15.33:0/826597648 learned_addr learned my addr 172.21.15.33:0/826597648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 -- 172.21.15.33:0/826597648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f0100680 msgr2=0x7f87f0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f0100680 0x7f87f0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 -- 172.21.15.33:0/826597648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f0101870 msgr2=0x7f87f0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f0101870 0x7f87f0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 -- 172.21.15.33:0/826597648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f019f6f0 con 0x7f87f0102230 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.597+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 0x7f87f019f080 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f87e000eb10 tx=0x7f87e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.598+0000 7f87f497c640 1 -- 172.21.15.33:0/826597648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004b20 con 0x7f87f0102230 2024-04-24T23:18:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.598+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f019f9e0 con 0x7f87f0102230 2024-04-24T23:18:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.598+0000 7f87f497c640 1 -- 172.21.15.33:0/826597648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e0004cc0 con 0x7f87f0102230 2024-04-24T23:18:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.598+0000 7f87f497c640 1 -- 172.21.15.33:0/826597648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0010760 con 0x7f87f0102230 2024-04-24T23:18:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.598+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f019ff00 con 0x7f87f0102230 2024-04-24T23:18:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.600+0000 7f87f497c640 1 -- 172.21.15.33:0/826597648 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f87e0005020 con 0x7f87f0102230 2024-04-24T23:18:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.600+0000 7f87f876d740 1 Processor -- start 2024-04-24T23:18:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.600+0000 7f87f497c640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87dc0a2b90 0x7f87dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.600+0000 7f87f876d740 1 -- start start 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f876d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac2b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f497c640 1 -- 172.21.15.33:0/826597648 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87e00cf980 con 0x7f87f0102230 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f876d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87dc0a2b90 0x7f87dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f876d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac33220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f876d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ab8f670 con 0x55fc9ac30d90 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f876d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ab37970 con 0x55fc9ac2b230 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87f876d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ab8b630 con 0x55fc9ac2bc20 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac2b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac33220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac2b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44848/0 (socket says 172.21.15.33:44848) 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 -- 172.21.15.33:0/3642576365 learned_addr learned my addr 172.21.15.33:0/3642576365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 -- 172.21.15.33:0/3642576365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 msgr2=0x55fc9ac30720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac30720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 -- 172.21.15.33:0/3642576365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 msgr2=0x55fc9ac33220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac33220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 -- 172.21.15.33:0/3642576365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9ab5b400 con 0x55fc9ac2b230 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac30720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87eeffd640 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac2b650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87e800abf0 tx=0x7f87e800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9746d3a8f904403 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87dc0a2b90 0x7f87dc0a5050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f87d80314e0 tx=0x7f87d8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87e5ffb640 1 -- 172.21.15.33:0/3642576365 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004220 con 0x55fc9ac2b230 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87e5ffb640 1 -- 172.21.15.33:0/3642576365 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e80043c0 con 0x55fc9ac2b230 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.601+0000 7f87e5ffb640 1 -- 172.21.15.33:0/3642576365 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80188f0 con 0x55fc9ac2b230 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 -- 172.21.15.33:0/3642576365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 msgr2=0x55fc9ac2b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac2b650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87e800abf0 tx=0x7f87e800f790 comp rx=0 tx=0).stop 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 -- 172.21.15.33:0/3642576365 shutdown_connections 2024-04-24T23:18:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac30720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 --2- 172.21.15.33:0/3642576365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac2b650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 -- 172.21.15.33:0/3642576365 >> 172.21.15.33:0/3642576365 conn(0x55fc9ac268a0 msgr2=0x55fc9ac26ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 -- 172.21.15.33:0/3642576365 shutdown_connections 2024-04-24T23:18:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.602+0000 7f87f876d740 1 -- 172.21.15.33:0/3642576365 wait complete. 2024-04-24T23:18:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 -- start start 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 Processor -- start 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 0x55fc9ac396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac39bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ac38150 con 0x55fc9ac2b230 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ab4b050 con 0x55fc9ac30d90 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ac382d0 con 0x55fc9ac2bc20 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87f876d740 10 client.0 _ll_register_callbacks cb 0x55fc9a8a2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 0x55fc9ac396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac39bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 0x55fc9ac396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54446/0 (socket says 172.21.15.33:54446) 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac39bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39470/0 (socket says 172.21.15.33:39470) 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44860/0 (socket says 172.21.15.33:44860) 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 -- 172.21.15.33:0/1731199959 learned_addr learned my addr 172.21.15.33:0/1731199959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 -- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 msgr2=0x55fc9ac39bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac39bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 -- 172.21.15.33:0/1731199959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 msgr2=0x55fc9ac40c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.603+0000 7f87eeffd640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87eeffd640 1 -- 172.21.15.33:0/1731199959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc9ab1cdc0 con 0x55fc9ac2b230 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac39bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87eeffd640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 0x55fc9ac396b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f87e800fc90 tx=0x7f87e80057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9bd25c4db342e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ceffd640 1 -- 172.21.15.33:0/1731199959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8027020 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ceffd640 1 -- 172.21.15.33:0/1731199959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87e80048d0 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e80043c0 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ceffd640 1 -- 172.21.15.33:0/1731199959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87e8032050 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 10 client.16701 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc9a8a33f0 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ceffd640 1 -- 172.21.15.33:0/1731199959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e80043c0 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87ceffd640 1 -- 172.21.15.33:0/1731199959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f87e8023470 con 0x55fc9ac2b230 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 10 client.16701 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 10 client.16701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 20 client.16701 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.604+0000 7f87f876d740 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9ac451a0 0x55fc9ac475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.605+0000 7f87f876d740 4 client.16701 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.605+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc9ab9cbb0 con 0x55fc9ac451a0 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.605+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9ac451a0 0x55fc9ac475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.605+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9ac451a0 0x55fc9ac475f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1dc994fd438329fc server_cookie=42fa532574e984a4 in_seq=0 out_seq=0 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.605+0000 7f87ceffd640 10 client.16701 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87ceffd640 1 -- 172.21.15.33:0/1731199959 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc9ab9cbb0 con 0x55fc9ac451a0 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87ceffd640 10 client.16701 handle_command_reply: tid=0 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 client.16701 shutdown 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 msgr2=0x55fc9ac396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 0x55fc9ac396b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f87e800fc90 tx=0x7f87e80057d0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 shutdown_connections 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9ac451a0 0x55fc9ac475f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac39bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87ceffd640 0 client.16701 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 --2- 172.21.15.33:0/1731199959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2b230 0x55fc9ac396b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 >> 172.21.15.33:0/1731199959 conn(0x55fc9ac268a0 msgr2=0x55fc9ac25b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 shutdown_connections 2024-04-24T23:18:05.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 1 -- 172.21.15.33:0/1731199959 wait complete. 2024-04-24T23:18:05.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.606+0000 7f87f876d740 20 client.16701 trim_cache size 0 max 16384 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 Processor -- start 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 -- start start 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac25d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac37960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac38cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01aaa60 con 0x55fc9ac30d90 2024-04-24T23:18:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01a8330 con 0x55fc9ac2b230 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87f876d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01a6180 con 0x55fc9ac2bc20 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac38cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac38cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54454/0 (socket says 172.21.15.33:54454) 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87ef7fe640 1 -- 172.21.15.33:0/1793406457 learned_addr learned my addr 172.21.15.33:0/1793406457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac37960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.620+0000 7f87eeffd640 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac25d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87ef7fe640 1 -- 172.21.15.33:0/1793406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 msgr2=0x55fc9ac37960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac37960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87ef7fe640 1 -- 172.21.15.33:0/1793406457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 msgr2=0x55fc9ac25d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac25d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87ef7fe640 1 -- 172.21.15.33:0/1793406457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9a96d7d0 con 0x55fc9ac30d90 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac38cd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f87e009bde0 tx=0x7f87e00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e055126b6107cc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87e57fa640 1 -- 172.21.15.33:0/1793406457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e009be50 con 0x55fc9ac30d90 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87e57fa640 1 -- 172.21.15.33:0/1793406457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e009e040 con 0x55fc9ac30d90 2024-04-24T23:18:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.621+0000 7f87e57fa640 1 -- 172.21.15.33:0/1793406457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0005310 con 0x55fc9ac30d90 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 -- 172.21.15.33:0/1793406457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 msgr2=0x55fc9ac38cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac38cd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f87e009bde0 tx=0x7f87e00058d0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 -- 172.21.15.33:0/1793406457 shutdown_connections 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac30d90 0x55fc9ac38cd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac37960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 --2- 172.21.15.33:0/1793406457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac25d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 -- 172.21.15.33:0/1793406457 >> 172.21.15.33:0/1793406457 conn(0x55fc9ac268a0 msgr2=0x55fc9ac2a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 -- 172.21.15.33:0/1793406457 shutdown_connections 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 -- 172.21.15.33:0/1793406457 wait complete. 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 -- start start 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.622+0000 7f87f876d740 1 Processor -- start 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac36a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac36f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01a67f0 con 0x55fc9ac2bc20 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01a7c60 con 0x55fc9ac2b230 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01a6ec0 con 0x55fc9ac30d90 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 10 client.0 _ll_register_callbacks cb 0x7f87e00057d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac36f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac36a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac36f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54466/0 (socket says 172.21.15.33:54466) 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac36a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44872/0 (socket says 172.21.15.33:44872) 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 -- 172.21.15.33:0/2699230958 learned_addr learned my addr 172.21.15.33:0/2699230958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 -- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac30d90 msgr2=0x55fc9ac40be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 -- 172.21.15.33:0/2699230958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 msgr2=0x55fc9ac36a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac36a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 -- 172.21.15.33:0/2699230958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc9acadf10 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87eeffd640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac36a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ef7fe640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ee7fc640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac36f70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f87d8031ba0 tx=0x7f87d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6372a8426b401754 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ceffd640 1 -- 172.21.15.33:0/2699230958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8053070 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e009e040 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.623+0000 7f87ceffd640 1 -- 172.21.15.33:0/2699230958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87d8005240 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87ceffd640 1 -- 172.21.15.33:0/2699230958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87d805c050 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 10 client.16707 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc9acaf2b0 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87ceffd640 1 -- 172.21.15.33:0/2699230958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e009e040 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87ceffd640 1 -- 172.21.15.33:0/2699230958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f87d8056020 con 0x55fc9ac2bc20 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 10 client.16707 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 10 client.16707 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 20 client.16707 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9acafc90 0x55fc9acb20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 4 client.16707 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc9ab96f00 con 0x55fc9acafc90 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.624+0000 7f87eeffd640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9acafc90 0x55fc9acb20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87eeffd640 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9acafc90 0x55fc9acb20b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=819b3bd3e67ed61f server_cookie=29e7612d039ccc4a in_seq=0 out_seq=0 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87ceffd640 10 client.16707 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87ceffd640 1 -- 172.21.15.33:0/2699230958 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc9ab96f00 con 0x55fc9acafc90 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87ceffd640 10 client.16707 handle_command_reply: tid=0 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 client.16707 shutdown 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 msgr2=0x55fc9ac36f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac36f70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f87d8031ba0 tx=0x7f87d8009260 comp rx=0 tx=0).stop 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 shutdown_connections 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55fc9acafc90 0x55fc9acb20b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87ceffd640 0 client.16707 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc9ac30d90 0x55fc9ac40be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc9ac2bc20 0x55fc9ac36f70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 --2- 172.21.15.33:0/2699230958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9ac2b230 0x55fc9ac36a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 >> 172.21.15.33:0/2699230958 conn(0x55fc9ac268a0 msgr2=0x55fc9ac2c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 shutdown_connections 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 1 -- 172.21.15.33:0/2699230958 wait complete. 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.625+0000 7f87f876d740 20 client.16707 trim_cache size 0 max 16384 2024-04-24T23:18:05.633 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:18:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87dc0a2b90 msgr2=0x7f87dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87dc0a2b90 0x7f87dc0a5050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f87d80314e0 tx=0x7f87d8005fd0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 msgr2=0x7f87f019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 0x7f87f019f080 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f87e000eb10 tx=0x7f87e000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 shutdown_connections 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87dc0a2b90 0x7f87dc0a5050 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87f0102230 0x7f87f019f080 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f0101870 0x7f87f0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 --2- 172.21.15.33:0/826597648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f0100680 0x7f87f0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 >> 172.21.15.33:0/826597648 conn(0x7f87f00fbbf0 msgr2=0x7f87f00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 shutdown_connections 2024-04-24T23:18:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.626+0000 7f87f597e640 1 -- 172.21.15.33:0/826597648 wait complete. 2024-04-24T23:18:05.638 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:05 smithi033 ceph-mon[29201]: pgmap v533: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-04-24T23:18:05.638 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:05 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2955509756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:05.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 -- 172.21.15.33:0/124451580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 msgr2=0x7ff2141045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 --2- 172.21.15.33:0/124451580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff2141045f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff204009ca0 tx=0x7ff20402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 -- 172.21.15.33:0/124451580 shutdown_connections 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 --2- 172.21.15.33:0/124451580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff214107460 0x7ff2141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 --2- 172.21.15.33:0/124451580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214104b30 0x7ff214106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 --2- 172.21.15.33:0/124451580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff2141045f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 -- 172.21.15.33:0/124451580 >> 172.21.15.33:0/124451580 conn(0x7ff2140fbbf0 msgr2=0x7ff2140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 -- 172.21.15.33:0/124451580 shutdown_connections 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.750+0000 7ff219b90640 1 -- 172.21.15.33:0/124451580 wait complete. 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 Processor -- start 2024-04-24T23:18:05.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 -- start start 2024-04-24T23:18:05.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff214197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff214104b30 0x7ff214198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214107460 0x7ff21419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff21410d430 con 0x7ff214102200 2024-04-24T23:18:05.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff21410d2b0 con 0x7ff214104b30 2024-04-24T23:18:05.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff219b90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff21410d5b0 con 0x7ff214107460 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff214197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff214197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49002/0 (socket says 172.21.15.33:49002) 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff2137fe640 1 -- 172.21.15.33:0/1486853926 learned_addr learned my addr 172.21.15.33:0/1486853926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff213fff640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214107460 0x7ff21419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.751+0000 7ff212ffd640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff214104b30 0x7ff214198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff2137fe640 1 -- 172.21.15.33:0/1486853926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214107460 msgr2=0x7ff21419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff2137fe640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214107460 0x7ff21419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff2137fe640 1 -- 172.21.15.33:0/1486853926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff214104b30 msgr2=0x7ff214198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff2137fe640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff214104b30 0x7ff214198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff2137fe640 1 -- 172.21.15.33:0/1486853926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419f760 con 0x7ff214102200 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff213fff640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214107460 0x7ff21419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff2137fe640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff214197b40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff204000f80 tx=0x7ff204031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff210ff9640 1 -- 172.21.15.33:0/1486853926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20402fa20 con 0x7ff214102200 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419f9f0 con 0x7ff214102200 2024-04-24T23:18:05.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff210ff9640 1 -- 172.21.15.33:0/1486853926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff20402fbc0 con 0x7ff214102200 2024-04-24T23:18:05.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff210ff9640 1 -- 172.21.15.33:0/1486853926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20403a6f0 con 0x7ff214102200 2024-04-24T23:18:05.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.752+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2140ffff0 con 0x7ff214102200 2024-04-24T23:18:05.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff210ff9640 1 -- 172.21.15.33:0/1486853926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff20403a890 con 0x7ff214102200 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 Processor -- start 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 -- start start 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e59c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e9e500670 con 0x561e9e59cc20 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561e9e4a8970 con 0x561e9e59c230 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff21c97a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561e9e4fc630 con 0x561e9e5a1d90 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49016/0 (socket says 172.21.15.33:49016) 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 -- 172.21.15.33:0/3982089774 learned_addr learned my addr 172.21.15.33:0/3982089774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff213fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e59c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff212ffd640 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff210ff9640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1fc0a6fa0 0x7ff1fc0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 -- 172.21.15.33:0/3982089774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 msgr2=0x561e9e5a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 -- 172.21.15.33:0/3982089774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 msgr2=0x561e9e59c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e59c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff2137fe640 1 -- 172.21.15.33:0/3982089774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e9e4cc400 con 0x561e9e59cc20 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff213fff640 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e59c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff212ffd640 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.755+0000 7ff213fff640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1fc0a6fa0 0x7ff1fc0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.756+0000 7ff210ff9640 1 -- 172.21.15.33:0/1486853926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2040f0a90 con 0x7ff214102200 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.756+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a1720 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff204008930 tx=0x7ff20404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98736caa21fb10b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.756+0000 7ff1f9ffb640 1 -- 172.21.15.33:0/3982089774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040bc950 con 0x561e9e59cc20 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.756+0000 7ff213fff640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1fc0a6fa0 0x7ff1fc0a9460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x561e9e59d480 tx=0x7ff20c0046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.756+0000 7ff1f9ffb640 1 -- 172.21.15.33:0/3982089774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2040fa040 con 0x561e9e59cc20 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.756+0000 7ff1f9ffb640 1 -- 172.21.15.33:0/3982089774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040bd6f0 con 0x561e9e59cc20 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 -- 172.21.15.33:0/3982089774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 msgr2=0x561e9e5a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a1720 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff204008930 tx=0x7ff20404d000 comp rx=0 tx=0).stop 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 -- 172.21.15.33:0/3982089774 shutdown_connections 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a1720 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3982089774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e59c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 -- 172.21.15.33:0/3982089774 >> 172.21.15.33:0/3982089774 conn(0x561e9e5978a0 msgr2=0x561e9e597ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 -- 172.21.15.33:0/3982089774 shutdown_connections 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 -- 172.21.15.33:0/3982089774 wait complete. 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 -- start start 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 Processor -- start 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e5a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.757+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e9e500670 con 0x561e9e59cc20 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561e9e4bc050 con 0x561e9e59c230 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561e9e5a4b00 con 0x561e9e5a1d90 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 10 client.0 _ll_register_callbacks cb 0x561e9e213be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e5a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e5a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51190/0 (socket says 172.21.15.33:51190) 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49020/0 (socket says 172.21.15.33:49020) 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff2137fe640 1 -- 172.21.15.33:0/3538998227 learned_addr learned my addr 172.21.15.33:0/3538998227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff213fff640 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 -- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 msgr2=0x561e9e5ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 -- 172.21.15.33:0/3538998227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 msgr2=0x561e9e5a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e5a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 -- 172.21.15.33:0/3538998227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e9e48ddc0 con 0x561e9e59cc20 2024-04-24T23:18:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff212ffd640 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a63c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff214199050 tx=0x7ff208012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e763699d41cb9ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff1f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3538998227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208004030 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3538998227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2080041d0 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2040fa040 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3538998227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff208025050 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 10 client.16719 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.758+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e9e2143f0 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3538998227 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2040fa040 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3538998227 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff208003940 con 0x561e9e59cc20 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff21c97a740 10 client.16719 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff21c97a740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff21c97a740 20 client.16719 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e5b1d20 0x561e9e5b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff21c97a740 4 client.16719 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e9e50dbb0 con 0x561e9e5b1d20 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e5b1d20 0x561e9e5b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e5b1d20 0x561e9e5b41a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b861105df7719f7d server_cookie=9f844cae4f7de51c in_seq=0 out_seq=0 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.759+0000 7ff1f2ffd640 10 client.16719 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3538998227 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e9e50dbb0 con 0x561e9e5b1d20 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff1f2ffd640 10 client.16719 handle_command_reply: tid=0 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 client.16719 shutdown 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 msgr2=0x561e9e5a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a63c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff214199050 tx=0x7ff208012b60 comp rx=0 tx=0).stop 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 shutdown_connections 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e5b1d20 0x561e9e5b41a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e5a1d90 0x561e9e5ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e59cc20 0x561e9e5a63c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3538998227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59c230 0x561e9e5a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 >> 172.21.15.33:0/3538998227 conn(0x561e9e5978a0 msgr2=0x561e9e596b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.760+0000 7ff1f2ffd640 0 client.16719 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:05.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.761+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 shutdown_connections 2024-04-24T23:18:05.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.761+0000 7ff21c97a740 1 -- 172.21.15.33:0/3538998227 wait complete. 2024-04-24T23:18:05.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.761+0000 7ff21c97a740 20 client.16719 trim_cache size 0 max 16384 2024-04-24T23:18:05.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.774+0000 7ff21c97a740 1 Processor -- start 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.774+0000 7ff21c97a740 1 -- start start 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e59a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff21c97a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141aaaa0 con 0x561e9e5a1d90 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff21c97a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141a8370 con 0x561e9e59cc20 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff21c97a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141a61c0 con 0x561e9e59c230 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff213fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e59a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51200/0 (socket says 172.21.15.33:51200) 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 -- 172.21.15.33:0/104745010 learned_addr learned my addr 172.21.15.33:0/104745010 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff2137fe640 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 -- 172.21.15.33:0/104745010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 msgr2=0x561e9e5b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 -- 172.21.15.33:0/104745010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 msgr2=0x561e9e59a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e59a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 -- 172.21.15.33:0/104745010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e9e2de7d0 con 0x561e9e59cc20 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff2137fe640 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff213fff640 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e59a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff212ffd640 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ad3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff208003f50 tx=0x7ff208009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0180b3094b0c585 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.775+0000 7ff1f97fa640 1 -- 172.21.15.33:0/104745010 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208022040 con 0x561e9e59cc20 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff1f97fa640 1 -- 172.21.15.33:0/104745010 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208003ad0 con 0x561e9e59cc20 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff1f97fa640 1 -- 172.21.15.33:0/104745010 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208010b00 con 0x561e9e59cc20 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 -- 172.21.15.33:0/104745010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 msgr2=0x561e9e5ad3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ad3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff208003f50 tx=0x7ff208009ee0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 -- 172.21.15.33:0/104745010 shutdown_connections 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e59a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ad3c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 --2- 172.21.15.33:0/104745010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 -- 172.21.15.33:0/104745010 >> 172.21.15.33:0/104745010 conn(0x561e9e5978a0 msgr2=0x561e9e596b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 -- 172.21.15.33:0/104745010 shutdown_connections 2024-04-24T23:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.776+0000 7ff21c97a740 1 -- 172.21.15.33:0/104745010 wait complete. 2024-04-24T23:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 -- start start 2024-04-24T23:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 Processor -- start 2024-04-24T23:18:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e5ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141a6830 con 0x561e9e5a1d90 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141a7ca0 con 0x561e9e59cc20 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141a6f00 con 0x561e9e59c230 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff21c97a740 10 client.0 _ll_register_callbacks cb 0x7ff208009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51202/0 (socket says 172.21.15.33:51202) 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff212ffd640 1 -- 172.21.15.33:0/3785060225 learned_addr learned my addr 172.21.15.33:0/3785060225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff213fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e5ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.777+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff213fff640 1 -- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 msgr2=0x561e9e5aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff213fff640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff213fff640 1 -- 172.21.15.33:0/3785060225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 msgr2=0x561e9e5ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff213fff640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff213fff640 1 -- 172.21.15.33:0/3785060225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e9e5ac6f0 con 0x561e9e5a1d90 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff212ffd640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff213fff640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e5ac100 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff20c003e50 tx=0x7ff20c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e9ee5b62991392 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff1f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3785060225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c010bc0 con 0x561e9e5a1d90 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3785060225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff20c026050 con 0x561e9e5a1d90 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff208003ad0 con 0x561e9e5a1d90 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3785060225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff20c0047a0 con 0x561e9e5a1d90 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff21c97a740 10 client.16725 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e9e618480 con 0x561e9e5a1d90 2024-04-24T23:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.778+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3785060225 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208003ad0 con 0x561e9e5a1d90 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3785060225 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff20c02a400 con 0x561e9e5a1d90 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff21c97a740 10 client.16725 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff21c97a740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff21c97a740 20 client.16725 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e618ed0 0x561e9e61b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff21c97a740 4 client.16725 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e9e507f00 con 0x561e9e618ed0 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e618ed0 0x561e9e61b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff1f2ffd640 10 client.16725 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.779+0000 7ff2137fe640 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e618ed0 0x561e9e61b2f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab6d64768823ce0 server_cookie=ed2692f04a5b0ec4 in_seq=0 out_seq=0 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff1f2ffd640 1 -- 172.21.15.33:0/3785060225 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e9e507f00 con 0x561e9e618ed0 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff1f2ffd640 10 client.16725 handle_command_reply: tid=0 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 client.16725 shutdown 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 msgr2=0x561e9e5ac100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e5ac100 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff20c003e50 tx=0x7ff20c004730 comp rx=0 tx=0).stop 2024-04-24T23:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 shutdown_connections 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e9e618ed0 0x561e9e61b2f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff1f2ffd640 0 client.16725 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e9e5a1d90 0x561e9e5ac100 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e9e59cc20 0x561e9e5ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 --2- 172.21.15.33:0/3785060225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e9e59c230 0x561e9e5aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 >> 172.21.15.33:0/3785060225 conn(0x561e9e5978a0 msgr2=0x561e9e59d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 shutdown_connections 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 1 -- 172.21.15.33:0/3785060225 wait complete. 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.780+0000 7ff21c97a740 20 client.16725 trim_cache size 0 max 16384 2024-04-24T23:18:05.788 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1fc0a6fa0 msgr2=0x7ff1fc0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1fc0a6fa0 0x7ff1fc0a9460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x561e9e59d480 tx=0x7ff20c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 msgr2=0x7ff214197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff214197b40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff204000f80 tx=0x7ff204031ee0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 shutdown_connections 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1fc0a6fa0 0x7ff1fc0a9460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214107460 0x7ff21419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff214104b30 0x7ff214198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 --2- 172.21.15.33:0/1486853926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff214102200 0x7ff214197b40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 >> 172.21.15.33:0/1486853926 conn(0x7ff2140fbbf0 msgr2=0x7ff214102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 shutdown_connections 2024-04-24T23:18:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:05.781+0000 7ff219b90640 1 -- 172.21.15.33:0/1486853926 wait complete. 2024-04-24T23:18:05.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:05 smithi038 ceph-mon[32450]: pgmap v533: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-04-24T23:18:05.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:05 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2955509756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:05 smithi117 ceph-mon[32529]: pgmap v533: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-04-24T23:18:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:05 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2955509756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:06.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 608147 12.87 MB/sec execute 131 sec latency 22.099 ms 2024-04-24T23:18:06.796 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:18:06.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:06.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:18:06.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.907+0000 7fd8c169a640 1 -- 172.21.15.33:0/3534755513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 msgr2=0x7fd8bc1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.907+0000 7fd8c169a640 1 --2- 172.21.15.33:0/3534755513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1057a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009860 tx=0x7fd8a402a340 comp rx=0 tx=0).stop 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 -- 172.21.15.33:0/3534755513 shutdown_connections 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 --2- 172.21.15.33:0/3534755513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1057a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 --2- 172.21.15.33:0/3534755513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc102970 0x7fd8bc104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 --2- 172.21.15.33:0/3534755513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8bc100000 0x7fd8bc102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 -- 172.21.15.33:0/3534755513 >> 172.21.15.33:0/3534755513 conn(0x7fd8bc0fbc00 msgr2=0x7fd8bc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 -- 172.21.15.33:0/3534755513 shutdown_connections 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 -- 172.21.15.33:0/3534755513 wait complete. 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 Processor -- start 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.908+0000 7fd8c169a640 1 -- start start 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8c169a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 0x7fd8bc19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8c169a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8bc102970 0x7fd8bc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8c169a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8c169a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1139e0 con 0x7fd8bc102970 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8c169a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc113860 con 0x7fd8bc105320 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8c169a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc113b60 con 0x7fd8bc100000 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 0x7fd8bc19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 0x7fd8bc19e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41382/0 (socket says 172.21.15.33:41382) 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 -- 172.21.15.33:0/4229203661 learned_addr learned my addr 172.21.15.33:0/4229203661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8bc102970 0x7fd8bc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 -- 172.21.15.33:0/4229203661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 msgr2=0x7fd8bc1a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 -- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8bc102970 msgr2=0x7fd8bc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8bc102970 0x7fd8bc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8baffd640 1 -- 172.21.15.33:0/4229203661 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc1a5ca0 con 0x7fd8bc100000 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.909+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.910+0000 7fd8baffd640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 0x7fd8bc19e080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00e9c0 tx=0x7fd8ac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.910+0000 7fd8b3fff640 1 -- 172.21.15.33:0/4229203661 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00cde0 con 0x7fd8bc100000 2024-04-24T23:18:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.910+0000 7fd8b3fff640 1 -- 172.21.15.33:0/4229203661 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac0035f0 con 0x7fd8bc100000 2024-04-24T23:18:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.910+0000 7fd8b3fff640 1 -- 172.21.15.33:0/4229203661 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac010690 con 0x7fd8bc100000 2024-04-24T23:18:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.910+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8bc1a5f90 con 0x7fd8bc100000 2024-04-24T23:18:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.910+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8bc1a6450 con 0x7fd8bc100000 2024-04-24T23:18:06.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.913+0000 7fd8b3fff640 1 -- 172.21.15.33:0/4229203661 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd8ac0037a0 con 0x7fd8bc100000 2024-04-24T23:18:06.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.913+0000 7fd8c43a2740 1 Processor -- start 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.913+0000 7fd8c43a2740 1 -- start start 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8b3fff640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8a80a2c10 0x7fd8a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312767280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312769d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5573126c52c0 con 0x557312761d90 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55731266d910 con 0x557312762780 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5573126c1280 con 0x5573127678f0 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8a80a2c10 0x7fd8a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8b3fff640 1 -- 172.21.15.33:0/4229203661 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd8ac0cf6b0 con 0x7fd8bc100000 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8ba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312767280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8ba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312767280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51230/0 (socket says 172.21.15.33:51230) 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/3956796037 learned_addr learned my addr 172.21.15.33:0/3956796037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312769d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8baffd640 1 -- 172.21.15.33:0/3956796037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 msgr2=0x557312769d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8baffd640 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312769d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8baffd640 1 -- 172.21.15.33:0/3956796037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 msgr2=0x557312767280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8baffd640 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312767280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8baffd640 1 -- 172.21.15.33:0/3956796037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55731267efb0 con 0x557312761d90 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312767280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.914+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8a80a2c10 0x7fd8a80a50d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40305b0 tx=0x7fd8a4005d10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.915+0000 7fd8baffd640 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127621b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c6a0 tx=0x7fd8ac09b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc80fd7b1f07b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.915+0000 7fd8b17fa640 1 -- 172.21.15.33:0/3956796037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00ad90 con 0x557312761d90 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.915+0000 7fd8b17fa640 1 -- 172.21.15.33:0/3956796037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac09ce50 con 0x557312761d90 2024-04-24T23:18:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.915+0000 7fd8b17fa640 1 -- 172.21.15.33:0/3956796037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac015db0 con 0x557312761d90 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.915+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3956796037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 msgr2=0x5573127621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.915+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127621b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c6a0 tx=0x7fd8ac09b480 comp rx=0 tx=0).stop 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3956796037 shutdown_connections 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312769d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312767280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3956796037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127621b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3956796037 >> 172.21.15.33:0/3956796037 conn(0x557312729950 msgr2=0x55731275d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3956796037 shutdown_connections 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3956796037 wait complete. 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- start start 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 Processor -- start 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731276c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731276c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x5573127737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55731276a8a0 con 0x557312762780 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557312680e90 con 0x557312761d90 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55731276aa20 con 0x5573127678f0 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8c43a2740 10 client.0 _ll_register_callbacks cb 0x5573126bc970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731276c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731276c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49080/0 (socket says 172.21.15.33:49080) 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1946776927 learned_addr learned my addr 172.21.15.33:0/1946776927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8baffd640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731276c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.916+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x5573127737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 msgr2=0x5573127737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x5573127737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1946776927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 msgr2=0x55731276c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731276c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1946776927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573124a17e0 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731276c790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4011ce0 tx=0x7fd8b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc24eedb48ea2c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8a27fc640 1 -- 172.21.15.33:0/1946776927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4003970 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8a27fc640 1 -- 172.21.15.33:0/1946776927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd8b4003b10 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8ac09ce50 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8a27fc640 1 -- 172.21.15.33:0/1946776927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8b4025050 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8c43a2740 10 client.16737 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557312501f70 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8a27fc640 1 -- 172.21.15.33:0/1946776927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac09ce50 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8a27fc640 1 -- 172.21.15.33:0/1946776927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd8b401a4a0 con 0x557312762780 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8c43a2740 10 client.16737 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.917+0000 7fd8c43a2740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8c43a2740 20 client.16737 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x55731277a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8c43a2740 4 client.16737 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573126995a0 con 0x557312777d70 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8baffd640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x55731277a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8baffd640 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x55731277a210 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9242c01b6f9bfc05 server_cookie=411f3fb4d0a31112 in_seq=0 out_seq=0 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.918+0000 7fd8a27fc640 10 client.16737 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8a27fc640 1 -- 172.21.15.33:0/1946776927 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573126995a0 con 0x557312777d70 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8a27fc640 10 client.16737 handle_command_reply: tid=0 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 client.16737 shutdown 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 msgr2=0x55731276c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731276c790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4011ce0 tx=0x7fd8b400f790 comp rx=0 tx=0).stop 2024-04-24T23:18:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 shutdown_connections 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x55731277a210 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x5573127737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731276c790 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8a27fc640 0 client.16737 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1946776927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731276c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 >> 172.21.15.33:0/1946776927 conn(0x557312729950 msgr2=0x55731275c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 shutdown_connections 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1946776927 wait complete. 2024-04-24T23:18:06.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.919+0000 7fd8c43a2740 20 client.16737 trim_cache size 0 max 16384 2024-04-24T23:18:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 Processor -- start 2024-04-24T23:18:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 -- start start 2024-04-24T23:18:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731275c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731275cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312773790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b33e0 con 0x557312762780 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b0cb0 con 0x557312761d90 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1aeb00 con 0x5573127678f0 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731275cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731275cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49090/0 (socket says 172.21.15.33:49090) 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1313483603 learned_addr learned my addr 172.21.15.33:0/1313483603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312773790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.940+0000 7fd8baffd640 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731275c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1313483603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 msgr2=0x557312773790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312773790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1313483603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 msgr2=0x55731275c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731275c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8ba7fc640 1 -- 172.21.15.33:0/1313483603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557312399170 con 0x557312762780 2024-04-24T23:18:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312773790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731275cd10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009730 tx=0x7fd8b401adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4660c2f0dbbab9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8b0ff9640 1 -- 172.21.15.33:0/1313483603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b402d020 con 0x557312762780 2024-04-24T23:18:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8b0ff9640 1 -- 172.21.15.33:0/1313483603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8b4003e50 con 0x557312762780 2024-04-24T23:18:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.941+0000 7fd8b0ff9640 1 -- 172.21.15.33:0/1313483603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4005310 con 0x557312762780 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1313483603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 msgr2=0x55731275cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731275cd10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009730 tx=0x7fd8b401adb0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1313483603 shutdown_connections 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312773790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312762780 0x55731275cd10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/1313483603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312761d90 0x55731275c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1313483603 >> 172.21.15.33:0/1313483603 conn(0x557312729950 msgr2=0x55731275c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1313483603 shutdown_connections 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 -- 172.21.15.33:0/1313483603 wait complete. 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 -- start start 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.942+0000 7fd8c43a2740 1 Processor -- start 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312771c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312772c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1af170 con 0x557312761d90 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1b05e0 con 0x557312762780 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1af840 con 0x5573127678f0 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 10 client.0 _ll_register_callbacks cb 0x7fd8b4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49100/0 (socket says 172.21.15.33:49100) 2024-04-24T23:18:06.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 -- 172.21.15.33:0/3669251303 learned_addr learned my addr 172.21.15.33:0/3669251303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312771c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8bb7fe640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312772c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 -- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 msgr2=0x557312772c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312772c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 -- 172.21.15.33:0/3669251303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 msgr2=0x557312771c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312771c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 -- 172.21.15.33:0/3669251303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557312773210 con 0x557312761d90 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8baffd640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127716d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0984f0 tx=0x7fd8ac097dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b385b2e6cf34b11c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8a27fc640 1 -- 172.21.15.33:0/3669251303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac07e070 con 0x557312761d90 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8a27fc640 1 -- 172.21.15.33:0/3669251303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd8ac003b10 con 0x557312761d90 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.943+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b4003e50 con 0x557312761d90 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8a27fc640 1 -- 172.21.15.33:0/3669251303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8ac09ce50 con 0x557312761d90 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 10 client.16743 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573127debd0 con 0x557312761d90 2024-04-24T23:18:06.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8a27fc640 1 -- 172.21.15.33:0/3669251303 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8b4003e50 con 0x557312761d90 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8a27fc640 1 -- 172.21.15.33:0/3669251303 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd8ac0cf070 con 0x557312761d90 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 10 client.16743 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 20 client.16743 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x5573127e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 4 client.16743 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573126d2800 con 0x557312777d70 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.944+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x5573127e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.945+0000 7fd8a27fc640 10 client.16743 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.945+0000 7fd8ba7fc640 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x5573127e11a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2bdec7adb3670 server_cookie=5ef60216562f8602 in_seq=0 out_seq=0 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.945+0000 7fd8a27fc640 1 -- 172.21.15.33:0/3669251303 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573126d2800 con 0x557312777d70 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.945+0000 7fd8a27fc640 10 client.16743 handle_command_reply: tid=0 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.945+0000 7fd8c43a2740 1 client.16743 shutdown 2024-04-24T23:18:06.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 msgr2=0x5573127716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127716d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0984f0 tx=0x7fd8ac097dc0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 shutdown_connections 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557312777d70 0x5573127e11a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573127678f0 0x557312772c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8a27fc640 0 client.16743 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557312762780 0x557312771c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 --2- 172.21.15.33:0/3669251303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557312761d90 0x5573127716d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 >> 172.21.15.33:0/3669251303 conn(0x557312729950 msgr2=0x55731275c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 shutdown_connections 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 1 -- 172.21.15.33:0/3669251303 wait complete. 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c43a2740 20 client.16743 trim_cache size 0 max 16384 2024-04-24T23:18:06.953 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:18:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:18:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "cc322fba-2c53-4cb5-9327-b5bf7e903dd0", 2024-04-24T23:18:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:18:06.954 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8a80a2c10 msgr2=0x7fd8a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c169a640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8a80a2c10 0x7fd8a80a50d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40305b0 tx=0x7fd8a4005d10 comp rx=0 tx=0).stop 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 msgr2=0x7fd8bc19e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.946+0000 7fd8c169a640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 0x7fd8bc19e080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00e9c0 tx=0x7fd8ac00ee80 comp rx=0 tx=0).stop 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 shutdown_connections 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8a80a2c10 0x7fd8a80a50d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8bc105320 0x7fd8bc1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8bc102970 0x7fd8bc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 --2- 172.21.15.33:0/4229203661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8bc100000 0x7fd8bc19e080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 >> 172.21.15.33:0/4229203661 conn(0x7fd8bc0fbc00 msgr2=0x7fd8bc100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 shutdown_connections 2024-04-24T23:18:06.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:06.947+0000 7fd8c169a640 1 -- 172.21.15.33:0/4229203661 wait complete. 2024-04-24T23:18:06.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:18:06.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 -- 172.21.15.33:0/663793656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 msgr2=0x7fee64105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 --2- 172.21.15.33:0/663793656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee64105c60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fee54009900 tx=0x7fee5402a410 comp rx=0 tx=0).stop 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 -- 172.21.15.33:0/663793656 shutdown_connections 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 --2- 172.21.15.33:0/663793656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64106280 0x7fee64108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 --2- 172.21.15.33:0/663793656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee64105c60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 --2- 172.21.15.33:0/663793656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee64100f40 0x7fee64103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 -- 172.21.15.33:0/663793656 >> 172.21.15.33:0/663793656 conn(0x7fee640faf40 msgr2=0x7fee640fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 -- 172.21.15.33:0/663793656 shutdown_connections 2024-04-24T23:18:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 -- 172.21.15.33:0/663793656 wait complete. 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 Processor -- start 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 -- start start 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64100f40 0x7fee64100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.072+0000 7fee6ab63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee6419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee6ab63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee64106280 0x7fee6419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee6ab63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410c320 con 0x7fee64103870 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee6ab63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410c1a0 con 0x7fee64106280 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee6ab63640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410c4a0 con 0x7fee64100f40 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee6419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee6419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49120/0 (socket says 172.21.15.33:49120) 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 -- 172.21.15.33:0/2691461257 learned_addr learned my addr 172.21.15.33:0/2691461257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee688ff640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64100f40 0x7fee64100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee69100640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee64106280 0x7fee6419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 -- 172.21.15.33:0/2691461257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64100f40 msgr2=0x7fee64100820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64100f40 0x7fee64100820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 -- 172.21.15.33:0/2691461257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee64106280 msgr2=0x7fee6419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee64106280 0x7fee6419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 -- 172.21.15.33:0/2691461257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6419f740 con 0x7fee64103870 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee63fff640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee6419c810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fee54009230 tx=0x7fee540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee61ffb640 1 -- 172.21.15.33:0/2691461257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5403f030 con 0x7fee64103870 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee61ffb640 1 -- 172.21.15.33:0/2691461257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54003ce0 con 0x7fee64103870 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6419f9d0 con 0x7fee64103870 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.073+0000 7fee61ffb640 1 -- 172.21.15.33:0/2691461257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54035070 con 0x7fee64103870 2024-04-24T23:18:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.074+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6419fec0 con 0x7fee64103870 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee61ffb640 1 -- 172.21.15.33:0/2691461257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fee54042030 con 0x7fee64103870 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 Processor -- start 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 -- start start 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59240 0x55ad1bd59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bcbd6b0 con 0x55ad1bd5eda0 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bc65880 con 0x55ad1bd59c30 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee6d878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bcb9670 con 0x55ad1bd59240 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee69100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59240 0x55ad1bd59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee688ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.076+0000 7fee61ffb640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fee500a2b90 0x7fee500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd5e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51270/0 (socket says 172.21.15.33:51270) 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee69100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59240 0x55ad1bd59660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41420/0 (socket says 172.21.15.33:41420) 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 -- 172.21.15.33:0/1534831175 learned_addr learned my addr 172.21.15.33:0/1534831175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd61230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49134/0 (socket says 172.21.15.33:49134) 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 -- 172.21.15.33:0/1534831175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59240 msgr2=0x55ad1bd59660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee61ffb640 1 -- 172.21.15.33:0/2691461257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fee540ef840 con 0x7fee64103870 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59240 0x55ad1bd59660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 -- 172.21.15.33:0/1534831175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd5eda0 msgr2=0x55ad1bd61230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd61230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 -- 172.21.15.33:0/1534831175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1ba9b7d0 con 0x55ad1bd59c30 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee69100640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fee500a2b90 0x7fee500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee688ff640 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd5e730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009c90 tx=0x7fee4c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77fb330f39bf744c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee69100640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fee500a2b90 0x7fee500a5050 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55ad1bd5a490 tx=0x7fee5c004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee5affd640 1 -- 172.21.15.33:0/1534831175 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c01e070 con 0x55ad1bd59c30 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee5affd640 1 -- 172.21.15.33:0/1534831175 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c019b10 con 0x55ad1bd59c30 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.077+0000 7fee5affd640 1 -- 172.21.15.33:0/1534831175 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00ea70 con 0x55ad1bd59c30 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 -- 172.21.15.33:0/1534831175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 msgr2=0x55ad1bd5e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd5e730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009c90 tx=0x7fee4c011810 comp rx=0 tx=0).stop 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 -- 172.21.15.33:0/1534831175 shutdown_connections 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd5e730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 --2- 172.21.15.33:0/1534831175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59240 0x55ad1bd59660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 -- 172.21.15.33:0/1534831175 >> 172.21.15.33:0/1534831175 conn(0x55ad1bd548b0 msgr2=0x55ad1bd54cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 -- 172.21.15.33:0/1534831175 shutdown_connections 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 -- 172.21.15.33:0/1534831175 wait complete. 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.078+0000 7fee6d878740 1 -- start start 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 Processor -- start 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd63690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59c30 0x55ad1bd63bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd6ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bd61d50 con 0x55ad1bd59240 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bc78f50 con 0x55ad1bd5eda0 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bd61ed0 con 0x55ad1bd59c30 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee6d878740 10 client.0 _ll_register_callbacks cb 0x55ad1bcb4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee688ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd63690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee63fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59c30 0x55ad1bd63bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee688ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd63690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49138/0 (socket says 172.21.15.33:49138) 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee688ff640 1 -- 172.21.15.33:0/502598072 learned_addr learned my addr 172.21.15.33:0/502598072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee69100640 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd6ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee63fff640 1 -- 172.21.15.33:0/502598072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd5eda0 msgr2=0x55ad1bd6ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee63fff640 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd6ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee63fff640 1 -- 172.21.15.33:0/502598072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 msgr2=0x55ad1bd63690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee63fff640 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd63690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.079+0000 7fee63fff640 1 -- 172.21.15.33:0/502598072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1b9d13f0 con 0x55ad1bd59c30 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee63fff640 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59c30 0x55ad1bd63bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee540ed000 tx=0x7fee54003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3b16fd97d8a0b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee43fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee43fff640 1 -- 172.21.15.33:0/502598072 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54049040 con 0x55ad1bd59c30 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4c019b10 con 0x55ad1bd59c30 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee43fff640 1 -- 172.21.15.33:0/502598072 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fee540b8430 con 0x55ad1bd59c30 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee43fff640 1 -- 172.21.15.33:0/502598072 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee540bb720 con 0x55ad1bd59c30 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee6d878740 10 client.26281 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad1b9910f0 con 0x55ad1bd59c30 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.080+0000 7fee43fff640 1 -- 172.21.15.33:0/502598072 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c019b10 con 0x55ad1bd59c30 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee43fff640 1 -- 172.21.15.33:0/502598072 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fee5403bc10 con 0x55ad1bd59c30 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee6d878740 10 client.26281 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee6d878740 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee6d878740 20 client.26281 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee6d878740 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bd6f4e0 0x55ad1bd71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee6d878740 4 client.26281 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad1bccabf0 con 0x55ad1bd6f4e0 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee688ff640 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bd6f4e0 0x55ad1bd71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee688ff640 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bd6f4e0 0x55ad1bd71950 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c95c4f28713b6a1 server_cookie=6fb4a75c3bde2f6b in_seq=0 out_seq=0 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.081+0000 7fee43fff640 10 client.26281 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee43fff640 1 -- 172.21.15.33:0/502598072 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad1bccabf0 con 0x55ad1bd6f4e0 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee43fff640 10 client.26281 handle_command_reply: tid=0 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 client.26281 shutdown 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59c30 msgr2=0x55ad1bd63bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59c30 0x55ad1bd63bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee540ed000 tx=0x7fee54003e90 comp rx=0 tx=0).stop 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 shutdown_connections 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bd6f4e0 0x55ad1bd71950 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd6ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd59c30 0x55ad1bd63bd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee43fff640 0 client.26281 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 --2- 172.21.15.33:0/502598072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd63690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 >> 172.21.15.33:0/502598072 conn(0x55ad1bd548b0 msgr2=0x55ad1bd53af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 shutdown_connections 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.082+0000 7fee6d878740 1 -- 172.21.15.33:0/502598072 wait complete. 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.083+0000 7fee6d878740 20 client.26281 trim_cache size 0 max 16384 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 Processor -- start 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 -- start start 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd54c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd6abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee641acd20 con 0x55ad1bd59240 2024-04-24T23:18:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee641aa5f0 con 0x55ad1bd59c30 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee6d878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee641a8440 con 0x55ad1bd5eda0 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd54c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd54c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49154/0 (socket says 172.21.15.33:49154) 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 -- 172.21.15.33:0/1228595350 learned_addr learned my addr 172.21.15.33:0/1228595350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee63fff640 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd6abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 -- 172.21.15.33:0/1228595350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 msgr2=0x55ad1bd57630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 -- 172.21.15.33:0/1228595350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 msgr2=0x55ad1bd6abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd6abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.101+0000 7fee688ff640 1 -- 172.21.15.33:0/1228595350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1bd71fa0 con 0x55ad1bd59240 2024-04-24T23:18:07.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.102+0000 7fee688ff640 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd54c90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fee4c011d10 tx=0x7fee4c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8315f83e229e877 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:07.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.102+0000 7fee5a7fc640 1 -- 172.21.15.33:0/1228595350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c01e040 con 0x55ad1bd59240 2024-04-24T23:18:07.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.102+0000 7fee5a7fc640 1 -- 172.21.15.33:0/1228595350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c031070 con 0x55ad1bd59240 2024-04-24T23:18:07.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.102+0000 7fee5a7fc640 1 -- 172.21.15.33:0/1228595350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c02b1d0 con 0x55ad1bd59240 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.102+0000 7fee6d878740 1 -- 172.21.15.33:0/1228595350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 msgr2=0x55ad1bd54c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.102+0000 7fee6d878740 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd54c90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fee4c011d10 tx=0x7fee4c0051d0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- 172.21.15.33:0/1228595350 shutdown_connections 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59c30 0x55ad1bd6abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 --2- 172.21.15.33:0/1228595350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59240 0x55ad1bd54c90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- 172.21.15.33:0/1228595350 >> 172.21.15.33:0/1228595350 conn(0x55ad1bd548b0 msgr2=0x55ad1bd5b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- 172.21.15.33:0/1228595350 shutdown_connections 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- 172.21.15.33:0/1228595350 wait complete. 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- start start 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 Processor -- start 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59240 0x55ad1bd68150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 0x55ad1bd68690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee641a8ab0 con 0x55ad1bd59c30 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee641a9f20 con 0x55ad1bd59240 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee641a9180 con 0x55ad1bd5eda0 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee6d878740 10 client.0 _ll_register_callbacks cb 0x7fee54031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.103+0000 7fee63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 0x55ad1bd68690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59240 0x55ad1bd68150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee69100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 0x55ad1bd68690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49160/0 (socket says 172.21.15.33:49160) 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee63fff640 1 -- 172.21.15.33:0/2272458220 learned_addr learned my addr 172.21.15.33:0/2272458220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 -- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 msgr2=0x55ad1bd696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 -- 172.21.15.33:0/2272458220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 msgr2=0x55ad1bd68690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 0x55ad1bd68690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 -- 172.21.15.33:0/2272458220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1bd69c90 con 0x55ad1bd59240 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee63fff640 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 0x55ad1bd68690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee688ff640 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59240 0x55ad1bd68150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fee4c02e040 tx=0x7fee4c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f8c68c9f87040a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee43fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee43fff640 1 -- 172.21.15.33:0/2272458220 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c005540 con 0x55ad1bd59240 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee43fff640 1 -- 172.21.15.33:0/2272458220 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fee4c019430 con 0x55ad1bd59240 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4c031070 con 0x55ad1bd59240 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.104+0000 7fee43fff640 1 -- 172.21.15.33:0/2272458220 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee4c048050 con 0x55ad1bd59240 2024-04-24T23:18:07.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 10 client.26288 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad1bd6afa0 con 0x55ad1bd59240 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee43fff640 1 -- 172.21.15.33:0/2272458220 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c031070 con 0x55ad1bd59240 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee43fff640 1 -- 172.21.15.33:0/2272458220 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fee4c0035f0 con 0x55ad1bd59240 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 10 client.26288 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 20 client.26288 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bdd6070 0x55ad1bdd8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 4 client.26288 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad1bcc4f40 con 0x55ad1bdd6070 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.105+0000 7fee63fff640 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bdd6070 0x55ad1bdd8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee63fff640 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bdd6070 0x55ad1bdd8490 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fb08e2b8208d2c server_cookie=d09741ddb270c8fa in_seq=0 out_seq=0 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee43fff640 10 client.26288 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee43fff640 1 -- 172.21.15.33:0/2272458220 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ad1bcc4f40 con 0x55ad1bdd6070 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee43fff640 10 client.26288 handle_command_reply: tid=0 2024-04-24T23:18:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 client.26288 shutdown 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59240 msgr2=0x55ad1bd68150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59240 0x55ad1bd68150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fee4c02e040 tx=0x7fee4c009f90 comp rx=0 tx=0).stop 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 shutdown_connections 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ad1bdd6070 0x55ad1bdd8490 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad1bd5eda0 0x55ad1bd696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bd59c30 0x55ad1bd68690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 --2- 172.21.15.33:0/2272458220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad1bd59240 0x55ad1bd68150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee43fff640 0 client.26288 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 >> 172.21.15.33:0/2272458220 conn(0x55ad1bd548b0 msgr2=0x55ad1bd53260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 shutdown_connections 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 1 -- 172.21.15.33:0/2272458220 wait complete. 2024-04-24T23:18:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.106+0000 7fee6d878740 20 client.26288 trim_cache size 0 max 16384 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "cc322fba-2c53-4cb5-9327-b5bf7e903dd0": { 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "cc322fba-2c53-4cb5-9327-b5bf7e903dd0", 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:18:07.115 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fee500a2b90 msgr2=0x7fee500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fee500a2b90 0x7fee500a5050 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55ad1bd5a490 tx=0x7fee5c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 msgr2=0x7fee6419c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee6419c810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fee54009230 tx=0x7fee540042f0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 shutdown_connections 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fee500a2b90 0x7fee500a5050 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee64106280 0x7fee6419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee64103870 0x7fee6419c810 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 --2- 172.21.15.33:0/2691461257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64100f40 0x7fee64100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 >> 172.21.15.33:0/2691461257 conn(0x7fee640faf40 msgr2=0x7fee64101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 shutdown_connections 2024-04-24T23:18:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:07.107+0000 7fee6ab63640 1 -- 172.21.15.33:0/2691461257 wait complete. 2024-04-24T23:18:07.120 INFO:tasks.cephfs.filesystem:scrub status for tag:cc322fba-2c53-4cb5-9327-b5bf7e903dd0 - {'path': '/', 'tag': 'cc322fba-2c53-4cb5-9327-b5bf7e903dd0', 'options': 'recursive,force'} 2024-04-24T23:18:07.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 610889 12.90 MB/sec execute 132 sec latency 80.040 ms 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: pgmap v534: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 257 op/s 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:07 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: pgmap v534: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 257 op/s 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:07 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: pgmap v534: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 257 op/s 2024-04-24T23:18:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:18:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:18:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:18:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:07 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:08.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 612329 12.85 MB/sec execute 133 sec latency 21.528 ms 2024-04-24T23:18:09.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614979 12.86 MB/sec execute 134 sec latency 19.851 ms 2024-04-24T23:18:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:09 smithi038 ceph-mon[32450]: pgmap v535: 129 pgs: 129 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-04-24T23:18:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:09 smithi117 ceph-mon[32529]: pgmap v535: 129 pgs: 129 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-04-24T23:18:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:09 smithi033 ceph-mon[29201]: pgmap v535: 129 pgs: 129 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-04-24T23:18:10.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 617926 12.89 MB/sec execute 135 sec latency 20.281 ms 2024-04-24T23:18:11.446 INFO:tasks.workunit.client.0.smithi033.stdout: 1 620074 12.85 MB/sec execute 136 sec latency 15.901 ms 2024-04-24T23:18:11.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:11 smithi038 ceph-mon[32450]: pgmap v536: 129 pgs: 129 active+clean; 232 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-04-24T23:18:11.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:11 smithi117 ceph-mon[32529]: pgmap v536: 129 pgs: 129 active+clean; 232 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-04-24T23:18:11.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:11 smithi033 ceph-mon[29201]: pgmap v536: 129 pgs: 129 active+clean; 232 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-04-24T23:18:12.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 622121 12.86 MB/sec execute 137 sec latency 85.817 ms 2024-04-24T23:18:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:12 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:18:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:12 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:18:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:12 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:18:13.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 624828 12.87 MB/sec execute 138 sec latency 25.805 ms 2024-04-24T23:18:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:13 smithi038 ceph-mon[32450]: pgmap v537: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 317 op/s 2024-04-24T23:18:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:18:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:13 smithi117 ceph-mon[32529]: pgmap v537: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 317 op/s 2024-04-24T23:18:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:18:13.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:13 smithi033 ceph-mon[29201]: pgmap v537: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 317 op/s 2024-04-24T23:18:13.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:18:14.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 626647 12.85 MB/sec execute 139 sec latency 28.366 ms 2024-04-24T23:18:15.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 629283 12.86 MB/sec execute 140 sec latency 14.209 ms 2024-04-24T23:18:15.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:15 smithi038 ceph-mon[32450]: pgmap v538: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T23:18:15.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:15 smithi117 ceph-mon[32529]: pgmap v538: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T23:18:15.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:15 smithi033 ceph-mon[29201]: pgmap v538: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T23:18:16.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 632108 12.89 MB/sec execute 141 sec latency 15.151 ms 2024-04-24T23:18:17.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 633405 12.82 MB/sec execute 142 sec latency 53.477 ms 2024-04-24T23:18:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:17 smithi038 ceph-mon[32450]: pgmap v539: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-04-24T23:18:17.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:17 smithi117 ceph-mon[32529]: pgmap v539: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-04-24T23:18:17.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:17 smithi033 ceph-mon[29201]: pgmap v539: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-04-24T23:18:18.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 636035 12.85 MB/sec execute 143 sec latency 17.995 ms 2024-04-24T23:18:19.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 638730 12.85 MB/sec execute 144 sec latency 12.791 ms 2024-04-24T23:18:19.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:19 smithi038 ceph-mon[32450]: pgmap v540: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2024-04-24T23:18:19.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:19 smithi117 ceph-mon[32529]: pgmap v540: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2024-04-24T23:18:19.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:19 smithi033 ceph-mon[29201]: pgmap v540: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 243 op/s 2024-04-24T23:18:20.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 640960 12.84 MB/sec execute 145 sec latency 18.718 ms 2024-04-24T23:18:21.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 643476 12.85 MB/sec execute 146 sec latency 24.124 ms 2024-04-24T23:18:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:21 smithi038 ceph-mon[32450]: pgmap v541: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-24T23:18:21.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:21 smithi117 ceph-mon[32529]: pgmap v541: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-24T23:18:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:21 smithi033 ceph-mon[29201]: pgmap v541: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-24T23:18:22.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 646094 12.86 MB/sec execute 147 sec latency 90.495 ms 2024-04-24T23:18:23.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 647417 12.82 MB/sec execute 148 sec latency 64.435 ms 2024-04-24T23:18:23.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:23 smithi038 ceph-mon[32450]: pgmap v542: 129 pgs: 129 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2024-04-24T23:18:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:23 smithi117 ceph-mon[32529]: pgmap v542: 129 pgs: 129 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2024-04-24T23:18:23.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:23 smithi033 ceph-mon[29201]: pgmap v542: 129 pgs: 129 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2024-04-24T23:18:24.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 650232 12.85 MB/sec execute 149 sec latency 88.922 ms 2024-04-24T23:18:25.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 652863 12.85 MB/sec execute 150 sec latency 14.011 ms 2024-04-24T23:18:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:25 smithi038 ceph-mon[32450]: pgmap v543: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-04-24T23:18:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:25 smithi117 ceph-mon[32529]: pgmap v543: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-04-24T23:18:25.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:25 smithi033 ceph-mon[29201]: pgmap v543: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-04-24T23:18:26.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 654702 12.82 MB/sec execute 151 sec latency 16.016 ms 2024-04-24T23:18:27.447 INFO:tasks.workunit.client.0.smithi033.stdout: 1 657226 12.85 MB/sec execute 152 sec latency 53.088 ms 2024-04-24T23:18:27.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:27 smithi038 ceph-mon[32450]: pgmap v544: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 239 op/s 2024-04-24T23:18:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:27 smithi117 ceph-mon[32529]: pgmap v544: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 239 op/s 2024-04-24T23:18:27.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:27 smithi033 ceph-mon[29201]: pgmap v544: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 239 op/s 2024-04-24T23:18:28.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 659330 12.81 MB/sec execute 153 sec latency 53.717 ms 2024-04-24T23:18:29.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 661149 12.80 MB/sec execute 154 sec latency 91.288 ms 2024-04-24T23:18:29.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:29 smithi038 ceph-mon[32450]: pgmap v545: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-04-24T23:18:29.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:29 smithi117 ceph-mon[32529]: pgmap v545: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-04-24T23:18:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:29 smithi033 ceph-mon[29201]: pgmap v545: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-04-24T23:18:30.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664075 12.83 MB/sec execute 155 sec latency 12.811 ms 2024-04-24T23:18:31.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 665837 12.80 MB/sec execute 156 sec latency 21.716 ms 2024-04-24T23:18:31.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:31 smithi038 ceph-mon[32450]: pgmap v546: 129 pgs: 129 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-04-24T23:18:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:31 smithi117 ceph-mon[32529]: pgmap v546: 129 pgs: 129 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-04-24T23:18:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:31 smithi033 ceph-mon[29201]: pgmap v546: 129 pgs: 129 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-04-24T23:18:32.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 668096 12.80 MB/sec execute 157 sec latency 50.173 ms 2024-04-24T23:18:33.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 670631 12.80 MB/sec execute 158 sec latency 17.042 ms 2024-04-24T23:18:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:33 smithi038 ceph-mon[32450]: pgmap v547: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 262 op/s 2024-04-24T23:18:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:33 smithi117 ceph-mon[32529]: pgmap v547: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 262 op/s 2024-04-24T23:18:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:33 smithi033 ceph-mon[29201]: pgmap v547: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 262 op/s 2024-04-24T23:18:34.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 672723 12.80 MB/sec execute 159 sec latency 15.121 ms 2024-04-24T23:18:35.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 675445 12.80 MB/sec execute 160 sec latency 13.920 ms 2024-04-24T23:18:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:35 smithi038 ceph-mon[32450]: pgmap v548: 129 pgs: 129 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:18:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:35 smithi117 ceph-mon[32529]: pgmap v548: 129 pgs: 129 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:18:35.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:35 smithi033 ceph-mon[29201]: pgmap v548: 129 pgs: 129 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:18:36.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 678430 12.83 MB/sec execute 161 sec latency 12.281 ms 2024-04-24T23:18:37.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:18:37.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:18:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 -- 172.21.15.33:0/596179356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c103350 msgr2=0x7f625c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 --2- 172.21.15.33:0/596179356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c103350 0x7f625c109be0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6250009900 tx=0x7f625002a410 comp rx=0 tx=0).stop 2024-04-24T23:18:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 -- 172.21.15.33:0/596179356 shutdown_connections 2024-04-24T23:18:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 --2- 172.21.15.33:0/596179356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c103350 0x7f625c109be0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 --2- 172.21.15.33:0/596179356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c102990 0x7f625c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 --2- 172.21.15.33:0/596179356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c1017a0 0x7f625c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.263+0000 7f6261163640 1 -- 172.21.15.33:0/596179356 >> 172.21.15.33:0/596179356 conn(0x7f625c0fcd10 msgr2=0x7f625c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.264+0000 7f6261163640 1 -- 172.21.15.33:0/596179356 shutdown_connections 2024-04-24T23:18:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.264+0000 7f6261163640 1 -- 172.21.15.33:0/596179356 wait complete. 2024-04-24T23:18:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.264+0000 7f6261163640 1 Processor -- start 2024-04-24T23:18:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.264+0000 7f6261163640 1 -- start start 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c1017a0 0x7f625c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 0x7f625c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c103350 0x7f625c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f625c10d420 con 0x7f625c102990 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f625c10d2a0 con 0x7f625c1017a0 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f625c10d5a0 con 0x7f625c103350 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 0x7f625c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c1017a0 0x7f625c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 0x7f625c197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42086/0 (socket says 172.21.15.33:42086) 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 -- 172.21.15.33:0/1932701882 learned_addr learned my addr 172.21.15.33:0/1932701882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625b59e640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c103350 0x7f625c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 -- 172.21.15.33:0/1932701882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c103350 msgr2=0x7f625c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c103350 0x7f625c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 -- 172.21.15.33:0/1932701882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c1017a0 msgr2=0x7f625c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c1017a0 0x7f625c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 -- 172.21.15.33:0/1932701882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c19f680 con 0x7f625c102990 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625b59e640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c103350 0x7f625c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625ad9d640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c1017a0 0x7f625c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f625a59c640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 0x7f625c197fa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e9f0 tx=0x7f624c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6243fff640 1 -- 172.21.15.33:0/1932701882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c00cd80 con 0x7f625c102990 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625c19f970 con 0x7f625c102990 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6243fff640 1 -- 172.21.15.33:0/1932701882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f624c010910 con 0x7f625c102990 2024-04-24T23:18:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625c19ff10 con 0x7f625c102990 2024-04-24T23:18:37.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.265+0000 7f6243fff640 1 -- 172.21.15.33:0/1932701882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c0188f0 con 0x7f625c102990 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6243fff640 1 -- 172.21.15.33:0/1932701882 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f624c010ac0 con 0x7f625c102990 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6263e78740 1 Processor -- start 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6263e78740 1 -- start start 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6243fff640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f62440a2b90 0x7f62440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6263e78740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 0x56052c382660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6263e78740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382c30 0x56052c387730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.268+0000 7f6263e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c38a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f6263e78740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56052c2e66b0 con 0x56052c382240 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625b59e640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f62440a2b90 0x7f62440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f6263e78740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56052c28e880 con 0x56052c382c30 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f6263e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56052c2e2670 con 0x56052c387da0 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f6243fff640 1 -- 172.21.15.33:0/1932701882 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f624c0cf7f0 con 0x7f625c102990 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 0x56052c382660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 0x56052c382660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42102/0 (socket says 172.21.15.33:42102) 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 -- 172.21.15.33:0/161500796 learned_addr learned my addr 172.21.15.33:0/161500796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625a59c640 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382c30 0x56052c387730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625b59e640 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c38a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 -- 172.21.15.33:0/161500796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 msgr2=0x56052c38a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c38a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 -- 172.21.15.33:0/161500796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382c30 msgr2=0x56052c387730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382c30 0x56052c387730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 -- 172.21.15.33:0/161500796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052c0c47d0 con 0x56052c382240 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625b59e640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f62440a2b90 0x7f62440a5050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f625002a930 tx=0x7f6250032f30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f625ad9d640 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 0x56052c382660 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6254009990 tx=0x7f6254011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b559a3c1cab8634a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f62417fa640 1 -- 172.21.15.33:0/161500796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254016290 con 0x56052c382240 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f62417fa640 1 -- 172.21.15.33:0/161500796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6254003e50 con 0x56052c382240 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.269+0000 7f62417fa640 1 -- 172.21.15.33:0/161500796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004050 con 0x56052c382240 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 -- 172.21.15.33:0/161500796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 msgr2=0x56052c382660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 0x56052c382660 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6254009990 tx=0x7f6254011cf0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 -- 172.21.15.33:0/161500796 shutdown_connections 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c38a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382c30 0x56052c387730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 --2- 172.21.15.33:0/161500796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382240 0x56052c382660 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 -- 172.21.15.33:0/161500796 >> 172.21.15.33:0/161500796 conn(0x56052c37d8b0 msgr2=0x56052c37dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 -- 172.21.15.33:0/161500796 shutdown_connections 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 -- 172.21.15.33:0/161500796 wait complete. 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.270+0000 7f6263e78740 1 -- start start 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 Processor -- start 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c38c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c38cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c393bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56052c38ad50 con 0x56052c382c30 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56052c2a1f50 con 0x56052c382240 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56052c38aed0 con 0x56052c387da0 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f6263e78740 10 client.0 _ll_register_callbacks cb 0x56052c2dd000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c38cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c38cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42116/0 (socket says 172.21.15.33:42116) 2024-04-24T23:18:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 -- 172.21.15.33:0/3044830786 learned_addr learned my addr 172.21.15.33:0/3044830786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625ad9d640 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c38c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 -- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 msgr2=0x56052c393bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c393bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 -- 172.21.15.33:0/3044830786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 msgr2=0x56052c38c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c38c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 -- 172.21.15.33:0/3044830786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052bffa3f0 con 0x56052c382c30 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f625a59c640 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c38cb80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f624c0ccfb0 tx=0x7f624c09c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df1de080c88f8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.271+0000 7f623eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f623eff5640 1 -- 172.21.15.33:0/3044830786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c018b70 con 0x56052c382c30 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254003e50 con 0x56052c382c30 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f623eff5640 1 -- 172.21.15.33:0/3044830786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f624c099fc0 con 0x56052c382c30 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f623eff5640 1 -- 172.21.15.33:0/3044830786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f624c018b70 con 0x56052c382c30 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 10 client.16773 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56052bfba0f0 con 0x56052c382c30 2024-04-24T23:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f623eff5640 1 -- 172.21.15.33:0/3044830786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6254003e50 con 0x56052c382c30 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f623eff5640 1 -- 172.21.15.33:0/3044830786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f624c097a10 con 0x56052c382c30 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 10 client.16773 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 20 client.16773 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c398510 0x56052c39a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 4 client.16773 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.272+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052c2f3bf0 con 0x56052c398510 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.273+0000 7f625ad9d640 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c398510 0x56052c39a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.273+0000 7f625ad9d640 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c398510 0x56052c39a9a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ed2f006b9f3bf3 server_cookie=42643e76fcf23e19 in_seq=0 out_seq=0 2024-04-24T23:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.273+0000 7f623eff5640 10 client.16773 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f623eff5640 1 -- 172.21.15.33:0/3044830786 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56052c2f3bf0 con 0x56052c398510 2024-04-24T23:18:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f623eff5640 10 client.16773 handle_command_reply: tid=0 2024-04-24T23:18:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 client.16773 shutdown 2024-04-24T23:18:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 msgr2=0x56052c38cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c38cb80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f624c0ccfb0 tx=0x7f624c09c500 comp rx=0 tx=0).stop 2024-04-24T23:18:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 shutdown_connections 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c398510 0x56052c39a9a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f623eff5640 0 client.16773 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c393bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c38cb80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 --2- 172.21.15.33:0/3044830786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c38c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 >> 172.21.15.33:0/3044830786 conn(0x56052c37d8b0 msgr2=0x56052c37caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 shutdown_connections 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 1 -- 172.21.15.33:0/3044830786 wait complete. 2024-04-24T23:18:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.274+0000 7f6263e78740 20 client.16773 trim_cache size 0 max 16384 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 Processor -- start 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 -- start start 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c380860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c382c30 0x56052c393b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 0x56052c380da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1aab80 con 0x56052c387da0 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1a8450 con 0x56052c382240 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f6263e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1a62a0 con 0x56052c382c30 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f625b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 0x56052c380da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f625b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 0x56052c380da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42118/0 (socket says 172.21.15.33:42118) 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f625b59e640 1 -- 172.21.15.33:0/3904276254 learned_addr learned my addr 172.21.15.33:0/3904276254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f625ad9d640 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c380860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.292+0000 7f625a59c640 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c382c30 0x56052c393b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625b59e640 1 -- 172.21.15.33:0/3904276254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c382c30 msgr2=0x56052c393b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625b59e640 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c382c30 0x56052c393b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625b59e640 1 -- 172.21.15.33:0/3904276254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 msgr2=0x56052c380860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625b59e640 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c380860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625b59e640 1 -- 172.21.15.33:0/3904276254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052c383010 con 0x56052c387da0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625a59c640 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c382c30 0x56052c393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f625b59e640 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 0x56052c380da0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6250009620 tx=0x7f6250041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd35c16273b43f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f6240ff9640 1 -- 172.21.15.33:0/3904276254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625004e070 con 0x56052c387da0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f6240ff9640 1 -- 172.21.15.33:0/3904276254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6250005510 con 0x56052c387da0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.293+0000 7f6240ff9640 1 -- 172.21.15.33:0/3904276254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250035050 con 0x56052c387da0 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- 172.21.15.33:0/3904276254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 msgr2=0x56052c380da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 0x56052c380da0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6250009620 tx=0x7f6250041040 comp rx=0 tx=0).stop 2024-04-24T23:18:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- 172.21.15.33:0/3904276254 shutdown_connections 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c387da0 0x56052c380da0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c382c30 0x56052c393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- 172.21.15.33:0/3904276254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c380860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- 172.21.15.33:0/3904276254 >> 172.21.15.33:0/3904276254 conn(0x56052c37d8b0 msgr2=0x56052c384020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- 172.21.15.33:0/3904276254 shutdown_connections 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- 172.21.15.33:0/3904276254 wait complete. 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- start start 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 Processor -- start 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c391830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c391d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c392d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1a6910 con 0x56052c382c30 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1a7d80 con 0x56052c382240 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1a6fe0 con 0x56052c387da0 2024-04-24T23:18:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f6263e78740 10 client.0 _ll_register_callbacks cb 0x7f624c09ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f625a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c391d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f625b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c392d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f625ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c391830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f625a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c391d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42120/0 (socket says 172.21.15.33:42120) 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.294+0000 7f625a59c640 1 -- 172.21.15.33:0/1615771501 learned_addr learned my addr 172.21.15.33:0/1615771501 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625ad9d640 1 -- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 msgr2=0x56052c392d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625ad9d640 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c392d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625ad9d640 1 -- 172.21.15.33:0/1615771501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 msgr2=0x56052c391d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625ad9d640 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c391d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625ad9d640 1 -- 172.21.15.33:0/1615771501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052c393370 con 0x56052c382240 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625a59c640 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c391d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f625ad9d640 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c391830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f625400f790 tx=0x7f62540096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ab56e27cffba0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f623eff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:18:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f623eff5640 1 -- 172.21.15.33:0/1615771501 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254036030 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f623eff5640 1 -- 172.21.15.33:0/1615771501 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6254023040 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6250005510 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f623eff5640 1 -- 172.21.15.33:0/1615771501 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6254040050 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f6263e78740 10 client.26306 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56052c3ff000 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.295+0000 7f623eff5640 1 -- 172.21.15.33:0/1615771501 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6250005510 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f623eff5640 1 -- 172.21.15.33:0/1615771501 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f62540323a0 con 0x56052c382240 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 10 client.26306 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 10 client.26306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 20 client.26306 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c3ff830 0x56052c401c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 4 client.26306 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56052c2edf40 con 0x56052c3ff830 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f625a59c640 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c3ff830 0x56052c401c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f625a59c640 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c3ff830 0x56052c401c50 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70eda88dc1f51f21 server_cookie=1b9bb463a851aee1 in_seq=0 out_seq=0 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f623eff5640 10 client.26306 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f623eff5640 1 -- 172.21.15.33:0/1615771501 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56052c2edf40 con 0x56052c3ff830 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f623eff5640 10 client.26306 handle_command_reply: tid=0 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 client.26306 shutdown 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 msgr2=0x56052c391830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c391830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f625400f790 tx=0x7f62540096c0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 shutdown_connections 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.296+0000 7f6263e78740 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56052c3ff830 0x56052c401c50 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052c387da0 0x56052c392d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052c382c30 0x56052c391d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 1 --2- 172.21.15.33:0/1615771501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56052c382240 0x56052c391830 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 >> 172.21.15.33:0/1615771501 conn(0x56052c37d8b0 msgr2=0x56052c37c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f623eff5640 0 client.26306 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 shutdown_connections 2024-04-24T23:18:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 1 -- 172.21.15.33:0/1615771501 wait complete. 2024-04-24T23:18:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6263e78740 20 client.26306 trim_cache size 0 max 16384 2024-04-24T23:18:37.306 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:18:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:18:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:18:37.306 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f62440a2b90 msgr2=0x7f62440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f62440a2b90 0x7f62440a5050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f625002a930 tx=0x7f6250032f30 comp rx=0 tx=0).stop 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 msgr2=0x7f625c197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 0x7f625c197fa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e9f0 tx=0x7f624c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 shutdown_connections 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f62440a2b90 0x7f62440a5050 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625c103350 0x7f625c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f625c102990 0x7f625c197fa0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 --2- 172.21.15.33:0/1932701882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f625c1017a0 0x7f625c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 >> 172.21.15.33:0/1932701882 conn(0x7f625c0fcd10 msgr2=0x7f625c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 shutdown_connections 2024-04-24T23:18:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.297+0000 7f6261163640 1 -- 172.21.15.33:0/1932701882 wait complete. 2024-04-24T23:18:37.310 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:18:37.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:18:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 -- 172.21.15.33:0/474310703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa090104bb0 msgr2=0x7fa090105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 --2- 172.21.15.33:0/474310703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa090104bb0 0x7fa090105030 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa080009900 tx=0x7fa08002a410 comp rx=0 tx=0).stop 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 -- 172.21.15.33:0/474310703 shutdown_connections 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 --2- 172.21.15.33:0/474310703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa090105570 0x7fa09010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 --2- 172.21.15.33:0/474310703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa090104bb0 0x7fa090105030 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 --2- 172.21.15.33:0/474310703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0901039c0 0x7fa090103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 -- 172.21.15.33:0/474310703 >> 172.21.15.33:0/474310703 conn(0x7fa0900fef30 msgr2=0x7fa090101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 -- 172.21.15.33:0/474310703 shutdown_connections 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.412+0000 7fa098509640 1 -- 172.21.15.33:0/474310703 wait complete. 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 Processor -- start 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 -- start start 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 0x7fa090199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa090104bb0 0x7fa09019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa090105570 0x7fa0901a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa09010f640 con 0x7fa0901039c0 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa09010f4c0 con 0x7fa090105570 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa098509640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa09010f7c0 con 0x7fa090104bb0 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 0x7fa090199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 0x7fa090199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42146/0 (socket says 172.21.15.33:42146) 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 -- 172.21.15.33:0/2118191546 learned_addr learned my addr 172.21.15.33:0/2118191546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa096aa6640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa090105570 0x7fa0901a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa095aa4640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa090104bb0 0x7fa09019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 -- 172.21.15.33:0/2118191546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa090104bb0 msgr2=0x7fa09019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa090104bb0 0x7fa09019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 -- 172.21.15.33:0/2118191546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa090105570 msgr2=0x7fa0901a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa090105570 0x7fa0901a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa0962a5640 1 -- 172.21.15.33:0/2118191546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0901a1910 con 0x7fa0901039c0 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.413+0000 7fa096aa6640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa090105570 0x7fa0901a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.414+0000 7fa0962a5640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 0x7fa090199d00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00e9c0 tx=0x7fa08c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.414+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00cd80 con 0x7fa0901039c0 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.414+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0901a1c00 con 0x7fa0901039c0 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.414+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa08c0035f0 con 0x7fa0901039c0 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.414+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c010690 con 0x7fa0901039c0 2024-04-24T23:18:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.414+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0901a2150 con 0x7fa0901039c0 2024-04-24T23:18:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.415+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09010ad60 con 0x7fa0901039c0 2024-04-24T23:18:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.416+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa08c0037a0 con 0x7fa0901039c0 2024-04-24T23:18:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.417+0000 7fa0877fe640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa07c0a2b90 0x7fa07c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.417+0000 7fa095aa4640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa07c0a2b90 0x7fa07c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.417+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa08c0cf830 con 0x7fa0901039c0 2024-04-24T23:18:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.417+0000 7fa095aa4640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa07c0a2b90 0x7fa07c0a5050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa0800314e0 tx=0x7fa080000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.422+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa08c09c400 con 0x7fa0901039c0 2024-04-24T23:18:37.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 680085 12.80 MB/sec execute 162 sec latency 112.426 ms 2024-04-24T23:18:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.704+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa09002d170 con 0x7fa0901039c0 2024-04-24T23:18:37.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.705+0000 7fa0877fe640 1 -- 172.21.15.33:0/2118191546 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fa08c09bd00 con 0x7fa0901039c0 2024-04-24T23:18:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:18:37.708 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:18:37.708 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:18:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.710+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa07c0a2b90 msgr2=0x7fa07c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.710+0000 7fa098509640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa07c0a2b90 0x7fa07c0a5050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa0800314e0 tx=0x7fa080000bc0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.710+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 msgr2=0x7fa090199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.710+0000 7fa098509640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 0x7fa090199d00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00e9c0 tx=0x7fa08c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:18:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 shutdown_connections 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa07c0a2b90 0x7fa07c0a5050 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa090105570 0x7fa0901a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa090104bb0 0x7fa09019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 --2- 172.21.15.33:0/2118191546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0901039c0 0x7fa090199d00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 >> 172.21.15.33:0/2118191546 conn(0x7fa0900fef30 msgr2=0x7fa090107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 shutdown_connections 2024-04-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.711+0000 7fa098509640 1 -- 172.21.15.33:0/2118191546 wait complete. 2024-04-24T23:18:37.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:18:37.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 -- 172.21.15.33:0/678509507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 msgr2=0x7fa710109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 --2- 172.21.15.33:0/678509507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 0x7fa710109be0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa708009ca0 tx=0x7fa70802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 -- 172.21.15.33:0/678509507 shutdown_connections 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 --2- 172.21.15.33:0/678509507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 0x7fa710109be0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 --2- 172.21.15.33:0/678509507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa710102990 0x7fa710102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 --2- 172.21.15.33:0/678509507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7101017a0 0x7fa710101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 -- 172.21.15.33:0/678509507 >> 172.21.15.33:0/678509507 conn(0x7fa7100fcd10 msgr2=0x7fa7100ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 -- 172.21.15.33:0/678509507 shutdown_connections 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.853+0000 7fa714872640 1 -- 172.21.15.33:0/678509507 wait complete. 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 Processor -- start 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 -- start start 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 0x7fa710197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa710102990 0x7fa710197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 0x7fa71019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa71010d420 con 0x7fa710103350 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa71010d2a0 con 0x7fa710102990 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa714872640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa71010d5a0 con 0x7fa7101017a0 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa707fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa710102990 0x7fa710197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 0x7fa710197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 0x7fa71019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa707fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa710102990 0x7fa710197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53398/0 (socket says 172.21.15.33:53398) 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 0x7fa710197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54668/0 (socket says 172.21.15.33:54668) 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.854+0000 7fa707fff640 1 -- 172.21.15.33:0/2299211198 learned_addr learned my addr 172.21.15.33:0/2299211198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa70e59c640 1 -- 172.21.15.33:0/2299211198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa710102990 msgr2=0x7fa710197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa70e59c640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa710102990 0x7fa710197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa70e59c640 1 -- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 msgr2=0x7fa71019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa70e59c640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 0x7fa71019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa70e59c640 1 -- 172.21.15.33:0/2299211198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71019f660 con 0x7fa7101017a0 2024-04-24T23:18:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa70e59c640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 0x7fa710197a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa70000e9c0 tx=0x7fa70000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa7077fe640 1 -- 172.21.15.33:0/2299211198 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70000ccc0 con 0x7fa7101017a0 2024-04-24T23:18:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71019f950 con 0x7fa7101017a0 2024-04-24T23:18:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa7077fe640 1 -- 172.21.15.33:0/2299211198 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7000035f0 con 0x7fa7101017a0 2024-04-24T23:18:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa7077fe640 1 -- 172.21.15.33:0/2299211198 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70000ce60 con 0x7fa7101017a0 2024-04-24T23:18:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.855+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71019fea0 con 0x7fa7101017a0 2024-04-24T23:18:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.857+0000 7fa7077fe640 1 -- 172.21.15.33:0/2299211198 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa7000037a0 con 0x7fa7101017a0 2024-04-24T23:18:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 Processor -- start 2024-04-24T23:18:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 -- start start 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 0x55c7c08cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c7c0833670 con 0x55c7c08d4d90 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c7c07db970 con 0x55c7c08cf230 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa71757e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c7c082f630 con 0x55c7c08cfc20 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa7077fe640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6fc0a2b90 0x7fa6fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 0x55c7c08cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 0x55c7c08cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53402/0 (socket says 172.21.15.33:53402) 2024-04-24T23:18:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa70ed9d640 1 -- 172.21.15.33:0/518940914 learned_addr learned my addr 172.21.15.33:0/518940914 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.858+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6fc0a2b90 0x7fa6fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 -- 172.21.15.33:0/518940914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 msgr2=0x55c7c08d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa7077fe640 1 -- 172.21.15.33:0/2299211198 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa7000cf7c0 con 0x7fa7101017a0 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 -- 172.21.15.33:0/518940914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 msgr2=0x55c7c08d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 -- 172.21.15.33:0/518940914 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c07ff400 con 0x55c7c08cf230 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70e59c640 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa707fff640 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 0x55c7c08cf650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa708035000 tx=0x7fa708031d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc93699091ce210 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6fc0a2b90 0x7fa6fc0a5050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa708048000 tx=0x7fa70804a000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa704ff9640 1 -- 172.21.15.33:0/518940914 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70804f070 con 0x55c7c08cf230 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa704ff9640 1 -- 172.21.15.33:0/518940914 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708036040 con 0x55c7c08cf230 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.859+0000 7fa704ff9640 1 -- 172.21.15.33:0/518940914 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70802f610 con 0x55c7c08cf230 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 -- 172.21.15.33:0/518940914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 msgr2=0x55c7c08cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 0x55c7c08cf650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa708035000 tx=0x7fa708031d30 comp rx=0 tx=0).stop 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 -- 172.21.15.33:0/518940914 shutdown_connections 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 --2- 172.21.15.33:0/518940914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cf230 0x55c7c08cf650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 -- 172.21.15.33:0/518940914 >> 172.21.15.33:0/518940914 conn(0x55c7c08ca8a0 msgr2=0x55c7c08caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 -- 172.21.15.33:0/518940914 shutdown_connections 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.860+0000 7fa71757e740 1 -- 172.21.15.33:0/518940914 wait complete. 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 -- start start 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 Processor -- start 2024-04-24T23:18:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cfc20 0x55c7c08d97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c7c08d7b60 con 0x55c7c08d4d90 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c7c07ef050 con 0x55c7c08cfc20 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c7c08d7ce0 con 0x55c7c08cf230 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa71757e740 10 client.0 _ll_register_callbacks cb 0x55c7c0546be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa707fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cfc20 0x55c7c08d97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08e0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42198/0 (socket says 172.21.15.33:42198) 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08d9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54692/0 (socket says 172.21.15.33:54692) 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 -- 172.21.15.33:0/4176169623 learned_addr learned my addr 172.21.15.33:0/4176169623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa707fff640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cfc20 0x55c7c08d97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53414/0 (socket says 172.21.15.33:53414) 2024-04-24T23:18:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 -- 172.21.15.33:0/4176169623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 msgr2=0x55c7c08d9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08d9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 -- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cfc20 msgr2=0x55c7c08d97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cfc20 0x55c7c08d97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70ed9d640 1 -- 172.21.15.33:0/4176169623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c07c0dc0 con 0x55c7c08d4d90 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.861+0000 7fa70e59c640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08d9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08e0780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa70802ade0 tx=0x7fa7080098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b87cb59125a44a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/4176169623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708056020 con 0x55c7c08d4d90 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/4176169623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa70802f610 con 0x55c7c08d4d90 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa708036040 con 0x55c7c08d4d90 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/4176169623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa708061050 con 0x55c7c08d4d90 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 10 client.16797 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c7c05473f0 con 0x55c7c08d4d90 2024-04-24T23:18:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/4176169623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708036040 con 0x55c7c08d4d90 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/4176169623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa7080306a0 con 0x55c7c08d4d90 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 10 client.16797 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 20 client.16797 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.862+0000 7fa71757e740 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c08e4d80 0x55c7c08e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.863+0000 7fa71757e740 4 client.16797 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.863+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c0840bb0 con 0x55c7c08e4d80 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.863+0000 7fa70e59c640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c08e4d80 0x55c7c08e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.863+0000 7fa6f5ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.863+0000 7fa70e59c640 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c08e4d80 0x55c7c08e71e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f3f4943cacb5823 server_cookie=dbccf0b4e86da545 in_seq=0 out_seq=0 2024-04-24T23:18:37.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/4176169623 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7c0840bb0 con 0x55c7c08e4d80 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa6f5ffb640 10 client.16797 handle_command_reply: tid=0 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 client.16797 shutdown 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 msgr2=0x55c7c08e0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08e0780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa70802ade0 tx=0x7fa7080098a0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 shutdown_connections 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c08e4d80 0x55c7c08e71e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08d4d90 0x55c7c08e0780 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08cfc20 0x55c7c08d97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 --2- 172.21.15.33:0/4176169623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08d9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa6f5ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:37.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 >> 172.21.15.33:0/4176169623 conn(0x55c7c08ca8a0 msgr2=0x55c7c08c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 shutdown_connections 2024-04-24T23:18:37.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 1 -- 172.21.15.33:0/4176169623 wait complete. 2024-04-24T23:18:37.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.864+0000 7fa71757e740 20 client.16797 trim_cache size 0 max 16384 2024-04-24T23:18:37.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 Processor -- start 2024-04-24T23:18:37.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 -- start start 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 0x55c7c08ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101accd0 con 0x55c7c08cf230 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101aa5a0 con 0x55c7c08d4d90 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.878+0000 7fa71757e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101a83f0 con 0x55c7c08cfc20 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 0x55c7c08ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 0x55c7c08ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42200/0 (socket says 172.21.15.33:42200) 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 -- 172.21.15.33:0/464644384 learned_addr learned my addr 172.21.15.33:0/464644384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa707fff640 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 -- 172.21.15.33:0/464644384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 msgr2=0x55c7c08e0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08e0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 -- 172.21.15.33:0/464644384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 msgr2=0x55c7c08d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 -- 172.21.15.33:0/464644384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c06117d0 con 0x55c7c08cf230 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa70e59c640 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 0x55c7c08ce040 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa70009bbc0 tx=0x7fa70009ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d39c15800a351a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa6f7fff640 1 -- 172.21.15.33:0/464644384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70009ffe0 con 0x55c7c08cf230 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa6f7fff640 1 -- 172.21.15.33:0/464644384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7000a48a0 con 0x55c7c08cf230 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.879+0000 7fa6f7fff640 1 -- 172.21.15.33:0/464644384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000cfbb0 con 0x55c7c08cf230 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 -- 172.21.15.33:0/464644384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 msgr2=0x55c7c08ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 0x55c7c08ce040 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa70009bbc0 tx=0x7fa70009ba00 comp rx=0 tx=0).stop 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 -- 172.21.15.33:0/464644384 shutdown_connections 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cfc20 0x55c7c08e0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- 172.21.15.33:0/464644384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cf230 0x55c7c08ce040 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 -- 172.21.15.33:0/464644384 >> 172.21.15.33:0/464644384 conn(0x55c7c08ca8a0 msgr2=0x55c7c08c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 -- 172.21.15.33:0/464644384 shutdown_connections 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 -- 172.21.15.33:0/464644384 wait complete. 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 -- start start 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 Processor -- start 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08ddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 0x55c7c08de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.880+0000 7fa71757e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08df380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa71757e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101a8a60 con 0x55c7c08cfc20 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa71757e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101a9ed0 con 0x55c7c08d4d90 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa71757e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101a9130 con 0x55c7c08cf230 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa71757e740 10 client.0 _ll_register_callbacks cb 0x7fa70804d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 0x55c7c08de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 0x55c7c08de4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42204/0 (socket says 172.21.15.33:42204) 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 -- 172.21.15.33:0/524968291 learned_addr learned my addr 172.21.15.33:0/524968291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa70ed9d640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08df380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa70e59c640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08ddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 -- 172.21.15.33:0/524968291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 msgr2=0x55c7c08ddf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08ddf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 -- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 msgr2=0x55c7c08df380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08df380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 -- 172.21.15.33:0/524968291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c08df970 con 0x55c7c08cfc20 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa707fff640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 0x55c7c08de4a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009640 tx=0x7fa6f8012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbdcf3dc5ee2439 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/524968291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8022020 con 0x55c7c08cfc20 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/524968291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa6f800fa90 con 0x55c7c08cfc20 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7000a48a0 con 0x55c7c08cfc20 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.881+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/524968291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6f802a050 con 0x55c7c08cfc20 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 10 client.16803 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c7c094b3f0 con 0x55c7c08cfc20 2024-04-24T23:18:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/524968291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7000a48a0 con 0x55c7c08cfc20 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/524968291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa6f80048c0 con 0x55c7c08cfc20 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 10 client.16803 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 20 client.16803 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c094be70 0x55c7c094e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 4 client.16803 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7c083af00 con 0x55c7c094be70 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa70e59c640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c094be70 0x55c7c094e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa70e59c640 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c094be70 0x55c7c094e290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1a474bd414a1ecf server_cookie=7d8022035873687d in_seq=0 out_seq=0 2024-04-24T23:18:37.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.882+0000 7fa6f5ffb640 10 client.16803 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa6f5ffb640 1 -- 172.21.15.33:0/524968291 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7c083af00 con 0x55c7c094be70 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa6f5ffb640 10 client.16803 handle_command_reply: tid=0 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 client.16803 shutdown 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 msgr2=0x55c7c08de4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 0x55c7c08de4a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009640 tx=0x7fa6f8012d70 comp rx=0 tx=0).stop 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 shutdown_connections 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55c7c094be70 0x55c7c094e290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c7c08d4d90 0x55c7c08df380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7c08cfc20 0x55c7c08de4a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 --2- 172.21.15.33:0/524968291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7c08cf230 0x55c7c08ddf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa6f5ffb640 0 client.16803 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 >> 172.21.15.33:0/524968291 conn(0x55c7c08ca8a0 msgr2=0x55c7c08c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 shutdown_connections 2024-04-24T23:18:37.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 1 -- 172.21.15.33:0/524968291 wait complete. 2024-04-24T23:18:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa71757e740 20 client.16803 trim_cache size 0 max 16384 2024-04-24T23:18:37.892 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6fc0a2b90 msgr2=0x7fa6fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6fc0a2b90 0x7fa6fc0a5050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa708048000 tx=0x7fa70804a000 comp rx=0 tx=0).stop 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 msgr2=0x7fa710197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 0x7fa710197a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa70000e9c0 tx=0x7fa70000ee80 comp rx=0 tx=0).stop 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 shutdown_connections 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6fc0a2b90 0x7fa6fc0a5050 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa710103350 0x7fa71019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa710102990 0x7fa710197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 --2- 172.21.15.33:0/2299211198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7101017a0 0x7fa710197a40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.883+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 >> 172.21.15.33:0/2299211198 conn(0x7fa7100fcd10 msgr2=0x7fa710105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.884+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 shutdown_connections 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:37.884+0000 7fa714872640 1 -- 172.21.15.33:0/2299211198 wait complete. 2024-04-24T23:18:37.896 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:37 smithi033 ceph-mon[29201]: pgmap v549: 129 pgs: 129 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-04-24T23:18:37.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:18:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:37 smithi038 ceph-mon[32450]: pgmap v549: 129 pgs: 129 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-04-24T23:18:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:37 smithi117 ceph-mon[32529]: pgmap v549: 129 pgs: 129 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 -- 172.21.15.33:0/3219567860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620102230 msgr2=0x7f0620108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 --2- 172.21.15.33:0/3219567860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620102230 0x7f0620108ac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0610009900 tx=0x7f061002a410 comp rx=0 tx=0).stop 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 -- 172.21.15.33:0/3219567860 shutdown_connections 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 --2- 172.21.15.33:0/3219567860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620102230 0x7f0620108ac0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 --2- 172.21.15.33:0/3219567860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0620101870 0x7f0620101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 --2- 172.21.15.33:0/3219567860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 0x7f0620100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 -- 172.21.15.33:0/3219567860 >> 172.21.15.33:0/3219567860 conn(0x7f06200fbbf0 msgr2=0x7f06200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 -- 172.21.15.33:0/3219567860 shutdown_connections 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 -- 172.21.15.33:0/3219567860 wait complete. 2024-04-24T23:18:38.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 Processor -- start 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.004+0000 7f06256ba640 1 -- start start 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f06256ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 0x7f0620104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f06256ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 0x7f0620102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f06256ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0620102230 0x7f0620103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f06256ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f062010c300 con 0x7f0620101870 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f06256ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f062010c180 con 0x7f0620100680 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f06256ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f062010c480 con 0x7f0620102230 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 0x7f0620102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 0x7f0620104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 0x7f0620102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42212/0 (socket says 172.21.15.33:42212) 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061e7fc640 1 -- 172.21.15.33:0/347145764 learned_addr learned my addr 172.21.15.33:0/347145764 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061f7fe640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0620102230 0x7f0620103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 -- 172.21.15.33:0/347145764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0620102230 msgr2=0x7f0620103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0620102230 0x7f0620103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 -- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 msgr2=0x7f0620102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 0x7f0620102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 -- 172.21.15.33:0/347145764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620103a20 con 0x7f0620100680 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061e7fc640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 0x7f0620102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.005+0000 7f061effd640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 0x7f0620104610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0618005fd0 tx=0x7f061800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.006+0000 7f0607fff640 1 -- 172.21.15.33:0/347145764 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618003970 con 0x7f0620100680 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.006+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06201a1bb0 con 0x7f0620100680 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.006+0000 7f0607fff640 1 -- 172.21.15.33:0/347145764 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0618004030 con 0x7f0620100680 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.006+0000 7f0607fff640 1 -- 172.21.15.33:0/347145764 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180107f0 con 0x7f0620100680 2024-04-24T23:18:38.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.006+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06201a2150 con 0x7f0620100680 2024-04-24T23:18:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f0607fff640 1 -- 172.21.15.33:0/347145764 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f06180041e0 con 0x7f0620100680 2024-04-24T23:18:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 Processor -- start 2024-04-24T23:18:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 -- start start 2024-04-24T23:18:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860964650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x563860969720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638608c8670 con 0x563860964230 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563860870970 con 0x563860969d90 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.008+0000 7f06283c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5638608c4630 con 0x563860964c20 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f0607fff640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06080a2b90 0x7f06080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860964650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x563860969720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860964650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42218/0 (socket says 172.21.15.33:42218) 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061f7fe640 1 -- 172.21.15.33:0/1087306874 learned_addr learned my addr 172.21.15.33:0/1087306874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061f7fe640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06080a2b90 0x7f06080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f0607fff640 1 -- 172.21.15.33:0/347145764 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06180cfcd0 con 0x7f0620100680 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 -- 172.21.15.33:0/1087306874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 msgr2=0x56386096c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 -- 172.21.15.33:0/1087306874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 msgr2=0x563860964650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860964650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 -- 172.21.15.33:0/1087306874 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563860894400 con 0x563860964c20 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061e7fc640 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061effd640 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x563860969720 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0618098b10 tx=0x7f0618004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e56ea58ef0b7c5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f06057fa640 1 -- 172.21.15.33:0/1087306874 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061809c3a0 con 0x563860964c20 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f061f7fe640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06080a2b90 0x7f06080a5050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x563860965480 tx=0x7f061002a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f06057fa640 1 -- 172.21.15.33:0/1087306874 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061809c140 con 0x563860964c20 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.009+0000 7f06057fa640 1 -- 172.21.15.33:0/1087306874 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180bc0f0 con 0x563860964c20 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 -- 172.21.15.33:0/1087306874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 msgr2=0x563860969720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x563860969720 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0618098b10 tx=0x7f0618004930 comp rx=0 tx=0).stop 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 -- 172.21.15.33:0/1087306874 shutdown_connections 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x563860969720 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 --2- 172.21.15.33:0/1087306874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860964650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 -- 172.21.15.33:0/1087306874 >> 172.21.15.33:0/1087306874 conn(0x56386095f8a0 msgr2=0x56386095fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 -- 172.21.15.33:0/1087306874 shutdown_connections 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 -- 172.21.15.33:0/1087306874 wait complete. 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 -- start start 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.010+0000 7f06283c7740 1 Processor -- start 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x56386096e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x56386096e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860969d90 0x563860975910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56386096cd40 con 0x563860964230 2024-04-24T23:18:38.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563860884050 con 0x563860964c20 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56386096cec0 con 0x563860969d90 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f06283c7740 10 client.0 _ll_register_callbacks cb 0x5638605dbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x56386096e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x56386096e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x56386096e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53484/0 (socket says 172.21.15.33:53484) 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061e7fc640 1 -- 172.21.15.33:0/2309871611 learned_addr learned my addr 172.21.15.33:0/2309871611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061f7fe640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860969d90 0x563860975910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 -- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860969d90 msgr2=0x563860975910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860969d90 0x563860975910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 -- 172.21.15.33:0/2309871611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 msgr2=0x56386096e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x56386096e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 -- 172.21.15.33:0/2309871611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563860855dc0 con 0x563860964230 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061e7fc640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x56386096e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f061effd640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x56386096e390 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f061801b040 tx=0x7f061809c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df2fb149eb8c3db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f0602ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.011+0000 7f0602ff5640 1 -- 172.21.15.33:0/2309871611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180b4020 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f0602ff5640 1 -- 172.21.15.33:0/2309871611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06180bc0f0 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638608d5bb0 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f0602ff5640 1 -- 172.21.15.33:0/2309871611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06180cd460 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 10 client.16815 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638605dc3f0 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f0602ff5640 1 -- 172.21.15.33:0/2309871611 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5638608d5bb0 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f0602ff5640 1 -- 172.21.15.33:0/2309871611 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f061809b590 con 0x563860964230 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 10 client.16815 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 20 client.16815 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x563860979e00 0x56386097c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 4 client.16815 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638608cff00 con 0x563860979e00 2024-04-24T23:18:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.012+0000 7f061e7fc640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x563860979e00 0x56386097c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.013+0000 7f061e7fc640 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x563860979e00 0x56386097c1f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78972cc2379c3baa server_cookie=11f7f24312a4a473 in_seq=0 out_seq=0 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.013+0000 7f0602ff5640 10 client.16815 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.013+0000 7f0602ff5640 1 -- 172.21.15.33:0/2309871611 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638608cff00 con 0x563860979e00 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.013+0000 7f0602ff5640 10 client.16815 handle_command_reply: tid=0 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.013+0000 7f06283c7740 1 client.16815 shutdown 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 msgr2=0x56386096e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x56386096e390 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f061801b040 tx=0x7f061809c840 comp rx=0 tx=0).stop 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 shutdown_connections 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x563860979e00 0x56386097c1f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860969d90 0x563860975910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x56386096e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 --2- 172.21.15.33:0/2309871611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x56386096e390 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f0602ff5640 0 client.16815 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 >> 172.21.15.33:0/2309871611 conn(0x56386095f8a0 msgr2=0x56386095eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 shutdown_connections 2024-04-24T23:18:38.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 1 -- 172.21.15.33:0/2309871611 wait complete. 2024-04-24T23:18:38.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.014+0000 7f06283c7740 20 client.16815 trim_cache size 0 max 16384 2024-04-24T23:18:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.028+0000 7f06283c7740 1 Processor -- start 2024-04-24T23:18:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 -- start start 2024-04-24T23:18:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860969650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x5638609758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06201aeee0 con 0x563860964230 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06201ac7b0 con 0x563860969d90 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f06283c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06201aa600 con 0x563860964c20 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860969650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860969650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42226/0 (socket says 172.21.15.33:42226) 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x5638609758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 -- 172.21.15.33:0/698326300 learned_addr learned my addr 172.21.15.33:0/698326300 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061f7fe640 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 -- 172.21.15.33:0/698326300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 msgr2=0x5638609758d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x5638609758d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 -- 172.21.15.33:0/698326300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 msgr2=0x56386096e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 -- 172.21.15.33:0/698326300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638606a67d0 con 0x563860964230 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061e7fc640 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x5638609758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.029+0000 7f061effd640 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860969650 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f061809a3f0 tx=0x7f06180b5220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963514a194e900cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f0604ff9640 1 -- 172.21.15.33:0/698326300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180b4070 con 0x563860964230 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f0604ff9640 1 -- 172.21.15.33:0/698326300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061809b590 con 0x563860964230 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f0604ff9640 1 -- 172.21.15.33:0/698326300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061809c3a0 con 0x563860964230 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 -- 172.21.15.33:0/698326300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 msgr2=0x563860969650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860969650 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f061809a3f0 tx=0x7f06180b5220 comp rx=0 tx=0).stop 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 -- 172.21.15.33:0/698326300 shutdown_connections 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860969d90 0x56386096e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964c20 0x5638609758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 --2- 172.21.15.33:0/698326300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860964230 0x563860969650 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 -- 172.21.15.33:0/698326300 >> 172.21.15.33:0/698326300 conn(0x56386095f8a0 msgr2=0x56386095eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 -- 172.21.15.33:0/698326300 shutdown_connections 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.030+0000 7f06283c7740 1 -- 172.21.15.33:0/698326300 wait complete. 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 -- start start 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 Processor -- start 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964230 0x563860975780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x563860972220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 0x563860972760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06201aac70 con 0x563860969d90 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06201ac0e0 con 0x563860964c20 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06201ab340 con 0x563860964230 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f06283c7740 10 client.0 _ll_register_callbacks cb 0x7f0614005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 0x563860972760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 0x563860972760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42234/0 (socket says 172.21.15.33:42234) 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 -- 172.21.15.33:0/328908844 learned_addr learned my addr 172.21.15.33:0/328908844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061e7fc640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x563860972220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061effd640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964230 0x563860975780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 -- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964230 msgr2=0x563860975780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964230 0x563860975780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 -- 172.21.15.33:0/328908844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 msgr2=0x563860972220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x563860972220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061f7fe640 1 -- 172.21.15.33:0/328908844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638609de170 con 0x563860969d90 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.031+0000 7f061e7fc640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x563860972220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f061f7fe640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 0x563860972760 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0610006010 tx=0x7f0610009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b695142112b6d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f0602ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f0602ff5640 1 -- 172.21.15.33:0/328908844 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610051070 con 0x563860969d90 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f0602ff5640 1 -- 172.21.15.33:0/328908844 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0610003bc0 con 0x563860969d90 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061809b590 con 0x563860969d90 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f0602ff5640 1 -- 172.21.15.33:0/328908844 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f061005e050 con 0x563860969d90 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 10 client.16821 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638609df3f0 con 0x563860969d90 2024-04-24T23:18:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f0602ff5640 1 -- 172.21.15.33:0/328908844 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061809b590 con 0x563860969d90 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f0602ff5640 1 -- 172.21.15.33:0/328908844 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f06100049a0 con 0x563860969d90 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 10 client.16821 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 20 client.16821 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5638609dfdd0 0x5638609e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 4 client.16821 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.032+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56386084d820 con 0x5638609dfdd0 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f061effd640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5638609dfdd0 0x5638609e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f0602ff5640 10 client.16821 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f061effd640 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5638609dfdd0 0x5638609e21f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75ce4aa31dd4bd33 server_cookie=5dc5f3a766e88bb4 in_seq=0 out_seq=0 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f0602ff5640 1 -- 172.21.15.33:0/328908844 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56386084d820 con 0x5638609dfdd0 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f0602ff5640 10 client.16821 handle_command_reply: tid=0 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 client.16821 shutdown 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 msgr2=0x563860972760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 0x563860972760 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0610006010 tx=0x7f0610009260 comp rx=0 tx=0).stop 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 shutdown_connections 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5638609dfdd0 0x5638609e21f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563860969d90 0x563860972760 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563860964c20 0x563860972220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 --2- 172.21.15.33:0/328908844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563860964230 0x563860975780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 >> 172.21.15.33:0/328908844 conn(0x56386095f8a0 msgr2=0x56386095eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f0602ff5640 0 client.16821 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.033+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 shutdown_connections 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06283c7740 1 -- 172.21.15.33:0/328908844 wait complete. 2024-04-24T23:18:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06283c7740 20 client.16821 trim_cache size 0 max 16384 2024-04-24T23:18:38.045 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:18:38.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06080a2b90 msgr2=0x7f06080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06080a2b90 0x7f06080a5050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x563860965480 tx=0x7f061002a410 comp rx=0 tx=0).stop 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 msgr2=0x7f0620104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 0x7f0620104610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0618005fd0 tx=0x7f061800ef90 comp rx=0 tx=0).stop 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 shutdown_connections 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06080a2b90 0x7f06080a5050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0620102230 0x7f0620103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0620101870 0x7f0620102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 --2- 172.21.15.33:0/347145764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0620100680 0x7f0620104610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 >> 172.21.15.33:0/347145764 conn(0x7f06200fbbf0 msgr2=0x7f06200fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 shutdown_connections 2024-04-24T23:18:38.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.034+0000 7f06256ba640 1 -- 172.21.15.33:0/347145764 wait complete. 2024-04-24T23:18:38.048 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:18:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2931492498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 msgr2=0x7f20b8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2931492498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8101d10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009900 tx=0x7f20ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:18:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2931492498 shutdown_connections 2024-04-24T23:18:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2931492498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b8102250 0x7f20b8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2931492498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8101d10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2931492498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 0x7f20b8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2931492498 >> 172.21.15.33:0/2931492498 conn(0x7f20b80fbc10 msgr2=0x7f20b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2931492498 shutdown_connections 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2931492498 wait complete. 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 Processor -- start 2024-04-24T23:18:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.154+0000 7f20bdcf5640 1 -- start start 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20bdcf5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 0x7f20b8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20bdcf5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20bdcf5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b8102250 0x7f20b819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20bdcf5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810c300 con 0x7f20b8101890 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20bdcf5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810c180 con 0x7f20b8102250 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20bdcf5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810c480 con 0x7f20b81006a0 2024-04-24T23:18:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42254/0 (socket says 172.21.15.33:42254) 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 -- 172.21.15.33:0/2425653587 learned_addr learned my addr 172.21.15.33:0/2425653587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b7fff640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b8102250 0x7f20b819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b77fe640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 0x7f20b8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 -- 172.21.15.33:0/2425653587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 msgr2=0x7f20b8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 0x7f20b8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 -- 172.21.15.33:0/2425653587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b8102250 msgr2=0x7f20b819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b8102250 0x7f20b819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 -- 172.21.15.33:0/2425653587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b819f720 con 0x7f20b8101890 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b77fe640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 0x7f20b8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8198040 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009230 tx=0x7f20ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.155+0000 7f20b4ff9640 1 -- 172.21.15.33:0/2425653587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac03f030 con 0x7f20b8101890 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.156+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b819f9b0 con 0x7f20b8101890 2024-04-24T23:18:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.156+0000 7f20b4ff9640 1 -- 172.21.15.33:0/2425653587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ac003ce0 con 0x7f20b8101890 2024-04-24T23:18:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.156+0000 7f20b4ff9640 1 -- 172.21.15.33:0/2425653587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac035070 con 0x7f20b8101890 2024-04-24T23:18:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.156+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b819fef0 con 0x7f20b8101890 2024-04-24T23:18:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.158+0000 7f20b4ff9640 1 -- 172.21.15.33:0/2425653587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f20ac042030 con 0x7f20b8101890 2024-04-24T23:18:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 Processor -- start 2024-04-24T23:18:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 -- start start 2024-04-24T23:18:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae588a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae589030 0x5627ae58db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae58e1a0 0x5627ae590630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627ae4e8670 con 0x5627ae589030 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5627ae490970 con 0x5627ae588640 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20c0a01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5627ae4e4630 con 0x5627ae58e1a0 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae588a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae589030 0x5627ae58db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae588a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53530/0 (socket says 172.21.15.33:53530) 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b7fff640 1 -- 172.21.15.33:0/176691007 learned_addr learned my addr 172.21.15.33:0/176691007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae58e1a0 0x5627ae590630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b4ff9640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20a00a2b90 0x7f20a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b77fe640 1 -- 172.21.15.33:0/176691007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae58e1a0 msgr2=0x5627ae590630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b77fe640 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae58e1a0 0x5627ae590630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b77fe640 1 -- 172.21.15.33:0/176691007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 msgr2=0x5627ae588a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b77fe640 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae588a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b77fe640 1 -- 172.21.15.33:0/176691007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ae4b4400 con 0x5627ae589030 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b7fff640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20a00a2b90 0x7f20a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.159+0000 7f20b4ff9640 1 -- 172.21.15.33:0/2425653587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20ac0ef970 con 0x7f20b8101890 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.160+0000 7f20b77fe640 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae589030 0x5627ae58db30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009c00 tx=0x7f20a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ddc4c3e5cdf04b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.160+0000 7f209dffb640 1 -- 172.21.15.33:0/176691007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8012450 con 0x5627ae589030 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.160+0000 7f209dffb640 1 -- 172.21.15.33:0/176691007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a8012ad0 con 0x5627ae589030 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.160+0000 7f209dffb640 1 -- 172.21.15.33:0/176691007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a800fbb0 con 0x5627ae589030 2024-04-24T23:18:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- 172.21.15.33:0/176691007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae589030 msgr2=0x5627ae58db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20b7fff640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20a00a2b90 0x7f20a00a5050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009fd0 tx=0x7f20b0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae589030 0x5627ae58db30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009c00 tx=0x7f20a80117b0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- 172.21.15.33:0/176691007 shutdown_connections 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae58e1a0 0x5627ae590630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae589030 0x5627ae58db30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- 172.21.15.33:0/176691007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae588a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- 172.21.15.33:0/176691007 >> 172.21.15.33:0/176691007 conn(0x5627ae57f8a0 msgr2=0x5627ae57fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- 172.21.15.33:0/176691007 shutdown_connections 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- 172.21.15.33:0/176691007 wait complete. 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- start start 2024-04-24T23:18:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 Processor -- start 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae592790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae592cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae599d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627ae591150 con 0x5627ae58e1a0 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5627ae4a4050 con 0x5627ae589030 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.161+0000 7f20c0a01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5627ae5912d0 con 0x5627ae588640 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20c0a01740 10 client.0 _ll_register_callbacks cb 0x5627ae1fbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae592cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae599d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae592790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae592cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53536/0 (socket says 172.21.15.33:53536) 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b6ffd640 1 -- 172.21.15.33:0/3456224138 learned_addr learned my addr 172.21.15.33:0/3456224138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae592790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54802/0 (socket says 172.21.15.33:54802) 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 -- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 msgr2=0x5627ae592790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae592790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 -- 172.21.15.33:0/3456224138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 msgr2=0x5627ae592cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae592cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 -- 172.21.15.33:0/3456224138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ae475dc0 con 0x5627ae58e1a0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae592cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b77fe640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae592790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20b7fff640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae599d10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f20b0020950 tx=0x7f20b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152de1d919d5ab1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f2096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f2096ffd640 1 -- 172.21.15.33:0/3456224138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0010dc0 con 0x5627ae58e1a0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f2096ffd640 1 -- 172.21.15.33:0/3456224138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20b001b440 con 0x5627ae58e1a0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a8012ad0 con 0x5627ae58e1a0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f2096ffd640 1 -- 172.21.15.33:0/3456224138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20b0020e10 con 0x5627ae58e1a0 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20c0a01740 10 client.16833 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.162+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627ae1fc3f0 con 0x5627ae58e1a0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f2096ffd640 1 -- 172.21.15.33:0/3456224138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a8012ad0 con 0x5627ae58e1a0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f2096ffd640 1 -- 172.21.15.33:0/3456224138 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20b0027020 con 0x5627ae58e1a0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20c0a01740 10 client.16833 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20c0a01740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20c0a01740 20 client.16833 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae59e6d0 0x5627ae5a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20c0a01740 4 client.16833 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627ae4f5bb0 con 0x5627ae59e6d0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20b77fe640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae59e6d0 0x5627ae5a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f20b77fe640 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae59e6d0 0x5627ae5a0b70 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327615b3833dd772 server_cookie=bce3545a6de3809a in_seq=0 out_seq=0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.163+0000 7f2096ffd640 10 client.16833 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f2096ffd640 1 -- 172.21.15.33:0/3456224138 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627ae4f5bb0 con 0x5627ae59e6d0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f2096ffd640 10 client.16833 handle_command_reply: tid=0 2024-04-24T23:18:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 client.16833 shutdown 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 msgr2=0x5627ae599d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae599d10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f20b0020950 tx=0x7f20b0004ab0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 shutdown_connections 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae59e6d0 0x5627ae5a0b70 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae599d10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f2096ffd640 0 client.16833 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae592cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3456224138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae592790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 >> 172.21.15.33:0/3456224138 conn(0x5627ae57f8a0 msgr2=0x5627ae57eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 shutdown_connections 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 1 -- 172.21.15.33:0/3456224138 wait complete. 2024-04-24T23:18:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.164+0000 7f20c0a01740 20 client.16833 trim_cache size 0 max 16384 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 Processor -- start 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 -- start start 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae58d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 0x5627ae599cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae5827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20b81aab00 con 0x5627ae58e1a0 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b81a83d0 con 0x5627ae588640 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20c0a01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20b81a6220 con 0x5627ae589030 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae5827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 0x5627ae599cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae5827b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42286/0 (socket says 172.21.15.33:42286) 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 0x5627ae599cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54808/0 (socket says 172.21.15.33:54808) 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b7fff640 1 -- 172.21.15.33:0/177175409 learned_addr learned my addr 172.21.15.33:0/177175409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b77fe640 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae58d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 -- 172.21.15.33:0/177175409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 msgr2=0x5627ae58d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae58d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 -- 172.21.15.33:0/177175409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 msgr2=0x5627ae5827b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae5827b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b6ffd640 1 -- 172.21.15.33:0/177175409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ae2c67d0 con 0x5627ae589030 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.178+0000 7f20b7fff640 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae5827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.179+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 0x5627ae599cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0b8810 tx=0x7f20ac003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702badf8c6a87048 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.179+0000 7f209d7fa640 1 -- 172.21.15.33:0/177175409 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac049070 con 0x5627ae589030 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.179+0000 7f209d7fa640 1 -- 172.21.15.33:0/177175409 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ac0bc540 con 0x5627ae589030 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.179+0000 7f209d7fa640 1 -- 172.21.15.33:0/177175409 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac03b460 con 0x5627ae589030 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.179+0000 7f20c0a01740 1 -- 172.21.15.33:0/177175409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 msgr2=0x5627ae599cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.179+0000 7f20c0a01740 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 0x5627ae599cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0b8810 tx=0x7f20ac003f50 comp rx=0 tx=0).stop 2024-04-24T23:18:38.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- 172.21.15.33:0/177175409 shutdown_connections 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae5827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae589030 0x5627ae599cd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 --2- 172.21.15.33:0/177175409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae588640 0x5627ae58d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- 172.21.15.33:0/177175409 >> 172.21.15.33:0/177175409 conn(0x5627ae57f8a0 msgr2=0x5627ae57eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- 172.21.15.33:0/177175409 shutdown_connections 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- 172.21.15.33:0/177175409 wait complete. 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- start start 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 Processor -- start 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae597640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae597b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae598b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20b81a6890 con 0x5627ae58e1a0 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b81a7d00 con 0x5627ae589030 2024-04-24T23:18:38.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20b81a6f60 con 0x5627ae588640 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20c0a01740 10 client.0 _ll_register_callbacks cb 0x7f20b0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae598b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae597640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae598b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42298/0 (socket says 172.21.15.33:42298) 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20b7fff640 1 -- 172.21.15.33:0/3202480400 learned_addr learned my addr 172.21.15.33:0/3202480400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.180+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae597b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b7fff640 1 -- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 msgr2=0x5627ae597640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b7fff640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae597640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b7fff640 1 -- 172.21.15.33:0/3202480400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 msgr2=0x5627ae597b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b7fff640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae597b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b7fff640 1 -- 172.21.15.33:0/3202480400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ae599180 con 0x5627ae58e1a0 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b77fe640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae597640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b6ffd640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae597b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20b7fff640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae598b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009e40 tx=0x7f20b001dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b95b3605410fbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f2096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f2096ffd640 1 -- 172.21.15.33:0/3202480400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b003a070 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20ac0bc540 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f2096ffd640 1 -- 172.21.15.33:0/3202480400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20b0004b20 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f2096ffd640 1 -- 172.21.15.33:0/3202480400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20b0045050 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20c0a01740 10 client.16839 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627ae604e90 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.181+0000 7f2096ffd640 1 -- 172.21.15.33:0/3202480400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ac0bc540 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f2096ffd640 1 -- 172.21.15.33:0/3202480400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20b0027040 con 0x5627ae58e1a0 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 10 client.16839 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:38.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 20 client.16839 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae6058c0 0x5627ae607ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 4 client.16839 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627ae4eff00 con 0x5627ae6058c0 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20b77fe640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae6058c0 0x5627ae607ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20b77fe640 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae6058c0 0x5627ae607ce0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9b8ac7dafc308a server_cookie=a7a90d7178c1b07a in_seq=0 out_seq=0 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f2096ffd640 10 client.16839 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f2096ffd640 1 -- 172.21.15.33:0/3202480400 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627ae4eff00 con 0x5627ae6058c0 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f2096ffd640 10 client.16839 handle_command_reply: tid=0 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.182+0000 7f20c0a01740 1 client.16839 shutdown 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 msgr2=0x5627ae598b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae598b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009e40 tx=0x7f20b001dcc0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 shutdown_connections 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5627ae6058c0 0x5627ae607ce0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627ae58e1a0 0x5627ae598b90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5627ae589030 0x5627ae597b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 --2- 172.21.15.33:0/3202480400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627ae588640 0x5627ae597640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 >> 172.21.15.33:0/3202480400 conn(0x5627ae57f8a0 msgr2=0x5627ae589ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f2096ffd640 0 client.16839 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 shutdown_connections 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 1 -- 172.21.15.33:0/3202480400 wait complete. 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20c0a01740 20 client.16839 trim_cache size 0 max 16384 2024-04-24T23:18:38.192 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20a00a2b90 msgr2=0x7f20a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20a00a2b90 0x7f20a00a5050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009fd0 tx=0x7f20b0004a40 comp rx=0 tx=0).stop 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 msgr2=0x7f20b8198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8198040 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009230 tx=0x7f20ac0042f0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 shutdown_connections 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20a00a2b90 0x7f20a00a5050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b8102250 0x7f20b819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b8101890 0x7f20b8198040 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 --2- 172.21.15.33:0/2425653587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20b81006a0 0x7f20b8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.183+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 >> 172.21.15.33:0/2425653587 conn(0x7f20b80fbc10 msgr2=0x7f20b80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.184+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 shutdown_connections 2024-04-24T23:18:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:38.184+0000 7f20bdcf5640 1 -- 172.21.15.33:0/2425653587 wait complete. 2024-04-24T23:18:38.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 682320 12.80 MB/sec execute 163 sec latency 60.376 ms 2024-04-24T23:18:38.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2118191546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:38.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2118191546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:38.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2118191546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:39.197 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:18:39.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:39.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:18:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 -- 172.21.15.33:0/3121268814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 msgr2=0x7f603c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 --2- 172.21.15.33:0/3121268814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c101cf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6028009900 tx=0x7f602802a410 comp rx=0 tx=0).stop 2024-04-24T23:18:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 -- 172.21.15.33:0/3121268814 shutdown_connections 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 --2- 172.21.15.33:0/3121268814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603c102230 0x7f603c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 --2- 172.21.15.33:0/3121268814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c101cf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 --2- 172.21.15.33:0/3121268814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f603c100680 0x7f603c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 -- 172.21.15.33:0/3121268814 >> 172.21.15.33:0/3121268814 conn(0x7f603c0fbbf0 msgr2=0x7f603c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 -- 172.21.15.33:0/3121268814 shutdown_connections 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.304+0000 7f6042f63640 1 -- 172.21.15.33:0/3121268814 wait complete. 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 Processor -- start 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 -- start start 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f603c100680 0x7f603c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603c102230 0x7f603c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10d420 con 0x7f603c101870 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10d2a0 con 0x7f603c100680 2024-04-24T23:18:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6042f63640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10d5a0 con 0x7f603c102230 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42316/0 (socket says 172.21.15.33:42316) 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 -- 172.21.15.33:0/3897546805 learned_addr learned my addr 172.21.15.33:0/3897546805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6040cff640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f603c100680 0x7f603c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f6041500640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603c102230 0x7f603c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 -- 172.21.15.33:0/3897546805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603c102230 msgr2=0x7f603c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603c102230 0x7f603c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 -- 172.21.15.33:0/3897546805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f603c100680 msgr2=0x7f603c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f603c100680 0x7f603c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.305+0000 7f603bfff640 1 -- 172.21.15.33:0/3897546805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c19f700 con 0x7f603c101870 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.306+0000 7f603bfff640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c198020 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6028009230 tx=0x7f6028031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.306+0000 7f6039ffb640 1 -- 172.21.15.33:0/3897546805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602803f030 con 0x7f603c101870 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.306+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c19f990 con 0x7f603c101870 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.306+0000 7f6039ffb640 1 -- 172.21.15.33:0/3897546805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6028003ad0 con 0x7f603c101870 2024-04-24T23:18:39.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.306+0000 7f6039ffb640 1 -- 172.21.15.33:0/3897546805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028035070 con 0x7f603c101870 2024-04-24T23:18:39.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.306+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c19fed0 con 0x7f603c101870 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6039ffb640 1 -- 172.21.15.33:0/3897546805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6028042030 con 0x7f603c101870 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 Processor -- start 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 -- start start 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a20e6c2c0 con 0x559a20f09780 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a20e14910 con 0x559a20f0e8f0 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6045c7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a20e68280 con 0x559a20f08d90 2024-04-24T23:18:39.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6041500640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6040cff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f603bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6041500640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54860/0 (socket says 172.21.15.33:54860) 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.309+0000 7f6041500640 1 -- 172.21.15.33:0/1925240654 learned_addr learned my addr 172.21.15.33:0/1925240654 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6039ffb640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f602c0a2b90 0x7f602c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 -- 172.21.15.33:0/1925240654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 msgr2=0x559a20f10d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f10d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 -- 172.21.15.33:0/1925240654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 msgr2=0x559a20f0e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f0e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 -- 172.21.15.33:0/1925240654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a20e25fb0 con 0x559a20f08d90 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6040cff640 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f603bfff640 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f10d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6039ffb640 1 -- 172.21.15.33:0/3897546805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f60280ef940 con 0x7f603c101870 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f602c0a2b90 0x7f602c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f091b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6034010510 tx=0x7f6034009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ec44aa5b72ea41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6026ffd640 1 -- 172.21.15.33:0/1925240654 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603401d400 con 0x559a20f08d90 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6026ffd640 1 -- 172.21.15.33:0/1925240654 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60340134c0 con 0x559a20f08d90 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6026ffd640 1 -- 172.21.15.33:0/1925240654 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340118a0 con 0x559a20f08d90 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.310+0000 7f6041500640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f602c0a2b90 0x7f602c0a5050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6034021000 tx=0x7f603401e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 -- 172.21.15.33:0/1925240654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 msgr2=0x559a20f091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f091b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6034010510 tx=0x7f6034009f90 comp rx=0 tx=0).stop 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 -- 172.21.15.33:0/1925240654 shutdown_connections 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f10d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 --2- 172.21.15.33:0/1925240654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f091b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 -- 172.21.15.33:0/1925240654 >> 172.21.15.33:0/1925240654 conn(0x559a20ed0950 msgr2=0x559a20f04810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 -- 172.21.15.33:0/1925240654 shutdown_connections 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 -- 172.21.15.33:0/1925240654 wait complete. 2024-04-24T23:18:39.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 -- start start 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.311+0000 7f6045c7b740 1 Processor -- start 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f08d90 0x559a20f13270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 0x559a20f1a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a20f118a0 con 0x559a20f0e8f0 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a20e27e90 con 0x559a20f08d90 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a20f11a20 con 0x559a20f09780 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6045c7b740 10 client.0 _ll_register_callbacks cb 0x559a20e63970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6041500640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 0x559a20f1a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6041500640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 0x559a20f1a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42336/0 (socket says 172.21.15.33:42336) 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6041500640 1 -- 172.21.15.33:0/766625778 learned_addr learned my addr 172.21.15.33:0/766625778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6040cff640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f08d90 0x559a20f13270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 -- 172.21.15.33:0/766625778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f08d90 msgr2=0x559a20f13270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f08d90 0x559a20f13270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 -- 172.21.15.33:0/766625778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 msgr2=0x559a20f1a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 0x559a20f1a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 -- 172.21.15.33:0/766625778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a20c487e0 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f6041500640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 0x559a20f1a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f603bfff640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f137b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60280b82c0 tx=0x7f6028009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6800047a140431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f601bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f601bfff640 1 -- 172.21.15.33:0/766625778 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028049070 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.312+0000 7f601bfff640 1 -- 172.21.15.33:0/766625778 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f602803b460 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60340134c0 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f601bfff640 1 -- 172.21.15.33:0/766625778 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60280bbe60 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 10 client.26365 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a20ca8f70 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f601bfff640 1 -- 172.21.15.33:0/766625778 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60340134c0 con 0x559a20f09780 2024-04-24T23:18:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f601bfff640 1 -- 172.21.15.33:0/766625778 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f60280b7bb0 con 0x559a20f09780 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 10 client.26365 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 20 client.26365 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f1ee20 0x559a20f21240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 4 client.26365 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a20e405a0 con 0x559a20f1ee20 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.313+0000 7f6040cff640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f1ee20 0x559a20f21240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.314+0000 7f6040cff640 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f1ee20 0x559a20f21240 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5464101d9b7b30 server_cookie=61053ac838990193 in_seq=0 out_seq=0 2024-04-24T23:18:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.314+0000 7f601bfff640 10 client.26365 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.314+0000 7f601bfff640 1 -- 172.21.15.33:0/766625778 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a20e405a0 con 0x559a20f1ee20 2024-04-24T23:18:39.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.314+0000 7f601bfff640 10 client.26365 handle_command_reply: tid=0 2024-04-24T23:18:39.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.314+0000 7f6045c7b740 1 client.26365 shutdown 2024-04-24T23:18:39.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.314+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 msgr2=0x559a20f137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f137b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60280b82c0 tx=0x7f6028009260 comp rx=0 tx=0).stop 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 shutdown_connections 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f1ee20 0x559a20f21240 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f0e8f0 0x559a20f1a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f137b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 --2- 172.21.15.33:0/766625778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f08d90 0x559a20f13270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f601bfff640 0 client.26365 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 >> 172.21.15.33:0/766625778 conn(0x559a20ed0950 msgr2=0x559a20f03950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 shutdown_connections 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 1 -- 172.21.15.33:0/766625778 wait complete. 2024-04-24T23:18:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.315+0000 7f6045c7b740 20 client.26365 trim_cache size 0 max 16384 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.334+0000 7f6045c7b740 1 Processor -- start 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.334+0000 7f6045c7b740 1 -- start start 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.334+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.334+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f1a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.334+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f6045c7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1accf0 con 0x559a20f09780 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f6045c7b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1aa5c0 con 0x559a20f0e8f0 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f6045c7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1a8410 con 0x559a20f08d90 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f1a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f6040cff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f6041500640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f1a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42340/0 (socket says 172.21.15.33:42340) 2024-04-24T23:18:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 -- 172.21.15.33:0/29152787 learned_addr learned my addr 172.21.15.33:0/29152787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 -- 172.21.15.33:0/29152787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 msgr2=0x559a20f030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 -- 172.21.15.33:0/29152787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 msgr2=0x559a20f03600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f03600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 -- 172.21.15.33:0/29152787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a20b40170 con 0x559a20f09780 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f603bfff640 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f1a7b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60280314e0 tx=0x7f60280bbcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3af8c4ea58e75e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f60267fc640 1 -- 172.21.15.33:0/29152787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028049070 con 0x559a20f09780 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f60267fc640 1 -- 172.21.15.33:0/29152787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60280d9070 con 0x559a20f09780 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.335+0000 7f60267fc640 1 -- 172.21.15.33:0/29152787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280b84d0 con 0x559a20f09780 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 -- 172.21.15.33:0/29152787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 msgr2=0x559a20f1a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f1a7b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60280314e0 tx=0x7f60280bbcc0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 -- 172.21.15.33:0/29152787 shutdown_connections 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f03600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f09780 0x559a20f1a7b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 --2- 172.21.15.33:0/29152787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f08d90 0x559a20f030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 -- 172.21.15.33:0/29152787 >> 172.21.15.33:0/29152787 conn(0x559a20ed0950 msgr2=0x559a20f0abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 -- 172.21.15.33:0/29152787 shutdown_connections 2024-04-24T23:18:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 -- 172.21.15.33:0/29152787 wait complete. 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.336+0000 7f6045c7b740 1 -- start start 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 Processor -- start 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 0x559a20f17a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f17fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1a8a80 con 0x559a20f08d90 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1a9ef0 con 0x559a20f0e8f0 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1a9150 con 0x559a20f09780 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6045c7b740 10 client.0 _ll_register_callbacks cb 0x7f60280f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 0x559a20f17a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 0x559a20f17a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42342/0 (socket says 172.21.15.33:42342) 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 -- 172.21.15.33:0/2706018734 learned_addr learned my addr 172.21.15.33:0/2706018734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6041500640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f603bfff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f17fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 -- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 msgr2=0x559a20f17fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f17fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 -- 172.21.15.33:0/2706018734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 msgr2=0x559a20f18f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f18f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6040cff640 1 -- 172.21.15.33:0/2706018734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a20f19560 con 0x559a20f08d90 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f6041500640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.337+0000 7f603bfff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f17fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6040cff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 0x559a20f17a70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6030000c00 tx=0x7f6030005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab9b3f4c0e6dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f601bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f601bfff640 1 -- 172.21.15.33:0/2706018734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030005020 con 0x559a20f08d90 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f601bfff640 1 -- 172.21.15.33:0/2706018734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f60300051c0 con 0x559a20f08d90 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60280d9070 con 0x559a20f08d90 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f601bfff640 1 -- 172.21.15.33:0/2706018734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60300053c0 con 0x559a20f08d90 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 10 client.16857 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a20f1a970 con 0x559a20f08d90 2024-04-24T23:18:39.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f601bfff640 1 -- 172.21.15.33:0/2706018734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60280d9070 con 0x559a20f08d90 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f601bfff640 1 -- 172.21.15.33:0/2706018734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f60300037b0 con 0x559a20f08d90 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 10 client.16857 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 20 client.16857 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f85a80 0x559a20f87ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 4 client.16857 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.338+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a20e79800 con 0x559a20f85a80 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.339+0000 7f603bfff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f85a80 0x559a20f87ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.339+0000 7f603bfff640 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f85a80 0x559a20f87ea0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77c1b7cebf31ee server_cookie=80ff88380c94580d in_seq=0 out_seq=0 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.339+0000 7f601bfff640 10 client.16857 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f601bfff640 1 -- 172.21.15.33:0/2706018734 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a20e79800 con 0x559a20f85a80 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f601bfff640 10 client.16857 handle_command_reply: tid=0 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 client.16857 shutdown 2024-04-24T23:18:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 msgr2=0x559a20f17a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 0x559a20f17a70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6030000c00 tx=0x7f6030005f10 comp rx=0 tx=0).stop 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 shutdown_connections 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a20f85a80 0x559a20f87ea0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a20f0e8f0 0x559a20f18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f601bfff640 0 client.16857 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a20f09780 0x559a20f17fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 --2- 172.21.15.33:0/2706018734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a20f08d90 0x559a20f17a70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.340+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 >> 172.21.15.33:0/2706018734 conn(0x559a20ed0950 msgr2=0x559a20f074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 shutdown_connections 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6045c7b740 1 -- 172.21.15.33:0/2706018734 wait complete. 2024-04-24T23:18:39.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6045c7b740 20 client.16857 trim_cache size 0 max 16384 2024-04-24T23:18:39.348 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:18:39.348 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:18:39.348 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "bd0037a5-0ccc-4805-ac41-93e2fafea676", 2024-04-24T23:18:39.348 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:18:39.348 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f602c0a2b90 msgr2=0x7f602c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f602c0a2b90 0x7f602c0a5050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6034021000 tx=0x7f603401e000 comp rx=0 tx=0).stop 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 msgr2=0x7f603c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c198020 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6028009230 tx=0x7f6028031b60 comp rx=0 tx=0).stop 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 shutdown_connections 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f602c0a2b90 0x7f602c0a5050 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603c102230 0x7f603c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101870 0x7f603c198020 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 --2- 172.21.15.33:0/3897546805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f603c100680 0x7f603c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 >> 172.21.15.33:0/3897546805 conn(0x7f603c0fbbf0 msgr2=0x7f603c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 shutdown_connections 2024-04-24T23:18:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.341+0000 7f6042f63640 1 -- 172.21.15.33:0/3897546805 wait complete. 2024-04-24T23:18:39.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:18:39.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:18:39.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 685346 12.83 MB/sec execute 164 sec latency 18.168 ms 2024-04-24T23:18:39.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 -- 172.21.15.33:0/3366289337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c103330 msgr2=0x7f332c109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 --2- 172.21.15.33:0/3366289337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c103330 0x7f332c109bc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3324009ca0 tx=0x7f332402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 -- 172.21.15.33:0/3366289337 shutdown_connections 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 --2- 172.21.15.33:0/3366289337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c103330 0x7f332c109bc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 --2- 172.21.15.33:0/3366289337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332c102970 0x7f332c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 --2- 172.21.15.33:0/3366289337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c101780 0x7f332c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 -- 172.21.15.33:0/3366289337 >> 172.21.15.33:0/3366289337 conn(0x7f332c0fccf0 msgr2=0x7f332c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 -- 172.21.15.33:0/3366289337 shutdown_connections 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.462+0000 7f333434c640 1 -- 172.21.15.33:0/3366289337 wait complete. 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 Processor -- start 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 -- start start 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 0x7f332c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c102970 0x7f332c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332c103330 0x7f332c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f332c10d3e0 con 0x7f332c101780 2024-04-24T23:18:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f332c10d260 con 0x7f332c102970 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f333434c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f332c10d560 con 0x7f332c103330 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 0x7f332c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33318e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c102970 0x7f332c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 0x7f332c0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42364/0 (socket says 172.21.15.33:42364) 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33328e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332c103330 0x7f332c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 -- 172.21.15.33:0/1166283582 learned_addr learned my addr 172.21.15.33:0/1166283582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 -- 172.21.15.33:0/1166283582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332c103330 msgr2=0x7f332c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332c103330 0x7f332c074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 -- 172.21.15.33:0/1166283582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c102970 msgr2=0x7f332c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c102970 0x7f332c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33320e8640 1 -- 172.21.15.33:0/1166283582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c074900 con 0x7f332c101780 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.463+0000 7f33318e7640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c102970 0x7f332c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.464+0000 7f33320e8640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 0x7f332c0754f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f332800e9c0 tx=0x7f332800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.464+0000 7f33237fe640 1 -- 172.21.15.33:0/1166283582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332800cd80 con 0x7f332c101780 2024-04-24T23:18:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.464+0000 7f33237fe640 1 -- 172.21.15.33:0/1166283582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33280035f0 con 0x7f332c101780 2024-04-24T23:18:39.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.464+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c1a1b50 con 0x7f332c101780 2024-04-24T23:18:39.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.464+0000 7f33237fe640 1 -- 172.21.15.33:0/1166283582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328010690 con 0x7f332c101780 2024-04-24T23:18:39.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.464+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c1a20a0 con 0x7f332c101780 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.466+0000 7f33237fe640 1 -- 172.21.15.33:0/1166283582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f33280037a0 con 0x7f332c101780 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 Processor -- start 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 -- start start 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09240 0x55a0d4a09660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0d496d6b0 con 0x55a0d4a0eda0 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4915880 con 0x55a0d4a09240 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f3337058740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4969670 con 0x55a0d4a09c30 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33237fe640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33180a2b90 0x7f33180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33320e8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a0e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33318e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33318e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a11230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42370/0 (socket says 172.21.15.33:42370) 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33318e7640 1 -- 172.21.15.33:0/921445784 learned_addr learned my addr 172.21.15.33:0/921445784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33328e9640 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09240 0x55a0d4a09660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33328e9640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33180a2b90 0x7f33180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.467+0000 7f33237fe640 1 -- 172.21.15.33:0/1166283582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33280cf870 con 0x7f332c101780 2024-04-24T23:18:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33320e8640 1 -- 172.21.15.33:0/921445784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09240 msgr2=0x55a0d4a09660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33320e8640 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09240 0x55a0d4a09660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33320e8640 1 -- 172.21.15.33:0/921445784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 msgr2=0x55a0d4a11230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33320e8640 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a11230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33320e8640 1 -- 172.21.15.33:0/921445784 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d474b7d0 con 0x55a0d4a09c30 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33318e7640 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a11230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33320e8640 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a0e730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33280986b0 tx=0x7f332806eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae9f3361f16a585 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f33328e9640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33180a2b90 0x7f33180a5050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3324004af0 tx=0x7f3324005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f3320bfe640 1 -- 172.21.15.33:0/921445784 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332809bce0 con 0x55a0d4a09c30 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f3320bfe640 1 -- 172.21.15.33:0/921445784 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33280a0090 con 0x55a0d4a09c30 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.468+0000 7f3320bfe640 1 -- 172.21.15.33:0/921445784 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328066a90 con 0x55a0d4a09c30 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- 172.21.15.33:0/921445784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 msgr2=0x55a0d4a0e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a0e730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33280986b0 tx=0x7f332806eda0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- 172.21.15.33:0/921445784 shutdown_connections 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a11230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a0e730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- 172.21.15.33:0/921445784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09240 0x55a0d4a09660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- 172.21.15.33:0/921445784 >> 172.21.15.33:0/921445784 conn(0x55a0d4a048b0 msgr2=0x55a0d4a04cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- 172.21.15.33:0/921445784 shutdown_connections 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- 172.21.15.33:0/921445784 wait complete. 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- start start 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 Processor -- start 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09c30 0x55a0d4a13c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4a11d50 con 0x55a0d4a09240 2024-04-24T23:18:39.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4928f50 con 0x55a0d4a09c30 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.469+0000 7f3337058740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4a11ed0 con 0x55a0d4a0eda0 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f3337058740 10 client.0 _ll_register_callbacks cb 0x55a0d4964000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33320e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09c30 0x55a0d4a13c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33320e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42378/0 (socket says 172.21.15.33:42378) 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33320e8640 1 -- 172.21.15.33:0/2658067279 learned_addr learned my addr 172.21.15.33:0/2658067279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33328e9640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 -- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a0eda0 msgr2=0x55a0d4a1ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a1ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 -- 172.21.15.33:0/2658067279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 msgr2=0x55a0d4a136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a136e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 -- 172.21.15.33:0/2658067279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d46813f0 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33320e8640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f33318e7640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09c30 0x55a0d4a13c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55a0d4a05820 tx=0x7f331c011bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96c5bfa8c6797f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f3315ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f3315ffb640 1 -- 172.21.15.33:0/2658067279 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c010c60 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f3315ffb640 1 -- 172.21.15.33:0/2658067279 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f331c010e00 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.470+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33280a0090 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3315ffb640 1 -- 172.21.15.33:0/2658067279 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f331c004020 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 10 client.26396 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0d46410f0 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3315ffb640 1 -- 172.21.15.33:0/2658067279 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33280a0090 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3315ffb640 1 -- 172.21.15.33:0/2658067279 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f331c005200 con 0x55a0d4a09c30 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 10 client.26396 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 20 client.26396 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a1f530 0x55a0d4a219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 4 client.26396 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d497abf0 con 0x55a0d4a1f530 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.471+0000 7f33320e8640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a1f530 0x55a0d4a219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f33320e8640 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a1f530 0x55a0d4a219b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a00809c26134de server_cookie=1c23b622e886eda2 in_seq=0 out_seq=0 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f3315ffb640 10 client.26396 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f3315ffb640 1 -- 172.21.15.33:0/2658067279 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0d497abf0 con 0x55a0d4a1f530 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f3315ffb640 10 client.26396 handle_command_reply: tid=0 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f3337058740 1 client.26396 shutdown 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09c30 msgr2=0x55a0d4a13c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.472+0000 7f3337058740 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09c30 0x55a0d4a13c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55a0d4a05820 tx=0x7f331c011bc0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 shutdown_connections 2024-04-24T23:18:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a1f530 0x55a0d4a219b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a1ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a09c30 0x55a0d4a13c20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3315ffb640 0 client.26396 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 --2- 172.21.15.33:0/2658067279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 >> 172.21.15.33:0/2658067279 conn(0x55a0d4a048b0 msgr2=0x55a0d4a03af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 shutdown_connections 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 1 -- 172.21.15.33:0/2658067279 wait complete. 2024-04-24T23:18:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.473+0000 7f3337058740 20 client.26396 trim_cache size 0 max 16384 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 Processor -- start 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 -- start start 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09240 0x55a0d4a129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 0x55a0d4a1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1aeeb0 con 0x55a0d4a09c30 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1ac780 con 0x55a0d4a0eda0 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f3337058740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1aa5d0 con 0x55a0d4a09240 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 0x55a0d4a1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 0x55a0d4a1ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42388/0 (socket says 172.21.15.33:42388) 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 -- 172.21.15.33:0/3120748443 learned_addr learned my addr 172.21.15.33:0/3120748443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33328e9640 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33320e8640 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09240 0x55a0d4a129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 -- 172.21.15.33:0/3120748443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09240 msgr2=0x55a0d4a129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09240 0x55a0d4a129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 -- 172.21.15.33:0/3120748443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 msgr2=0x55a0d4a07860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a07860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 -- 172.21.15.33:0/3120748443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d4a0a010 con 0x55a0d4a09c30 2024-04-24T23:18:39.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.492+0000 7f33318e7640 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 0x55a0d4a1ac20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f331c00c010 tx=0x7f331c010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7819cef4ec5ef542 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3317fff640 1 -- 172.21.15.33:0/3120748443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0041c0 con 0x55a0d4a09c30 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3317fff640 1 -- 172.21.15.33:0/3120748443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f331c02e070 con 0x55a0d4a09c30 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3317fff640 1 -- 172.21.15.33:0/3120748443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0296b0 con 0x55a0d4a09c30 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 -- 172.21.15.33:0/3120748443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 msgr2=0x55a0d4a1ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 0x55a0d4a1ac20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f331c00c010 tx=0x7f331c010e70 comp rx=0 tx=0).stop 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 -- 172.21.15.33:0/3120748443 shutdown_connections 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a07860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09c30 0x55a0d4a1ac20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 --2- 172.21.15.33:0/3120748443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09240 0x55a0d4a129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 -- 172.21.15.33:0/3120748443 >> 172.21.15.33:0/3120748443 conn(0x55a0d4a048b0 msgr2=0x55a0d4a0b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 -- 172.21.15.33:0/3120748443 shutdown_connections 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.493+0000 7f3337058740 1 -- 172.21.15.33:0/3120748443 wait complete. 2024-04-24T23:18:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 -- start start 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 Processor -- start 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a18740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a18c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1aac40 con 0x55a0d4a09240 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1ac0b0 con 0x55a0d4a0eda0 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1ab310 con 0x55a0d4a09c30 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f3337058740 10 client.0 _ll_register_callbacks cb 0x7f331c00bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a18740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a18740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42404/0 (socket says 172.21.15.33:42404) 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 -- 172.21.15.33:0/443691735 learned_addr learned my addr 172.21.15.33:0/443691735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33318e7640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a18c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33328e9640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 -- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 msgr2=0x55a0d4a18c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a18c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 -- 172.21.15.33:0/443691735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 msgr2=0x55a0d4a19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.494+0000 7f33320e8640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f33320e8640 1 -- 172.21.15.33:0/443691735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d4a1a280 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f33318e7640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a18c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f33320e8640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a18740 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f332809bc70 tx=0x7f332809b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2a9886f05d8a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3315ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3315ffb640 1 -- 172.21.15.33:0/443691735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280a0090 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3315ffb640 1 -- 172.21.15.33:0/443691735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f332809b130 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f331c02e070 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3315ffb640 1 -- 172.21.15.33:0/443691735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3328003ba0 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 10 client.16875 fetch_fsmap learned FSMap version 27 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0d4a85fd0 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3315ffb640 1 -- 172.21.15.33:0/443691735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f331c02e070 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3315ffb640 1 -- 172.21.15.33:0/443691735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f33280ba520 con 0x55a0d4a09240 2024-04-24T23:18:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 10 client.16875 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 20 client.16875 populate_metadata read hostname 'smithi033' 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.495+0000 7f3337058740 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a864e0 0x55a0d4a888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 4 client.16875 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0d4974f40 con 0x55a0d4a864e0 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f33318e7640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a864e0 0x55a0d4a888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f33318e7640 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a864e0 0x55a0d4a888d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcfcbaaff4a10ab server_cookie=7148e7bd0daa7014 in_seq=0 out_seq=0 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3315ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3315ffb640 1 -- 172.21.15.33:0/443691735 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a0d4974f40 con 0x55a0d4a864e0 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3315ffb640 10 client.16875 handle_command_reply: tid=0 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 client.16875 shutdown 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 msgr2=0x55a0d4a18740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a18740 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f332809bc70 tx=0x7f332809b740 comp rx=0 tx=0).stop 2024-04-24T23:18:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 shutdown_connections 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a0d4a864e0 0x55a0d4a888d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d4a0eda0 0x55a0d4a19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0d4a09c30 0x55a0d4a18c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3315ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 --2- 172.21.15.33:0/443691735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0d4a09240 0x55a0d4a18740 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.496+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 >> 172.21.15.33:0/443691735 conn(0x55a0d4a048b0 msgr2=0x55a0d4a03260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 shutdown_connections 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f3337058740 1 -- 172.21.15.33:0/443691735 wait complete. 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f3337058740 20 client.16875 trim_cache size 0 max 16384 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:18:39.507 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: "bd0037a5-0ccc-4805-ac41-93e2fafea676": { 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "bd0037a5-0ccc-4805-ac41-93e2fafea676", 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:18:39.508 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33180a2b90 msgr2=0x7f33180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33180a2b90 0x7f33180a5050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3324004af0 tx=0x7f3324005d80 comp rx=0 tx=0).stop 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 msgr2=0x7f332c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 0x7f332c0754f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f332800e9c0 tx=0x7f332800ee80 comp rx=0 tx=0).stop 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 shutdown_connections 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33180a2b90 0x7f33180a5050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332c103330 0x7f332c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f332c102970 0x7f332c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 --2- 172.21.15.33:0/1166283582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c101780 0x7f332c0754f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 >> 172.21.15.33:0/1166283582 conn(0x7f332c0fccf0 msgr2=0x7f332c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 shutdown_connections 2024-04-24T23:18:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:18:39.497+0000 7f333434c640 1 -- 172.21.15.33:0/1166283582 wait complete. 2024-04-24T23:18:39.510 INFO:tasks.cephfs.filesystem:scrub status for tag:bd0037a5-0ccc-4805-ac41-93e2fafea676 - {'path': '/', 'tag': 'bd0037a5-0ccc-4805-ac41-93e2fafea676', 'options': 'recursive,force'} 2024-04-24T23:18:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:39 smithi038 ceph-mon[32450]: pgmap v550: 129 pgs: 129 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 289 op/s 2024-04-24T23:18:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:39 smithi117 ceph-mon[32529]: pgmap v550: 129 pgs: 129 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 289 op/s 2024-04-24T23:18:39.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:39 smithi033 ceph-mon[29201]: pgmap v550: 129 pgs: 129 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 289 op/s 2024-04-24T23:18:40.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 687176 12.79 MB/sec execute 165 sec latency 13.244 ms 2024-04-24T23:18:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:40 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:18:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:18:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:18:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:40 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:18:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:18:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:18:40.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:40 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:18:40.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:18:40.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:18:40.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:40.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:18:41.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 689634 12.80 MB/sec execute 166 sec latency 17.081 ms 2024-04-24T23:18:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:41 smithi038 ceph-mon[32450]: pgmap v551: 129 pgs: 129 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-04-24T23:18:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:41 smithi117 ceph-mon[32529]: pgmap v551: 129 pgs: 129 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-04-24T23:18:41.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:41 smithi033 ceph-mon[29201]: pgmap v551: 129 pgs: 129 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-04-24T23:18:42.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 691730 12.79 MB/sec execute 167 sec latency 120.643 ms 2024-04-24T23:18:43.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 693470 12.77 MB/sec execute 168 sec latency 32.866 ms 2024-04-24T23:18:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:43 smithi038 ceph-mon[32450]: pgmap v552: 129 pgs: 129 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-04-24T23:18:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:43 smithi117 ceph-mon[32529]: pgmap v552: 129 pgs: 129 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-04-24T23:18:43.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:43 smithi033 ceph-mon[29201]: pgmap v552: 129 pgs: 129 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-04-24T23:18:44.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 696341 12.80 MB/sec execute 169 sec latency 8.968 ms 2024-04-24T23:18:45.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 698610 12.77 MB/sec execute 170 sec latency 16.341 ms 2024-04-24T23:18:45.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:45 smithi038 ceph-mon[32450]: pgmap v553: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-04-24T23:18:45.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:45 smithi117 ceph-mon[32529]: pgmap v553: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-04-24T23:18:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:45 smithi033 ceph-mon[29201]: pgmap v553: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-04-24T23:18:46.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 700556 12.77 MB/sec execute 171 sec latency 24.227 ms 2024-04-24T23:18:47.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 703422 12.80 MB/sec execute 172 sec latency 23.762 ms 2024-04-24T23:18:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:47 smithi038 ceph-mon[32450]: pgmap v554: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-04-24T23:18:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:47 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:18:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:47 smithi117 ceph-mon[32529]: pgmap v554: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-04-24T23:18:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:47 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:18:47.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:47 smithi033 ceph-mon[29201]: pgmap v554: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-04-24T23:18:47.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:47 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:18:48.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 705459 12.76 MB/sec execute 173 sec latency 53.516 ms 2024-04-24T23:18:48.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:48 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:18:48.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:48 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:18:48.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:48 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:18:49.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 707430 12.76 MB/sec execute 174 sec latency 23.279 ms 2024-04-24T23:18:49.827 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:49 smithi117 ceph-mon[32529]: pgmap v555: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-04-24T23:18:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:49 smithi038 ceph-mon[32450]: pgmap v555: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-04-24T23:18:49.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:49 smithi033 ceph-mon[29201]: pgmap v555: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-04-24T23:18:50.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 710358 12.79 MB/sec execute 175 sec latency 17.241 ms 2024-04-24T23:18:51.448 INFO:tasks.workunit.client.0.smithi033.stdout: 1 712624 12.77 MB/sec execute 176 sec latency 19.588 ms 2024-04-24T23:18:51.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:51 smithi038 ceph-mon[32450]: pgmap v556: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2024-04-24T23:18:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:51 smithi117 ceph-mon[32529]: pgmap v556: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2024-04-24T23:18:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:51 smithi033 ceph-mon[29201]: pgmap v556: 129 pgs: 129 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2024-04-24T23:18:52.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 714587 12.76 MB/sec execute 177 sec latency 18.785 ms 2024-04-24T23:18:53.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 717321 12.79 MB/sec execute 178 sec latency 100.351 ms 2024-04-24T23:18:53.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:53 smithi038 ceph-mon[32450]: pgmap v557: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-04-24T23:18:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:53 smithi117 ceph-mon[32529]: pgmap v557: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-04-24T23:18:53.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:53 smithi033 ceph-mon[29201]: pgmap v557: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-04-24T23:18:54.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 718843 12.75 MB/sec execute 179 sec latency 19.709 ms 2024-04-24T23:18:55.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 721468 12.76 MB/sec execute 180 sec latency 16.843 ms 2024-04-24T23:18:55.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:55 smithi038 ceph-mon[32450]: pgmap v558: 129 pgs: 129 active+clean; 264 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-04-24T23:18:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:55 smithi117 ceph-mon[32529]: pgmap v558: 129 pgs: 129 active+clean; 264 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-04-24T23:18:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:55 smithi033 ceph-mon[29201]: pgmap v558: 129 pgs: 129 active+clean; 264 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-04-24T23:18:56.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 724412 12.79 MB/sec execute 181 sec latency 22.012 ms 2024-04-24T23:18:57.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 726026 12.75 MB/sec execute 182 sec latency 14.000 ms 2024-04-24T23:18:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:57 smithi038 ceph-mon[32450]: pgmap v559: 129 pgs: 129 active+clean; 267 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-04-24T23:18:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:57 smithi117 ceph-mon[32529]: pgmap v559: 129 pgs: 129 active+clean; 267 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-04-24T23:18:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:57 smithi033 ceph-mon[29201]: pgmap v559: 129 pgs: 129 active+clean; 267 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-04-24T23:18:58.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 728418 12.76 MB/sec execute 183 sec latency 75.808 ms 2024-04-24T23:18:59.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 730861 12.76 MB/sec execute 184 sec latency 22.711 ms 2024-04-24T23:18:59.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:18:59 smithi038 ceph-mon[32450]: pgmap v560: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2024-04-24T23:18:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:18:59 smithi117 ceph-mon[32529]: pgmap v560: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2024-04-24T23:18:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:18:59 smithi033 ceph-mon[29201]: pgmap v560: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2024-04-24T23:19:00.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 732757 12.74 MB/sec execute 185 sec latency 23.669 ms 2024-04-24T23:19:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:00 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:00 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:00 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:01.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 735507 12.76 MB/sec execute 186 sec latency 29.503 ms 2024-04-24T23:19:01.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: pgmap v561: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 246 op/s 2024-04-24T23:19:01.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:01.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: pgmap v561: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 246 op/s 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:01.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:01.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:19:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: pgmap v561: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 246 op/s 2024-04-24T23:19:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:01.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:19:02.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 738396 12.77 MB/sec execute 187 sec latency 24.239 ms 2024-04-24T23:19:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:02 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-24T23:19:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:02 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-24T23:19:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:02 smithi038 ceph-mon[32450]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T23:19:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:02 smithi038 ceph-mon[32450]: Unable to set osd_memory_target on smithi038 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-24T23:19:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:02 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-24T23:19:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:02 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-24T23:19:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:02 smithi117 ceph-mon[32529]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T23:19:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:02 smithi117 ceph-mon[32529]: Unable to set osd_memory_target on smithi038 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-24T23:19:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:02 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-24T23:19:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:02 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-24T23:19:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:02 smithi033 ceph-mon[29201]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T23:19:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:02 smithi033 ceph-mon[29201]: Unable to set osd_memory_target on smithi038 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-24T23:19:03.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 739774 12.74 MB/sec execute 188 sec latency 41.945 ms 2024-04-24T23:19:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:03 smithi038 ceph-mon[32450]: pgmap v562: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-04-24T23:19:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:03 smithi117 ceph-mon[32529]: pgmap v562: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-04-24T23:19:03.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:03 smithi033 ceph-mon[29201]: pgmap v562: 129 pgs: 129 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-04-24T23:19:04.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 742442 12.76 MB/sec execute 189 sec latency 17.869 ms 2024-04-24T23:19:05.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 744842 12.75 MB/sec execute 190 sec latency 33.684 ms 2024-04-24T23:19:05.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:05 smithi038 ceph-mon[32450]: pgmap v563: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-04-24T23:19:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:05 smithi117 ceph-mon[32529]: pgmap v563: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-04-24T23:19:05.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:05 smithi033 ceph-mon[29201]: pgmap v563: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-04-24T23:19:06.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 746769 12.74 MB/sec execute 191 sec latency 12.921 ms 2024-04-24T23:19:07.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 749662 12.76 MB/sec execute 192 sec latency 9.765 ms 2024-04-24T23:19:07.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:07 smithi038 ceph-mon[32450]: pgmap v564: 129 pgs: 129 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-04-24T23:19:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:07 smithi117 ceph-mon[32529]: pgmap v564: 129 pgs: 129 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-04-24T23:19:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:07 smithi033 ceph-mon[29201]: pgmap v564: 129 pgs: 129 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-04-24T23:19:08.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 751667 12.73 MB/sec execute 193 sec latency 121.195 ms 2024-04-24T23:19:09.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 753491 12.73 MB/sec execute 194 sec latency 20.390 ms 2024-04-24T23:19:09.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:19:09.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 -- 172.21.15.33:0/1702989282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 msgr2=0x7f2630100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 --2- 172.21.15.33:0/1702989282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630100aa0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f262c009900 tx=0x7f262c02a410 comp rx=0 tx=0).stop 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 -- 172.21.15.33:0/1702989282 shutdown_connections 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 --2- 172.21.15.33:0/1702989282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630102230 0x7f2630108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 --2- 172.21.15.33:0/1702989282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630101870 0x7f2630101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 --2- 172.21.15.33:0/1702989282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630100aa0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 -- 172.21.15.33:0/1702989282 >> 172.21.15.33:0/1702989282 conn(0x7f26300fbbf0 msgr2=0x7f26300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 -- 172.21.15.33:0/1702989282 shutdown_connections 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.625+0000 7f263806d640 1 -- 172.21.15.33:0/1702989282 wait complete. 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 Processor -- start 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 -- start start 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 0x7f263019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630102230 0x7f263019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f263010c300 con 0x7f2630100680 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f263010c180 con 0x7f2630101870 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.626+0000 7f263806d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f263010c480 con 0x7f2630102230 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 0x7f263019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f263660a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630102230 0x7f263019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45736/0 (socket says 172.21.15.33:45736) 2024-04-24T23:19:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 0x7f263019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35504/0 (socket says 172.21.15.33:35504) 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 -- 172.21.15.33:0/3980323008 learned_addr learned my addr 172.21.15.33:0/3980323008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 -- 172.21.15.33:0/3980323008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630102230 msgr2=0x7f263019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630102230 0x7f263019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 -- 172.21.15.33:0/3980323008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 msgr2=0x7f263019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 0x7f263019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 -- 172.21.15.33:0/3980323008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263019f760 con 0x7f2630100680 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f263660a640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630102230 0x7f263019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635608640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 0x7f263019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2635e09640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630069440 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f262c031610 tx=0x7f262c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2626ffd640 1 -- 172.21.15.33:0/3980323008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c03f030 con 0x7f2630100680 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2626ffd640 1 -- 172.21.15.33:0/3980323008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c003ad0 con 0x7f2630100680 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263019f990 con 0x7f2630100680 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f2626ffd640 1 -- 172.21.15.33:0/3980323008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c035070 con 0x7f2630100680 2024-04-24T23:19:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.627+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263019fe80 con 0x7f2630100680 2024-04-24T23:19:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.629+0000 7f2626ffd640 1 -- 172.21.15.33:0/3980323008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f262c042030 con 0x7f2630100680 2024-04-24T23:19:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 Processor -- start 2024-04-24T23:19:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 -- start start 2024-04-24T23:19:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d6e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e37cd26b0 con 0x561e37d73da0 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561e37c7a880 con 0x561e37d6e240 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263ad76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561e37cce670 con 0x561e37d6ec30 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f2626ffd640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f261c0a2b90 0x7f261c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f2635e09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f2635608640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f2635e09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d73730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36818/0 (socket says 172.21.15.33:36818) 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f263660a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d6e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f2635e09640 1 -- 172.21.15.33:0/167619901 learned_addr learned my addr 172.21.15.33:0/167619901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.630+0000 7f2635608640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d76230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45746/0 (socket says 172.21.15.33:45746) 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f263660a640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f261c0a2b90 0x7f261c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2635608640 1 -- 172.21.15.33:0/167619901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 msgr2=0x561e37d73730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2635608640 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d73730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2635608640 1 -- 172.21.15.33:0/167619901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 msgr2=0x561e37d6e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2635608640 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d6e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2626ffd640 1 -- 172.21.15.33:0/3980323008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f262c0ef960 con 0x7f2630100680 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2635608640 1 -- 172.21.15.33:0/167619901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e37ab07d0 con 0x561e37d73da0 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f2635608640 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d76230 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2620009fa0 tx=0x7f26200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac94c3ac032451ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f263660a640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f261c0a2b90 0x7f261c0a5050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2628005fd0 tx=0x7f2628011810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f261bfff640 1 -- 172.21.15.33:0/167619901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620004310 con 0x561e37d73da0 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f261bfff640 1 -- 172.21.15.33:0/167619901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26200044b0 con 0x561e37d73da0 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.631+0000 7f261bfff640 1 -- 172.21.15.33:0/167619901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262000fa50 con 0x561e37d73da0 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 -- 172.21.15.33:0/167619901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 msgr2=0x561e37d76230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d76230 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2620009fa0 tx=0x7f26200118d0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 -- 172.21.15.33:0/167619901 shutdown_connections 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d76230 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d73730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 --2- 172.21.15.33:0/167619901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d6e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 -- 172.21.15.33:0/167619901 >> 172.21.15.33:0/167619901 conn(0x561e37d698b0 msgr2=0x561e37d69cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 -- 172.21.15.33:0/167619901 shutdown_connections 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 -- 172.21.15.33:0/167619901 wait complete. 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.632+0000 7f263ad76740 1 -- start start 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 Processor -- start 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 0x561e37d78c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d73da0 0x561e37d7fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e37d76d50 con 0x561e37d6ec30 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561e37c8df50 con 0x561e37d73da0 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561e37d76ed0 con 0x561e37d6e240 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263ad76740 10 client.0 _ll_register_callbacks cb 0x561e37cc9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 0x561e37d78c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 0x561e37d78c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45758/0 (socket says 172.21.15.33:45758) 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 -- 172.21.15.33:0/1648779194 learned_addr learned my addr 172.21.15.33:0/1648779194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635e09640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f263660a640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d73da0 0x561e37d7fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 -- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 msgr2=0x561e37d786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 -- 172.21.15.33:0/1648779194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d73da0 msgr2=0x561e37d7fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d73da0 0x561e37d7fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.633+0000 7f2635608640 1 -- 172.21.15.33:0/1648779194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e379e63f0 con 0x561e37d6ec30 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f2635608640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 0x561e37d78c10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f262001b040 tx=0x7f26200052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec68a413e243f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f26197fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f26197fa640 1 -- 172.21.15.33:0/1648779194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620029030 con 0x561e37d6ec30 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f26197fa640 1 -- 172.21.15.33:0/1648779194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f262000f790 con 0x561e37d6ec30 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26200044b0 con 0x561e37d6ec30 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f26197fa640 1 -- 172.21.15.33:0/1648779194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2620033050 con 0x561e37d6ec30 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 10 client.16887 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e379a60f0 con 0x561e37d6ec30 2024-04-24T23:19:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f26197fa640 1 -- 172.21.15.33:0/1648779194 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26200044b0 con 0x561e37d6ec30 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f26197fa640 1 -- 172.21.15.33:0/1648779194 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f26200035f0 con 0x561e37d6ec30 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 10 client.16887 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 20 client.16887 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37d841f0 0x561e37d86660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 4 client.16887 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.634+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e37cdfbf0 con 0x561e37d841f0 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.635+0000 7f2635e09640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37d841f0 0x561e37d86660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.635+0000 7f26197fa640 10 client.16887 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.635+0000 7f2635e09640 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37d841f0 0x561e37d86660 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb26e4d1e0595e0c server_cookie=d1fed8529c60273d in_seq=0 out_seq=0 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f26197fa640 1 -- 172.21.15.33:0/1648779194 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e37cdfbf0 con 0x561e37d841f0 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f26197fa640 10 client.16887 handle_command_reply: tid=0 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 client.16887 shutdown 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 msgr2=0x561e37d78c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 0x561e37d78c10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f262001b040 tx=0x7f26200052f0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 shutdown_connections 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37d841f0 0x561e37d86660 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d73da0 0x561e37d7fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f26197fa640 0 client.16887 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d6ec30 0x561e37d78c10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 --2- 172.21.15.33:0/1648779194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 >> 172.21.15.33:0/1648779194 conn(0x561e37d698b0 msgr2=0x561e37d68af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 shutdown_connections 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 1 -- 172.21.15.33:0/1648779194 wait complete. 2024-04-24T23:19:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.636+0000 7f263ad76740 20 client.16887 trim_cache size 0 max 16384 2024-04-24T23:19:09.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 Processor -- start 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 -- start start 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d6c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6ec30 0x561e37d7fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d6cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26301acc80 con 0x561e37d73da0 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26301aa550 con 0x561e37d6ec30 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.654+0000 7f263ad76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26301a83a0 con 0x561e37d6e240 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d6cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f2635e09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d6c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f2635608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6ec30 0x561e37d7fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d6cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45772/0 (socket says 172.21.15.33:45772) 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 -- 172.21.15.33:0/1764372437 learned_addr learned my addr 172.21.15.33:0/1764372437 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 -- 172.21.15.33:0/1764372437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 msgr2=0x561e37d6c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d6c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 -- 172.21.15.33:0/1764372437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6ec30 msgr2=0x561e37d7fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6ec30 0x561e37d7fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 -- 172.21.15.33:0/1764372437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e37d5b090 con 0x561e37d73da0 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f2635e09640 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f263660a640 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d6cb70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2628003a50 tx=0x7f2628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9d0848c0a3361c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f261b7fe640 1 -- 172.21.15.33:0/1764372437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628004d20 con 0x561e37d73da0 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f261b7fe640 1 -- 172.21.15.33:0/1764372437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2628005020 con 0x561e37d73da0 2024-04-24T23:19:09.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.655+0000 7f261b7fe640 1 -- 172.21.15.33:0/1764372437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262801f6a0 con 0x561e37d73da0 2024-04-24T23:19:09.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 -- 172.21.15.33:0/1764372437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 msgr2=0x561e37d6cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d6cb70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2628003a50 tx=0x7f2628004a40 comp rx=0 tx=0).stop 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 -- 172.21.15.33:0/1764372437 shutdown_connections 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d6cb70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6ec30 0x561e37d7fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 --2- 172.21.15.33:0/1764372437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6e240 0x561e37d6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 -- 172.21.15.33:0/1764372437 >> 172.21.15.33:0/1764372437 conn(0x561e37d698b0 msgr2=0x561e37d69c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 -- 172.21.15.33:0/1764372437 shutdown_connections 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 -- 172.21.15.33:0/1764372437 wait complete. 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 -- start start 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.656+0000 7f263ad76740 1 Processor -- start 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d7dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d7ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26301a8a10 con 0x561e37d73da0 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26301a9e80 con 0x561e37d6e240 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26301a90e0 con 0x561e37d6ec30 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263ad76740 10 client.0 _ll_register_callbacks cb 0x7f2628011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263660a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d7ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635e09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635608640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d7dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263660a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d7ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45786/0 (socket says 172.21.15.33:45786) 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263660a640 1 -- 172.21.15.33:0/472898066 learned_addr learned my addr 172.21.15.33:0/472898066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635608640 1 -- 172.21.15.33:0/472898066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 msgr2=0x561e37d7d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635608640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d7d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635608640 1 -- 172.21.15.33:0/472898066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 msgr2=0x561e37d7ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635608640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d7ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635608640 1 -- 172.21.15.33:0/472898066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e37d7f080 con 0x561e37d6ec30 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f2635e09640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.657+0000 7f263660a640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d7ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f2635608640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d7dad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f26200059f0 tx=0x7f2620003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01c8ddde4dd128b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f26197fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f26197fa640 1 -- 172.21.15.33:0/472898066 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262003c060 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f26197fa640 1 -- 172.21.15.33:0/472898066 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2620003db0 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2628005020 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f26197fa640 1 -- 172.21.15.33:0/472898066 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2620043050 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 10 client.26407 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e37dea9e0 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f26197fa640 1 -- 172.21.15.33:0/472898066 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2628005020 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f26197fa640 1 -- 172.21.15.33:0/472898066 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2620025470 con 0x561e37d6ec30 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 10 client.26407 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 10 client.26407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 20 client.26407 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37deb2f0 0x561e37ded710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 4 client.26407 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.658+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e37cd9f40 con 0x561e37deb2f0 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f2635e09640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37deb2f0 0x561e37ded710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f26197fa640 10 client.26407 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f2635e09640 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37deb2f0 0x561e37ded710 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760cfeaf46172e91 server_cookie=8f9924555fdea00e in_seq=0 out_seq=0 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f26197fa640 1 -- 172.21.15.33:0/472898066 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561e37cd9f40 con 0x561e37deb2f0 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f26197fa640 10 client.26407 handle_command_reply: tid=0 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 client.26407 shutdown 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 msgr2=0x561e37d7dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d7dad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f26200059f0 tx=0x7f2620003820 comp rx=0 tx=0).stop 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 shutdown_connections 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561e37deb2f0 0x561e37ded710 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e37d73da0 0x561e37d7ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e37d6ec30 0x561e37d7dad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f26197fa640 0 client.26407 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:09.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 --2- 172.21.15.33:0/472898066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561e37d6e240 0x561e37d7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 >> 172.21.15.33:0/472898066 conn(0x561e37d698b0 msgr2=0x561e37d68260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 shutdown_connections 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.659+0000 7f263ad76740 1 -- 172.21.15.33:0/472898066 wait complete. 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263ad76740 20 client.26407 trim_cache size 0 max 16384 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:19:09.668 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:19:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f261c0a2b90 msgr2=0x7f261c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f261c0a2b90 0x7f261c0a5050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2628005fd0 tx=0x7f2628011810 comp rx=0 tx=0).stop 2024-04-24T23:19:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 msgr2=0x7f2630069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630069440 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f262c031610 tx=0x7f262c004270 comp rx=0 tx=0).stop 2024-04-24T23:19:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 shutdown_connections 2024-04-24T23:19:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f261c0a2b90 0x7f261c0a5050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2630102230 0x7f263019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2630101870 0x7f263019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 --2- 172.21.15.33:0/3980323008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630100680 0x7f2630069440 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 >> 172.21.15.33:0/3980323008 conn(0x7f26300fbbf0 msgr2=0x7f2630104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 shutdown_connections 2024-04-24T23:19:09.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.660+0000 7f263806d640 1 -- 172.21.15.33:0/3980323008 wait complete. 2024-04-24T23:19:09.674 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:19:09.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:19:09.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- 172.21.15.33:0/2098816530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 msgr2=0x7fb1b8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/2098816530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8101bd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009900 tx=0x7fb1b002a410 comp rx=0 tx=0).stop 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- 172.21.15.33:0/2098816530 shutdown_connections 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/2098816530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1b8103360 0x7fb1b8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/2098816530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1b81029a0 0x7fb1b8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/2098816530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8101bd0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- 172.21.15.33:0/2098816530 >> 172.21.15.33:0/2098816530 conn(0x7fb1b80fcd20 msgr2=0x7fb1b80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- 172.21.15.33:0/2098816530 shutdown_connections 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- 172.21.15.33:0/2098816530 wait complete. 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 Processor -- start 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- start start 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1b81029a0 0x7fb1b8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1b8103360 0x7fb1b819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810d480 con 0x7fb1b81017b0 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810d300 con 0x7fb1b8103360 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.793+0000 7fb1bfd34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810d600 con 0x7fb1b81029a0 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45800/0 (socket says 172.21.15.33:45800) 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 -- 172.21.15.33:0/240723587 learned_addr learned my addr 172.21.15.33:0/240723587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bd2cf640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1b81029a0 0x7fb1b8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1be2d1640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1b8103360 0x7fb1b819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 -- 172.21.15.33:0/240723587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1b81029a0 msgr2=0x7fb1b8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1b81029a0 0x7fb1b8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 -- 172.21.15.33:0/240723587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1b8103360 msgr2=0x7fb1b819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1b8103360 0x7fb1b819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:09.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 -- 172.21.15.33:0/240723587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b819f710 con 0x7fb1b81017b0 2024-04-24T23:19:09.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bdad0640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8197af0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0031440 tx=0x7fb1b0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b003f030 con 0x7fb1b81017b0 2024-04-24T23:19:09.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b819f9a0 con 0x7fb1b81017b0 2024-04-24T23:19:09.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0003ce0 con 0x7fb1b81017b0 2024-04-24T23:19:09.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0035070 con 0x7fb1b81017b0 2024-04-24T23:19:09.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.794+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b819fee0 con 0x7fb1b81017b0 2024-04-24T23:19:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.795+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8108ad0 con 0x7fb1b81017b0 2024-04-24T23:19:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.797+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb1b0042030 con 0x7fb1b81017b0 2024-04-24T23:19:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.797+0000 7fb1b6ffd640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1a00a2b90 0x7fb1a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.798+0000 7fb1bd2cf640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1a00a2b90 0x7fb1a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.798+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb1b00efc50 con 0x7fb1b81017b0 2024-04-24T23:19:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.798+0000 7fb1bd2cf640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1a00a2b90 0x7fb1a00a5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8199000 tx=0x7fb1ac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:09.801+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1b00bc1d0 con 0x7fb1b81017b0 2024-04-24T23:19:09.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:09 smithi038 ceph-mon[32450]: pgmap v565: 129 pgs: 129 active+clean; 275 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-04-24T23:19:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:09 smithi117 ceph-mon[32529]: pgmap v565: 129 pgs: 129 active+clean; 275 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-04-24T23:19:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:09 smithi033 ceph-mon[29201]: pgmap v565: 129 pgs: 129 active+clean; 275 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-04-24T23:19:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.035+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1b802d3e0 con 0x7fb1b81017b0 2024-04-24T23:19:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.036+0000 7fb1b6ffd640 1 -- 172.21.15.33:0/240723587 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fb1b00c04d0 con 0x7fb1b81017b0 2024-04-24T23:19:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:19:10.039 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:19:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1a00a2b90 msgr2=0x7fb1a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1a00a2b90 0x7fb1a00a5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8199000 tx=0x7fb1ac005f90 comp rx=0 tx=0).stop 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 msgr2=0x7fb1b8197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8197af0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0031440 tx=0x7fb1b0032ce0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 shutdown_connections 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb1a00a2b90 0x7fb1a00a5050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1b8103360 0x7fb1b819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1b81029a0 0x7fb1b8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 --2- 172.21.15.33:0/240723587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1b81017b0 0x7fb1b8197af0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 >> 172.21.15.33:0/240723587 conn(0x7fb1b80fcd20 msgr2=0x7fb1b8105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 shutdown_connections 2024-04-24T23:19:10.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.038+0000 7fb1bfd34640 1 -- 172.21.15.33:0/240723587 wait complete. 2024-04-24T23:19:10.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:19:10.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 -- 172.21.15.33:0/534368193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68103340 msgr2=0x7f1d68109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/534368193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68103340 0x7f1d68109bd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009ca0 tx=0x7f1d6002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 -- 172.21.15.33:0/534368193 shutdown_connections 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/534368193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68103340 0x7f1d68109bd0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/534368193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 0x7f1d68102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/534368193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d68101790 0x7f1d68101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 -- 172.21.15.33:0/534368193 >> 172.21.15.33:0/534368193 conn(0x7f1d680fcd20 msgr2=0x7f1d680ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 -- 172.21.15.33:0/534368193 shutdown_connections 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.163+0000 7f1d6e05d640 1 -- 172.21.15.33:0/534368193 wait complete. 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 Processor -- start 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 -- start start 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 0x7f1d68197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 0x7f1d68198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d68103340 0x7f1d6819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d460 con 0x7f1d68101790 2024-04-24T23:19:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d2e0 con 0x7f1d68102980 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d6e05d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d5e0 con 0x7f1d68103340 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 0x7f1d68197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 0x7f1d68197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45812/0 (socket says 172.21.15.33:45812) 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d677fe640 1 -- 172.21.15.33:0/1760235462 learned_addr learned my addr 172.21.15.33:0/1760235462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 0x7f1d68198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.164+0000 7f1d67fff640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d68103340 0x7f1d6819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d677fe640 1 -- 172.21.15.33:0/1760235462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d68103340 msgr2=0x7f1d6819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d68103340 0x7f1d6819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d677fe640 1 -- 172.21.15.33:0/1760235462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 msgr2=0x7f1d68198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 0x7f1d68198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d677fe640 1 -- 172.21.15.33:0/1760235462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6819f6e0 con 0x7f1d68101790 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 0x7f1d68198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 0x7f1d68197ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9c0 tx=0x7f1d5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d64ff9640 1 -- 172.21.15.33:0/1760235462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800cd80 con 0x7f1d68101790 2024-04-24T23:19:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d64ff9640 1 -- 172.21.15.33:0/1760235462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d580035f0 con 0x7f1d68101790 2024-04-24T23:19:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6819f9d0 con 0x7f1d68101790 2024-04-24T23:19:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d64ff9640 1 -- 172.21.15.33:0/1760235462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58010690 con 0x7f1d68101790 2024-04-24T23:19:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.165+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6819fed0 con 0x7f1d68101790 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.167+0000 7f1d64ff9640 1 -- 172.21.15.33:0/1760235462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1d580037a0 con 0x7f1d68101790 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 Processor -- start 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 -- start start 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35314640 0x562e35314a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35319b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e3531c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d64ff9640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1d500a2b90 0x7f1d500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562e35274670 con 0x562e3531a1a0 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e3521c970 con 0x562e35314640 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e35270630 con 0x562e35315030 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35314640 0x562e35314a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d66ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e3531c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35314640 0x562e35314a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35594/0 (socket says 172.21.15.33:35594) 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d66ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e3531c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45826/0 (socket says 172.21.15.33:45826) 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d67fff640 1 -- 172.21.15.33:0/1503369463 learned_addr learned my addr 172.21.15.33:0/1503369463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d67fff640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1d500a2b90 0x7f1d500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d64ff9640 1 -- 172.21.15.33:0/1760235462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1d580cf720 con 0x7f1d68101790 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35319b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d66ffd640 1 -- 172.21.15.33:0/1503369463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 msgr2=0x562e35319b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.168+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35319b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d66ffd640 1 -- 172.21.15.33:0/1503369463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35314640 msgr2=0x562e35314a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35314640 0x562e35314a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d66ffd640 1 -- 172.21.15.33:0/1503369463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e35240400 con 0x562e3531a1a0 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e3531c630 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fa0 tx=0x7f1d5c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306e8dd11287a664 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d67fff640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1d500a2b90 0x7f1d500a5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x562e35315890 tx=0x7f1d60031bb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d4dffb640 1 -- 172.21.15.33:0/1503369463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004030 con 0x562e3531a1a0 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d4dffb640 1 -- 172.21.15.33:0/1503369463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c005500 con 0x562e3531a1a0 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.169+0000 7f1d4dffb640 1 -- 172.21.15.33:0/1503369463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004d80 con 0x562e3531a1a0 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1503369463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 msgr2=0x562e3531c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e3531c630 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fa0 tx=0x7f1d5c0118d0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1503369463 shutdown_connections 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e3531c630 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35319b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1503369463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35314640 0x562e35314a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1503369463 >> 172.21.15.33:0/1503369463 conn(0x562e3530b8a0 msgr2=0x562e3530bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1503369463 shutdown_connections 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1503369463 wait complete. 2024-04-24T23:19:10.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 -- start start 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 Processor -- start 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.170+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35314640 0x562e3531e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e3531ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e35325ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562e3531d150 con 0x562e3531a1a0 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e35230050 con 0x562e35315030 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e3531d2d0 con 0x562e35314640 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 10 client.0 _ll_register_callbacks cb 0x562e34f87be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e35325ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e3531ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e35325ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45828/0 (socket says 172.21.15.33:45828) 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d67fff640 1 -- 172.21.15.33:0/2055310309 learned_addr learned my addr 172.21.15.33:0/2055310309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d677fe640 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35314640 0x562e3531e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 -- 172.21.15.33:0/2055310309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35314640 msgr2=0x562e3531e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35314640 0x562e3531e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 -- 172.21.15.33:0/2055310309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 msgr2=0x562e35325ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e35325ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 -- 172.21.15.33:0/2055310309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e35201dc0 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e3531ec60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01b040 tx=0x7f1d5c004f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0aa670603fd4043 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d46ffd640 1 -- 172.21.15.33:0/2055310309 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c027070 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5c005500 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.171+0000 7f1d46ffd640 1 -- 172.21.15.33:0/2055310309 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1d5c00f5a0 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d46ffd640 1 -- 172.21.15.33:0/2055310309 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d5c032050 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 10 client.26438 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e34f883f0 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d46ffd640 1 -- 172.21.15.33:0/2055310309 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c005500 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d46ffd640 1 -- 172.21.15.33:0/2055310309 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1d5c022580 con 0x562e35315030 2024-04-24T23:19:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 10 client.26438 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 10 client.26438 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 20 client.26438 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e3532a2b0 0x562e3532c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 4 client.26438 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e35281bb0 con 0x562e3532a2b0 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.172+0000 7f1d677fe640 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e3532a2b0 0x562e3532c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.173+0000 7f1d46ffd640 10 client.26438 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:10.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.173+0000 7f1d677fe640 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e3532a2b0 0x562e3532c6e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76457ee6400863b1 server_cookie=69970f41d27aa1fc in_seq=0 out_seq=0 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d46ffd640 1 -- 172.21.15.33:0/2055310309 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e35281bb0 con 0x562e3532a2b0 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d46ffd640 10 client.26438 handle_command_reply: tid=0 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 client.26438 shutdown 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 msgr2=0x562e3531ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e3531ec60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01b040 tx=0x7f1d5c004f50 comp rx=0 tx=0).stop 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 shutdown_connections 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e3532a2b0 0x562e3532c6e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e3531a1a0 0x562e35325ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e3531ec60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/2055310309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35314640 0x562e3531e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d46ffd640 0 client.26438 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 >> 172.21.15.33:0/2055310309 conn(0x562e3530b8a0 msgr2=0x562e3530ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 shutdown_connections 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 1 -- 172.21.15.33:0/2055310309 wait complete. 2024-04-24T23:19:10.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.174+0000 7f1d70d6b740 20 client.26438 trim_cache size 0 max 16384 2024-04-24T23:19:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 Processor -- start 2024-04-24T23:19:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 -- start start 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e3531dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e35325c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e3531a1a0 0x562e3530e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681aaaa0 con 0x562e35314640 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681a8370 con 0x562e35315030 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.188+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681a61c0 con 0x562e3531a1a0 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e3531dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e3531a1a0 0x562e3530e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e3531dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45838/0 (socket says 172.21.15.33:45838) 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e3531a1a0 0x562e3530e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36888/0 (socket says 172.21.15.33:36888) 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d66ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e35325c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 -- 172.21.15.33:0/1999435615 learned_addr learned my addr 172.21.15.33:0/1999435615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 -- 172.21.15.33:0/1999435615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e3531a1a0 msgr2=0x562e3530e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e3531a1a0 0x562e3530e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 -- 172.21.15.33:0/1999435615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 msgr2=0x562e35325c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e35325c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 -- 172.21.15.33:0/1999435615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e350527d0 con 0x562e35314640 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e3531dd40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c6a0 tx=0x7f1d580096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b9192d0b6709bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d4d7fa640 1 -- 172.21.15.33:0/1999435615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5809ffa0 con 0x562e35314640 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d4d7fa640 1 -- 172.21.15.33:0/1999435615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d580b2040 con 0x562e35314640 2024-04-24T23:19:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.189+0000 7f1d4d7fa640 1 -- 172.21.15.33:0/1999435615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58003c30 con 0x562e35314640 2024-04-24T23:19:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1999435615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 msgr2=0x562e3531dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e3531dd40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c6a0 tx=0x7f1d580096e0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1999435615 shutdown_connections 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e3531a1a0 0x562e3530e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e35315030 0x562e35325c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1999435615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e3531dd40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1999435615 >> 172.21.15.33:0/1999435615 conn(0x562e3530b8a0 msgr2=0x562e3530ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1999435615 shutdown_connections 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1999435615 wait complete. 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 -- start start 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.190+0000 7f1d70d6b740 1 Processor -- start 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e353231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35323710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e3531a1a0 0x562e353246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681a6830 con 0x562e35314640 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681a7ca0 con 0x562e3531a1a0 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681a6f00 con 0x562e35315030 2024-04-24T23:19:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 10 client.0 _ll_register_callbacks cb 0x7f1d58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e353231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e353231d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45842/0 (socket says 172.21.15.33:45842) 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 -- 172.21.15.33:0/1077660776 learned_addr learned my addr 172.21.15.33:0/1077660776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35323710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d67fff640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e3531a1a0 0x562e353246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 -- 172.21.15.33:0/1077660776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 msgr2=0x562e35323710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35323710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 -- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e3531a1a0 msgr2=0x562e353246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e3531a1a0 0x562e353246d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 -- 172.21.15.33:0/1077660776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e35324cc0 con 0x562e35314640 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d677fe640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e353231d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1d580a4330 tx=0x7f1d5809df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71c5fd8b3432cde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d46ffd640 1 -- 172.21.15.33:0/1077660776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58099c90 con 0x562e35314640 2024-04-24T23:19:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d46ffd640 1 -- 172.21.15.33:0/1077660776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1d58098260 con 0x562e35314640 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d580b2040 con 0x562e35314640 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d46ffd640 1 -- 172.21.15.33:0/1077660776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d58003c10 con 0x562e35314640 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 10 client.16917 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.191+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e35390860 con 0x562e35314640 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d46ffd640 1 -- 172.21.15.33:0/1077660776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d580b2040 con 0x562e35314640 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d46ffd640 1 -- 172.21.15.33:0/1077660776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1d580a43d0 con 0x562e35314640 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d70d6b740 10 client.16917 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d70d6b740 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d70d6b740 20 client.16917 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e35391050 0x562e35393470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d70d6b740 4 client.16917 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e3531e510 con 0x562e35391050 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e35391050 0x562e35393470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d66ffd640 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e35391050 0x562e35393470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5853c803e7933e5b server_cookie=f8b2709db33938e9 in_seq=0 out_seq=0 2024-04-24T23:19:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.192+0000 7f1d46ffd640 10 client.16917 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d46ffd640 1 -- 172.21.15.33:0/1077660776 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e3531e510 con 0x562e35391050 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d46ffd640 10 client.16917 handle_command_reply: tid=0 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 client.16917 shutdown 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 msgr2=0x562e353231d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e353231d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1d580a4330 tx=0x7f1d5809df30 comp rx=0 tx=0).stop 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 shutdown_connections 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562e35391050 0x562e35393470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e3531a1a0 0x562e353246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e35315030 0x562e35323710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 --2- 172.21.15.33:0/1077660776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e35314640 0x562e353231d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 >> 172.21.15.33:0/1077660776 conn(0x562e3530b8a0 msgr2=0x562e3530ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d46ffd640 0 client.16917 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 shutdown_connections 2024-04-24T23:19:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 1 -- 172.21.15.33:0/1077660776 wait complete. 2024-04-24T23:19:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.193+0000 7f1d70d6b740 20 client.16917 trim_cache size 0 max 16384 2024-04-24T23:19:10.202 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:19:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1d500a2b90 msgr2=0x7f1d500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1d500a2b90 0x7f1d500a5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x562e35315890 tx=0x7f1d60031bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 msgr2=0x7f1d68197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 0x7f1d68197ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9c0 tx=0x7f1d5800ee80 comp rx=0 tx=0).stop 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 shutdown_connections 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1d500a2b90 0x7f1d500a5050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d68103340 0x7f1d6819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d68102980 0x7f1d68198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 --2- 172.21.15.33:0/1760235462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d68101790 0x7f1d68197ac0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 >> 172.21.15.33:0/1760235462 conn(0x7f1d680fcd20 msgr2=0x7f1d68105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 shutdown_connections 2024-04-24T23:19:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.194+0000 7f1d6e05d640 1 -- 172.21.15.33:0/1760235462 wait complete. 2024-04-24T23:19:10.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:19:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 -- 172.21.15.33:0/2456674219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 msgr2=0x7f04240ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 --2- 172.21.15.33:0/2456674219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f04240ffe80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0414009900 tx=0x7f041402a410 comp rx=0 tx=0).stop 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 -- 172.21.15.33:0/2456674219 shutdown_connections 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 --2- 172.21.15.33:0/2456674219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0424101180 0x7f0424105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 --2- 172.21.15.33:0/2456674219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04241007c0 0x7f0424100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 --2- 172.21.15.33:0/2456674219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f04240ffe80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 -- 172.21.15.33:0/2456674219 >> 172.21.15.33:0/2456674219 conn(0x7f04240fb370 msgr2=0x7f04240fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 -- 172.21.15.33:0/2456674219 shutdown_connections 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 -- 172.21.15.33:0/2456674219 wait complete. 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.316+0000 7f0429763640 1 Processor -- start 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 -- start start 2024-04-24T23:19:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f042419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241007c0 0x7f042419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0424101180 0x7f04241a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113ae0 con 0x7f04240ffa60 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113960 con 0x7f0424101180 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113c60 con 0x7f04241007c0 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f042419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f042419e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45864/0 (socket says 172.21.15.33:45864) 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 -- 172.21.15.33:0/471745033 learned_addr learned my addr 172.21.15.33:0/471745033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f04237fe640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0424101180 0x7f04241a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f04227fc640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241007c0 0x7f042419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 -- 172.21.15.33:0/471745033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241007c0 msgr2=0x7f042419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241007c0 0x7f042419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 -- 172.21.15.33:0/471745033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0424101180 msgr2=0x7f04241a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0424101180 0x7f04241a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 -- 172.21.15.33:0/471745033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04241a5ce0 con 0x7f04240ffa60 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0422ffd640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f042419e0c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0414031440 tx=0x7f0414032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f040bfff640 1 -- 172.21.15.33:0/471745033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041403f030 con 0x7f04240ffa60 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.317+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04241a5f70 con 0x7f04240ffa60 2024-04-24T23:19:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.318+0000 7f040bfff640 1 -- 172.21.15.33:0/471745033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0414003ce0 con 0x7f04240ffa60 2024-04-24T23:19:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.318+0000 7f040bfff640 1 -- 172.21.15.33:0/471745033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414035070 con 0x7f04240ffa60 2024-04-24T23:19:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.318+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241a63d0 con 0x7f04240ffa60 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f040bfff640 1 -- 172.21.15.33:0/471745033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0414042030 con 0x7f04240ffa60 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f042c477740 1 Processor -- start 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f042c477740 1 -- start start 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f040bfff640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f040c0a2b90 0x7f040c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f042c477740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bcb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f042c477740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.320+0000 7f042c477740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bd3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04237fe640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f040c0a2b90 0x7f040c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45868/0 (socket says 172.21.15.33:45868) 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bd3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f040bfff640 1 -- 172.21.15.33:0/471745033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04140ef4e0 con 0x7f04240ffa60 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04227fc640 1 -- 172.21.15.33:0/3135739832 learned_addr learned my addr 172.21.15.33:0/3135739832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bcb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556776b2f670 con 0x556776bcbc20 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556776ad7970 con 0x556776bd0d90 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556776b2b630 con 0x556776bcb230 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 -- 172.21.15.33:0/3135739832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 msgr2=0x556776bd3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bd3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 -- 172.21.15.33:0/3135739832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 msgr2=0x556776bd0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 -- 172.21.15.33:0/3135739832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556776afb400 con 0x556776bcb230 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04227fc640 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04237fe640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f040c0a2b90 0x7f040c0a5050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f041c005fd0 tx=0x7f041c009e00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bcb650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0414032ed0 tx=0x7f04140b7dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee33699aa9ea5ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04097fa640 1 -- 172.21.15.33:0/3135739832 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414049070 con 0x556776bcb230 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04097fa640 1 -- 172.21.15.33:0/3135739832 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04140bba60 con 0x556776bcb230 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.321+0000 7f04097fa640 1 -- 172.21.15.33:0/3135739832 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140ef250 con 0x556776bcb230 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 msgr2=0x556776bcb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bcb650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0414032ed0 tx=0x7f04140b7dc0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 shutdown_connections 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bd3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 --2- 172.21.15.33:0/3135739832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bcb650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 >> 172.21.15.33:0/3135739832 conn(0x556776bc68a0 msgr2=0x556776bc6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 shutdown_connections 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.322+0000 7f042c477740 1 -- 172.21.15.33:0/3135739832 wait complete. 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 -- start start 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 Processor -- start 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bd50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bdc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556776bd3b60 con 0x556776bcbc20 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556776aeb050 con 0x556776bd0d90 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556776bd3ce0 con 0x556776bcb230 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f042c477740 10 client.0 _ll_register_callbacks cb 0x556776842be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45880/0 (socket says 172.21.15.33:45880) 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 -- 172.21.15.33:0/3889830327 learned_addr learned my addr 172.21.15.33:0/3889830327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bd50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04237fe640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bdc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 -- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 msgr2=0x556776bd50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bd50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 -- 172.21.15.33:0/3889830327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 msgr2=0x556776bdc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bdc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.323+0000 7f04227fc640 1 -- 172.21.15.33:0/3889830327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556776abcdc0 con 0x556776bcbc20 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f04227fc640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd55f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f04180126d0 tx=0x7f0418012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1c703a43a87900 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f0406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f0406ff5640 1 -- 172.21.15.33:0/3889830327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418004050 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f0406ff5640 1 -- 172.21.15.33:0/3889830327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04180041f0 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04140bba60 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f0406ff5640 1 -- 172.21.15.33:0/3889830327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04180043f0 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 10 client.16929 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567768433f0 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f0406ff5640 1 -- 172.21.15.33:0/3889830327 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04140bba60 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f0406ff5640 1 -- 172.21.15.33:0/3889830327 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0418003b60 con 0x556776bcbc20 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 10 client.16929 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 10 client.16929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 20 client.16929 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776be0ff0 0x556776be3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 4 client.16929 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.324+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556776b3cbb0 con 0x556776be0ff0 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.325+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776be0ff0 0x556776be3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.325+0000 7f0406ff5640 10 client.16929 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.325+0000 7f0422ffd640 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776be0ff0 0x556776be3450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce04c5a17e8628ff server_cookie=3b1afc3b64e01ba0 in_seq=0 out_seq=0 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.325+0000 7f0406ff5640 1 -- 172.21.15.33:0/3889830327 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556776b3cbb0 con 0x556776be0ff0 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.325+0000 7f0406ff5640 10 client.16929 handle_command_reply: tid=0 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 client.16929 shutdown 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 msgr2=0x556776bd55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd55f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f04180126d0 tx=0x7f0418012b90 comp rx=0 tx=0).stop 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 shutdown_connections 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776be0ff0 0x556776be3450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bd0d90 0x556776bdc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bcbc20 0x556776bd55f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f0406ff5640 0 client.16929 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 --2- 172.21.15.33:0/3889830327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bd50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 >> 172.21.15.33:0/3889830327 conn(0x556776bc68a0 msgr2=0x556776bc5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 shutdown_connections 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 1 -- 172.21.15.33:0/3889830327 wait complete. 2024-04-24T23:19:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.326+0000 7f042c477740 20 client.16929 trim_cache size 0 max 16384 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 Processor -- start 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 -- start start 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bdd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 0x556776bdc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bcc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04241b30e0 con 0x556776bd0d90 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04241b09b0 con 0x556776bcbc20 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f042c477740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04241ae800 con 0x556776bcb230 2024-04-24T23:19:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f04237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bcc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f04227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 0x556776bdc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f04237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bcc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45892/0 (socket says 172.21.15.33:45892) 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f04227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 0x556776bdc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35688/0 (socket says 172.21.15.33:35688) 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f04237fe640 1 -- 172.21.15.33:0/1061769970 learned_addr learned my addr 172.21.15.33:0/1061769970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.339+0000 7f0422ffd640 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bdd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04227fc640 1 -- 172.21.15.33:0/1061769970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 msgr2=0x556776bdd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04227fc640 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bdd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04227fc640 1 -- 172.21.15.33:0/1061769970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 msgr2=0x556776bcc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04227fc640 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bcc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04227fc640 1 -- 172.21.15.33:0/1061769970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677690d7d0 con 0x556776bcbc20 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04237fe640 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bcc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f04227fc640 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 0x556776bdc5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x556776bd6590 tx=0x7f0418001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb2bab19f347637 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f0408ff9640 1 -- 172.21.15.33:0/1061769970 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041802d030 con 0x556776bcbc20 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f0408ff9640 1 -- 172.21.15.33:0/1061769970 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0418003fa0 con 0x556776bcbc20 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.340+0000 7f0408ff9640 1 -- 172.21.15.33:0/1061769970 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041800fe60 con 0x556776bcbc20 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- 172.21.15.33:0/1061769970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 msgr2=0x556776bdc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 0x556776bdc5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x556776bd6590 tx=0x7f0418001e70 comp rx=0 tx=0).stop 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- 172.21.15.33:0/1061769970 shutdown_connections 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bcc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcbc20 0x556776bdc5f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- 172.21.15.33:0/1061769970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcb230 0x556776bdd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- 172.21.15.33:0/1061769970 >> 172.21.15.33:0/1061769970 conn(0x556776bc68a0 msgr2=0x556776bc5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- 172.21.15.33:0/1061769970 shutdown_connections 2024-04-24T23:19:10.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- 172.21.15.33:0/1061769970 wait complete. 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- start start 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 Processor -- start 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcb230 0x556776bd9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcbc20 0x556776bda400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bdb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04241aee70 con 0x556776bd0d90 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04241b02e0 con 0x556776bcb230 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04241af540 con 0x556776bcbc20 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f042c477740 10 client.0 _ll_register_callbacks cb 0x7f0418009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.341+0000 7f04237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bdb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f0422ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcb230 0x556776bd9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcbc20 0x556776bda400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bdb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45908/0 (socket says 172.21.15.33:45908) 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 -- 172.21.15.33:0/1027383809 learned_addr learned my addr 172.21.15.33:0/1027383809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 -- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcbc20 msgr2=0x556776bda400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcbc20 0x556776bda400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 -- 172.21.15.33:0/1027383809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcb230 msgr2=0x556776bd9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcb230 0x556776bd9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 -- 172.21.15.33:0/1027383809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556776bdb990 con 0x556776bd0d90 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04227fc640 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcbc20 0x556776bda400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f04237fe640 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bdb3a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f041c013350 tx=0x7f041c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7def91d3ff0e4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f0406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f0406ff5640 1 -- 172.21.15.33:0/1027383809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c004b20 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f0406ff5640 1 -- 172.21.15.33:0/1027383809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f041c01c040 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0418003fa0 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f0406ff5640 1 -- 172.21.15.33:0/1027383809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f041c034050 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f042c477740 10 client.16935 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.342+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556776c47780 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0406ff5640 1 -- 172.21.15.33:0/1027383809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0418003fa0 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0406ff5640 1 -- 172.21.15.33:0/1027383809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f041c020050 con 0x556776bd0d90 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 10 client.16935 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 20 client.16935 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776c481d0 0x556776c4a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 4 client.16935 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556776b36f00 con 0x556776c481d0 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0422ffd640 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776c481d0 0x556776c4a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0406ff5640 10 client.16935 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0422ffd640 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776c481d0 0x556776c4a5f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a38769fc10634bc6 server_cookie=e74102d41d410ca0 in_seq=0 out_seq=0 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0406ff5640 1 -- 172.21.15.33:0/1027383809 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556776b36f00 con 0x556776c481d0 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f0406ff5640 10 client.16935 handle_command_reply: tid=0 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.343+0000 7f042c477740 1 client.16935 shutdown 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 msgr2=0x556776bdb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bdb3a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f041c013350 tx=0x7f041c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 shutdown_connections 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556776c481d0 0x556776c4a5f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556776bd0d90 0x556776bdb3a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0406ff5640 0 client.16935 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556776bcbc20 0x556776bda400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 --2- 172.21.15.33:0/1027383809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556776bcb230 0x556776bd9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 >> 172.21.15.33:0/1027383809 conn(0x556776bc68a0 msgr2=0x556776bc9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 shutdown_connections 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 1 -- 172.21.15.33:0/1027383809 wait complete. 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f042c477740 20 client.16935 trim_cache size 0 max 16384 2024-04-24T23:19:10.355 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:19:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f040c0a2b90 msgr2=0x7f040c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f040c0a2b90 0x7f040c0a5050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f041c005fd0 tx=0x7f041c009e00 comp rx=0 tx=0).stop 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 msgr2=0x7f042419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f042419e0c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0414031440 tx=0x7f0414032ce0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 shutdown_connections 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f040c0a2b90 0x7f040c0a5050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0424101180 0x7f04241a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241007c0 0x7f042419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 --2- 172.21.15.33:0/471745033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04240ffa60 0x7f042419e0c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.344+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 >> 172.21.15.33:0/471745033 conn(0x7f04240fb370 msgr2=0x7f0424101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.345+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 shutdown_connections 2024-04-24T23:19:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.345+0000 7f0429763640 1 -- 172.21.15.33:0/471745033 wait complete. 2024-04-24T23:19:10.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:19:10.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 756469 12.75 MB/sec execute 195 sec latency 21.744 ms 2024-04-24T23:19:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 -- 172.21.15.33:0/2841258236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8102230 msgr2=0x7f8cd8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 --2- 172.21.15.33:0/2841258236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8102230 0x7f8cd8108ac0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009ca0 tx=0x7f8cd002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 -- 172.21.15.33:0/2841258236 shutdown_connections 2024-04-24T23:19:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 --2- 172.21.15.33:0/2841258236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8102230 0x7f8cd8108ac0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 --2- 172.21.15.33:0/2841258236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8101870 0x7f8cd8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 --2- 172.21.15.33:0/2841258236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd8100680 0x7f8cd8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 -- 172.21.15.33:0/2841258236 >> 172.21.15.33:0/2841258236 conn(0x7f8cd80fbbf0 msgr2=0x7f8cd80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 -- 172.21.15.33:0/2841258236 shutdown_connections 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.463+0000 7f8cddab3640 1 -- 172.21.15.33:0/2841258236 wait complete. 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 Processor -- start 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 -- start start 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8100680 0x7f8cd8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 0x7f8cd8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd8102230 0x7f8cd819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810d3e0 con 0x7f8cd8101870 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810d260 con 0x7f8cd8100680 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cddab3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810d560 con 0x7f8cd8102230 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.464+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 0x7f8cd8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 0x7f8cd8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45916/0 (socket says 172.21.15.33:45916) 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/1160528543 learned_addr learned my addr 172.21.15.33:0/1160528543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8100680 0x7f8cd8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd7fff640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd8102230 0x7f8cd819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/1160528543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd8102230 msgr2=0x7f8cd819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd8102230 0x7f8cd819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/1160528543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8100680 msgr2=0x7f8cd8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8100680 0x7f8cd8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/1160528543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd819f670 con 0x7f8cd8101870 2024-04-24T23:19:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 0x7f8cd8197f90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e9f0 tx=0x7f8ccc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd4ff9640 1 -- 172.21.15.33:0/1160528543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00cd80 con 0x7f8cd8101870 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd4ff9640 1 -- 172.21.15.33:0/1160528543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc010910 con 0x7f8cd8101870 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd819f960 con 0x7f8cd8101870 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cd4ff9640 1 -- 172.21.15.33:0/1160528543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc018990 con 0x7f8cd8101870 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.465+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd819feb0 con 0x7f8cd8101870 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.467+0000 7f8cd4ff9640 1 -- 172.21.15.33:0/1160528543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8ccc010ac0 con 0x7f8cd8101870 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 Processor -- start 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 -- start start 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf70720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf73220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbecf670 con 0x55b5fbf6b230 2024-04-24T23:19:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbe77970 con 0x55b5fbf6bc20 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbecb630 con 0x55b5fbf70d90 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd4ff9640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8cc00a2b90 0x7f8cc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf70720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf70720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35714/0 (socket says 172.21.15.33:35714) 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd77fe640 1 -- 172.21.15.33:0/341536998 learned_addr learned my addr 172.21.15.33:0/341536998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf73220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.468+0000 7f8cd7fff640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8cc00a2b90 0x7f8cc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd77fe640 1 -- 172.21.15.33:0/341536998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 msgr2=0x55b5fbf73220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd4ff9640 1 -- 172.21.15.33:0/1160528543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ccc0cf660 con 0x7f8cd8101870 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf73220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd77fe640 1 -- 172.21.15.33:0/341536998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 msgr2=0x55b5fbf6b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf6b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd77fe640 1 -- 172.21.15.33:0/341536998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fbe9b400 con 0x55b5fbf6bc20 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd7fff640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8cc00a2b90 0x7f8cc00a5050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0004af0 tx=0x7f8cd0005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf70720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800b7d0 tx=0x7f8cc8004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301774b46780ce4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cbdffb640 1 -- 172.21.15.33:0/341536998 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8003dd0 con 0x55b5fbf6bc20 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cbdffb640 1 -- 172.21.15.33:0/341536998 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8015070 con 0x55b5fbf6bc20 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.469+0000 7f8cbdffb640 1 -- 172.21.15.33:0/341536998 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80117a0 con 0x55b5fbf6bc20 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 -- 172.21.15.33:0/341536998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 msgr2=0x55b5fbf70720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf70720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800b7d0 tx=0x7f8cc8004610 comp rx=0 tx=0).stop 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 -- 172.21.15.33:0/341536998 shutdown_connections 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf73220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf70720 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/341536998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf6b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 -- 172.21.15.33:0/341536998 >> 172.21.15.33:0/341536998 conn(0x55b5fbf668a0 msgr2=0x55b5fbf66ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 -- 172.21.15.33:0/341536998 shutdown_connections 2024-04-24T23:19:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 -- 172.21.15.33:0/341536998 wait complete. 2024-04-24T23:19:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 -- start start 2024-04-24T23:19:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.470+0000 7f8ce07c0740 1 Processor -- start 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf75380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbf73d40 con 0x55b5fbf6b230 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbe8b050 con 0x55b5fbf70d90 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbf73ec0 con 0x55b5fbf6bc20 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 10 client.0 _ll_register_callbacks cb 0x55b5fbbe2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf75380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf75380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45930/0 (socket says 172.21.15.33:45930) 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 -- 172.21.15.33:0/1320415809 learned_addr learned my addr 172.21.15.33:0/1320415809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd7fff640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 -- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 msgr2=0x55b5fbf758c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf758c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 -- 172.21.15.33:0/1320415809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 msgr2=0x55b5fbf7c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 -- 172.21.15.33:0/1320415809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fbe5cdc0 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf75380 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80039c0 tx=0x7f8cc8003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7e719d10b1e5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/1320415809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800be70 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/1320415809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8cc801a680 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.471+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc8015070 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/1320415809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8cc8030050 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 10 client.16947 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5fbbe33f0 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/1320415809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8015070 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/1320415809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8cc8011cd0 con 0x55b5fbf6b230 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 10 client.16947 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 20 client.16947 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbf83290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 4 client.16947 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5fbedcbb0 con 0x55b5fbf80ea0 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.472+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbf83290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8cb6ffd640 10 client.16947 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbf83290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26393364520d98d server_cookie=e55b8588a1cd305b in_seq=0 out_seq=0 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/1320415809 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5fbedcbb0 con 0x55b5fbf80ea0 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8cb6ffd640 10 client.16947 handle_command_reply: tid=0 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 client.16947 shutdown 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 msgr2=0x55b5fbf75380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf75380 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80039c0 tx=0x7f8cc8003e00 comp rx=0 tx=0).stop 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 shutdown_connections 2024-04-24T23:19:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbf83290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8cb6ffd640 0 client.16947 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.473+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/1320415809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf75380 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.474+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 >> 172.21.15.33:0/1320415809 conn(0x55b5fbf668a0 msgr2=0x55b5fbf65b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.474+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 shutdown_connections 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.474+0000 7f8ce07c0740 1 -- 172.21.15.33:0/1320415809 wait complete. 2024-04-24T23:19:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.474+0000 7f8ce07c0740 20 client.16947 trim_cache size 0 max 16384 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 Processor -- start 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 -- start start 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf65d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81aaa80 con 0x55b5fbf6b230 2024-04-24T23:19:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81a8350 con 0x55b5fbf70d90 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81a61a0 con 0x55b5fbf6bc20 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf65d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf65d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45942/0 (socket says 172.21.15.33:45942) 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 -- 172.21.15.33:0/2603578990 learned_addr learned my addr 172.21.15.33:0/2603578990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd7fff640 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 -- 172.21.15.33:0/2603578990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 msgr2=0x55b5fbf7c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 -- 172.21.15.33:0/2603578990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 msgr2=0x55b5fbf749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.488+0000 7f8cd77fe640 1 -- 172.21.15.33:0/2603578990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fbcad7d0 con 0x55b5fbf6b230 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.489+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf65d80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8006010 tx=0x7f8cc8025780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63090fd243b8b688 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.489+0000 7f8cbd7fa640 1 -- 172.21.15.33:0/2603578990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800be70 con 0x55b5fbf6b230 2024-04-24T23:19:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.489+0000 7f8cbd7fa640 1 -- 172.21.15.33:0/2603578990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8015040 con 0x55b5fbf6b230 2024-04-24T23:19:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.489+0000 7f8cbd7fa640 1 -- 172.21.15.33:0/2603578990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc801a680 con 0x55b5fbf6b230 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2603578990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 msgr2=0x55b5fbf65d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf65d80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8006010 tx=0x7f8cc8025780 comp rx=0 tx=0).stop 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2603578990 shutdown_connections 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf70d90 0x55b5fbf749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2603578990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6b230 0x55b5fbf65d80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2603578990 >> 172.21.15.33:0/2603578990 conn(0x55b5fbf668a0 msgr2=0x55b5fbf6a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2603578990 shutdown_connections 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2603578990 wait complete. 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- start start 2024-04-24T23:19:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 Processor -- start 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6b230 0x55b5fbf79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81a6810 con 0x55b5fbf6bc20 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81a7c80 con 0x55b5fbf6b230 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81a6ee0 con 0x55b5fbf70d90 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.490+0000 7f8ce07c0740 10 client.0 _ll_register_callbacks cb 0x7f8cd0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6b230 0x55b5fbf79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45954/0 (socket says 172.21.15.33:45954) 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/2336373198 learned_addr learned my addr 172.21.15.33:0/2336373198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6b230 0x55b5fbf79d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35732/0 (socket says 172.21.15.33:35732) 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd7fff640 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 msgr2=0x55b5fbf7b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/2336373198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6b230 msgr2=0x55b5fbf79d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6b230 0x55b5fbf79d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 -- 172.21.15.33:0/2336373198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fbf7b840 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cd6ffd640 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7a240 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c6a0 tx=0x7f8ccc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d7019b75faba28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/2336373198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc086040 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/2336373198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ccc0cf380 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc8015040 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/2336373198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc026040 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8ce07c0740 10 client.16953 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.491+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5fbf7cbb0 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/2336373198 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8015040 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/2336373198 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8ccc082b50 con 0x55b5fbf6bc20 2024-04-24T23:19:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 10 client.16953 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 20 client.16953 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbfe9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 4 client.16953 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5fbed6f00 con 0x55b5fbf80ea0 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbfe9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cd77fe640 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbfe9b30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ddf9e6750de568 server_cookie=e1f601a233f65a7 in_seq=0 out_seq=0 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cb6ffd640 10 client.16953 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cb6ffd640 1 -- 172.21.15.33:0/2336373198 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5fbed6f00 con 0x55b5fbf80ea0 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8cb6ffd640 10 client.16953 handle_command_reply: tid=0 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.492+0000 7f8ce07c0740 1 client.16953 shutdown 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 msgr2=0x55b5fbf7a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7a240 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c6a0 tx=0x7f8ccc018420 comp rx=0 tx=0).stop 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 shutdown_connections 2024-04-24T23:19:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b5fbf80ea0 0x55b5fbfe9b30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5fbf70d90 0x55b5fbf7b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5fbf6bc20 0x55b5fbf7a240 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cb6ffd640 0 client.16953 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 --2- 172.21.15.33:0/2336373198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5fbf6b230 0x55b5fbf79d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 >> 172.21.15.33:0/2336373198 conn(0x55b5fbf668a0 msgr2=0x55b5fbf6c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 shutdown_connections 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 1 -- 172.21.15.33:0/2336373198 wait complete. 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8ce07c0740 20 client.16953 trim_cache size 0 max 16384 2024-04-24T23:19:10.501 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:19:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8cc00a2b90 msgr2=0x7f8cc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8cc00a2b90 0x7f8cc00a5050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0004af0 tx=0x7f8cd0005d80 comp rx=0 tx=0).stop 2024-04-24T23:19:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 msgr2=0x7f8cd8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 0x7f8cd8197f90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e9f0 tx=0x7f8ccc00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 shutdown_connections 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8cc00a2b90 0x7f8cc00a5050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd8102230 0x7f8cd819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cd8101870 0x7f8cd8197f90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 --2- 172.21.15.33:0/1160528543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8100680 0x7f8cd8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.493+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 >> 172.21.15.33:0/1160528543 conn(0x7f8cd80fbbf0 msgr2=0x7f8cd8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.494+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 shutdown_connections 2024-04-24T23:19:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:10.494+0000 7f8cddab3640 1 -- 172.21.15.33:0/1160528543 wait complete. 2024-04-24T23:19:10.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:10 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/240723587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:19:10.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:10 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/240723587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:19:10.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:10 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/240723587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:19:11.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 758380 12.72 MB/sec execute 196 sec latency 18.425 ms 2024-04-24T23:19:11.507 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:19:11.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:19:11.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 -- 172.21.15.33:0/3119908093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 msgr2=0x7fc464101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 --2- 172.21.15.33:0/3119908093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464101cf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc454009900 tx=0x7fc45402a410 comp rx=0 tx=0).stop 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 -- 172.21.15.33:0/3119908093 shutdown_connections 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 --2- 172.21.15.33:0/3119908093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc464102230 0x7fc464108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 --2- 172.21.15.33:0/3119908093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464101cf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 --2- 172.21.15.33:0/3119908093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 0x7fc464100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.618+0000 7fc46baa5640 1 -- 172.21.15.33:0/3119908093 >> 172.21.15.33:0/3119908093 conn(0x7fc4640fbbf0 msgr2=0x7fc4640fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 -- 172.21.15.33:0/3119908093 shutdown_connections 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 -- 172.21.15.33:0/3119908093 wait complete. 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 Processor -- start 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 -- start start 2024-04-24T23:19:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 0x7fc464197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc464102230 0x7fc46419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410c300 con 0x7fc464101870 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410c180 con 0x7fc464100680 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.619+0000 7fc46baa5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410c480 con 0x7fc464102230 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469841640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 0x7fc464197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469841640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 0x7fc464197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35758/0 (socket says 172.21.15.33:35758) 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45972/0 (socket says 172.21.15.33:45972) 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469841640 1 -- 172.21.15.33:0/1590150527 learned_addr learned my addr 172.21.15.33:0/1590150527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc46a042640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc464102230 0x7fc46419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 -- 172.21.15.33:0/1590150527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc464102230 msgr2=0x7fc46419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc464102230 0x7fc46419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 -- 172.21.15.33:0/1590150527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 msgr2=0x7fc464197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 0x7fc464197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 -- 172.21.15.33:0/1590150527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46419f6f0 con 0x7fc464101870 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc469040640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464198010 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc454031440 tx=0x7fc454004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc45affd640 1 -- 172.21.15.33:0/1590150527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45403f070 con 0x7fc464101870 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc45affd640 1 -- 172.21.15.33:0/1590150527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc454003cd0 con 0x7fc464101870 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46419f980 con 0x7fc464101870 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc45affd640 1 -- 172.21.15.33:0/1590150527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454035070 con 0x7fc464101870 2024-04-24T23:19:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.620+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46419fe40 con 0x7fc464101870 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.622+0000 7fc45affd640 1 -- 172.21.15.33:0/1590150527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc45403b4f0 con 0x7fc464101870 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 Processor -- start 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 -- start start 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fac31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac3780 0x55fa7fac8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 0x55fa7facad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fa262c0 con 0x55fa7fac2d90 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa7f9ce910 con 0x55fa7fac88f0 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.623+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fa22280 con 0x55fa7fac3780 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 0x55fa7facad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc46a042640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fac31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469841640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac3780 0x55fa7fac8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc46a042640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fac31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45984/0 (socket says 172.21.15.33:45984) 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc46a042640 1 -- 172.21.15.33:0/707590938 learned_addr learned my addr 172.21.15.33:0/707590938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 0x55fa7facad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35766/0 (socket says 172.21.15.33:35766) 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc45affd640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4500a2b90 0x7fc4500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc46a042640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4500a2b90 0x7fc4500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc45affd640 1 -- 172.21.15.33:0/1590150527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4540efb60 con 0x7fc464101870 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 -- 172.21.15.33:0/707590938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac3780 msgr2=0x55fa7fac8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac3780 0x55fa7fac8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 -- 172.21.15.33:0/707590938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 msgr2=0x55fa7fac31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fac31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 -- 172.21.15.33:0/707590938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7f9dffb0 con 0x55fa7fac88f0 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc46a042640 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fac31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc46a042640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4500a2b90 0x7fc4500a5050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009fd0 tx=0x7fc45c009c40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.624+0000 7fc469040640 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 0x55fa7facad80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc4540ed320 tx=0x7fc45402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a483c5911b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc44ffff640 1 -- 172.21.15.33:0/707590938 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454048070 con 0x55fa7fac88f0 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc44ffff640 1 -- 172.21.15.33:0/707590938 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4540bc730 con 0x55fa7fac88f0 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc44ffff640 1 -- 172.21.15.33:0/707590938 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4540ba0d0 con 0x55fa7fac88f0 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 -- 172.21.15.33:0/707590938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 msgr2=0x55fa7facad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 0x55fa7facad80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc4540ed320 tx=0x7fc45402a410 comp rx=0 tx=0).stop 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 -- 172.21.15.33:0/707590938 shutdown_connections 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac88f0 0x55fa7facad80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac3780 0x55fa7fac8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/707590938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fac31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.625+0000 7fc46e7ba740 1 -- 172.21.15.33:0/707590938 >> 172.21.15.33:0/707590938 conn(0x55fa7fa8a950 msgr2=0x55fa7fabe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 -- 172.21.15.33:0/707590938 shutdown_connections 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 -- 172.21.15.33:0/707590938 wait complete. 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 -- start start 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 Processor -- start 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7faccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7facd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fa7f9ce910 con 0x55fa7fac2d90 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa7f9e1e90 con 0x55fa7fac3780 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fa7facb660 con 0x55fa7fac88f0 2024-04-24T23:19:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc46e7ba740 10 client.0 _ll_register_callbacks cb 0x55fa7fa1d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc469841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7faccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc469841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7faccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45994/0 (socket says 172.21.15.33:45994) 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.626+0000 7fc469841640 1 -- 172.21.15.33:0/4105450094 learned_addr learned my addr 172.21.15.33:0/4105450094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469040640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7facd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc46a042640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469841640 1 -- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 msgr2=0x55fa7fad42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469841640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469841640 1 -- 172.21.15.33:0/4105450094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 msgr2=0x55fa7facd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469841640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7facd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469841640 1 -- 172.21.15.33:0/4105450094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7f8027e0 con 0x55fa7fac2d90 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc469841640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7faccd60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc460012740 tx=0x7fc460012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49660808ed9c3f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc44d7fa640 1 -- 172.21.15.33:0/4105450094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460012190 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc44d7fa640 1 -- 172.21.15.33:0/4105450094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc460004030 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4540bc730 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc44d7fa640 1 -- 172.21.15.33:0/4105450094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc460025050 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc46e7ba740 10 client.16965 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.627+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa7f862f70 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc44d7fa640 1 -- 172.21.15.33:0/4105450094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4540bc730 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc44d7fa640 1 -- 172.21.15.33:0/4105450094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc460003780 con 0x55fa7fac2d90 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc46e7ba740 10 client.16965 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc46e7ba740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc46e7ba740 20 client.16965 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fad8890 0x55fa7fadad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc46e7ba740 4 client.16965 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa7f9fa5a0 con 0x55fa7fad8890 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc469040640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fad8890 0x55fa7fadad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc469040640 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fad8890 0x55fa7fadad50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccb3762b018cc87 server_cookie=fac790701c9d2565 in_seq=0 out_seq=0 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.628+0000 7fc44d7fa640 10 client.16965 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc44d7fa640 1 -- 172.21.15.33:0/4105450094 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa7f9fa5a0 con 0x55fa7fad8890 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc44d7fa640 10 client.16965 handle_command_reply: tid=0 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 client.16965 shutdown 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 msgr2=0x55fa7faccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7faccd60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc460012740 tx=0x7fc460012c00 comp rx=0 tx=0).stop 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 shutdown_connections 2024-04-24T23:19:11.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fad8890 0x55fa7fadad50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc44d7fa640 0 client.16965 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7facd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4105450094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7faccd60 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.629+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 >> 172.21.15.33:0/4105450094 conn(0x55fa7fa8a950 msgr2=0x55fa7fabd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.630+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 shutdown_connections 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.630+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4105450094 wait complete. 2024-04-24T23:19:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.630+0000 7fc46e7ba740 20 client.16965 trim_cache size 0 max 16384 2024-04-24T23:19:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.649+0000 7fc46e7ba740 1 Processor -- start 2024-04-24T23:19:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.649+0000 7fc46e7ba740 1 -- start start 2024-04-24T23:19:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7facd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fabd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641acc80 con 0x55fa7fac2d90 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641aa550 con 0x55fa7fac3780 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641a83a0 con 0x55fa7fac88f0 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7facd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7facd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46004/0 (socket says 172.21.15.33:46004) 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 -- 172.21.15.33:0/4172161932 learned_addr learned my addr 172.21.15.33:0/4172161932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469040640 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc46a042640 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fabd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 -- 172.21.15.33:0/4172161932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 msgr2=0x55fa7fabd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fabd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 -- 172.21.15.33:0/4172161932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 msgr2=0x55fa7fad42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 -- 172.21.15.33:0/4172161932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7f6fa170 con 0x55fa7fac2d90 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469040640 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc469841640 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7facd040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc460001eb0 tx=0x7fc4600098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee3e6942ce01cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc44f7fe640 1 -- 172.21.15.33:0/4172161932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46002e070 con 0x55fa7fac2d90 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc44f7fe640 1 -- 172.21.15.33:0/4172161932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4600050c0 con 0x55fa7fac2d90 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.650+0000 7fc44f7fe640 1 -- 172.21.15.33:0/4172161932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460022040 con 0x55fa7fac2d90 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4172161932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 msgr2=0x55fa7facd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7facd040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc460001eb0 tx=0x7fc4600098e0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4172161932 shutdown_connections 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fabd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/4172161932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7facd040 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4172161932 >> 172.21.15.33:0/4172161932 conn(0x55fa7fa8a950 msgr2=0x55fa7fabd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4172161932 shutdown_connections 2024-04-24T23:19:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.651+0000 7fc46e7ba740 1 -- 172.21.15.33:0/4172161932 wait complete. 2024-04-24T23:19:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 -- start start 2024-04-24T23:19:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 Processor -- start 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fad2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641a8a10 con 0x55fa7fac2d90 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641a9e80 con 0x55fa7fac3780 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641a90e0 con 0x55fa7fac88f0 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.652+0000 7fc46e7ba740 10 client.0 _ll_register_callbacks cb 0x7fc4600099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc469841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fad2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc469040640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc469040640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35802/0 (socket says 172.21.15.33:35802) 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc469841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fad2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46020/0 (socket says 172.21.15.33:46020) 2024-04-24T23:19:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc469040640 1 -- 172.21.15.33:0/3413371695 learned_addr learned my addr 172.21.15.33:0/3413371695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 -- 172.21.15.33:0/3413371695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 msgr2=0x55fa7fad2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 -- 172.21.15.33:0/3413371695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 msgr2=0x55fa7fad2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fad2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 -- 172.21.15.33:0/3413371695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7fad3aa0 con 0x55fa7fac88f0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46a042640 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad34b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005fd0 tx=0x7fc45c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=888169d7ae94365b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc44d7fa640 1 -- 172.21.15.33:0/3413371695 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c02e060 con 0x55fa7fac88f0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc44d7fa640 1 -- 172.21.15.33:0/3413371695 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc45c003c70 con 0x55fa7fac88f0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4600050c0 con 0x55fa7fac88f0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc44d7fa640 1 -- 172.21.15.33:0/3413371695 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc45c035050 con 0x55fa7fac88f0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46e7ba740 10 client.26485 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.653+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa7fb43910 con 0x55fa7fac88f0 2024-04-24T23:19:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc44d7fa640 1 -- 172.21.15.33:0/3413371695 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4600050c0 con 0x55fa7fac88f0 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc44d7fa640 1 -- 172.21.15.33:0/3413371695 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc45c029400 con 0x55fa7fac88f0 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc46e7ba740 10 client.26485 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc46e7ba740 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc46e7ba740 20 client.26485 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fb44370 0x55fa7fb46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc46e7ba740 4 client.26485 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa7fa33800 con 0x55fa7fb44370 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc469841640 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fb44370 0x55fa7fb46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc44d7fa640 10 client.26485 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.654+0000 7fc469841640 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fb44370 0x55fa7fb46760 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb6ad5ae585788d server_cookie=6a1044527babac63 in_seq=0 out_seq=0 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.655+0000 7fc44d7fa640 1 -- 172.21.15.33:0/3413371695 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa7fa33800 con 0x55fa7fb44370 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.655+0000 7fc44d7fa640 10 client.26485 handle_command_reply: tid=0 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.655+0000 7fc46e7ba740 1 client.26485 shutdown 2024-04-24T23:19:11.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.655+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 msgr2=0x55fa7fad34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.655+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad34b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005fd0 tx=0x7fc45c01c000 comp rx=0 tx=0).stop 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 shutdown_connections 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fa7fb44370 0x55fa7fb46760 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa7fac88f0 0x55fa7fad34b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa7fac3780 0x55fa7fad2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc44d7fa640 0 client.26485 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 --2- 172.21.15.33:0/3413371695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa7fac2d90 0x55fa7fad2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 >> 172.21.15.33:0/3413371695 conn(0x55fa7fa8a950 msgr2=0x55fa7fabd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 shutdown_connections 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 1 -- 172.21.15.33:0/3413371695 wait complete. 2024-04-24T23:19:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46e7ba740 20 client.26485 trim_cache size 0 max 16384 2024-04-24T23:19:11.664 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:19:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:19:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0b5bd33a-93c1-4831-b869-77d2b5619a14", 2024-04-24T23:19:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:19:11.664 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4500a2b90 msgr2=0x7fc4500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4500a2b90 0x7fc4500a5050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009fd0 tx=0x7fc45c009c40 comp rx=0 tx=0).stop 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 msgr2=0x7fc464198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464198010 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc454031440 tx=0x7fc454004270 comp rx=0 tx=0).stop 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 shutdown_connections 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc4500a2b90 0x7fc4500a5050 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc464102230 0x7fc46419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc464101870 0x7fc464198010 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 --2- 172.21.15.33:0/1590150527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464100680 0x7fc464197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.656+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 >> 172.21.15.33:0/1590150527 conn(0x7fc4640fbbf0 msgr2=0x7fc4640fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.657+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 shutdown_connections 2024-04-24T23:19:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.657+0000 7fc46baa5640 1 -- 172.21.15.33:0/1590150527 wait complete. 2024-04-24T23:19:11.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:19:11.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.795+0000 7f1223742640 1 -- 172.21.15.33:0/3876608203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 msgr2=0x7f121c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.795+0000 7f1223742640 1 --2- 172.21.15.33:0/3876608203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c101bc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1218009900 tx=0x7f121802a410 comp rx=0 tx=0).stop 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- 172.21.15.33:0/3876608203 shutdown_connections 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 --2- 172.21.15.33:0/3876608203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f121c103350 0x7f121c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 --2- 172.21.15.33:0/3876608203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f121c102990 0x7f121c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 --2- 172.21.15.33:0/3876608203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c101bc0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- 172.21.15.33:0/3876608203 >> 172.21.15.33:0/3876608203 conn(0x7f121c0fcd10 msgr2=0x7f121c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- 172.21.15.33:0/3876608203 shutdown_connections 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- 172.21.15.33:0/3876608203 wait complete. 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 Processor -- start 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- start start 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f121c102990 0x7f121c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f121c103350 0x7f121c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f121c10d420 con 0x7f121c1017a0 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f121c10d2a0 con 0x7f121c103350 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.796+0000 7f1223742640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f121c10d5a0 con 0x7f121c102990 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46032/0 (socket says 172.21.15.33:46032) 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 -- 172.21.15.33:0/2066924063 learned_addr learned my addr 172.21.15.33:0/2066924063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f1221cdf640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f121c103350 0x7f121c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f1220cdd640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f121c102990 0x7f121c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 -- 172.21.15.33:0/2066924063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f121c102990 msgr2=0x7f121c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f121c102990 0x7f121c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 -- 172.21.15.33:0/2066924063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f121c103350 msgr2=0x7f121c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f121c103350 0x7f121c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 -- 172.21.15.33:0/2066924063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c19f6e0 con 0x7f121c1017a0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12214de640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c197ac0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1218031610 tx=0x7f1218004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12127fc640 1 -- 172.21.15.33:0/2066924063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121803f030 con 0x7f121c1017a0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c19f970 con 0x7f121c1017a0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12127fc640 1 -- 172.21.15.33:0/2066924063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1218003ad0 con 0x7f121c1017a0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f12127fc640 1 -- 172.21.15.33:0/2066924063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218035070 con 0x7f121c1017a0 2024-04-24T23:19:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.797+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c19feb0 con 0x7f121c1017a0 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f12127fc640 1 -- 172.21.15.33:0/2066924063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1218042030 con 0x7f121c1017a0 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 Processor -- start 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 -- start start 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4e240 0x55bfc6d4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d53730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 0x55bfc6d56230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bfc6cb26b0 con 0x55bfc6d53da0 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bfc6c5a880 con 0x55bfc6d4ec30 2024-04-24T23:19:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f1226453740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bfc6cae670 con 0x55bfc6d4e240 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.800+0000 7f12127fc640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12080a2c10 0x7f12080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12214de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d53730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 0x55bfc6d56230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 0x55bfc6d56230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46034/0 (socket says 172.21.15.33:46034) 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 -- 172.21.15.33:0/3954648130 learned_addr learned my addr 172.21.15.33:0/3954648130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12214de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d53730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35822/0 (socket says 172.21.15.33:35822) 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1221cdf640 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4e240 0x55bfc6d4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1221cdf640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12080a2c10 0x7f12080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12127fc640 1 -- 172.21.15.33:0/2066924063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f12180ef8c0 con 0x7f121c1017a0 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 -- 172.21.15.33:0/3954648130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4e240 msgr2=0x55bfc6d4e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4e240 0x55bfc6d4e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 -- 172.21.15.33:0/3954648130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 msgr2=0x55bfc6d53730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d53730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 -- 172.21.15.33:0/3954648130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc6a907d0 con 0x55bfc6d53da0 2024-04-24T23:19:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12214de640 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d53730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1220cdd640 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 0x55bfc6d56230 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f120c009c90 tx=0x7f120c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e051e966c389b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f1221cdf640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12080a2c10 0x7f12080a50d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55bfc6d4f490 tx=0x7f1214009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12077fe640 1 -- 172.21.15.33:0/3954648130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c01e070 con 0x55bfc6d53da0 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12077fe640 1 -- 172.21.15.33:0/3954648130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c019b10 con 0x55bfc6d53da0 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.801+0000 7f12077fe640 1 -- 172.21.15.33:0/3954648130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00ea00 con 0x55bfc6d53da0 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.802+0000 7f1226453740 1 -- 172.21.15.33:0/3954648130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 msgr2=0x55bfc6d56230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.802+0000 7f1226453740 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 0x55bfc6d56230 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f120c009c90 tx=0x7f120c011810 comp rx=0 tx=0).stop 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- 172.21.15.33:0/3954648130 shutdown_connections 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d53da0 0x55bfc6d56230 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d53730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 --2- 172.21.15.33:0/3954648130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4e240 0x55bfc6d4e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- 172.21.15.33:0/3954648130 >> 172.21.15.33:0/3954648130 conn(0x55bfc6d498b0 msgr2=0x55bfc6d49cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- 172.21.15.33:0/3954648130 shutdown_connections 2024-04-24T23:19:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- 172.21.15.33:0/3954648130 wait complete. 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- start start 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 Processor -- start 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d58880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bfc6d56d50 con 0x55bfc6d4e240 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bfc6c6df50 con 0x55bfc6d4ec30 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bfc6d56ed0 con 0x55bfc6d5ddf0 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.803+0000 7f1226453740 10 client.0 _ll_register_callbacks cb 0x55bfc6ca9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d58880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d58880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46044/0 (socket says 172.21.15.33:46044) 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 -- 172.21.15.33:0/364684989 learned_addr learned my addr 172.21.15.33:0/364684989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1220cdd640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1221cdf640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 -- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d5ddf0 msgr2=0x55bfc6d601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 -- 172.21.15.33:0/364684989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 msgr2=0x55bfc6d58dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 -- 172.21.15.33:0/364684989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc69c63f0 con 0x55bfc6d4e240 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1220cdd640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f12214de640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d58880 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f12180ed030 tx=0x7f12180bb860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7703a5c7452042f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1204ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1204ff9640 1 -- 172.21.15.33:0/364684989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218049040 con 0x55bfc6d4e240 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c019b10 con 0x55bfc6d4e240 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1204ff9640 1 -- 172.21.15.33:0/364684989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1218003c80 con 0x55bfc6d4e240 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1204ff9640 1 -- 172.21.15.33:0/364684989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12180bbe40 con 0x55bfc6d4e240 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1226453740 10 client.16983 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:11.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.804+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfc69860f0 con 0x55bfc6d4e240 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1204ff9640 1 -- 172.21.15.33:0/364684989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c019b10 con 0x55bfc6d4e240 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1204ff9640 1 -- 172.21.15.33:0/364684989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f12180b84b0 con 0x55bfc6d4e240 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1226453740 10 client.16983 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1226453740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1226453740 20 client.16983 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1226453740 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6d64a60 0x55bfc6d66ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1226453740 4 client.16983 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfc6cbfbf0 con 0x55bfc6d64a60 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1220cdd640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6d64a60 0x55bfc6d66ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1204ff9640 10 client.16983 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.805+0000 7f1220cdd640 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6d64a60 0x55bfc6d66ef0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58d43e4249602e9 server_cookie=1e9a6dc7ded174db in_seq=0 out_seq=0 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.806+0000 7f1204ff9640 1 -- 172.21.15.33:0/364684989 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfc6cbfbf0 con 0x55bfc6d64a60 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.806+0000 7f1204ff9640 10 client.16983 handle_command_reply: tid=0 2024-04-24T23:19:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.806+0000 7f1226453740 1 client.16983 shutdown 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.806+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 msgr2=0x55bfc6d58880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.806+0000 7f1226453740 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d58880 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f12180ed030 tx=0x7f12180bb860 comp rx=0 tx=0).stop 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 shutdown_connections 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6d64a60 0x55bfc6d66ef0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 --2- 172.21.15.33:0/364684989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d58880 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1204ff9640 0 client.16983 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 >> 172.21.15.33:0/364684989 conn(0x55bfc6d498b0 msgr2=0x55bfc6d48af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 shutdown_connections 2024-04-24T23:19:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 1 -- 172.21.15.33:0/364684989 wait complete. 2024-04-24T23:19:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.807+0000 7f1226453740 20 client.16983 trim_cache size 0 max 16384 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 Processor -- start 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 -- start start 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4e240 0x55bfc6d44980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d4c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1acc60 con 0x55bfc6d5ddf0 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1aa530 con 0x55bfc6d4e240 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1226453740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1a8380 con 0x55bfc6d4ec30 2024-04-24T23:19:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1221cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d4c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1220cdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1221cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d4c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46060/0 (socket says 172.21.15.33:46060) 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1221cdf640 1 -- 172.21.15.33:0/106693100 learned_addr learned my addr 172.21.15.33:0/106693100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f12214de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4e240 0x55bfc6d44980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1220cdd640 1 -- 172.21.15.33:0/106693100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4e240 msgr2=0x55bfc6d44980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1220cdd640 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4e240 0x55bfc6d44980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1220cdd640 1 -- 172.21.15.33:0/106693100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 msgr2=0x55bfc6d4c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1220cdd640 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d4c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1220cdd640 1 -- 172.21.15.33:0/106693100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc6d50920 con 0x55bfc6d4ec30 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.828+0000 7f1221cdf640 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d4c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.829+0000 7f1220cdd640 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f120c008000 tx=0x7f120c0093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5877ad75d05ce774 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.829+0000 7f1206ffd640 1 -- 172.21.15.33:0/106693100 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c01e030 con 0x55bfc6d4ec30 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.829+0000 7f1206ffd640 1 -- 172.21.15.33:0/106693100 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c02b070 con 0x55bfc6d4ec30 2024-04-24T23:19:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.829+0000 7f1206ffd640 1 -- 172.21.15.33:0/106693100 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c02b220 con 0x55bfc6d4ec30 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- 172.21.15.33:0/106693100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 msgr2=0x55bfc6d58ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f120c008000 tx=0x7f120c0093f0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- 172.21.15.33:0/106693100 shutdown_connections 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d4c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d58ba0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- 172.21.15.33:0/106693100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d4e240 0x55bfc6d44980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- 172.21.15.33:0/106693100 >> 172.21.15.33:0/106693100 conn(0x55bfc6d498b0 msgr2=0x55bfc6d58490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- 172.21.15.33:0/106693100 shutdown_connections 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- 172.21.15.33:0/106693100 wait complete. 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- start start 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 Processor -- start 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1a89f0 con 0x55bfc6d4e240 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.830+0000 7f1226453740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1a9e60 con 0x55bfc6d5ddf0 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1226453740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1a90c0 con 0x55bfc6d4ec30 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1226453740 10 client.0 _ll_register_callbacks cb 0x7f1218031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d75e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46072/0 (socket says 172.21.15.33:46072) 2024-04-24T23:19:11.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1221cdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 -- 172.21.15.33:0/3447082943 learned_addr learned my addr 172.21.15.33:0/3447082943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1220cdd640 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 -- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 msgr2=0x55bfc6d76350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d76350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 -- 172.21.15.33:0/3447082943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d5ddf0 msgr2=0x55bfc6d772a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d772a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 -- 172.21.15.33:0/3447082943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc6d77890 con 0x55bfc6d4e240 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f12214de640 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d75e10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1218031b80 tx=0x7f12180d4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c32c74ba2a9de8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1204ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1204ff9640 1 -- 172.21.15.33:0/3447082943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218049040 con 0x55bfc6d4e240 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1204ff9640 1 -- 172.21.15.33:0/3447082943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1218031dd0 con 0x55bfc6d4e240 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c02b070 con 0x55bfc6d4e240 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1204ff9640 1 -- 172.21.15.33:0/3447082943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12180c0140 con 0x55bfc6d4e240 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1226453740 10 client.16989 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.831+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfc6dcf280 con 0x55bfc6d4e240 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1204ff9640 1 -- 172.21.15.33:0/3447082943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c02b070 con 0x55bfc6d4e240 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1204ff9640 1 -- 172.21.15.33:0/3447082943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f12180bb290 con 0x55bfc6d4e240 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1226453740 10 client.16989 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1226453740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1226453740 20 client.16989 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1226453740 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6dcf8e0 0x55bfc6dd1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1226453740 4 client.16989 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bfc6cb9f40 con 0x55bfc6dcf8e0 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1220cdd640 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6dcf8e0 0x55bfc6dd1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1220cdd640 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6dcf8e0 0x55bfc6dd1cf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7645aca9d780a8b4 server_cookie=b41d0161e81a5245 in_seq=0 out_seq=0 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.832+0000 7f1204ff9640 10 client.16989 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1204ff9640 1 -- 172.21.15.33:0/3447082943 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bfc6cb9f40 con 0x55bfc6dcf8e0 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1204ff9640 10 client.16989 handle_command_reply: tid=0 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 client.16989 shutdown 2024-04-24T23:19:11.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 msgr2=0x55bfc6d75e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d75e10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1218031b80 tx=0x7f12180d4f90 comp rx=0 tx=0).stop 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 shutdown_connections 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bfc6dcf8e0 0x55bfc6dd1cf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfc6d5ddf0 0x55bfc6d772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1204ff9640 0 client.16989 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfc6d4ec30 0x55bfc6d76350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 --2- 172.21.15.33:0/3447082943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfc6d4e240 0x55bfc6d75e10 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 >> 172.21.15.33:0/3447082943 conn(0x55bfc6d498b0 msgr2=0x55bfc6d4c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 shutdown_connections 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 1 -- 172.21.15.33:0/3447082943 wait complete. 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.833+0000 7f1226453740 20 client.16989 trim_cache size 0 max 16384 2024-04-24T23:19:11.841 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "0b5bd33a-93c1-4831-b869-77d2b5619a14": { 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0b5bd33a-93c1-4831-b869-77d2b5619a14", 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:19:11.842 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12080a2c10 msgr2=0x7f12080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12080a2c10 0x7f12080a50d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55bfc6d4f490 tx=0x7f1214009500 comp rx=0 tx=0).stop 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 msgr2=0x7f121c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c197ac0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1218031610 tx=0x7f1218004270 comp rx=0 tx=0).stop 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 shutdown_connections 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f12080a2c10 0x7f12080a50d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f121c103350 0x7f121c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f121c102990 0x7f121c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 --2- 172.21.15.33:0/2066924063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1017a0 0x7f121c197ac0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 >> 172.21.15.33:0/2066924063 conn(0x7f121c0fcd10 msgr2=0x7f121c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 shutdown_connections 2024-04-24T23:19:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:11.834+0000 7f1223742640 1 -- 172.21.15.33:0/2066924063 wait complete. 2024-04-24T23:19:11.846 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:11 smithi033 ceph-mon[29201]: pgmap v566: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-04-24T23:19:11.847 INFO:tasks.cephfs.filesystem:scrub status for tag:0b5bd33a-93c1-4831-b869-77d2b5619a14 - {'path': '/', 'tag': '0b5bd33a-93c1-4831-b869-77d2b5619a14', 'options': 'recursive,force'} 2024-04-24T23:19:11.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:11 smithi038 ceph-mon[32450]: pgmap v566: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-04-24T23:19:11.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:11 smithi117 ceph-mon[32529]: pgmap v566: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-04-24T23:19:12.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 760577 12.73 MB/sec execute 197 sec latency 21.321 ms 2024-04-24T23:19:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:12 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:19:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:19:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:19:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:12 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:19:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:19:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:19:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:12 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:19:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:19:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:19:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:12.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:13.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 763192 12.74 MB/sec execute 198 sec latency 78.723 ms 2024-04-24T23:19:13.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:13 smithi038 ceph-mon[32450]: pgmap v567: 129 pgs: 129 active+clean; 280 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-04-24T23:19:13.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:13 smithi117 ceph-mon[32529]: pgmap v567: 129 pgs: 129 active+clean; 280 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-04-24T23:19:13.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:13 smithi033 ceph-mon[29201]: pgmap v567: 129 pgs: 129 active+clean; 280 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-04-24T23:19:14.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 764630 12.71 MB/sec execute 199 sec latency 24.005 ms 2024-04-24T23:19:15.449 INFO:tasks.workunit.client.0.smithi033.stdout: 1 767371 12.73 MB/sec execute 200 sec latency 13.870 ms 2024-04-24T23:19:15.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:15 smithi038 ceph-mon[32450]: pgmap v568: 129 pgs: 129 active+clean; 280 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-04-24T23:19:15.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:15 smithi117 ceph-mon[32529]: pgmap v568: 129 pgs: 129 active+clean; 280 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-04-24T23:19:15.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:15 smithi033 ceph-mon[29201]: pgmap v568: 129 pgs: 129 active+clean; 280 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-04-24T23:19:16.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 769978 12.73 MB/sec execute 201 sec latency 23.824 ms 2024-04-24T23:19:17.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 772158 12.72 MB/sec execute 202 sec latency 12.243 ms 2024-04-24T23:19:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:17 smithi038 ceph-mon[32450]: pgmap v569: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-04-24T23:19:17.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:17 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:19:17.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:17 smithi117 ceph-mon[32529]: pgmap v569: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-04-24T23:19:17.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:17 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:19:17.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:17 smithi033 ceph-mon[29201]: pgmap v569: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-04-24T23:19:17.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:17 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:19:18.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 774464 12.73 MB/sec execute 203 sec latency 70.915 ms 2024-04-24T23:19:18.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:18 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:19:18.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:18 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:19:18.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:18 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:19:19.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 776967 12.73 MB/sec execute 204 sec latency 24.922 ms 2024-04-24T23:19:19.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:19 smithi038 ceph-mon[32450]: pgmap v570: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-04-24T23:19:19.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:19 smithi117 ceph-mon[32529]: pgmap v570: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-04-24T23:19:19.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:19 smithi033 ceph-mon[29201]: pgmap v570: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-04-24T23:19:20.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 778778 12.71 MB/sec execute 205 sec latency 61.004 ms 2024-04-24T23:19:21.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 781646 12.73 MB/sec execute 206 sec latency 14.861 ms 2024-04-24T23:19:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:21 smithi038 ceph-mon[32450]: pgmap v571: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2024-04-24T23:19:21.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:21 smithi117 ceph-mon[32529]: pgmap v571: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2024-04-24T23:19:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:21 smithi033 ceph-mon[29201]: pgmap v571: 129 pgs: 129 active+clean; 283 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2024-04-24T23:19:22.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 784345 12.73 MB/sec execute 207 sec latency 18.293 ms 2024-04-24T23:19:23.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 785674 12.70 MB/sec execute 208 sec latency 118.399 ms 2024-04-24T23:19:23.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:23 smithi038 ceph-mon[32450]: pgmap v572: 129 pgs: 129 active+clean; 284 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 280 op/s 2024-04-24T23:19:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:23 smithi117 ceph-mon[32529]: pgmap v572: 129 pgs: 129 active+clean; 284 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 280 op/s 2024-04-24T23:19:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:23 smithi033 ceph-mon[29201]: pgmap v572: 129 pgs: 129 active+clean; 284 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 280 op/s 2024-04-24T23:19:24.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 788489 12.73 MB/sec execute 209 sec latency 36.827 ms 2024-04-24T23:19:25.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 790825 12.71 MB/sec execute 210 sec latency 15.898 ms 2024-04-24T23:19:25.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:25 smithi038 ceph-mon[32450]: pgmap v573: 129 pgs: 129 active+clean; 286 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-04-24T23:19:25.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:25 smithi117 ceph-mon[32529]: pgmap v573: 129 pgs: 129 active+clean; 286 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-04-24T23:19:25.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:25 smithi033 ceph-mon[29201]: pgmap v573: 129 pgs: 129 active+clean; 286 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-04-24T23:19:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 792841 12.71 MB/sec execute 211 sec latency 12.259 ms 2024-04-24T23:19:27.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 795688 12.73 MB/sec execute 212 sec latency 23.270 ms 2024-04-24T23:19:27.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:27 smithi038 ceph-mon[32450]: pgmap v574: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-04-24T23:19:27.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:27 smithi117 ceph-mon[32529]: pgmap v574: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-04-24T23:19:27.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:27 smithi033 ceph-mon[29201]: pgmap v574: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-04-24T23:19:28.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 798083 12.72 MB/sec execute 213 sec latency 50.687 ms 2024-04-24T23:19:29.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 799652 12.70 MB/sec execute 214 sec latency 47.276 ms 2024-04-24T23:19:29.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:29 smithi038 ceph-mon[32450]: pgmap v575: 129 pgs: 129 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-24T23:19:29.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:29 smithi117 ceph-mon[32529]: pgmap v575: 129 pgs: 129 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-24T23:19:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:29 smithi033 ceph-mon[29201]: pgmap v575: 129 pgs: 129 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-24T23:19:30.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 802647 12.73 MB/sec execute 215 sec latency 15.951 ms 2024-04-24T23:19:31.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 805012 12.71 MB/sec execute 216 sec latency 13.296 ms 2024-04-24T23:19:31.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:31 smithi038 ceph-mon[32450]: pgmap v576: 129 pgs: 129 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-04-24T23:19:31.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:31 smithi117 ceph-mon[32529]: pgmap v576: 129 pgs: 129 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-04-24T23:19:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:31 smithi033 ceph-mon[29201]: pgmap v576: 129 pgs: 129 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-04-24T23:19:32.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 807124 12.71 MB/sec execute 217 sec latency 15.899 ms 2024-04-24T23:19:33.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 809960 12.73 MB/sec execute 218 sec latency 15.720 ms 2024-04-24T23:19:33.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:33 smithi038 ceph-mon[32450]: pgmap v577: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-04-24T23:19:33.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:33 smithi117 ceph-mon[32529]: pgmap v577: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-04-24T23:19:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:33 smithi033 ceph-mon[29201]: pgmap v577: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-04-24T23:19:34.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 811927 12.70 MB/sec execute 219 sec latency 35.850 ms 2024-04-24T23:19:35.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 813958 12.70 MB/sec execute 220 sec latency 12.813 ms 2024-04-24T23:19:35.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:35 smithi038 ceph-mon[32450]: pgmap v578: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:19:35.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:35 smithi117 ceph-mon[32529]: pgmap v578: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:19:35.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:35 smithi033 ceph-mon[29201]: pgmap v578: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:19:36.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 816837 12.73 MB/sec execute 221 sec latency 12.725 ms 2024-04-24T23:19:37.450 INFO:tasks.workunit.client.0.smithi033.stdout: 1 819370 12.72 MB/sec execute 222 sec latency 17.602 ms 2024-04-24T23:19:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:37 smithi038 ceph-mon[32450]: pgmap v579: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-04-24T23:19:37.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:37 smithi117 ceph-mon[32529]: pgmap v579: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-04-24T23:19:37.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:37 smithi033 ceph-mon[29201]: pgmap v579: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-04-24T23:19:38.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 821175 12.71 MB/sec execute 223 sec latency 17.891 ms 2024-04-24T23:19:39.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 823648 12.72 MB/sec execute 224 sec latency 71.576 ms 2024-04-24T23:19:39.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:39 smithi038 ceph-mon[32450]: pgmap v580: 129 pgs: 129 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-04-24T23:19:39.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:39 smithi117 ceph-mon[32529]: pgmap v580: 129 pgs: 129 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-04-24T23:19:39.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:39 smithi033 ceph-mon[29201]: pgmap v580: 129 pgs: 129 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-04-24T23:19:40.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 824935 12.68 MB/sec execute 225 sec latency 79.211 ms 2024-04-24T23:19:41.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 827768 12.70 MB/sec execute 226 sec latency 11.204 ms 2024-04-24T23:19:41.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:19:41.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:19:41.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:41 smithi038 ceph-mon[32450]: pgmap v581: 129 pgs: 129 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-04-24T23:19:41.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:41 smithi117 ceph-mon[32529]: pgmap v581: 129 pgs: 129 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-04-24T23:19:41.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:41 smithi033 ceph-mon[29201]: pgmap v581: 129 pgs: 129 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-04-24T23:19:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 -- 172.21.15.33:0/111059402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a41006a0 msgr2=0x7f30a4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 --2- 172.21.15.33:0/111059402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a41006a0 0x7f30a4100ac0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3094009900 tx=0x7f309402a410 comp rx=0 tx=0).stop 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 -- 172.21.15.33:0/111059402 shutdown_connections 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 --2- 172.21.15.33:0/111059402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30a4102250 0x7f30a4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 --2- 172.21.15.33:0/111059402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a4101890 0x7f30a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 --2- 172.21.15.33:0/111059402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a41006a0 0x7f30a4100ac0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 -- 172.21.15.33:0/111059402 >> 172.21.15.33:0/111059402 conn(0x7f30a40fbc30 msgr2=0x7f30a40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 -- 172.21.15.33:0/111059402 shutdown_connections 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 -- 172.21.15.33:0/111059402 wait complete. 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 Processor -- start 2024-04-24T23:19:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.963+0000 7f30a9a41640 1 -- start start 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a9a41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30a41006a0 0x7f30a410c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a9a41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 0x7f30a410b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a9a41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a4102250 0x7f30a410b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a9a41640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30a410d410 con 0x7f30a4101890 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a9a41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30a410d290 con 0x7f30a4102250 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a9a41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30a410d590 con 0x7f30a41006a0 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 0x7f30a410b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30a41006a0 0x7f30a410c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 0x7f30a410b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36200/0 (socket says 172.21.15.33:36200) 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 -- 172.21.15.33:0/972748046 learned_addr learned my addr 172.21.15.33:0/972748046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a37fe640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a4102250 0x7f30a410b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 -- 172.21.15.33:0/972748046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30a41006a0 msgr2=0x7f30a410c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30a41006a0 0x7f30a410c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 -- 172.21.15.33:0/972748046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a4102250 msgr2=0x7f30a410b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a4102250 0x7f30a410b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 -- 172.21.15.33:0/972748046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a410be00 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f309bfff640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 0x7f30a410b030 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ea60 tx=0x7f308c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.964+0000 7f30a8a3f640 1 -- 172.21.15.33:0/972748046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00cde0 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.965+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41a1be0 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.965+0000 7f30a8a3f640 1 -- 172.21.15.33:0/972748046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c0035f0 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.965+0000 7f30a8a3f640 1 -- 172.21.15.33:0/972748046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c010670 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.965+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a2130 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30a8a3f640 1 -- 172.21.15.33:0/972748046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f308c0037a0 con 0x7f30a4101890 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 Processor -- start 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 -- start start 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30a8a3f640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30900a2b90 0x7f30900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fba352d6b0 con 0x55fba35ceda0 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fba34d5880 con 0x55fba35c9c30 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.967+0000 7f30ac820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fba3529670 con 0x55fba35c9240 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a37fe640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30900a2b90 0x7f30900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46642/0 (socket says 172.21.15.33:46642) 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 -- 172.21.15.33:0/601068099 learned_addr learned my addr 172.21.15.33:0/601068099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f309bfff640 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a8a3f640 1 -- 172.21.15.33:0/972748046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f308c0cf690 con 0x7f30a4101890 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 -- 172.21.15.33:0/601068099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 msgr2=0x55fba35ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 -- 172.21.15.33:0/601068099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 msgr2=0x55fba35d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 -- 172.21.15.33:0/601068099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba330b7d0 con 0x55fba35c9240 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a37fe640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30900a2b90 0x7f30900a5050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f309c005fd0 tx=0x7f309c009e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35c9660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3094031610 tx=0x7f3094005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=681d39ef8d1a00c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f3099ffb640 1 -- 172.21.15.33:0/601068099 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094040070 con 0x55fba35c9240 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f3099ffb640 1 -- 172.21.15.33:0/601068099 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094004030 con 0x55fba35c9240 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.968+0000 7f3099ffb640 1 -- 172.21.15.33:0/601068099 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094035040 con 0x55fba35c9240 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 -- 172.21.15.33:0/601068099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 msgr2=0x55fba35c9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35c9660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3094031610 tx=0x7f3094005de0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 -- 172.21.15.33:0/601068099 shutdown_connections 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 --2- 172.21.15.33:0/601068099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35c9660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 -- 172.21.15.33:0/601068099 >> 172.21.15.33:0/601068099 conn(0x55fba35c48b0 msgr2=0x55fba35c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 -- 172.21.15.33:0/601068099 shutdown_connections 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 -- 172.21.15.33:0/601068099 wait complete. 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.969+0000 7f30ac820740 1 -- start start 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 Processor -- start 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9c30 0x55fba35d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35ceda0 0x55fba35dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fba35d1cf0 con 0x55fba35c9240 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fba34e8f50 con 0x55fba35ceda0 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fba35d1e70 con 0x55fba35c9c30 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30ac820740 10 client.0 _ll_register_callbacks cb 0x55fba3524000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35d3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36208/0 (socket says 172.21.15.33:36208) 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3829354061 learned_addr learned my addr 172.21.15.33:0/3829354061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f309bfff640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9c30 0x55fba35d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a37fe640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35ceda0 0x55fba35dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9c30 msgr2=0x55fba35d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9c30 0x55fba35d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3829354061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35ceda0 msgr2=0x55fba35dab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35ceda0 0x55fba35dab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3829354061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba32413f0 con 0x55fba35c9240 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f309bfff640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9c30 0x55fba35d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35d3660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a5510b3044c441 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.970+0000 7f3082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f3082ffd640 1 -- 172.21.15.33:0/3829354061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309404c070 con 0x55fba35c9240 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f3082ffd640 1 -- 172.21.15.33:0/3829354061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30940043e0 con 0x55fba35c9240 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094004030 con 0x55fba35c9240 2024-04-24T23:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f3082ffd640 1 -- 172.21.15.33:0/3829354061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3094058050 con 0x55fba35c9240 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 10 client.17001 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fba32010f0 con 0x55fba35c9240 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f3082ffd640 1 -- 172.21.15.33:0/3829354061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094004030 con 0x55fba35c9240 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f3082ffd640 1 -- 172.21.15.33:0/3829354061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30940035f0 con 0x55fba35c9240 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 10 client.17001 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 20 client.17001 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba35df060 0x55fba35e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 4 client.17001 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fba353abf0 con 0x55fba35df060 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.971+0000 7f309bfff640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba35df060 0x55fba35e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.972+0000 7f3082ffd640 10 client.17001 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.972+0000 7f309bfff640 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba35df060 0x55fba35e1450 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee9d483575a9020 server_cookie=3188ed3262f61e35 in_seq=0 out_seq=0 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.972+0000 7f3082ffd640 1 -- 172.21.15.33:0/3829354061 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fba353abf0 con 0x55fba35df060 2024-04-24T23:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.972+0000 7f3082ffd640 10 client.17001 handle_command_reply: tid=0 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.972+0000 7f30ac820740 1 client.17001 shutdown 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 msgr2=0x55fba35d3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35d3660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094009590 comp rx=0 tx=0).stop 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 shutdown_connections 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba35df060 0x55fba35e1450 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35ceda0 0x55fba35dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f3082ffd640 0 client.17001 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9c30 0x55fba35d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 --2- 172.21.15.33:0/3829354061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35d3660 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 >> 172.21.15.33:0/3829354061 conn(0x55fba35c48b0 msgr2=0x55fba35c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 shutdown_connections 2024-04-24T23:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 1 -- 172.21.15.33:0/3829354061 wait complete. 2024-04-24T23:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.973+0000 7f30ac820740 20 client.17001 trim_cache size 0 max 16384 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 Processor -- start 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 -- start start 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35ceda0 0x55fba35c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41aeeb0 con 0x55fba35c9240 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ac780 con 0x55fba35c9c30 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30ac820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41aa5d0 con 0x55fba35ceda0 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36212/0 (socket says 172.21.15.33:36212) 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30a2ffd640 1 -- 172.21.15.33:0/2229823828 learned_addr learned my addr 172.21.15.33:0/2229823828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f30a37fe640 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35ceda0 0x55fba35c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.992+0000 7f309bfff640 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30a2ffd640 1 -- 172.21.15.33:0/2229823828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35ceda0 msgr2=0x55fba35c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35ceda0 0x55fba35c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30a2ffd640 1 -- 172.21.15.33:0/2229823828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 msgr2=0x55fba35dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30a2ffd640 1 -- 172.21.15.33:0/2229823828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba35ce370 con 0x55fba35c9240 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35c7d40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f309402a930 tx=0x7f3094003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6810a2f31833263b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30997fa640 1 -- 172.21.15.33:0/2229823828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309404c070 con 0x55fba35c9240 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30997fa640 1 -- 172.21.15.33:0/2229823828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094031d90 con 0x55fba35c9240 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.993+0000 7f30997fa640 1 -- 172.21.15.33:0/2229823828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094040030 con 0x55fba35c9240 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 -- 172.21.15.33:0/2229823828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 msgr2=0x55fba35c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35c7d40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f309402a930 tx=0x7f3094003780 comp rx=0 tx=0).stop 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 -- 172.21.15.33:0/2229823828 shutdown_connections 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35ceda0 0x55fba35c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 --2- 172.21.15.33:0/2229823828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35c9240 0x55fba35c7d40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 -- 172.21.15.33:0/2229823828 >> 172.21.15.33:0/2229823828 conn(0x55fba35c48b0 msgr2=0x55fba35c7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 -- 172.21.15.33:0/2229823828 shutdown_connections 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 -- 172.21.15.33:0/2229823828 wait complete. 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 -- start start 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.994+0000 7f30ac820740 1 Processor -- start 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41aac40 con 0x55fba35ceda0 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ac0b0 con 0x55fba35c9c30 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ab310 con 0x55fba35c9240 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30ac820740 10 client.0 _ll_register_callbacks cb 0x7f309c010670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f309bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35d8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46674/0 (socket says 172.21.15.33:46674) 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3184485710 learned_addr learned my addr 172.21.15.33:0/3184485710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3184485710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 msgr2=0x55fba35d8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35d8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3184485710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 msgr2=0x55fba35d9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 -- 172.21.15.33:0/3184485710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba35da190 con 0x55fba35c9240 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f30a2ffd640 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35d8650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4e951311029cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f3082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.995+0000 7f3082ffd640 1 -- 172.21.15.33:0/3184485710 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094040070 con 0x55fba35c9240 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094031d90 con 0x55fba35c9240 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f3082ffd640 1 -- 172.21.15.33:0/3184485710 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30940316b0 con 0x55fba35c9240 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f3082ffd640 1 -- 172.21.15.33:0/3184485710 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f309406d050 con 0x55fba35c9240 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 10 client.26521 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fba36457a0 con 0x55fba35c9240 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f3082ffd640 1 -- 172.21.15.33:0/3184485710 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094031d90 con 0x55fba35c9240 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f3082ffd640 1 -- 172.21.15.33:0/3184485710 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f309403c9b0 con 0x55fba35c9240 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 10 client.26521 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 20 client.26521 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba3646090 0x55fba36484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 4 client.26521 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fba3534f40 con 0x55fba3646090 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.996+0000 7f309bfff640 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba3646090 0x55fba36484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f309bfff640 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba3646090 0x55fba36484b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f900cc7b62f2060 server_cookie=c1adf12efeb61bbe in_seq=0 out_seq=0 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f3082ffd640 10 client.26521 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f3082ffd640 1 -- 172.21.15.33:0/3184485710 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fba3534f40 con 0x55fba3646090 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f3082ffd640 10 client.26521 handle_command_reply: tid=0 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 client.26521 shutdown 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 msgr2=0x55fba35d8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35d8650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094005730 comp rx=0 tx=0).stop 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 shutdown_connections 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fba3646090 0x55fba36484b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fba35ceda0 0x55fba35d9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fba35c9c30 0x55fba35d8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f3082ffd640 0 client.26521 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 --2- 172.21.15.33:0/3184485710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fba35c9240 0x55fba35d8650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 >> 172.21.15.33:0/3184485710 conn(0x55fba35c48b0 msgr2=0x55fba35c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 shutdown_connections 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 1 -- 172.21.15.33:0/3184485710 wait complete. 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.997+0000 7f30ac820740 20 client.26521 trim_cache size 0 max 16384 2024-04-24T23:19:42.008 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:19:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:19:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:19:42.008 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30900a2b90 msgr2=0x7f30900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30900a2b90 0x7f30900a5050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f309c005fd0 tx=0x7f309c009e70 comp rx=0 tx=0).stop 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 msgr2=0x7f30a410b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 0x7f30a410b030 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ea60 tx=0x7f308c00ef20 comp rx=0 tx=0).stop 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 shutdown_connections 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30900a2b90 0x7f30900a5050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a4102250 0x7f30a410b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a4101890 0x7f30a410b030 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 --2- 172.21.15.33:0/972748046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30a41006a0 0x7f30a410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 >> 172.21.15.33:0/972748046 conn(0x7f30a40fbc30 msgr2=0x7f30a4104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 shutdown_connections 2024-04-24T23:19:42.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:41.998+0000 7f30a9a41640 1 -- 172.21.15.33:0/972748046 wait complete. 2024-04-24T23:19:42.012 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:19:42.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:19:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 -- 172.21.15.33:0/3518609529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca0104bd0 msgr2=0x7feca0106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 --2- 172.21.15.33:0/3518609529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca0104bd0 0x7feca0106fc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec94009900 tx=0x7fec9402a410 comp rx=0 tx=0).stop 2024-04-24T23:19:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 -- 172.21.15.33:0/3518609529 shutdown_connections 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 --2- 172.21.15.33:0/3518609529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feca0107500 0x7feca01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 --2- 172.21.15.33:0/3518609529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca0104bd0 0x7feca0106fc0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 --2- 172.21.15.33:0/3518609529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca01022a0 0x7feca0104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 -- 172.21.15.33:0/3518609529 >> 172.21.15.33:0/3518609529 conn(0x7feca00fbbf0 msgr2=0x7feca00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 -- 172.21.15.33:0/3518609529 shutdown_connections 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 -- 172.21.15.33:0/3518609529 wait complete. 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.116+0000 7feca5672640 1 Processor -- start 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 -- start start 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 0x7feca0199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feca0104bd0 0x7feca019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca0107500 0x7feca01a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feca010d3d0 con 0x7feca01022a0 2024-04-24T23:19:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feca010d250 con 0x7feca0104bd0 2024-04-24T23:19:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7feca5672640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feca010d550 con 0x7feca0107500 2024-04-24T23:19:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 0x7feca0199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 0x7feca0199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36244/0 (socket says 172.21.15.33:36244) 2024-04-24T23:19:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 -- 172.21.15.33:0/1941877283 learned_addr learned my addr 172.21.15.33:0/1941877283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9f7fe640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca0107500 0x7feca01a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9e7fc640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feca0104bd0 0x7feca019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 -- 172.21.15.33:0/1941877283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca0107500 msgr2=0x7feca01a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca0107500 0x7feca01a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 -- 172.21.15.33:0/1941877283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feca0104bd0 msgr2=0x7feca019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feca0104bd0 0x7feca019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.117+0000 7fec9effd640 1 -- 172.21.15.33:0/1941877283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca01a18c0 con 0x7feca01022a0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.118+0000 7fec9effd640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 0x7feca0199c80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fec90009d50 tx=0x7fec9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.118+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90003b70 con 0x7feca01022a0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.118+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca01a1bb0 con 0x7feca01022a0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.118+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90003d10 con 0x7feca01022a0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.118+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90010a20 con 0x7feca01022a0 2024-04-24T23:19:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.118+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca01a20d0 con 0x7feca01022a0 2024-04-24T23:19:42.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.119+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca0108ff0 con 0x7feca01022a0 2024-04-24T23:19:42.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.120+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fec90010bc0 con 0x7feca01022a0 2024-04-24T23:19:42.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.121+0000 7fec87fff640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fec880a2b90 0x7fec880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.121+0000 7fec9e7fc640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fec880a2b90 0x7fec880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.121+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fec900cfaf0 con 0x7feca01022a0 2024-04-24T23:19:42.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.121+0000 7fec9e7fc640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fec880a2b90 0x7fec880a5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec94005ab0 tx=0x7fec94000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.125+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec9009c750 con 0x7feca01022a0 2024-04-24T23:19:42.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.356+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feca002d3c0 con 0x7feca01022a0 2024-04-24T23:19:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.357+0000 7fec87fff640 1 -- 172.21.15.33:0/1941877283 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fec9009c050 con 0x7feca01022a0 2024-04-24T23:19:42.359 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:19:42.359 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:19:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:19:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fec880a2b90 msgr2=0x7fec880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fec880a2b90 0x7fec880a5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec94005ab0 tx=0x7fec94000bc0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 msgr2=0x7feca0199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 0x7feca0199c80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fec90009d50 tx=0x7fec9000c6a0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 shutdown_connections 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fec880a2b90 0x7fec880a5050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca0107500 0x7feca01a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feca0104bd0 0x7feca019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 --2- 172.21.15.33:0/1941877283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feca01022a0 0x7feca0199c80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 >> 172.21.15.33:0/1941877283 conn(0x7feca00fbbf0 msgr2=0x7feca01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 shutdown_connections 2024-04-24T23:19:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.359+0000 7feca5672640 1 -- 172.21.15.33:0/1941877283 wait complete. 2024-04-24T23:19:42.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:19:42.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 830396 12.70 MB/sec execute 227 sec latency 19.327 ms 2024-04-24T23:19:42.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 -- 172.21.15.33:0/3731706462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c102200 msgr2=0x7fd84c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 --2- 172.21.15.33:0/3731706462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c102200 0x7fd84c1045f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009ca0 tx=0x7fd83c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 -- 172.21.15.33:0/3731706462 shutdown_connections 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 --2- 172.21.15.33:0/3731706462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd84c107460 0x7fd84c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 --2- 172.21.15.33:0/3731706462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c104b30 0x7fd84c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 --2- 172.21.15.33:0/3731706462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c102200 0x7fd84c1045f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 -- 172.21.15.33:0/3731706462 >> 172.21.15.33:0/3731706462 conn(0x7fd84c0fbbf0 msgr2=0x7fd84c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 -- 172.21.15.33:0/3731706462 shutdown_connections 2024-04-24T23:19:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.478+0000 7fd850d89640 1 -- 172.21.15.33:0/3731706462 wait complete. 2024-04-24T23:19:42.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 Processor -- start 2024-04-24T23:19:42.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 -- start start 2024-04-24T23:19:42.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c102200 0x7fd84c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 0x7fd84c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd84c107460 0x7fd84c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10d420 con 0x7fd84c104b30 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10d2a0 con 0x7fd84c102200 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd850d89640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10d5a0 con 0x7fd84c107460 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 0x7fd84c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c102200 0x7fd84c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 0x7fd84c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36254/0 (socket says 172.21.15.33:36254) 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 -- 172.21.15.33:0/526431951 learned_addr learned my addr 172.21.15.33:0/526431951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd84c107460 0x7fd84c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 -- 172.21.15.33:0/526431951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd84c107460 msgr2=0x7fd84c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd84c107460 0x7fd84c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 -- 172.21.15.33:0/526431951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c102200 msgr2=0x7fd84c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c102200 0x7fd84c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd849d9b640 1 -- 172.21.15.33:0/526431951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c19f710 con 0x7fd84c104b30 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.479+0000 7fd84a59c640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c102200 0x7fd84c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.480+0000 7fd849d9b640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 0x7fd84c198030 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9c0 tx=0x7fd83400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.480+0000 7fd8437fe640 1 -- 172.21.15.33:0/526431951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83400cd80 con 0x7fd84c104b30 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.480+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c19fa00 con 0x7fd84c104b30 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.480+0000 7fd8437fe640 1 -- 172.21.15.33:0/526431951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd834005500 con 0x7fd84c104b30 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.480+0000 7fd8437fe640 1 -- 172.21.15.33:0/526431951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834010690 con 0x7fd84c104b30 2024-04-24T23:19:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.480+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c19ffa0 con 0x7fd84c104b30 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.482+0000 7fd8437fe640 1 -- 172.21.15.33:0/526431951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd8340046c0 con 0x7fd84c104b30 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.482+0000 7fd853b71740 1 Processor -- start 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 -- start start 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c5542650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5547720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd8437fe640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8380a2b90 0x7fd8380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c554a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631c54a6670 con 0x5631c5542230 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5631c544e970 con 0x5631c5542c20 2024-04-24T23:19:42.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd853b71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631c54a2630 con 0x5631c5547d90 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5547720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c5542650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd849d9b640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8380a2b90 0x7fd8380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5547720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59688/0 (socket says 172.21.15.33:59688) 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84a59c640 1 -- 172.21.15.33:0/591875416 learned_addr learned my addr 172.21.15.33:0/591875416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c5542650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36270/0 (socket says 172.21.15.33:36270) 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd8437fe640 1 -- 172.21.15.33:0/526431951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd8340cf780 con 0x7fd84c104b30 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c554a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84a59c640 1 -- 172.21.15.33:0/591875416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 msgr2=0x5631c554a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84a59c640 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c554a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.483+0000 7fd84a59c640 1 -- 172.21.15.33:0/591875416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 msgr2=0x5631c5542650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd84a59c640 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c5542650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd84a59c640 1 -- 172.21.15.33:0/591875416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c5472400 con 0x5631c5542c20 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd849d9b640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8380a2b90 0x7fd8380a5050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd834098620 tx=0x7fd834091130 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd84a59c640 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5547720 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd83c035000 tx=0x7fd83c004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3ee7662498b6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd840ff9640 1 -- 172.21.15.33:0/591875416 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c049070 con 0x5631c5542c20 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd840ff9640 1 -- 172.21.15.33:0/591875416 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c036040 con 0x5631c5542c20 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.484+0000 7fd840ff9640 1 -- 172.21.15.33:0/591875416 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c02feb0 con 0x5631c5542c20 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- 172.21.15.33:0/591875416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 msgr2=0x5631c5547720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5547720 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd83c035000 tx=0x7fd83c004aa0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- 172.21.15.33:0/591875416 shutdown_connections 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c554a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5547720 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- 172.21.15.33:0/591875416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c5542650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- 172.21.15.33:0/591875416 >> 172.21.15.33:0/591875416 conn(0x5631c553d8a0 msgr2=0x5631c553dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- 172.21.15.33:0/591875416 shutdown_connections 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- 172.21.15.33:0/591875416 wait complete. 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- start start 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 Processor -- start 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c554c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542c20 0x5631c554c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c5553700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631c554ab60 con 0x5631c5542230 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5631c5462050 con 0x5631c5547d90 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.485+0000 7fd853b71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631c554ace0 con 0x5631c5542c20 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd853b71740 10 client.0 _ll_register_callbacks cb 0x5631c51b9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c554c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542c20 0x5631c554c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c554c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36280/0 (socket says 172.21.15.33:36280) 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 -- 172.21.15.33:0/2033293459 learned_addr learned my addr 172.21.15.33:0/2033293459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c5553700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 -- 172.21.15.33:0/2033293459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542c20 msgr2=0x5631c554c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542c20 0x5631c554c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 -- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 msgr2=0x5631c5553700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c5553700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 -- 172.21.15.33:0/2033293459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c5433dc0 con 0x5631c5542230 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c554c180 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd844013da0 tx=0x7fd844009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5005aa4e09a87e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd831ffb640 1 -- 172.21.15.33:0/2033293459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844005020 con 0x5631c5542230 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83c036040 con 0x5631c5542230 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd831ffb640 1 -- 172.21.15.33:0/2033293459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd844009040 con 0x5631c5542230 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd831ffb640 1 -- 172.21.15.33:0/2033293459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8440051c0 con 0x5631c5542230 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd853b71740 10 client.17025 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.486+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631c51ba3f0 con 0x5631c5542230 2024-04-24T23:19:42.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd831ffb640 1 -- 172.21.15.33:0/2033293459 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c036040 con 0x5631c5542230 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd831ffb640 1 -- 172.21.15.33:0/2033293459 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd844022470 con 0x5631c5542230 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd853b71740 10 client.17025 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd853b71740 10 client.17025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd853b71740 20 client.17025 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd853b71740 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55583a0 0x5631c555a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd853b71740 4 client.17025 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631c54b3bb0 con 0x5631c55583a0 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd84a59c640 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55583a0 0x5631c555a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd84a59c640 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55583a0 0x5631c555a840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2103d204ff93e99c server_cookie=43ba880387866795 in_seq=0 out_seq=0 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.487+0000 7fd831ffb640 10 client.17025 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:42.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd831ffb640 1 -- 172.21.15.33:0/2033293459 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631c54b3bb0 con 0x5631c55583a0 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd831ffb640 10 client.17025 handle_command_reply: tid=0 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 client.17025 shutdown 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 msgr2=0x5631c554c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c554c180 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd844013da0 tx=0x7fd844009ad0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 shutdown_connections 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55583a0 0x5631c555a840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c5553700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542c20 0x5631c554c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd831ffb640 0 client.17025 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 --2- 172.21.15.33:0/2033293459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c554c180 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 >> 172.21.15.33:0/2033293459 conn(0x5631c553d8a0 msgr2=0x5631c553cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 shutdown_connections 2024-04-24T23:19:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 1 -- 172.21.15.33:0/2033293459 wait complete. 2024-04-24T23:19:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.488+0000 7fd853b71740 20 client.17025 trim_cache size 0 max 16384 2024-04-24T23:19:42.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 Processor -- start 2024-04-24T23:19:42.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 -- start start 2024-04-24T23:19:42.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542230 0x5631c554c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 0x5631c55536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c55436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1acd10 con 0x5631c5542c20 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1aa5e0 con 0x5631c5547d90 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd853b71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1a8430 con 0x5631c5542230 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 0x5631c55536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 0x5631c55536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36294/0 (socket says 172.21.15.33:36294) 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 -- 172.21.15.33:0/1966045615 learned_addr learned my addr 172.21.15.33:0/1966045615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c55436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542230 0x5631c554c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 -- 172.21.15.33:0/1966045615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542230 msgr2=0x5631c554c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542230 0x5631c554c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 -- 172.21.15.33:0/1966045615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 msgr2=0x5631c55436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c55436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84a59c640 1 -- 172.21.15.33:0/1966045615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c52847d0 con 0x5631c5542c20 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.502+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c55436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd84a59c640 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 0x5631c55536c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd83c031ae0 tx=0x7fd83c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a733cb3855f91ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd833fff640 1 -- 172.21.15.33:0/1966045615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c049040 con 0x5631c5542c20 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd833fff640 1 -- 172.21.15.33:0/1966045615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c036040 con 0x5631c5542c20 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd833fff640 1 -- 172.21.15.33:0/1966045615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c02f610 con 0x5631c5542c20 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 -- 172.21.15.33:0/1966045615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 msgr2=0x5631c55536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 0x5631c55536c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd83c031ae0 tx=0x7fd83c005dc0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 -- 172.21.15.33:0/1966045615 shutdown_connections 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5547d90 0x5631c55436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542c20 0x5631c55536c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 --2- 172.21.15.33:0/1966045615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5542230 0x5631c554c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 -- 172.21.15.33:0/1966045615 >> 172.21.15.33:0/1966045615 conn(0x5631c553d8a0 msgr2=0x5631c553cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 -- 172.21.15.33:0/1966045615 shutdown_connections 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.503+0000 7fd853b71740 1 -- 172.21.15.33:0/1966045615 wait complete. 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 -- start start 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 Processor -- start 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c55504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5550a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c55bc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1a8aa0 con 0x5631c5542230 2024-04-24T23:19:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1a9f10 con 0x5631c5542c20 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1a9170 con 0x5631c5547d90 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd853b71740 10 client.0 _ll_register_callbacks cb 0x7fd844009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c55504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c55504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36304/0 (socket says 172.21.15.33:36304) 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5550a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 -- 172.21.15.33:0/1752677441 learned_addr learned my addr 172.21.15.33:0/1752677441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c55bc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 -- 172.21.15.33:0/1752677441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 msgr2=0x5631c55bc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c55bc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 -- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 msgr2=0x5631c5550a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5550a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 -- 172.21.15.33:0/1752677441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c5551070 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd84ad9d640 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c55504f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd844005eb0 tx=0x7fd844009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73a68835a61503e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.504+0000 7fd831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd831ffb640 1 -- 172.21.15.33:0/1752677441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844005570 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd831ffb640 1 -- 172.21.15.33:0/1752677441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd844022a60 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83c036040 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd831ffb640 1 -- 172.21.15.33:0/1752677441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd844041050 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 10 client.17031 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631c55be020 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd831ffb640 1 -- 172.21.15.33:0/1752677441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c036040 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd831ffb640 1 -- 172.21.15.33:0/1752677441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd844024640 con 0x5631c5542230 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 10 client.17031 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 10 client.17031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:19:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 20 client.17031 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55bea00 0x5631c55c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 4 client.17031 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:19:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631c54adf00 con 0x5631c55bea00 2024-04-24T23:19:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.505+0000 7fd84a59c640 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55bea00 0x5631c55c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.506+0000 7fd84a59c640 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55bea00 0x5631c55c0e20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b38c27871ec04762 server_cookie=bb9c94cc5b6a763f in_seq=0 out_seq=0 2024-04-24T23:19:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.506+0000 7fd831ffb640 10 client.17031 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:42.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd831ffb640 1 -- 172.21.15.33:0/1752677441 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631c54adf00 con 0x5631c55bea00 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd831ffb640 10 client.17031 handle_command_reply: tid=0 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 client.17031 shutdown 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 msgr2=0x5631c55504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c55504f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd844005eb0 tx=0x7fd844009ad0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 shutdown_connections 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5631c55bea00 0x5631c55c0e20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631c5547d90 0x5631c55bc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5631c5542c20 0x5631c5550a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 --2- 172.21.15.33:0/1752677441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631c5542230 0x5631c55504f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd831ffb640 0 client.17031 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 >> 172.21.15.33:0/1752677441 conn(0x5631c553d8a0 msgr2=0x5631c553cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 shutdown_connections 2024-04-24T23:19:42.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 1 -- 172.21.15.33:0/1752677441 wait complete. 2024-04-24T23:19:42.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.507+0000 7fd853b71740 20 client.17031 trim_cache size 0 max 16384 2024-04-24T23:19:42.517 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8380a2b90 msgr2=0x7fd8380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8380a2b90 0x7fd8380a5050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd834098620 tx=0x7fd834091130 comp rx=0 tx=0).stop 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 msgr2=0x7fd84c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 0x7fd84c198030 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9c0 tx=0x7fd83400ee80 comp rx=0 tx=0).stop 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 shutdown_connections 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd8380a2b90 0x7fd8380a5050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd84c107460 0x7fd84c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84c104b30 0x7fd84c198030 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 --2- 172.21.15.33:0/526431951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd84c102200 0x7fd84c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 >> 172.21.15.33:0/526431951 conn(0x7fd84c0fbbf0 msgr2=0x7fd84c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 shutdown_connections 2024-04-24T23:19:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.508+0000 7fd850d89640 1 -- 172.21.15.33:0/526431951 wait complete. 2024-04-24T23:19:42.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:19:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 -- 172.21.15.33:0/3281965120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204102230 msgr2=0x7f4204108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 --2- 172.21.15.33:0/3281965120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204102230 0x7f4204108ac0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009ca0 tx=0x7f41f402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 -- 172.21.15.33:0/3281965120 shutdown_connections 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 --2- 172.21.15.33:0/3281965120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204102230 0x7f4204108ac0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 --2- 172.21.15.33:0/3281965120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4204101870 0x7f4204101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 --2- 172.21.15.33:0/3281965120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4204100680 0x7f4204100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 -- 172.21.15.33:0/3281965120 >> 172.21.15.33:0/3281965120 conn(0x7f42040fbbf0 msgr2=0x7f42040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 -- 172.21.15.33:0/3281965120 shutdown_connections 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.628+0000 7f420999e640 1 -- 172.21.15.33:0/3281965120 wait complete. 2024-04-24T23:19:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 Processor -- start 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 -- start start 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 0x7f4204197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4204101870 0x7f4204197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4204102230 0x7f420419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f420410c300 con 0x7f4204100680 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f420410c180 con 0x7f4204101870 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f420999e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f420410c480 con 0x7f4204102230 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f4202ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 0x7f4204197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f4202ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 0x7f4204197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36330/0 (socket says 172.21.15.33:36330) 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f4202ffd640 1 -- 172.21.15.33:0/2735872458 learned_addr learned my addr 172.21.15.33:0/2735872458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f42027fc640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4204101870 0x7f4204197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.629+0000 7f42037fe640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4204102230 0x7f420419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.630+0000 7f4202ffd640 1 -- 172.21.15.33:0/2735872458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4204102230 msgr2=0x7f420419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.630+0000 7f4202ffd640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4204102230 0x7f420419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.630+0000 7f4202ffd640 1 -- 172.21.15.33:0/2735872458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4204101870 msgr2=0x7f4204197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.630+0000 7f4202ffd640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4204101870 0x7f4204197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.630+0000 7f4202ffd640 1 -- 172.21.15.33:0/2735872458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420419f6b0 con 0x7f4204100680 2024-04-24T23:19:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.630+0000 7f4202ffd640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 0x7f4204197a90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00e9c0 tx=0x7f41fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.632+0000 7f420899c640 1 -- 172.21.15.33:0/2735872458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00cd80 con 0x7f4204100680 2024-04-24T23:19:42.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.632+0000 7f420899c640 1 -- 172.21.15.33:0/2735872458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc0035f0 con 0x7f4204100680 2024-04-24T23:19:42.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.632+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420419f9a0 con 0x7f4204100680 2024-04-24T23:19:42.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.632+0000 7f420899c640 1 -- 172.21.15.33:0/2735872458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc010690 con 0x7f4204100680 2024-04-24T23:19:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.633+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420419fef0 con 0x7f4204100680 2024-04-24T23:19:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.635+0000 7f420899c640 1 -- 172.21.15.33:0/2735872458 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f41fc0037a0 con 0x7f4204100680 2024-04-24T23:19:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.635+0000 7f420c786740 1 Processor -- start 2024-04-24T23:19:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.635+0000 7f420899c640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41ec0a2b90 0x7f41ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.635+0000 7f420c786740 1 -- start start 2024-04-24T23:19:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420c786740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420c786740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc57720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420c786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc57d90 0x56398dc5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42037fe640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41ec0a2b90 0x7f41ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420c786740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56398dbb6670 con 0x56398dc52230 2024-04-24T23:19:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420c786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56398db5e970 con 0x56398dc57d90 2024-04-24T23:19:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420c786740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56398dbb2630 con 0x56398dc52c20 2024-04-24T23:19:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f420899c640 1 -- 172.21.15.33:0/2735872458 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41fc0cf7b0 con 0x7f4204100680 2024-04-24T23:19:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42027fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc57720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42027fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc57720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46792/0 (socket says 172.21.15.33:46792) 2024-04-24T23:19:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42027fc640 1 -- 172.21.15.33:0/4189015282 learned_addr learned my addr 172.21.15.33:0/4189015282 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42037fe640 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc57d90 0x56398dc5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 -- 172.21.15.33:0/4189015282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 msgr2=0x56398dc57720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc57720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 -- 172.21.15.33:0/4189015282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc57d90 msgr2=0x56398dc5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc57d90 0x56398dc5a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 -- 172.21.15.33:0/4189015282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398db82400 con 0x56398dc52230 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42027fc640 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc57720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42037fe640 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc57d90 0x56398dc5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f42037fe640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41ec0a2b90 0x7f41ec0a5050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005e30 tx=0x7f41f40099b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.636+0000 7f4202ffd640 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc52650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c6a0 tx=0x7f41fc09bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a3bb811e499d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.637+0000 7f41e9ffb640 1 -- 172.21.15.33:0/4189015282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0a0030 con 0x56398dc52230 2024-04-24T23:19:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.637+0000 7f41e9ffb640 1 -- 172.21.15.33:0/4189015282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc0a48f0 con 0x56398dc52230 2024-04-24T23:19:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.637+0000 7f41e9ffb640 1 -- 172.21.15.33:0/4189015282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc09b330 con 0x56398dc52230 2024-04-24T23:19:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 -- 172.21.15.33:0/4189015282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 msgr2=0x56398dc52650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc52650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c6a0 tx=0x7f41fc09bb80 comp rx=0 tx=0).stop 2024-04-24T23:19:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 -- 172.21.15.33:0/4189015282 shutdown_connections 2024-04-24T23:19:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc57d90 0x56398dc5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc57720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 --2- 172.21.15.33:0/4189015282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc52650 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 -- 172.21.15.33:0/4189015282 >> 172.21.15.33:0/4189015282 conn(0x56398dc4d8a0 msgr2=0x56398dc4dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 -- 172.21.15.33:0/4189015282 shutdown_connections 2024-04-24T23:19:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 -- 172.21.15.33:0/4189015282 wait complete. 2024-04-24T23:19:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.638+0000 7f420c786740 1 -- start start 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 Processor -- start 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52230 0x56398dc5c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc5c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dc63930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56398dc5ace0 con 0x56398dc57d90 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56398db72050 con 0x56398dc52230 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56398dc5ae60 con 0x56398dc52c20 2024-04-24T23:19:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f420c786740 10 client.0 _ll_register_callbacks cb 0x56398d8c9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dc63930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dc63930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36356/0 (socket says 172.21.15.33:36356) 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f4202ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52230 0x56398dc5c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 -- 172.21.15.33:0/1076851694 learned_addr learned my addr 172.21.15.33:0/1076851694 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42027fc640 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc5c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 -- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 msgr2=0x56398dc5c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc5c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 -- 172.21.15.33:0/1076851694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52230 msgr2=0x56398dc5c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52230 0x56398dc5c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 -- 172.21.15.33:0/1076851694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398db43dc0 con 0x56398dc57d90 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f42037fe640 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dc63930 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f41f4035000 tx=0x7f41f402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5012ce54c8ee16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.639+0000 7f41e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1076851694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4045040 con 0x56398dc57d90 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1076851694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41f4004d40 con 0x56398dc57d90 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41fc0a48f0 con 0x56398dc57d90 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1076851694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41f405b050 con 0x56398dc57d90 2024-04-24T23:19:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 10 client.17043 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56398d8ca3f0 con 0x56398dc57d90 2024-04-24T23:19:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1076851694 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc0a48f0 con 0x56398dc57d90 2024-04-24T23:19:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1076851694 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f41f402f870 con 0x56398dc57d90 2024-04-24T23:19:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 10 client.17043 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 10 client.17043 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 20 client.17043 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dc68280 0x56398dc6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 4 client.17043 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56398dbc3bb0 con 0x56398dc68280 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.640+0000 7f4202ffd640 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dc68280 0x56398dc6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.641+0000 7f4202ffd640 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dc68280 0x56398dc6a720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d838eb5da0fc921 server_cookie=3c3ca5b9f256d6da in_seq=0 out_seq=0 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.641+0000 7f41e2ffd640 10 client.17043 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.641+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1076851694 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56398dbc3bb0 con 0x56398dc68280 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.641+0000 7f41e2ffd640 10 client.17043 handle_command_reply: tid=0 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.641+0000 7f420c786740 1 client.17043 shutdown 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 msgr2=0x56398dc63930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dc63930 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f41f4035000 tx=0x7f41f402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 shutdown_connections 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dc68280 0x56398dc6a720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dc63930 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52c20 0x56398dc5c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 --2- 172.21.15.33:0/1076851694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52230 0x56398dc5c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 >> 172.21.15.33:0/1076851694 conn(0x56398dc4d8a0 msgr2=0x56398dc4cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f41e2ffd640 0 client.17043 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 shutdown_connections 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 1 -- 172.21.15.33:0/1076851694 wait complete. 2024-04-24T23:19:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.642+0000 7f420c786740 20 client.17043 trim_cache size 0 max 16384 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 Processor -- start 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 -- start start 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc51040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dc4c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc57d90 0x56398dc6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42041acd10 con 0x56398dc52230 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f42041aa5e0 con 0x56398dc52c20 2024-04-24T23:19:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.655+0000 7f420c786740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42041a8430 con 0x56398dc57d90 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc51040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc51040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36366/0 (socket says 172.21.15.33:36366) 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f42037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc57d90 0x56398dc6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 -- 172.21.15.33:0/2959808300 learned_addr learned my addr 172.21.15.33:0/2959808300 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f42027fc640 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dc4c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 -- 172.21.15.33:0/2959808300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc57d90 msgr2=0x56398dc6efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc57d90 0x56398dc6efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 -- 172.21.15.33:0/2959808300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 msgr2=0x56398dc4c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dc4c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 -- 172.21.15.33:0/2959808300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398d9947d0 con 0x56398dc52230 2024-04-24T23:19:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f42027fc640 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dc4c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f4202ffd640 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc51040 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0ccf70 tx=0x7f41fc09bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e319f8f26fe98709 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f41e97fa640 1 -- 172.21.15.33:0/2959808300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc07e070 con 0x56398dc52230 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f41e97fa640 1 -- 172.21.15.33:0/2959808300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc0a48f0 con 0x56398dc52230 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.656+0000 7f41e97fa640 1 -- 172.21.15.33:0/2959808300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0ba040 con 0x56398dc52230 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 -- 172.21.15.33:0/2959808300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 msgr2=0x56398dc51040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc51040 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0ccf70 tx=0x7f41fc09bde0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 -- 172.21.15.33:0/2959808300 shutdown_connections 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc57d90 0x56398dc6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dc4c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 --2- 172.21.15.33:0/2959808300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc52230 0x56398dc51040 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 -- 172.21.15.33:0/2959808300 >> 172.21.15.33:0/2959808300 conn(0x56398dc4d8a0 msgr2=0x56398dc4c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 -- 172.21.15.33:0/2959808300 shutdown_connections 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 -- 172.21.15.33:0/2959808300 wait complete. 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 -- start start 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.657+0000 7f420c786740 1 Processor -- start 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52230 0x56398dcc9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dcc9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dcd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42041a8aa0 con 0x56398dc57d90 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f42041a9f10 con 0x56398dc52c20 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42041a9170 con 0x56398dc52230 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 10 client.0 _ll_register_callbacks cb 0x7f41f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dcd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42027fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dcc9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dcd0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36376/0 (socket says 172.21.15.33:36376) 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 -- 172.21.15.33:0/1863578390 learned_addr learned my addr 172.21.15.33:0/1863578390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f4202ffd640 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52230 0x56398dcc9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 -- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52230 msgr2=0x56398dcc9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52230 0x56398dcc9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 -- 172.21.15.33:0/1863578390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 msgr2=0x56398dcc9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dcc9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 -- 172.21.15.33:0/1863578390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398dcd13c0 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f42037fe640 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dcd0dd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009ba0 tx=0x7f41f402f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7af368e41937a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f41e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1863578390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4064060 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1863578390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41f4004410 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.658+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41fc0a48f0 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1863578390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41f406b050 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 10 client.17049 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56398dcd2730 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1863578390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc0a48f0 con 0x56398dc57d90 2024-04-24T23:19:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1863578390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f41f4030aa0 con 0x56398dc57d90 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 10 client.17049 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 10 client.17049 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 20 client.17049 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dcd30d0 0x56398dcd54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 4 client.17049 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56398dbbdf00 con 0x56398dcd30d0 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f4202ffd640 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dcd30d0 0x56398dcd54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f41e2ffd640 10 client.17049 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.659+0000 7f4202ffd640 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dcd30d0 0x56398dcd54f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23d41c3465bf6b14 server_cookie=b627738f578c9e04 in_seq=0 out_seq=0 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f41e2ffd640 1 -- 172.21.15.33:0/1863578390 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56398dbbdf00 con 0x56398dcd30d0 2024-04-24T23:19:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f41e2ffd640 10 client.17049 handle_command_reply: tid=0 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 client.17049 shutdown 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 msgr2=0x56398dcd0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dcd0dd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009ba0 tx=0x7f41f402f8a0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 shutdown_connections 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56398dcd30d0 0x56398dcd54f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56398dc57d90 0x56398dcd0dd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56398dc52c20 0x56398dcc9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f41e2ffd640 0 client.17049 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 --2- 172.21.15.33:0/1863578390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56398dc52230 0x56398dcc9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 >> 172.21.15.33:0/1863578390 conn(0x56398dc4d8a0 msgr2=0x56398dc4cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 shutdown_connections 2024-04-24T23:19:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 1 -- 172.21.15.33:0/1863578390 wait complete. 2024-04-24T23:19:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420c786740 20 client.17049 trim_cache size 0 max 16384 2024-04-24T23:19:42.677 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:19:42.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41ec0a2b90 msgr2=0x7f41ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420999e640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41ec0a2b90 0x7f41ec0a5050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005e30 tx=0x7f41f40099b0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 msgr2=0x7f4204197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.660+0000 7f420999e640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 0x7f4204197a90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00e9c0 tx=0x7f41fc00ee80 comp rx=0 tx=0).stop 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 shutdown_connections 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41ec0a2b90 0x7f41ec0a5050 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4204102230 0x7f420419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4204101870 0x7f4204197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 --2- 172.21.15.33:0/2735872458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204100680 0x7f4204197a90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 >> 172.21.15.33:0/2735872458 conn(0x7f42040fbbf0 msgr2=0x7f42040fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 shutdown_connections 2024-04-24T23:19:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.661+0000 7f420999e640 1 -- 172.21.15.33:0/2735872458 wait complete. 2024-04-24T23:19:42.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:19:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 -- 172.21.15.33:0/2525741885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c100950 msgr2=0x7f525c100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 --2- 172.21.15.33:0/2525741885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c100950 0x7f525c100db0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f524c009900 tx=0x7f524c02a410 comp rx=0 tx=0).stop 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 -- 172.21.15.33:0/2525741885 shutdown_connections 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 --2- 172.21.15.33:0/2525741885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c1012f0 0x7f525c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 --2- 172.21.15.33:0/2525741885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c100950 0x7f525c100db0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 --2- 172.21.15.33:0/2525741885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 0x7f525c100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 -- 172.21.15.33:0/2525741885 >> 172.21.15.33:0/2525741885 conn(0x7f525c0fbbf0 msgr2=0x7f525c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 -- 172.21.15.33:0/2525741885 shutdown_connections 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 -- 172.21.15.33:0/2525741885 wait complete. 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 Processor -- start 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.784+0000 7f5261709640 1 -- start start 2024-04-24T23:19:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 0x7f525c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c100950 0x7f525c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c1012f0 0x7f525c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10f5f0 con 0x7f525c1012f0 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10f470 con 0x7f525c0ffff0 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10f770 con 0x7f525c100950 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c1012f0 0x7f525c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 0x7f525c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c100950 0x7f525c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 0x7f525c199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59794/0 (socket says 172.21.15.33:59794) 2024-04-24T23:19:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525affd640 1 -- 172.21.15.33:0/2664731588 learned_addr learned my addr 172.21.15.33:0/2664731588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 -- 172.21.15.33:0/2664731588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c100950 msgr2=0x7f525c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c100950 0x7f525c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 -- 172.21.15.33:0/2664731588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 msgr2=0x7f525c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 0x7f525c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 -- 172.21.15.33:0/2664731588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c1a1900 con 0x7f525c1012f0 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525a7fc640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c100950 0x7f525c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f525b7fe640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c1012f0 0x7f525c1a1260 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5254009fd0 tx=0x7f525400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5243fff640 1 -- 172.21.15.33:0/2664731588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525400ce20 con 0x7f525c1012f0 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c1a1bf0 con 0x7f525c1012f0 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5243fff640 1 -- 172.21.15.33:0/2664731588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5254005020 con 0x7f525c1012f0 2024-04-24T23:19:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5243fff640 1 -- 172.21.15.33:0/2664731588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254010720 con 0x7f525c1012f0 2024-04-24T23:19:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.785+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c1a2110 con 0x7f525c1012f0 2024-04-24T23:19:42.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5243fff640 1 -- 172.21.15.33:0/2664731588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5254005540 con 0x7f525c1012f0 2024-04-24T23:19:42.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 Processor -- start 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 -- start start 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924feb230 0x55c924feb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 0x55c924ff0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924ff0d90 0x55c924ff3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c924f4f670 con 0x55c924febc20 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c924ef7970 con 0x55c924feb230 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.788+0000 7f5264415740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c924f4b630 con 0x55c924ff0d90 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 0x55c924ff0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924feb230 0x55c924feb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 0x55c924ff0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36396/0 (socket says 172.21.15.33:36396) 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 -- 172.21.15.33:0/1595980198 learned_addr learned my addr 172.21.15.33:0/1595980198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525a7fc640 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924ff0d90 0x55c924ff3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f5243fff640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52440a2b90 0x7f52440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 -- 172.21.15.33:0/1595980198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924ff0d90 msgr2=0x55c924ff3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924ff0d90 0x55c924ff3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 -- 172.21.15.33:0/1595980198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924feb230 msgr2=0x55c924feb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924feb230 0x55c924feb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 -- 172.21.15.33:0/1595980198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c924f1b400 con 0x55c924febc20 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525a7fc640 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924feb230 0x55c924feb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525b7fe640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52440a2b90 0x7f52440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f525affd640 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 0x55c924ff0720 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5250009d70 tx=0x7f5250011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cf24142b584d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f5243fff640 1 -- 172.21.15.33:0/2664731588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f52540cfa40 con 0x7f525c1012f0 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f52417fa640 1 -- 172.21.15.33:0/1595980198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525001e070 con 0x55c924febc20 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f52417fa640 1 -- 172.21.15.33:0/1595980198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5250019b10 con 0x55c924febc20 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.789+0000 7f52417fa640 1 -- 172.21.15.33:0/1595980198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525000eba0 con 0x55c924febc20 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 -- 172.21.15.33:0/1595980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 msgr2=0x55c924ff0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 0x55c924ff0720 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5250009d70 tx=0x7f5250011810 comp rx=0 tx=0).stop 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f525b7fe640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52440a2b90 0x7f52440a5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5254098880 tx=0x7f52540c6a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 -- 172.21.15.33:0/1595980198 shutdown_connections 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924ff0d90 0x55c924ff3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924febc20 0x55c924ff0720 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 --2- 172.21.15.33:0/1595980198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924feb230 0x55c924feb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 -- 172.21.15.33:0/1595980198 >> 172.21.15.33:0/1595980198 conn(0x55c924fe68a0 msgr2=0x55c924fe6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 -- 172.21.15.33:0/1595980198 shutdown_connections 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 -- 172.21.15.33:0/1595980198 wait complete. 2024-04-24T23:19:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 -- start start 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.790+0000 7f5264415740 1 Processor -- start 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ff5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c924ff3d40 con 0x55c924feb230 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c924f0b050 con 0x55c924ff0d90 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c924ff3ec0 con 0x55c924febc20 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 10 client.0 _ll_register_callbacks cb 0x55c924c62be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36408/0 (socket says 172.21.15.33:36408) 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 -- 172.21.15.33:0/366424502 learned_addr learned my addr 172.21.15.33:0/366424502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525affd640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ff5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 -- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 msgr2=0x55c924ff5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ff5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 -- 172.21.15.33:0/366424502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 msgr2=0x55c924ffc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 -- 172.21.15.33:0/366424502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c924edcdc0 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525affd640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ff5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f525a7fc640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff52c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f524c02a930 tx=0x7f524c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e91c386cd5d47d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f523eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f523eff5640 1 -- 172.21.15.33:0/366424502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c041070 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f523eff5640 1 -- 172.21.15.33:0/366424502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f524c005230 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.791+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5250019b10 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f523eff5640 1 -- 172.21.15.33:0/366424502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f524c052050 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 10 client.17061 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c924c633f0 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f523eff5640 1 -- 172.21.15.33:0/366424502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5250019b10 con 0x55c924feb230 2024-04-24T23:19:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f523eff5640 1 -- 172.21.15.33:0/366424502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f524c0035f0 con 0x55c924feb230 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 10 client.17061 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 20 client.17061 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925001070 0x55c925003510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 4 client.17061 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c924f5cbb0 con 0x55c925001070 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.792+0000 7f525affd640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925001070 0x55c925003510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f525affd640 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925001070 0x55c925003510 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79352e9440373900 server_cookie=53e44acb92451df9 in_seq=0 out_seq=0 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f523eff5640 10 client.17061 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f523eff5640 1 -- 172.21.15.33:0/366424502 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c924f5cbb0 con 0x55c925001070 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f523eff5640 10 client.17061 handle_command_reply: tid=0 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 client.17061 shutdown 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 msgr2=0x55c924ff52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff52c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f524c02a930 tx=0x7f524c02a410 comp rx=0 tx=0).stop 2024-04-24T23:19:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 shutdown_connections 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925001070 0x55c925003510 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ff5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f523eff5640 0 client.17061 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 --2- 172.21.15.33:0/366424502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff52c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.793+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 >> 172.21.15.33:0/366424502 conn(0x55c924fe68a0 msgr2=0x55c924fe5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.794+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 shutdown_connections 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.794+0000 7f5264415740 1 -- 172.21.15.33:0/366424502 wait complete. 2024-04-24T23:19:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.794+0000 7f5264415740 20 client.17061 trim_cache size 0 max 16384 2024-04-24T23:19:42.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 Processor -- start 2024-04-24T23:19:42.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 -- start start 2024-04-24T23:19:42.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924feb230 0x55c924fe97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924febc20 0x55c924ffc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 0x55c924fe9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1aee80 con 0x55c924ff0d90 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1ac750 con 0x55c924febc20 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.807+0000 7f5264415740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1aa5a0 con 0x55c924feb230 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 0x55c924fe9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 0x55c924fe9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36418/0 (socket says 172.21.15.33:36418) 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 -- 172.21.15.33:0/2921077408 learned_addr learned my addr 172.21.15.33:0/2921077408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525affd640 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924febc20 0x55c924ffc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924feb230 0x55c924fe97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 -- 172.21.15.33:0/2921077408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924feb230 msgr2=0x55c924fe97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924feb230 0x55c924fe97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 -- 172.21.15.33:0/2921077408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924febc20 msgr2=0x55c924ffc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924febc20 0x55c924ffc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 -- 172.21.15.33:0/2921077408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c924d2d7d0 con 0x55c924ff0d90 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f525a7fc640 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 0x55c924fe9cf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f524c033000 tx=0x7f524c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cbba2ba528c973 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f5240ff9640 1 -- 172.21.15.33:0/2921077408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c041020 con 0x55c924ff0d90 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f5240ff9640 1 -- 172.21.15.33:0/2921077408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524c003e30 con 0x55c924ff0d90 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.808+0000 7f5240ff9640 1 -- 172.21.15.33:0/2921077408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c035050 con 0x55c924ff0d90 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- 172.21.15.33:0/2921077408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 msgr2=0x55c924fe9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 0x55c924fe9cf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f524c033000 tx=0x7f524c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- 172.21.15.33:0/2921077408 shutdown_connections 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924ff0d90 0x55c924fe9cf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924febc20 0x55c924ffc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- 172.21.15.33:0/2921077408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924feb230 0x55c924fe97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- 172.21.15.33:0/2921077408 >> 172.21.15.33:0/2921077408 conn(0x55c924fe68a0 msgr2=0x55c924fe5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- 172.21.15.33:0/2921077408 shutdown_connections 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- 172.21.15.33:0/2921077408 wait complete. 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- start start 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 Processor -- start 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ffa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1aac10 con 0x55c924feb230 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1ac080 con 0x55c924ff0d90 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1ab2e0 con 0x55c924febc20 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f5264415740 10 client.0 _ll_register_callbacks cb 0x7f524c031a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.809+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36422/0 (socket says 172.21.15.33:36422) 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 -- 172.21.15.33:0/3210145740 learned_addr learned my addr 172.21.15.33:0/3210145740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525affd640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ffa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 -- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 msgr2=0x55c924ffa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ffa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 -- 172.21.15.33:0/3210145740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 msgr2=0x55c924ffb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 -- 172.21.15.33:0/3210145740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c924ffb770 con 0x55c924feb230 2024-04-24T23:19:42.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525affd640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ffa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f525a7fc640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff9c30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f524c001eb0 tx=0x7f524c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b29f95eb4e3b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.810+0000 7f523eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.811+0000 7f523eff5640 1 -- 172.21.15.33:0/3210145740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c035070 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.812+0000 7f523eff5640 1 -- 172.21.15.33:0/3210145740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f524c005030 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.813+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524c003e30 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.813+0000 7f523eff5640 1 -- 172.21.15.33:0/3210145740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f524c041030 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.813+0000 7f5264415740 10 client.17067 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.813+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c924ffca70 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f523eff5640 1 -- 172.21.15.33:0/3210145740 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524c003e30 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f523eff5640 1 -- 172.21.15.33:0/3210145740 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f524c004a90 con 0x55c924feb230 2024-04-24T23:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f5264415740 10 client.17067 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f5264415740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f5264415740 20 client.17067 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f5264415740 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925067c20 0x55c92506a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f5264415740 4 client.17067 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.814+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c924f56f00 con 0x55c925067c20 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.815+0000 7f525affd640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925067c20 0x55c92506a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.815+0000 7f525affd640 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925067c20 0x55c92506a040 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8223ed8600f08bc5 server_cookie=870f93f38338b4f8 in_seq=0 out_seq=0 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.815+0000 7f523eff5640 10 client.17067 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.815+0000 7f523eff5640 1 -- 172.21.15.33:0/3210145740 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c924f56f00 con 0x55c925067c20 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.815+0000 7f523eff5640 10 client.17067 handle_command_reply: tid=0 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.815+0000 7f5264415740 1 client.17067 shutdown 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 msgr2=0x55c924ff9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff9c30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f524c001eb0 tx=0x7f524c02a410 comp rx=0 tx=0).stop 2024-04-24T23:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 shutdown_connections 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c925067c20 0x55c92506a040 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c924ff0d90 0x55c924ffb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c924febc20 0x55c924ffa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 --2- 172.21.15.33:0/3210145740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c924feb230 0x55c924ff9c30 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f523eff5640 0 client.17067 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 >> 172.21.15.33:0/3210145740 conn(0x55c924fe68a0 msgr2=0x55c924fec6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 shutdown_connections 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 1 -- 172.21.15.33:0/3210145740 wait complete. 2024-04-24T23:19:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.816+0000 7f5264415740 20 client.17067 trim_cache size 0 max 16384 2024-04-24T23:19:42.822 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:19:42.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52440a2b90 msgr2=0x7f52440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52440a2b90 0x7f52440a5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5254098880 tx=0x7f52540c6a40 comp rx=0 tx=0).stop 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c1012f0 msgr2=0x7f525c1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c1012f0 0x7f525c1a1260 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5254009fd0 tx=0x7f525400ef10 comp rx=0 tx=0).stop 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 shutdown_connections 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52440a2b90 0x7f52440a5050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c1012f0 0x7f525c1a1260 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525c100950 0x7f525c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 --2- 172.21.15.33:0/2664731588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f525c0ffff0 0x7f525c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 >> 172.21.15.33:0/2664731588 conn(0x7f525c0fbbf0 msgr2=0x7f525c0fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 shutdown_connections 2024-04-24T23:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:42.819+0000 7f5261709640 1 -- 172.21.15.33:0/2664731588 wait complete. 2024-04-24T23:19:42.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:42 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1941877283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:19:42.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:42 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1941877283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:19:42.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:42 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1941877283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:19:43.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 832298 12.70 MB/sec execute 228 sec latency 15.201 ms 2024-04-24T23:19:43.839 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:19:43.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:19:43.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:19:43.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:43 smithi038 ceph-mon[32450]: pgmap v582: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 319 op/s 2024-04-24T23:19:43.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:43 smithi117 ceph-mon[32529]: pgmap v582: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 319 op/s 2024-04-24T23:19:43.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:43 smithi033 ceph-mon[29201]: pgmap v582: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 319 op/s 2024-04-24T23:19:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 -- 172.21.15.33:0/2498415054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 msgr2=0x7f93cc102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 --2- 172.21.15.33:0/2498415054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc102df0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009900 tx=0x7f93bc02a410 comp rx=0 tx=0).stop 2024-04-24T23:19:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 -- 172.21.15.33:0/2498415054 shutdown_connections 2024-04-24T23:19:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 --2- 172.21.15.33:0/2498415054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc103330 0x7f93cc109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 --2- 172.21.15.33:0/2498415054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc102df0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 --2- 172.21.15.33:0/2498415054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc101780 0x7f93cc101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 -- 172.21.15.33:0/2498415054 >> 172.21.15.33:0/2498415054 conn(0x7f93cc0fccf0 msgr2=0x7f93cc0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 -- 172.21.15.33:0/2498415054 shutdown_connections 2024-04-24T23:19:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.957+0000 7f93d4741640 1 -- 172.21.15.33:0/2498415054 wait complete. 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 Processor -- start 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 -- start start 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc101780 0x7f93cc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc103330 0x7f93cc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc10d450 con 0x7f93cc102970 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc10d2d0 con 0x7f93cc103330 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d4741640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc10d5d0 con 0x7f93cc101780 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36438/0 (socket says 172.21.15.33:36438) 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 -- 172.21.15.33:0/2216100396 learned_addr learned my addr 172.21.15.33:0/2216100396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d2cde640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc103330 0x7f93cc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d24dd640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc101780 0x7f93cc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 -- 172.21.15.33:0/2216100396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc101780 msgr2=0x7f93cc197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc101780 0x7f93cc197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 -- 172.21.15.33:0/2216100396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc103330 msgr2=0x7f93cc19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc103330 0x7f93cc19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.958+0000 7f93d1cdc640 1 -- 172.21.15.33:0/2216100396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc19f6b0 con 0x7f93cc102970 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.959+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc197fd0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f93bc031440 tx=0x7f93bc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.959+0000 7f93c37fe640 1 -- 172.21.15.33:0/2216100396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc03f030 con 0x7f93cc102970 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.959+0000 7f93c37fe640 1 -- 172.21.15.33:0/2216100396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93bc003ce0 con 0x7f93cc102970 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.959+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc19f940 con 0x7f93cc102970 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.959+0000 7f93c37fe640 1 -- 172.21.15.33:0/2216100396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc035070 con 0x7f93cc102970 2024-04-24T23:19:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.959+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc19fe30 con 0x7f93cc102970 2024-04-24T23:19:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.961+0000 7f93c37fe640 1 -- 172.21.15.33:0/2216100396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f93bc042030 con 0x7f93cc102970 2024-04-24T23:19:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.962+0000 7f93c37fe640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93b80a2b90 0x7f93b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.962+0000 7f93d7445740 1 Processor -- start 2024-04-24T23:19:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.962+0000 7f93d24dd640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93b80a2b90 0x7f93b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.962+0000 7f93c37fe640 1 -- 172.21.15.33:0/2216100396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f93bc0ef840 con 0x7f93cc102970 2024-04-24T23:19:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.962+0000 7f93d7445740 1 -- start start 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d7445740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d7445740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288eb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d7445740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d7445740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562d288492c0 con 0x562d288eb8f0 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d7445740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d287f1910 con 0x562d288e6780 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d7445740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d28845280 con 0x562d288e5d90 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93b80a2b90 0x7f93b80a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f93cc1027d0 tx=0x7f93c8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d2cde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d2cde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288edd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36442/0 (socket says 172.21.15.33:36442) 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288eb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59854/0 (socket says 172.21.15.33:59854) 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d2cde640 1 -- 172.21.15.33:0/1356420585 learned_addr learned my addr 172.21.15.33:0/1356420585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 -- 172.21.15.33:0/1356420585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 msgr2=0x562d288eb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288eb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 -- 172.21.15.33:0/1356420585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 msgr2=0x562d288edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 -- 172.21.15.33:0/1356420585 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d28802fb0 con 0x562d288e5d90 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d2cde640 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.963+0000 7f93d24dd640 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288e61b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f93c80118d0 tx=0x7f93c8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d122cc29462fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93c0ff9640 1 -- 172.21.15.33:0/1356420585 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8026430 con 0x562d288e5d90 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93c0ff9640 1 -- 172.21.15.33:0/1356420585 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c801d8f0 con 0x562d288e5d90 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93c0ff9640 1 -- 172.21.15.33:0/1356420585 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8025950 con 0x562d288e5d90 2024-04-24T23:19:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 -- 172.21.15.33:0/1356420585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 msgr2=0x562d288e61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288e61b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f93c80118d0 tx=0x7f93c8014000 comp rx=0 tx=0).stop 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 -- 172.21.15.33:0/1356420585 shutdown_connections 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 --2- 172.21.15.33:0/1356420585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288e61b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 -- 172.21.15.33:0/1356420585 >> 172.21.15.33:0/1356420585 conn(0x562d288ad950 msgr2=0x562d288e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.964+0000 7f93d7445740 1 -- 172.21.15.33:0/1356420585 shutdown_connections 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 -- 172.21.15.33:0/1356420585 wait complete. 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 -- start start 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 Processor -- start 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 0x562d288f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288f7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562d288ee8a0 con 0x562d288e5d90 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562d28804e90 con 0x562d288e6780 2024-04-24T23:19:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d288eea20 con 0x562d288eb8f0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 0x562d288f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 0x562d288f01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36456/0 (socket says 172.21.15.33:36456) 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 -- 172.21.15.33:0/3542681897 learned_addr learned my addr 172.21.15.33:0/3542681897 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d2cde640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d7445740 10 client.0 _ll_register_callbacks cb 0x562d28840970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 -- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 msgr2=0x562d288f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 -- 172.21.15.33:0/3542681897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 msgr2=0x562d288f06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.965+0000 7f93d24dd640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d24dd640 1 -- 172.21.15.33:0/3542681897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d286257e0 con 0x562d288e5d90 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c801d8f0 con 0x562d288e5d90 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d24dd640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 0x562d288f01b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f93c801d410 tx=0x7f93c801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e086105f81384c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93b5ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3542681897 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c801daa0 con 0x562d288e5d90 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3542681897 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f93c801dc40 con 0x562d288e5d90 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3542681897 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93c8004020 con 0x562d288e5d90 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d7445740 10 client.17079 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=27}) -- 0x562d28685f70 con 0x562d288e5d90 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3542681897 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f93c80041c0 con 0x562d288e5d90 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d7445740 10 client.17079 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.966+0000 7f93d7445740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93d7445740 20 client.17079 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93d7445740 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d288fb800 0x562d288fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93d7445740 4 client.17079 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d2881d5a0 con 0x562d288fb800 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d288fb800 0x562d288fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93b5ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.967+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d288fb800 0x562d288fdc30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6213d854817172c6 server_cookie=bf3cddfaed326ce8 in_seq=0 out_seq=0 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3542681897 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d2881d5a0 con 0x562d288fb800 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93b5ffb640 10 client.17079 handle_command_reply: tid=0 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 client.17079 shutdown 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 msgr2=0x562d288f01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 0x562d288f01b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f93c801d410 tx=0x7f93c801d440 comp rx=0 tx=0).stop 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 shutdown_connections 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d288fb800 0x562d288fdc30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93b5ffb640 0 client.17079 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 --2- 172.21.15.33:0/3542681897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e5d90 0x562d288f01b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 >> 172.21.15.33:0/3542681897 conn(0x562d288ad950 msgr2=0x562d288e0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 shutdown_connections 2024-04-24T23:19:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.968+0000 7f93d7445740 1 -- 172.21.15.33:0/3542681897 wait complete. 2024-04-24T23:19:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.969+0000 7f93d7445740 20 client.17079 trim_cache size 0 max 16384 2024-04-24T23:19:43.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 Processor -- start 2024-04-24T23:19:43.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 -- start start 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 0x562d288f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e6780 0x562d288f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288e4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1aee70 con 0x562d288e6780 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1ac740 con 0x562d288e5d90 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d7445740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc073dc0 con 0x562d288eb8f0 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e6780 0x562d288f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d24dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 0x562d288f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e6780 0x562d288f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36464/0 (socket says 172.21.15.33:36464) 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d24dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 0x562d288f04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59862/0 (socket says 172.21.15.33:59862) 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d1cdc640 1 -- 172.21.15.33:0/2517610925 learned_addr learned my addr 172.21.15.33:0/2517610925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.988+0000 7f93d2cde640 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d24dd640 1 -- 172.21.15.33:0/2517610925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 msgr2=0x562d288e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d24dd640 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288e4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d24dd640 1 -- 172.21.15.33:0/2517610925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e6780 msgr2=0x562d288f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d24dd640 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e6780 0x562d288f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d24dd640 1 -- 172.21.15.33:0/2517610925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2851d170 con 0x562d288e5d90 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d2cde640 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93d24dd640 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 0x562d288f04d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f93c8016040 tx=0x7f93c802c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f37c92dd239ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93b7fff640 1 -- 172.21.15.33:0/2517610925 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8047070 con 0x562d288e5d90 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93b7fff640 1 -- 172.21.15.33:0/2517610925 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c801d8f0 con 0x562d288e5d90 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.989+0000 7f93b7fff640 1 -- 172.21.15.33:0/2517610925 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c801dca0 con 0x562d288e5d90 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- 172.21.15.33:0/2517610925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 msgr2=0x562d288f04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 0x562d288f04d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f93c8016040 tx=0x7f93c802c000 comp rx=0 tx=0).stop 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- 172.21.15.33:0/2517610925 shutdown_connections 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288eb8f0 0x562d288e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288e6780 0x562d288f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- 172.21.15.33:0/2517610925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e5d90 0x562d288f04d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- 172.21.15.33:0/2517610925 >> 172.21.15.33:0/2517610925 conn(0x562d288ad950 msgr2=0x562d288efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- 172.21.15.33:0/2517610925 shutdown_connections 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- 172.21.15.33:0/2517610925 wait complete. 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- start start 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 Processor -- start 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288f5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc074430 con 0x562d288eb8f0 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc0758a0 con 0x562d288e6780 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc074b00 con 0x562d288e5d90 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.990+0000 7f93d7445740 10 client.0 _ll_register_callbacks cb 0x7f93c4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36480/0 (socket says 172.21.15.33:36480) 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d1cdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 -- 172.21.15.33:0/3939992737 learned_addr learned my addr 172.21.15.33:0/3939992737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d24dd640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 -- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 msgr2=0x562d288f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 -- 172.21.15.33:0/3939992737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 msgr2=0x562d288f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 -- 172.21.15.33:0/3939992737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d288f6cb0 con 0x562d288eb8f0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d1cdc640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d2cde640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288f66c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f93c40042c0 tx=0x7f93c4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7accbdf84466d98a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3939992737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4021060 con 0x562d288eb8f0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c801d8f0 con 0x562d288eb8f0 2024-04-24T23:19:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3939992737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f93c4010d40 con 0x562d288eb8f0 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3939992737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93c4028050 con 0x562d288eb8f0 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d7445740 10 client.17085 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.991+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d28962110 con 0x562d288eb8f0 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3939992737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c801d8f0 con 0x562d288eb8f0 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3939992737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f93c400f450 con 0x562d288eb8f0 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d7445740 10 client.17085 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d7445740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d7445740 20 client.17085 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d7445740 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d28962ae0 0x562d28964f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d7445740 4 client.17085 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d28856800 con 0x562d28962ae0 2024-04-24T23:19:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d24dd640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d28962ae0 0x562d28964f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93d24dd640 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d28962ae0 0x562d28964f00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707a24e704329b64 server_cookie=e18892e394a899bd in_seq=0 out_seq=0 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.992+0000 7f93b5ffb640 10 client.17085 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93b5ffb640 1 -- 172.21.15.33:0/3939992737 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d28856800 con 0x562d28962ae0 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93b5ffb640 10 client.17085 handle_command_reply: tid=0 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 client.17085 shutdown 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 msgr2=0x562d288f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288f66c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f93c40042c0 tx=0x7f93c4003f90 comp rx=0 tx=0).stop 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 shutdown_connections 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562d28962ae0 0x562d28964f00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d288eb8f0 0x562d288f66c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93b5ffb640 0 client.17085 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d288e6780 0x562d288f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 --2- 172.21.15.33:0/3939992737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d288e5d90 0x562d288f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 >> 172.21.15.33:0/3939992737 conn(0x562d288ad950 msgr2=0x562d288e00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 shutdown_connections 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 1 -- 172.21.15.33:0/3939992737 wait complete. 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.994+0000 7f93d7445740 20 client.17085 trim_cache size 0 max 16384 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:19:44.003 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "025ada61-5348-4e55-a673-e75576e0dc00", 2024-04-24T23:19:44.004 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:19:44.004 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:19:44.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93b80a2b90 msgr2=0x7f93b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93b80a2b90 0x7f93b80a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f93cc1027d0 tx=0x7f93c8005f90 comp rx=0 tx=0).stop 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 msgr2=0x7f93cc197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc197fd0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f93bc031440 tx=0x7f93bc032ce0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 shutdown_connections 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f93b80a2b90 0x7f93b80a5050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc103330 0x7f93cc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93cc102970 0x7f93cc197fd0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 --2- 172.21.15.33:0/2216100396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc101780 0x7f93cc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 >> 172.21.15.33:0/2216100396 conn(0x7f93cc0fccf0 msgr2=0x7f93cc105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 shutdown_connections 2024-04-24T23:19:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:43.995+0000 7f93d4741640 1 -- 172.21.15.33:0/2216100396 wait complete. 2024-04-24T23:19:44.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:19:44.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:19:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 -- 172.21.15.33:0/740191846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d41006a0 msgr2=0x7ff6d4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 --2- 172.21.15.33:0/740191846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d41006a0 0x7ff6d4100ac0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009900 tx=0x7ff6c402a410 comp rx=0 tx=0).stop 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 -- 172.21.15.33:0/740191846 shutdown_connections 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 --2- 172.21.15.33:0/740191846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d4102250 0x7ff6d4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 --2- 172.21.15.33:0/740191846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6d4101890 0x7ff6d4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 --2- 172.21.15.33:0/740191846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d41006a0 0x7ff6d4100ac0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 -- 172.21.15.33:0/740191846 >> 172.21.15.33:0/740191846 conn(0x7ff6d40fbc30 msgr2=0x7ff6d40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 -- 172.21.15.33:0/740191846 shutdown_connections 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.120+0000 7ff6d8989640 1 -- 172.21.15.33:0/740191846 wait complete. 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 Processor -- start 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 -- start start 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41006a0 0x7ff6d406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 0x7ff6d419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6d4102250 0x7ff6d419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410c320 con 0x7ff6d4101890 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410c1a0 con 0x7ff6d4102250 2024-04-24T23:19:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d8989640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410c4a0 con 0x7ff6d41006a0 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 0x7ff6d419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 0x7ff6d419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36512/0 (socket says 172.21.15.33:36512) 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6d4102250 0x7ff6d419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/1132499340 learned_addr learned my addr 172.21.15.33:0/1132499340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d259c640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41006a0 0x7ff6d406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/1132499340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41006a0 msgr2=0x7ff6d406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41006a0 0x7ff6d406a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/1132499340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6d4102250 msgr2=0x7ff6d419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.121+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6d4102250 0x7ff6d419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/1132499340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d419f6e0 con 0x7ff6d4101890 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 0x7ff6d419c7b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e9f0 tx=0x7ff6bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6cb7fe640 1 -- 172.21.15.33:0/1132499340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00cd80 con 0x7ff6d4101890 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d419f9d0 con 0x7ff6d4101890 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6cb7fe640 1 -- 172.21.15.33:0/1132499340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6bc010910 con 0x7ff6d4101890 2024-04-24T23:19:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d419ff70 con 0x7ff6d4101890 2024-04-24T23:19:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.122+0000 7ff6cb7fe640 1 -- 172.21.15.33:0/1132499340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0188f0 con 0x7ff6d4101890 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 Processor -- start 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 -- start start 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6cb7fe640 1 -- 172.21.15.33:0/1132499340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff6bc010ac0 con 0x7ff6d4101890 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc36b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc370730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc373230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eadc2cf6b0 con 0x55eadc36bc30 2024-04-24T23:19:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eadc277880 con 0x55eadc36b240 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6db774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eadc2cb670 con 0x55eadc370da0 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc370730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc373230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc36b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc370730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36526/0 (socket says 172.21.15.33:36526) 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc36b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59904/0 (socket says 172.21.15.33:59904) 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc373230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46926/0 (socket says 172.21.15.33:46926) 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.125+0000 7ff6d259c640 1 -- 172.21.15.33:0/3250661946 learned_addr learned my addr 172.21.15.33:0/3250661946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d2d9d640 1 -- 172.21.15.33:0/3250661946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 msgr2=0x55eadc373230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc373230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d2d9d640 1 -- 172.21.15.33:0/3250661946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 msgr2=0x55eadc370730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc370730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d2d9d640 1 -- 172.21.15.33:0/3250661946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadc0ad7d0 con 0x55eadc36b240 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc373230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc36b660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009f30 tx=0x7ff6cc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c66e7ee5e6f7bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6cb7fe640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6c00a2b90 0x7ff6c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6d259c640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6c00a2b90 0x7ff6c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6c8ff9640 1 -- 172.21.15.33:0/3250661946 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc016070 con 0x55eadc36b240 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6cb7fe640 1 -- 172.21.15.33:0/1132499340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6bc0cf740 con 0x7ff6d4101890 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6c8ff9640 1 -- 172.21.15.33:0/3250661946 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc004d20 con 0x55eadc36b240 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.126+0000 7ff6c8ff9640 1 -- 172.21.15.33:0/3250661946 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc012870 con 0x55eadc36b240 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 -- 172.21.15.33:0/3250661946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 msgr2=0x55eadc36b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6d259c640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6c00a2b90 0x7ff6c00a5050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc36b660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009f30 tx=0x7ff6cc010800 comp rx=0 tx=0).stop 2024-04-24T23:19:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 -- 172.21.15.33:0/3250661946 shutdown_connections 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc373230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc370730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 --2- 172.21.15.33:0/3250661946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc36b660 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 -- 172.21.15.33:0/3250661946 >> 172.21.15.33:0/3250661946 conn(0x55eadc3668b0 msgr2=0x55eadc366cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 -- 172.21.15.33:0/3250661946 shutdown_connections 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 -- 172.21.15.33:0/3250661946 wait complete. 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 -- start start 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.127+0000 7ff6db774740 1 Processor -- start 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36b240 0x55eadc375690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36bc30 0x55eadc375bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eadc373d50 con 0x55eadc36b240 2024-04-24T23:19:44.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eadc28af50 con 0x55eadc36bc30 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eadc373ed0 con 0x55eadc370da0 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 10 client.0 _ll_register_callbacks cb 0x55eadc2c6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36b240 0x55eadc375690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36bc30 0x55eadc375bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36b240 0x55eadc375690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36542/0 (socket says 172.21.15.33:36542) 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d259c640 1 -- 172.21.15.33:0/3169662662 learned_addr learned my addr 172.21.15.33:0/3169662662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 msgr2=0x55eadc37cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/3169662662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36b240 msgr2=0x55eadc375690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36b240 0x55eadc375690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/3169662662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadbfe33f0 con 0x55eadc36bc30 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36bc30 0x55eadc375bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0ccf00 tx=0x7ff6bc009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e2bc81fa01adbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/3169662662 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc09b620 con 0x55eadc36bc30 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/3169662662 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6bc018b50 con 0x55eadc36bc30 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.128+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6cc004d20 con 0x55eadc36bc30 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/3169662662 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc09bcc0 con 0x55eadc36bc30 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 10 client.26624 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eadbfa30f0 con 0x55eadc36bc30 2024-04-24T23:19:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/3169662662 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc004d20 con 0x55eadc36bc30 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/3169662662 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff6bc097a10 con 0x55eadc36bc30 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 10 client.26624 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 10 client.26624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 20 client.26624 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc381530 0x55eadc383940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 4 client.26624 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eadc2dcbf0 con 0x55eadc381530 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.129+0000 7ff6d259c640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc381530 0x55eadc383940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6b9ffb640 10 client.26624 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6d259c640 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc381530 0x55eadc383940 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131d1c456be3188 server_cookie=abe2647b88defbb3 in_seq=0 out_seq=0 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/3169662662 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eadc2dcbf0 con 0x55eadc381530 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6b9ffb640 10 client.26624 handle_command_reply: tid=0 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 client.26624 shutdown 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36bc30 msgr2=0x55eadc375bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36bc30 0x55eadc375bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0ccf00 tx=0x7ff6bc009700 comp rx=0 tx=0).stop 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 shutdown_connections 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc381530 0x55eadc383940 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.130+0000 7ff6db774740 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36bc30 0x55eadc375bd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.131+0000 7ff6db774740 1 --2- 172.21.15.33:0/3169662662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36b240 0x55eadc375690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.131+0000 7ff6b9ffb640 0 client.26624 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.131+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 >> 172.21.15.33:0/3169662662 conn(0x55eadc3668b0 msgr2=0x55eadc365af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.131+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 shutdown_connections 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.131+0000 7ff6db774740 1 -- 172.21.15.33:0/3169662662 wait complete. 2024-04-24T23:19:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.131+0000 7ff6db774740 20 client.26624 trim_cache size 0 max 16384 2024-04-24T23:19:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 Processor -- start 2024-04-24T23:19:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 -- start start 2024-04-24T23:19:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc369630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc36bc30 0x55eadc37cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc370da0 0x55eadc369b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41acd10 con 0x55eadc370da0 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41aa5e0 con 0x55eadc36b240 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6db774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41a8430 con 0x55eadc36bc30 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc370da0 0x55eadc369b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc369630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc369630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59926/0 (socket says 172.21.15.33:59926) 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 -- 172.21.15.33:0/1413942234 learned_addr learned my addr 172.21.15.33:0/1413942234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc370da0 0x55eadc369b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36546/0 (socket says 172.21.15.33:36546) 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc36bc30 0x55eadc37cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 -- 172.21.15.33:0/1413942234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc36bc30 msgr2=0x55eadc37cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc36bc30 0x55eadc37cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 -- 172.21.15.33:0/1413942234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc370da0 msgr2=0x55eadc369b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc370da0 0x55eadc369b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.152+0000 7ff6d259c640 1 -- 172.21.15.33:0/1413942234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadc373fd0 con 0x55eadc36b240 2024-04-24T23:19:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.153+0000 7ff6d259c640 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc369630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40314e0 tx=0x7ff6c403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=446b2cc51ee50cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.153+0000 7ff6bbfff640 1 -- 172.21.15.33:0/1413942234 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4035040 con 0x55eadc36b240 2024-04-24T23:19:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.153+0000 7ff6bbfff640 1 -- 172.21.15.33:0/1413942234 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4056070 con 0x55eadc36b240 2024-04-24T23:19:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.153+0000 7ff6bbfff640 1 -- 172.21.15.33:0/1413942234 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4051310 con 0x55eadc36b240 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- 172.21.15.33:0/1413942234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 msgr2=0x55eadc369630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc369630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40314e0 tx=0x7ff6c403f000 comp rx=0 tx=0).stop 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- 172.21.15.33:0/1413942234 shutdown_connections 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc370da0 0x55eadc369b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc36bc30 0x55eadc37cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- 172.21.15.33:0/1413942234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc369630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- 172.21.15.33:0/1413942234 >> 172.21.15.33:0/1413942234 conn(0x55eadc3668b0 msgr2=0x55eadc375690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- 172.21.15.33:0/1413942234 shutdown_connections 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- 172.21.15.33:0/1413942234 wait complete. 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- start start 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 Processor -- start 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc37a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc37aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41a8aa0 con 0x55eadc36bc30 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41a9f10 con 0x55eadc36b240 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41a9170 con 0x55eadc370da0 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.154+0000 7ff6db774740 10 client.0 _ll_register_callbacks cb 0x7ff6bc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc37aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc37a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc37aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36550/0 (socket says 172.21.15.33:36550) 2024-04-24T23:19:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/4289593889 learned_addr learned my addr 172.21.15.33:0/4289593889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d2d9d640 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 msgr2=0x55eadc37ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/4289593889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 msgr2=0x55eadc37a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc37a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 -- 172.21.15.33:0/4289593889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadc37c080 con 0x55eadc36bc30 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6d1d9b640 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc37aa80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc09b9c0 tx=0x7ff6bc09b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1708b45f9cd15b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/4289593889 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0b8060 con 0x55eadc36bc30 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/4289593889 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6bc0ba5d0 con 0x55eadc36bc30 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c4056070 con 0x55eadc36bc30 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/4289593889 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc0d5240 con 0x55eadc36bc30 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6db774740 10 client.17103 fetch_fsmap learned FSMap version 27 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.155+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eadc3e7d40 con 0x55eadc36bc30 2024-04-24T23:19:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/4289593889 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4056070 con 0x55eadc36bc30 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/4289593889 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff6bc018be0 con 0x55eadc36bc30 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6db774740 10 client.17103 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6db774740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6db774740 20 client.17103 populate_metadata read hostname 'smithi033' 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6db774740 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc3e8550 0x55eadc3ea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6db774740 4 client.17103 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eadc2d6f40 con 0x55eadc3e8550 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.156+0000 7ff6d259c640 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc3e8550 0x55eadc3ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6d259c640 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc3e8550 0x55eadc3ea970 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7818d3496b81c38 server_cookie=f771cb33c1e428fc in_seq=0 out_seq=0 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6b9ffb640 10 client.17103 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6b9ffb640 1 -- 172.21.15.33:0/4289593889 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eadc2d6f40 con 0x55eadc3e8550 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6b9ffb640 10 client.17103 handle_command_reply: tid=0 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 client.17103 shutdown 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 msgr2=0x55eadc37aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc37aa80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc09b9c0 tx=0x7ff6bc09b710 comp rx=0 tx=0).stop 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 shutdown_connections 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55eadc3e8550 0x55eadc3ea970 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eadc370da0 0x55eadc37ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eadc36bc30 0x55eadc37aa80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6b9ffb640 0 client.17103 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 --2- 172.21.15.33:0/4289593889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eadc36b240 0x55eadc37a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 >> 172.21.15.33:0/4289593889 conn(0x55eadc3668b0 msgr2=0x55eadc365260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 shutdown_connections 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 1 -- 172.21.15.33:0/4289593889 wait complete. 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6db774740 20 client.17103 trim_cache size 0 max 16384 2024-04-24T23:19:44.165 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: "025ada61-5348-4e55-a673-e75576e0dc00": { 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "025ada61-5348-4e55-a673-e75576e0dc00", 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:19:44.166 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6c00a2b90 msgr2=0x7ff6c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6d8989640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6c00a2b90 0x7ff6c00a5050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c4005fd0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 msgr2=0x7ff6d419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.157+0000 7ff6d8989640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 0x7ff6d419c7b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e9f0 tx=0x7ff6bc00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 shutdown_connections 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6c00a2b90 0x7ff6c00a5050 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6d4102250 0x7ff6d419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d4101890 0x7ff6d419c7b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 --2- 172.21.15.33:0/1132499340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41006a0 0x7ff6d406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 >> 172.21.15.33:0/1132499340 conn(0x7ff6d40fbc30 msgr2=0x7ff6d4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 shutdown_connections 2024-04-24T23:19:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:19:44.158+0000 7ff6d8989640 1 -- 172.21.15.33:0/1132499340 wait complete. 2024-04-24T23:19:44.174 INFO:tasks.cephfs.filesystem:scrub status for tag:025ada61-5348-4e55-a673-e75576e0dc00 - {'path': '/', 'tag': '025ada61-5348-4e55-a673-e75576e0dc00', 'options': 'recursive,force'} 2024-04-24T23:19:44.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 834637 12.70 MB/sec execute 229 sec latency 62.802 ms 2024-04-24T23:19:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:44 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:19:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:44 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:44 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:19:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:44 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:19:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:44 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:44 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:44 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:19:44.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:44 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:44 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:19:44.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:44 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:19:44.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:44 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:44 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:44 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:19:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:44 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:44 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:19:44.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:44 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:19:44.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:44 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:44.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:44 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:19:45.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 836848 12.68 MB/sec execute 230 sec latency 23.699 ms 2024-04-24T23:19:45.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:45 smithi038 ceph-mon[32450]: pgmap v583: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:19:45.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:45 smithi117 ceph-mon[32529]: pgmap v583: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:19:45.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:45 smithi033 ceph-mon[29201]: pgmap v583: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:19:46.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 838833 12.68 MB/sec execute 231 sec latency 25.259 ms 2024-04-24T23:19:47.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 841748 12.70 MB/sec execute 232 sec latency 21.085 ms 2024-04-24T23:19:47.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:47 smithi038 ceph-mon[32450]: pgmap v584: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-04-24T23:19:47.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:47 smithi117 ceph-mon[32529]: pgmap v584: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-04-24T23:19:47.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:47 smithi033 ceph-mon[29201]: pgmap v584: 129 pgs: 129 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-04-24T23:19:48.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 844368 12.70 MB/sec execute 233 sec latency 13.765 ms 2024-04-24T23:19:49.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 845904 12.68 MB/sec execute 234 sec latency 104.193 ms 2024-04-24T23:19:49.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:49 smithi038 ceph-mon[32450]: pgmap v585: 129 pgs: 129 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 232 op/s 2024-04-24T23:19:49.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:49 smithi117 ceph-mon[32529]: pgmap v585: 129 pgs: 129 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 232 op/s 2024-04-24T23:19:49.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:49 smithi033 ceph-mon[29201]: pgmap v585: 129 pgs: 129 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 232 op/s 2024-04-24T23:19:50.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 848798 12.70 MB/sec execute 235 sec latency 26.595 ms 2024-04-24T23:19:51.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 851005 12.68 MB/sec execute 236 sec latency 13.551 ms 2024-04-24T23:19:51.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:51 smithi038 ceph-mon[32450]: pgmap v586: 129 pgs: 129 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T23:19:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:51 smithi117 ceph-mon[32529]: pgmap v586: 129 pgs: 129 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T23:19:51.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:51 smithi033 ceph-mon[29201]: pgmap v586: 129 pgs: 129 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T23:19:52.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 852923 12.68 MB/sec execute 237 sec latency 13.864 ms 2024-04-24T23:19:52.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:19:52.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:19:52.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:19:53.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 855889 12.70 MB/sec execute 238 sec latency 10.674 ms 2024-04-24T23:19:53.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:53 smithi038 ceph-mon[32450]: pgmap v587: 129 pgs: 129 active+clean; 283 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-04-24T23:19:53.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:53 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:19:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:53 smithi117 ceph-mon[32529]: pgmap v587: 129 pgs: 129 active+clean; 283 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-04-24T23:19:53.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:53 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:19:53.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:53 smithi033 ceph-mon[29201]: pgmap v587: 129 pgs: 129 active+clean; 283 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-04-24T23:19:53.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:53 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:19:54.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 857429 12.67 MB/sec execute 239 sec latency 98.294 ms 2024-04-24T23:19:55.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 859785 12.68 MB/sec execute 240 sec latency 24.906 ms 2024-04-24T23:19:55.922 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:55 smithi038 ceph-mon[32450]: pgmap v588: 129 pgs: 129 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:19:55.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:55 smithi117 ceph-mon[32529]: pgmap v588: 129 pgs: 129 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:19:55.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:55 smithi033 ceph-mon[29201]: pgmap v588: 129 pgs: 129 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-24T23:19:56.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 862564 12.69 MB/sec execute 241 sec latency 27.455 ms 2024-04-24T23:19:57.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 864587 12.67 MB/sec execute 242 sec latency 13.628 ms 2024-04-24T23:19:57.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:57 smithi038 ceph-mon[32450]: pgmap v589: 129 pgs: 129 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:19:57.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:57 smithi117 ceph-mon[32529]: pgmap v589: 129 pgs: 129 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:19:57.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:57 smithi033 ceph-mon[29201]: pgmap v589: 129 pgs: 129 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:19:58.451 INFO:tasks.workunit.client.0.smithi033.stdout: 1 867025 12.68 MB/sec execute 243 sec latency 15.840 ms 2024-04-24T23:19:59.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 869676 12.69 MB/sec execute 244 sec latency 50.969 ms 2024-04-24T23:19:59.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:19:59 smithi038 ceph-mon[32450]: pgmap v590: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-24T23:19:59.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:19:59 smithi117 ceph-mon[32529]: pgmap v590: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-24T23:19:59.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:19:59 smithi033 ceph-mon[29201]: pgmap v590: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-24T23:20:00.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 871119 12.66 MB/sec execute 245 sec latency 33.215 ms 2024-04-24T23:20:00.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:00 smithi038 ceph-mon[32450]: overall HEALTH_OK 2024-04-24T23:20:00.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:00 smithi117 ceph-mon[32529]: overall HEALTH_OK 2024-04-24T23:20:00.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:00 smithi033 ceph-mon[29201]: overall HEALTH_OK 2024-04-24T23:20:01.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 873859 12.68 MB/sec execute 246 sec latency 11.228 ms 2024-04-24T23:20:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:01 smithi038 ceph-mon[32450]: pgmap v591: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-04-24T23:20:01.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:01 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:01 smithi117 ceph-mon[32529]: pgmap v591: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-04-24T23:20:01.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:01 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:01 smithi033 ceph-mon[29201]: pgmap v591: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-04-24T23:20:01.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:01 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:02.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 876571 12.68 MB/sec execute 247 sec latency 12.932 ms 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:02.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:02.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:02.927 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:20:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:02.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:02.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:02.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:20:03.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 878063 12.66 MB/sec execute 248 sec latency 44.075 ms 2024-04-24T23:20:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:03 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi117 to 176.2M 2024-04-24T23:20:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:03 smithi038 ceph-mon[32450]: Unable to set osd_memory_target on smithi117 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:20:03.921 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:03 smithi038 ceph-mon[32450]: pgmap v592: 129 pgs: 129 active+clean; 291 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 302 op/s 2024-04-24T23:20:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:03 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi117 to 176.2M 2024-04-24T23:20:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:03 smithi117 ceph-mon[32529]: Unable to set osd_memory_target on smithi117 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:20:03.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:03 smithi117 ceph-mon[32529]: pgmap v592: 129 pgs: 129 active+clean; 291 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 302 op/s 2024-04-24T23:20:03.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:03 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi117 to 176.2M 2024-04-24T23:20:03.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:03 smithi033 ceph-mon[29201]: Unable to set osd_memory_target on smithi117 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:20:03.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:03 smithi033 ceph-mon[29201]: pgmap v592: 129 pgs: 129 active+clean; 291 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 302 op/s 2024-04-24T23:20:04.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 880720 12.68 MB/sec execute 249 sec latency 79.400 ms 2024-04-24T23:20:05.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 882168 12.65 MB/sec execute 250 sec latency 78.394 ms 2024-04-24T23:20:05.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:05 smithi117 ceph-mon[32529]: pgmap v593: 129 pgs: 129 active+clean; 292 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-04-24T23:20:05.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:05 smithi033 ceph-mon[29201]: pgmap v593: 129 pgs: 129 active+clean; 292 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-04-24T23:20:06.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:05 smithi038 ceph-mon[32450]: pgmap v593: 129 pgs: 129 active+clean; 292 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-04-24T23:20:06.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 884679 12.66 MB/sec execute 251 sec latency 12.532 ms 2024-04-24T23:20:07.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 886785 12.65 MB/sec execute 252 sec latency 62.350 ms 2024-04-24T23:20:07.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:07 smithi117 ceph-mon[32529]: pgmap v594: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-04-24T23:20:07.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:07 smithi033 ceph-mon[29201]: pgmap v594: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-04-24T23:20:08.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:07 smithi038 ceph-mon[32450]: pgmap v594: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-04-24T23:20:08.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 888721 12.64 MB/sec execute 253 sec latency 17.890 ms 2024-04-24T23:20:09.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 891352 12.66 MB/sec execute 254 sec latency 97.939 ms 2024-04-24T23:20:09.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:09 smithi117 ceph-mon[32529]: pgmap v595: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-04-24T23:20:09.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:09 smithi033 ceph-mon[29201]: pgmap v595: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-04-24T23:20:10.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:09 smithi038 ceph-mon[32450]: pgmap v595: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-04-24T23:20:10.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 893066 12.63 MB/sec execute 255 sec latency 32.000 ms 2024-04-24T23:20:11.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 895446 12.63 MB/sec execute 256 sec latency 19.520 ms 2024-04-24T23:20:11.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:11 smithi033 ceph-mon[29201]: pgmap v596: 129 pgs: 129 active+clean; 287 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-04-24T23:20:12.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:11 smithi038 ceph-mon[32450]: pgmap v596: 129 pgs: 129 active+clean; 287 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-04-24T23:20:12.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:11 smithi117 ceph-mon[32529]: pgmap v596: 129 pgs: 129 active+clean; 287 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-04-24T23:20:12.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 898411 12.66 MB/sec execute 257 sec latency 25.056 ms 2024-04-24T23:20:13.452 INFO:tasks.workunit.client.0.smithi033.stdout: 1 900637 12.64 MB/sec execute 258 sec latency 29.027 ms 2024-04-24T23:20:13.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:13 smithi033 ceph-mon[29201]: pgmap v597: 129 pgs: 129 active+clean; 275 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-04-24T23:20:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:13 smithi038 ceph-mon[32450]: pgmap v597: 129 pgs: 129 active+clean; 275 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-04-24T23:20:14.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:20:14.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:20:14.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:13 smithi117 ceph-mon[32529]: pgmap v597: 129 pgs: 129 active+clean; 275 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-04-24T23:20:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 -- 172.21.15.33:0/2228955385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd534101890 msgr2=0x7fd534101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 --2- 172.21.15.33:0/2228955385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd534101890 0x7fd534101d10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd528009900 tx=0x7fd52802a410 comp rx=0 tx=0).stop 2024-04-24T23:20:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 -- 172.21.15.33:0/2228955385 shutdown_connections 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 --2- 172.21.15.33:0/2228955385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd534102250 0x7fd534108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 --2- 172.21.15.33:0/2228955385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd534101890 0x7fd534101d10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 --2- 172.21.15.33:0/2228955385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5341006a0 0x7fd534100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 -- 172.21.15.33:0/2228955385 >> 172.21.15.33:0/2228955385 conn(0x7fd5340fbc30 msgr2=0x7fd5340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 -- 172.21.15.33:0/2228955385 shutdown_connections 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.281+0000 7fd53beb3640 1 -- 172.21.15.33:0/2228955385 wait complete. 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 Processor -- start 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 -- start start 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 0x7fd53406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd534101890 0x7fd53419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd534102250 0x7fd5341a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd53410c320 con 0x7fd5341006a0 2024-04-24T23:20:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd53410c1a0 con 0x7fd534102250 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53beb3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd53410c4a0 con 0x7fd534101890 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd539c4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 0x7fd53406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd539c4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 0x7fd53406a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58740/0 (socket says 172.21.15.33:58740) 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd539c4f640 1 -- 172.21.15.33:0/3284835643 learned_addr learned my addr 172.21.15.33:0/3284835643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53944e640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd534101890 0x7fd53419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53a450640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd534102250 0x7fd5341a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53944e640 1 -- 172.21.15.33:0/3284835643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd534102250 msgr2=0x7fd5341a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53944e640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd534102250 0x7fd5341a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53944e640 1 -- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 msgr2=0x7fd53406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53944e640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 0x7fd53406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd53944e640 1 -- 172.21.15.33:0/3284835643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5341a19d0 con 0x7fd534101890 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.282+0000 7fd539c4f640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 0x7fd53406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.283+0000 7fd53944e640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd534101890 0x7fd53419ec90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd528031440 tx=0x7fd528004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.283+0000 7fd526ffd640 1 -- 172.21.15.33:0/3284835643 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52803f070 con 0x7fd534101890 2024-04-24T23:20:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.283+0000 7fd526ffd640 1 -- 172.21.15.33:0/3284835643 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd528003cd0 con 0x7fd534101890 2024-04-24T23:20:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.283+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5341a1c00 con 0x7fd534101890 2024-04-24T23:20:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.283+0000 7fd526ffd640 1 -- 172.21.15.33:0/3284835643 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528035070 con 0x7fd534101890 2024-04-24T23:20:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.283+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5341a20c0 con 0x7fd534101890 2024-04-24T23:20:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd526ffd640 1 -- 172.21.15.33:0/3284835643 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd52803b460 con 0x7fd534101890 2024-04-24T23:20:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 Processor -- start 2024-04-24T23:20:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 -- start start 2024-04-24T23:20:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c08730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c0b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561a7b676b0 con 0x5561a7c08da0 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5561a7b0f880 con 0x5561a7c03c30 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5561a7b63670 con 0x5561a7c03240 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53944e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c0b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53944e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c0b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58754/0 (socket says 172.21.15.33:58754) 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd53944e640 1 -- 172.21.15.33:0/1409633684 learned_addr learned my addr 172.21.15.33:0/1409633684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.286+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c08730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd53a450640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd539c4f640 1 -- 172.21.15.33:0/1409633684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 msgr2=0x5561a7c03660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c03660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd539c4f640 1 -- 172.21.15.33:0/1409633684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 msgr2=0x5561a7c0b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c0b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd526ffd640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5200a2bc0 0x7fd5200a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd539c4f640 1 -- 172.21.15.33:0/1409633684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a79457d0 con 0x5561a7c03c30 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd53944e640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c0b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd53a450640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd53a450640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5200a2bc0 0x7fd5200a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd526ffd640 1 -- 172.21.15.33:0/3284835643 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd5280ef820 con 0x7fd534101890 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c08730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5300058c0 tx=0x7fd530011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90099ef28f2068e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd51ffff640 1 -- 172.21.15.33:0/1409633684 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53001e070 con 0x5561a7c03c30 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd51ffff640 1 -- 172.21.15.33:0/1409633684 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5300035f0 con 0x5561a7c03c30 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.287+0000 7fd51ffff640 1 -- 172.21.15.33:0/1409633684 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53000f780 con 0x5561a7c03c30 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1409633684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 msgr2=0x5561a7c08730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53a450640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5200a2bc0 0x7fd5200a5080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5561a7c04490 tx=0x7fd52c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c08730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5300058c0 tx=0x7fd530011950 comp rx=0 tx=0).stop 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1409633684 shutdown_connections 2024-04-24T23:20:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c0b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c08730 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1409633684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1409633684 >> 172.21.15.33:0/1409633684 conn(0x5561a7bfe8b0 msgr2=0x5561a7bfecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1409633684 shutdown_connections 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1409633684 wait complete. 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 -- start start 2024-04-24T23:20:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.288+0000 7fd53ebc0740 1 Processor -- start 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c0d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c0dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c14bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561a7c0bd50 con 0x5561a7c08da0 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5561a7b22f50 con 0x5561a7c03c30 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5561a7c0bed0 con 0x5561a7c03240 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53ebc0740 10 client.0 _ll_register_callbacks cb 0x5561a7b5e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c14bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c14bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58758/0 (socket says 172.21.15.33:58758) 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 -- 172.21.15.33:0/1280566327 learned_addr learned my addr 172.21.15.33:0/1280566327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53944e640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c0dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c0d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 -- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 msgr2=0x5561a7c0d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c0d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 -- 172.21.15.33:0/1280566327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 msgr2=0x5561a7c0dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c0dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 -- 172.21.15.33:0/1280566327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a787b3f0 con 0x5561a7c08da0 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c0d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53944e640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c0dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd53a450640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c14bd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004070 tx=0x7fd52c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf0a1238c067b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.289+0000 7fd51d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd51d7fa640 1 -- 172.21.15.33:0/1280566327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004b20 con 0x5561a7c08da0 2024-04-24T23:20:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd51d7fa640 1 -- 172.21.15.33:0/1280566327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd52c004cc0 con 0x5561a7c08da0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5300035f0 con 0x5561a7c08da0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd51d7fa640 1 -- 172.21.15.33:0/1280566327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd52c031050 con 0x5561a7c08da0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 10 client.17115 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561a783b0f0 con 0x5561a7c08da0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd51d7fa640 1 -- 172.21.15.33:0/1280566327 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5300035f0 con 0x5561a7c08da0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd51d7fa640 1 -- 172.21.15.33:0/1280566327 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd52c027030 con 0x5561a7c08da0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 10 client.17115 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 20 client.17115 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c19520 0x5561a7c1b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 4 client.17115 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561a7b74bf0 con 0x5561a7c19520 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.290+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c19520 0x5561a7c1b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd51d7fa640 10 client.17115 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd539c4f640 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c19520 0x5561a7c1b980 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da4edfcf777af61 server_cookie=4512d0fc9862ebf0 in_seq=0 out_seq=0 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd51d7fa640 1 -- 172.21.15.33:0/1280566327 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561a7b74bf0 con 0x5561a7c19520 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd51d7fa640 10 client.17115 handle_command_reply: tid=0 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd53ebc0740 1 client.17115 shutdown 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 msgr2=0x5561a7c14bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.291+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c14bd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004070 tx=0x7fd52c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 shutdown_connections 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c19520 0x5561a7c1b980 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c14bd0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c0dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd51d7fa640 0 client.17115 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1280566327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c0d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 >> 172.21.15.33:0/1280566327 conn(0x5561a7bfe8b0 msgr2=0x5561a7bfdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 shutdown_connections 2024-04-24T23:20:14.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1280566327 wait complete. 2024-04-24T23:20:14.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.292+0000 7fd53ebc0740 20 client.17115 trim_cache size 0 max 16384 2024-04-24T23:20:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 Processor -- start 2024-04-24T23:20:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 -- start start 2024-04-24T23:20:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c14b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341aeef0 con 0x5561a7c08da0 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341ac7c0 con 0x5561a7c03c30 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.310+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341aa610 con 0x5561a7c03240 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53944e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c14b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd539c4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58764/0 (socket says 172.21.15.33:58764) 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 -- 172.21.15.33:0/1275035416 learned_addr learned my addr 172.21.15.33:0/1275035416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53944e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c14b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56696/0 (socket says 172.21.15.33:56696) 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 -- 172.21.15.33:0/1275035416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 msgr2=0x5561a7c05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 -- 172.21.15.33:0/1275035416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 msgr2=0x5561a7c14b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c14b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 -- 172.21.15.33:0/1275035416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a7bff620 con 0x5561a7c08da0 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53944e640 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c14b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd53a450640 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c055d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004040 tx=0x7fd52c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cb54cd712d68df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd51f7fe640 1 -- 172.21.15.33:0/1275035416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00faf0 con 0x5561a7c08da0 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd51f7fe640 1 -- 172.21.15.33:0/1275035416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c00fc90 con 0x5561a7c08da0 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.311+0000 7fd51f7fe640 1 -- 172.21.15.33:0/1275035416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004d80 con 0x5561a7c08da0 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1275035416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 msgr2=0x5561a7c055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c055d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004040 tx=0x7fd52c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1275035416 shutdown_connections 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c08da0 0x5561a7c055d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c14b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/1275035416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c03240 0x5561a7c05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1275035416 >> 172.21.15.33:0/1275035416 conn(0x5561a7bfe8b0 msgr2=0x5561a7c0d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1275035416 shutdown_connections 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 -- 172.21.15.33:0/1275035416 wait complete. 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.312+0000 7fd53ebc0740 1 -- start start 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 Processor -- start 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 0x5561a7c11c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c12190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c08da0 0x5561a7c13130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341aac80 con 0x5561a7c03240 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341ac0f0 con 0x5561a7c03c30 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5341ab350 con 0x5561a7c08da0 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53ebc0740 10 client.0 _ll_register_callbacks cb 0x7fd52c003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd539c4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 0x5561a7c11c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd539c4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 0x5561a7c11c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58766/0 (socket says 172.21.15.33:58766) 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53944e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c12190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd539c4f640 1 -- 172.21.15.33:0/3352550330 learned_addr learned my addr 172.21.15.33:0/3352550330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53a450640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c08da0 0x5561a7c13130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53944e640 1 -- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c08da0 msgr2=0x5561a7c13130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53944e640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c08da0 0x5561a7c13130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53944e640 1 -- 172.21.15.33:0/3352550330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 msgr2=0x5561a7c11c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.313+0000 7fd53944e640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 0x5561a7c11c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd53944e640 1 -- 172.21.15.33:0/3352550330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a7c13720 con 0x5561a7c03c30 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd539c4f640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 0x5561a7c11c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd53944e640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c12190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd528006010 tx=0x7fd528003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ee1ef0bcadbef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd51d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd51d7fa640 1 -- 172.21.15.33:0/3352550330 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528048040 con 0x5561a7c03c30 2024-04-24T23:20:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd51d7fa640 1 -- 172.21.15.33:0/3352550330 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd5280bc150 con 0x5561a7c03c30 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c00fc90 con 0x5561a7c03c30 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd51d7fa640 1 -- 172.21.15.33:0/3352550330 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5280bb830 con 0x5561a7c03c30 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd53ebc0740 10 client.26648 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561a7c14aa0 con 0x5561a7c03c30 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.314+0000 7fd51d7fa640 1 -- 172.21.15.33:0/3352550330 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c00fc90 con 0x5561a7c03c30 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd51d7fa640 1 -- 172.21.15.33:0/3352550330 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd52804b060 con 0x5561a7c03c30 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 10 client.26648 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 20 client.26648 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c7fe80 0x5561a7c822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 4 client.26648 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561a7b6ef40 con 0x5561a7c7fe80 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd539c4f640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c7fe80 0x5561a7c822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd51d7fa640 10 client.26648 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd539c4f640 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c7fe80 0x5561a7c822a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103d93b03460132f server_cookie=fb7ce09f190e60b5 in_seq=0 out_seq=0 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd51d7fa640 1 -- 172.21.15.33:0/3352550330 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561a7b6ef40 con 0x5561a7c7fe80 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd51d7fa640 10 client.26648 handle_command_reply: tid=0 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.315+0000 7fd53ebc0740 1 client.26648 shutdown 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 msgr2=0x5561a7c12190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c12190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd528006010 tx=0x7fd528003ed0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 shutdown_connections 2024-04-24T23:20:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5561a7c7fe80 0x5561a7c822a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd51d7fa640 0 client.26648 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561a7c08da0 0x5561a7c13130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561a7c03c30 0x5561a7c12190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 --2- 172.21.15.33:0/3352550330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561a7c03240 0x5561a7c11c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 >> 172.21.15.33:0/3352550330 conn(0x5561a7bfe8b0 msgr2=0x5561a7c0d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 shutdown_connections 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 1 -- 172.21.15.33:0/3352550330 wait complete. 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53ebc0740 20 client.26648 trim_cache size 0 max 16384 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:20:14.327 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:20:14.328 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:20:14.328 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5200a2bc0 msgr2=0x7fd5200a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.316+0000 7fd53beb3640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5200a2bc0 0x7fd5200a5080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5561a7c04490 tx=0x7fd52c009500 comp rx=0 tx=0).stop 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd534101890 msgr2=0x7fd53419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd534101890 0x7fd53419ec90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd528031440 tx=0x7fd528004270 comp rx=0 tx=0).stop 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 shutdown_connections 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5200a2bc0 0x7fd5200a5080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd534102250 0x7fd5341a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd534101890 0x7fd53419ec90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 --2- 172.21.15.33:0/3284835643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5341006a0 0x7fd53406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 >> 172.21.15.33:0/3284835643 conn(0x7fd5340fbc30 msgr2=0x7fd534104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 shutdown_connections 2024-04-24T23:20:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.317+0000 7fd53beb3640 1 -- 172.21.15.33:0/3284835643 wait complete. 2024-04-24T23:20:14.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:20:14.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:20:14.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 902632 12.64 MB/sec execute 259 sec latency 27.870 ms 2024-04-24T23:20:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.456+0000 7fa300eda640 1 -- 172.21.15.33:0/2177787734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc1090b0 msgr2=0x7fa2fc109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.456+0000 7fa300eda640 1 --2- 172.21.15.33:0/2177787734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc109560 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0013f10 tx=0x7fa2f00329c0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.458+0000 7fa300eda640 1 -- 172.21.15.33:0/2177787734 shutdown_connections 2024-04-24T23:20:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.458+0000 7fa300eda640 1 --2- 172.21.15.33:0/2177787734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc109560 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.458+0000 7fa300eda640 1 --2- 172.21.15.33:0/2177787734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc10c360 0x7fa2fc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.458+0000 7fa300eda640 1 --2- 172.21.15.33:0/2177787734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc06a590 0x7fa2fc1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.458+0000 7fa300eda640 1 -- 172.21.15.33:0/2177787734 >> 172.21.15.33:0/2177787734 conn(0x7fa2fc06e980 msgr2=0x7fa2fc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.463+0000 7fa300eda640 1 -- 172.21.15.33:0/2177787734 shutdown_connections 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.466+0000 7fa300eda640 1 -- 172.21.15.33:0/2177787734 wait complete. 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 Processor -- start 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 -- start start 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc06a590 0x7fa2fc1af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc1af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc10c360 0x7fa2fc19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc113b10 con 0x7fa2fc06a590 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc113990 con 0x7fa2fc1090b0 2024-04-24T23:20:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.467+0000 7fa300eda640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc113c90 con 0x7fa2fc10c360 2024-04-24T23:20:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.469+0000 7fa2f9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc1af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.470+0000 7fa2fad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc10c360 0x7fa2fc19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.471+0000 7fa2f9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc1af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56726/0 (socket says 172.21.15.33:56726) 2024-04-24T23:20:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.471+0000 7fa2f9d9b640 1 -- 172.21.15.33:0/283949025 learned_addr learned my addr 172.21.15.33:0/283949025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.471+0000 7fa2fa59c640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc06a590 0x7fa2fc1af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.474+0000 7fa2fa59c640 1 -- 172.21.15.33:0/283949025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc10c360 msgr2=0x7fa2fc19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.474+0000 7fa2fa59c640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc10c360 0x7fa2fc19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.474+0000 7fa2fa59c640 1 -- 172.21.15.33:0/283949025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc1090b0 msgr2=0x7fa2fc1af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.474+0000 7fa2fa59c640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc1af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.474+0000 7fa2fa59c640 1 -- 172.21.15.33:0/283949025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc19f8c0 con 0x7fa2fc06a590 2024-04-24T23:20:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.474+0000 7fa2fad9d640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc10c360 0x7fa2fc19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.477+0000 7fa2fa59c640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc06a590 0x7fa2fc1af250 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400bdf0 tx=0x7fa2f4009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.480+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4019070 con 0x7fa2fc06a590 2024-04-24T23:20:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.480+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f4014920 con 0x7fa2fc06a590 2024-04-24T23:20:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.480+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40129b0 con 0x7fa2fc06a590 2024-04-24T23:20:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.480+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc19fbb0 con 0x7fa2fc06a590 2024-04-24T23:20:14.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.480+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc1a0130 con 0x7fa2fc06a590 2024-04-24T23:20:14.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.487+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc0052c0 con 0x7fa2fc06a590 2024-04-24T23:20:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.489+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa2f4012b50 con 0x7fa2fc06a590 2024-04-24T23:20:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.489+0000 7fa2eb7fe640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2e40a2d90 0x7fa2e40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.490+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa2f40ceb90 con 0x7fa2fc06a590 2024-04-24T23:20:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.491+0000 7fa2f9d9b640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2e40a2d90 0x7fa2e40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.491+0000 7fa2f9d9b640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2e40a2d90 0x7fa2e40a5250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc1b0320 tx=0x7fa2ec009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.501+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa2f409b700 con 0x7fa2fc06a590 2024-04-24T23:20:14.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.743+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2dc0055b0 con 0x7fa2fc06a590 2024-04-24T23:20:14.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.744+0000 7fa2eb7fe640 1 -- 172.21.15.33:0/283949025 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fa2f409b000 con 0x7fa2fc06a590 2024-04-24T23:20:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:20:14.746 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:20:14.746 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:20:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2e40a2d90 msgr2=0x7fa2e40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2e40a2d90 0x7fa2e40a5250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc1b0320 tx=0x7fa2ec009190 comp rx=0 tx=0).stop 2024-04-24T23:20:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc06a590 msgr2=0x7fa2fc1af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc06a590 0x7fa2fc1af250 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400bdf0 tx=0x7fa2f4009f70 comp rx=0 tx=0).stop 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 shutdown_connections 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa2e40a2d90 0x7fa2e40a5250 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2fc10c360 0x7fa2fc19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2fc1090b0 0x7fa2fc1af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 --2- 172.21.15.33:0/283949025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2fc06a590 0x7fa2fc1af250 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 >> 172.21.15.33:0/283949025 conn(0x7fa2fc06e980 msgr2=0x7fa2fc1112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 shutdown_connections 2024-04-24T23:20:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.746+0000 7fa300eda640 1 -- 172.21.15.33:0/283949025 wait complete. 2024-04-24T23:20:14.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:20:14.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 -- 172.21.15.33:0/2715473711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4100680 msgr2=0x7fb0a4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2715473711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4100680 0x7fb0a4100aa0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb094009900 tx=0x7fb09402a410 comp rx=0 tx=0).stop 2024-04-24T23:20:14.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 -- 172.21.15.33:0/2715473711 shutdown_connections 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2715473711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a4102230 0x7fb0a4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2715473711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a4101870 0x7fb0a4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2715473711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4100680 0x7fb0a4100aa0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 -- 172.21.15.33:0/2715473711 >> 172.21.15.33:0/2715473711 conn(0x7fb0a40fbbf0 msgr2=0x7fb0a40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 -- 172.21.15.33:0/2715473711 shutdown_connections 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 -- 172.21.15.33:0/2715473711 wait complete. 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 Processor -- start 2024-04-24T23:20:14.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.869+0000 7fb0aa961640 1 -- start start 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0aa961640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a4100680 0x7fb0a41958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0aa961640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a4101870 0x7fb0a4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0aa961640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 0x7fb0a419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0aa961640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b200 con 0x7fb0a4102230 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0aa961640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b080 con 0x7fb0a4100680 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0aa961640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b380 con 0x7fb0a4101870 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 0x7fb0a419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 0x7fb0a419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58778/0 (socket says 172.21.15.33:58778) 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a4100680 0x7fb0a41958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 -- 172.21.15.33:0/2197005249 learned_addr learned my addr 172.21.15.33:0/2197005249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a4101870 0x7fb0a4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 -- 172.21.15.33:0/2197005249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a4101870 msgr2=0x7fb0a4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a4101870 0x7fb0a4195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 -- 172.21.15.33:0/2197005249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a4100680 msgr2=0x7fb0a41958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a4100680 0x7fb0a41958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 -- 172.21.15.33:0/2197005249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a419d4e0 con 0x7fb0a4102230 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.870+0000 7fb0a8efe640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 0x7fb0a419ce70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00eb10 tx=0x7fb09c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.871+0000 7fb0a17fa640 1 -- 172.21.15.33:0/2197005249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c004b20 con 0x7fb0a4102230 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.871+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a419d7d0 con 0x7fb0a4102230 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.871+0000 7fb0a17fa640 1 -- 172.21.15.33:0/2197005249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c004cc0 con 0x7fb0a4102230 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.871+0000 7fb0a17fa640 1 -- 172.21.15.33:0/2197005249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c010760 con 0x7fb0a4102230 2024-04-24T23:20:14.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.871+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a419dcd0 con 0x7fb0a4102230 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a17fa640 1 -- 172.21.15.33:0/2197005249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb09c005020 con 0x7fb0a4102230 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 Processor -- start 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 -- start start 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3062650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3067720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f306a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2fc6670 con 0x55b3f3067d90 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2f6e970 con 0x55b3f3062c20 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2fc2630 con 0x55b3f3062230 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f306a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3067720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3067720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56756/0 (socket says 172.21.15.33:56756) 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 -- 172.21.15.33:0/1766508072 learned_addr learned my addr 172.21.15.33:0/1766508072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a17fa640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb08c0a4da0 0x7fb08c0a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3062650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a17fa640 1 -- 172.21.15.33:0/2197005249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb09c0cf980 con 0x7fb0a4102230 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a8efe640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb08c0a4da0 0x7fb08c0a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 -- 172.21.15.33:0/1766508072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 msgr2=0x55b3f3062650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3062650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 -- 172.21.15.33:0/1766508072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 msgr2=0x55b3f306a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f306a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.874+0000 7fb0a3fff640 1 -- 172.21.15.33:0/1766508072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f2f92400 con 0x55b3f3062c20 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.875+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f306a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.875+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3067720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb094031610 tx=0x7fb094005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228b58cbcab8740e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.875+0000 7fb0a8efe640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb08c0a4da0 0x7fb08c0a7260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0a4590 tx=0x7fb09c00efd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.875+0000 7fb08a7fc640 1 -- 172.21.15.33:0/1766508072 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094040070 con 0x55b3f3062c20 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.875+0000 7fb08a7fc640 1 -- 172.21.15.33:0/1766508072 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb094004030 con 0x55b3f3062c20 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.875+0000 7fb08a7fc640 1 -- 172.21.15.33:0/1766508072 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094035040 con 0x55b3f3062c20 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 -- 172.21.15.33:0/1766508072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 msgr2=0x55b3f3067720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3067720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb094031610 tx=0x7fb094005de0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 -- 172.21.15.33:0/1766508072 shutdown_connections 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f306a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3067720 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/1766508072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3062650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 -- 172.21.15.33:0/1766508072 >> 172.21.15.33:0/1766508072 conn(0x55b3f305d8a0 msgr2=0x55b3f305dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 -- 172.21.15.33:0/1766508072 shutdown_connections 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 -- 172.21.15.33:0/1766508072 wait complete. 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 -- start start 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.876+0000 7fb0ad66e740 1 Processor -- start 2024-04-24T23:20:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062230 0x55b3f306c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f306c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f30738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3f306ace0 con 0x55b3f3067d90 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2f82050 con 0x55b3f3062230 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3f306ae60 con 0x55b3f3062c20 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0ad66e740 10 client.0 _ll_register_callbacks cb 0x55b3f2cd9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f30738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f306c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f30738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58796/0 (socket says 172.21.15.33:58796) 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a37fe640 1 -- 172.21.15.33:0/2894021412 learned_addr learned my addr 172.21.15.33:0/2894021412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062230 0x55b3f306c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 -- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062230 msgr2=0x55b3f306c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062230 0x55b3f306c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 -- 172.21.15.33:0/2894021412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 msgr2=0x55b3f30738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f30738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 -- 172.21.15.33:0/2894021412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f2f53dc0 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f30738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f306c890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55b3f305e770 tx=0x7fb094009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d6d9c36f8a6059 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.877+0000 7fb0837fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0837fe640 1 -- 172.21.15.33:0/2894021412 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09404c070 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0837fe640 1 -- 172.21.15.33:0/2894021412 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0940043e0 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb094004030 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0837fe640 1 -- 172.21.15.33:0/2894021412 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb094058050 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 10 client.26647 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3f2cda3f0 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0837fe640 1 -- 172.21.15.33:0/2894021412 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb094004030 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0837fe640 1 -- 172.21.15.33:0/2894021412 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0940035f0 con 0x55b3f3062c20 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 10 client.26647 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 10 client.26647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:20:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 20 client.26647 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f3077dc0 0x55b3f307a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 4 client.26647 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.878+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3f2fd3bb0 con 0x55b3f3077dc0 2024-04-24T23:20:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.879+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f3077dc0 0x55b3f307a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.879+0000 7fb0837fe640 10 client.26647 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.879+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f3077dc0 0x55b3f307a1b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1fb006d43c9a071 server_cookie=9a84618a68307b93 in_seq=0 out_seq=0 2024-04-24T23:20:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0837fe640 1 -- 172.21.15.33:0/2894021412 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3f2fd3bb0 con 0x55b3f3077dc0 2024-04-24T23:20:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0837fe640 10 client.26647 handle_command_reply: tid=0 2024-04-24T23:20:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 client.26647 shutdown 2024-04-24T23:20:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 msgr2=0x55b3f306c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f306c890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55b3f305e770 tx=0x7fb094009590 comp rx=0 tx=0).stop 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 shutdown_connections 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f3077dc0 0x55b3f307a1b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f30738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0837fe640 0 client.26647 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f306c890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/2894021412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062230 0x55b3f306c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 >> 172.21.15.33:0/2894021412 conn(0x55b3f305d8a0 msgr2=0x55b3f305cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 shutdown_connections 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 1 -- 172.21.15.33:0/2894021412 wait complete. 2024-04-24T23:20:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.880+0000 7fb0ad66e740 20 client.26647 trim_cache size 0 max 16384 2024-04-24T23:20:14.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 Processor -- start 2024-04-24T23:20:14.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 -- start start 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 0x55b3f3063690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f3073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3067d90 0x55b3f3063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41aaac0 con 0x55b3f3062230 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41a8390 con 0x55b3f3067d90 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41a61e0 con 0x55b3f3062c20 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.895+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 0x55b3f3063690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 0x55b3f3063690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58804/0 (socket says 172.21.15.33:58804) 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 -- 172.21.15.33:0/4074918477 learned_addr learned my addr 172.21.15.33:0/4074918477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f3073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3067d90 0x55b3f3063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 -- 172.21.15.33:0/4074918477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 msgr2=0x55b3f3073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f3073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 -- 172.21.15.33:0/4074918477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3067d90 msgr2=0x55b3f3063bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3067d90 0x55b3f3063bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 -- 172.21.15.33:0/4074918477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f3067a70 con 0x55b3f3062230 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f3073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 0x55b3f3063690 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb098004650 tx=0x7fb0980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f79db799493939 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb089ffb640 1 -- 172.21.15.33:0/4074918477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09801a040 con 0x55b3f3062230 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb089ffb640 1 -- 172.21.15.33:0/4074918477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb098004c80 con 0x55b3f3062230 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.896+0000 7fb089ffb640 1 -- 172.21.15.33:0/4074918477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0980052f0 con 0x55b3f3062230 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 -- 172.21.15.33:0/4074918477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 msgr2=0x55b3f3063690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 0x55b3f3063690 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb098004650 tx=0x7fb0980046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 -- 172.21.15.33:0/4074918477 shutdown_connections 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3067d90 0x55b3f3063bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062c20 0x55b3f3073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/4074918477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3062230 0x55b3f3063690 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 -- 172.21.15.33:0/4074918477 >> 172.21.15.33:0/4074918477 conn(0x55b3f305d8a0 msgr2=0x55b3f305cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 -- 172.21.15.33:0/4074918477 shutdown_connections 2024-04-24T23:20:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 -- 172.21.15.33:0/4074918477 wait complete. 2024-04-24T23:20:14.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.897+0000 7fb0ad66e740 1 -- start start 2024-04-24T23:20:14.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 Processor -- start 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f3072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41a6850 con 0x55b3f3067d90 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41a7cc0 con 0x55b3f3062c20 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41a6f20 con 0x55b3f3062230 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0ad66e740 10 client.0 _ll_register_callbacks cb 0x7fb098009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f3072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56782/0 (socket says 172.21.15.33:56782) 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 -- 172.21.15.33:0/854696921 learned_addr learned my addr 172.21.15.33:0/854696921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 -- 172.21.15.33:0/854696921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 msgr2=0x55b3f3070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 -- 172.21.15.33:0/854696921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 msgr2=0x55b3f3072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f3072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.898+0000 7fb0a3fff640 1 -- 172.21.15.33:0/854696921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f2da47d0 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0a3fff640 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3071320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb09402a450 tx=0x7fb094003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad90b8983a4de329 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0837fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0837fe640 1 -- 172.21.15.33:0/854696921 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09404c040 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0837fe640 1 -- 172.21.15.33:0/854696921 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0940318b0 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb098004c80 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0837fe640 1 -- 172.21.15.33:0/854696921 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb094040030 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 10 client.26669 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3f3073a80 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0837fe640 1 -- 172.21.15.33:0/854696921 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb098004c80 con 0x55b3f3062c20 2024-04-24T23:20:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0837fe640 1 -- 172.21.15.33:0/854696921 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb094051440 con 0x55b3f3062c20 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 10 client.26669 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 10 client.26669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 20 client.26669 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f30de730 0x55b3f30e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 4 client.26669 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3f306c140 con 0x55b3f30de730 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.899+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f30de730 0x55b3f30e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.900+0000 7fb0837fe640 10 client.26669 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.900+0000 7fb0a37fe640 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f30de730 0x55b3f30e0b50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc1aaffc47c886c0 server_cookie=90d9fdf96bfc76bf in_seq=0 out_seq=0 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.901+0000 7fb0837fe640 1 -- 172.21.15.33:0/854696921 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3f306c140 con 0x55b3f30de730 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.901+0000 7fb0837fe640 10 client.26669 handle_command_reply: tid=0 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.901+0000 7fb0ad66e740 1 client.26669 shutdown 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 msgr2=0x55b3f3071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3071320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb09402a450 tx=0x7fb094003e10 comp rx=0 tx=0).stop 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 shutdown_connections 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b3f30de730 0x55b3f30e0b50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3f3067d90 0x55b3f3072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b3f3062c20 0x55b3f3071320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0837fe640 0 client.26669 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 --2- 172.21.15.33:0/854696921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3f3062230 0x55b3f3070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 >> 172.21.15.33:0/854696921 conn(0x55b3f305d8a0 msgr2=0x55b3f305c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 shutdown_connections 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 1 -- 172.21.15.33:0/854696921 wait complete. 2024-04-24T23:20:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0ad66e740 20 client.26669 trim_cache size 0 max 16384 2024-04-24T23:20:14.911 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb08c0a4da0 msgr2=0x7fb08c0a7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb08c0a4da0 0x7fb08c0a7260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0a4590 tx=0x7fb09c00efd0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 msgr2=0x7fb0a419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 0x7fb0a419ce70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00eb10 tx=0x7fb09c00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 shutdown_connections 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.902+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb08c0a4da0 0x7fb08c0a7260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.903+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a4102230 0x7fb0a419ce70 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.903+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a4101870 0x7fb0a4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.903+0000 7fb0aa961640 1 --2- 172.21.15.33:0/2197005249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a4100680 0x7fb0a41958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.903+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 >> 172.21.15.33:0/2197005249 conn(0x7fb0a40fbbf0 msgr2=0x7fb0a4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.903+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 shutdown_connections 2024-04-24T23:20:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:14.903+0000 7fb0aa961640 1 -- 172.21.15.33:0/2197005249 wait complete. 2024-04-24T23:20:14.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.079+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/2754996336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0102980 msgr2=0x7fa4b0104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.079+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/2754996336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0102980 0x7fa4b0104df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009860 tx=0x7fa4ac02a340 comp rx=0 tx=0).stop 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.080+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/2754996336 shutdown_connections 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.080+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/2754996336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0105330 0x7fa4b01057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.080+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/2754996336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0102980 0x7fa4b0104df0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.080+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/2754996336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0100010 0x7fa4b0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.080+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/2754996336 >> 172.21.15.33:0/2754996336 conn(0x7fa4b00fbc30 msgr2=0x7fa4b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.081+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/2754996336 shutdown_connections 2024-04-24T23:20:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.081+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/2754996336 wait complete. 2024-04-24T23:20:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.082+0000 7fa4b7b9e640 1 Processor -- start 2024-04-24T23:20:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.083+0000 7fa4b7b9e640 1 -- start start 2024-04-24T23:20:15.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.084+0000 7fa4b7b9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0100010 0x7fa4b019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.084+0000 7fa4b7b9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0102980 0x7fa4b019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.084+0000 7fa4b7b9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0105330 0x7fa4b01a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.084+0000 7fa4b7b9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01139f0 con 0x7fa4b0100010 2024-04-24T23:20:15.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.084+0000 7fa4b7b9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0113870 con 0x7fa4b0102980 2024-04-24T23:20:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.084+0000 7fa4b7b9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0113b70 con 0x7fa4b0105330 2024-04-24T23:20:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.085+0000 7fa4b5139640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0102980 0x7fa4b019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.085+0000 7fa4b5139640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0102980 0x7fa4b019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56808/0 (socket says 172.21.15.33:56808) 2024-04-24T23:20:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.085+0000 7fa4b5139640 1 -- 172.21.15.33:0/3034537448 learned_addr learned my addr 172.21.15.33:0/3034537448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.085+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0100010 0x7fa4b019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.085+0000 7fa4b613b640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0105330 0x7fa4b01a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.086+0000 7fa4b593a640 1 -- 172.21.15.33:0/3034537448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0105330 msgr2=0x7fa4b01a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.086+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0105330 0x7fa4b01a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.086+0000 7fa4b593a640 1 -- 172.21.15.33:0/3034537448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0102980 msgr2=0x7fa4b019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.086+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0102980 0x7fa4b019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.086+0000 7fa4b593a640 1 -- 172.21.15.33:0/3034537448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b01a5c80 con 0x7fa4b0100010 2024-04-24T23:20:15.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.087+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0100010 0x7fa4b019e060 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800ec80 tx=0x7fa4a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.088+0000 7fa4a6ffd640 1 -- 172.21.15.33:0/3034537448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8003b70 con 0x7fa4b0100010 2024-04-24T23:20:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.088+0000 7fa4a6ffd640 1 -- 172.21.15.33:0/3034537448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a8003d10 con 0x7fa4b0100010 2024-04-24T23:20:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.088+0000 7fa4a6ffd640 1 -- 172.21.15.33:0/3034537448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80108a0 con 0x7fa4b0100010 2024-04-24T23:20:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.089+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b01a5f70 con 0x7fa4b0100010 2024-04-24T23:20:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.089+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01a6320 con 0x7fa4b0100010 2024-04-24T23:20:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.093+0000 7fa4ba989740 1 Processor -- start 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.093+0000 7fa4ba989740 1 -- start start 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4a6ffd640 1 -- 172.21.15.33:0/3034537448 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa4a8004000 con 0x7fa4b0100010 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6c20 0x55a8f4dab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4dabd90 0x55a8f4dae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4ba989740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a8f4d0a670 con 0x55a8f4da6c20 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4ba989740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a8f4cb2970 con 0x55a8f4dabd90 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.094+0000 7fa4ba989740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a8f4d06630 con 0x55a8f4da6230 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.095+0000 7fa4b613b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.095+0000 7fa4b593a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6c20 0x55a8f4dab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.095+0000 7fa4b613b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56338/0 (socket says 172.21.15.33:56338) 2024-04-24T23:20:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.095+0000 7fa4b613b640 1 -- 172.21.15.33:0/1847734697 learned_addr learned my addr 172.21.15.33:0/1847734697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.095+0000 7fa4b5139640 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4dabd90 0x55a8f4dae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.095+0000 7fa4a6ffd640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa49c0a2ce0 0x7fa49c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4a6ffd640 1 -- 172.21.15.33:0/3034537448 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4a80cf6f0 con 0x7fa4b0100010 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b593a640 1 -- 172.21.15.33:0/1847734697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 msgr2=0x55a8f4da6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b593a640 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b593a640 1 -- 172.21.15.33:0/1847734697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4dabd90 msgr2=0x55a8f4dae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b593a640 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4dabd90 0x55a8f4dae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b593a640 1 -- 172.21.15.33:0/1847734697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f4cd6400 con 0x55a8f4da6c20 2024-04-24T23:20:15.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b613b640 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:15.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.096+0000 7fa4b613b640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa49c0a2ce0 0x7fa49c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.098+0000 7fa4b593a640 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6c20 0x55a8f4dab720 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80cce30 tx=0x7fa4a800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77aaa876ebbfec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.098+0000 7fa4b613b640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa49c0a2ce0 0x7fa49c0a51a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55a8f4da7480 tx=0x7fa4a00046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.101+0000 7fa49bfff640 1 -- 172.21.15.33:0/1847734697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a809b700 con 0x55a8f4da6c20 2024-04-24T23:20:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.101+0000 7fa49bfff640 1 -- 172.21.15.33:0/1847734697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a809c2c0 con 0x55a8f4da6c20 2024-04-24T23:20:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.101+0000 7fa49bfff640 1 -- 172.21.15.33:0/1847734697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a809b520 con 0x55a8f4da6c20 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.102+0000 7fa4ba989740 1 -- 172.21.15.33:0/1847734697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6c20 msgr2=0x55a8f4dab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.102+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6c20 0x55a8f4dab720 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80cce30 tx=0x7fa4a800eec0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.103+0000 7fa4ba989740 1 -- 172.21.15.33:0/1847734697 shutdown_connections 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.103+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4dabd90 0x55a8f4dae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.103+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6c20 0x55a8f4dab720 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.103+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1847734697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.103+0000 7fa4ba989740 1 -- 172.21.15.33:0/1847734697 >> 172.21.15.33:0/1847734697 conn(0x55a8f4da18a0 msgr2=0x55a8f4da1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.104+0000 7fa4ba989740 1 -- 172.21.15.33:0/1847734697 shutdown_connections 2024-04-24T23:20:15.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.104+0000 7fa4ba989740 1 -- 172.21.15.33:0/1847734697 wait complete. 2024-04-24T23:20:15.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.104+0000 7fa4ba989740 1 -- start start 2024-04-24T23:20:15.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.104+0000 7fa4ba989740 1 Processor -- start 2024-04-24T23:20:15.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.105+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6230 0x55a8f4db0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.105+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 0x55a8f4db0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.105+0000 7fa4b593a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6230 0x55a8f4db0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4b5139640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 0x55a8f4db0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4b5139640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 0x55a8f4db0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56352/0 (socket says 172.21.15.33:56352) 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4b5139640 1 -- 172.21.15.33:0/3291148051 learned_addr learned my addr 172.21.15.33:0/3291148051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4db78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a8f4daece0 con 0x55a8f4dabd90 2024-04-24T23:20:15.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a8f4cc6050 con 0x55a8f4da6230 2024-04-24T23:20:15.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.106+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a8f4daee60 con 0x55a8f4da6c20 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4b5139640 1 -- 172.21.15.33:0/3291148051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6230 msgr2=0x55a8f4db0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4b5139640 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6230 0x55a8f4db0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4b5139640 1 -- 172.21.15.33:0/3291148051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 msgr2=0x55a8f4db78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4b5139640 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4db78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4b5139640 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f4c97dc0 con 0x55a8f4da6c20 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4b5139640 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 0x55a8f4db0890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009980 tx=0x7fa4ac030a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac86fc4af2220cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.107+0000 7fa4ba989740 10 client.0 _ll_register_callbacks cb 0x55a8f4a1dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.108+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a809c2c0 con 0x55a8f4da6c20 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.109+0000 7fa4997fa640 10 client.26665 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.109+0000 7fa4997fa640 1 -- 172.21.15.33:0/3291148051 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac041070 con 0x55a8f4da6c20 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.109+0000 7fa4997fa640 1 -- 172.21.15.33:0/3291148051 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4ac0035f0 con 0x55a8f4da6c20 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.109+0000 7fa4997fa640 1 -- 172.21.15.33:0/3291148051 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac04e050 con 0x55a8f4da6c20 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.109+0000 7fa4ba989740 10 client.26665 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.109+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55a8f4a1e3f0 con 0x55a8f4da6c20 2024-04-24T23:20:15.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.110+0000 7fa4997fa640 1 -- 172.21.15.33:0/3291148051 <== mon.1 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4ac03a040 con 0x55a8f4da6c20 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4ba989740 10 client.26665 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4ba989740 10 client.26665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4ba989740 20 client.26665 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4dbe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4ba989740 4 client.26665 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8f4d17bb0 con 0x55a8f4dbbf40 2024-04-24T23:20:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.111+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4dbe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.114+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4dbe3d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=960966c2ae4fe2d0 server_cookie=612b8b155ec39efc in_seq=0 out_seq=0 2024-04-24T23:20:15.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.114+0000 7fa4997fa640 10 client.26665 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.115+0000 7fa4997fa640 1 -- 172.21.15.33:0/3291148051 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8f4d17bb0 con 0x55a8f4dbbf40 2024-04-24T23:20:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.115+0000 7fa4997fa640 10 client.26665 handle_command_reply: tid=0 2024-04-24T23:20:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.115+0000 7fa4ba989740 1 client.26665 shutdown 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.116+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 msgr2=0x55a8f4db0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.116+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 0x55a8f4db0890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009980 tx=0x7fa4ac030a50 comp rx=0 tx=0).stop 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.116+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 shutdown_connections 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.116+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4dbe3d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.116+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4db78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4997fa640 0 client.26665 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6c20 0x55a8f4db0890 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3291148051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6230 0x55a8f4db0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 >> 172.21.15.33:0/3291148051 conn(0x55a8f4da18a0 msgr2=0x55a8f4da0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 shutdown_connections 2024-04-24T23:20:15.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4ba989740 1 -- 172.21.15.33:0/3291148051 wait complete. 2024-04-24T23:20:15.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.117+0000 7fa4ba989740 20 client.26665 trim_cache size 0 max 16384 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.133+0000 7fa4ba989740 1 Processor -- start 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.133+0000 7fa4ba989740 1 -- start start 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4da4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4ba989740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01b3340 con 0x55a8f4dabd90 2024-04-24T23:20:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4ba989740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01b0c10 con 0x55a8f4da6c20 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4ba989740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01aea60 con 0x55a8f4da6230 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b5139640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b613b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4da4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b613b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4da4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58846/0 (socket says 172.21.15.33:58846) 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b613b640 1 -- 172.21.15.33:0/3054941674 learned_addr learned my addr 172.21.15.33:0/3054941674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b593a640 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b5139640 1 -- 172.21.15.33:0/3054941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 msgr2=0x55a8f4da09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b5139640 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b5139640 1 -- 172.21.15.33:0/3054941674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 msgr2=0x55a8f4da4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b5139640 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4da4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.134+0000 7fa4b5139640 1 -- 172.21.15.33:0/3054941674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f4ae87d0 con 0x55a8f4da6c20 2024-04-24T23:20:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.135+0000 7fa4b5139640 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db7890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0045a0 tx=0x7fa4ac03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9785de614deed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.135+0000 7fa49b7fe640 1 -- 172.21.15.33:0/3054941674 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac057070 con 0x55a8f4da6c20 2024-04-24T23:20:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.135+0000 7fa49b7fe640 1 -- 172.21.15.33:0/3054941674 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a809c2c0 con 0x55a8f4da6c20 2024-04-24T23:20:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.135+0000 7fa49b7fe640 1 -- 172.21.15.33:0/3054941674 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac041040 con 0x55a8f4da6c20 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.135+0000 7fa4ba989740 1 -- 172.21.15.33:0/3054941674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 msgr2=0x55a8f4db7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.135+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db7890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0045a0 tx=0x7fa4ac03e000 comp rx=0 tx=0).stop 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/3054941674 shutdown_connections 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4dabd90 0x55a8f4da4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db7890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 --2- 172.21.15.33:0/3054941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4da6230 0x55a8f4da09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/3054941674 >> 172.21.15.33:0/3054941674 conn(0x55a8f4da18a0 msgr2=0x55a8f4da02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/3054941674 shutdown_connections 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/3054941674 wait complete. 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- start start 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 Processor -- start 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 0x55a8f4db4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4dabd90 0x55a8f4db64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4b593a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 0x55a8f4db4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4b593a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 0x55a8f4db4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58856/0 (socket says 172.21.15.33:58856) 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4b593a640 1 -- 172.21.15.33:0/1246757948 learned_addr learned my addr 172.21.15.33:0/1246757948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01af0d0 con 0x55a8f4da6230 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01b0540 con 0x55a8f4da6c20 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.136+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01af7a0 con 0x55a8f4dabd90 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b5139640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b613b640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4dabd90 0x55a8f4db64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4ba989740 10 client.0 _ll_register_callbacks cb 0x7fa4a0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b593a640 1 -- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4dabd90 msgr2=0x55a8f4db64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b593a640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4dabd90 0x55a8f4db64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b593a640 1 -- 172.21.15.33:0/1246757948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 msgr2=0x55a8f4db54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b593a640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b593a640 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f4db6ad0 con 0x55a8f4da6230 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b613b640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4dabd90 0x55a8f4db64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4b593a640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 0x55a8f4db4f90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa4a809c470 tx=0x7fa4a8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b3f7a7b1c45ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4997fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4997fa640 1 -- 172.21.15.33:0/1246757948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80a8070 con 0x55a8f4da6230 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a809c2c0 con 0x55a8f4da6230 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4997fa640 1 -- 172.21.15.33:0/1246757948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4a80d5300 con 0x55a8f4da6230 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4997fa640 1 -- 172.21.15.33:0/1246757948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4a80ba070 con 0x55a8f4da6230 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4ba989740 10 client.17160 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.137+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8f4e22580 con 0x55a8f4da6230 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4997fa640 1 -- 172.21.15.33:0/1246757948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a809ff70 con 0x55a8f4da6230 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4997fa640 1 -- 172.21.15.33:0/1246757948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4a80ab020 con 0x55a8f4da6230 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4ba989740 10 client.17160 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4ba989740 10 client.17160 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4ba989740 20 client.17160 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4e24a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4ba989740 4 client.17160 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa4a809c2c0 con 0x55a8f4dbbf40 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.138+0000 7fa4b5139640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4e24a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4b5139640 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4e24a90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a1ed1ae52375193 server_cookie=a69548b08b5c0c95 in_seq=0 out_seq=0 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4997fa640 10 client.17160 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4997fa640 1 -- 172.21.15.33:0/1246757948 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa4a809c2c0 con 0x55a8f4dbbf40 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4997fa640 10 client.17160 handle_command_reply: tid=0 2024-04-24T23:20:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 client.17160 shutdown 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 msgr2=0x55a8f4db4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 0x55a8f4db4f90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa4a809c470 tx=0x7fa4a8003f70 comp rx=0 tx=0).stop 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 shutdown_connections 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a8f4dbbf40 0x55a8f4e24a90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a8f4dabd90 0x55a8f4db64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a8f4da6c20 0x55a8f4db54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 --2- 172.21.15.33:0/1246757948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a8f4da6230 0x55a8f4db4f90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4997fa640 0 client.17160 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 >> 172.21.15.33:0/1246757948 conn(0x55a8f4da18a0 msgr2=0x55a8f4da0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 shutdown_connections 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 1 -- 172.21.15.33:0/1246757948 wait complete. 2024-04-24T23:20:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.139+0000 7fa4ba989740 20 client.17160 trim_cache size 0 max 16384 2024-04-24T23:20:15.148 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:20:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa49c0a2ce0 msgr2=0x7fa49c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa49c0a2ce0 0x7fa49c0a51a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55a8f4da7480 tx=0x7fa4a00046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0100010 msgr2=0x7fa4b019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0100010 0x7fa4b019e060 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800ec80 tx=0x7fa4a800c6a0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 shutdown_connections 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa49c0a2ce0 0x7fa49c0a51a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4b0105330 0x7fa4b01a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4b0102980 0x7fa4b019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 --2- 172.21.15.33:0/3034537448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4b0100010 0x7fa4b019e060 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 >> 172.21.15.33:0/3034537448 conn(0x7fa4b00fbc30 msgr2=0x7fa4b0100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 shutdown_connections 2024-04-24T23:20:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.140+0000 7fa4b7b9e640 1 -- 172.21.15.33:0/3034537448 wait complete. 2024-04-24T23:20:15.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 -- 172.21.15.33:0/2036997818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 msgr2=0x7fc9ac1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2036997818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac1045f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009900 tx=0x7fc99c02a410 comp rx=0 tx=0).stop 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 -- 172.21.15.33:0/2036997818 shutdown_connections 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2036997818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 0x7fc9ac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2036997818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 0x7fc9ac106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2036997818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac1045f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 -- 172.21.15.33:0/2036997818 >> 172.21.15.33:0/2036997818 conn(0x7fc9ac0fbbf0 msgr2=0x7fc9ac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 -- 172.21.15.33:0/2036997818 shutdown_connections 2024-04-24T23:20:15.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.262+0000 7fc9b2528640 1 -- 172.21.15.33:0/2036997818 wait complete. 2024-04-24T23:20:15.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 Processor -- start 2024-04-24T23:20:15.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 -- start start 2024-04-24T23:20:15.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 0x7fc9ac198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 0x7fc9ac19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d420 con 0x7fc9ac102200 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d2a0 con 0x7fc9ac104b30 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9b2528640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d5a0 con 0x7fc9ac107460 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.263+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58864/0 (socket says 172.21.15.33:58864) 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9abfff640 1 -- 172.21.15.33:0/2772900477 learned_addr learned my addr 172.21.15.33:0/2772900477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 0x7fc9ac198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 0x7fc9ac19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/2772900477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 msgr2=0x7fc9ac19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 0x7fc9ac19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 msgr2=0x7fc9ac197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/2772900477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac19f710 con 0x7fc9ac104b30 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9abfff640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 0x7fc9ac19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 0x7fc9ac198030 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000e9c0 tx=0x7fc9a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9a97fa640 1 -- 172.21.15.33:0/2772900477 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a000ccc0 con 0x7fc9ac104b30 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9a97fa640 1 -- 172.21.15.33:0/2772900477 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0010910 con 0x7fc9ac104b30 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac19fa00 con 0x7fc9ac104b30 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9a97fa640 1 -- 172.21.15.33:0/2772900477 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00188c0 con 0x7fc9ac104b30 2024-04-24T23:20:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.264+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac19ff50 con 0x7fc9ac104b30 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.266+0000 7fc9a97fa640 1 -- 172.21.15.33:0/2772900477 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc9a0010ac0 con 0x7fc9ac104b30 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 Processor -- start 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 -- start start 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfc640 0x55d514bfca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c01b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514c021a0 0x55d514c04630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d514b5c670 con 0x55d514c021a0 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d514b04970 con 0x55d514bfc640 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b5231740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d514b58630 con 0x55d514bfd030 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514c021a0 0x55d514c04630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b0ac5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfc640 0x55d514bfca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c01b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9a97fa640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9940a2b90 0x7fc9940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c01b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56406/0 (socket says 172.21.15.33:56406) 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b0ac5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfc640 0x55d514bfca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56850/0 (socket says 172.21.15.33:56850) 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9abfff640 1 -- 172.21.15.33:0/3860427753 learned_addr learned my addr 172.21.15.33:0/3860427753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9940a2b90 0x7fc9940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.267+0000 7fc9abfff640 1 -- 172.21.15.33:0/3860427753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfc640 msgr2=0x55d514bfca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9abfff640 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfc640 0x55d514bfca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9abfff640 1 -- 172.21.15.33:0/3860427753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514c021a0 msgr2=0x55d514c04630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9abfff640 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514c021a0 0x55d514c04630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9abfff640 1 -- 172.21.15.33:0/3860427753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d514b28400 con 0x55d514bfd030 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9a97fa640 1 -- 172.21.15.33:0/2772900477 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc9a00cf730 con 0x7fc9ac104b30 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9abfff640 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c01b30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009a20 tx=0x7fc99c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5115aa9f069408bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9940a2b90 0x7fc9940a5050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55d514bfd890 tx=0x7fc9a4004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9927fc640 1 -- 172.21.15.33:0/3860427753 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c035040 con 0x55d514bfd030 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9927fc640 1 -- 172.21.15.33:0/3860427753 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0319b0 con 0x55d514bfd030 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.268+0000 7fc9927fc640 1 -- 172.21.15.33:0/3860427753 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c004a00 con 0x55d514bfd030 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 -- 172.21.15.33:0/3860427753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 msgr2=0x55d514c01b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c01b30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009a20 tx=0x7fc99c005de0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 -- 172.21.15.33:0/3860427753 shutdown_connections 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514c021a0 0x55d514c04630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c01b30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3860427753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfc640 0x55d514bfca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 -- 172.21.15.33:0/3860427753 >> 172.21.15.33:0/3860427753 conn(0x55d514bf38a0 msgr2=0x55d514bf3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 -- 172.21.15.33:0/3860427753 shutdown_connections 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 -- 172.21.15.33:0/3860427753 wait complete. 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 -- start start 2024-04-24T23:20:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.269+0000 7fc9b5231740 1 Processor -- start 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfc640 0x55d514c06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 0x55d514c06c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514c0dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d514c050f0 con 0x55d514bfd030 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d514b18050 con 0x55d514c021a0 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d514c05270 con 0x55d514bfc640 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b5231740 10 client.0 _ll_register_callbacks cb 0x55d51486fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 0x55d514c06c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfc640 0x55d514c06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 0x55d514c06c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58882/0 (socket says 172.21.15.33:58882) 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/1775917222 learned_addr learned my addr 172.21.15.33:0/1775917222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514c0dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfc640 msgr2=0x55d514c06700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfc640 0x55d514c06700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/1775917222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 msgr2=0x55d514c0dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514c0dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 -- 172.21.15.33:0/1775917222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d514ae9dc0 con 0x55d514bfd030 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9abfff640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfc640 0x55d514c06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 0x55d514c06c40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0000c00 tx=0x7fc9a0009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbef09f36533ecf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.270+0000 7fc98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1775917222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00b2060 con 0x55d514bfd030 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1775917222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc9a00cfbe0 con 0x55d514bfd030 2024-04-24T23:20:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c0319b0 con 0x55d514bfd030 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1775917222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9a009ffb0 con 0x55d514bfd030 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 10 client.17172 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5148703f0 con 0x55d514bfd030 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1775917222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0319b0 con 0x55d514bfd030 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1775917222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a009fbb0 con 0x55d514bfd030 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 10 client.17172 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 20 client.17172 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c12590 0x55d514c14a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 4 client.17172 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.271+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d514b69bb0 con 0x55d514c12590 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.272+0000 7fc9abfff640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c12590 0x55d514c14a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.272+0000 7fc98b7fe640 10 client.17172 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.272+0000 7fc9abfff640 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c12590 0x55d514c14a00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=658f9a77578f3bd1 server_cookie=7c2d07cef0084bf in_seq=0 out_seq=0 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1775917222 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d514b69bb0 con 0x55d514c12590 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc98b7fe640 10 client.17172 handle_command_reply: tid=0 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 client.17172 shutdown 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 msgr2=0x55d514c06c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 0x55d514c06c40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0000c00 tx=0x7fc9a0009700 comp rx=0 tx=0).stop 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 shutdown_connections 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c12590 0x55d514c14a00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514c0dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfd030 0x55d514c06c40 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc98b7fe640 0 client.17172 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1775917222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfc640 0x55d514c06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 >> 172.21.15.33:0/1775917222 conn(0x55d514bf38a0 msgr2=0x55d514bf2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 shutdown_connections 2024-04-24T23:20:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 1 -- 172.21.15.33:0/1775917222 wait complete. 2024-04-24T23:20:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.273+0000 7fc9b5231740 20 client.17172 trim_cache size 0 max 16384 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.287+0000 7fc9b5231740 1 Processor -- start 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.287+0000 7fc9b5231740 1 -- start start 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c01750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514bfdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b5231740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1aab10 con 0x55d514bfc640 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b5231740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a83e0 con 0x55d514c021a0 2024-04-24T23:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b5231740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a6230 con 0x55d514bfd030 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c01750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c01750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58888/0 (socket says 172.21.15.33:58888) 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 -- 172.21.15.33:0/3820143011 learned_addr learned my addr 172.21.15.33:0/3820143011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514bfdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c0dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 -- 172.21.15.33:0/3820143011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 msgr2=0x55d514c0dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c0dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 -- 172.21.15.33:0/3820143011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 msgr2=0x55d514bfdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514bfdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 -- 172.21.15.33:0/3820143011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d51493a7d0 con 0x55d514bfc640 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc9abfff640 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c01750 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009a20 tx=0x7fc99c049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d2000872b8bf46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc991ffb640 1 -- 172.21.15.33:0/3820143011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c035040 con 0x55d514bfc640 2024-04-24T23:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc991ffb640 1 -- 172.21.15.33:0/3820143011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0054f0 con 0x55d514bfc640 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.288+0000 7fc991ffb640 1 -- 172.21.15.33:0/3820143011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c003e60 con 0x55d514bfc640 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 -- 172.21.15.33:0/3820143011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 msgr2=0x55d514c01750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c01750 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009a20 tx=0x7fc99c049040 comp rx=0 tx=0).stop 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 -- 172.21.15.33:0/3820143011 shutdown_connections 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514c021a0 0x55d514bfdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514bfd030 0x55d514c0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 --2- 172.21.15.33:0/3820143011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c01750 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 -- 172.21.15.33:0/3820143011 >> 172.21.15.33:0/3820143011 conn(0x55d514bf38a0 msgr2=0x55d514bf2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 -- 172.21.15.33:0/3820143011 shutdown_connections 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.289+0000 7fc9b5231740 1 -- 172.21.15.33:0/3820143011 wait complete. 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 -- start start 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 Processor -- start 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c0b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfd030 0x55d514c0bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514c021a0 0x55d514c0cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a68a0 con 0x55d514bfc640 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a7d10 con 0x55d514bfd030 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1a6f70 con 0x55d514c021a0 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b5231740 10 client.0 _ll_register_callbacks cb 0x7fc9a000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c0b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c0b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58896/0 (socket says 172.21.15.33:58896) 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 -- 172.21.15.33:0/1268079426 learned_addr learned my addr 172.21.15.33:0/1268079426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9b0ac5640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514c021a0 0x55d514c0cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfd030 0x55d514c0bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 -- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514c021a0 msgr2=0x55d514c0cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514c021a0 0x55d514c0cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 -- 172.21.15.33:0/1268079426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfd030 msgr2=0x55d514c0bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfd030 0x55d514c0bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.290+0000 7fc9abfff640 1 -- 172.21.15.33:0/1268079426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d514c0d280 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9abfff640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c0b740 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0092a0 tx=0x7fc99c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b12c3338d78781 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1268079426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c05f060 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1268079426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc99c031930 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c0054f0 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1268079426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc99c035040 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 10 client.17178 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d514c78e00 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1268079426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0054f0 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1268079426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc99c031bf0 con 0x55d514bfc640 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 10 client.17178 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 20 client.17178 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c791f0 0x55d514c7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 4 client.17178 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.291+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d514b63f00 con 0x55d514c791f0 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c791f0 0x55d514c7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc98b7fe640 10 client.17178 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9ab7fe640 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c791f0 0x55d514c7b5e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb38786a83f5221c server_cookie=57cacbbda6e27f93 in_seq=0 out_seq=0 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc98b7fe640 1 -- 172.21.15.33:0/1268079426 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d514b63f00 con 0x55d514c791f0 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc98b7fe640 10 client.17178 handle_command_reply: tid=0 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 client.17178 shutdown 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 msgr2=0x55d514c0b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c0b740 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0092a0 tx=0x7fc99c02a410 comp rx=0 tx=0).stop 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 shutdown_connections 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d514c791f0 0x55d514c7b5e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc98b7fe640 0 client.17178 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d514c021a0 0x55d514c0cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d514bfd030 0x55d514c0bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 --2- 172.21.15.33:0/1268079426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d514bfc640 0x55d514c0b740 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 >> 172.21.15.33:0/1268079426 conn(0x55d514bf38a0 msgr2=0x55d514bf6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 shutdown_connections 2024-04-24T23:20:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.292+0000 7fc9b5231740 1 -- 172.21.15.33:0/1268079426 wait complete. 2024-04-24T23:20:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b5231740 20 client.17178 trim_cache size 0 max 16384 2024-04-24T23:20:15.301 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9940a2b90 msgr2=0x7fc9940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9940a2b90 0x7fc9940a5050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55d514bfd890 tx=0x7fc9a4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 msgr2=0x7fc9ac198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 0x7fc9ac198030 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000e9c0 tx=0x7fc9a000ee80 comp rx=0 tx=0).stop 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 shutdown_connections 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc9940a2b90 0x7fc9940a5050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9ac107460 0x7fc9ac19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9ac104b30 0x7fc9ac198030 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 --2- 172.21.15.33:0/2772900477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9ac102200 0x7fc9ac197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 >> 172.21.15.33:0/2772900477 conn(0x7fc9ac0fbbf0 msgr2=0x7fc9ac102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 shutdown_connections 2024-04-24T23:20:15.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:15.293+0000 7fc9b2528640 1 -- 172.21.15.33:0/2772900477 wait complete. 2024-04-24T23:20:15.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 905292 12.65 MB/sec execute 260 sec latency 120.974 ms 2024-04-24T23:20:15.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:15 smithi033 ceph-mon[29201]: pgmap v598: 129 pgs: 129 active+clean; 275 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:20:15.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:15 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/283949025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:20:16.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:15 smithi038 ceph-mon[32450]: pgmap v598: 129 pgs: 129 active+clean; 275 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:20:16.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:15 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/283949025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:20:16.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:15 smithi117 ceph-mon[32529]: pgmap v598: 129 pgs: 129 active+clean; 275 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:20:16.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:15 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/283949025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:20:16.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:20:16.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:20:16.308 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:20:16.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 -- 172.21.15.33:0/3620986017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94102230 msgr2=0x7f4c94108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/3620986017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94102230 0x7f4c94108ac0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ca0 tx=0x7f4c8c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 -- 172.21.15.33:0/3620986017 shutdown_connections 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/3620986017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94102230 0x7f4c94108ac0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/3620986017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94101870 0x7f4c94101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/3620986017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 0x7f4c94100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 -- 172.21.15.33:0/3620986017 >> 172.21.15.33:0/3620986017 conn(0x7f4c940fbbf0 msgr2=0x7f4c940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 -- 172.21.15.33:0/3620986017 shutdown_connections 2024-04-24T23:20:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.418+0000 7f4c9bb16640 1 -- 172.21.15.33:0/3620986017 wait complete. 2024-04-24T23:20:16.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 Processor -- start 2024-04-24T23:20:16.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 -- start start 2024-04-24T23:20:16.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 0x7f4c94197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 0x7f4c94198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94102230 0x7f4c9419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410c300 con 0x7f4c94101870 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410c180 con 0x7f4c94100680 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9bb16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410c480 con 0x7f4c94102230 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c990b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 0x7f4c94198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 0x7f4c94197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c990b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 0x7f4c94198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49640/0 (socket says 172.21.15.33:49640) 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 0x7f4c94197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58986/0 (socket says 172.21.15.33:58986) 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c990b1640 1 -- 172.21.15.33:0/2086776863 learned_addr learned my addr 172.21.15.33:0/2086776863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.419+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94102230 0x7f4c9419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c990b1640 1 -- 172.21.15.33:0/2086776863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94102230 msgr2=0x7f4c9419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c990b1640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94102230 0x7f4c9419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c990b1640 1 -- 172.21.15.33:0/2086776863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 msgr2=0x7f4c94197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c990b1640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 0x7f4c94197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c990b1640 1 -- 172.21.15.33:0/2086776863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9419f6b0 con 0x7f4c94101870 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94102230 0x7f4c9419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c990b1640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 0x7f4c94198020 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400e9f0 tx=0x7f4c8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c8affd640 1 -- 172.21.15.33:0/2086776863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400cd80 con 0x7f4c94101870 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c8affd640 1 -- 172.21.15.33:0/2086776863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84005500 con 0x7f4c94101870 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9419f9a0 con 0x7f4c94101870 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c8affd640 1 -- 172.21.15.33:0/2086776863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84010690 con 0x7f4c94101870 2024-04-24T23:20:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.420+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9419fea0 con 0x7f4c94101870 2024-04-24T23:20:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.422+0000 7f4c8affd640 1 -- 172.21.15.33:0/2086776863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4c840046c0 con 0x7f4c94101870 2024-04-24T23:20:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.422+0000 7f4c9e81a740 1 Processor -- start 2024-04-24T23:20:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 -- start start 2024-04-24T23:20:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55706c1432c0 con 0x55706c1e9d00 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55706c0eb910 con 0x55706c1e41a0 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55706c13f280 con 0x55706c1e4b90 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9a0b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c8affd640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4c800a2b90 0x7f4c800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c990b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9a0b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1e45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58996/0 (socket says 172.21.15.33:58996) 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1220126741 learned_addr learned my addr 172.21.15.33:0/1220126741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36490/0 (socket says 172.21.15.33:36490) 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.423+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4c800a2b90 0x7f4c800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c8affd640 1 -- 172.21.15.33:0/2086776863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4c840cf8a0 con 0x7f4c94101870 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1220126741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 msgr2=0x55706c1e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1220126741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 msgr2=0x55706c1ec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1ec190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1220126741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706c0fcfb0 con 0x55706c1e41a0 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c990b1640 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c998b2640 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1e45c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c006010 tx=0x7f4c8c031e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e366ed25671e3716 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4c800a2b90 0x7f4c800a5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c047000 tx=0x7f4c8c049000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c7ffff640 1 -- 172.21.15.33:0/1220126741 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c04e070 con 0x55706c1e41a0 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c7ffff640 1 -- 172.21.15.33:0/1220126741 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8c036070 con 0x55706c1e41a0 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.424+0000 7f4c7ffff640 1 -- 172.21.15.33:0/1220126741 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c02f610 con 0x55706c1e41a0 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1220126741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 msgr2=0x55706c1e45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1e45c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c006010 tx=0x7f4c8c031e90 comp rx=0 tx=0).stop 2024-04-24T23:20:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1220126741 shutdown_connections 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1220126741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1e45c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1220126741 >> 172.21.15.33:0/1220126741 conn(0x55706c1a7950 msgr2=0x55706c1db810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1220126741 shutdown_connections 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1220126741 wait complete. 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 -- start start 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.425+0000 7f4c9e81a740 1 Processor -- start 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55706c1ecad0 con 0x55706c1e9d00 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55706c0fee90 con 0x55706c1e41a0 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55706c1ecc50 con 0x55706c1e4b90 2024-04-24T23:20:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9e81a740 10 client.0 _ll_register_callbacks cb 0x55706c13a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1ee4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59008/0 (socket says 172.21.15.33:59008) 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c998b2640 1 -- 172.21.15.33:0/1591006678 learned_addr learned my addr 172.21.15.33:0/1591006678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c990b1640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 msgr2=0x55706c1eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1591006678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 msgr2=0x55706c1ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/1591006678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706bf1f7e0 con 0x55706c1e9d00 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c990b1640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.426+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1f59e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c02ade0 tx=0x7f4c8c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86da5de5392da910 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/1591006678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c04e070 con 0x55706c1e9d00 2024-04-24T23:20:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/1591006678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4c8c004620 con 0x55706c1e9d00 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c8c036070 con 0x55706c1e9d00 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/1591006678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c8c004820 con 0x55706c1e9d00 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 10 client.17190 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55706bf7ff70 con 0x55706c1e9d00 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/1591006678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8c036070 con 0x55706c1e9d00 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/1591006678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4c8c030750 con 0x55706c1e9d00 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 10 client.17190 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 20 client.17190 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c1f9c80 0x55706c1fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 4 client.17190 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.427+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706c1175a0 con 0x55706c1f9c80 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.428+0000 7f4c998b2640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c1f9c80 0x55706c1fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.428+0000 7f4c7d7fa640 10 client.17190 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.428+0000 7f4c998b2640 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c1f9c80 0x55706c1fc0c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a17aad0a740afd5 server_cookie=bb278d37d6b5475a in_seq=0 out_seq=0 2024-04-24T23:20:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/1591006678 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55706c1175a0 con 0x55706c1f9c80 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c7d7fa640 10 client.17190 handle_command_reply: tid=0 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 client.17190 shutdown 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 msgr2=0x55706c1f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1f59e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c02ade0 tx=0x7f4c8c0095c0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 shutdown_connections 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c1f9c80 0x55706c1fc0c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e9d00 0x55706c1f59e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c7d7fa640 0 client.17190 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/1591006678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 >> 172.21.15.33:0/1591006678 conn(0x55706c1a7950 msgr2=0x55706c1da950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 shutdown_connections 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 1 -- 172.21.15.33:0/1591006678 wait complete. 2024-04-24T23:20:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.429+0000 7f4c9e81a740 20 client.17190 trim_cache size 0 max 16384 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.449+0000 7f4c9e81a740 1 Processor -- start 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 -- start start 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 0x55706c1e6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 0x55706c1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941aaae0 con 0x55706c1e41a0 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941a83b0 con 0x55706c1e9d00 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941a6200 con 0x55706c1e4b90 2024-04-24T23:20:16.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 0x55706c1e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9a0b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 0x55706c1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c990b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9a0b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 0x55706c1f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59014/0 (socket says 172.21.15.33:59014) 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9a0b3640 1 -- 172.21.15.33:0/2122972844 learned_addr learned my addr 172.21.15.33:0/2122972844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 0x55706c1e6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49674/0 (socket says 172.21.15.33:49674) 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c990b1640 1 -- 172.21.15.33:0/2122972844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 msgr2=0x55706c1f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c990b1640 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 0x55706c1f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c990b1640 1 -- 172.21.15.33:0/2122972844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 msgr2=0x55706c1e6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c990b1640 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 0x55706c1e6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c990b1640 1 -- 172.21.15.33:0/2122972844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706be17170 con 0x55706c1e4b90 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 0x55706c1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.450+0000 7f4c998b2640 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 0x55706c1e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c990b1640 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e6570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c8409ba70 tx=0x7f4c840b0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f3f28425a16010 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c7f7fe640 1 -- 172.21.15.33:0/2122972844 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8409b6d0 con 0x55706c1e4b90 2024-04-24T23:20:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c7f7fe640 1 -- 172.21.15.33:0/2122972844 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8409c470 con 0x55706c1e4b90 2024-04-24T23:20:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c7f7fe640 1 -- 172.21.15.33:0/2122972844 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840a4550 con 0x55706c1e4b90 2024-04-24T23:20:16.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 907284 12.63 MB/sec execute 261 sec latency 16.279 ms 2024-04-24T23:20:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c9e81a740 1 -- 172.21.15.33:0/2122972844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 msgr2=0x55706c1e6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e6570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c8409ba70 tx=0x7f4c840b0040 comp rx=0 tx=0).stop 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c9e81a740 1 -- 172.21.15.33:0/2122972844 shutdown_connections 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e9d00 0x55706c1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e4b90 0x55706c1e6570 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.451+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/2122972844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e41a0 0x55706c1e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- 172.21.15.33:0/2122972844 >> 172.21.15.33:0/2122972844 conn(0x55706c1a7950 msgr2=0x55706c1ee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- 172.21.15.33:0/2122972844 shutdown_connections 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- 172.21.15.33:0/2122972844 wait complete. 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- start start 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 Processor -- start 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 0x55706c1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e9d00 0x55706c1f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941a6870 con 0x55706c1e4b90 2024-04-24T23:20:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941a7ce0 con 0x55706c1e41a0 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941a6f40 con 0x55706c1e9d00 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.452+0000 7f4c9e81a740 10 client.0 _ll_register_callbacks cb 0x7f4c8c04c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 0x55706c1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c998b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 0x55706c1f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49682/0 (socket says 172.21.15.33:49682) 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 -- 172.21.15.33:0/3152175675 learned_addr learned my addr 172.21.15.33:0/3152175675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e9d00 0x55706c1f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 -- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e9d00 msgr2=0x55706c1f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e9d00 0x55706c1f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 -- 172.21.15.33:0/3152175675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 msgr2=0x55706c1f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 -- 172.21.15.33:0/3152175675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706c1f4870 con 0x55706c1e4b90 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c9a0b3640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e9d00 0x55706c1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c998b2640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c990b1640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 0x55706c1f32c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4c840049d0 tx=0x7f4c84005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd1091d36172bad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/3152175675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8409a070 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/3152175675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4c8409b3d0 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c8409c470 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/3152175675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c8409bd70 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c9e81a740 10 client.17196 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.453+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55706c1f5bb0 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/3152175675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c840a9020 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/3152175675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4c840b2070 con 0x55706c1e4b90 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c9e81a740 10 client.17196 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c9e81a740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c9e81a740 20 client.17196 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c2608c0 0x55706c262ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c9e81a740 4 client.17196 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55706c150800 con 0x55706c2608c0 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c998b2640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c2608c0 0x55706c262ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c998b2640 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c2608c0 0x55706c262ce0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfc7aef93e6c5f3 server_cookie=7ab9410cc5d83ae3 in_seq=0 out_seq=0 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.454+0000 7f4c7d7fa640 10 client.17196 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c7d7fa640 1 -- 172.21.15.33:0/3152175675 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55706c150800 con 0x55706c2608c0 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c7d7fa640 10 client.17196 handle_command_reply: tid=0 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c9e81a740 1 client.17196 shutdown 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 msgr2=0x55706c1f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 0x55706c1f32c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4c840049d0 tx=0x7f4c84005cc0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 shutdown_connections 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55706c2608c0 0x55706c262ce0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.455+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55706c1e9d00 0x55706c1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55706c1e4b90 0x55706c1f32c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9e81a740 1 --2- 172.21.15.33:0/3152175675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55706c1e41a0 0x55706c1f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c7d7fa640 0 client.17196 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 >> 172.21.15.33:0/3152175675 conn(0x55706c1a7950 msgr2=0x55706c1ee3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 shutdown_connections 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9e81a740 1 -- 172.21.15.33:0/3152175675 wait complete. 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9e81a740 20 client.17196 trim_cache size 0 max 16384 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "06ca523a-b7c0-4e8c-b989-e557178a97a8", 2024-04-24T23:20:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:20:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:20:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4c800a2b90 msgr2=0x7f4c800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4c800a2b90 0x7f4c800a5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c047000 tx=0x7f4c8c049000 comp rx=0 tx=0).stop 2024-04-24T23:20:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 msgr2=0x7f4c94198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 0x7f4c94198020 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400e9f0 tx=0x7f4c8400edf0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 shutdown_connections 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4c800a2b90 0x7f4c800a5050 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c94102230 0x7f4c9419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c94101870 0x7f4c94198020 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 --2- 172.21.15.33:0/2086776863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c94100680 0x7f4c94197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 >> 172.21.15.33:0/2086776863 conn(0x7f4c940fbbf0 msgr2=0x7f4c940688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 shutdown_connections 2024-04-24T23:20:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.456+0000 7f4c9bb16640 1 -- 172.21.15.33:0/2086776863 wait complete. 2024-04-24T23:20:16.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:20:16.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 -- 172.21.15.33:0/3958929451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 msgr2=0x7f8a6c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 --2- 172.21.15.33:0/3958929451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c100aa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c02a410 comp rx=0 tx=0).stop 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 -- 172.21.15.33:0/3958929451 shutdown_connections 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 --2- 172.21.15.33:0/3958929451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c102230 0x7f8a6c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 --2- 172.21.15.33:0/3958929451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c101870 0x7f8a6c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 --2- 172.21.15.33:0/3958929451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c100aa0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 -- 172.21.15.33:0/3958929451 >> 172.21.15.33:0/3958929451 conn(0x7f8a6c0fbbf0 msgr2=0x7f8a6c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 -- 172.21.15.33:0/3958929451 shutdown_connections 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.580+0000 7f8a723dc640 1 -- 172.21.15.33:0/3958929451 wait complete. 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 Processor -- start 2024-04-24T23:20:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 -- start start 2024-04-24T23:20:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c101870 0x7f8a6c196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c102230 0x7f8a6c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10c350 con 0x7f8a6c100680 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10c1d0 con 0x7f8a6c102230 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a723dc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10c4d0 con 0x7f8a6c101870 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c1969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49704/0 (socket says 172.21.15.33:49704) 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a6bfff640 1 -- 172.21.15.33:0/602532060 learned_addr learned my addr 172.21.15.33:0/602532060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c101870 0x7f8a6c196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.581+0000 7f8a70979640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c102230 0x7f8a6c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6bfff640 1 -- 172.21.15.33:0/602532060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c101870 msgr2=0x7f8a6c196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c101870 0x7f8a6c196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6bfff640 1 -- 172.21.15.33:0/602532060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c102230 msgr2=0x7f8a6c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c102230 0x7f8a6c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6bfff640 1 -- 172.21.15.33:0/602532060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c19e5f0 con 0x7f8a6c100680 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c101870 0x7f8a6c196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a70979640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c102230 0x7f8a6c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c1969d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c031440 tx=0x7f8a5c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a697fa640 1 -- 172.21.15.33:0/602532060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c03f030 con 0x7f8a6c100680 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a697fa640 1 -- 172.21.15.33:0/602532060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c003ce0 con 0x7f8a6c100680 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c19e880 con 0x7f8a6c100680 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a697fa640 1 -- 172.21.15.33:0/602532060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c035070 con 0x7f8a6c100680 2024-04-24T23:20:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.582+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c19ed70 con 0x7f8a6c100680 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.584+0000 7f8a697fa640 1 -- 172.21.15.33:0/602532060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8a5c042030 con 0x7f8a6c100680 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 Processor -- start 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 -- start start 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a697fa640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8a540a2b90 0x7f8a540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd015e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0163940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0166440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbd00c06b0 con 0x55fbd015ee40 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbd0068880 con 0x55fbd0163fb0 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a751c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fbd00bc670 con 0x55fbd015e450 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a70979640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8a540a2b90 0x7f8a540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0163940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49712/0 (socket says 172.21.15.33:49712) 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/57034534 learned_addr learned my addr 172.21.15.33:0/57034534 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a697fa640 1 -- 172.21.15.33:0/602532060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8a5c0efc60 con 0x7f8a6c100680 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a70979640 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/57034534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 msgr2=0x55fbd015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd015e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/57034534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 msgr2=0x55fbd0166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.585+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/57034534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbcfe9e7d0 con 0x55fbd015ee40 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0163940 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8a600114f0 tx=0x7f8a600119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d74eece1a017588 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a70979640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8a540a2b90 0x7f8a540a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8a64005fd0 tx=0x7f8a64013660 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a527fc640 1 -- 172.21.15.33:0/57034534 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6001e070 con 0x55fbd015ee40 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a527fc640 1 -- 172.21.15.33:0/57034534 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a60011030 con 0x55fbd015ee40 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a527fc640 1 -- 172.21.15.33:0/57034534 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60010670 con 0x55fbd015ee40 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a751c6740 1 -- 172.21.15.33:0/57034534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 msgr2=0x55fbd0163940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.586+0000 7f8a751c6740 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0163940 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8a600114f0 tx=0x7f8a600119b0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- 172.21.15.33:0/57034534 shutdown_connections 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0163940 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 --2- 172.21.15.33:0/57034534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- 172.21.15.33:0/57034534 >> 172.21.15.33:0/57034534 conn(0x55fbd01578b0 msgr2=0x55fbd0157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- 172.21.15.33:0/57034534 shutdown_connections 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- 172.21.15.33:0/57034534 wait complete. 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- start start 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 Processor -- start 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 0x55fbd0168870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0168db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd0163fb0 0x55fbd016fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbd0166f60 con 0x55fbd015ee40 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbd007bf50 con 0x55fbd015e450 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fbd01670e0 con 0x55fbd0163fb0 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a751c6740 10 client.0 _ll_register_callbacks cb 0x55fbd00b7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 0x55fbd0168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.587+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0168db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49716/0 (socket says 172.21.15.33:49716) 2024-04-24T23:20:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 0x55fbd0168870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:59066/0 (socket says 172.21.15.33:59066) 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/3567586600 learned_addr learned my addr 172.21.15.33:0/3567586600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a70979640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd0163fb0 0x55fbd016fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd0163fb0 msgr2=0x55fbd016fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd0163fb0 0x55fbd016fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/3567586600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 msgr2=0x55fbd0168870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 0x55fbd0168870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 -- 172.21.15.33:0/3567586600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbcfdd43f0 con 0x55fbd015ee40 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 0x55fbd0168870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0168db0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8a600058c0 tx=0x7f8a60011410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c045d3634fc093 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/3567586600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60026020 con 0x55fbd015ee40 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/3567586600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8a6000f5c0 con 0x55fbd015ee40 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a60011030 con 0x55fbd015ee40 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/3567586600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a60031050 con 0x55fbd015ee40 2024-04-24T23:20:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a751c6740 10 client.17208 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.588+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbcfd940f0 con 0x55fbd015ee40 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/3567586600 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a60011030 con 0x55fbd015ee40 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/3567586600 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8a60021030 con 0x55fbd015ee40 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a751c6740 10 client.17208 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a751c6740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a751c6740 20 client.17208 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a751c6740 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd015d1b0 0x55fbd015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a751c6740 4 client.17208 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbd00cdbf0 con 0x55fbd015d1b0 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd015d1b0 0x55fbd015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd015d1b0 0x55fbd015d610 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9725523bcc7ded server_cookie=ed8e1b7c63286d4c in_seq=0 out_seq=0 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.589+0000 7f8a4b7fe640 10 client.17208 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/3567586600 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbd00cdbf0 con 0x55fbd015d1b0 2024-04-24T23:20:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a4b7fe640 10 client.17208 handle_command_reply: tid=0 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 client.17208 shutdown 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 msgr2=0x55fbd0168db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0168db0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8a600058c0 tx=0x7f8a60011410 comp rx=0 tx=0).stop 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 shutdown_connections 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd015d1b0 0x55fbd015d610 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd0163fb0 0x55fbd016fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd0168db0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 --2- 172.21.15.33:0/3567586600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd015e450 0x55fbd0168870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 >> 172.21.15.33:0/3567586600 conn(0x55fbd01578b0 msgr2=0x55fbd0156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a4b7fe640 0 client.17208 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 shutdown_connections 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.590+0000 7f8a751c6740 1 -- 172.21.15.33:0/3567586600 wait complete. 2024-04-24T23:20:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.591+0000 7f8a751c6740 20 client.17208 trim_cache size 0 max 16384 2024-04-24T23:20:16.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 Processor -- start 2024-04-24T23:20:16.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 -- start start 2024-04-24T23:20:16.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 0x55fbd015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015ee40 0x55fbd016fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1aaad0 con 0x55fbd015e450 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a83a0 con 0x55fbd0163fb0 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a751c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a61f0 con 0x55fbd015ee40 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 0x55fbd015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a70979640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 0x55fbd015d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49730/0 (socket says 172.21.15.33:49730) 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a6bfff640 1 -- 172.21.15.33:0/818841806 learned_addr learned my addr 172.21.15.33:0/818841806 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.609+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015ee40 0x55fbd016fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6bfff640 1 -- 172.21.15.33:0/818841806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015ee40 msgr2=0x55fbd016fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015ee40 0x55fbd016fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6bfff640 1 -- 172.21.15.33:0/818841806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 msgr2=0x55fbd0160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6bfff640 1 -- 172.21.15.33:0/818841806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbd015d980 con 0x55fbd015e450 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015ee40 0x55fbd016fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 0x55fbd015d1b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aeb56e9358bbe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a51ffb640 1 -- 172.21.15.33:0/818841806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c049040 con 0x55fbd015e450 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a51ffb640 1 -- 172.21.15.33:0/818841806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c0c0480 con 0x55fbd015e450 2024-04-24T23:20:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.610+0000 7f8a51ffb640 1 -- 172.21.15.33:0/818841806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0c0170 con 0x55fbd015e450 2024-04-24T23:20:16.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 -- 172.21.15.33:0/818841806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 msgr2=0x55fbd015d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 0x55fbd015d1b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c0dc040 comp rx=0 tx=0).stop 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 -- 172.21.15.33:0/818841806 shutdown_connections 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd0160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015ee40 0x55fbd016fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 --2- 172.21.15.33:0/818841806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015e450 0x55fbd015d1b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 -- 172.21.15.33:0/818841806 >> 172.21.15.33:0/818841806 conn(0x55fbd01578b0 msgr2=0x55fbd0168870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 -- 172.21.15.33:0/818841806 shutdown_connections 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 -- 172.21.15.33:0/818841806 wait complete. 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 -- start start 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 Processor -- start 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.611+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd016fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd016c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd016cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a6860 con 0x55fbd015ee40 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a7cd0 con 0x55fbd0163fb0 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a6f30 con 0x55fbd015e450 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 10 client.0 _ll_register_callbacks cb 0x7f8a6001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd016fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a70979640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd016cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd016c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd016fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36554/0 (socket says 172.21.15.33:36554) 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 -- 172.21.15.33:0/773428342 learned_addr learned my addr 172.21.15.33:0/773428342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 -- 172.21.15.33:0/773428342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 msgr2=0x55fbd016cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd016cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 -- 172.21.15.33:0/773428342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 msgr2=0x55fbd016c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd016c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 -- 172.21.15.33:0/773428342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbd01d6a00 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a6bfff640 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd016fc30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0bbe80 tx=0x7f8a5c0b84d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=963d009024ee3f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/773428342 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c049050 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/773428342 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8a5c0dc0b0 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.612+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a5c0c0480 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/773428342 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a5c0ba220 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 10 client.26725 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbd01d7bb0 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/773428342 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c0c0480 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/773428342 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8a5c03bcc0 con 0x55fbd015e450 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 10 client.26725 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 20 client.26725 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd01d83a0 0x55fbd01da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 4 client.26725 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbd00c7f40 con 0x55fbd01d83a0 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd01d83a0 0x55fbd01da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a6b7fe640 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd01d83a0 0x55fbd01da7c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6c63ef9a7a16ba server_cookie=755e1fc697798383 in_seq=0 out_seq=0 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.613+0000 7f8a4b7fe640 10 client.26725 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a4b7fe640 1 -- 172.21.15.33:0/773428342 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fbd00c7f40 con 0x55fbd01d83a0 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a4b7fe640 10 client.26725 handle_command_reply: tid=0 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 client.26725 shutdown 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 msgr2=0x55fbd016fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd016fc30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0bbe80 tx=0x7f8a5c0b84d0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 shutdown_connections 2024-04-24T23:20:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fbd01d83a0 0x55fbd01da7c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbd0163fb0 0x55fbd016cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbd015ee40 0x55fbd016c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 --2- 172.21.15.33:0/773428342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fbd015e450 0x55fbd016fc30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a4b7fe640 0 client.26725 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 >> 172.21.15.33:0/773428342 conn(0x55fbd01578b0 msgr2=0x55fbd0168850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 shutdown_connections 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 1 -- 172.21.15.33:0/773428342 wait complete. 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a751c6740 20 client.26725 trim_cache size 0 max 16384 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:20:16.623 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: "06ca523a-b7c0-4e8c-b989-e557178a97a8": { 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "06ca523a-b7c0-4e8c-b989-e557178a97a8", 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:20:16.624 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8a540a2b90 msgr2=0x7f8a540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.614+0000 7f8a723dc640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8a540a2b90 0x7f8a540a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8a64005fd0 tx=0x7f8a64013660 comp rx=0 tx=0).stop 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 msgr2=0x7f8a6c1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c1969d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c031440 tx=0x7f8a5c032ce0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 shutdown_connections 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8a540a2b90 0x7f8a540a5050 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a6c102230 0x7f8a6c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c101870 0x7f8a6c196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 --2- 172.21.15.33:0/602532060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a6c100680 0x7f8a6c1969d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:16.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 >> 172.21.15.33:0/602532060 conn(0x7f8a6c0fbbf0 msgr2=0x7f8a6c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:16.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 shutdown_connections 2024-04-24T23:20:16.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:16.615+0000 7f8a723dc640 1 -- 172.21.15.33:0/602532060 wait complete. 2024-04-24T23:20:16.628 INFO:tasks.cephfs.filesystem:scrub status for tag:06ca523a-b7c0-4e8c-b989-e557178a97a8 - {'path': '/', 'tag': '06ca523a-b7c0-4e8c-b989-e557178a97a8', 'options': 'recursive,force'} 2024-04-24T23:20:17.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 909658 12.64 MB/sec execute 262 sec latency 13.328 ms 2024-04-24T23:20:17.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: pgmap v599: 129 pgs: 129 active+clean; 279 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-04-24T23:20:17.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:20:17.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:17.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:20:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:20:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:17 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: pgmap v599: 129 pgs: 129 active+clean; 279 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:17 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: pgmap v599: 129 pgs: 129 active+clean; 279 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-04-24T23:20:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:20:18.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:20:18.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:20:18.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:17 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:18.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 912625 12.66 MB/sec execute 263 sec latency 21.943 ms 2024-04-24T23:20:19.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 914880 12.64 MB/sec execute 264 sec latency 24.269 ms 2024-04-24T23:20:19.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:19 smithi033 ceph-mon[29201]: pgmap v600: 129 pgs: 129 active+clean; 280 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-04-24T23:20:19.979 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:19 smithi117 ceph-mon[32529]: pgmap v600: 129 pgs: 129 active+clean; 280 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-04-24T23:20:20.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:19 smithi038 ceph-mon[32450]: pgmap v600: 129 pgs: 129 active+clean; 280 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-04-24T23:20:20.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 916530 12.64 MB/sec execute 265 sec latency 96.783 ms 2024-04-24T23:20:21.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 918322 12.69 MB/sec execute 266 sec latency 54.506 ms 2024-04-24T23:20:21.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:21 smithi033 ceph-mon[29201]: pgmap v601: 129 pgs: 129 active+clean; 280 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:20:21.973 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:21 smithi038 ceph-mon[32450]: pgmap v601: 129 pgs: 129 active+clean; 280 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:20:22.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:21 smithi117 ceph-mon[32529]: pgmap v601: 129 pgs: 129 active+clean; 280 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:20:22.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 920970 12.69 MB/sec execute 267 sec latency 12.284 ms 2024-04-24T23:20:22.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:22 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:20:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:22 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:20:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:22 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:20:23.453 INFO:tasks.workunit.client.0.smithi033.stdout: 1 924012 12.72 MB/sec execute 268 sec latency 8.274 ms 2024-04-24T23:20:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:23 smithi033 ceph-mon[29201]: pgmap v602: 129 pgs: 129 active+clean; 282 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 306 op/s 2024-04-24T23:20:23.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:23 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:20:24.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:23 smithi038 ceph-mon[32450]: pgmap v602: 129 pgs: 129 active+clean; 282 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 306 op/s 2024-04-24T23:20:24.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:23 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:20:24.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:23 smithi117 ceph-mon[32529]: pgmap v602: 129 pgs: 129 active+clean; 282 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 306 op/s 2024-04-24T23:20:24.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:23 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:20:24.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 926208 12.70 MB/sec execute 269 sec latency 27.098 ms 2024-04-24T23:20:25.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 927784 12.69 MB/sec execute 270 sec latency 71.786 ms 2024-04-24T23:20:25.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:25 smithi033 ceph-mon[29201]: pgmap v603: 129 pgs: 129 active+clean; 282 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T23:20:26.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:25 smithi038 ceph-mon[32450]: pgmap v603: 129 pgs: 129 active+clean; 282 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T23:20:26.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:25 smithi117 ceph-mon[32529]: pgmap v603: 129 pgs: 129 active+clean; 282 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T23:20:26.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 930702 12.70 MB/sec execute 271 sec latency 12.506 ms 2024-04-24T23:20:27.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 932839 12.69 MB/sec execute 272 sec latency 12.101 ms 2024-04-24T23:20:27.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:27 smithi033 ceph-mon[29201]: pgmap v604: 129 pgs: 129 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 292 op/s 2024-04-24T23:20:28.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:27 smithi038 ceph-mon[32450]: pgmap v604: 129 pgs: 129 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 292 op/s 2024-04-24T23:20:28.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:27 smithi117 ceph-mon[32529]: pgmap v604: 129 pgs: 129 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 292 op/s 2024-04-24T23:20:28.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 935221 12.70 MB/sec execute 273 sec latency 12.059 ms 2024-04-24T23:20:29.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 938154 12.71 MB/sec execute 274 sec latency 32.395 ms 2024-04-24T23:20:29.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:29 smithi033 ceph-mon[29201]: pgmap v605: 129 pgs: 129 active+clean; 274 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-04-24T23:20:30.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:29 smithi038 ceph-mon[32450]: pgmap v605: 129 pgs: 129 active+clean; 274 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-04-24T23:20:30.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:29 smithi117 ceph-mon[32529]: pgmap v605: 129 pgs: 129 active+clean; 274 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-04-24T23:20:30.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 939661 12.69 MB/sec execute 275 sec latency 62.376 ms 2024-04-24T23:20:31.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 942123 12.69 MB/sec execute 276 sec latency 21.036 ms 2024-04-24T23:20:31.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:31 smithi033 ceph-mon[29201]: pgmap v606: 129 pgs: 129 active+clean; 273 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-04-24T23:20:32.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:31 smithi038 ceph-mon[32450]: pgmap v606: 129 pgs: 129 active+clean; 273 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-04-24T23:20:32.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:31 smithi117 ceph-mon[32529]: pgmap v606: 129 pgs: 129 active+clean; 273 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-04-24T23:20:32.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 945212 12.71 MB/sec execute 277 sec latency 23.389 ms 2024-04-24T23:20:33.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 947878 12.71 MB/sec execute 278 sec latency 13.623 ms 2024-04-24T23:20:33.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:33 smithi033 ceph-mon[29201]: pgmap v607: 129 pgs: 129 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 287 op/s 2024-04-24T23:20:34.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:33 smithi038 ceph-mon[32450]: pgmap v607: 129 pgs: 129 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 287 op/s 2024-04-24T23:20:34.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:33 smithi117 ceph-mon[32529]: pgmap v607: 129 pgs: 129 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 287 op/s 2024-04-24T23:20:34.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 949760 12.70 MB/sec execute 279 sec latency 18.009 ms 2024-04-24T23:20:35.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 952507 12.72 MB/sec execute 280 sec latency 66.258 ms 2024-04-24T23:20:36.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:35 smithi038 ceph-mon[32450]: pgmap v608: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-24T23:20:36.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:35 smithi117 ceph-mon[32529]: pgmap v608: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-24T23:20:36.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:35 smithi033 ceph-mon[29201]: pgmap v608: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-24T23:20:36.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 954754 12.70 MB/sec execute 281 sec latency 15.293 ms 2024-04-24T23:20:37.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 957169 12.71 MB/sec execute 282 sec latency 13.500 ms 2024-04-24T23:20:38.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:37 smithi038 ceph-mon[32450]: pgmap v609: 129 pgs: 129 active+clean; 282 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-04-24T23:20:38.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:37 smithi117 ceph-mon[32529]: pgmap v609: 129 pgs: 129 active+clean; 282 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-04-24T23:20:38.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:37 smithi033 ceph-mon[29201]: pgmap v609: 129 pgs: 129 active+clean; 282 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-04-24T23:20:38.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 959971 12.72 MB/sec execute 283 sec latency 22.703 ms 2024-04-24T23:20:39.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 963068 12.74 MB/sec execute 284 sec latency 21.409 ms 2024-04-24T23:20:40.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:39 smithi038 ceph-mon[32450]: pgmap v610: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-04-24T23:20:40.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:39 smithi117 ceph-mon[32529]: pgmap v610: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-04-24T23:20:40.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:39 smithi033 ceph-mon[29201]: pgmap v610: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-04-24T23:20:40.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 965169 12.72 MB/sec execute 285 sec latency 68.515 ms 2024-04-24T23:20:41.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 967128 12.72 MB/sec execute 286 sec latency 18.821 ms 2024-04-24T23:20:42.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:41 smithi038 ceph-mon[32450]: pgmap v611: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.9 MiB/s wr, 209 op/s 2024-04-24T23:20:42.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:41 smithi117 ceph-mon[32529]: pgmap v611: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.9 MiB/s wr, 209 op/s 2024-04-24T23:20:42.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:41 smithi033 ceph-mon[29201]: pgmap v611: 129 pgs: 129 active+clean; 285 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.9 MiB/s wr, 209 op/s 2024-04-24T23:20:42.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 970285 12.74 MB/sec execute 287 sec latency 6.933 ms 2024-04-24T23:20:43.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 972847 12.74 MB/sec execute 288 sec latency 12.017 ms 2024-04-24T23:20:44.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:43 smithi038 ceph-mon[32450]: pgmap v612: 129 pgs: 129 active+clean; 266 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-04-24T23:20:44.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:43 smithi117 ceph-mon[32529]: pgmap v612: 129 pgs: 129 active+clean; 266 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-04-24T23:20:44.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:43 smithi033 ceph-mon[29201]: pgmap v612: 129 pgs: 129 active+clean; 266 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-04-24T23:20:44.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 974887 12.74 MB/sec execute 289 sec latency 21.002 ms 2024-04-24T23:20:45.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 977439 12.74 MB/sec execute 290 sec latency 55.924 ms 2024-04-24T23:20:46.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:45 smithi038 ceph-mon[32450]: pgmap v613: 129 pgs: 129 active+clean; 263 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:20:46.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:45 smithi117 ceph-mon[32529]: pgmap v613: 129 pgs: 129 active+clean; 263 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:20:46.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:45 smithi033 ceph-mon[29201]: pgmap v613: 129 pgs: 129 active+clean; 263 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:20:46.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 979912 12.74 MB/sec execute 291 sec latency 22.483 ms 2024-04-24T23:20:46.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:20:46.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:20:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 -- 172.21.15.33:0/3705300019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4100680 msgr2=0x7f9ac4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 --2- 172.21.15.33:0/3705300019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4100680 0x7f9ac4100aa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009900 tx=0x7f9ab402a410 comp rx=0 tx=0).stop 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 -- 172.21.15.33:0/3705300019 shutdown_connections 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 --2- 172.21.15.33:0/3705300019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4102230 0x7f9ac4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 --2- 172.21.15.33:0/3705300019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4101870 0x7f9ac4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 --2- 172.21.15.33:0/3705300019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4100680 0x7f9ac4100aa0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 -- 172.21.15.33:0/3705300019 >> 172.21.15.33:0/3705300019 conn(0x7f9ac40fbbf0 msgr2=0x7f9ac40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 -- 172.21.15.33:0/3705300019 shutdown_connections 2024-04-24T23:20:46.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.736+0000 7f9aca666640 1 -- 172.21.15.33:0/3705300019 wait complete. 2024-04-24T23:20:46.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 Processor -- start 2024-04-24T23:20:46.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 -- start start 2024-04-24T23:20:46.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 0x7f9ac41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 0x7f9ac4196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4102230 0x7f9ac419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac410c300 con 0x7f9ac4101870 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac410c180 con 0x7f9ac4100680 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9aca666640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac410c480 con 0x7f9ac4102230 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 0x7f9ac41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 0x7f9ac4196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac8c03640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4102230 0x7f9ac419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 0x7f9ac41969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38450/0 (socket says 172.21.15.33:38450) 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 0x7f9ac4196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52620/0 (socket says 172.21.15.33:52620) 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac3fff640 1 -- 172.21.15.33:0/3523838481 learned_addr learned my addr 172.21.15.33:0/3523838481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.737+0000 7f9ac3fff640 1 -- 172.21.15.33:0/3523838481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4102230 msgr2=0x7f9ac419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4102230 0x7f9ac419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac3fff640 1 -- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 msgr2=0x7f9ac4196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 0x7f9ac4196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac3fff640 1 -- 172.21.15.33:0/3523838481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac40688d0 con 0x7f9ac4100680 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 0x7f9ac4196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 0x7f9ac41969d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4031610 tx=0x7f9ab4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac17fa640 1 -- 172.21.15.33:0/3523838481 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab403f030 con 0x7f9ac4100680 2024-04-24T23:20:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac17fa640 1 -- 172.21.15.33:0/3523838481 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4003ad0 con 0x7f9ac4100680 2024-04-24T23:20:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac4068b60 con 0x7f9ac4100680 2024-04-24T23:20:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9ac17fa640 1 -- 172.21.15.33:0/3523838481 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4035070 con 0x7f9ac4100680 2024-04-24T23:20:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.738+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac4069050 con 0x7f9ac4100680 2024-04-24T23:20:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9ac17fa640 1 -- 172.21.15.33:0/3523838481 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9ab4042030 con 0x7f9ac4100680 2024-04-24T23:20:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 Processor -- start 2024-04-24T23:20:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 -- start start 2024-04-24T23:20:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef834650 0x55b0ef834a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef839b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef83c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef7946b0 con 0x55b0ef83a1b0 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef73c880 con 0x55b0ef834650 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.741+0000 7f9acd370740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef790670 con 0x55b0ef835040 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef839b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef834650 0x55b0ef834a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef83c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef839b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36634/0 (socket says 172.21.15.33:36634) 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac3fff640 1 -- 172.21.15.33:0/2174783060 learned_addr learned my addr 172.21.15.33:0/2174783060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 -- 172.21.15.33:0/2174783060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 msgr2=0x55b0ef839b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef839b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac17fa640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9aac0a2b90 0x7f9aac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 -- 172.21.15.33:0/2174783060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 msgr2=0x55b0ef83c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef83c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 -- 172.21.15.33:0/2174783060 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef5727d0 con 0x55b0ef834650 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac17fa640 1 -- 172.21.15.33:0/3523838481 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ab40efab0 con 0x7f9ac4100680 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9aac0a2b90 0x7f9aac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.742+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef834650 0x55b0ef834a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009f30 tx=0x7f9abc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4336e24303a6fbe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9aaa7fc640 1 -- 172.21.15.33:0/2174783060 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc022070 con 0x55b0ef834650 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9aaa7fc640 1 -- 172.21.15.33:0/2174783060 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc012e40 con 0x55b0ef834650 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9aaa7fc640 1 -- 172.21.15.33:0/2174783060 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc016070 con 0x55b0ef834650 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 -- 172.21.15.33:0/2174783060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef834650 msgr2=0x55b0ef834a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9aac0a2b90 0x7f9aac0a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9abc01f000 tx=0x7f9abc022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef834650 0x55b0ef834a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009f30 tx=0x7f9abc010800 comp rx=0 tx=0).stop 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 -- 172.21.15.33:0/2174783060 shutdown_connections 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef83c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef839b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 --2- 172.21.15.33:0/2174783060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef834650 0x55b0ef834a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 -- 172.21.15.33:0/2174783060 >> 172.21.15.33:0/2174783060 conn(0x55b0ef82b8b0 msgr2=0x55b0ef82bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 -- 172.21.15.33:0/2174783060 shutdown_connections 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.743+0000 7f9acd370740 1 -- 172.21.15.33:0/2174783060 wait complete. 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 -- start start 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 Processor -- start 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef83ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 0x55b0ef83efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef846000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef83d160 con 0x55b0ef835040 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef74ff50 con 0x55b0ef83a1b0 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef83d2e0 con 0x55b0ef834650 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.744+0000 7f9acd370740 10 client.0 _ll_register_callbacks cb 0x55b0ef78b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 0x55b0ef83efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef83ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef83ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36642/0 (socket says 172.21.15.33:36642) 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac8c03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef846000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 0x55b0ef83efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52634/0 (socket says 172.21.15.33:52634) 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac3fff640 1 -- 172.21.15.33:0/2576007703 learned_addr learned my addr 172.21.15.33:0/2576007703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 -- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 msgr2=0x55b0ef83ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef83ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 -- 172.21.15.33:0/2576007703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 msgr2=0x55b0ef846000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef846000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 -- 172.21.15.33:0/2576007703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef4a83f0 con 0x55b0ef835040 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef83ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 0x55b0ef83efc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4197ee0 tx=0x7f9ab800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50d1e728c622b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9aa37fe640 1 -- 172.21.15.33:0/2576007703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8012110 con 0x55b0ef835040 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9aa37fe640 1 -- 172.21.15.33:0/2576007703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ab8004050 con 0x55b0ef835040 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0ef7a1bf0 con 0x55b0ef835040 2024-04-24T23:20:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9aa37fe640 1 -- 172.21.15.33:0/2576007703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ab8025050 con 0x55b0ef835040 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9acd370740 10 client.17226 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.745+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0ef4680f0 con 0x55b0ef835040 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9aa37fe640 1 -- 172.21.15.33:0/2576007703 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b0ef7a1bf0 con 0x55b0ef835040 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9aa37fe640 1 -- 172.21.15.33:0/2576007703 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ab8010430 con 0x55b0ef835040 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9acd370740 10 client.17226 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9acd370740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9acd370740 20 client.17226 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9acd370740 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef84a920 0x55b0ef84cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9acd370740 4 client.17226 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ef79bf40 con 0x55b0ef84a920 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef84a920 0x55b0ef84cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9aa37fe640 10 client.17226 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.746+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef84a920 0x55b0ef84cde0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00d68502044cf15 server_cookie=2b82004619b89629 in_seq=0 out_seq=0 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9aa37fe640 1 -- 172.21.15.33:0/2576007703 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0ef79bf40 con 0x55b0ef84a920 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9aa37fe640 10 client.17226 handle_command_reply: tid=0 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 client.17226 shutdown 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 msgr2=0x55b0ef83efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 0x55b0ef83efc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4197ee0 tx=0x7f9ab800f790 comp rx=0 tx=0).stop 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 shutdown_connections 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef84a920 0x55b0ef84cde0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef846000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef835040 0x55b0ef83efc0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 --2- 172.21.15.33:0/2576007703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef83ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9aa37fe640 0 client.17226 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 >> 172.21.15.33:0/2576007703 conn(0x55b0ef82b8b0 msgr2=0x55b0ef82aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 shutdown_connections 2024-04-24T23:20:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 1 -- 172.21.15.33:0/2576007703 wait complete. 2024-04-24T23:20:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.747+0000 7f9acd370740 20 client.17226 trim_cache size 0 max 16384 2024-04-24T23:20:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 Processor -- start 2024-04-24T23:20:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 -- start start 2024-04-24T23:20:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 0x55b0ef82a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef845fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef851690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41aaae0 con 0x55b0ef834650 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41a83b0 con 0x55b0ef83a1b0 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.766+0000 7f9acd370740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41a6200 con 0x55b0ef835040 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 0x55b0ef82a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 0x55b0ef82a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52642/0 (socket says 172.21.15.33:52642) 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac3fff640 1 -- 172.21.15.33:0/146292245 learned_addr learned my addr 172.21.15.33:0/146292245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef851690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef845fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 -- 172.21.15.33:0/146292245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 msgr2=0x55b0ef845fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef845fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 -- 172.21.15.33:0/146292245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 msgr2=0x55b0ef82a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 0x55b0ef82a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 -- 172.21.15.33:0/146292245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef81d090 con 0x55b0ef83a1b0 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 0x55b0ef82a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef851690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009f30 tx=0x7f9abc025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4c42b83431181b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9aa9ffb640 1 -- 172.21.15.33:0/146292245 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc016040 con 0x55b0ef83a1b0 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9aa9ffb640 1 -- 172.21.15.33:0/146292245 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc035070 con 0x55b0ef83a1b0 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.767+0000 7f9aa9ffb640 1 -- 172.21.15.33:0/146292245 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc01c860 con 0x55b0ef83a1b0 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 -- 172.21.15.33:0/146292245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 msgr2=0x55b0ef851690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef851690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009f30 tx=0x7f9abc025000 comp rx=0 tx=0).stop 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 -- 172.21.15.33:0/146292245 shutdown_connections 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef83a1b0 0x55b0ef851690 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef835040 0x55b0ef845fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 --2- 172.21.15.33:0/146292245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef834650 0x55b0ef82a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 -- 172.21.15.33:0/146292245 >> 172.21.15.33:0/146292245 conn(0x55b0ef82b8b0 msgr2=0x55b0ef82a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 -- 172.21.15.33:0/146292245 shutdown_connections 2024-04-24T23:20:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 -- 172.21.15.33:0/146292245 wait complete. 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.768+0000 7f9acd370740 1 -- start start 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 Processor -- start 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef843540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef835040 0x55b0ef843a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef844a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41a6870 con 0x55b0ef83a1b0 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41a7ce0 con 0x55b0ef835040 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41a6f40 con 0x55b0ef834650 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9acd370740 10 client.0 _ll_register_callbacks cb 0x7f9ab8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef843540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac8c03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef844a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef835040 0x55b0ef843a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac8c03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef844a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52654/0 (socket says 172.21.15.33:52654) 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac8c03640 1 -- 172.21.15.33:0/728824241 learned_addr learned my addr 172.21.15.33:0/728824241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef843540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36658/0 (socket says 172.21.15.33:36658) 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac37fe640 1 -- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 msgr2=0x55b0ef843540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef843540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac37fe640 1 -- 172.21.15.33:0/728824241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 msgr2=0x55b0ef844a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef844a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac37fe640 1 -- 172.21.15.33:0/728824241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef845080 con 0x55b0ef835040 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef843540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.769+0000 7f9ac8c03640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef844a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9ac37fe640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef835040 0x55b0ef843a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8010820 tx=0x7f9ab800ffb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329c464f9927b8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9aa37fe640 1 -- 172.21.15.33:0/728824241 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8010e60 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9abc035070 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9aa37fe640 1 -- 172.21.15.33:0/728824241 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ab8018040 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9aa37fe640 1 -- 172.21.15.33:0/728824241 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ab8010a40 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 10 client.26759 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0ef8b0f10 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9aa37fe640 1 -- 172.21.15.33:0/728824241 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc035070 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9aa37fe640 1 -- 172.21.15.33:0/728824241 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ab8003710 con 0x55b0ef835040 2024-04-24T23:20:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 10 client.26759 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 20 client.26759 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef8b17d0 0x55b0ef8b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 4 client.26759 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.770+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0ef7196f0 con 0x55b0ef8b17d0 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef8b17d0 0x55b0ef8b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9aa37fe640 10 client.26759 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9ac3fff640 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef8b17d0 0x55b0ef8b3bf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4052f21b0a1d245 server_cookie=ad02defbfaf22780 in_seq=0 out_seq=0 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9aa37fe640 1 -- 172.21.15.33:0/728824241 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0ef7196f0 con 0x55b0ef8b17d0 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9aa37fe640 10 client.26759 handle_command_reply: tid=0 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 client.26759 shutdown 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef835040 msgr2=0x55b0ef843a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef835040 0x55b0ef843a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8010820 tx=0x7f9ab800ffb0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 shutdown_connections 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b0ef8b17d0 0x55b0ef8b3bf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef83a1b0 0x55b0ef844a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9aa37fe640 0 client.26759 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef835040 0x55b0ef843a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 --2- 172.21.15.33:0/728824241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0ef834650 0x55b0ef843540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 >> 172.21.15.33:0/728824241 conn(0x55b0ef82b8b0 msgr2=0x55b0ef82aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 shutdown_connections 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 1 -- 172.21.15.33:0/728824241 wait complete. 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.771+0000 7f9acd370740 20 client.26759 trim_cache size 0 max 16384 2024-04-24T23:20:46.782 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:20:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:20:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:20:46.783 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9aac0a2b90 msgr2=0x7f9aac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9aac0a2b90 0x7f9aac0a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9abc01f000 tx=0x7f9abc022000 comp rx=0 tx=0).stop 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 msgr2=0x7f9ac41969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 0x7f9ac41969d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4031610 tx=0x7f9ab4004270 comp rx=0 tx=0).stop 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 shutdown_connections 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9aac0a2b90 0x7f9aac0a5050 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4102230 0x7f9ac419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac4101870 0x7f9ac4196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 --2- 172.21.15.33:0/3523838481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ac4100680 0x7f9ac41969d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 >> 172.21.15.33:0/3523838481 conn(0x7f9ac40fbbf0 msgr2=0x7f9ac4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 shutdown_connections 2024-04-24T23:20:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.772+0000 7f9aca666640 1 -- 172.21.15.33:0/3523838481 wait complete. 2024-04-24T23:20:46.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:20:46.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 -- 172.21.15.33:0/1827508881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780075720 msgr2=0x7fc780073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1827508881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780075720 0x7fc780073b70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc770009900 tx=0x7fc77002a480 comp rx=0 tx=0).stop 2024-04-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 -- 172.21.15.33:0/1827508881 shutdown_connections 2024-04-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1827508881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780109630 0x7fc78010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1827508881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc780074140 0x7fc7800745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1827508881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780075720 0x7fc780073b70 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 -- 172.21.15.33:0/1827508881 >> 172.21.15.33:0/1827508881 conn(0x7fc7800fccf0 msgr2=0x7fc7800ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 -- 172.21.15.33:0/1827508881 shutdown_connections 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.894+0000 7fc7853a2640 1 -- 172.21.15.33:0/1827508881 wait complete. 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 Processor -- start 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 -- start start 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 0x7fc780199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780075720 0x7fc78019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc780109630 0x7fc7801a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc78010f610 con 0x7fc780074140 2024-04-24T23:20:46.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc78010f490 con 0x7fc780109630 2024-04-24T23:20:46.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc7853a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc78010f790 con 0x7fc780075720 2024-04-24T23:20:46.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 0x7fc780199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 0x7fc780199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52672/0 (socket says 172.21.15.33:52672) 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 -- 172.21.15.33:0/1187670959 learned_addr learned my addr 172.21.15.33:0/1187670959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc777fff640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780075720 0x7fc78019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77f7fe640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc780109630 0x7fc7801a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 -- 172.21.15.33:0/1187670959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780075720 msgr2=0x7fc78019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780075720 0x7fc78019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 -- 172.21.15.33:0/1187670959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc780109630 msgr2=0x7fc7801a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc780109630 0x7fc7801a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc77effd640 1 -- 172.21.15.33:0/1187670959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7801a1890 con 0x7fc780074140 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.895+0000 7fc777fff640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780075720 0x7fc78019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.896+0000 7fc77effd640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 0x7fc780199c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc770009230 tx=0x7fc770031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.896+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77003f030 con 0x7fc780074140 2024-04-24T23:20:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.896+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7801a1b20 con 0x7fc780074140 2024-04-24T23:20:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.896+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc770003ad0 con 0x7fc780074140 2024-04-24T23:20:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.896+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770035070 con 0x7fc780074140 2024-04-24T23:20:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.896+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7801a2010 con 0x7fc780074140 2024-04-24T23:20:46.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.897+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7801083f0 con 0x7fc780074140 2024-04-24T23:20:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.898+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc770042030 con 0x7fc780074140 2024-04-24T23:20:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.898+0000 7fc77cff9640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc76c0a2b90 0x7fc76c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.899+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc7700ef9f0 con 0x7fc780074140 2024-04-24T23:20:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.899+0000 7fc777fff640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc76c0a2b90 0x7fc76c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.899+0000 7fc777fff640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc76c0a2b90 0x7fc76c0a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc768009c20 tx=0x7fc768005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:46.903+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc7700bc5c0 con 0x7fc780074140 2024-04-24T23:20:47.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.135+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc780062370 con 0x7fc780074140 2024-04-24T23:20:47.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.136+0000 7fc77cff9640 1 -- 172.21.15.33:0/1187670959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fc7700bbec0 con 0x7fc780074140 2024-04-24T23:20:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:20:47.139 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:20:47.139 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc76c0a2b90 msgr2=0x7fc76c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc76c0a2b90 0x7fc76c0a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc768009c20 tx=0x7fc768005f90 comp rx=0 tx=0).stop 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 msgr2=0x7fc780199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 0x7fc780199c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc770009230 tx=0x7fc770031b60 comp rx=0 tx=0).stop 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 shutdown_connections 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc76c0a2b90 0x7fc76c0a5050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc780109630 0x7fc7801a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc780075720 0x7fc78019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 --2- 172.21.15.33:0/1187670959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc780074140 0x7fc780199c80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.138+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 >> 172.21.15.33:0/1187670959 conn(0x7fc7800fccf0 msgr2=0x7fc7801055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.139+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 shutdown_connections 2024-04-24T23:20:47.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.139+0000 7fc7853a2640 1 -- 172.21.15.33:0/1187670959 wait complete. 2024-04-24T23:20:47.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/3526104605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98100680 msgr2=0x7f0a98100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/3526104605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98100680 0x7f0a98100aa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009900 tx=0x7f0a8802a410 comp rx=0 tx=0).stop 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/3526104605 shutdown_connections 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/3526104605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a98102230 0x7f0a98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/3526104605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a98101870 0x7f0a98101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/3526104605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98100680 0x7f0a98100aa0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/3526104605 >> 172.21.15.33:0/3526104605 conn(0x7f0a980fbbf0 msgr2=0x7f0a980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/3526104605 shutdown_connections 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/3526104605 wait complete. 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 Processor -- start 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.264+0000 7f0a9e1b7640 1 -- start start 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a98100680 0x7f0a98069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 0x7f0a9819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a98102230 0x7f0a9819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810c2d0 con 0x7f0a98101870 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810c150 con 0x7f0a98102230 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810c450 con 0x7f0a98100680 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 0x7f0a9819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 0x7f0a9819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52690/0 (socket says 172.21.15.33:52690) 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 -- 172.21.15.33:0/1333122281 learned_addr learned my addr 172.21.15.33:0/1333122281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a977fe640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a98100680 0x7f0a98069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a97fff640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a98102230 0x7f0a9819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 -- 172.21.15.33:0/1333122281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a98100680 msgr2=0x7f0a98069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a98100680 0x7f0a98069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 -- 172.21.15.33:0/1333122281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a98102230 msgr2=0x7f0a9819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a98102230 0x7f0a9819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 -- 172.21.15.33:0/1333122281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9819f6f0 con 0x7f0a98101870 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 0x7f0a9819c7f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9f0 tx=0x7f0a8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a94ff9640 1 -- 172.21.15.33:0/1333122281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00cd80 con 0x7f0a98101870 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9819f9e0 con 0x7f0a98101870 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a94ff9640 1 -- 172.21.15.33:0/1333122281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c010910 con 0x7f0a98101870 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.265+0000 7f0a94ff9640 1 -- 172.21.15.33:0/1333122281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0188f0 con 0x7f0a98101870 2024-04-24T23:20:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.266+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9819ff80 con 0x7f0a98101870 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.268+0000 7f0a94ff9640 1 -- 172.21.15.33:0/1333122281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0a8c010ac0 con 0x7f0a98101870 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.268+0000 7f0a94ff9640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0a800a2b90 0x7f0a800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.268+0000 7f0aa0fa4740 1 Processor -- start 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.268+0000 7f0a977fe640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0a800a2b90 0x7f0a800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.268+0000 7f0a94ff9640 1 -- 172.21.15.33:0/1333122281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0a8c0cf790 con 0x7f0a98101870 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.268+0000 7f0aa0fa4740 1 -- start start 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d35541af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d35541da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d355371670 con 0x55d355416a80 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d355319970 con 0x55d35541b5d0 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d35536d630 con 0x55d355416090 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d35541af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a977fe640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0a800a2b90 0x7f0a800a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0a88005ab0 tx=0x7f0a88000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d35541da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d35541af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52692/0 (socket says 172.21.15.33:52692) 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a96ffd640 1 -- 172.21.15.33:0/1536661743 learned_addr learned my addr 172.21.15.33:0/1536661743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 -- 172.21.15.33:0/1536661743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 msgr2=0x55d3554164b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554164b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 -- 172.21.15.33:0/1536661743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 msgr2=0x55d35541af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d35541af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 -- 172.21.15.33:0/1536661743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d35533d400 con 0x55d35541b5d0 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d35541af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a97fff640 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d35541da20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009f00 tx=0x7f0a90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b74d32bd80846b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.269+0000 7f0a7dffb640 1 -- 172.21.15.33:0/1536661743 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90004d20 con 0x55d35541b5d0 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0a7dffb640 1 -- 172.21.15.33:0/1536661743 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a90004050 con 0x55d35541b5d0 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0a7dffb640 1 -- 172.21.15.33:0/1536661743 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90005280 con 0x55d35541b5d0 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/1536661743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 msgr2=0x55d35541da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d35541da20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009f00 tx=0x7f0a90011810 comp rx=0 tx=0).stop 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/1536661743 shutdown_connections 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d35541da20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d35541af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/1536661743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/1536661743 >> 172.21.15.33:0/1536661743 conn(0x55d355408820 msgr2=0x55d355408c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/1536661743 shutdown_connections 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.270+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/1536661743 wait complete. 2024-04-24T23:20:47.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 -- start start 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 Processor -- start 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d355416090 0x55d35541fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416a80 0x55d355420090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 0x55d3554270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d35541e540 con 0x55d35541b5d0 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d35532d050 con 0x55d355416090 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d35541e6c0 con 0x55d355416a80 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0aa0fa4740 10 client.0 _ll_register_callbacks cb 0x55d355084be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 0x55d3554270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0a977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d355416090 0x55d35541fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 0x55d3554270d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52698/0 (socket says 172.21.15.33:52698) 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416a80 0x55d355420090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0a97fff640 1 -- 172.21.15.33:0/4012021559 learned_addr learned my addr 172.21.15.33:0/4012021559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.271+0000 7f0a97fff640 1 -- 172.21.15.33:0/4012021559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416a80 msgr2=0x55d355420090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a97fff640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416a80 0x55d355420090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a97fff640 1 -- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d355416090 msgr2=0x55d35541fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a97fff640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d355416090 0x55d35541fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a97fff640 1 -- 172.21.15.33:0/4012021559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3552fedc0 con 0x55d35541b5d0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a977fe640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d355416090 0x55d35541fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416a80 0x55d355420090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a97fff640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 0x55d3554270d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001b040 tx=0x7f0a90004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02222ffc1c98dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a76ffd640 1 -- 172.21.15.33:0/4012021559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90029020 con 0x55d35541b5d0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a76ffd640 1 -- 172.21.15.33:0/4012021559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0a90025470 con 0x55d35541b5d0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a90004050 con 0x55d35541b5d0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a76ffd640 1 -- 172.21.15.33:0/4012021559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a90034050 con 0x55d35541b5d0 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0aa0fa4740 10 client.17250 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3550853f0 con 0x55d35541b5d0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a76ffd640 1 -- 172.21.15.33:0/4012021559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a90004050 con 0x55d35541b5d0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.272+0000 7f0a76ffd640 1 -- 172.21.15.33:0/4012021559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a90019730 con 0x55d35541b5d0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0aa0fa4740 10 client.17250 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0aa0fa4740 10 client.17250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0aa0fa4740 20 client.17250 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d35542b7e0 0x55d35542dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0aa0fa4740 4 client.17250 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d35537ebb0 con 0x55d35542b7e0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0a977fe640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d35542b7e0 0x55d35542dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0a977fe640 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d35542b7e0 0x55d35542dc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f71ef92a4894477 server_cookie=81dbe782910b03f8 in_seq=0 out_seq=0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.273+0000 7f0a76ffd640 10 client.17250 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0a76ffd640 1 -- 172.21.15.33:0/4012021559 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d35537ebb0 con 0x55d35542b7e0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0a76ffd640 10 client.17250 handle_command_reply: tid=0 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 client.17250 shutdown 2024-04-24T23:20:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 msgr2=0x55d3554270d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 0x55d3554270d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001b040 tx=0x7f0a90004430 comp rx=0 tx=0).stop 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 shutdown_connections 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d35542b7e0 0x55d35542dc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d35541b5d0 0x55d3554270d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416a80 0x55d355420090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0a76ffd640 0 client.17250 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/4012021559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d355416090 0x55d35541fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 >> 172.21.15.33:0/4012021559 conn(0x55d355408820 msgr2=0x55d355407b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 shutdown_connections 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/4012021559 wait complete. 2024-04-24T23:20:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.274+0000 7f0aa0fa4740 20 client.17250 trim_cache size 0 max 16384 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 Processor -- start 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 -- start start 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d355416e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355427090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d3554173a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981aab60 con 0x55d355416a80 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a8430 con 0x55d35541b5d0 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a6280 con 0x55d355416090 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.288+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355427090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d355416e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355427090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52714/0 (socket says 172.21.15.33:52714) 2024-04-24T23:20:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 -- 172.21.15.33:0/3691939340 learned_addr learned my addr 172.21.15.33:0/3691939340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a97fff640 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d3554173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 -- 172.21.15.33:0/3691939340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 msgr2=0x55d355416e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d355416e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 -- 172.21.15.33:0/3691939340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 msgr2=0x55d3554173a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d3554173a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 -- 172.21.15.33:0/3691939340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d35514f7d0 con 0x55d355416a80 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a977fe640 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d355416e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355427090 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c098690 tx=0x7f0a8c0a00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c04c7ec8bbd00b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a7d7fa640 1 -- 172.21.15.33:0/3691939340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0b4e60 con 0x55d355416a80 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a7d7fa640 1 -- 172.21.15.33:0/3691939340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c0b1030 con 0x55d355416a80 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.289+0000 7f0a7d7fa640 1 -- 172.21.15.33:0/3691939340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c09abd0 con 0x55d355416a80 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/3691939340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 msgr2=0x55d355427090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355427090 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c098690 tx=0x7f0a8c0a00d0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/3691939340 shutdown_connections 2024-04-24T23:20:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d3554173a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355427090 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/3691939340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d355416e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/3691939340 >> 172.21.15.33:0/3691939340 conn(0x55d355408820 msgr2=0x55d355407b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/3691939340 shutdown_connections 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/3691939340 wait complete. 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 -- start start 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 Processor -- start 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.290+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355424830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d355425840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a68f0 con 0x55d355416a80 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a7d60 con 0x55d35541b5d0 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a6fc0 con 0x55d355416090 2024-04-24T23:20:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 10 client.0 _ll_register_callbacks cb 0x7f0a88040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355424830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d355425840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355424830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52726/0 (socket says 172.21.15.33:52726) 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 -- 172.21.15.33:0/458616140 learned_addr learned my addr 172.21.15.33:0/458616140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a977fe640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 -- 172.21.15.33:0/458616140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 msgr2=0x55d3554242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 -- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 msgr2=0x55d355425840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d355425840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 -- 172.21.15.33:0/458616140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d355425e30 con 0x55d355416a80 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a97fff640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d355425840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a96ffd640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355424830 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09bed0 tx=0x7f0a8c09a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc2a04be64bfb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a76ffd640 1 -- 172.21.15.33:0/458616140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0a8020 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a76ffd640 1 -- 172.21.15.33:0/458616140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0a8c0a4d20 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8c0b1030 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0a76ffd640 1 -- 172.21.15.33:0/458616140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c0ba070 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 10 client.17256 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.291+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d355427130 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0a76ffd640 1 -- 172.21.15.33:0/458616140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c0b1030 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0a76ffd640 1 -- 172.21.15.33:0/458616140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a8c018ca0 con 0x55d355416a80 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0aa0fa4740 10 client.17256 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0aa0fa4740 10 client.17256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0aa0fa4740 20 client.17256 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d3554922d0 0x55d3554946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0aa0fa4740 4 client.17256 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d35541f940 con 0x55d3554922d0 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0a977fe640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d3554922d0 0x55d3554946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0a977fe640 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d3554922d0 0x55d3554946f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f5e0e6592907871 server_cookie=75de8df874fcf47b in_seq=0 out_seq=0 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.292+0000 7f0a76ffd640 10 client.17256 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a76ffd640 1 -- 172.21.15.33:0/458616140 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d35541f940 con 0x55d3554922d0 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a76ffd640 10 client.17256 handle_command_reply: tid=0 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 client.17256 shutdown 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 msgr2=0x55d355424830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355424830 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09bed0 tx=0x7f0a8c09a010 comp rx=0 tx=0).stop 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 shutdown_connections 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55d3554922d0 0x55d3554946f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d35541b5d0 0x55d355425840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d355416a80 0x55d355424830 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a76ffd640 0 client.17256 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 --2- 172.21.15.33:0/458616140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d355416090 0x55d3554242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 >> 172.21.15.33:0/458616140 conn(0x55d355408820 msgr2=0x55d355414e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 shutdown_connections 2024-04-24T23:20:47.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 1 -- 172.21.15.33:0/458616140 wait complete. 2024-04-24T23:20:47.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0aa0fa4740 20 client.17256 trim_cache size 0 max 16384 2024-04-24T23:20:47.302 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0a800a2b90 msgr2=0x7f0a800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0a800a2b90 0x7f0a800a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0a88005ab0 tx=0x7f0a88000bc0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 msgr2=0x7f0a9819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.293+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 0x7f0a9819c7f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9f0 tx=0x7f0a8c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 shutdown_connections 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0a800a2b90 0x7f0a800a5050 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a98102230 0x7f0a9819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a98101870 0x7f0a9819c7f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 --2- 172.21.15.33:0/1333122281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a98100680 0x7f0a98069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 >> 172.21.15.33:0/1333122281 conn(0x7f0a980fbbf0 msgr2=0x7f0a98104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 shutdown_connections 2024-04-24T23:20:47.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.294+0000 7f0a9e1b7640 1 -- 172.21.15.33:0/1333122281 wait complete. 2024-04-24T23:20:47.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:20:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 -- 172.21.15.33:0/3903723092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 msgr2=0x7f13a0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 --2- 172.21.15.33:0/3903723092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0101bb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1394009900 tx=0x7f139402a410 comp rx=0 tx=0).stop 2024-04-24T23:20:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 -- 172.21.15.33:0/3903723092 shutdown_connections 2024-04-24T23:20:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 --2- 172.21.15.33:0/3903723092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a0103340 0x7f13a0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 --2- 172.21.15.33:0/3903723092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0102980 0x7f13a0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 --2- 172.21.15.33:0/3903723092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0101bb0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 -- 172.21.15.33:0/3903723092 >> 172.21.15.33:0/3903723092 conn(0x7f13a00fcd20 msgr2=0x7f13a00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 -- 172.21.15.33:0/3903723092 shutdown_connections 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.413+0000 7f13a7058640 1 -- 172.21.15.33:0/3903723092 wait complete. 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 Processor -- start 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 -- start start 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0102980 0x7f13a0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a0103340 0x7f13a019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010d410 con 0x7f13a0101790 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010d290 con 0x7f13a0102980 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a7058640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010d590 con 0x7f13a0103340 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52742/0 (socket says 172.21.15.33:52742) 2024-04-24T23:20:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 -- 172.21.15.33:0/4054857121 learned_addr learned my addr 172.21.15.33:0/4054857121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f139ffff640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0102980 0x7f13a0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a55f5640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a0103340 0x7f13a019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 -- 172.21.15.33:0/4054857121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a0103340 msgr2=0x7f13a019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a0103340 0x7f13a019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 -- 172.21.15.33:0/4054857121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0102980 msgr2=0x7f13a0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0102980 0x7f13a0198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.414+0000 7f13a4df4640 1 -- 172.21.15.33:0/4054857121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a019f700 con 0x7f13a0101790 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.415+0000 7f13a4df4640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0197ae0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1394031610 tx=0x7f1394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.415+0000 7f139dffb640 1 -- 172.21.15.33:0/4054857121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139403f030 con 0x7f13a0101790 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.415+0000 7f139dffb640 1 -- 172.21.15.33:0/4054857121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394003ad0 con 0x7f13a0101790 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.415+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a019f990 con 0x7f13a0101790 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.415+0000 7f139dffb640 1 -- 172.21.15.33:0/4054857121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394035070 con 0x7f13a0101790 2024-04-24T23:20:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.415+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a019fed0 con 0x7f13a0101790 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f139dffb640 1 -- 172.21.15.33:0/4054857121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1394042030 con 0x7f13a0101790 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 Processor -- start 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 -- start start 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5592cf727670 con 0x5592cf7c3c20 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592cf6cf970 con 0x5592cf7c3230 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a9d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592cf723630 con 0x5592cf7c8d90 2024-04-24T23:20:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a4df4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38584/0 (socket says 172.21.15.33:38584) 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 -- 172.21.15.33:0/1140221510 learned_addr learned my addr 172.21.15.33:0/1140221510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f139ffff640 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f139dffb640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13900a2c10 0x7f13900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 -- 172.21.15.33:0/1140221510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 msgr2=0x5592cf7cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 -- 172.21.15.33:0/1140221510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 msgr2=0x5592cf7c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a55f5640 1 -- 172.21.15.33:0/1140221510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cf6f3400 con 0x5592cf7c3230 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.418+0000 7f13a4df4640 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.419+0000 7f139dffb640 1 -- 172.21.15.33:0/4054857121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f13940ef9b0 con 0x7f13a0101790 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.419+0000 7f13a55f5640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13900a2c10 0x7f13900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.419+0000 7f13a55f5640 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7c3650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1398009fc0 tx=0x7f1398010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6aca741a781e7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.419+0000 7f138affd640 1 -- 172.21.15.33:0/1140221510 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398003b30 con 0x5592cf7c3230 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.419+0000 7f138affd640 1 -- 172.21.15.33:0/1140221510 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13980119e0 con 0x5592cf7c3230 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.419+0000 7f138affd640 1 -- 172.21.15.33:0/1140221510 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139801e9d0 con 0x5592cf7c3230 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- 172.21.15.33:0/1140221510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 msgr2=0x5592cf7c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a55f5640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13900a2c10 0x7f13900a50d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f139801f000 tx=0x7f1398020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7c3650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1398009fc0 tx=0x7f1398010860 comp rx=0 tx=0).stop 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- 172.21.15.33:0/1140221510 shutdown_connections 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1140221510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7c3650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- 172.21.15.33:0/1140221510 >> 172.21.15.33:0/1140221510 conn(0x5592cf7be8a0 msgr2=0x5592cf7beca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- 172.21.15.33:0/1140221510 shutdown_connections 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- 172.21.15.33:0/1140221510 wait complete. 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- start start 2024-04-24T23:20:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 Processor -- start 2024-04-24T23:20:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5592cf7cbd40 con 0x5592cf7c3c20 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592cf6e3050 con 0x5592cf7c3230 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.420+0000 7f13a9d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592cf7cbec0 con 0x5592cf7c8d90 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a9d66740 10 client.0 _ll_register_callbacks cb 0x5592cf43abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f139ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a55f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a55f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7d48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36774/0 (socket says 172.21.15.33:36774) 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a55f5640 1 -- 172.21.15.33:0/2465354280 learned_addr learned my addr 172.21.15.33:0/2465354280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 -- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 msgr2=0x5592cf7d48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7d48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 -- 172.21.15.33:0/2465354280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 msgr2=0x5592cf7cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 -- 172.21.15.33:0/2465354280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cf6b4dc0 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f139ffff640 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a4df4640 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7cd340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13940ed170 tx=0x7f1394009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ceb773fda328b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f137ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f137ffff640 1 -- 172.21.15.33:0/2465354280 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13940efe60 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f137ffff640 1 -- 172.21.15.33:0/2465354280 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1394003c80 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.421+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13980119e0 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f137ffff640 1 -- 172.21.15.33:0/2465354280 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1394049070 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 10 client.26795 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592cf43b3f0 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f137ffff640 1 -- 172.21.15.33:0/2465354280 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13980119e0 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f137ffff640 1 -- 172.21.15.33:0/2465354280 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f13940035f0 con 0x5592cf7c3230 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 10 client.26795 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 10 client.26795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 20 client.26795 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf7db510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 4 client.26795 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592cf734bb0 con 0x5592cf7d9080 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f139ffff640 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf7db510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f137ffff640 10 client.26795 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.422+0000 7f139ffff640 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf7db510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=686ab022c6fa44d2 server_cookie=284a67e8b8aeaf55 in_seq=0 out_seq=0 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f137ffff640 1 -- 172.21.15.33:0/2465354280 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592cf734bb0 con 0x5592cf7d9080 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f137ffff640 10 client.26795 handle_command_reply: tid=0 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 client.26795 shutdown 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 msgr2=0x5592cf7cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7cd340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13940ed170 tx=0x7f1394009560 comp rx=0 tx=0).stop 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 shutdown_connections 2024-04-24T23:20:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf7db510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f137ffff640 0 client.26795 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 --2- 172.21.15.33:0/2465354280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7cd340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 >> 172.21.15.33:0/2465354280 conn(0x5592cf7be8a0 msgr2=0x5592cf7bdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 shutdown_connections 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.423+0000 7f13a9d66740 1 -- 172.21.15.33:0/2465354280 wait complete. 2024-04-24T23:20:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.424+0000 7f13a9d66740 20 client.26795 trim_cache size 0 max 16384 2024-04-24T23:20:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 Processor -- start 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 -- start start 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01acd20 con 0x5592cf7c3c20 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01aa5f0 con 0x5592cf7c3230 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a9d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01a8440 con 0x5592cf7c8d90 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7d4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52762/0 (socket says 172.21.15.33:52762) 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 -- 172.21.15.33:0/1139609934 learned_addr learned my addr 172.21.15.33:0/1139609934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a4df4640 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f13a55f5640 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 -- 172.21.15.33:0/1139609934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 msgr2=0x5592cf7cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 -- 172.21.15.33:0/1139609934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 msgr2=0x5592cf7bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7bdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.437+0000 7f139ffff640 1 -- 172.21.15.33:0/1139609934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cf5057d0 con 0x5592cf7c3c20 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f139ffff640 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7d4880 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x5592cf7bf770 tx=0x7f138c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71140ce5f26728b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f138a7fc640 1 -- 172.21.15.33:0/1139609934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0122d0 con 0x5592cf7c3c20 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f138a7fc640 1 -- 172.21.15.33:0/1139609934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f138c026030 con 0x5592cf7c3c20 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f138a7fc640 1 -- 172.21.15.33:0/1139609934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c022350 con 0x5592cf7c3c20 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 -- 172.21.15.33:0/1139609934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 msgr2=0x5592cf7d4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7d4880 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x5592cf7bf770 tx=0x7f138c005f90 comp rx=0 tx=0).stop 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 -- 172.21.15.33:0/1139609934 shutdown_connections 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c8d90 0x5592cf7cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c3c20 0x5592cf7d4880 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1139609934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3230 0x5592cf7bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 -- 172.21.15.33:0/1139609934 >> 172.21.15.33:0/1139609934 conn(0x5592cf7be8a0 msgr2=0x5592cf7c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 -- 172.21.15.33:0/1139609934 shutdown_connections 2024-04-24T23:20:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.438+0000 7f13a9d66740 1 -- 172.21.15.33:0/1139609934 wait complete. 2024-04-24T23:20:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 -- start start 2024-04-24T23:20:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 Processor -- start 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 0x5592cf7d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3c20 0x5592cf7d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c8d90 0x5592cf7d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01a8ab0 con 0x5592cf7c8d90 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01a9f20 con 0x5592cf7c3c20 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01a9180 con 0x5592cf7c3230 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a9d66740 10 client.0 _ll_register_callbacks cb 0x7f1394031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a4df4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 0x5592cf7d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a55f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c8d90 0x5592cf7d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f139ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3c20 0x5592cf7d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a4df4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 0x5592cf7d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36784/0 (socket says 172.21.15.33:36784) 2024-04-24T23:20:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.439+0000 7f13a4df4640 1 -- 172.21.15.33:0/1815740179 learned_addr learned my addr 172.21.15.33:0/1815740179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a4df4640 1 -- 172.21.15.33:0/1815740179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3c20 msgr2=0x5592cf7d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a4df4640 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3c20 0x5592cf7d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a4df4640 1 -- 172.21.15.33:0/1815740179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c8d90 msgr2=0x5592cf7d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a4df4640 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c8d90 0x5592cf7d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a4df4640 1 -- 172.21.15.33:0/1815740179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cf7d3530 con 0x5592cf7c3230 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a4df4640 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 0x5592cf7d19f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1394043040 tx=0x7f13940bbd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f110ed4dd8574bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f137ffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f137ffff640 1 -- 172.21.15.33:0/1815740179 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13940bb9c0 con 0x5592cf7c3230 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f137ffff640 1 -- 172.21.15.33:0/1815740179 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f13940035f0 con 0x5592cf7c3230 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138c026030 con 0x5592cf7c3230 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f137ffff640 1 -- 172.21.15.33:0/1815740179 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13940f51f0 con 0x5592cf7c3230 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a9d66740 10 client.26785 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592cf7d48a0 con 0x5592cf7c3230 2024-04-24T23:20:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f137ffff640 1 -- 172.21.15.33:0/1815740179 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13940d8080 con 0x5592cf7c3230 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f137ffff640 1 -- 172.21.15.33:0/1815740179 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f13940c46f0 con 0x5592cf7c3230 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a9d66740 10 client.26785 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a9d66740 10 client.26785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.440+0000 7f13a9d66740 20 client.26785 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf8419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 4 client.26785 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592cf72ef00 con 0x5592cf7d9080 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f139ffff640 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf8419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f137ffff640 10 client.26785 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f139ffff640 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf8419a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a174a8df5e29376 server_cookie=1b3f15a4a8500e4c in_seq=0 out_seq=0 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f137ffff640 1 -- 172.21.15.33:0/1815740179 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592cf72ef00 con 0x5592cf7d9080 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f137ffff640 10 client.26785 handle_command_reply: tid=0 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 client.26785 shutdown 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 msgr2=0x5592cf7d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 0x5592cf7d19f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1394043040 tx=0x7f13940bbd10 comp rx=0 tx=0).stop 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 shutdown_connections 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5592cf7d9080 0x5592cf8419a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592cf7c8d90 0x5592cf7d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592cf7c3c20 0x5592cf7d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 --2- 172.21.15.33:0/1815740179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592cf7c3230 0x5592cf7d19f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f137ffff640 0 client.26785 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.441+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 >> 172.21.15.33:0/1815740179 conn(0x5592cf7be8a0 msgr2=0x5592cf7c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 shutdown_connections 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a9d66740 1 -- 172.21.15.33:0/1815740179 wait complete. 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a9d66740 20 client.26785 trim_cache size 0 max 16384 2024-04-24T23:20:47.449 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:20:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13900a2c10 msgr2=0x7f13900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13900a2c10 0x7f13900a50d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f139801f000 tx=0x7f1398020000 comp rx=0 tx=0).stop 2024-04-24T23:20:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 msgr2=0x7f13a0197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0197ae0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1394031610 tx=0x7f1394004270 comp rx=0 tx=0).stop 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 shutdown_connections 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f13900a2c10 0x7f13900a50d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a0103340 0x7f13a019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0102980 0x7f13a0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 --2- 172.21.15.33:0/4054857121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13a0101790 0x7f13a0197ae0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 >> 172.21.15.33:0/4054857121 conn(0x7f13a00fcd20 msgr2=0x7f13a0105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 shutdown_connections 2024-04-24T23:20:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.442+0000 7f13a7058640 1 -- 172.21.15.33:0/4054857121 wait complete. 2024-04-24T23:20:47.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 982066 12.74 MB/sec execute 292 sec latency 13.068 ms 2024-04-24T23:20:47.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.561+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/1661026230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec101870 msgr2=0x7f9eec101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.561+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/1661026230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec101870 0x7f9eec101cf0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402a410 comp rx=0 tx=0).stop 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.561+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/1661026230 shutdown_connections 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/1661026230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec102230 0x7f9eec108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/1661026230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec101870 0x7f9eec101cf0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/1661026230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eec100680 0x7f9eec100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/1661026230 >> 172.21.15.33:0/1661026230 conn(0x7f9eec0fbbf0 msgr2=0x7f9eec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/1661026230 shutdown_connections 2024-04-24T23:20:47.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/1661026230 wait complete. 2024-04-24T23:20:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 Processor -- start 2024-04-24T23:20:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 -- start start 2024-04-24T23:20:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.562+0000 7f9ef0b8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 0x7f9eec197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eec101870 0x7f9eec198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec102230 0x7f9eec19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10c300 con 0x7f9eec100680 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10c180 con 0x7f9eec102230 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10c480 con 0x7f9eec101870 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 0x7f9eec197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 0x7f9eec197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52792/0 (socket says 172.21.15.33:52792) 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 -- 172.21.15.33:0/2919442889 learned_addr learned my addr 172.21.15.33:0/2919442889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eead9d640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec102230 0x7f9eec19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eec101870 0x7f9eec198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 -- 172.21.15.33:0/2919442889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eec101870 msgr2=0x7f9eec198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eec101870 0x7f9eec198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 -- 172.21.15.33:0/2919442889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec102230 msgr2=0x7f9eec19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec102230 0x7f9eec19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 -- 172.21.15.33:0/2919442889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec19f710 con 0x7f9eec100680 2024-04-24T23:20:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9eea59c640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 0x7f9eec197af0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e9c0 tx=0x7f9edc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ee37fe640 1 -- 172.21.15.33:0/2919442889 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00cd80 con 0x7f9eec100680 2024-04-24T23:20:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec19fa00 con 0x7f9eec100680 2024-04-24T23:20:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec19ff20 con 0x7f9eec100680 2024-04-24T23:20:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ee37fe640 1 -- 172.21.15.33:0/2919442889 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc0035f0 con 0x7f9eec100680 2024-04-24T23:20:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.563+0000 7f9ee37fe640 1 -- 172.21.15.33:0/2919442889 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc010690 con 0x7f9eec100680 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ee37fe640 1 -- 172.21.15.33:0/2919442889 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9edc0037a0 con 0x7f9eec100680 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 Processor -- start 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 -- start start 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defa2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defa7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa7d90 0x55a2defaa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a2def06670 con 0x55a2defa7d90 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2deeae970 con 0x55a2defa2c20 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ef3970740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2def02630 con 0x55a2defa2230 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9eead9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defa2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa7d90 0x55a2defaa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9eea59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defa7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.566+0000 7f9eead9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defa2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36822/0 (socket says 172.21.15.33:36822) 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9eead9d640 1 -- 172.21.15.33:0/1189482026 learned_addr learned my addr 172.21.15.33:0/1189482026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee37fe640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9ed80a2b90 0x7f9ed80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9eead9d640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9ed80a2b90 0x7f9ed80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee3fff640 1 -- 172.21.15.33:0/1189482026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 msgr2=0x55a2defa2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defa2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee3fff640 1 -- 172.21.15.33:0/1189482026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 msgr2=0x55a2defa7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defa7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee37fe640 1 -- 172.21.15.33:0/2919442889 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9edc0cf7d0 con 0x7f9eec100680 2024-04-24T23:20:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee3fff640 1 -- 172.21.15.33:0/1189482026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2deed2400 con 0x55a2defa7d90 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9eea59c640 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defa7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa7d90 0x55a2defaa220 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40314e0 tx=0x7f9ed4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1529378d3557cec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9eead9d640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9ed80a2b90 0x7f9ed80a5050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55a2defa3480 tx=0x7f9ee4004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee0ff9640 1 -- 172.21.15.33:0/1189482026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed403f070 con 0x55a2defa7d90 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee0ff9640 1 -- 172.21.15.33:0/1189482026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4004030 con 0x55a2defa7d90 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.567+0000 7f9ee0ff9640 1 -- 172.21.15.33:0/1189482026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4035070 con 0x55a2defa7d90 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 -- 172.21.15.33:0/1189482026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa7d90 msgr2=0x55a2defaa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa7d90 0x55a2defaa220 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40314e0 tx=0x7f9ed4005e60 comp rx=0 tx=0).stop 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 -- 172.21.15.33:0/1189482026 shutdown_connections 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa7d90 0x55a2defaa220 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defa7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 --2- 172.21.15.33:0/1189482026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defa2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 -- 172.21.15.33:0/1189482026 >> 172.21.15.33:0/1189482026 conn(0x55a2def9d8a0 msgr2=0x55a2def9dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 -- 172.21.15.33:0/1189482026 shutdown_connections 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 -- 172.21.15.33:0/1189482026 wait complete. 2024-04-24T23:20:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.568+0000 7f9ef3970740 1 -- start start 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 Processor -- start 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2230 0x55a2defac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a2defaace0 con 0x55a2defa2c20 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2deec2050 con 0x55a2defa2230 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2defaae60 con 0x55a2defa7d90 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 10 client.0 _ll_register_callbacks cb 0x55a2dec19be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52800/0 (socket says 172.21.15.33:52800) 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9eea59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2230 0x55a2defac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 -- 172.21.15.33:0/4272891698 learned_addr learned my addr 172.21.15.33:0/4272891698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9eead9d640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 -- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 msgr2=0x55a2defb38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 -- 172.21.15.33:0/4272891698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2230 msgr2=0x55a2defac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2230 0x55a2defac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 -- 172.21.15.33:0/4272891698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2dee93dc0 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9eead9d640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defac860 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4031440 tx=0x7f9ed4004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb78792b56d71690 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/4272891698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed404b020 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/4272891698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ed40035f0 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.569+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed4004030 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/4272891698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4056050 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 10 client.17286 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2dec1a3f0 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/4272891698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4004030 con 0x55a2defa2c20 2024-04-24T23:20:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/4272891698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ed403b850 con 0x55a2defa2c20 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 10 client.17286 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 20 client.17286 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2defb7d90 0x55a2defba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 4 client.17286 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2def13bb0 con 0x55a2defb7d90 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.570+0000 7f9eea59c640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2defb7d90 0x55a2defba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9eea59c640 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2defb7d90 0x55a2defba180 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5065884db21a67f7 server_cookie=fd71fcfa976b27 in_seq=0 out_seq=0 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9ed1ffb640 10 client.17286 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/4272891698 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2def13bb0 con 0x55a2defb7d90 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9ed1ffb640 10 client.17286 handle_command_reply: tid=0 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9ef3970740 1 client.17286 shutdown 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 msgr2=0x55a2defac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.571+0000 7f9ef3970740 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defac860 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4031440 tx=0x7f9ed4004280 comp rx=0 tx=0).stop 2024-04-24T23:20:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 shutdown_connections 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2defb7d90 0x55a2defba180 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ed1ffb640 0 client.17286 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defac860 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 --2- 172.21.15.33:0/4272891698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2230 0x55a2defac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 >> 172.21.15.33:0/4272891698 conn(0x55a2def9d8a0 msgr2=0x55a2def9cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 shutdown_connections 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 1 -- 172.21.15.33:0/4272891698 wait complete. 2024-04-24T23:20:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.572+0000 7f9ef3970740 20 client.17286 trim_cache size 0 max 16384 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 Processor -- start 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 -- start start 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defb3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa7d90 0x55a2defa3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1acce0 con 0x55a2defa2c20 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1aa5b0 con 0x55a2defa7d90 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.585+0000 7f9ef3970740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1a8400 con 0x55a2defa2230 2024-04-24T23:20:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defb3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defb3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52804/0 (socket says 172.21.15.33:52804) 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 -- 172.21.15.33:0/3098206803 learned_addr learned my addr 172.21.15.33:0/3098206803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9eead9d640 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa7d90 0x55a2defa3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9eea59c640 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 -- 172.21.15.33:0/3098206803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 msgr2=0x55a2defab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 -- 172.21.15.33:0/3098206803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa7d90 msgr2=0x55a2defa3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa7d90 0x55a2defa3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 -- 172.21.15.33:0/3098206803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2dece47d0 con 0x55a2defa2c20 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defb3860 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9ed403bca0 tx=0x7f9ed4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7a84b08b51b294 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ed3fff640 1 -- 172.21.15.33:0/3098206803 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed404b040 con 0x55a2defa2c20 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ed3fff640 1 -- 172.21.15.33:0/3098206803 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4031da0 con 0x55a2defa2c20 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.586+0000 7f9ed3fff640 1 -- 172.21.15.33:0/3098206803 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4035070 con 0x55a2defa2c20 2024-04-24T23:20:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- 172.21.15.33:0/3098206803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 msgr2=0x55a2defb3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defb3860 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9ed403bca0 tx=0x7f9ed4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- 172.21.15.33:0/3098206803 shutdown_connections 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa7d90 0x55a2defa3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2c20 0x55a2defb3860 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- 172.21.15.33:0/3098206803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa2230 0x55a2defab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- 172.21.15.33:0/3098206803 >> 172.21.15.33:0/3098206803 conn(0x55a2def9d8a0 msgr2=0x55a2def9cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- 172.21.15.33:0/3098206803 shutdown_connections 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- 172.21.15.33:0/3098206803 wait complete. 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- start start 2024-04-24T23:20:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 Processor -- start 2024-04-24T23:20:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 0x55a2defb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1a8a70 con 0x55a2defa2230 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1a9ee0 con 0x55a2defa2c20 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1a9140 con 0x55a2defa7d90 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.587+0000 7f9ef3970740 10 client.0 _ll_register_callbacks cb 0x7f9edc097fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 0x55a2defb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 0x55a2defb1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52812/0 (socket says 172.21.15.33:52812) 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9eead9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9eea59c640 1 -- 172.21.15.33:0/2879438144 learned_addr learned my addr 172.21.15.33:0/2879438144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 -- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 msgr2=0x55a2defb2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 -- 172.21.15.33:0/2879438144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 msgr2=0x55a2defb1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 0x55a2defb1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 -- 172.21.15.33:0/2879438144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2defb2c80 con 0x55a2defa2c20 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9eea59c640 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 0x55a2defb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ee3fff640 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defb1680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a2def99610 tx=0x7f9ed4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec972505bbc9ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/2879438144 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed404b070 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/2879438144 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ed403b430 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed4031da0 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/2879438144 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4035040 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.588+0000 7f9ef3970740 10 client.26819 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2df01e4b0 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/2879438144 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4031da0 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/2879438144 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ed4061030 con 0x55a2defa2c20 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 10 client.26819 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 20 client.26819 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2df01ec50 0x55a2df021070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 4 client.26819 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2defac110 con 0x55a2df01ec50 2024-04-24T23:20:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.589+0000 7f9eea59c640 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2df01ec50 0x55a2df021070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ed1ffb640 10 client.26819 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9eea59c640 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2df01ec50 0x55a2df021070 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdacbb4cb7ca619a server_cookie=d30f6fca68ae5905 in_seq=0 out_seq=0 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ed1ffb640 1 -- 172.21.15.33:0/2879438144 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2defac110 con 0x55a2df01ec50 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ed1ffb640 10 client.26819 handle_command_reply: tid=0 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 client.26819 shutdown 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 msgr2=0x55a2defb1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defb1680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a2def99610 tx=0x7f9ed4005e80 comp rx=0 tx=0).stop 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 shutdown_connections 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a2df01ec50 0x55a2df021070 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2defa7d90 0x55a2defb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2defa2c20 0x55a2defb1680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 --2- 172.21.15.33:0/2879438144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2defa2230 0x55a2defb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ed1ffb640 0 client.26819 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 >> 172.21.15.33:0/2879438144 conn(0x55a2def9d8a0 msgr2=0x55a2def9cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 shutdown_connections 2024-04-24T23:20:47.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 1 -- 172.21.15.33:0/2879438144 wait complete. 2024-04-24T23:20:47.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef3970740 20 client.26819 trim_cache size 0 max 16384 2024-04-24T23:20:47.598 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9ed80a2b90 msgr2=0x7f9ed80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9ed80a2b90 0x7f9ed80a5050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55a2defa3480 tx=0x7f9ee4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 msgr2=0x7f9eec197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.590+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 0x7f9eec197af0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e9c0 tx=0x7f9edc00ee80 comp rx=0 tx=0).stop 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 shutdown_connections 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9ed80a2b90 0x7f9ed80a5050 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eec102230 0x7f9eec19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eec101870 0x7f9eec198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 --2- 172.21.15.33:0/2919442889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eec100680 0x7f9eec197af0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 >> 172.21.15.33:0/2919442889 conn(0x7f9eec0fbbf0 msgr2=0x7f9eec0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 shutdown_connections 2024-04-24T23:20:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:47.591+0000 7f9ef0b8b640 1 -- 172.21.15.33:0/2919442889 wait complete. 2024-04-24T23:20:48.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:47 smithi038 ceph-mon[32450]: pgmap v614: 129 pgs: 129 active+clean; 264 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-24T23:20:48.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:47 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1187670959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:20:48.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:47 smithi117 ceph-mon[32529]: pgmap v614: 129 pgs: 129 active+clean; 264 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-24T23:20:48.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:47 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1187670959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:20:48.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:47 smithi033 ceph-mon[29201]: pgmap v614: 129 pgs: 129 active+clean; 264 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-24T23:20:48.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:47 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1187670959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:20:48.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 984794 12.74 MB/sec execute 293 sec latency 23.177 ms 2024-04-24T23:20:48.604 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:20:48.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:20:48.605 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:20:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 -- 172.21.15.33:0/2526102760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe901006a0 msgr2=0x7fbe90100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 --2- 172.21.15.33:0/2526102760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe901006a0 0x7fbe90100ac0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009ca0 tx=0x7fbe8002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 -- 172.21.15.33:0/2526102760 shutdown_connections 2024-04-24T23:20:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 --2- 172.21.15.33:0/2526102760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90102250 0x7fbe90108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 --2- 172.21.15.33:0/2526102760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe90101890 0x7fbe90101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 --2- 172.21.15.33:0/2526102760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe901006a0 0x7fbe90100ac0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 -- 172.21.15.33:0/2526102760 >> 172.21.15.33:0/2526102760 conn(0x7fbe900fbc30 msgr2=0x7fbe900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 -- 172.21.15.33:0/2526102760 shutdown_connections 2024-04-24T23:20:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 -- 172.21.15.33:0/2526102760 wait complete. 2024-04-24T23:20:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.713+0000 7fbe950a5640 1 Processor -- start 2024-04-24T23:20:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 -- start start 2024-04-24T23:20:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe901006a0 0x7fbe90104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90101890 0x7fbe90102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 0x7fbe901031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010c360 con 0x7fbe90102250 2024-04-24T23:20:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010c1e0 con 0x7fbe90101890 2024-04-24T23:20:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe950a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010c4e0 con 0x7fbe901006a0 2024-04-24T23:20:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 0x7fbe901031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90101890 0x7fbe90102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 0x7fbe901031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52828/0 (socket says 172.21.15.33:52828) 2024-04-24T23:20:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8f59e640 1 -- 172.21.15.33:0/668961511 learned_addr learned my addr 172.21.15.33:0/668961511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe901006a0 0x7fbe90104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8e59c640 1 -- 172.21.15.33:0/668961511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe901006a0 msgr2=0x7fbe90104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe901006a0 0x7fbe90104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8e59c640 1 -- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 msgr2=0x7fbe901031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 0x7fbe901031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8e59c640 1 -- 172.21.15.33:0/668961511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe90103a40 con 0x7fbe90101890 2024-04-24T23:20:48.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.714+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 0x7fbe901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.715+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90101890 0x7fbe90102c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400e9f0 tx=0x7fbe8400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.715+0000 7fbe77fff640 1 -- 172.21.15.33:0/668961511 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400ccc0 con 0x7fbe90101890 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.715+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe901a1c10 con 0x7fbe90101890 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.715+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901a2160 con 0x7fbe90101890 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.715+0000 7fbe77fff640 1 -- 172.21.15.33:0/668961511 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe84005020 con 0x7fbe90101890 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.715+0000 7fbe77fff640 1 -- 172.21.15.33:0/668961511 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8401c070 con 0x7fbe90101890 2024-04-24T23:20:48.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.717+0000 7fbe97db8740 1 Processor -- start 2024-04-24T23:20:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.717+0000 7fbe97db8740 1 -- start start 2024-04-24T23:20:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.717+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ab71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.717+0000 7fbe77fff640 1 -- 172.21.15.33:0/668961511 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbe84004020 con 0x7fbe90101890 2024-04-24T23:20:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.717+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1abc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.717+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1abed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe97db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acc1a1a2c0 con 0x55acc1ab7780 2024-04-24T23:20:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe97db8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55acc19c2910 con 0x55acc1abc8f0 2024-04-24T23:20:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe97db8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55acc1a16280 con 0x55acc1ab6d90 2024-04-24T23:20:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1abc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ab71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1abc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52842/0 (socket says 172.21.15.33:52842) 2024-04-24T23:20:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1abed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8ed9d640 1 -- 172.21.15.33:0/1011300760 learned_addr learned my addr 172.21.15.33:0/1011300760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 -- 172.21.15.33:0/1011300760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 msgr2=0x55acc1abed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1abed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 -- 172.21.15.33:0/1011300760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 msgr2=0x55acc1abc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1abc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 -- 172.21.15.33:0/1011300760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc19d3fb0 con 0x55acc1ab6d90 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1abc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ab71b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe88011490 tx=0x7fbe88011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=775077ae2a82a693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe77fff640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe780a2b90 0x7fbe780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe757fa640 1 -- 172.21.15.33:0/1011300760 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8801e070 con 0x55acc1ab6d90 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe780a2b90 0x7fbe780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe757fa640 1 -- 172.21.15.33:0/1011300760 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8800fa20 con 0x55acc1ab6d90 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.718+0000 7fbe757fa640 1 -- 172.21.15.33:0/1011300760 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88019bc0 con 0x55acc1ab6d90 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe77fff640 1 -- 172.21.15.33:0/668961511 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe840d0870 con 0x7fbe90101890 2024-04-24T23:20:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe780a2b90 0x7fbe780a5050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55acc1ab7b80 tx=0x7fbe80009380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 -- 172.21.15.33:0/1011300760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 msgr2=0x55acc1ab71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ab71b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe88011490 tx=0x7fbe88011950 comp rx=0 tx=0).stop 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 -- 172.21.15.33:0/1011300760 shutdown_connections 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1abed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1abc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 --2- 172.21.15.33:0/1011300760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ab71b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 -- 172.21.15.33:0/1011300760 >> 172.21.15.33:0/1011300760 conn(0x55acc1a7e950 msgr2=0x55acc1ab2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 -- 172.21.15.33:0/1011300760 shutdown_connections 2024-04-24T23:20:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.719+0000 7fbe97db8740 1 -- 172.21.15.33:0/1011300760 wait complete. 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 -- start start 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 Processor -- start 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1abc8f0 0x55acc1ac87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acc1abf8a0 con 0x55acc1ab6d90 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55acc19d5e90 con 0x55acc1ab7780 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55acc1abfa20 con 0x55acc1abc8f0 2024-04-24T23:20:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe97db8740 10 client.0 _ll_register_callbacks cb 0x55acc1a11970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1abc8f0 0x55acc1ac87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52848/0 (socket says 172.21.15.33:52848) 2024-04-24T23:20:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 -- 172.21.15.33:0/369481176 learned_addr learned my addr 172.21.15.33:0/369481176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 -- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1abc8f0 msgr2=0x55acc1ac87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1abc8f0 0x55acc1ac87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 -- 172.21.15.33:0/369481176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 msgr2=0x55acc1ac1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8ed9d640 1 -- 172.21.15.33:0/369481176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc17f67e0 con 0x55acc1ab6d90 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.720+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1abc8f0 0x55acc1ac87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1240 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbe80031ae0 tx=0x7fbe8002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4fd7c914834c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe72ff5640 1 -- 172.21.15.33:0/369481176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80003e80 con 0x55acc1ab6d90 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe72ff5640 1 -- 172.21.15.33:0/369481176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe80004250 con 0x55acc1ab6d90 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe8800fa20 con 0x55acc1ab6d90 2024-04-24T23:20:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe72ff5640 1 -- 172.21.15.33:0/369481176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe80004450 con 0x55acc1ab6d90 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 10 client.17304 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acc1856f70 con 0x55acc1ab6d90 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe72ff5640 1 -- 172.21.15.33:0/369481176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8800fa20 con 0x55acc1ab6d90 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe72ff5640 1 -- 172.21.15.33:0/369481176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe80059070 con 0x55acc1ab6d90 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 10 client.17304 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 20 client.17304 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1acccb0 0x55acc1acf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 4 client.17304 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.721+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acc19ee5a0 con 0x55acc1acccb0 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.722+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1acccb0 0x55acc1acf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.722+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1acccb0 0x55acc1acf150 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e07b044dd6d05df server_cookie=2bedc9c3e0fc4f04 in_seq=0 out_seq=0 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.722+0000 7fbe72ff5640 10 client.17304 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe72ff5640 1 -- 172.21.15.33:0/369481176 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acc19ee5a0 con 0x55acc1acccb0 2024-04-24T23:20:48.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe72ff5640 10 client.17304 handle_command_reply: tid=0 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 client.17304 shutdown 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 msgr2=0x55acc1ac1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1240 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbe80031ae0 tx=0x7fbe8002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 shutdown_connections 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1acccb0 0x55acc1acf150 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1abc8f0 0x55acc1ac87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe72ff5640 0 client.17304 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 --2- 172.21.15.33:0/369481176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1240 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 >> 172.21.15.33:0/369481176 conn(0x55acc1a7e950 msgr2=0x55acc1ab1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 shutdown_connections 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 1 -- 172.21.15.33:0/369481176 wait complete. 2024-04-24T23:20:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.723+0000 7fbe97db8740 20 client.17304 trim_cache size 0 max 16384 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 Processor -- start 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 -- start start 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1ac8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1ab54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c012490 con 0x55acc1ab7780 2024-04-24T23:20:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c00fd60 con 0x55acc1abc8f0 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe97db8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c00dbb0 con 0x55acc1ab6d90 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1ac8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1ac8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52860/0 (socket says 172.21.15.33:52860) 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 -- 172.21.15.33:0/2750656964 learned_addr learned my addr 172.21.15.33:0/2750656964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1ab54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 -- 172.21.15.33:0/2750656964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 msgr2=0x55acc1ac1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 -- 172.21.15.33:0/2750656964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 msgr2=0x55acc1ab54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1ab54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.742+0000 7fbe8e59c640 1 -- 172.21.15.33:0/2750656964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc16ee170 con 0x55acc1ab7780 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1ac8780 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbe84004570 tx=0x7fbe840cc900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8d847515b3eea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe74ff9640 1 -- 172.21.15.33:0/2750656964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840b3030 con 0x55acc1ab7780 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe74ff9640 1 -- 172.21.15.33:0/2750656964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8409be90 con 0x55acc1ab7780 2024-04-24T23:20:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe74ff9640 1 -- 172.21.15.33:0/2750656964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84099a90 con 0x55acc1ab7780 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 -- 172.21.15.33:0/2750656964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 msgr2=0x55acc1ac8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1ac8780 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbe84004570 tx=0x7fbe840cc900 comp rx=0 tx=0).stop 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 -- 172.21.15.33:0/2750656964 shutdown_connections 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1abc8f0 0x55acc1ab54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1ab7780 0x55acc1ac8780 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 --2- 172.21.15.33:0/2750656964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 -- 172.21.15.33:0/2750656964 >> 172.21.15.33:0/2750656964 conn(0x55acc1a7e950 msgr2=0x55acc1ac0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 -- 172.21.15.33:0/2750656964 shutdown_connections 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.743+0000 7fbe97db8740 1 -- 172.21.15.33:0/2750656964 wait complete. 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 -- start start 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 Processor -- start 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 0x55acc1ac78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c00e220 con 0x55acc1abc8f0 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c00f690 con 0x55acc1ab7780 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c00e8f0 con 0x55acc1ab6d90 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe97db8740 10 client.0 _ll_register_callbacks cb 0x7fbe80009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 0x55acc1ac78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 0x55acc1ac78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52874/0 (socket says 172.21.15.33:52874) 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 -- 172.21.15.33:0/546639408 learned_addr learned my addr 172.21.15.33:0/546639408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8e59c640 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 -- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 msgr2=0x55acc1ac6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 -- 172.21.15.33:0/546639408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 msgr2=0x55acc1ac69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.744+0000 7fbe8f59e640 1 -- 172.21.15.33:0/546639408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc1ac7ec0 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe8f59e640 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 0x55acc1ac78d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbe88011490 tx=0x7fbe8800f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723a11d73bec2c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe72ff5640 1 -- 172.21.15.33:0/546639408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8802b070 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe72ff5640 1 -- 172.21.15.33:0/546639408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe88005250 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe8409be90 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe72ff5640 1 -- 172.21.15.33:0/546639408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe88036050 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 10 client.17310 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acc1b337e0 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe72ff5640 1 -- 172.21.15.33:0/546639408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8409be90 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe72ff5640 1 -- 172.21.15.33:0/546639408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe88024450 con 0x55acc1abc8f0 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 10 client.17310 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 20 client.17310 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1b34020 0x55acc1b36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 4 client.17310 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55acc1a27800 con 0x55acc1b34020 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.745+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1b34020 0x55acc1b36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.746+0000 7fbe8ed9d640 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1b34020 0x55acc1b36440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94d220288b6ee14 server_cookie=877791f8d844b1d4 in_seq=0 out_seq=0 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.746+0000 7fbe72ff5640 10 client.17310 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe72ff5640 1 -- 172.21.15.33:0/546639408 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55acc1a27800 con 0x55acc1b34020 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe72ff5640 10 client.17310 handle_command_reply: tid=0 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 client.17310 shutdown 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 msgr2=0x55acc1ac78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 0x55acc1ac78d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbe88011490 tx=0x7fbe8800f9d0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 shutdown_connections 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acc1b34020 0x55acc1b36440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acc1abc8f0 0x55acc1ac78d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acc1ab7780 0x55acc1ac69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe72ff5640 0 client.17310 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 --2- 172.21.15.33:0/546639408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acc1ab6d90 0x55acc1ac6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 >> 172.21.15.33:0/546639408 conn(0x55acc1a7e950 msgr2=0x55acc1ab10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 shutdown_connections 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 1 -- 172.21.15.33:0/546639408 wait complete. 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.747+0000 7fbe97db8740 20 client.17310 trim_cache size 0 max 16384 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0cc30ce5-8bac-47fe-8302-b61ffd4095c1", 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:20:48.758 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:20:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe780a2b90 msgr2=0x7fbe780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe780a2b90 0x7fbe780a5050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55acc1ab7b80 tx=0x7fbe80009380 comp rx=0 tx=0).stop 2024-04-24T23:20:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90101890 msgr2=0x7fbe90102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90101890 0x7fbe90102c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400e9f0 tx=0x7fbe8400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 shutdown_connections 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbe780a2b90 0x7fbe780a5050 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102250 0x7fbe901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe90101890 0x7fbe90102c70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 --2- 172.21.15.33:0/668961511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe901006a0 0x7fbe90104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 >> 172.21.15.33:0/668961511 conn(0x7fbe900fbc30 msgr2=0x7fbe900fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 shutdown_connections 2024-04-24T23:20:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.748+0000 7fbe950a5640 1 -- 172.21.15.33:0/668961511 wait complete. 2024-04-24T23:20:48.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:20:48.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 -- 172.21.15.33:0/1044745198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 msgr2=0x7f1120108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 --2- 172.21.15.33:0/1044745198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f1120108ae0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1118009be0 tx=0x7f111802a6c0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 -- 172.21.15.33:0/1044745198 shutdown_connections 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 --2- 172.21.15.33:0/1044745198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f1120108ae0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 --2- 172.21.15.33:0/1044745198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1120101890 0x7f1120101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 --2- 172.21.15.33:0/1044745198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f1120100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 -- 172.21.15.33:0/1044745198 >> 172.21.15.33:0/1044745198 conn(0x7f11200fbc30 msgr2=0x7f11200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 -- 172.21.15.33:0/1044745198 shutdown_connections 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.866+0000 7f11272f7640 1 -- 172.21.15.33:0/1044745198 wait complete. 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 Processor -- start 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 -- start start 2024-04-24T23:20:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f11201045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1120101890 0x7f1120102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f11201031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f112010c2b0 con 0x7f1120102250 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f112010c130 con 0x7f11201006a0 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f11272f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f112010c430 con 0x7f1120101890 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1125894640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f11201031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1125093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f11201045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1125093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f11201045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:38744/0 (socket says 172.21.15.33:38744) 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1125093640 1 -- 172.21.15.33:0/2985969846 learned_addr learned my addr 172.21.15.33:0/2985969846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1125894640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f11201031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52890/0 (socket says 172.21.15.33:52890) 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1124892640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1120101890 0x7f1120102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.867+0000 7f1125093640 1 -- 172.21.15.33:0/2985969846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1120101890 msgr2=0x7f1120102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f1125093640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1120101890 0x7f1120102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f1125093640 1 -- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 msgr2=0x7f11201031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f1125093640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f11201031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f1125093640 1 -- 172.21.15.33:0/2985969846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11201039d0 con 0x7f11201006a0 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f1125093640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f11201045c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e9a0 tx=0x7f111c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f11167fc640 1 -- 172.21.15.33:0/2985969846 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c00cd80 con 0x7f11201006a0 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11201a1bc0 con 0x7f11201006a0 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11201a2160 con 0x7f11201006a0 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f11167fc640 1 -- 172.21.15.33:0/2985969846 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c010910 con 0x7f11201006a0 2024-04-24T23:20:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.868+0000 7f11167fc640 1 -- 172.21.15.33:0/2985969846 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c0189c0 con 0x7f11201006a0 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 Processor -- start 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 -- start start 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f11167fc640 1 -- 172.21.15.33:0/2985969846 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f111c010ac0 con 0x7f11201006a0 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b382660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b382c30 0x555f2b387730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 0x555f2b38a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f2b2e66b0 con 0x555f2b387da0 2024-04-24T23:20:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f2b28e880 con 0x555f2b382c30 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1129ffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555f2b2e2670 con 0x555f2b382240 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1124892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 0x555f2b38a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b382660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b382c30 0x555f2b387730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b382660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36914/0 (socket says 172.21.15.33:36914) 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1124892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 0x555f2b38a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52892/0 (socket says 172.21.15.33:52892) 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 -- 172.21.15.33:0/368363771 learned_addr learned my addr 172.21.15.33:0/368363771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 -- 172.21.15.33:0/368363771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b382c30 msgr2=0x555f2b387730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b382c30 0x555f2b387730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 -- 172.21.15.33:0/368363771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 msgr2=0x555f2b38a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 0x555f2b38a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1125894640 1 -- 172.21.15.33:0/368363771 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2b0c47d0 con 0x555f2b382240 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f1124892640 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 0x555f2b38a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.871+0000 7f11167fc640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f110c0a2c10 0x7f110c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.872+0000 7f1125894640 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b382660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1118009be0 tx=0x7f1118005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfb90305183f1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.872+0000 7f1125093640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f110c0a2c10 0x7f110c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.872+0000 7f11167fc640 1 -- 172.21.15.33:0/2985969846 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f111c0cf6e0 con 0x7f11201006a0 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.872+0000 7f110b7fe640 1 -- 172.21.15.33:0/368363771 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118042030 con 0x555f2b382240 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.872+0000 7f110b7fe640 1 -- 172.21.15.33:0/368363771 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118037040 con 0x555f2b382240 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.872+0000 7f110b7fe640 1 -- 172.21.15.33:0/368363771 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118036070 con 0x555f2b382240 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- 172.21.15.33:0/368363771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 msgr2=0x555f2b382660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1125093640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f110c0a2c10 0x7f110c0a50d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x555f2b383030 tx=0x7f111c0d5550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b382660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1118009be0 tx=0x7f1118005ec0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- 172.21.15.33:0/368363771 shutdown_connections 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b387da0 0x555f2b38a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b382c30 0x555f2b387730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- 172.21.15.33:0/368363771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b382660 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- 172.21.15.33:0/368363771 >> 172.21.15.33:0/368363771 conn(0x555f2b37d8b0 msgr2=0x555f2b37dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- 172.21.15.33:0/368363771 shutdown_connections 2024-04-24T23:20:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- 172.21.15.33:0/368363771 wait complete. 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- start start 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 Processor -- start 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 0x555f2b38c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382c30 0x555f2b38cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b393bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f2b38acf0 con 0x555f2b382240 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f2b2a1f50 con 0x555f2b387da0 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555f2b38ae70 con 0x555f2b382c30 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.873+0000 7f1129ffd740 10 client.0 _ll_register_callbacks cb 0x555f2b2dd000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 0x555f2b38c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 0x555f2b38c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52908/0 (socket says 172.21.15.33:52908) 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1124892640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382c30 0x555f2b38cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 -- 172.21.15.33:0/773570424 learned_addr learned my addr 172.21.15.33:0/773570424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b393bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 -- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382c30 msgr2=0x555f2b38cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382c30 0x555f2b38cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 -- 172.21.15.33:0/773570424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 msgr2=0x555f2b393bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b393bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 -- 172.21.15.33:0/773570424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2affa3f0 con 0x555f2b382240 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1125894640 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 0x555f2b38c640 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f11180099a0 tx=0x7f1118030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c20820824043eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1108ff9640 1 -- 172.21.15.33:0/773570424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111804e020 con 0x555f2b382240 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1118037040 con 0x555f2b382240 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1108ff9640 1 -- 172.21.15.33:0/773570424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1118031440 con 0x555f2b382240 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1108ff9640 1 -- 172.21.15.33:0/773570424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1118059050 con 0x555f2b382240 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1129ffd740 10 client.17322 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.874+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f2afba0f0 con 0x555f2b382240 2024-04-24T23:20:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1108ff9640 1 -- 172.21.15.33:0/773570424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118037040 con 0x555f2b382240 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1108ff9640 1 -- 172.21.15.33:0/773570424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f111803c040 con 0x555f2b382240 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1129ffd740 10 client.17322 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1129ffd740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1129ffd740 20 client.17322 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1129ffd740 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3980b0 0x555f2b39a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1129ffd740 4 client.17322 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f2b2f3bf0 con 0x555f2b3980b0 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.875+0000 7f1124892640 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3980b0 0x555f2b39a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1124892640 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3980b0 0x555f2b39a4a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7800e52126252f server_cookie=ad7b697d4caa75eb in_seq=0 out_seq=0 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1108ff9640 10 client.17322 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1108ff9640 1 -- 172.21.15.33:0/773570424 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f2b2f3bf0 con 0x555f2b3980b0 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1108ff9640 10 client.17322 handle_command_reply: tid=0 2024-04-24T23:20:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 client.17322 shutdown 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 msgr2=0x555f2b38c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 0x555f2b38c640 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f11180099a0 tx=0x7f1118030fd0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 shutdown_connections 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3980b0 0x555f2b39a4a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b393bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382c30 0x555f2b38cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 --2- 172.21.15.33:0/773570424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382240 0x555f2b38c640 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1108ff9640 0 client.17322 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 >> 172.21.15.33:0/773570424 conn(0x555f2b37d8b0 msgr2=0x555f2b37caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.876+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 shutdown_connections 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.877+0000 7f1129ffd740 1 -- 172.21.15.33:0/773570424 wait complete. 2024-04-24T23:20:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.877+0000 7f1129ffd740 20 client.17322 trim_cache size 0 max 16384 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 Processor -- start 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 -- start start 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b38cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b393b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b37dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11201aedd0 con 0x555f2b382c30 2024-04-24T23:20:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11201ac6a0 con 0x555f2b387da0 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1129ffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11201aa4f0 con 0x555f2b382240 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1124892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b393b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1124892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b393b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52924/0 (socket says 172.21.15.33:52924) 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1124892640 1 -- 172.21.15.33:0/1603594429 learned_addr learned my addr 172.21.15.33:0/1603594429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.894+0000 7f1125894640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b38cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1125894640 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b37dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1124892640 1 -- 172.21.15.33:0/1603594429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 msgr2=0x555f2b38cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1124892640 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b38cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1124892640 1 -- 172.21.15.33:0/1603594429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 msgr2=0x555f2b37dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1124892640 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b37dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1124892640 1 -- 172.21.15.33:0/1603594429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2b36f090 con 0x555f2b382c30 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f1124892640 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b393b50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x555f2b37e820 tx=0x7f1110005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4d5aa317bea0f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f110affd640 1 -- 172.21.15.33:0/1603594429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100200f0 con 0x555f2b382c30 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f110affd640 1 -- 172.21.15.33:0/1603594429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110020770 con 0x555f2b382c30 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.895+0000 7f110affd640 1 -- 172.21.15.33:0/1603594429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110010b70 con 0x555f2b382c30 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 -- 172.21.15.33:0/1603594429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 msgr2=0x555f2b393b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b393b50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x555f2b37e820 tx=0x7f1110005f90 comp rx=0 tx=0).stop 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 -- 172.21.15.33:0/1603594429 shutdown_connections 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b37dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b393b50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1603594429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b38cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 -- 172.21.15.33:0/1603594429 >> 172.21.15.33:0/1603594429 conn(0x555f2b37d8b0 msgr2=0x555f2b380860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 -- 172.21.15.33:0/1603594429 shutdown_connections 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 -- 172.21.15.33:0/1603594429 wait complete. 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 -- start start 2024-04-24T23:20:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.896+0000 7f1129ffd740 1 Processor -- start 2024-04-24T23:20:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b391770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b391cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b392c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11201aab60 con 0x555f2b382c30 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11201abfd0 con 0x555f2b387da0 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11201ab230 con 0x555f2b382240 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 10 client.0 _ll_register_callbacks cb 0x7f11100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b391cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b391cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52928/0 (socket says 172.21.15.33:52928) 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 -- 172.21.15.33:0/1467793684 learned_addr learned my addr 172.21.15.33:0/1467793684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1125894640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b391770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1125894640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b392c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 -- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 msgr2=0x555f2b391770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b391770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 -- 172.21.15.33:0/1467793684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 msgr2=0x555f2b392c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b392c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 -- 172.21.15.33:0/1467793684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2b393260 con 0x555f2b382c30 2024-04-24T23:20:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1125894640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b392c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1124892640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b391cb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11100107c0 tx=0x7f1110005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71e276fdce87f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1108ff9640 1 -- 172.21.15.33:0/1467793684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110030070 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1108ff9640 1 -- 172.21.15.33:0/1467793684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11100200f0 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.897+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1110020770 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1108ff9640 1 -- 172.21.15.33:0/1467793684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f111003b050 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 10 client.17328 fetch_fsmap learned FSMap version 27 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f2b3fe9a0 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1108ff9640 1 -- 172.21.15.33:0/1467793684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110020770 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1108ff9640 1 -- 172.21.15.33:0/1467793684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f111002c550 con 0x555f2b382c30 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 10 client.17328 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 20 client.17328 populate_metadata read hostname 'smithi033' 2024-04-24T23:20:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3ff2c0 0x555f2b4016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 4 client.17328 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:20:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f2b2edf40 con 0x555f2b3ff2c0 2024-04-24T23:20:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1125894640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3ff2c0 0x555f2b4016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1108ff9640 10 client.17328 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.898+0000 7f1125894640 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3ff2c0 0x555f2b4016e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71adb020bd578964 server_cookie=f7277095200699b9 in_seq=0 out_seq=0 2024-04-24T23:20:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1108ff9640 1 -- 172.21.15.33:0/1467793684 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555f2b2edf40 con 0x555f2b3ff2c0 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1108ff9640 10 client.17328 handle_command_reply: tid=0 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 client.17328 shutdown 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 msgr2=0x555f2b391cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b391cb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11100107c0 tx=0x7f1110005d90 comp rx=0 tx=0).stop 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 shutdown_connections 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555f2b3ff2c0 0x555f2b4016e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f2b387da0 0x555f2b392c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f2b382c30 0x555f2b391cb0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 --2- 172.21.15.33:0/1467793684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2b382240 0x555f2b391770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1108ff9640 0 client.17328 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 >> 172.21.15.33:0/1467793684 conn(0x555f2b37d8b0 msgr2=0x555f2b38c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 shutdown_connections 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 1 -- 172.21.15.33:0/1467793684 wait complete. 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.899+0000 7f1129ffd740 20 client.17328 trim_cache size 0 max 16384 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stdout: "0cc30ce5-8bac-47fe-8302-b61ffd4095c1": { 2024-04-24T23:20:48.910 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:20:48.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0cc30ce5-8bac-47fe-8302-b61ffd4095c1", 2024-04-24T23:20:48.911 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:20:48.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:20:48.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:20:48.911 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f110c0a2c10 msgr2=0x7f110c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f110c0a2c10 0x7f110c0a50d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x555f2b383030 tx=0x7f111c0d5550 comp rx=0 tx=0).stop 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 msgr2=0x7f11201045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f11201045c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e9a0 tx=0x7f111c00ee60 comp rx=0 tx=0).stop 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 shutdown_connections 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f110c0a2c10 0x7f110c0a50d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1120102250 0x7f11201031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1120101890 0x7f1120102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 --2- 172.21.15.33:0/2985969846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11201006a0 0x7f11201045c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 >> 172.21.15.33:0/2985969846 conn(0x7f11200fbc30 msgr2=0x7f11200fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 shutdown_connections 2024-04-24T23:20:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:20:48.900+0000 7f11272f7640 1 -- 172.21.15.33:0/2985969846 wait complete. 2024-04-24T23:20:48.915 INFO:tasks.cephfs.filesystem:scrub status for tag:0cc30ce5-8bac-47fe-8302-b61ffd4095c1 - {'path': '/', 'tag': '0cc30ce5-8bac-47fe-8302-b61ffd4095c1', 'options': 'recursive,force'} 2024-04-24T23:20:49.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 987758 12.76 MB/sec execute 294 sec latency 22.045 ms 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: pgmap v615: 129 pgs: 129 active+clean; 268 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 3.4 MiB/s wr, 277 op/s 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:49.978 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:49 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: pgmap v615: 129 pgs: 129 active+clean; 268 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 3.4 MiB/s wr, 277 op/s 2024-04-24T23:20:50.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:20:50.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:20:50.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:20:50.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.172 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:49 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: pgmap v615: 129 pgs: 129 active+clean; 268 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 3.4 MiB/s wr, 277 op/s 2024-04-24T23:20:50.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:20:50.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:20:50.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:20:50.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:49 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:20:50.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 989542 12.74 MB/sec execute 295 sec latency 57.441 ms 2024-04-24T23:20:51.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 991842 12.74 MB/sec execute 296 sec latency 30.283 ms 2024-04-24T23:20:51.973 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:51 smithi038 ceph-mon[32450]: pgmap v616: 129 pgs: 129 active+clean; 265 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:20:52.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:51 smithi117 ceph-mon[32529]: pgmap v616: 129 pgs: 129 active+clean; 265 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:20:52.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:51 smithi033 ceph-mon[29201]: pgmap v616: 129 pgs: 129 active+clean; 265 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:20:52.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 994870 12.76 MB/sec execute 297 sec latency 13.511 ms 2024-04-24T23:20:53.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 996414 12.74 MB/sec execute 298 sec latency 14.723 ms 2024-04-24T23:20:54.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:53 smithi038 ceph-mon[32450]: pgmap v617: 129 pgs: 129 active+clean; 264 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-04-24T23:20:54.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:53 smithi117 ceph-mon[32529]: pgmap v617: 129 pgs: 129 active+clean; 264 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-04-24T23:20:54.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:53 smithi033 ceph-mon[29201]: pgmap v617: 129 pgs: 129 active+clean; 264 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-04-24T23:20:54.454 INFO:tasks.workunit.client.0.smithi033.stdout: 1 998790 12.74 MB/sec execute 299 sec latency 9.281 ms 2024-04-24T23:20:55.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1001222 12.74 MB/sec execute 300 sec latency 64.172 ms 2024-04-24T23:20:56.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:55 smithi038 ceph-mon[32450]: pgmap v618: 129 pgs: 129 active+clean; 268 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-04-24T23:20:56.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:55 smithi117 ceph-mon[32529]: pgmap v618: 129 pgs: 129 active+clean; 268 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-04-24T23:20:56.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:55 smithi033 ceph-mon[29201]: pgmap v618: 129 pgs: 129 active+clean; 268 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-04-24T23:20:56.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1002541 12.72 MB/sec execute 301 sec latency 103.185 ms 2024-04-24T23:20:57.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1005356 12.74 MB/sec execute 302 sec latency 8.061 ms 2024-04-24T23:20:58.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:57 smithi038 ceph-mon[32450]: pgmap v619: 129 pgs: 129 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 4.2 MiB/s wr, 272 op/s 2024-04-24T23:20:58.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:57 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:20:58.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:57 smithi117 ceph-mon[32529]: pgmap v619: 129 pgs: 129 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 4.2 MiB/s wr, 272 op/s 2024-04-24T23:20:58.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:57 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:20:58.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:57 smithi033 ceph-mon[29201]: pgmap v619: 129 pgs: 129 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 4.2 MiB/s wr, 272 op/s 2024-04-24T23:20:58.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:57 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:20:58.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1006867 12.72 MB/sec execute 303 sec latency 22.698 ms 2024-04-24T23:20:59.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:58 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:20:59.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:58 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:20:59.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:58 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:20:59.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1009413 12.72 MB/sec execute 304 sec latency 10.770 ms 2024-04-24T23:21:00.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:20:59 smithi038 ceph-mon[32450]: pgmap v620: 129 pgs: 129 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T23:21:00.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:20:59 smithi117 ceph-mon[32529]: pgmap v620: 129 pgs: 129 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T23:21:00.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:20:59 smithi033 ceph-mon[29201]: pgmap v620: 129 pgs: 129 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T23:21:00.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1011889 12.72 MB/sec execute 305 sec latency 58.124 ms 2024-04-24T23:21:01.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1013124 12.70 MB/sec execute 306 sec latency 55.986 ms 2024-04-24T23:21:02.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:01 smithi033 ceph-mon[29201]: pgmap v621: 129 pgs: 129 active+clean; 281 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 3.7 MiB/s wr, 186 op/s 2024-04-24T23:21:02.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:01 smithi038 ceph-mon[32450]: pgmap v621: 129 pgs: 129 active+clean; 281 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 3.7 MiB/s wr, 186 op/s 2024-04-24T23:21:02.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:01 smithi117 ceph-mon[32529]: pgmap v621: 129 pgs: 129 active+clean; 281 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 3.7 MiB/s wr, 186 op/s 2024-04-24T23:21:02.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1015693 12.70 MB/sec execute 307 sec latency 14.614 ms 2024-04-24T23:21:03.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:02 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:03.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:02 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:03.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:02 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:03.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1017192 12.69 MB/sec execute 308 sec latency 28.834 ms 2024-04-24T23:21:04.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:03 smithi038 ceph-mon[32450]: pgmap v622: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 285 op/s 2024-04-24T23:21:04.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:04.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:04.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:21:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:03 smithi117 ceph-mon[32529]: pgmap v622: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 285 op/s 2024-04-24T23:21:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:04.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:21:04.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:03 smithi033 ceph-mon[29201]: pgmap v622: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 285 op/s 2024-04-24T23:21:04.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:04.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:04.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:21:04.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1020019 12.70 MB/sec execute 309 sec latency 11.214 ms 2024-04-24T23:21:05.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1022471 12.70 MB/sec execute 310 sec latency 29.059 ms 2024-04-24T23:21:06.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:05 smithi038 ceph-mon[32450]: pgmap v623: 129 pgs: 129 active+clean; 290 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-04-24T23:21:06.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:05 smithi117 ceph-mon[32529]: pgmap v623: 129 pgs: 129 active+clean; 290 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-04-24T23:21:06.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:05 smithi033 ceph-mon[29201]: pgmap v623: 129 pgs: 129 active+clean; 290 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-04-24T23:21:06.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1023745 12.68 MB/sec execute 311 sec latency 47.515 ms 2024-04-24T23:21:07.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1026548 12.69 MB/sec execute 312 sec latency 22.174 ms 2024-04-24T23:21:08.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:07 smithi038 ceph-mon[32450]: pgmap v624: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 224 op/s 2024-04-24T23:21:08.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:07 smithi117 ceph-mon[32529]: pgmap v624: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 224 op/s 2024-04-24T23:21:08.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:07 smithi033 ceph-mon[29201]: pgmap v624: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 224 op/s 2024-04-24T23:21:08.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1028220 12.68 MB/sec execute 313 sec latency 13.192 ms 2024-04-24T23:21:09.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1030818 12.68 MB/sec execute 314 sec latency 17.593 ms 2024-04-24T23:21:10.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:09 smithi038 ceph-mon[32450]: pgmap v625: 129 pgs: 129 active+clean; 295 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-04-24T23:21:10.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:09 smithi117 ceph-mon[32529]: pgmap v625: 129 pgs: 129 active+clean; 295 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-04-24T23:21:10.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:09 smithi033 ceph-mon[29201]: pgmap v625: 129 pgs: 129 active+clean; 295 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-04-24T23:21:10.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1033751 12.70 MB/sec execute 315 sec latency 12.781 ms 2024-04-24T23:21:11.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1034722 12.66 MB/sec execute 316 sec latency 95.320 ms 2024-04-24T23:21:12.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:11 smithi038 ceph-mon[32450]: pgmap v626: 129 pgs: 129 active+clean; 295 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 222 op/s 2024-04-24T23:21:12.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:11 smithi117 ceph-mon[32529]: pgmap v626: 129 pgs: 129 active+clean; 295 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 222 op/s 2024-04-24T23:21:12.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:11 smithi033 ceph-mon[29201]: pgmap v626: 129 pgs: 129 active+clean; 295 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 222 op/s 2024-04-24T23:21:12.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1037632 12.68 MB/sec execute 317 sec latency 11.963 ms 2024-04-24T23:21:13.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1040198 12.68 MB/sec execute 318 sec latency 18.921 ms 2024-04-24T23:21:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:13 smithi038 ceph-mon[32450]: pgmap v627: 129 pgs: 129 active+clean; 296 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 292 op/s 2024-04-24T23:21:14.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:13 smithi117 ceph-mon[32529]: pgmap v627: 129 pgs: 129 active+clean; 296 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 292 op/s 2024-04-24T23:21:14.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:13 smithi033 ceph-mon[29201]: pgmap v627: 129 pgs: 129 active+clean; 296 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 292 op/s 2024-04-24T23:21:14.455 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1041851 12.66 MB/sec execute 319 sec latency 32.595 ms 2024-04-24T23:21:15.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1044715 12.68 MB/sec execute 320 sec latency 13.460 ms 2024-04-24T23:21:16.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:15 smithi038 ceph-mon[32450]: pgmap v628: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-04-24T23:21:16.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:15 smithi117 ceph-mon[32529]: pgmap v628: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-04-24T23:21:16.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:15 smithi033 ceph-mon[29201]: pgmap v628: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-04-24T23:21:16.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1046305 12.66 MB/sec execute 321 sec latency 84.236 ms 2024-04-24T23:21:17.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1048686 12.66 MB/sec execute 322 sec latency 15.672 ms 2024-04-24T23:21:18.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:17 smithi038 ceph-mon[32450]: pgmap v629: 129 pgs: 129 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-04-24T23:21:18.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:17 smithi117 ceph-mon[32529]: pgmap v629: 129 pgs: 129 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-04-24T23:21:18.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:17 smithi033 ceph-mon[29201]: pgmap v629: 129 pgs: 129 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-04-24T23:21:18.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1051738 12.68 MB/sec execute 323 sec latency 13.001 ms 2024-04-24T23:21:18.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:21:18.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 -- 172.21.15.33:0/1109162900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 msgr2=0x7fad7c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 --2- 172.21.15.33:0/1109162900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c100aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009900 tx=0x7fad6c02a410 comp rx=0 tx=0).stop 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 -- 172.21.15.33:0/1109162900 shutdown_connections 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 --2- 172.21.15.33:0/1109162900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad7c102230 0x7fad7c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 --2- 172.21.15.33:0/1109162900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad7c101870 0x7fad7c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 --2- 172.21.15.33:0/1109162900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c100aa0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.028+0000 7fad8267b640 1 -- 172.21.15.33:0/1109162900 >> 172.21.15.33:0/1109162900 conn(0x7fad7c0fbbf0 msgr2=0x7fad7c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.029+0000 7fad8267b640 1 -- 172.21.15.33:0/1109162900 shutdown_connections 2024-04-24T23:21:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.029+0000 7fad8267b640 1 -- 172.21.15.33:0/1109162900 wait complete. 2024-04-24T23:21:19.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.029+0000 7fad8267b640 1 Processor -- start 2024-04-24T23:21:19.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.029+0000 7fad8267b640 1 -- start start 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.029+0000 7fad8267b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad7c101870 0x7fad7c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad7c102230 0x7fad7c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10c340 con 0x7fad7c100680 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10c1c0 con 0x7fad7c101870 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10c4c0 con 0x7fad7c102230 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32878/0 (socket says 172.21.15.33:32878) 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 -- 172.21.15.33:0/738088941 learned_addr learned my addr 172.21.15.33:0/738088941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad7c101870 0x7fad7c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad80c18640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad7c102230 0x7fad7c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 -- 172.21.15.33:0/738088941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad7c102230 msgr2=0x7fad7c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad7c102230 0x7fad7c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 -- 172.21.15.33:0/738088941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad7c101870 msgr2=0x7fad7c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad7c101870 0x7fad7c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 -- 172.21.15.33:0/738088941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c19f730 con 0x7fad7c100680 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad7bfff640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c197b10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fad6c031440 tx=0x7fad6c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad797fa640 1 -- 172.21.15.33:0/738088941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c03f030 con 0x7fad7c100680 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad797fa640 1 -- 172.21.15.33:0/738088941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6c003ce0 con 0x7fad7c100680 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c19f9c0 con 0x7fad7c100680 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad797fa640 1 -- 172.21.15.33:0/738088941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c035070 con 0x7fad7c100680 2024-04-24T23:21:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.030+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c19feb0 con 0x7fad7c100680 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.033+0000 7fad797fa640 1 -- 172.21.15.33:0/738088941 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fad6c042030 con 0x7fad7c100680 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.033+0000 7fad85382740 1 Processor -- start 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.033+0000 7fad85382740 1 -- start start 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.033+0000 7fad85382740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67240 0x555bb7d67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.033+0000 7fad85382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d6c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad85382740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d6f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad85382740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555bb7ccb6b0 con 0x555bb7d6cda0 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad85382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555bb7c73880 con 0x555bb7d67c30 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad85382740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555bb7cc7670 con 0x555bb7d67240 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad797fa640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad640a2b90 0x7fad640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d6f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d6c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d6c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51568/0 (socket says 172.21.15.33:51568) 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d6f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32890/0 (socket says 172.21.15.33:32890) 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7bfff640 1 -- 172.21.15.33:0/1660625876 learned_addr learned my addr 172.21.15.33:0/1660625876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad80c18640 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67240 0x555bb7d67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad80c18640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad640a2b90 0x7fad640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 -- 172.21.15.33:0/1660625876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67240 msgr2=0x555bb7d67660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67240 0x555bb7d67660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 -- 172.21.15.33:0/1660625876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 msgr2=0x555bb7d6c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad797fa640 1 -- 172.21.15.33:0/738088941 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad6c0ef930 con 0x7fad7c100680 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d6c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 -- 172.21.15.33:0/1660625876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb7aa97d0 con 0x555bb7d6cda0 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7bfff640 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d6c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad80c18640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad640a2b90 0x7fad640a5050 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fad74005c40 tx=0x7fad74009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d6f230 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fad70009bd0 tx=0x7fad70011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115e944b6b39d3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.034+0000 7fad627fc640 1 -- 172.21.15.33:0/1660625876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70004230 con 0x555bb7d6cda0 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad627fc640 1 -- 172.21.15.33:0/1660625876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70009e40 con 0x555bb7d6cda0 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad627fc640 1 -- 172.21.15.33:0/1660625876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70004d50 con 0x555bb7d6cda0 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 -- 172.21.15.33:0/1660625876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 msgr2=0x555bb7d6f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d6f230 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fad70009bd0 tx=0x7fad70011810 comp rx=0 tx=0).stop 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 -- 172.21.15.33:0/1660625876 shutdown_connections 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d6f230 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d6c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 --2- 172.21.15.33:0/1660625876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67240 0x555bb7d67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 -- 172.21.15.33:0/1660625876 >> 172.21.15.33:0/1660625876 conn(0x555bb7d628b0 msgr2=0x555bb7d62cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 -- 172.21.15.33:0/1660625876 shutdown_connections 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.035+0000 7fad85382740 1 -- 172.21.15.33:0/1660625876 wait complete. 2024-04-24T23:21:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 -- start start 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 Processor -- start 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d71660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d71ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d6cda0 0x555bb7d78b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555bb7d6fd50 con 0x555bb7d67240 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555bb7c86f50 con 0x555bb7d67c30 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555bb7d6fed0 con 0x555bb7d6cda0 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad85382740 10 client.0 _ll_register_callbacks cb 0x555bb7cc2000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d71660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d71ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d71660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32900/0 (socket says 172.21.15.33:32900) 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d71ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51584/0 (socket says 172.21.15.33:51584) 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 -- 172.21.15.33:0/3321636154 learned_addr learned my addr 172.21.15.33:0/3321636154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad80c18640 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d6cda0 0x555bb7d78b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 -- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d6cda0 msgr2=0x555bb7d78b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d6cda0 0x555bb7d78b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 -- 172.21.15.33:0/3321636154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 msgr2=0x555bb7d71ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.036+0000 7fad7bfff640 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d71ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad7bfff640 1 -- 172.21.15.33:0/3321636154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb79df3f0 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad7bfff640 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d71660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000c00 tx=0x7fad6c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d760158c6ce0322 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad5b7fe640 1 -- 172.21.15.33:0/3321636154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c049070 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad5b7fe640 1 -- 172.21.15.33:0/3321636154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad6c0b8450 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bb7cd8bf0 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad5b7fe640 1 -- 172.21.15.33:0/3321636154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad6c0bb740 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 10 client.17340 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bb799f0f0 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad5b7fe640 1 -- 172.21.15.33:0/3321636154 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555bb7cd8bf0 con 0x555bb7d67240 2024-04-24T23:21:19.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad5b7fe640 1 -- 172.21.15.33:0/3321636154 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad6c03bca0 con 0x555bb7d67240 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 10 client.17340 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 20 client.17340 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7d7d3f0 0x555bb7d7f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 4 client.17340 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.037+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb7cd2f40 con 0x555bb7d7d3f0 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.038+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7d7d3f0 0x555bb7d7f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.038+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7d7d3f0 0x555bb7d7f880 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c231ee71b02893 server_cookie=2947fd8dc58f610b in_seq=0 out_seq=0 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.038+0000 7fad5b7fe640 10 client.17340 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad5b7fe640 1 -- 172.21.15.33:0/3321636154 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bb7cd2f40 con 0x555bb7d7d3f0 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad5b7fe640 10 client.17340 handle_command_reply: tid=0 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 client.17340 shutdown 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 msgr2=0x555bb7d71660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d71660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000c00 tx=0x7fad6c032ed0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 shutdown_connections 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7d7d3f0 0x555bb7d7f880 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d6cda0 0x555bb7d78b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67c30 0x555bb7d71ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad5b7fe640 0 client.17340 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 --2- 172.21.15.33:0/3321636154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d71660 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 >> 172.21.15.33:0/3321636154 conn(0x555bb7d628b0 msgr2=0x555bb7d61af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 shutdown_connections 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 1 -- 172.21.15.33:0/3321636154 wait complete. 2024-04-24T23:21:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.039+0000 7fad85382740 20 client.17340 trim_cache size 0 max 16384 2024-04-24T23:21:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.057+0000 7fad85382740 1 Processor -- start 2024-04-24T23:21:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.057+0000 7fad85382740 1 -- start start 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad85382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67240 0x555bb7d65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad85382740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad85382740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad85382740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1acc00 con 0x555bb7d6cda0 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad85382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1aa4d0 con 0x555bb7d67240 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad85382740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1a8320 con 0x555bb7d67c30 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad80c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67240 0x555bb7d65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad80c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d65da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32916/0 (socket says 172.21.15.33:32916) 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad80c18640 1 -- 172.21.15.33:0/624009671 learned_addr learned my addr 172.21.15.33:0/624009671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 -- 172.21.15.33:0/624009671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67240 msgr2=0x555bb7d65860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67240 0x555bb7d65860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 -- 172.21.15.33:0/624009671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 msgr2=0x555bb7d65da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d65da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 -- 172.21.15.33:0/624009671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb7d62450 con 0x555bb7d67c30 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7bfff640 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67240 0x555bb7d65860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d78b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad70005e00 tx=0x7fad70009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fead818468152e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.058+0000 7fad61ffb640 1 -- 172.21.15.33:0/624009671 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70030070 con 0x555bb7d67c30 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad61ffb640 1 -- 172.21.15.33:0/624009671 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70004630 con 0x555bb7d67c30 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad61ffb640 1 -- 172.21.15.33:0/624009671 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7001e040 con 0x555bb7d67c30 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 -- 172.21.15.33:0/624009671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 msgr2=0x555bb7d78b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d78b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad70005e00 tx=0x7fad70009910 comp rx=0 tx=0).stop 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 -- 172.21.15.33:0/624009671 shutdown_connections 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d6cda0 0x555bb7d65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d78b30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 --2- 172.21.15.33:0/624009671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d67240 0x555bb7d65860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 -- 172.21.15.33:0/624009671 >> 172.21.15.33:0/624009671 conn(0x555bb7d628b0 msgr2=0x555bb7d71270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 -- 172.21.15.33:0/624009671 shutdown_connections 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.059+0000 7fad85382740 1 -- 172.21.15.33:0/624009671 wait complete. 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 -- start start 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 Processor -- start 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d76800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d76d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d6cda0 0x555bb7d77c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1a8990 con 0x555bb7d67240 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1a9e00 con 0x555bb7d6cda0 2024-04-24T23:21:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1a9060 con 0x555bb7d67c30 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad85382740 10 client.0 _ll_register_callbacks cb 0x7fad6c0ecfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d76800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d76800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32926/0 (socket says 172.21.15.33:32926) 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 -- 172.21.15.33:0/1997797969 learned_addr learned my addr 172.21.15.33:0/1997797969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad80c18640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d6cda0 0x555bb7d77c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d76d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 -- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 msgr2=0x555bb7d76d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d76d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 -- 172.21.15.33:0/1997797969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d6cda0 msgr2=0x555bb7d77c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d6cda0 0x555bb7d77c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.060+0000 7fad7bfff640 1 -- 172.21.15.33:0/1997797969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb7d78250 con 0x555bb7d67240 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad7bfff640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d76800 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0c4bc0 tx=0x7fad6c0b8660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdfa692a8e3c1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad5b7fe640 1 -- 172.21.15.33:0/1997797969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0d8030 con 0x555bb7d67240 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad5b7fe640 1 -- 172.21.15.33:0/1997797969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad6c03b5e0 con 0x555bb7d67240 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad70004630 con 0x555bb7d67240 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad5b7fe640 1 -- 172.21.15.33:0/1997797969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad6c0d0050 con 0x555bb7d67240 2024-04-24T23:21:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 10 client.17346 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bb7de3e70 con 0x555bb7d67240 2024-04-24T23:21:19.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad5b7fe640 1 -- 172.21.15.33:0/1997797969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70004630 con 0x555bb7d67240 2024-04-24T23:21:19.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad5b7fe640 1 -- 172.21.15.33:0/1997797969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad6c03bca0 con 0x555bb7d67240 2024-04-24T23:21:19.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 10 client.17346 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:19.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 20 client.17346 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7de46d0 0x555bb7de6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 4 client.17346 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.061+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bb7d68ba0 con 0x555bb7de46d0 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7de46d0 0x555bb7de6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad7b7fe640 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7de46d0 0x555bb7de6af0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea38d0dec78547f server_cookie=6c70669d5d01f981 in_seq=0 out_seq=0 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad5b7fe640 10 client.17346 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad5b7fe640 1 -- 172.21.15.33:0/1997797969 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bb7d68ba0 con 0x555bb7de46d0 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad5b7fe640 10 client.17346 handle_command_reply: tid=0 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 client.17346 shutdown 2024-04-24T23:21:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 msgr2=0x555bb7d76800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d76800 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0c4bc0 tx=0x7fad6c0b8660 comp rx=0 tx=0).stop 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 shutdown_connections 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555bb7de46d0 0x555bb7de6af0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bb7d6cda0 0x555bb7d77c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb7d67c30 0x555bb7d76d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad5b7fe640 0 client.17346 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 --2- 172.21.15.33:0/1997797969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555bb7d67240 0x555bb7d76800 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 >> 172.21.15.33:0/1997797969 conn(0x555bb7d628b0 msgr2=0x555bb7d61260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.062+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 shutdown_connections 2024-04-24T23:21:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad85382740 1 -- 172.21.15.33:0/1997797969 wait complete. 2024-04-24T23:21:19.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad85382740 20 client.17346 trim_cache size 0 max 16384 2024-04-24T23:21:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:21:19.076 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:21:19.076 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:21:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:21:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad640a2b90 msgr2=0x7fad640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad640a2b90 0x7fad640a5050 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fad74005c40 tx=0x7fad74009500 comp rx=0 tx=0).stop 2024-04-24T23:21:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 msgr2=0x7fad7c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c197b10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fad6c031440 tx=0x7fad6c032ce0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 shutdown_connections 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad640a2b90 0x7fad640a5050 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad7c102230 0x7fad7c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad7c101870 0x7fad7c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 --2- 172.21.15.33:0/738088941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad7c100680 0x7fad7c197b10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 >> 172.21.15.33:0/738088941 conn(0x7fad7c0fbbf0 msgr2=0x7fad7c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 shutdown_connections 2024-04-24T23:21:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.063+0000 7fad8267b640 1 -- 172.21.15.33:0/738088941 wait complete. 2024-04-24T23:21:19.080 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:21:19.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.197+0000 7f1c10415640 1 -- 172.21.15.33:0/3530976982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08102230 msgr2=0x7f1c08108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.197+0000 7f1c10415640 1 --2- 172.21.15.33:0/3530976982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08102230 0x7f1c08108ac0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c0002a410 comp rx=0 tx=0).stop 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- 172.21.15.33:0/3530976982 shutdown_connections 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 --2- 172.21.15.33:0/3530976982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08102230 0x7f1c08108ac0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 --2- 172.21.15.33:0/3530976982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08101870 0x7f1c08101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 --2- 172.21.15.33:0/3530976982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08100680 0x7f1c08100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- 172.21.15.33:0/3530976982 >> 172.21.15.33:0/3530976982 conn(0x7f1c080fbbf0 msgr2=0x7f1c080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- 172.21.15.33:0/3530976982 shutdown_connections 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- 172.21.15.33:0/3530976982 wait complete. 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 Processor -- start 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- start start 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 0x7f1c08104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08101870 0x7f1c08102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08102230 0x7f1c08103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0810c300 con 0x7f1c08100680 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0810c180 con 0x7f1c08101870 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.198+0000 7f1c10415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0810c480 con 0x7f1c08102230 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 0x7f1c08104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 0x7f1c08104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32954/0 (socket says 172.21.15.33:32954) 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 -- 172.21.15.33:0/171996638 learned_addr learned my addr 172.21.15.33:0/171996638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e9b2640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08102230 0x7f1c08103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0d9b0640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08101870 0x7f1c08102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 -- 172.21.15.33:0/171996638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08102230 msgr2=0x7f1c08103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08102230 0x7f1c08103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 -- 172.21.15.33:0/171996638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08101870 msgr2=0x7f1c08102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08101870 0x7f1c08102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 -- 172.21.15.33:0/171996638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c08103a20 con 0x7f1c08100680 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e9b2640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08102230 0x7f1c08103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c0e1b1640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 0x7f1c08104610 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400ebc0 tx=0x7f1c0400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04003b70 con 0x7f1c08100680 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c04003d10 con 0x7f1c08100680 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a1bb0 con 0x7f1c08100680 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04019980 con 0x7f1c08100680 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.199+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a2100 con 0x7f1c08100680 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.200+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c081a2330 con 0x7f1c08100680 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.201+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1c04019b20 con 0x7f1c08100680 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.202+0000 7f1bff7fe640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bf40a2ce0 0x7f1bf40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.202+0000 7f1c0d9b0640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bf40a2ce0 0x7f1bf40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.202+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c040cfb30 con 0x7f1c08100680 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.203+0000 7f1c0d9b0640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bf40a2ce0 0x7f1bf40a51a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1c08104380 tx=0x7f1bf8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.207+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c0409c700 con 0x7f1c08100680 2024-04-24T23:21:19.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.452+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c081a2500 con 0x7f1c08100680 2024-04-24T23:21:19.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.453+0000 7f1bff7fe640 1 -- 172.21.15.33:0/171996638 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f1c0409c000 con 0x7f1c08100680 2024-04-24T23:21:19.455 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:21:19.455 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:21:19.455 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:21:19.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1053938 12.67 MB/sec execute 324 sec latency 17.143 ms 2024-04-24T23:21:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bf40a2ce0 msgr2=0x7f1bf40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bf40a2ce0 0x7f1bf40a51a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1c08104380 tx=0x7f1bf8005f90 comp rx=0 tx=0).stop 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 msgr2=0x7f1c08104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 0x7f1c08104610 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400ebc0 tx=0x7f1c0400c6a0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 shutdown_connections 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1bf40a2ce0 0x7f1bf40a51a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c08102230 0x7f1c08103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c08101870 0x7f1c08102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 --2- 172.21.15.33:0/171996638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c08100680 0x7f1c08104610 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 >> 172.21.15.33:0/171996638 conn(0x7f1c080fbbf0 msgr2=0x7f1c080fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 shutdown_connections 2024-04-24T23:21:19.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.455+0000 7f1c10415640 1 -- 172.21.15.33:0/171996638 wait complete. 2024-04-24T23:21:19.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:21:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 -- 172.21.15.33:0/2333252569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 msgr2=0x7fce10108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 --2- 172.21.15.33:0/2333252569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10108ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fce08009ca0 tx=0x7fce0802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 -- 172.21.15.33:0/2333252569 shutdown_connections 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 --2- 172.21.15.33:0/2333252569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10108ac0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 --2- 172.21.15.33:0/2333252569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce10101870 0x7fce10101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 --2- 172.21.15.33:0/2333252569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10100680 0x7fce10100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 -- 172.21.15.33:0/2333252569 >> 172.21.15.33:0/2333252569 conn(0x7fce100fbbf0 msgr2=0x7fce100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 -- 172.21.15.33:0/2333252569 shutdown_connections 2024-04-24T23:21:19.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.576+0000 7fce15072640 1 -- 172.21.15.33:0/2333252569 wait complete. 2024-04-24T23:21:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 Processor -- start 2024-04-24T23:21:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 -- start start 2024-04-24T23:21:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce10100680 0x7fce10104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10101870 0x7fce10102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce1010c300 con 0x7fce10102230 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce1010c180 con 0x7fce10100680 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.577+0000 7fce15072640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce1010c480 con 0x7fce10101870 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32958/0 (socket says 172.21.15.33:32958) 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce10100680 0x7fce10104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 -- 172.21.15.33:0/655932285 learned_addr learned my addr 172.21.15.33:0/655932285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0e59c640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10101870 0x7fce10102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 -- 172.21.15.33:0/655932285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10101870 msgr2=0x7fce10102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10101870 0x7fce10102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 -- 172.21.15.33:0/655932285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce10100680 msgr2=0x7fce10104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce10100680 0x7fce10104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 -- 172.21.15.33:0/655932285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce10103a20 con 0x7fce10102230 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0e59c640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10101870 0x7fce10102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce0f59e640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10103190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fce08031ae0 tx=0x7fce08031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce07fff640 1 -- 172.21.15.33:0/655932285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08003b70 con 0x7fce10102230 2024-04-24T23:21:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce101a1bb0 con 0x7fce10102230 2024-04-24T23:21:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce07fff640 1 -- 172.21.15.33:0/655932285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce08003d10 con 0x7fce10102230 2024-04-24T23:21:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce07fff640 1 -- 172.21.15.33:0/655932285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0802fc40 con 0x7fce10102230 2024-04-24T23:21:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.578+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce101a2100 con 0x7fce10102230 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.581+0000 7fce17d7d740 1 Processor -- start 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.581+0000 7fce07fff640 1 -- 172.21.15.33:0/655932285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fce08004050 con 0x7fce10102230 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.581+0000 7fce17d7d740 1 -- start start 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae65650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 0x55623ae6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae6ad90 0x55623ae6d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce17d7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55623adc9670 con 0x55623ae65230 2024-04-24T23:21:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce17d7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55623ad71970 con 0x55623ae6ad90 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce17d7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55623adc5630 con 0x55623ae65c20 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 0x55623ae6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae65650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 0x55623ae6a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:48838/0 (socket says 172.21.15.33:48838) 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 -- 172.21.15.33:0/649819990 learned_addr learned my addr 172.21.15.33:0/649819990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0e59c640 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae6ad90 0x55623ae6d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce07fff640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdfc0a2b90 0x7fcdfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 -- 172.21.15.33:0/649819990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae6ad90 msgr2=0x55623ae6d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae6ad90 0x55623ae6d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 -- 172.21.15.33:0/649819990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 msgr2=0x55623ae65650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae65650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 -- 172.21.15.33:0/649819990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623ad95400 con 0x55623ae65c20 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0f59e640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdfc0a2b90 0x7fcdfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 0x55623ae6a720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce00009ca0 tx=0x7fce00011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90113730f43a667c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce07fff640 1 -- 172.21.15.33:0/655932285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fce080f0450 con 0x7fce10102230 2024-04-24T23:21:19.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.582+0000 7fce057fa640 1 -- 172.21.15.33:0/649819990 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00004220 con 0x55623ae65c20 2024-04-24T23:21:19.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce057fa640 1 -- 172.21.15.33:0/649819990 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce000043c0 con 0x55623ae65c20 2024-04-24T23:21:19.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce057fa640 1 -- 172.21.15.33:0/649819990 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00019830 con 0x55623ae65c20 2024-04-24T23:21:19.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 -- 172.21.15.33:0/649819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 msgr2=0x55623ae6a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 0x55623ae6a720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce00009ca0 tx=0x7fce00011810 comp rx=0 tx=0).stop 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce0f59e640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdfc0a2b90 0x7fcdfc0a5050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce080edf30 tx=0x7fce0802a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 -- 172.21.15.33:0/649819990 shutdown_connections 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae6ad90 0x55623ae6d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae65c20 0x55623ae6a720 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 --2- 172.21.15.33:0/649819990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae65650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 -- 172.21.15.33:0/649819990 >> 172.21.15.33:0/649819990 conn(0x55623ae608a0 msgr2=0x55623ae60ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 -- 172.21.15.33:0/649819990 shutdown_connections 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.583+0000 7fce17d7d740 1 -- 172.21.15.33:0/649819990 wait complete. 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 -- start start 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 Processor -- start 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae6f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65c20 0x55623ae6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae76880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55623ae6dd40 con 0x55623ae65230 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55623ad85050 con 0x55623ae65c20 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55623ae6dec0 con 0x55623ae6ad90 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce17d7d740 10 client.0 _ll_register_callbacks cb 0x55623aadcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae6f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae6f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32976/0 (socket says 172.21.15.33:32976) 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 -- 172.21.15.33:0/2436742530 learned_addr learned my addr 172.21.15.33:0/2436742530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65c20 0x55623ae6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae76880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 -- 172.21.15.33:0/2436742530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 msgr2=0x55623ae76880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae76880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 -- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65c20 msgr2=0x55623ae6f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65c20 0x55623ae6f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.584+0000 7fce0e59c640 1 -- 172.21.15.33:0/2436742530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623ad56dc0 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae6f300 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8012fa0 tx=0x7fcdf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405da0a80447b12f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fcdf67fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fcdf67fc640 1 -- 172.21.15.33:0/2436742530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8003f10 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce000043c0 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fcdf67fc640 1 -- 172.21.15.33:0/2436742530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcdf8005020 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fcdf67fc640 1 -- 172.21.15.33:0/2436742530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdf802d050 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 10 client.17364 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55623aadd3f0 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fcdf67fc640 1 -- 172.21.15.33:0/2436742530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce000043c0 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fcdf67fc640 1 -- 172.21.15.33:0/2436742530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcdf801d070 con 0x55623ae65230 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 10 client.17364 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 10 client.17364 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 20 client.17364 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623ae7b110 0x55623ae7d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 4 client.17364 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.585+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55623add6bb0 con 0x55623ae7b110 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.586+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623ae7b110 0x55623ae7d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.586+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623ae7b110 0x55623ae7d580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f3c5123e4f54219 server_cookie=68880b96d5431078 in_seq=0 out_seq=0 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.586+0000 7fcdf67fc640 10 client.17364 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fcdf67fc640 1 -- 172.21.15.33:0/2436742530 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55623add6bb0 con 0x55623ae7b110 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fcdf67fc640 10 client.17364 handle_command_reply: tid=0 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 client.17364 shutdown 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 msgr2=0x55623ae6f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae6f300 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8012fa0 tx=0x7fcdf8004730 comp rx=0 tx=0).stop 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 shutdown_connections 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623ae7b110 0x55623ae7d580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae76880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65c20 0x55623ae6f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fcdf67fc640 0 client.17364 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 --2- 172.21.15.33:0/2436742530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65230 0x55623ae6f300 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 >> 172.21.15.33:0/2436742530 conn(0x55623ae608a0 msgr2=0x55623ae5fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 shutdown_connections 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 1 -- 172.21.15.33:0/2436742530 wait complete. 2024-04-24T23:21:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.587+0000 7fce17d7d740 20 client.17364 trim_cache size 0 max 16384 2024-04-24T23:21:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 Processor -- start 2024-04-24T23:21:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 -- start start 2024-04-24T23:21:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae76840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce101aee60 con 0x55623ae65c20 2024-04-24T23:21:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce101ac730 con 0x55623ae65230 2024-04-24T23:21:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce17d7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce101aa580 con 0x55623ae6ad90 2024-04-24T23:21:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae76840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce0e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce0e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae76840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32992/0 (socket says 172.21.15.33:32992) 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.601+0000 7fce0ed9d640 1 -- 172.21.15.33:0/376522341 learned_addr learned my addr 172.21.15.33:0/376522341 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0e59c640 1 -- 172.21.15.33:0/376522341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 msgr2=0x55623ae637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0e59c640 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0e59c640 1 -- 172.21.15.33:0/376522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 msgr2=0x55623ae76840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0e59c640 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae76840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0e59c640 1 -- 172.21.15.33:0/376522341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623aba77d0 con 0x55623ae6ad90 2024-04-24T23:21:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae76840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce0e59c640 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae63cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8012fd0 tx=0x7fcdf8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21773ecbb1d3180 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce04ff9640 1 -- 172.21.15.33:0/376522341 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8003f10 con 0x55623ae6ad90 2024-04-24T23:21:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce04ff9640 1 -- 172.21.15.33:0/376522341 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf80054e0 con 0x55623ae6ad90 2024-04-24T23:21:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.602+0000 7fce04ff9640 1 -- 172.21.15.33:0/376522341 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8004ce0 con 0x55623ae6ad90 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 -- 172.21.15.33:0/376522341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 msgr2=0x55623ae63cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae63cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8012fd0 tx=0x7fcdf8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 -- 172.21.15.33:0/376522341 shutdown_connections 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae63cf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae76840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 --2- 172.21.15.33:0/376522341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 -- 172.21.15.33:0/376522341 >> 172.21.15.33:0/376522341 conn(0x55623ae608a0 msgr2=0x55623ae5fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 -- 172.21.15.33:0/376522341 shutdown_connections 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 -- 172.21.15.33:0/376522341 wait complete. 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 -- start start 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.603+0000 7fce17d7d740 1 Processor -- start 2024-04-24T23:21:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae75b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce101aabf0 con 0x55623ae65c20 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce101ac060 con 0x55623ae65230 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce101ab2c0 con 0x55623ae6ad90 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 10 client.0 _ll_register_callbacks cb 0x7fcdf80114a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae74bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33000/0 (socket says 172.21.15.33:33000) 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 -- 172.21.15.33:0/519539244 learned_addr learned my addr 172.21.15.33:0/519539244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0e59c640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0e59c640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae75b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 -- 172.21.15.33:0/519539244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 msgr2=0x55623ae75b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae75b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 -- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 msgr2=0x55623ae74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 -- 172.21.15.33:0/519539244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623ae76120 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae74bc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x55623ae66690 tx=0x7fce00003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8c4e9bd6c8ed0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fcdf67fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fcdf67fc640 1 -- 172.21.15.33:0/519539244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00005250 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fcdf67fc640 1 -- 172.21.15.33:0/519539244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fce0001f040 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.604+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf80054e0 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fcdf67fc640 1 -- 172.21.15.33:0/519539244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce0003b050 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 10 client.17370 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55623aee1e00 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fcdf67fc640 1 -- 172.21.15.33:0/519539244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf80054e0 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fcdf67fc640 1 -- 172.21.15.33:0/519539244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fce00019430 con 0x55623ae65c20 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 10 client.17370 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 10 client.17370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 20 client.17370 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623aee2660 0x55623aee4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 4 client.17370 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55623ae5c070 con 0x55623aee2660 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.605+0000 7fce0e59c640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623aee2660 0x55623aee4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fcdf67fc640 10 client.17370 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce0e59c640 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623aee2660 0x55623aee4a50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73d9ae52e1229c12 server_cookie=3c77a90b36188126 in_seq=0 out_seq=0 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fcdf67fc640 1 -- 172.21.15.33:0/519539244 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55623ae5c070 con 0x55623aee2660 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fcdf67fc640 10 client.17370 handle_command_reply: tid=0 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 client.17370 shutdown 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 msgr2=0x55623ae74bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae74bc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x55623ae66690 tx=0x7fce00003ee0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 shutdown_connections 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55623aee2660 0x55623aee4a50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55623ae6ad90 0x55623ae75b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55623ae65c20 0x55623ae74bc0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fcdf67fc640 0 client.17370 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 --2- 172.21.15.33:0/519539244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55623ae65230 0x55623ae74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.606+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 >> 172.21.15.33:0/519539244 conn(0x55623ae608a0 msgr2=0x55623ae5f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 shutdown_connections 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce17d7d740 1 -- 172.21.15.33:0/519539244 wait complete. 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce17d7d740 20 client.17370 trim_cache size 0 max 16384 2024-04-24T23:21:19.616 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdfc0a2b90 msgr2=0x7fcdfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdfc0a2b90 0x7fcdfc0a5050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce080edf30 tx=0x7fce0802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 msgr2=0x7fce10103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10103190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fce08031ae0 tx=0x7fce08031ee0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 shutdown_connections 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdfc0a2b90 0x7fcdfc0a5050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce10102230 0x7fce10103190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce10101870 0x7fce10102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 --2- 172.21.15.33:0/655932285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce10100680 0x7fce10104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 >> 172.21.15.33:0/655932285 conn(0x7fce100fbbf0 msgr2=0x7fce100fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 shutdown_connections 2024-04-24T23:21:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.607+0000 7fce15072640 1 -- 172.21.15.33:0/655932285 wait complete. 2024-04-24T23:21:19.621 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:21:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 -- 172.21.15.33:0/2302829584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 msgr2=0x7ff5581046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 --2- 172.21.15.33:0/2302829584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff5581046c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff554009900 tx=0x7ff55402a410 comp rx=0 tx=0).stop 2024-04-24T23:21:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 -- 172.21.15.33:0/2302829584 shutdown_connections 2024-04-24T23:21:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 --2- 172.21.15.33:0/2302829584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558107530 0x7ff558109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 --2- 172.21.15.33:0/2302829584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558104c00 0x7ff558106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 --2- 172.21.15.33:0/2302829584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff5581046c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 -- 172.21.15.33:0/2302829584 >> 172.21.15.33:0/2302829584 conn(0x7ff5580fbc00 msgr2=0x7ff5580fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 -- 172.21.15.33:0/2302829584 shutdown_connections 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.732+0000 7ff55f838640 1 -- 172.21.15.33:0/2302829584 wait complete. 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 Processor -- start 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 -- start start 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff558197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558104c00 0x7ff558198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558107530 0x7ff55819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff55810d410 con 0x7ff558107530 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff55810d290 con 0x7ff5581022d0 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55f838640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff55810d590 con 0x7ff558104c00 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff558197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55ddd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558107530 0x7ff55819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55cdd3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558104c00 0x7ff558198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff558197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51710/0 (socket says 172.21.15.33:51710) 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.733+0000 7ff55d5d4640 1 -- 172.21.15.33:0/3150177675 learned_addr learned my addr 172.21.15.33:0/3150177675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55cdd3640 1 -- 172.21.15.33:0/3150177675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 msgr2=0x7ff558197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff558197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55cdd3640 1 -- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558107530 msgr2=0x7ff55819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558107530 0x7ff55819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55cdd3640 1 -- 172.21.15.33:0/3150177675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55819f740 con 0x7ff558104c00 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55ddd5640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558107530 0x7ff55819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558104c00 0x7ff558198060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e9c0 tx=0x7ff54800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff54e7fc640 1 -- 172.21.15.33:0/3150177675 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800ccc0 con 0x7ff558104c00 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55819fa30 con 0x7ff558104c00 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff54e7fc640 1 -- 172.21.15.33:0/3150177675 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff548010910 con 0x7ff558104c00 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff54e7fc640 1 -- 172.21.15.33:0/3150177675 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548018880 con 0x7ff558104c00 2024-04-24T23:21:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.734+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55819ff80 con 0x7ff558104c00 2024-04-24T23:21:19.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.736+0000 7ff54e7fc640 1 -- 172.21.15.33:0/3150177675 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff548010ac0 con 0x7ff558104c00 2024-04-24T23:21:19.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 Processor -- start 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 -- start start 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e7743a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7748b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e774b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5615e76a3670 con 0x5615e7743640 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5615e764b970 con 0x5615e77491a0 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff562629740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615e769f630 con 0x5615e7744030 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff55ddd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e7743a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff55ddd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e7743a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33040/0 (socket says 172.21.15.33:33040) 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff55ddd5640 1 -- 172.21.15.33:0/1739062749 learned_addr learned my addr 172.21.15.33:0/1739062749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e774b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7748b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff54e7fc640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5440a2b90 0x7ff5440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.737+0000 7ff55d5d4640 1 -- 172.21.15.33:0/1739062749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 msgr2=0x5615e774b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e774b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55d5d4640 1 -- 172.21.15.33:0/1739062749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 msgr2=0x5615e7743a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e7743a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55d5d4640 1 -- 172.21.15.33:0/1739062749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e766f400 con 0x5615e7744030 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff54e7fc640 1 -- 172.21.15.33:0/3150177675 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff5480cf790 con 0x7ff558104c00 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55ddd5640 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e7743a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55ddd5640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5440a2b90 0x7ff5440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7748b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff5540314e0 tx=0x7ff554005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2cd85ccdb19331 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff5437fe640 1 -- 172.21.15.33:0/1739062749 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554040070 con 0x5615e7744030 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff5437fe640 1 -- 172.21.15.33:0/1739062749 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff554005030 con 0x5615e7744030 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.738+0000 7ff5437fe640 1 -- 172.21.15.33:0/1739062749 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554035040 con 0x5615e7744030 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 -- 172.21.15.33:0/1739062749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 msgr2=0x5615e7748b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7748b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff5540314e0 tx=0x7ff554005dd0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff55ddd5640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5440a2b90 0x7ff5440a5050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5615e7744890 tx=0x7ff550004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 -- 172.21.15.33:0/1739062749 shutdown_connections 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e774b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7748b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 --2- 172.21.15.33:0/1739062749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e7743a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 -- 172.21.15.33:0/1739062749 >> 172.21.15.33:0/1739062749 conn(0x5615e773a8a0 msgr2=0x5615e773aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 -- 172.21.15.33:0/1739062749 shutdown_connections 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 -- 172.21.15.33:0/1739062749 wait complete. 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 -- start start 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.739+0000 7ff562629740 1 Processor -- start 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e774d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e774dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7754cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5615e774c0f0 con 0x5615e7743640 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5615e765f050 con 0x5615e77491a0 2024-04-24T23:21:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615e774c270 con 0x5615e7744030 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 10 client.0 _ll_register_callbacks cb 0x5615e73b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e774d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e774dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e774d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33052/0 (socket says 172.21.15.33:33052) 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55ddd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7754cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55d5d4640 1 -- 172.21.15.33:0/1311474792 learned_addr learned my addr 172.21.15.33:0/1311474792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 -- 172.21.15.33:0/1311474792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 msgr2=0x5615e7754cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7754cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 -- 172.21.15.33:0/1311474792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 msgr2=0x5615e774d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e774d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 -- 172.21.15.33:0/1311474792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e7630dc0 con 0x5615e7744030 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e774d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e774dc70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff54809b9b0 tx=0x7ff548010ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b76dada7a593f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff540ff9640 1 -- 172.21.15.33:0/1311474792 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5480cfd80 con 0x5615e7744030 2024-04-24T23:21:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff554005030 con 0x5615e7744030 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.740+0000 7ff540ff9640 1 -- 172.21.15.33:0/1311474792 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff54809b100 con 0x5615e7744030 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff540ff9640 1 -- 172.21.15.33:0/1311474792 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff54809ffb0 con 0x5615e7744030 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 10 client.26893 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615e73b73f0 con 0x5615e7744030 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff540ff9640 1 -- 172.21.15.33:0/1311474792 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff554005030 con 0x5615e7744030 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff540ff9640 1 -- 172.21.15.33:0/1311474792 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff54809ab70 con 0x5615e7744030 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 10 client.26893 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 10 client.26893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 20 client.26893 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77595a0 0x5615e775ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 4 client.26893 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e76b0bb0 con 0x5615e77595a0 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.741+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77595a0 0x5615e775ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77595a0 0x5615e775ba10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25960c9da5d4e7e1 server_cookie=98e9747caa1a0b87 in_seq=0 out_seq=0 2024-04-24T23:21:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff540ff9640 10 client.26893 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff540ff9640 1 -- 172.21.15.33:0/1311474792 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615e76b0bb0 con 0x5615e77595a0 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff540ff9640 10 client.26893 handle_command_reply: tid=0 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 client.26893 shutdown 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 msgr2=0x5615e774dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e774dc70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff54809b9b0 tx=0x7ff548010ef0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 shutdown_connections 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77595a0 0x5615e775ba10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7754cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff540ff9640 0 client.26893 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e774dc70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 --2- 172.21.15.33:0/1311474792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e774d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.742+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 >> 172.21.15.33:0/1311474792 conn(0x5615e773a8a0 msgr2=0x5615e7739b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.743+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 shutdown_connections 2024-04-24T23:21:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.743+0000 7ff562629740 1 -- 172.21.15.33:0/1311474792 wait complete. 2024-04-24T23:21:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.743+0000 7ff562629740 20 client.26893 trim_cache size 0 max 16384 2024-04-24T23:21:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 Processor -- start 2024-04-24T23:21:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 -- start start 2024-04-24T23:21:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7743640 0x5615e774ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 0x5615e7754c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e773d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581aaa40 con 0x5615e7744030 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581a8310 con 0x5615e77491a0 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff562629740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581a6160 con 0x5615e7743640 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff55cdd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 0x5615e7754c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7743640 0x5615e774ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff55cdd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 0x5615e7754c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33058/0 (socket says 172.21.15.33:33058) 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff55cdd3640 1 -- 172.21.15.33:0/533893524 learned_addr learned my addr 172.21.15.33:0/533893524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.757+0000 7ff55ddd5640 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e773d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55cdd3640 1 -- 172.21.15.33:0/533893524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7743640 msgr2=0x5615e774ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7743640 0x5615e774ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55cdd3640 1 -- 172.21.15.33:0/533893524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 msgr2=0x5615e773d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e773d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55cdd3640 1 -- 172.21.15.33:0/533893524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e74817d0 con 0x5615e7744030 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7743640 0x5615e774ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 0x5615e7754c70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c6a0 tx=0x7ff54809bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc5f701a8a23c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff542ffd640 1 -- 172.21.15.33:0/533893524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54809ab70 con 0x5615e7744030 2024-04-24T23:21:19.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff542ffd640 1 -- 172.21.15.33:0/533893524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5480b8020 con 0x5615e7744030 2024-04-24T23:21:19.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.758+0000 7ff542ffd640 1 -- 172.21.15.33:0/533893524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5480a1380 con 0x5615e7744030 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 -- 172.21.15.33:0/533893524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 msgr2=0x5615e7754c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 0x5615e7754c70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c6a0 tx=0x7ff54809bb90 comp rx=0 tx=0).stop 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 -- 172.21.15.33:0/533893524 shutdown_connections 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e773d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7744030 0x5615e7754c70 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 --2- 172.21.15.33:0/533893524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7743640 0x5615e774ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 -- 172.21.15.33:0/533893524 >> 172.21.15.33:0/533893524 conn(0x5615e773a8a0 msgr2=0x5615e7739b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 -- 172.21.15.33:0/533893524 shutdown_connections 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 -- 172.21.15.33:0/533893524 wait complete. 2024-04-24T23:21:19.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 -- start start 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.759+0000 7ff562629740 1 Processor -- start 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e77521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7752710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7753720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581a67d0 con 0x5615e7743640 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581a7c40 con 0x5615e77491a0 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581a6ea0 con 0x5615e7744030 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 10 client.0 _ll_register_callbacks cb 0x7ff5480d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e77521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e77521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33074/0 (socket says 172.21.15.33:33074) 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 -- 172.21.15.33:0/2133069855 learned_addr learned my addr 172.21.15.33:0/2133069855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55ddd5640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7753720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7752710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 -- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 msgr2=0x5615e7752710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7752710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 -- 172.21.15.33:0/2133069855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 msgr2=0x5615e7753720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7753720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 -- 172.21.15.33:0/2133069855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e7753d10 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff55d5d4640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e77521d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff554004730 tx=0x7ff554031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231ead88d9abaa47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff540ff9640 1 -- 172.21.15.33:0/2133069855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554040050 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff540ff9640 1 -- 172.21.15.33:0/2133069855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff554004450 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5480b8020 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff540ff9640 1 -- 172.21.15.33:0/2133069855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff55405c050 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.760+0000 7ff562629740 10 client.17388 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615e77bfb00 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff540ff9640 1 -- 172.21.15.33:0/2133069855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5480b8020 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff540ff9640 1 -- 172.21.15.33:0/2133069855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff554004ca0 con 0x5615e7743640 2024-04-24T23:21:19.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 10 client.17388 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 10 client.17388 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 20 client.17388 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77bff00 0x5615e77c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 4 client.17388 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615e76aaf00 con 0x5615e77bff00 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.761+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77bff00 0x5615e77c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff55cdd3640 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77bff00 0x5615e77c2320 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=275d9c2147f4b037 server_cookie=63901a8c053efed7 in_seq=0 out_seq=0 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff540ff9640 10 client.17388 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff540ff9640 1 -- 172.21.15.33:0/2133069855 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615e76aaf00 con 0x5615e77bff00 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff540ff9640 10 client.17388 handle_command_reply: tid=0 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 client.17388 shutdown 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 msgr2=0x5615e77521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e77521d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff554004730 tx=0x7ff554031b20 comp rx=0 tx=0).stop 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 shutdown_connections 2024-04-24T23:21:19.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5615e77bff00 0x5615e77c2320 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615e77491a0 0x5615e7753720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff540ff9640 0 client.17388 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615e7744030 0x5615e7752710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 --2- 172.21.15.33:0/2133069855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615e7743640 0x5615e77521d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 >> 172.21.15.33:0/2133069855 conn(0x5615e773a8a0 msgr2=0x5615e77449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 shutdown_connections 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 1 -- 172.21.15.33:0/2133069855 wait complete. 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.762+0000 7ff562629740 20 client.17388 trim_cache size 0 max 16384 2024-04-24T23:21:19.770 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:21:19.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5440a2b90 msgr2=0x7ff5440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5440a2b90 0x7ff5440a5050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5615e7744890 tx=0x7ff550004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:19.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558104c00 msgr2=0x7ff558198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558104c00 0x7ff558198060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e9c0 tx=0x7ff54800ee80 comp rx=0 tx=0).stop 2024-04-24T23:21:19.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 shutdown_connections 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5440a2b90 0x7ff5440a5050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff558107530 0x7ff55819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff558104c00 0x7ff558198060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 --2- 172.21.15.33:0/3150177675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5581022d0 0x7ff558197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 >> 172.21.15.33:0/3150177675 conn(0x7ff5580fbc00 msgr2=0x7ff558102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 shutdown_connections 2024-04-24T23:21:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.763+0000 7ff55f838640 1 -- 172.21.15.33:0/3150177675 wait complete. 2024-04-24T23:21:19.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 -- 172.21.15.33:0/2793417992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 msgr2=0x7f697c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 --2- 172.21.15.33:0/2793417992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c101cf0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6970009900 tx=0x7f697002a410 comp rx=0 tx=0).stop 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 -- 172.21.15.33:0/2793417992 shutdown_connections 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 --2- 172.21.15.33:0/2793417992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f697c102230 0x7f697c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 --2- 172.21.15.33:0/2793417992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c101cf0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 --2- 172.21.15.33:0/2793417992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697c100680 0x7f697c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 -- 172.21.15.33:0/2793417992 >> 172.21.15.33:0/2793417992 conn(0x7f697c0fbbf0 msgr2=0x7f697c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 -- 172.21.15.33:0/2793417992 shutdown_connections 2024-04-24T23:21:19.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 -- 172.21.15.33:0/2793417992 wait complete. 2024-04-24T23:21:19.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 Processor -- start 2024-04-24T23:21:19.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.882+0000 7f69829e7640 1 -- start start 2024-04-24T23:21:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697c100680 0x7f697c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f697c102230 0x7f697c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10c300 con 0x7f697c101870 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10c180 con 0x7f697c102230 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10c480 con 0x7f697c100680 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33096/0 (socket says 172.21.15.33:33096) 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 -- 172.21.15.33:0/571641343 learned_addr learned my addr 172.21.15.33:0/571641343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697bfff640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697c100680 0x7f697c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f6980f84640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f697c102230 0x7f697c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 -- 172.21.15.33:0/571641343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697c100680 msgr2=0x7f697c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697c100680 0x7f697c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 -- 172.21.15.33:0/571641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f697c102230 msgr2=0x7f697c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f697c102230 0x7f697c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 -- 172.21.15.33:0/571641343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c19f6d0 con 0x7f697c101870 2024-04-24T23:21:19.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f697b7fe640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c197ff0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6970009230 tx=0x7f69700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69797fa640 1 -- 172.21.15.33:0/571641343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697003f030 con 0x7f697c101870 2024-04-24T23:21:19.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c19f960 con 0x7f697c101870 2024-04-24T23:21:19.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69797fa640 1 -- 172.21.15.33:0/571641343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970003ce0 con 0x7f697c101870 2024-04-24T23:21:19.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.883+0000 7f69797fa640 1 -- 172.21.15.33:0/571641343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970035070 con 0x7f697c101870 2024-04-24T23:21:19.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.884+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c19fe50 con 0x7f697c101870 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69797fa640 1 -- 172.21.15.33:0/571641343 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6970042030 con 0x7f697c101870 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 Processor -- start 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 -- start start 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e55720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e58220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d154db4670 con 0x55d154e50230 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d154d5c970 con 0x55d154e50c20 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f69857d2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d154db0630 con 0x55d154e55d90 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e50650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33100/0 (socket says 172.21.15.33:33100) 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 -- 172.21.15.33:0/4213156389 learned_addr learned my addr 172.21.15.33:0/4213156389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f697bfff640 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e55720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f697b7fe640 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e58220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 -- 172.21.15.33:0/4213156389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 msgr2=0x55d154e58220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e58220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 -- 172.21.15.33:0/4213156389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 msgr2=0x55d154e55720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.886+0000 7f6980f84640 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e55720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f6980f84640 1 -- 172.21.15.33:0/4213156389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d154d80400 con 0x55d154e50230 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f697b7fe640 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e58220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f69797fa640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69640a2b90 0x7f69640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f697bfff640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69640a2b90 0x7f69640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f69797fa640 1 -- 172.21.15.33:0/571641343 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f69700ef840 con 0x7f697c101870 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f6980f84640 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e50650 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6974009fd0 tx=0x7f69740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311342878a9d474e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f69627fc640 1 -- 172.21.15.33:0/4213156389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974004d20 con 0x55d154e50230 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f69627fc640 1 -- 172.21.15.33:0/4213156389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6974004050 con 0x55d154e50230 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.887+0000 7f69627fc640 1 -- 172.21.15.33:0/4213156389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974005240 con 0x55d154e50230 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 -- 172.21.15.33:0/4213156389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 msgr2=0x55d154e50650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f697bfff640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69640a2b90 0x7f69640a5050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55d154e51020 tx=0x7f696c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e50650 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6974009fd0 tx=0x7f69740118d0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 -- 172.21.15.33:0/4213156389 shutdown_connections 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e58220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e55720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 --2- 172.21.15.33:0/4213156389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e50650 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 -- 172.21.15.33:0/4213156389 >> 172.21.15.33:0/4213156389 conn(0x55d154e4b8a0 msgr2=0x55d154e4bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 -- 172.21.15.33:0/4213156389 shutdown_connections 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 -- 172.21.15.33:0/4213156389 wait complete. 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 -- start start 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 Processor -- start 2024-04-24T23:21:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.888+0000 7f69857d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e5a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f69857d2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e61910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f69857d2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d154e58d40 con 0x55d154e50230 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f69857d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d154d70050 con 0x55d154e50c20 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f69857d2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d154e58ec0 con 0x55d154e55d90 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f69857d2740 10 client.0 _ll_register_callbacks cb 0x55d154ac7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33116/0 (socket says 172.21.15.33:33116) 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 -- 172.21.15.33:0/3128345094 learned_addr learned my addr 172.21.15.33:0/3128345094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697b7fe640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e5a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f6980f84640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e61910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 -- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 msgr2=0x55d154e61910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e61910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 -- 172.21.15.33:0/3128345094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 msgr2=0x55d154e5a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e5a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 -- 172.21.15.33:0/3128345094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d154d41dc0 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f6980f84640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e61910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697b7fe640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f697bfff640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a390 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f696c004620 tx=0x7f696c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0c8849795f5eff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f695b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f695b7fe640 1 -- 172.21.15.33:0/3128345094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c009e60 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f695b7fe640 1 -- 172.21.15.33:0/3128345094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f696c01f630 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6974004050 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.889+0000 7f695b7fe640 1 -- 172.21.15.33:0/3128345094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f696c01f830 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 10 client.17400 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d154ac83f0 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f695b7fe640 1 -- 172.21.15.33:0/3128345094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6974004050 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f695b7fe640 1 -- 172.21.15.33:0/3128345094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f696c027070 con 0x55d154e50230 2024-04-24T23:21:19.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 10 client.17400 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 20 client.17400 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154e662d0 0x55d154e68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 4 client.17400 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d154dc1bb0 con 0x55d154e662d0 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.890+0000 7f697b7fe640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154e662d0 0x55d154e68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f697b7fe640 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154e662d0 0x55d154e68790 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef42b3a0cb371719 server_cookie=9e8a65b5260fb522 in_seq=0 out_seq=0 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f695b7fe640 10 client.17400 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f695b7fe640 1 -- 172.21.15.33:0/3128345094 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d154dc1bb0 con 0x55d154e662d0 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f695b7fe640 10 client.17400 handle_command_reply: tid=0 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 client.17400 shutdown 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 msgr2=0x55d154e5a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a390 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f696c004620 tx=0x7f696c004730 comp rx=0 tx=0).stop 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 shutdown_connections 2024-04-24T23:21:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154e662d0 0x55d154e68790 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e55d90 0x55d154e61910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f695b7fe640 0 client.17400 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e50c20 0x55d154e5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 --2- 172.21.15.33:0/3128345094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a390 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.891+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 >> 172.21.15.33:0/3128345094 conn(0x55d154e4b8a0 msgr2=0x55d154e4ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.892+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 shutdown_connections 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.892+0000 7f69857d2740 1 -- 172.21.15.33:0/3128345094 wait complete. 2024-04-24T23:21:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.892+0000 7f69857d2740 20 client.17400 trim_cache size 0 max 16384 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 Processor -- start 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 -- start start 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50c20 0x55d154e618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e4a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1acc90 con 0x55d154e50230 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1aa560 con 0x55d154e55d90 2024-04-24T23:21:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f69857d2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1a83b0 con 0x55d154e50c20 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33122/0 (socket says 172.21.15.33:33122) 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 -- 172.21.15.33:0/858744894 learned_addr learned my addr 172.21.15.33:0/858744894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f6980f84640 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e4a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697b7fe640 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50c20 0x55d154e618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 -- 172.21.15.33:0/858744894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50c20 msgr2=0x55d154e618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50c20 0x55d154e618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 -- 172.21.15.33:0/858744894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 msgr2=0x55d154e4a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e4a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.907+0000 7f697bfff640 1 -- 172.21.15.33:0/858744894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d154b927d0 con 0x55d154e50230 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f697bfff640 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a830 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f696c019040 tx=0x7f696c012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c467538da95361 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f6961ffb640 1 -- 172.21.15.33:0/858744894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c01f9d0 con 0x55d154e50230 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f6961ffb640 1 -- 172.21.15.33:0/858744894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f696c029920 con 0x55d154e50230 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f6961ffb640 1 -- 172.21.15.33:0/858744894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c037850 con 0x55d154e50230 2024-04-24T23:21:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f69857d2740 1 -- 172.21.15.33:0/858744894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 msgr2=0x55d154e5a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f69857d2740 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a830 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f696c019040 tx=0x7f696c012d00 comp rx=0 tx=0).stop 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f69857d2740 1 -- 172.21.15.33:0/858744894 shutdown_connections 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.908+0000 7f69857d2740 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e4a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50c20 0x55d154e618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 --2- 172.21.15.33:0/858744894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50230 0x55d154e5a830 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- 172.21.15.33:0/858744894 >> 172.21.15.33:0/858744894 conn(0x55d154e4b8a0 msgr2=0x55d154e4a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- 172.21.15.33:0/858744894 shutdown_connections 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- 172.21.15.33:0/858744894 wait complete. 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- start start 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 Processor -- start 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50230 0x55d154e5eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 0x55d154e5f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e60400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1a8a20 con 0x55d154e50c20 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1a9e90 con 0x55d154e55d90 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1a90f0 con 0x55d154e50230 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f69857d2740 10 client.0 _ll_register_callbacks cb 0x7f696c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 0x55d154e5f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f697bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50230 0x55d154e5eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 0x55d154e5f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33124/0 (socket says 172.21.15.33:33124) 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.909+0000 7f697b7fe640 1 -- 172.21.15.33:0/1868934278 learned_addr learned my addr 172.21.15.33:0/1868934278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f6980f84640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e60400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697bfff640 1 -- 172.21.15.33:0/1868934278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 msgr2=0x55d154e60400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697bfff640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e60400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697bfff640 1 -- 172.21.15.33:0/1868934278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 msgr2=0x55d154e5f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697bfff640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 0x55d154e5f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697bfff640 1 -- 172.21.15.33:0/1868934278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d154e609f0 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697b7fe640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 0x55d154e5f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f697bfff640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50230 0x55d154e5eeb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f696c004020 tx=0x7f696c01fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32fa2eb4c26b1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f695b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f695b7fe640 1 -- 172.21.15.33:0/1868934278 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c023070 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f695b7fe640 1 -- 172.21.15.33:0/1868934278 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f696c005250 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696c029920 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f695b7fe640 1 -- 172.21.15.33:0/1868934278 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f696c04d050 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f69857d2740 10 client.26914 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.910+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d154ecc8d0 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f695b7fe640 1 -- 172.21.15.33:0/1868934278 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f696c029920 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f695b7fe640 1 -- 172.21.15.33:0/1868934278 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f696c027070 con 0x55d154e50230 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 10 client.26914 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 20 client.26914 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154ecd160 0x55d154ecf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 4 client.26914 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d154dbbf00 con 0x55d154ecd160 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f697b7fe640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154ecd160 0x55d154ecf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f695b7fe640 10 client.26914 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f697b7fe640 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154ecd160 0x55d154ecf580 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360377bfe3489834 server_cookie=2eb2d20f812f5a2e in_seq=0 out_seq=0 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f695b7fe640 1 -- 172.21.15.33:0/1868934278 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d154dbbf00 con 0x55d154ecd160 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f695b7fe640 10 client.26914 handle_command_reply: tid=0 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.911+0000 7f69857d2740 1 client.26914 shutdown 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50230 msgr2=0x55d154e5eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50230 0x55d154e5eeb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f696c004020 tx=0x7f696c01fc00 comp rx=0 tx=0).stop 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 shutdown_connections 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d154ecd160 0x55d154ecf580 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d154e55d90 0x55d154e60400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d154e50c20 0x55d154e5f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 --2- 172.21.15.33:0/1868934278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d154e50230 0x55d154e5eeb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f695b7fe640 0 client.26914 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 >> 172.21.15.33:0/1868934278 conn(0x55d154e4b8a0 msgr2=0x55d154e4a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 shutdown_connections 2024-04-24T23:21:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 1 -- 172.21.15.33:0/1868934278 wait complete. 2024-04-24T23:21:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69857d2740 20 client.26914 trim_cache size 0 max 16384 2024-04-24T23:21:19.920 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69640a2b90 msgr2=0x7f69640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69640a2b90 0x7f69640a5050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55d154e51020 tx=0x7f696c005f90 comp rx=0 tx=0).stop 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 msgr2=0x7f697c197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c197ff0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6970009230 tx=0x7f69700042f0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 shutdown_connections 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69640a2b90 0x7f69640a5050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f697c102230 0x7f697c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101870 0x7f697c197ff0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 --2- 172.21.15.33:0/571641343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697c100680 0x7f697c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.912+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 >> 172.21.15.33:0/571641343 conn(0x7f697c0fbbf0 msgr2=0x7f697c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.913+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 shutdown_connections 2024-04-24T23:21:19.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:19.913+0000 7f69829e7640 1 -- 172.21.15.33:0/571641343 wait complete. 2024-04-24T23:21:20.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:19 smithi038 ceph-mon[32450]: pgmap v630: 129 pgs: 129 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-04-24T23:21:20.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:19 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/171996638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:21:20.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:19 smithi117 ceph-mon[32529]: pgmap v630: 129 pgs: 129 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-04-24T23:21:20.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:19 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/171996638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:21:20.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:19 smithi033 ceph-mon[29201]: pgmap v630: 129 pgs: 129 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-04-24T23:21:20.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:19 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/171996638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:21:20.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1055863 12.67 MB/sec execute 325 sec latency 18.626 ms 2024-04-24T23:21:20.926 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:21:20.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:20.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:21:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.081+0000 7f6d3c321640 1 -- 172.21.15.33:0/4208926983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d340741a0 msgr2=0x7f6d34074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.081+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4208926983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d340741a0 0x7f6d34074620 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009cd0 tx=0x7f6d2c02a930 comp rx=0 tx=0).stop 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.083+0000 7f6d3c321640 1 -- 172.21.15.33:0/4208926983 shutdown_connections 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.083+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4208926983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d34110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.083+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4208926983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d340741a0 0x7f6d34074620 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.083+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4208926983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d34075780 0x7f6d34073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.083+0000 7f6d3c321640 1 -- 172.21.15.33:0/4208926983 >> 172.21.15.33:0/4208926983 conn(0x7f6d340fcd20 msgr2=0x7f6d340ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.086+0000 7f6d3c321640 1 -- 172.21.15.33:0/4208926983 shutdown_connections 2024-04-24T23:21:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.086+0000 7f6d3c321640 1 -- 172.21.15.33:0/4208926983 wait complete. 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 Processor -- start 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 -- start start 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d340741a0 0x7f6d3419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d34075780 0x7f6d3419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d341a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34113b70 con 0x7f6d34075780 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341139f0 con 0x7f6d340741a0 2024-04-24T23:21:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.087+0000 7f6d3c321640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34113cf0 con 0x7f6d3410dd60 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d341a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d398bc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d34075780 0x7f6d3419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d341a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:48996/0 (socket says 172.21.15.33:48996) 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d398bc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d34075780 0x7f6d3419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33140/0 (socket says 172.21.15.33:33140) 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 -- 172.21.15.33:0/4166780189 learned_addr learned my addr 172.21.15.33:0/4166780189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a0bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d340741a0 0x7f6d3419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 -- 172.21.15.33:0/4166780189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d340741a0 msgr2=0x7f6d3419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d340741a0 0x7f6d3419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 -- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d34075780 msgr2=0x7f6d3419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d34075780 0x7f6d3419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.088+0000 7f6d3a8be640 1 -- 172.21.15.33:0/4166780189 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d341a5ec0 con 0x7f6d3410dd60 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.089+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d341a5850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b3f0 tx=0x7f6d2400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.089+0000 7f6d2b7fe640 1 -- 172.21.15.33:0/4166780189 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24015020 con 0x7f6d3410dd60 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.089+0000 7f6d2b7fe640 1 -- 172.21.15.33:0/4166780189 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d24011d10 con 0x7f6d3410dd60 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.089+0000 7f6d2b7fe640 1 -- 172.21.15.33:0/4166780189 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400f2e0 con 0x7f6d3410dd60 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.089+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d341a61b0 con 0x7f6d3410dd60 2024-04-24T23:21:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.089+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d341a6700 con 0x7f6d3410dd60 2024-04-24T23:21:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.093+0000 7f6d2b7fe640 1 -- 172.21.15.33:0/4166780189 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6d24018020 con 0x7f6d3410dd60 2024-04-24T23:21:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.094+0000 7f6d3f02a740 1 Processor -- start 2024-04-24T23:21:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.094+0000 7f6d2b7fe640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d200a2c60 0x7f6d200a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.095+0000 7f6d2b7fe640 1 -- 172.21.15.33:0/4166780189 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d240ceb10 con 0x7f6d3410dd60 2024-04-24T23:21:21.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.095+0000 7f6d398bc640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d200a2c60 0x7f6d200a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.095+0000 7f6d3f02a740 1 -- start start 2024-04-24T23:21:21.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557afd8232c0 con 0x557afd8bfd90 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51818/0 (socket says 172.21.15.33:51818) 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3a8be640 1 -- 172.21.15.33:0/3823505516 learned_addr learned my addr 172.21.15.33:0/3823505516 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557afd7cb910 con 0x557afd8c58f0 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.096+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557afd81f280 con 0x557afd8c0780 2024-04-24T23:21:21.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.097+0000 7f6d398bc640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.097+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.097+0000 7f6d398bc640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d200a2c60 0x7f6d200a5120 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006010 tx=0x7f6d2c000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a8be640 1 -- 172.21.15.33:0/3823505516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 msgr2=0x557afd8c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a8be640 1 -- 172.21.15.33:0/3823505516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 msgr2=0x557afd8c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a8be640 1 -- 172.21.15.33:0/3823505516 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afd7dcfb0 con 0x557afd8c58f0 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d398bc640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8c7d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d24008050 tx=0x7f6d24097540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c03560dce00686f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d28bfe640 1 -- 172.21.15.33:0/3823505516 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d240a3c50 con 0x557afd8c58f0 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.098+0000 7f6d28bfe640 1 -- 172.21.15.33:0/3823505516 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d240977b0 con 0x557afd8c58f0 2024-04-24T23:21:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d28bfe640 1 -- 172.21.15.33:0/3823505516 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d240a0520 con 0x557afd8c58f0 2024-04-24T23:21:21.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 msgr2=0x557afd8c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8c7d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d24008050 tx=0x7f6d24097540 comp rx=0 tx=0).stop 2024-04-24T23:21:21.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 shutdown_connections 2024-04-24T23:21:21.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8c7d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3823505516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 >> 172.21.15.33:0/3823505516 conn(0x557afd887950 msgr2=0x557afd8bb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 shutdown_connections 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.099+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3823505516 wait complete. 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.100+0000 7f6d3f02a740 1 -- start start 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.100+0000 7f6d3f02a740 1 Processor -- start 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.100+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c0780 0x557afd8ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c58f0 0x557afd8d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557afd8c88a0 con 0x557afd8bfd90 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557afd7dee90 con 0x557afd8c0780 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557afd8c8a20 con 0x557afd8c58f0 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3f02a740 10 client.0 _ll_register_callbacks cb 0x557afd81a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3a0bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3a0bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33154/0 (socket says 172.21.15.33:33154) 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3a0bd640 1 -- 172.21.15.33:0/2715284577 learned_addr learned my addr 172.21.15.33:0/2715284577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c0780 0x557afd8ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c58f0 0x557afd8d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 -- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c58f0 msgr2=0x557afd8d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c58f0 0x557afd8d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 -- 172.21.15.33:0/2715284577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 msgr2=0x557afd8ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 -- 172.21.15.33:0/2715284577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afd5ff7e0 con 0x557afd8c0780 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d398bc640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c0780 0x557afd8ca720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009bf0 tx=0x7f6d2c03e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa91b98300117c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.101+0000 7f6d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d1dffb640 1 -- 172.21.15.33:0/2715284577 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c056060 con 0x557afd8c0780 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d1dffb640 1 -- 172.21.15.33:0/2715284577 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d2c004240 con 0x557afd8c0780 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d240977b0 con 0x557afd8c0780 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d1dffb640 1 -- 172.21.15.33:0/2715284577 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d2c004440 con 0x557afd8c0780 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d3f02a740 10 client.26942 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557afd65ff70 con 0x557afd8c0780 2024-04-24T23:21:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d1dffb640 1 -- 172.21.15.33:0/2715284577 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d240977b0 con 0x557afd8c0780 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.102+0000 7f6d1dffb640 1 -- 172.21.15.33:0/2715284577 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6d2c043040 con 0x557afd8c0780 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3f02a740 10 client.26942 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3f02a740 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3f02a740 20 client.26942 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd8d6040 0x557afd8d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3f02a740 4 client.26942 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557afd7f75a0 con 0x557afd8d6040 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd8d6040 0x557afd8d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d1dffb640 10 client.26942 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.103+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd8d6040 0x557afd8d84e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5397c64154542fc7 server_cookie=b7cbc63b675a896b in_seq=0 out_seq=0 2024-04-24T23:21:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d1dffb640 1 -- 172.21.15.33:0/2715284577 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557afd7f75a0 con 0x557afd8d6040 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d1dffb640 10 client.26942 handle_command_reply: tid=0 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 client.26942 shutdown 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c0780 msgr2=0x557afd8ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c0780 0x557afd8ca720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009bf0 tx=0x7f6d2c03e040 comp rx=0 tx=0).stop 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 shutdown_connections 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd8d6040 0x557afd8d84e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c58f0 0x557afd8d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c0780 0x557afd8ca720 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/2715284577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 >> 172.21.15.33:0/2715284577 conn(0x557afd887950 msgr2=0x557afd8ba950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d1dffb640 0 client.26942 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 shutdown_connections 2024-04-24T23:21:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 1 -- 172.21.15.33:0/2715284577 wait complete. 2024-04-24T23:21:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.104+0000 7f6d3f02a740 20 client.26942 trim_cache size 0 max 16384 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 Processor -- start 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 -- start start 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8ba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b34c0 con 0x557afd8bfd90 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b0d90 con 0x557afd8c58f0 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341aebe0 con 0x557afd8c0780 2024-04-24T23:21:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.123+0000 7f6d3a0bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8ba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33170/0 (socket says 172.21.15.33:33170) 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 -- 172.21.15.33:0/3043884462 learned_addr learned my addr 172.21.15.33:0/3043884462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d398bc640 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 -- 172.21.15.33:0/3043884462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 msgr2=0x557afd8d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 -- 172.21.15.33:0/3043884462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 msgr2=0x557afd8ba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8ba600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 -- 172.21.15.33:0/3043884462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afd4f7170 con 0x557afd8bfd90 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8ba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ba0c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6d30006fd0 tx=0x7f6d300090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b023b7913495d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d1ffff640 1 -- 172.21.15.33:0/3043884462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3001fdd0 con 0x557afd8bfd90 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d1ffff640 1 -- 172.21.15.33:0/3043884462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30025070 con 0x557afd8bfd90 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.124+0000 7f6d1ffff640 1 -- 172.21.15.33:0/3043884462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300147e0 con 0x557afd8bfd90 2024-04-24T23:21:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3043884462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 msgr2=0x557afd8ba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ba0c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6d30006fd0 tx=0x7f6d300090e0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3043884462 shutdown_connections 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8c58f0 0x557afd8ba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/3043884462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8bfd90 0x557afd8ba0c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3043884462 >> 172.21.15.33:0/3043884462 conn(0x557afd887950 msgr2=0x557afd8c1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3043884462 shutdown_connections 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.125+0000 7f6d3f02a740 1 -- 172.21.15.33:0/3043884462 wait complete. 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 -- start start 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 Processor -- start 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8bfd90 0x557afd8cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 0x557afd8d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341af250 con 0x557afd8c58f0 2024-04-24T23:21:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b06c0 con 0x557afd8bfd90 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341af920 con 0x557afd8c0780 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3f02a740 10 client.0 _ll_register_callbacks cb 0x7f6d2c030830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 0x557afd8d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 0x557afd8d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33172/0 (socket says 172.21.15.33:33172) 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 -- 172.21.15.33:0/1060630603 learned_addr learned my addr 172.21.15.33:0/1060630603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8bfd90 0x557afd8cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d398bc640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 -- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 msgr2=0x557afd8cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 -- 172.21.15.33:0/1060630603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8bfd90 msgr2=0x557afd8cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8bfd90 0x557afd8cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a8be640 1 -- 172.21.15.33:0/1060630603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afd8d07f0 con 0x557afd8c58f0 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d398bc640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.126+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8bfd90 0x557afd8cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3a8be640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 0x557afd8d0200 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b3c0 tx=0x7f6d2409f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8f9a40137413ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d1dffb640 1 -- 172.21.15.33:0/1060630603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d240a3c50 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d1dffb640 1 -- 172.21.15.33:0/1060630603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d240977b0 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d30025070 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d1dffb640 1 -- 172.21.15.33:0/1060630603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d240b1c70 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 10 client.17424 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557afd93c660 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d1dffb640 1 -- 172.21.15.33:0/1060630603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30025070 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d1dffb640 1 -- 172.21.15.33:0/1060630603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6d240a0520 con 0x557afd8c58f0 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 10 client.17424 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 20 client.17424 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd93c9f0 0x557afd93ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 4 client.17424 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:21:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.127+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557afd830800 con 0x557afd93c9f0 2024-04-24T23:21:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.128+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd93c9f0 0x557afd93ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.128+0000 7f6d3a0bd640 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd93c9f0 0x557afd93ede0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81820b437b5d1c66 server_cookie=7245358ff26be915 in_seq=0 out_seq=0 2024-04-24T23:21:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.128+0000 7f6d1dffb640 10 client.17424 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d1dffb640 1 -- 172.21.15.33:0/1060630603 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557afd830800 con 0x557afd93c9f0 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d1dffb640 10 client.17424 handle_command_reply: tid=0 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 client.17424 shutdown 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 msgr2=0x557afd8d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 0x557afd8d0200 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b3c0 tx=0x7f6d2409f450 comp rx=0 tx=0).stop 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 shutdown_connections 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557afd93c9f0 0x557afd93ede0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557afd8c58f0 0x557afd8d0200 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557afd8c0780 0x557afd8cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d1dffb640 0 client.17424 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 --2- 172.21.15.33:0/1060630603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557afd8bfd90 0x557afd8cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 >> 172.21.15.33:0/1060630603 conn(0x557afd887950 msgr2=0x557afd8be7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 shutdown_connections 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 1 -- 172.21.15.33:0/1060630603 wait complete. 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.129+0000 7f6d3f02a740 20 client.17424 trim_cache size 0 max 16384 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "b51b3e23-6287-4fe5-b964-c1113f7bc8f0", 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:21:21.137 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:21:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d200a2c60 msgr2=0x7f6d200a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d200a2c60 0x7f6d200a5120 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006010 tx=0x7f6d2c000f40 comp rx=0 tx=0).stop 2024-04-24T23:21:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 msgr2=0x7f6d341a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d341a5850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b3f0 tx=0x7f6d2400b8b0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 shutdown_connections 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d200a2c60 0x7f6d200a5120 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3410dd60 0x7f6d341a5850 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d34075780 0x7f6d3419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 --2- 172.21.15.33:0/4166780189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d340741a0 0x7f6d3419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 >> 172.21.15.33:0/4166780189 conn(0x7f6d340fcd20 msgr2=0x7f6d34101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 shutdown_connections 2024-04-24T23:21:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.130+0000 7f6d3c321640 1 -- 172.21.15.33:0/4166780189 wait complete. 2024-04-24T23:21:21.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:21:21.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:21:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/3210254732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec101890 msgr2=0x7f0eec101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/3210254732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec101890 0x7f0eec101d10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02a410 comp rx=0 tx=0).stop 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/3210254732 shutdown_connections 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/3210254732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec102250 0x7f0eec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/3210254732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec101890 0x7f0eec101d10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/3210254732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eec1006a0 0x7f0eec100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.275+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/3210254732 >> 172.21.15.33:0/3210254732 conn(0x7f0eec0fbc10 msgr2=0x7f0eec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.276+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/3210254732 shutdown_connections 2024-04-24T23:21:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.276+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/3210254732 wait complete. 2024-04-24T23:21:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 Processor -- start 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 -- start start 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 0x7f0eec196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec101890 0x7f0eec196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eec102250 0x7f0eec19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10c3e0 con 0x7f0eec1006a0 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10c260 con 0x7f0eec101890 2024-04-24T23:21:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.277+0000 7f0ef0fb6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10c560 con 0x7f0eec102250 2024-04-24T23:21:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 0x7f0eec196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0ee9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec101890 0x7f0eec196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0ee9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec101890 0x7f0eec196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51888/0 (socket says 172.21.15.33:51888) 2024-04-24T23:21:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 0x7f0eec196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33198/0 (socket says 172.21.15.33:33198) 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0ee9d9b640 1 -- 172.21.15.33:0/745674734 learned_addr learned my addr 172.21.15.33:0/745674734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eead9d640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eec102250 0x7f0eec19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 -- 172.21.15.33:0/745674734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eec102250 msgr2=0x7f0eec19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eec102250 0x7f0eec19e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 -- 172.21.15.33:0/745674734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec101890 msgr2=0x7f0eec196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec101890 0x7f0eec196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.278+0000 7f0eea59c640 1 -- 172.21.15.33:0/745674734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec19e6a0 con 0x7f0eec1006a0 2024-04-24T23:21:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.279+0000 7f0eea59c640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 0x7f0eec196a80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b3c0 tx=0x7f0ee000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.280+0000 7f0edb7fe640 1 -- 172.21.15.33:0/745674734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0011830 con 0x7f0eec1006a0 2024-04-24T23:21:21.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.280+0000 7f0edb7fe640 1 -- 172.21.15.33:0/745674734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee0016070 con 0x7f0eec1006a0 2024-04-24T23:21:21.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.280+0000 7f0edb7fe640 1 -- 172.21.15.33:0/745674734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000f2e0 con 0x7f0eec1006a0 2024-04-24T23:21:21.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.280+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eec069da0 con 0x7f0eec1006a0 2024-04-24T23:21:21.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.280+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eec06a320 con 0x7f0eec1006a0 2024-04-24T23:21:21.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.284+0000 7f0ef3d9b740 1 Processor -- start 2024-04-24T23:21:21.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.285+0000 7f0edb7fe640 1 -- 172.21.15.33:0/745674734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0ee000f480 con 0x7f0eec1006a0 2024-04-24T23:21:21.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.285+0000 7f0ef3d9b740 1 -- start start 2024-04-24T23:21:21.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78b240 0x55c48b78b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b790730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b793230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c48b6ef6b0 con 0x55c48b790da0 2024-04-24T23:21:21.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c48b697880 con 0x55c48b78b240 2024-04-24T23:21:21.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c48b6eb670 con 0x55c48b78bc30 2024-04-24T23:21:21.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0edb7fe640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ed40a2ce0 0x7f0ed40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0eead9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78b240 0x55c48b78b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ee9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b793230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ee9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b793230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33212/0 (socket says 172.21.15.33:33212) 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.286+0000 7f0ee9d9b640 1 -- 172.21.15.33:0/2924045271 learned_addr learned my addr 172.21.15.33:0/2924045271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0edb7fe640 1 -- 172.21.15.33:0/745674734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ee00ce970 con 0x7f0eec1006a0 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0ee9d9b640 1 -- 172.21.15.33:0/2924045271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 msgr2=0x55c48b790730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0eea59c640 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b790730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0eead9d640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ed40a2ce0 0x7f0ed40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b790730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0ee9d9b640 1 -- 172.21.15.33:0/2924045271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78b240 msgr2=0x55c48b78b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78b240 0x55c48b78b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0ee9d9b640 1 -- 172.21.15.33:0/2924045271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48b4cd7d0 con 0x55c48b790da0 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0eead9d640 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78b240 0x55c48b78b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.287+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b793230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0edc02aa00 tx=0x7f0edc004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cad96685e2a115 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.288+0000 7f0ed8ff9640 1 -- 172.21.15.33:0/2924045271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc040070 con 0x55c48b790da0 2024-04-24T23:21:21.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.288+0000 7f0eead9d640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ed40a2ce0 0x7f0ed40a51a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55c48b78c490 tx=0x7f0ee4004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.288+0000 7f0ed8ff9640 1 -- 172.21.15.33:0/2924045271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0edc005500 con 0x55c48b790da0 2024-04-24T23:21:21.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.288+0000 7f0ed8ff9640 1 -- 172.21.15.33:0/2924045271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc035040 con 0x55c48b790da0 2024-04-24T23:21:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/2924045271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 msgr2=0x55c48b793230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b793230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0edc02aa00 tx=0x7f0edc004170 comp rx=0 tx=0).stop 2024-04-24T23:21:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/2924045271 shutdown_connections 2024-04-24T23:21:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b793230 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b790730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/2924045271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78b240 0x55c48b78b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/2924045271 >> 172.21.15.33:0/2924045271 conn(0x55c48b7868b0 msgr2=0x55c48b786cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.289+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/2924045271 shutdown_connections 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/2924045271 wait complete. 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 -- start start 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 Processor -- start 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78b240 0x55c48b7a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78bc30 0x55c48b7a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b7adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c48b7a4d00 con 0x55c48b790da0 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c48b6aaf50 con 0x55c48b78bc30 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c48b7a4e80 con 0x55c48b78b240 2024-04-24T23:21:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.290+0000 7f0ef3d9b740 10 client.0 _ll_register_callbacks cb 0x55c48b6e6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:21.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b7adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b7adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33228/0 (socket says 172.21.15.33:33228) 2024-04-24T23:21:21.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0ee9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78bc30 0x55c48b7a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 -- 172.21.15.33:0/1659443119 learned_addr learned my addr 172.21.15.33:0/1659443119 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eea59c640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78b240 0x55c48b7a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 -- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78b240 msgr2=0x55c48b7a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78b240 0x55c48b7a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 -- 172.21.15.33:0/1659443119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78bc30 msgr2=0x55c48b7a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78bc30 0x55c48b7a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 -- 172.21.15.33:0/1659443119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48b4033f0 con 0x55c48b790da0 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78bc30 0x55c48b7a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0eead9d640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b7adc50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009f80 tx=0x7f0ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89e1b03c58aac50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.291+0000 7f0ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/1659443119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4025070 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/1659443119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ee4021480 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0edc005500 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/1659443119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ee4004b20 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ef3d9b740 10 client.17436 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c48b3c30f0 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/1659443119 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0edc005500 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/1659443119 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ee4032020 con 0x55c48b790da0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ef3d9b740 10 client.17436 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ef3d9b740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.292+0000 7f0ef3d9b740 20 client.17436 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.293+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b7b2680 0x55c48b7b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.293+0000 7f0ef3d9b740 4 client.17436 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.293+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c48b6fcbf0 con 0x55c48b7b2680 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.293+0000 7f0eea59c640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b7b2680 0x55c48b7b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.293+0000 7f0ed1ffb640 10 client.17436 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.293+0000 7f0eea59c640 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b7b2680 0x55c48b7b4b00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9ebea41f731434 server_cookie=2eba389ccccd679d in_seq=0 out_seq=0 2024-04-24T23:21:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.294+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/1659443119 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c48b6fcbf0 con 0x55c48b7b2680 2024-04-24T23:21:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.294+0000 7f0ed1ffb640 10 client.17436 handle_command_reply: tid=0 2024-04-24T23:21:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.294+0000 7f0ef3d9b740 1 client.17436 shutdown 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.294+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 msgr2=0x55c48b7adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b7adc50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009f80 tx=0x7f0ee4004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 shutdown_connections 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b7b2680 0x55c48b7b4b00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b790da0 0x55c48b7adc50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b78bc30 0x55c48b7a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1659443119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78b240 0x55c48b7a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 >> 172.21.15.33:0/1659443119 conn(0x55c48b7868b0 msgr2=0x55c48b785af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ed1ffb640 0 client.17436 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 shutdown_connections 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1659443119 wait complete. 2024-04-24T23:21:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.295+0000 7f0ef3d9b740 20 client.17436 trim_cache size 0 max 16384 2024-04-24T23:21:21.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.319+0000 7f0ef3d9b740 1 Processor -- start 2024-04-24T23:21:21.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.319+0000 7f0ef3d9b740 1 -- start start 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b789860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1aab00 con 0x55c48b78b240 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1a83d0 con 0x55c48b790da0 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1a6220 con 0x55c48b78bc30 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33236/0 (socket says 172.21.15.33:33236) 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0eea59c640 1 -- 172.21.15.33:0/1770796144 learned_addr learned my addr 172.21.15.33:0/1770796144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0eead9d640 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b789860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.320+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0eea59c640 1 -- 172.21.15.33:0/1770796144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 msgr2=0x55c48b7adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0eea59c640 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0eea59c640 1 -- 172.21.15.33:0/1770796144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 msgr2=0x55c48b789860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0eea59c640 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b789860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0eea59c640 1 -- 172.21.15.33:0/1770796144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48b78c010 con 0x55c48b78b240 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0eea59c640 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7a5cf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0ee009add0 tx=0x7f0ee0010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9229ad9795df59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0ed3fff640 1 -- 172.21.15.33:0/1770796144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00b9070 con 0x55c48b78b240 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0ed3fff640 1 -- 172.21.15.33:0/1770796144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee009ae40 con 0x55c48b78b240 2024-04-24T23:21:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.321+0000 7f0ed3fff640 1 -- 172.21.15.33:0/1770796144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee009d040 con 0x55c48b78b240 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1770796144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 msgr2=0x55c48b7a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7a5cf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0ee009add0 tx=0x7f0ee0010970 comp rx=0 tx=0).stop 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1770796144 shutdown_connections 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b789860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/1770796144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7a5cf0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1770796144 >> 172.21.15.33:0/1770796144 conn(0x55c48b7868b0 msgr2=0x55c48b78d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1770796144 shutdown_connections 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/1770796144 wait complete. 2024-04-24T23:21:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 -- start start 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.322+0000 7f0ef3d9b740 1 Processor -- start 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7ab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7ab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b7ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1a6890 con 0x55c48b78b240 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1a7d00 con 0x55c48b790da0 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1a6f60 con 0x55c48b78bc30 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ef3d9b740 10 client.0 _ll_register_callbacks cb 0x7f0ee4009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7ab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33246/0 (socket says 172.21.15.33:33246) 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 -- 172.21.15.33:0/752409436 learned_addr learned my addr 172.21.15.33:0/752409436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eead9d640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b7ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 -- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 msgr2=0x55c48b7ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7ab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 -- 172.21.15.33:0/752409436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 msgr2=0x55c48b7ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b7ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 -- 172.21.15.33:0/752409436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48b7acde0 con 0x55c48b78b240 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eead9d640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b7ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0eea59c640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7ab340 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00cc0e0 tx=0x7f0ee0006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5529178aa517d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.323+0000 7f0ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/752409436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee009d070 con 0x55c48b78b240 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/752409436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ee00974b0 con 0x55c48b78b240 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee009ae40 con 0x55c48b78b240 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/752409436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ee00ad050 con 0x55c48b78b240 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 10 client.17442 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c48b818c40 con 0x55c48b78b240 2024-04-24T23:21:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/752409436 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee009ae40 con 0x55c48b78b240 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/752409436 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ee0099e20 con 0x55c48b78b240 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 10 client.17442 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 20 client.17442 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b819500 0x55c48b81b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 4 client.17442 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c48b6f6f40 con 0x55c48b819500 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.324+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b819500 0x55c48b81b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ee9d9b640 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b819500 0x55c48b81b920 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b037b36870eb17a7 server_cookie=d624ddb4441edfd3 in_seq=0 out_seq=0 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ed1ffb640 10 client.17442 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ed1ffb640 1 -- 172.21.15.33:0/752409436 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c48b6f6f40 con 0x55c48b819500 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ed1ffb640 10 client.17442 handle_command_reply: tid=0 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 client.17442 shutdown 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 msgr2=0x55c48b7ab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7ab340 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00cc0e0 tx=0x7f0ee0006d80 comp rx=0 tx=0).stop 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 shutdown_connections 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c48b819500 0x55c48b81b920 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ed1ffb640 0 client.17442 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c48b790da0 0x55c48b7ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c48b78bc30 0x55c48b7ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 --2- 172.21.15.33:0/752409436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c48b78b240 0x55c48b7ab340 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 >> 172.21.15.33:0/752409436 conn(0x55c48b7868b0 msgr2=0x55c48b785260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 shutdown_connections 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 1 -- 172.21.15.33:0/752409436 wait complete. 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.325+0000 7f0ef3d9b740 20 client.17442 trim_cache size 0 max 16384 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:21:21.333 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: "b51b3e23-6287-4fe5-b964-c1113f7bc8f0": { 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "b51b3e23-6287-4fe5-b964-c1113f7bc8f0", 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:21:21.334 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ed40a2ce0 msgr2=0x7f0ed40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ed40a2ce0 0x7f0ed40a51a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55c48b78c490 tx=0x7f0ee4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 msgr2=0x7f0eec196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 0x7f0eec196a80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b3c0 tx=0x7f0ee000b880 comp rx=0 tx=0).stop 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 shutdown_connections 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ed40a2ce0 0x7f0ed40a51a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eec102250 0x7f0eec19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eec101890 0x7f0eec196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 --2- 172.21.15.33:0/745674734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eec1006a0 0x7f0eec196a80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 >> 172.21.15.33:0/745674734 conn(0x7f0eec0fbc10 msgr2=0x7f0eec104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 shutdown_connections 2024-04-24T23:21:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:21.326+0000 7f0ef0fb6640 1 -- 172.21.15.33:0/745674734 wait complete. 2024-04-24T23:21:21.339 INFO:tasks.cephfs.filesystem:scrub status for tag:b51b3e23-6287-4fe5-b964-c1113f7bc8f0 - {'path': '/', 'tag': 'b51b3e23-6287-4fe5-b964-c1113f7bc8f0', 'options': 'recursive,force'} 2024-04-24T23:21:21.459 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1058278 12.67 MB/sec execute 326 sec latency 80.611 ms 2024-04-24T23:21:22.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:21 smithi038 ceph-mon[32450]: pgmap v631: 129 pgs: 129 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2024-04-24T23:21:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:21 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:21:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:21 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:21 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:21:22.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:21 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:22.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:21 smithi117 ceph-mon[32529]: pgmap v631: 129 pgs: 129 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2024-04-24T23:21:22.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:21 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:21:22.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:21 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:22.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:21 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:21:22.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:21 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:21 smithi033 ceph-mon[29201]: pgmap v631: 129 pgs: 129 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2024-04-24T23:21:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:21 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:21:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:21 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:21 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:21:22.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:21 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:22.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1059807 12.65 MB/sec execute 327 sec latency 66.023 ms 2024-04-24T23:21:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:22 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:21:23.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:22 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:22 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:21:23.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:22 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:22 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:21:23.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:22 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:23.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1062517 12.66 MB/sec execute 328 sec latency 38.178 ms 2024-04-24T23:21:24.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:23 smithi038 ceph-mon[32450]: pgmap v632: 129 pgs: 129 active+clean; 278 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-04-24T23:21:24.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:23 smithi117 ceph-mon[32529]: pgmap v632: 129 pgs: 129 active+clean; 278 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-04-24T23:21:24.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:23 smithi033 ceph-mon[29201]: pgmap v632: 129 pgs: 129 active+clean; 278 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-04-24T23:21:24.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1064350 12.64 MB/sec execute 329 sec latency 100.675 ms 2024-04-24T23:21:25.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:24 smithi038 ceph-mon[32450]: pgmap v633: 129 pgs: 129 active+clean; 277 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-04-24T23:21:25.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:24 smithi117 ceph-mon[32529]: pgmap v633: 129 pgs: 129 active+clean; 277 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-04-24T23:21:25.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:24 smithi033 ceph-mon[29201]: pgmap v633: 129 pgs: 129 active+clean; 277 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-04-24T23:21:25.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1066447 12.65 MB/sec execute 330 sec latency 14.112 ms 2024-04-24T23:21:26.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1069047 12.65 MB/sec execute 331 sec latency 91.715 ms 2024-04-24T23:21:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:27 smithi038 ceph-mon[32450]: pgmap v634: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 298 op/s 2024-04-24T23:21:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:27 smithi117 ceph-mon[32529]: pgmap v634: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 298 op/s 2024-04-24T23:21:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:27 smithi033 ceph-mon[29201]: pgmap v634: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 298 op/s 2024-04-24T23:21:27.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1070460 12.63 MB/sec execute 332 sec latency 26.204 ms 2024-04-24T23:21:28.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:28 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:21:28.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:28 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:21:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:28 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:21:28.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1073201 12.65 MB/sec execute 333 sec latency 10.788 ms 2024-04-24T23:21:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:29 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:21:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:29 smithi038 ceph-mon[32450]: pgmap v635: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:21:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:29 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:21:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:29 smithi117 ceph-mon[32529]: pgmap v635: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:21:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:29 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:21:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:29 smithi033 ceph-mon[29201]: pgmap v635: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:21:29.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1075802 12.65 MB/sec execute 334 sec latency 23.946 ms 2024-04-24T23:21:30.458 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1077607 12.63 MB/sec execute 335 sec latency 29.084 ms 2024-04-24T23:21:31.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:31 smithi038 ceph-mon[32450]: pgmap v636: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-24T23:21:31.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:31 smithi117 ceph-mon[32529]: pgmap v636: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-24T23:21:31.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:31 smithi033 ceph-mon[29201]: pgmap v636: 129 pgs: 129 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-24T23:21:31.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1080120 12.65 MB/sec execute 336 sec latency 60.673 ms 2024-04-24T23:21:32.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1082272 12.63 MB/sec execute 337 sec latency 23.023 ms 2024-04-24T23:21:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:33 smithi038 ceph-mon[32450]: pgmap v637: 129 pgs: 129 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-04-24T23:21:33.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:33 smithi117 ceph-mon[32529]: pgmap v637: 129 pgs: 129 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-04-24T23:21:33.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:33 smithi033 ceph-mon[29201]: pgmap v637: 129 pgs: 129 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-04-24T23:21:33.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1084230 12.63 MB/sec execute 338 sec latency 20.466 ms 2024-04-24T23:21:34.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1087173 12.65 MB/sec execute 339 sec latency 14.493 ms 2024-04-24T23:21:35.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:35 smithi038 ceph-mon[32450]: pgmap v638: 129 pgs: 129 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-04-24T23:21:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:35 smithi117 ceph-mon[32529]: pgmap v638: 129 pgs: 129 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-04-24T23:21:35.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:35 smithi033 ceph-mon[29201]: pgmap v638: 129 pgs: 129 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-04-24T23:21:35.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1089308 12.63 MB/sec execute 340 sec latency 23.006 ms 2024-04-24T23:21:36.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1091199 12.63 MB/sec execute 341 sec latency 70.840 ms 2024-04-24T23:21:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:37 smithi038 ceph-mon[32450]: pgmap v639: 129 pgs: 129 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-04-24T23:21:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:37 smithi117 ceph-mon[32529]: pgmap v639: 129 pgs: 129 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-04-24T23:21:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:37 smithi033 ceph-mon[29201]: pgmap v639: 129 pgs: 129 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-04-24T23:21:37.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1093438 12.63 MB/sec execute 342 sec latency 39.739 ms 2024-04-24T23:21:38.458 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1094968 12.61 MB/sec execute 343 sec latency 71.112 ms 2024-04-24T23:21:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:39 smithi038 ceph-mon[32450]: pgmap v640: 129 pgs: 129 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-04-24T23:21:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:39 smithi117 ceph-mon[32529]: pgmap v640: 129 pgs: 129 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-04-24T23:21:39.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:39 smithi033 ceph-mon[29201]: pgmap v640: 129 pgs: 129 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-04-24T23:21:39.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1097904 12.63 MB/sec execute 344 sec latency 14.164 ms 2024-04-24T23:21:40.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1100368 12.63 MB/sec execute 345 sec latency 16.662 ms 2024-04-24T23:21:41.176 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:40 smithi117 ceph-mon[32529]: pgmap v641: 129 pgs: 129 active+clean; 290 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-04-24T23:21:41.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:40 smithi033 ceph-mon[29201]: pgmap v641: 129 pgs: 129 active+clean; 290 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-04-24T23:21:41.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:40 smithi038 ceph-mon[32450]: pgmap v641: 129 pgs: 129 active+clean; 290 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-04-24T23:21:41.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1102133 12.62 MB/sec execute 346 sec latency 13.788 ms 2024-04-24T23:21:42.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1104688 12.63 MB/sec execute 347 sec latency 65.379 ms 2024-04-24T23:21:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:43 smithi038 ceph-mon[32450]: pgmap v642: 129 pgs: 129 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 298 op/s 2024-04-24T23:21:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:43 smithi117 ceph-mon[32529]: pgmap v642: 129 pgs: 129 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 298 op/s 2024-04-24T23:21:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:43 smithi033 ceph-mon[29201]: pgmap v642: 129 pgs: 129 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 298 op/s 2024-04-24T23:21:43.456 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1106673 12.61 MB/sec execute 348 sec latency 16.111 ms 2024-04-24T23:21:44.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1108979 12.62 MB/sec execute 349 sec latency 26.208 ms 2024-04-24T23:21:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:45 smithi038 ceph-mon[32450]: pgmap v643: 129 pgs: 129 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-04-24T23:21:45.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:45 smithi117 ceph-mon[32529]: pgmap v643: 129 pgs: 129 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-04-24T23:21:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:45 smithi033 ceph-mon[29201]: pgmap v643: 129 pgs: 129 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-04-24T23:21:45.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1111946 12.63 MB/sec execute 350 sec latency 8.597 ms 2024-04-24T23:21:46.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1114161 12.62 MB/sec execute 351 sec latency 16.389 ms 2024-04-24T23:21:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:47 smithi038 ceph-mon[32450]: pgmap v644: 129 pgs: 129 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-04-24T23:21:47.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:47 smithi117 ceph-mon[32529]: pgmap v644: 129 pgs: 129 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-04-24T23:21:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:47 smithi033 ceph-mon[29201]: pgmap v644: 129 pgs: 129 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-04-24T23:21:47.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1115720 12.62 MB/sec execute 352 sec latency 69.599 ms 2024-04-24T23:21:48.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1118500 12.62 MB/sec execute 353 sec latency 14.118 ms 2024-04-24T23:21:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:49 smithi038 ceph-mon[32450]: pgmap v645: 129 pgs: 129 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-04-24T23:21:49.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:49 smithi117 ceph-mon[32529]: pgmap v645: 129 pgs: 129 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-04-24T23:21:49.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:49 smithi033 ceph-mon[29201]: pgmap v645: 129 pgs: 129 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-04-24T23:21:49.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1120309 12.61 MB/sec execute 354 sec latency 15.965 ms 2024-04-24T23:21:50.457 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1122852 12.62 MB/sec execute 355 sec latency 19.169 ms 2024-04-24T23:21:51.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:21:51.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:21:51.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:51 smithi033 ceph-mon[29201]: pgmap v646: 129 pgs: 129 active+clean; 270 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-04-24T23:21:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:51 smithi038 ceph-mon[32450]: pgmap v646: 129 pgs: 129 active+clean; 270 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-04-24T23:21:51.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:51 smithi117 ceph-mon[32529]: pgmap v646: 129 pgs: 129 active+clean; 270 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 -- 172.21.15.33:0/2227349913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70102980 msgr2=0x7f8e70102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 --2- 172.21.15.33:0/2227349913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70102980 0x7f8e70102e00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009900 tx=0x7f8e6402a410 comp rx=0 tx=0).stop 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 -- 172.21.15.33:0/2227349913 shutdown_connections 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 --2- 172.21.15.33:0/2227349913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e70103340 0x7f8e70109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 --2- 172.21.15.33:0/2227349913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70102980 0x7f8e70102e00 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 --2- 172.21.15.33:0/2227349913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e70101790 0x7f8e70101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 -- 172.21.15.33:0/2227349913 >> 172.21.15.33:0/2227349913 conn(0x7f8e700fcd20 msgr2=0x7f8e700ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 -- 172.21.15.33:0/2227349913 shutdown_connections 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.449+0000 7f8e75458640 1 -- 172.21.15.33:0/2227349913 wait complete. 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 Processor -- start 2024-04-24T23:21:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 -- start start 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e70101790 0x7f8e70199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e70102980 0x7f8e7019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 0x7f8e701a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010d460 con 0x7f8e70103340 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010d2e0 con 0x7f8e70102980 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e75458640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010d5e0 con 0x7f8e70101790 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 0x7f8e701a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 0x7f8e701a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56616/0 (socket says 172.21.15.33:56616) 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 -- 172.21.15.33:0/2356629250 learned_addr learned my addr 172.21.15.33:0/2356629250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e70102980 0x7f8e7019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6effd640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e70101790 0x7f8e70199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 -- 172.21.15.33:0/2356629250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e70101790 msgr2=0x7f8e70199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e70101790 0x7f8e70199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 -- 172.21.15.33:0/2356629250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e70102980 msgr2=0x7f8e7019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e70102980 0x7f8e7019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 -- 172.21.15.33:0/2356629250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e701a1900 con 0x7f8e70103340 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.450+0000 7f8e6f7fe640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 0x7f8e701a1260 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800eb10 tx=0x7f8e6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.451+0000 7f8e57fff640 1 -- 172.21.15.33:0/2356629250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68004b20 con 0x7f8e70103340 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.451+0000 7f8e57fff640 1 -- 172.21.15.33:0/2356629250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e68004cc0 con 0x7f8e70103340 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.451+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e701a1bf0 con 0x7f8e70103340 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.451+0000 7f8e57fff640 1 -- 172.21.15.33:0/2356629250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68010760 con 0x7f8e70103340 2024-04-24T23:21:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.451+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a2110 con 0x7f8e70103340 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.453+0000 7f8e57fff640 1 -- 172.21.15.33:0/2356629250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8e68005020 con 0x7f8e70103340 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.453+0000 7f8e78165740 1 Processor -- start 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.453+0000 7f8e78165740 1 -- start start 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e78165740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e78165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e78165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e85b0 0x5637347eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e78165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56373473e6b0 con 0x5637347e3440 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e78165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5637346e6880 con 0x5637347e85b0 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e78165740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56373473a670 con 0x5637347e2a50 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e57fff640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e580a2b90 0x7f8e580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56624/0 (socket says 172.21.15.33:56624) 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6effd640 1 -- 172.21.15.33:0/1696219889 learned_addr learned my addr 172.21.15.33:0/1696219889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e85b0 0x5637347eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6f7fe640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e580a2b90 0x7f8e580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e57fff640 1 -- 172.21.15.33:0/2356629250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e680cfa70 con 0x7f8e70103340 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1696219889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e85b0 msgr2=0x5637347eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e85b0 0x5637347eaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1696219889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 msgr2=0x5637347e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1696219889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373451c7d0 con 0x5637347e2a50 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347e2e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402a930 tx=0x7f8e64005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df001cd784d0e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e6f7fe640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e580a2b90 0x7f8e580a5050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e68098910 tx=0x7f8e6801bb30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.454+0000 7f8e557fa640 1 -- 172.21.15.33:0/1696219889 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64035070 con 0x5637347e2a50 2024-04-24T23:21:51.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e557fa640 1 -- 172.21.15.33:0/1696219889 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64032e50 con 0x5637347e2a50 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e557fa640 1 -- 172.21.15.33:0/1696219889 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6403ddf0 con 0x5637347e2a50 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 -- 172.21.15.33:0/1696219889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 msgr2=0x5637347e2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347e2e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402a930 tx=0x7f8e64005780 comp rx=0 tx=0).stop 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 -- 172.21.15.33:0/1696219889 shutdown_connections 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e85b0 0x5637347eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 --2- 172.21.15.33:0/1696219889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347e2e70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 -- 172.21.15.33:0/1696219889 >> 172.21.15.33:0/1696219889 conn(0x5637347de0c0 msgr2=0x5637347de4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 -- 172.21.15.33:0/1696219889 shutdown_connections 2024-04-24T23:21:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.455+0000 7f8e78165740 1 -- 172.21.15.33:0/1696219889 wait complete. 2024-04-24T23:21:51.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1125603 12.62 MB/sec execute 356 sec latency 13.542 ms 2024-04-24T23:21:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 -- start start 2024-04-24T23:21:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 Processor -- start 2024-04-24T23:21:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 0x5637347ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347f4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637347eb500 con 0x5637347e85b0 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5637346f9f50 con 0x5637347e2a50 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637347eb680 con 0x5637347e3440 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e78165740 10 client.0 _ll_register_callbacks cb 0x563734735000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 0x5637347ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 0x5637347ed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57844/0 (socket says 172.21.15.33:57844) 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6effd640 1 -- 172.21.15.33:0/398176280 learned_addr learned my addr 172.21.15.33:0/398176280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6effd640 1 -- 172.21.15.33:0/398176280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 msgr2=0x5637347ecea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.456+0000 7f8e6effd640 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347ecea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e6effd640 1 -- 172.21.15.33:0/398176280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 msgr2=0x5637347f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e6effd640 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e6effd640 1 -- 172.21.15.33:0/398176280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637344523f0 con 0x5637347e3440 2024-04-24T23:21:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e6effd640 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 0x5637347ed3e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e600126e0 tx=0x7f8e60012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9154f6607f815e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e52ff5640 1 -- 172.21.15.33:0/398176280 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60012190 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e64032e50 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e52ff5640 1 -- 172.21.15.33:0/398176280 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e6000f8c0 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e52ff5640 1 -- 172.21.15.33:0/398176280 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e6000fb00 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e78165740 10 client.26953 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637344120f0 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e52ff5640 1 -- 172.21.15.33:0/398176280 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64032e50 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.457+0000 7f8e52ff5640 1 -- 172.21.15.33:0/398176280 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8e6001c420 con 0x5637347e3440 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e78165740 10 client.26953 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e78165740 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e78165740 20 client.26953 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e78165740 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637347f8dc0 0x5637347fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e78165740 4 client.26953 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373474bbf0 con 0x5637347f8dc0 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637347f8dc0 0x5637347fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637347f8dc0 0x5637347fb200 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c28bf0d5d93220 server_cookie=3ba20469a518451c in_seq=0 out_seq=0 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.458+0000 7f8e52ff5640 10 client.26953 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e52ff5640 1 -- 172.21.15.33:0/398176280 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56373474bbf0 con 0x5637347f8dc0 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e52ff5640 10 client.26953 handle_command_reply: tid=0 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 client.26953 shutdown 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 msgr2=0x5637347ed3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 0x5637347ed3e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e600126e0 tx=0x7f8e60012ba0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 shutdown_connections 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637347f8dc0 0x5637347fb200 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e3440 0x5637347ed3e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e52ff5640 0 client.26953 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:51.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 --2- 172.21.15.33:0/398176280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347ecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 >> 172.21.15.33:0/398176280 conn(0x5637347de0c0 msgr2=0x5637347d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 shutdown_connections 2024-04-24T23:21:51.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 1 -- 172.21.15.33:0/398176280 wait complete. 2024-04-24T23:21:51.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.459+0000 7f8e78165740 20 client.26953 trim_cache size 0 max 16384 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 Processor -- start 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 -- start start 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e3440 0x5637347f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701aaaa0 con 0x5637347e85b0 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701a8370 con 0x5637347e3440 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e78165740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701a61c0 con 0x5637347e2a50 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347de4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57850/0 (socket says 172.21.15.33:57850) 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1158292271 learned_addr learned my addr 172.21.15.33:0/1158292271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.479+0000 7f8e6effd640 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e3440 0x5637347f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1158292271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e3440 msgr2=0x5637347f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e3440 0x5637347f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1158292271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 msgr2=0x5637347e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/1158292271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637347c7090 con 0x5637347e2a50 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347de4a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e64004540 tx=0x7f8e64031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=867d26f96b21461e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e54ff9640 1 -- 172.21.15.33:0/1158292271 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64035040 con 0x5637347e2a50 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e54ff9640 1 -- 172.21.15.33:0/1158292271 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64005500 con 0x5637347e2a50 2024-04-24T23:21:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.480+0000 7f8e54ff9640 1 -- 172.21.15.33:0/1158292271 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6403d8e0 con 0x5637347e2a50 2024-04-24T23:21:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 -- 172.21.15.33:0/1158292271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 msgr2=0x5637347de4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347de4a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e64004540 tx=0x7f8e64031400 comp rx=0 tx=0).stop 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 -- 172.21.15.33:0/1158292271 shutdown_connections 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e85b0 0x5637347e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e3440 0x5637347f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 --2- 172.21.15.33:0/1158292271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e2a50 0x5637347de4a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 -- 172.21.15.33:0/1158292271 >> 172.21.15.33:0/1158292271 conn(0x5637347de0c0 msgr2=0x5637347ecea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 -- 172.21.15.33:0/1158292271 shutdown_connections 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 -- 172.21.15.33:0/1158292271 wait complete. 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 -- start start 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.481+0000 7f8e78165740 1 Processor -- start 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44050/0 (socket says 172.21.15.33:44050) 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/2820318617 learned_addr learned my addr 172.21.15.33:0/2820318617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6effd640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e85b0 0x5637347f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701a6830 con 0x5637347e3440 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701a7ca0 con 0x5637347e2a50 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701a6f00 con 0x5637347e85b0 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e78165740 10 client.0 _ll_register_callbacks cb 0x7f8e60009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e85b0 0x5637347f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e85b0 msgr2=0x5637347f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e85b0 0x5637347f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/2820318617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 msgr2=0x5637347f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.482+0000 7f8e6e7fc640 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637347f3190 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e6effd640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e6e7fc640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347f1650 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e64006010 tx=0x7f8e640315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9029509b50333ede server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e52ff5640 1 -- 172.21.15.33:0/2820318617 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64067020 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e64005500 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e52ff5640 1 -- 172.21.15.33:0/2820318617 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e6403d860 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e52ff5640 1 -- 172.21.15.33:0/2820318617 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e64035040 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e78165740 10 client.26984 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637347f4480 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.483+0000 7f8e52ff5640 1 -- 172.21.15.33:0/2820318617 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64005500 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e52ff5640 1 -- 172.21.15.33:0/2820318617 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8e64059ce0 con 0x5637347e2a50 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 10 client.26984 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 20 client.26984 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56373485f490 0x563734861880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 4 client.26984 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563734745f40 con 0x56373485f490 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e6effd640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56373485f490 0x563734861880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e52ff5640 10 client.26984 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e6effd640 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56373485f490 0x563734861880 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49171d08f48b512c server_cookie=1cd9b82bc02e1ff7 in_seq=0 out_seq=0 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e52ff5640 1 -- 172.21.15.33:0/2820318617 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563734745f40 con 0x56373485f490 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e52ff5640 10 client.26984 handle_command_reply: tid=0 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.484+0000 7f8e78165740 1 client.26984 shutdown 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 msgr2=0x5637347f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347f1650 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e64006010 tx=0x7f8e640315b0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 shutdown_connections 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56373485f490 0x563734861880 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637347e85b0 0x5637347f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637347e3440 0x5637347f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e52ff5640 0 client.26984 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 --2- 172.21.15.33:0/2820318617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637347e2a50 0x5637347f1650 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 >> 172.21.15.33:0/2820318617 conn(0x5637347de0c0 msgr2=0x5637347e48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 shutdown_connections 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 1 -- 172.21.15.33:0/2820318617 wait complete. 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.485+0000 7f8e78165740 20 client.26984 trim_cache size 0 max 16384 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:21:51.492 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:21:51.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e580a2b90 msgr2=0x7f8e580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e580a2b90 0x7f8e580a5050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e68098910 tx=0x7f8e6801bb30 comp rx=0 tx=0).stop 2024-04-24T23:21:51.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 msgr2=0x7f8e701a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 0x7f8e701a1260 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800eb10 tx=0x7f8e6800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 shutdown_connections 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8e580a2b90 0x7f8e580a5050 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70103340 0x7f8e701a1260 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e70102980 0x7f8e7019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 --2- 172.21.15.33:0/2356629250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e70101790 0x7f8e70199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 >> 172.21.15.33:0/2356629250 conn(0x7f8e700fcd20 msgr2=0x7f8e70073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 shutdown_connections 2024-04-24T23:21:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.486+0000 7f8e75458640 1 -- 172.21.15.33:0/2356629250 wait complete. 2024-04-24T23:21:51.500 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:21:51.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 -- 172.21.15.33:0/3024376037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23941074e0 msgr2=0x7f23941098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 --2- 172.21.15.33:0/3024376037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23941074e0 0x7f23941098d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f238c009ca0 tx=0x7f238c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 -- 172.21.15.33:0/3024376037 shutdown_connections 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 --2- 172.21.15.33:0/3024376037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23941074e0 0x7f23941098d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 --2- 172.21.15.33:0/3024376037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2394104bb0 0x7f2394106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 --2- 172.21.15.33:0/3024376037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394102280 0x7f2394104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 -- 172.21.15.33:0/3024376037 >> 172.21.15.33:0/3024376037 conn(0x7f23940fbbf0 msgr2=0x7f23940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 -- 172.21.15.33:0/3024376037 shutdown_connections 2024-04-24T23:21:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 -- 172.21.15.33:0/3024376037 wait complete. 2024-04-24T23:21:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.643+0000 7f239b57e640 1 Processor -- start 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 -- start start 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 0x7f2394199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394104bb0 0x7f239419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23941074e0 0x7f23941a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f239410d420 con 0x7f2394104bb0 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f239410d2a0 con 0x7f2394102280 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239b57e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f239410d5a0 con 0x7f23941074e0 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f2398b19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394104bb0 0x7f239419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 0x7f2394199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f2398b19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394104bb0 0x7f239419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56662/0 (socket says 172.21.15.33:56662) 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 0x7f2394199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44074/0 (socket says 172.21.15.33:44074) 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f2398b19640 1 -- 172.21.15.33:0/2835257199 learned_addr learned my addr 172.21.15.33:0/2835257199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f2399b1b640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23941074e0 0x7f23941a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 -- 172.21.15.33:0/2835257199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23941074e0 msgr2=0x7f23941a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23941074e0 0x7f23941a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 -- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394104bb0 msgr2=0x7f239419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394104bb0 0x7f239419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 -- 172.21.15.33:0/2835257199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941a18b0 con 0x7f2394102280 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.644+0000 7f239931a640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 0x7f2394199c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f239000e9f0 tx=0x7f239000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.645+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239000ccc0 con 0x7f2394102280 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.645+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2390010910 con 0x7f2394102280 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.645+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a1ba0 con 0x7f2394102280 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.645+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390018960 con 0x7f2394102280 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.645+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a20f0 con 0x7f2394102280 2024-04-24T23:21:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.646+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2394108fd0 con 0x7f2394102280 2024-04-24T23:21:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.647+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2390010ac0 con 0x7f2394102280 2024-04-24T23:21:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.647+0000 7f238a7fc640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23800a2b90 0x7f23800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.647+0000 7f2398b19640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23800a2b90 0x7f23800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.647+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23900cfae0 con 0x7f2394102280 2024-04-24T23:21:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.648+0000 7f2398b19640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23800a2b90 0x7f23800a5050 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f239419b170 tx=0x7f2384005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.651+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f239009c060 con 0x7f2394102280 2024-04-24T23:21:51.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.934+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f239402d170 con 0x7f2394102280 2024-04-24T23:21:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.935+0000 7f238a7fc640 1 -- 172.21.15.33:0/2835257199 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f23900a0360 con 0x7f2394102280 2024-04-24T23:21:51.937 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:21:51.937 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:21:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23800a2b90 msgr2=0x7f23800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23800a2b90 0x7f23800a5050 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f239419b170 tx=0x7f2384005f90 comp rx=0 tx=0).stop 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 msgr2=0x7f2394199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 0x7f2394199c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f239000e9f0 tx=0x7f239000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 shutdown_connections 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f23800a2b90 0x7f23800a5050 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23941074e0 0x7f23941a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2394104bb0 0x7f239419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 --2- 172.21.15.33:0/2835257199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2394102280 0x7f2394199c70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.937+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 >> 172.21.15.33:0/2835257199 conn(0x7f23940fbbf0 msgr2=0x7f23941019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.938+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 shutdown_connections 2024-04-24T23:21:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:51.938+0000 7f239b57e640 1 -- 172.21.15.33:0/2835257199 wait complete. 2024-04-24T23:21:51.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:21:52.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 -- 172.21.15.33:0/4237176398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0102230 msgr2=0x7f20d0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 --2- 172.21.15.33:0/4237176398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0102230 0x7f20d0108ac0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009ca0 tx=0x7f20c802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 -- 172.21.15.33:0/4237176398 shutdown_connections 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 --2- 172.21.15.33:0/4237176398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0102230 0x7f20d0108ac0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 --2- 172.21.15.33:0/4237176398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0101870 0x7f20d0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 --2- 172.21.15.33:0/4237176398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20d0100680 0x7f20d0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.056+0000 7f20d8742640 1 -- 172.21.15.33:0/4237176398 >> 172.21.15.33:0/4237176398 conn(0x7f20d00fbbf0 msgr2=0x7f20d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 -- 172.21.15.33:0/4237176398 shutdown_connections 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 -- 172.21.15.33:0/4237176398 wait complete. 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 Processor -- start 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 -- start start 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 0x7f20d0104760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0101870 0x7f20d0102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20d0102230 0x7f20d0103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20d010c300 con 0x7f20d0101870 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20d010c180 con 0x7f20d0102230 2024-04-24T23:21:52.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d8742640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20d010c480 con 0x7f20d0100680 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d64de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 0x7f20d0104760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d64de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 0x7f20d0104760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57920/0 (socket says 172.21.15.33:57920) 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d64de640 1 -- 172.21.15.33:0/683533718 learned_addr learned my addr 172.21.15.33:0/683533718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0101870 0x7f20d0102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.057+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20d0102230 0x7f20d0103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d64de640 1 -- 172.21.15.33:0/683533718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20d0102230 msgr2=0x7f20d0103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d64de640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20d0102230 0x7f20d0103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d64de640 1 -- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0101870 msgr2=0x7f20d0102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d64de640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0101870 0x7f20d0102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d64de640 1 -- 172.21.15.33:0/683533718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d0103bf0 con 0x7f20d0100680 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0101870 0x7f20d0102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d64de640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 0x7f20d0104760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e9c0 tx=0x7f20cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20c77fe640 1 -- 172.21.15.33:0/683533718 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00cd80 con 0x7f20d0100680 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d01a1c00 con 0x7f20d0100680 2024-04-24T23:21:52.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20c77fe640 1 -- 172.21.15.33:0/683533718 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc0035f0 con 0x7f20d0100680 2024-04-24T23:21:52.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20c77fe640 1 -- 172.21.15.33:0/683533718 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc010690 con 0x7f20d0100680 2024-04-24T23:21:52.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.058+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d01a2120 con 0x7f20d0100680 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.060+0000 7f20c77fe640 1 -- 172.21.15.33:0/683533718 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f20cc0037a0 con 0x7f20d0100680 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.060+0000 7f20db44e740 1 Processor -- start 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 -- start start 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849230 0x56107e849650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20c77fe640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20bc0a2b90 0x7f20bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 0x56107e84e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e851220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56107e7ad670 con 0x56107e84ed90 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56107e755970 con 0x56107e849c20 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20db44e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56107e7a9630 con 0x56107e849230 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d64de640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20bc0a2b90 0x7f20bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 0x56107e84e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d6cdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849230 0x56107e849650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20c77fe640 1 -- 172.21.15.33:0/683533718 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20cc0cf7c0 con 0x7f20d0100680 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 0x56107e84e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44102/0 (socket says 172.21.15.33:44102) 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d6cdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849230 0x56107e849650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57932/0 (socket says 172.21.15.33:57932) 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 -- 172.21.15.33:0/713440817 learned_addr learned my addr 172.21.15.33:0/713440817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e851220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 -- 172.21.15.33:0/713440817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849230 msgr2=0x56107e849650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849230 0x56107e849650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 -- 172.21.15.33:0/713440817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 msgr2=0x56107e851220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e851220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d5cdd640 1 -- 172.21.15.33:0/713440817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107e779400 con 0x56107e849c20 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.061+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e851220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.062+0000 7f20d64de640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20bc0a2b90 0x7f20bc0a5050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f20cc09b970 tx=0x7f20cc01b6d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.062+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 0x56107e84e720 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20c0010450 tx=0x7f20c0010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428947d37b71487c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.062+0000 7f20c4ff9640 1 -- 172.21.15.33:0/713440817 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0011760 con 0x56107e849c20 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.062+0000 7f20c4ff9640 1 -- 172.21.15.33:0/713440817 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0011de0 con 0x56107e849c20 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.062+0000 7f20c4ff9640 1 -- 172.21.15.33:0/713440817 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c001a380 con 0x56107e849c20 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 -- 172.21.15.33:0/713440817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 msgr2=0x56107e84e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 0x56107e84e720 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20c0010450 tx=0x7f20c0010910 comp rx=0 tx=0).stop 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 -- 172.21.15.33:0/713440817 shutdown_connections 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e851220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849c20 0x56107e84e720 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 --2- 172.21.15.33:0/713440817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849230 0x56107e849650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 -- 172.21.15.33:0/713440817 >> 172.21.15.33:0/713440817 conn(0x56107e8448a0 msgr2=0x56107e844ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 -- 172.21.15.33:0/713440817 shutdown_connections 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 -- 172.21.15.33:0/713440817 wait complete. 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 -- start start 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.063+0000 7f20db44e740 1 Processor -- start 2024-04-24T23:21:52.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e853320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e853860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e85a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56107e851d40 con 0x56107e84ed90 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56107e769050 con 0x56107e849230 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56107e851ec0 con 0x56107e849c20 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 10 client.0 _ll_register_callbacks cb 0x56107e4c0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e85a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e853860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e853320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e853860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57934/0 (socket says 172.21.15.33:57934) 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d5cdd640 1 -- 172.21.15.33:0/1900871129 learned_addr learned my addr 172.21.15.33:0/1900871129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 -- 172.21.15.33:0/1900871129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 msgr2=0x56107e853860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e853860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 -- 172.21.15.33:0/1900871129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 msgr2=0x56107e85a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e85a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 -- 172.21.15.33:0/1900871129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107e73adc0 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e853860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e853320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f20c8031ae0 tx=0x7f20c8005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a04ece58aa74dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1900871129 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8003a30 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c0011de0 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.064+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1900871129 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20c8003bd0 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1900871129 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20c802f890 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 10 client.27002 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56107e4c13f0 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1900871129 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0011de0 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1900871129 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20c804b040 con 0x56107e849230 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 10 client.27002 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 10 client.27002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:21:52.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 20 client.27002 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:52.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e85f090 0x56107e861500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 4 client.27002 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:52.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56107e7babb0 con 0x56107e85f090 2024-04-24T23:21:52.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.065+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e85f090 0x56107e861500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.066+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e85f090 0x56107e861500 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39fe85aa81f9a192 server_cookie=dca2b51689b64ecf in_seq=0 out_seq=0 2024-04-24T23:21:52.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.066+0000 7f20b9ffb640 10 client.27002 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1900871129 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56107e7babb0 con 0x56107e85f090 2024-04-24T23:21:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20b9ffb640 10 client.27002 handle_command_reply: tid=0 2024-04-24T23:21:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 client.27002 shutdown 2024-04-24T23:21:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 msgr2=0x56107e853320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e853320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f20c8031ae0 tx=0x7f20c8005d80 comp rx=0 tx=0).stop 2024-04-24T23:21:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 shutdown_connections 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e85f090 0x56107e861500 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e85a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e853860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20db44e740 1 --2- 172.21.15.33:0/1900871129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e853320 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.068+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 >> 172.21.15.33:0/1900871129 conn(0x56107e8448a0 msgr2=0x56107e843b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.067+0000 7f20b9ffb640 0 client.27002 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.068+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 shutdown_connections 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.068+0000 7f20db44e740 1 -- 172.21.15.33:0/1900871129 wait complete. 2024-04-24T23:21:52.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.068+0000 7f20db44e740 20 client.27002 trim_cache size 0 max 16384 2024-04-24T23:21:52.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 Processor -- start 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 -- start start 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e8535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 0x56107e85a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e84ed90 0x56107e847480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01aee90 con 0x56107e849c20 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01ac760 con 0x56107e849230 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20db44e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01aa5b0 con 0x56107e84ed90 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 0x56107e85a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 0x56107e85a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56716/0 (socket says 172.21.15.33:56716) 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 -- 172.21.15.33:0/742276946 learned_addr learned my addr 172.21.15.33:0/742276946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e8535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e84ed90 0x56107e847480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 -- 172.21.15.33:0/742276946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e84ed90 msgr2=0x56107e847480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e84ed90 0x56107e847480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 -- 172.21.15.33:0/742276946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 msgr2=0x56107e8535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e8535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d5cdd640 1 -- 172.21.15.33:0/742276946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107e58b7d0 con 0x56107e849c20 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.083+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e8535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 0x56107e85a860 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f20c0010e10 tx=0x7f20c001c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1403ef59cc9e9433 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20bbfff640 1 -- 172.21.15.33:0/742276946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0030070 con 0x56107e849c20 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20bbfff640 1 -- 172.21.15.33:0/742276946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c001c800 con 0x56107e849c20 2024-04-24T23:21:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20bbfff640 1 -- 172.21.15.33:0/742276946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c002a470 con 0x56107e849c20 2024-04-24T23:21:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 -- 172.21.15.33:0/742276946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 msgr2=0x56107e85a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 0x56107e85a860 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f20c0010e10 tx=0x7f20c001c790 comp rx=0 tx=0).stop 2024-04-24T23:21:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 -- 172.21.15.33:0/742276946 shutdown_connections 2024-04-24T23:21:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e84ed90 0x56107e847480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e849c20 0x56107e85a860 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 --2- 172.21.15.33:0/742276946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e8535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 -- 172.21.15.33:0/742276946 >> 172.21.15.33:0/742276946 conn(0x56107e8448a0 msgr2=0x56107e843b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.084+0000 7f20db44e740 1 -- 172.21.15.33:0/742276946 shutdown_connections 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 -- 172.21.15.33:0/742276946 wait complete. 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 -- start start 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 Processor -- start 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e857a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e857fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e858f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01aac20 con 0x56107e84ed90 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01ac090 con 0x56107e849230 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01ab2f0 con 0x56107e849c20 2024-04-24T23:21:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20db44e740 10 client.0 _ll_register_callbacks cb 0x7f20c8031930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e857fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20d6cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e858f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20d5cdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e857fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57942/0 (socket says 172.21.15.33:57942) 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20d5cdd640 1 -- 172.21.15.33:0/1543092604 learned_addr learned my addr 172.21.15.33:0/1543092604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20d6cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e858f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56718/0 (socket says 172.21.15.33:56718) 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.085+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e857a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20d5cdd640 1 -- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 msgr2=0x56107e857a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e857a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20d5cdd640 1 -- 172.21.15.33:0/1543092604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 msgr2=0x56107e858f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e858f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20d5cdd640 1 -- 172.21.15.33:0/1543092604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107e859540 con 0x56107e849c20 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20d5cdd640 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e857fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b3d0 tx=0x7f20c0005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1bb6710c60ee00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1543092604 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0030070 con 0x56107e849c20 2024-04-24T23:21:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1543092604 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20c001cdc0 con 0x56107e849c20 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c001c800 con 0x56107e849c20 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1543092604 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20c0043050 con 0x56107e849c20 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 10 client.26983 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56107e85a840 con 0x56107e849c20 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1543092604 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c001c800 con 0x56107e849c20 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1543092604 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20c00035f0 con 0x56107e849c20 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 10 client.26983 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 10 client.26983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 20 client.26983 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.086+0000 7f20db44e740 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e8c5820 0x56107e8c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.087+0000 7f20db44e740 4 client.26983 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.087+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56107e7b4f00 con 0x56107e8c5820 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.087+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e8c5820 0x56107e8c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.087+0000 7f20b9ffb640 10 client.26983 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.087+0000 7f20d6cdf640 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e8c5820 0x56107e8c7c10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ddde0d3cc302ba1 server_cookie=4f920b4ebc0186d in_seq=0 out_seq=0 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20b9ffb640 1 -- 172.21.15.33:0/1543092604 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56107e7b4f00 con 0x56107e8c5820 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20b9ffb640 10 client.26983 handle_command_reply: tid=0 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 client.26983 shutdown 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 msgr2=0x56107e857fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e857fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b3d0 tx=0x7f20c0005cc0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 shutdown_connections 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56107e8c5820 0x56107e8c7c10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56107e84ed90 0x56107e858f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56107e849c20 0x56107e857fb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 --2- 172.21.15.33:0/1543092604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56107e849230 0x56107e857a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20b9ffb640 0 client.26983 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 >> 172.21.15.33:0/1543092604 conn(0x56107e8448a0 msgr2=0x56107e843b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 shutdown_connections 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 1 -- 172.21.15.33:0/1543092604 wait complete. 2024-04-24T23:21:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.088+0000 7f20db44e740 20 client.26983 trim_cache size 0 max 16384 2024-04-24T23:21:52.096 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20bc0a2b90 msgr2=0x7f20bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20bc0a2b90 0x7f20bc0a5050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f20cc09b970 tx=0x7f20cc01b6d0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 msgr2=0x7f20d0104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 0x7f20d0104760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e9c0 tx=0x7f20cc00ee80 comp rx=0 tx=0).stop 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 shutdown_connections 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20bc0a2b90 0x7f20bc0a5050 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20d0102230 0x7f20d0103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d0101870 0x7f20d0102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 --2- 172.21.15.33:0/683533718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d0100680 0x7f20d0104760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 >> 172.21.15.33:0/683533718 conn(0x7f20d00fbbf0 msgr2=0x7f20d00688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 shutdown_connections 2024-04-24T23:21:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.089+0000 7f20d8742640 1 -- 172.21.15.33:0/683533718 wait complete. 2024-04-24T23:21:52.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:21:52.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:52 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2835257199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:21:52.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 -- 172.21.15.33:0/1635145057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078101790 msgr2=0x7fd078101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 --2- 172.21.15.33:0/1635145057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078101790 0x7fd078101bb0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd068009900 tx=0x7fd06802a410 comp rx=0 tx=0).stop 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 -- 172.21.15.33:0/1635145057 shutdown_connections 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 --2- 172.21.15.33:0/1635145057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103340 0x7fd078109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 --2- 172.21.15.33:0/1635145057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 0x7fd078102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 --2- 172.21.15.33:0/1635145057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078101790 0x7fd078101bb0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 -- 172.21.15.33:0/1635145057 >> 172.21.15.33:0/1635145057 conn(0x7fd0780fcd20 msgr2=0x7fd0780ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 -- 172.21.15.33:0/1635145057 shutdown_connections 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.207+0000 7fd07e058640 1 -- 172.21.15.33:0/1635145057 wait complete. 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 Processor -- start 2024-04-24T23:21:52.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 -- start start 2024-04-24T23:21:52.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101790 0x7fd078197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 0x7fd078198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 0x7fd07819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810d460 con 0x7fd078103340 2024-04-24T23:21:52.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810d2e0 con 0x7fd078102980 2024-04-24T23:21:52.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd07e058640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810d5e0 con 0x7fd078101790 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 0x7fd07819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd06ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 0x7fd078198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 0x7fd07819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56744/0 (socket says 172.21.15.33:56744) 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd077fff640 1 -- 172.21.15.33:0/2879482910 learned_addr learned my addr 172.21.15.33:0/2879482910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.208+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101790 0x7fd078197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd077fff640 1 -- 172.21.15.33:0/2879482910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101790 msgr2=0x7fd078197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd077fff640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101790 0x7fd078197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd077fff640 1 -- 172.21.15.33:0/2879482910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 msgr2=0x7fd078198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd077fff640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 0x7fd078198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd077fff640 1 -- 172.21.15.33:0/2879482910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07819f750 con 0x7fd078103340 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 0x7fd078198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd077fff640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 0x7fd07819f0e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd07000b970 tx=0x7fd07000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd0757fa640 1 -- 172.21.15.33:0/2879482910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070019070 con 0x7fd078103340 2024-04-24T23:21:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd0757fa640 1 -- 172.21.15.33:0/2879482910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07000cb40 con 0x7fd078103340 2024-04-24T23:21:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd0757fa640 1 -- 172.21.15.33:0/2879482910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070007650 con 0x7fd078103340 2024-04-24T23:21:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07819fa40 con 0x7fd078103340 2024-04-24T23:21:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.209+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07819ffc0 con 0x7fd078103340 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.211+0000 7fd0757fa640 1 -- 172.21.15.33:0/2879482910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd07001c070 con 0x7fd078103340 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 Processor -- start 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 -- start start 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd0757fa640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0640a2b90 0x7fd0640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f1030 0x5570063f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 0x5570063f8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557006350670 con 0x5570063f61a0 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5570062f8970 con 0x5570063f0640 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd080d67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55700634c630 con 0x5570063f1030 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0640a2b90 0x7fd0640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 0x5570063f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 0x5570063f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56750/0 (socket says 172.21.15.33:56750) 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 -- 172.21.15.33:0/2640958998 learned_addr learned my addr 172.21.15.33:0/2640958998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd0757fa640 1 -- 172.21.15.33:0/2879482910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd0700cea10 con 0x7fd078103340 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f1030 0x5570063f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 -- 172.21.15.33:0/2640958998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f1030 msgr2=0x5570063f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f1030 0x5570063f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 -- 172.21.15.33:0/2640958998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 msgr2=0x5570063f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 -- 172.21.15.33:0/2640958998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700631c400 con 0x5570063f61a0 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0640a2b90 0x7fd0640a5050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd078199040 tx=0x7fd060013c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.212+0000 7fd077fff640 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 0x5570063f8630 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd07009aec0 tx=0x7fd07000f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c04176291277e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd06dffb640 1 -- 172.21.15.33:0/2640958998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07009b630 con 0x5570063f61a0 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd06dffb640 1 -- 172.21.15.33:0/2640958998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0700a49b0 con 0x5570063f61a0 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd06dffb640 1 -- 172.21.15.33:0/2640958998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07000f160 con 0x5570063f61a0 2024-04-24T23:21:52.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 -- 172.21.15.33:0/2640958998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 msgr2=0x5570063f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 0x5570063f8630 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd07009aec0 tx=0x7fd07000f300 comp rx=0 tx=0).stop 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 -- 172.21.15.33:0/2640958998 shutdown_connections 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f61a0 0x5570063f8630 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f1030 0x5570063f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 --2- 172.21.15.33:0/2640958998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 -- 172.21.15.33:0/2640958998 >> 172.21.15.33:0/2640958998 conn(0x5570063e78a0 msgr2=0x5570063e7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 -- 172.21.15.33:0/2640958998 shutdown_connections 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.213+0000 7fd080d67740 1 -- 172.21.15.33:0/2640958998 wait complete. 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 -- start start 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 Processor -- start 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x557006401c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5570063f90f0 con 0x5570063f1030 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55700630c050 con 0x5570063f0640 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570063f9270 con 0x5570063f61a0 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd080d67740 10 client.0 _ll_register_callbacks cb 0x557006063be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd06ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44158/0 (socket says 172.21.15.33:44158) 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd0777fe640 1 -- 172.21.15.33:0/2692047460 learned_addr learned my addr 172.21.15.33:0/2692047460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.214+0000 7fd077fff640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x557006401c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd06ffff640 1 -- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 msgr2=0x557006401c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x557006401c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd06ffff640 1 -- 172.21.15.33:0/2692047460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 msgr2=0x5570063fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd06ffff640 1 -- 172.21.15.33:0/2692047460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570062dddc0 con 0x5570063f1030 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fac40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x5570063f1aa0 tx=0x7fd060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab101a10b03aed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd056ffd640 1 -- 172.21.15.33:0/2692047460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060024030 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd056ffd640 1 -- 172.21.15.33:0/2692047460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd060003ac0 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0700a49b0 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd056ffd640 1 -- 172.21.15.33:0/2692047460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd060030050 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd080d67740 10 client.17496 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570060643f0 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd056ffd640 1 -- 172.21.15.33:0/2692047460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0700a49b0 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.215+0000 7fd056ffd640 1 -- 172.21.15.33:0/2692047460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd06001f070 con 0x5570063f1030 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd080d67740 10 client.17496 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd080d67740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd080d67740 20 client.17496 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd080d67740 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5570064064d0 0x557006408970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd080d67740 4 client.17496 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:52.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55700635dbb0 con 0x5570064064d0 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5570064064d0 0x557006408970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5570064064d0 0x557006408970 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be53a1724fcd1304 server_cookie=2e16b32cf0408495 in_seq=0 out_seq=0 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.216+0000 7fd056ffd640 10 client.17496 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd056ffd640 1 -- 172.21.15.33:0/2692047460 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55700635dbb0 con 0x5570064064d0 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd056ffd640 10 client.17496 handle_command_reply: tid=0 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 client.17496 shutdown 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 msgr2=0x5570063fac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fac40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x5570063f1aa0 tx=0x7fd060004730 comp rx=0 tx=0).stop 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 shutdown_connections 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5570064064d0 0x557006408970 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x557006401c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fac40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd056ffd640 0 client.17496 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 --2- 172.21.15.33:0/2692047460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f0640 0x5570063fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 >> 172.21.15.33:0/2692047460 conn(0x5570063e78a0 msgr2=0x5570063e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 shutdown_connections 2024-04-24T23:21:52.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 1 -- 172.21.15.33:0/2692047460 wait complete. 2024-04-24T23:21:52.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.217+0000 7fd080d67740 20 client.17496 trim_cache size 0 max 16384 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 Processor -- start 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 -- start start 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 0x5570063ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f1030 0x557006401c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x5570063ead50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781aab80 con 0x5570063f0640 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a8450 con 0x5570063f1030 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd080d67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a62a0 con 0x5570063f61a0 2024-04-24T23:21:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 0x5570063ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd077fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x5570063ead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 0x5570063ea810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56774/0 (socket says 172.21.15.33:56774) 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd0777fe640 1 -- 172.21.15.33:0/2240987219 learned_addr learned my addr 172.21.15.33:0/2240987219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.232+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f1030 0x557006401c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd0777fe640 1 -- 172.21.15.33:0/2240987219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 msgr2=0x5570063ead50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x5570063ead50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd0777fe640 1 -- 172.21.15.33:0/2240987219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f1030 msgr2=0x557006401c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f1030 0x557006401c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd0777fe640 1 -- 172.21.15.33:0/2240987219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700612e7d0 con 0x5570063f0640 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd077fff640 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x5570063ead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 0x5570063ea810 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd068009900 tx=0x7fd068000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dc9f03e1281dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd06d7fa640 1 -- 172.21.15.33:0/2240987219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068035040 con 0x5570063f0640 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd06d7fa640 1 -- 172.21.15.33:0/2240987219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06804b070 con 0x5570063f0640 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.233+0000 7fd06d7fa640 1 -- 172.21.15.33:0/2240987219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680462c0 con 0x5570063f0640 2024-04-24T23:21:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 -- 172.21.15.33:0/2240987219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 msgr2=0x5570063ea810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 0x5570063ea810 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd068009900 tx=0x7fd068000bc0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 -- 172.21.15.33:0/2240987219 shutdown_connections 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570063f61a0 0x5570063ead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f1030 0x557006401c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 --2- 172.21.15.33:0/2240987219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f0640 0x5570063ea810 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 -- 172.21.15.33:0/2240987219 >> 172.21.15.33:0/2240987219 conn(0x5570063e78a0 msgr2=0x5570063e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 -- 172.21.15.33:0/2240987219 shutdown_connections 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 -- 172.21.15.33:0/2240987219 wait complete. 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 -- start start 2024-04-24T23:21:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.234+0000 7fd080d67740 1 Processor -- start 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f61a0 0x5570063ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570064004c0 0x557006400940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a6910 con 0x5570063f1030 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a7d80 con 0x5570063f61a0 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a6fe0 con 0x5570064004c0 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd080d67740 10 client.0 _ll_register_callbacks cb 0x7fd060013550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56778/0 (socket says 172.21.15.33:56778) 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd0777fe640 1 -- 172.21.15.33:0/2391898543 learned_addr learned my addr 172.21.15.33:0/2391898543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f61a0 0x5570063ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd077fff640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570064004c0 0x557006400940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd06ffff640 1 -- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570064004c0 msgr2=0x557006400940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570064004c0 0x557006400940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd06ffff640 1 -- 172.21.15.33:0/2391898543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 msgr2=0x5570063fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd06ffff640 1 -- 172.21.15.33:0/2391898543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557006400e80 con 0x5570063f61a0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.235+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd06ffff640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f61a0 0x5570063ff4b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd060012420 tx=0x7fd060004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7b5c97ba0f085 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd056ffd640 1 -- 172.21.15.33:0/2391898543 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060039060 con 0x5570063f61a0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd06804b070 con 0x5570063f61a0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd056ffd640 1 -- 172.21.15.33:0/2391898543 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd0600039c0 con 0x5570063f61a0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd056ffd640 1 -- 172.21.15.33:0/2391898543 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd060040050 con 0x5570063f61a0 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd080d67740 10 client.27026 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55700646cb20 con 0x5570063f61a0 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.236+0000 7fd056ffd640 1 -- 172.21.15.33:0/2391898543 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06804b070 con 0x5570063f61a0 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd056ffd640 1 -- 172.21.15.33:0/2391898543 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd06001f050 con 0x5570063f61a0 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd080d67740 10 client.27026 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd080d67740 10 client.27026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd080d67740 20 client.27026 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd080d67740 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55700646d500 0x55700646f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd080d67740 4 client.27026 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557006357f00 con 0x55700646d500 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55700646d500 0x55700646f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd0777fe640 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55700646d500 0x55700646f920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f8764a945a72dac server_cookie=7df8d54cbf15d8b8 in_seq=0 out_seq=0 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.237+0000 7fd056ffd640 10 client.27026 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd056ffd640 1 -- 172.21.15.33:0/2391898543 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557006357f00 con 0x55700646d500 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd056ffd640 10 client.27026 handle_command_reply: tid=0 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 client.27026 shutdown 2024-04-24T23:21:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f61a0 msgr2=0x5570063ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f61a0 0x5570063ff4b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd060012420 tx=0x7fd060004a30 comp rx=0 tx=0).stop 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 shutdown_connections 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55700646d500 0x55700646f920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570064004c0 0x557006400940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570063f61a0 0x5570063ff4b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd056ffd640 0 client.27026 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 --2- 172.21.15.33:0/2391898543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5570063f1030 0x5570063fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 >> 172.21.15.33:0/2391898543 conn(0x5570063e78a0 msgr2=0x5570063e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 shutdown_connections 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 1 -- 172.21.15.33:0/2391898543 wait complete. 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.238+0000 7fd080d67740 20 client.27026 trim_cache size 0 max 16384 2024-04-24T23:21:52.245 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:21:52.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0640a2b90 msgr2=0x7fd0640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0640a2b90 0x7fd0640a5050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd078199040 tx=0x7fd060013c30 comp rx=0 tx=0).stop 2024-04-24T23:21:52.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 msgr2=0x7fd07819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 0x7fd07819f0e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd07000b970 tx=0x7fd07000be30 comp rx=0 tx=0).stop 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 shutdown_connections 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0640a2b90 0x7fd0640a5050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078103340 0x7fd07819f0e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102980 0x7fd078198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 --2- 172.21.15.33:0/2879482910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101790 0x7fd078197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 >> 172.21.15.33:0/2879482910 conn(0x7fd0780fcd20 msgr2=0x7fd078105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 shutdown_connections 2024-04-24T23:21:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.239+0000 7fd07e058640 1 -- 172.21.15.33:0/2879482910 wait complete. 2024-04-24T23:21:52.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:21:52.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 -- 172.21.15.33:0/4004085827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4100960 msgr2=0x7f92b4100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/4004085827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4100960 0x7f92b4100dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f929c009900 tx=0x7f929c02a410 comp rx=0 tx=0).stop 2024-04-24T23:21:52.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 -- 172.21.15.33:0/4004085827 shutdown_connections 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/4004085827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4101300 0x7f92b4101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/4004085827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4100960 0x7f92b4100dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/4004085827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b4100000 0x7f92b4100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.394+0000 7f92bb0b2640 1 -- 172.21.15.33:0/4004085827 >> 172.21.15.33:0/4004085827 conn(0x7f92b40fbc00 msgr2=0x7f92b40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.395+0000 7f92bb0b2640 1 -- 172.21.15.33:0/4004085827 shutdown_connections 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.395+0000 7f92bb0b2640 1 -- 172.21.15.33:0/4004085827 wait complete. 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 Processor -- start 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 -- start start 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b4100000 0x7f92b4070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 0x7f92b4070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4101300 0x7f92b41a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92b410f620 con 0x7f92b4100960 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92b410f4a0 con 0x7f92b4101300 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92bb0b2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92b410f7a0 con 0x7f92b4100000 2024-04-24T23:21:52.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 0x7f92b4070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 0x7f92b4070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56798/0 (socket says 172.21.15.33:56798) 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92b3fff640 1 -- 172.21.15.33:0/777514716 learned_addr learned my addr 172.21.15.33:0/777514716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.396+0000 7f92b964f640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4101300 0x7f92b41a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b4100000 0x7f92b4070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b3fff640 1 -- 172.21.15.33:0/777514716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b4100000 msgr2=0x7f92b4070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b3fff640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b4100000 0x7f92b4070100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b3fff640 1 -- 172.21.15.33:0/777514716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4101300 msgr2=0x7f92b41a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b3fff640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4101300 0x7f92b41a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b3fff640 1 -- 172.21.15.33:0/777514716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41aa270 con 0x7f92b4100960 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.397+0000 7f92b3fff640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 0x7f92b4070640 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f929c031440 tx=0x7f929c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.398+0000 7f92b1ffb640 1 -- 172.21.15.33:0/777514716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c03f030 con 0x7f92b4100960 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.398+0000 7f92b1ffb640 1 -- 172.21.15.33:0/777514716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c003ce0 con 0x7f92b4100960 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.398+0000 7f92b1ffb640 1 -- 172.21.15.33:0/777514716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c035070 con 0x7f92b4100960 2024-04-24T23:21:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.398+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41aa440 con 0x7f92b4100960 2024-04-24T23:21:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.398+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41aa7f0 con 0x7f92b4100960 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.402+0000 7f92b1ffb640 1 -- 172.21.15.33:0/777514716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f929c042030 con 0x7f92b4100960 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.402+0000 7f92bddbd740 1 Processor -- start 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.402+0000 7f92bddbd740 1 -- start start 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c38a40 0x556c07c38e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c39430 0x556c07c3df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c40a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92bddbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c07b94670 con 0x556c07c38a40 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92bddbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556c07b3c970 con 0x556c07c3e5a0 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92bddbd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c07b90630 con 0x556c07c39430 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c40a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b1ffb640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92a00a2c10 0x7f92a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c40a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44208/0 (socket says 172.21.15.33:44208) 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 -- 172.21.15.33:0/3193072664 learned_addr learned my addr 172.21.15.33:0/3193072664 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c39430 0x556c07c3df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b1ffb640 1 -- 172.21.15.33:0/777514716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f929c0f2170 con 0x7f92b4100960 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 -- 172.21.15.33:0/3193072664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c39430 msgr2=0x556c07c3df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c39430 0x556c07c3df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 -- 172.21.15.33:0/3193072664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c38a40 msgr2=0x556c07c38e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:21:52.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c38a40 0x556c07c38e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.403+0000 7f92b3fff640 1 -- 172.21.15.33:0/3193072664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c07b60400 con 0x556c07c3e5a0 2024-04-24T23:21:52.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.404+0000 7f92b964f640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92a00a2c10 0x7f92a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.404+0000 7f92b3fff640 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c40a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f929c0ec3e0 tx=0x7f929c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ce17e4abcfe2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.404+0000 7f92aaffd640 1 -- 172.21.15.33:0/3193072664 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0f3060 con 0x556c07c3e5a0 2024-04-24T23:21:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.404+0000 7f92aaffd640 1 -- 172.21.15.33:0/3193072664 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c0bb900 con 0x556c07c3e5a0 2024-04-24T23:21:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.404+0000 7f92b964f640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92a00a2c10 0x7f92a00a50d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x556c07c39c90 tx=0x7f92a4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.404+0000 7f92aaffd640 1 -- 172.21.15.33:0/3193072664 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c03bcd0 con 0x556c07c3e5a0 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 -- 172.21.15.33:0/3193072664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 msgr2=0x556c07c40a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c40a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f929c0ec3e0 tx=0x7f929c009260 comp rx=0 tx=0).stop 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 -- 172.21.15.33:0/3193072664 shutdown_connections 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c40a30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c39430 0x556c07c3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3193072664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c38a40 0x556c07c38e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.405+0000 7f92bddbd740 1 -- 172.21.15.33:0/3193072664 >> 172.21.15.33:0/3193072664 conn(0x556c07c2b8a0 msgr2=0x556c07c2bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.406+0000 7f92bddbd740 1 -- 172.21.15.33:0/3193072664 shutdown_connections 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.406+0000 7f92bddbd740 1 -- 172.21.15.33:0/3193072664 wait complete. 2024-04-24T23:21:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.406+0000 7f92bddbd740 1 -- start start 2024-04-24T23:21:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.407+0000 7f92bddbd740 1 Processor -- start 2024-04-24T23:21:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c49db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c07c41310 con 0x556c07c39430 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556c07b50050 con 0x556c07c3e5a0 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c07c41490 con 0x556c07c38a40 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92b8e4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92b8e4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c42830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:58026/0 (socket says 172.21.15.33:58026) 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92b8e4e640 1 -- 172.21.15.33:0/1063860317 learned_addr learned my addr 172.21.15.33:0/1063860317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.408+0000 7f92bddbd740 10 client.0 _ll_register_callbacks cb 0x556c078a7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b3fff640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c42d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b8e4e640 1 -- 172.21.15.33:0/1063860317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 msgr2=0x556c07c49db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c49db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b8e4e640 1 -- 172.21.15.33:0/1063860317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 msgr2=0x556c07c42d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c42d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b8e4e640 1 -- 172.21.15.33:0/1063860317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c07b21dc0 con 0x556c07c38a40 2024-04-24T23:21:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b3fff640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c42830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92ac003f30 tx=0x7f92ac0044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcbd38e562d2b1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f9293fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.409+0000 7f9293fff640 1 -- 172.21.15.33:0/1063860317 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac012a80 con 0x556c07c38a40 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.410+0000 7f9293fff640 1 -- 172.21.15.33:0/1063860317 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f92ac012c20 con 0x556c07c38a40 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.410+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c0bb900 con 0x556c07c38a40 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.410+0000 7f9293fff640 1 -- 172.21.15.33:0/1063860317 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92ac012e20 con 0x556c07c38a40 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.410+0000 7f92bddbd740 10 client.27010 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.410+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c078a83f0 con 0x556c07c38a40 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.410+0000 7f9293fff640 1 -- 172.21.15.33:0/1063860317 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ac01b960 con 0x556c07c38a40 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f9293fff640 1 -- 172.21.15.33:0/1063860317 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f92ac010920 con 0x556c07c38a40 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92bddbd740 10 client.27010 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92bddbd740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92bddbd740 20 client.27010 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92bddbd740 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c4e770 0x556c07c50c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92bddbd740 4 client.27010 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f929c0bb900 con 0x556c07c4e770 2024-04-24T23:21:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.411+0000 7f92b3fff640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c4e770 0x556c07c50c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.413+0000 7f92b3fff640 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c4e770 0x556c07c50c10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb701fd68ec0c54 server_cookie=2b1ab4a4a0291490 in_seq=0 out_seq=0 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.413+0000 7f9293fff640 10 client.27010 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.414+0000 7f9293fff640 1 -- 172.21.15.33:0/1063860317 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f929c0bb900 con 0x556c07c4e770 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.414+0000 7f9293fff640 10 client.27010 handle_command_reply: tid=0 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.414+0000 7f92bddbd740 1 client.27010 shutdown 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.415+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 msgr2=0x556c07c42830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.415+0000 7f92bddbd740 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c42830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92ac003f30 tx=0x7f92ac0044d0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.415+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 shutdown_connections 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.415+0000 7f92bddbd740 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c4e770 0x556c07c50c10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 1 --2- 172.21.15.33:0/1063860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c42830 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 >> 172.21.15.33:0/1063860317 conn(0x556c07c2b8a0 msgr2=0x556c07c2ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f9293fff640 0 client.27010 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 shutdown_connections 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 1 -- 172.21.15.33:0/1063860317 wait complete. 2024-04-24T23:21:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.416+0000 7f92bddbd740 20 client.27010 trim_cache size 0 max 16384 2024-04-24T23:21:52.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:52 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2835257199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:21:52.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:52 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2835257199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 Processor -- start 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 -- start start 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c38a40 0x556c07c2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c49d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c3e5a0 0x556c07c2eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41b3310 con 0x556c07c39430 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41b0be0 con 0x556c07c38a40 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.443+0000 7f92bddbd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41aea30 con 0x556c07c3e5a0 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b8e4e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c38a40 0x556c07c2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b8e4e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c38a40 0x556c07c2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44232/0 (socket says 172.21.15.33:44232) 2024-04-24T23:21:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b8e4e640 1 -- 172.21.15.33:0/2529634174 learned_addr learned my addr 172.21.15.33:0/2529634174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c3e5a0 0x556c07c2eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b3fff640 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c49d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 -- 172.21.15.33:0/2529634174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c38a40 msgr2=0x556c07c2e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c38a40 0x556c07c2e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 -- 172.21.15.33:0/2529634174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 msgr2=0x556c07c49d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c49d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 -- 172.21.15.33:0/2529634174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c079727d0 con 0x556c07c3e5a0 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.444+0000 7f92b964f640 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c3e5a0 0x556c07c2eac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92a4011810 tx=0x7f92a4013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20169d8559253ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.445+0000 7f92aa7fc640 1 -- 172.21.15.33:0/2529634174 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4003af0 con 0x556c07c3e5a0 2024-04-24T23:21:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.445+0000 7f92aa7fc640 1 -- 172.21.15.33:0/2529634174 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a4005500 con 0x556c07c3e5a0 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.445+0000 7f92aa7fc640 1 -- 172.21.15.33:0/2529634174 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a401e420 con 0x556c07c3e5a0 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 -- 172.21.15.33:0/2529634174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c3e5a0 msgr2=0x556c07c2eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c3e5a0 0x556c07c2eac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92a4011810 tx=0x7f92a4013040 comp rx=0 tx=0).stop 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 -- 172.21.15.33:0/2529634174 shutdown_connections 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c3e5a0 0x556c07c2eac0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c49d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 --2- 172.21.15.33:0/2529634174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c38a40 0x556c07c2e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 -- 172.21.15.33:0/2529634174 >> 172.21.15.33:0/2529634174 conn(0x556c07c2b8a0 msgr2=0x556c07c2ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 -- 172.21.15.33:0/2529634174 shutdown_connections 2024-04-24T23:21:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.446+0000 7f92bddbd740 1 -- 172.21.15.33:0/2529634174 wait complete. 2024-04-24T23:21:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.447+0000 7f92bddbd740 1 -- start start 2024-04-24T23:21:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.447+0000 7f92bddbd740 1 Processor -- start 2024-04-24T23:21:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c47170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92bddbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92bddbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41af0a0 con 0x556c07c39430 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56824/0 (socket says 172.21.15.33:56824) 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92b3fff640 1 -- 172.21.15.33:0/3799281270 learned_addr learned my addr 172.21.15.33:0/3799281270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.448+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c47170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41b0510 con 0x556c07c3e5a0 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92b964f640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c48670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41af770 con 0x556c07c38a40 2024-04-24T23:21:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92bddbd740 10 client.0 _ll_register_callbacks cb 0x7f92ac009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92b3fff640 1 -- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 msgr2=0x556c07c47170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92b3fff640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c47170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92b3fff640 1 -- 172.21.15.33:0/3799281270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 msgr2=0x556c07c48670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92b3fff640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c48670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.449+0000 7f92b3fff640 1 -- 172.21.15.33:0/3799281270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c07c48c60 con 0x556c07c39430 2024-04-24T23:21:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f92b3fff640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c476b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f929c0bb190 tx=0x7f929c0db040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db682e79bbeff9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f9293fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f9293fff640 1 -- 172.21.15.33:0/3799281270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c03bcd0 con 0x556c07c39430 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f9293fff640 1 -- 172.21.15.33:0/3799281270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f929c003800 con 0x556c07c39430 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a4005500 con 0x556c07c39430 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f9293fff640 1 -- 172.21.15.33:0/3799281270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f929c0bab60 con 0x556c07c39430 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f92bddbd740 10 client.17514 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.450+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c07c49f90 con 0x556c07c39430 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f9293fff640 1 -- 172.21.15.33:0/3799281270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a4005500 con 0x556c07c39430 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f9293fff640 1 -- 172.21.15.33:0/3799281270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f929c0b7870 con 0x556c07c39430 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f92bddbd740 10 client.17514 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f92bddbd740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f92bddbd740 20 client.17514 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c36460 0x556c07c36870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.451+0000 7f92bddbd740 4 client.17514 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.452+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c07ba1bb0 con 0x556c07c36460 2024-04-24T23:21:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.452+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c36460 0x556c07c36870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.452+0000 7f92b8e4e640 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c36460 0x556c07c36870 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d2fac3ed4f24cc server_cookie=de2b42885b91d1a in_seq=0 out_seq=0 2024-04-24T23:21:52.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.452+0000 7f9293fff640 10 client.17514 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:52.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.453+0000 7f9293fff640 1 -- 172.21.15.33:0/3799281270 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c07ba1bb0 con 0x556c07c36460 2024-04-24T23:21:52.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.453+0000 7f9293fff640 10 client.17514 handle_command_reply: tid=0 2024-04-24T23:21:52.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.453+0000 7f92bddbd740 1 client.17514 shutdown 2024-04-24T23:21:52.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.454+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 msgr2=0x556c07c476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.454+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c476b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f929c0bb190 tx=0x7f929c0db040 comp rx=0 tx=0).stop 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 shutdown_connections 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556c07c36460 0x556c07c36870 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556c07c3e5a0 0x556c07c48670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c07c39430 0x556c07c476b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 --2- 172.21.15.33:0/3799281270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c07c38a40 0x556c07c47170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 >> 172.21.15.33:0/3799281270 conn(0x556c07c2b8a0 msgr2=0x556c07c39e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f9293fff640 0 client.17514 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 shutdown_connections 2024-04-24T23:21:52.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.455+0000 7f92bddbd740 1 -- 172.21.15.33:0/3799281270 wait complete. 2024-04-24T23:21:52.458 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:21:52.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.456+0000 7f92bddbd740 20 client.17514 trim_cache size 0 max 16384 2024-04-24T23:21:52.459 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1126772 12.60 MB/sec execute 357 sec latency 43.501 ms 2024-04-24T23:21:52.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92a00a2c10 msgr2=0x7f92a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92a00a2c10 0x7f92a00a50d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x556c07c39c90 tx=0x7f92a4005f90 comp rx=0 tx=0).stop 2024-04-24T23:21:52.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 msgr2=0x7f92b4070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 0x7f92b4070640 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f929c031440 tx=0x7f929c032ce0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 shutdown_connections 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f92a00a2c10 0x7f92a00a50d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92b4101300 0x7f92b41a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4100960 0x7f92b4070640 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 --2- 172.21.15.33:0/777514716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b4100000 0x7f92b4070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.459+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 >> 172.21.15.33:0/777514716 conn(0x7f92b40fbc00 msgr2=0x7f92b40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.460+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 shutdown_connections 2024-04-24T23:21:52.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:52.460+0000 7f92bb0b2640 1 -- 172.21.15.33:0/777514716 wait complete. 2024-04-24T23:21:53.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:53 smithi038 ceph-mon[32450]: pgmap v647: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-04-24T23:21:53.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:53 smithi117 ceph-mon[32529]: pgmap v647: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-04-24T23:21:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:53 smithi033 ceph-mon[29201]: pgmap v647: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-04-24T23:21:53.459 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1129672 12.61 MB/sec execute 358 sec latency 10.939 ms 2024-04-24T23:21:53.482 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:21:53.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:53.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:21:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 -- 172.21.15.33:0/1631220402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 msgr2=0x7f6844109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1631220402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 0x7f6844109bd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f683c009ca0 tx=0x7f683c02a8b0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 -- 172.21.15.33:0/1631220402 shutdown_connections 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1631220402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 0x7f6844109bd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1631220402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6844102980 0x7f6844102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1631220402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f6844101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 -- 172.21.15.33:0/1631220402 >> 172.21.15.33:0/1631220402 conn(0x7f68440fcd20 msgr2=0x7f68440ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 -- 172.21.15.33:0/1631220402 shutdown_connections 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.596+0000 7f6848ecd640 1 -- 172.21.15.33:0/1631220402 wait complete. 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 Processor -- start 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 -- start start 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f68440754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6844102980 0x7f6844073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 0x7f6844074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f684410d3f0 con 0x7f6844101790 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f684410d270 con 0x7f6844103340 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6848ecd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f684410d570 con 0x7f6844102980 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f684259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f68440754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f684259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f68440754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56836/0 (socket says 172.21.15.33:56836) 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f684259c640 1 -- 172.21.15.33:0/1960890898 learned_addr learned my addr 172.21.15.33:0/1960890898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f6842d9d640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 0x7f6844074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.597+0000 7f683bfff640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6844102980 0x7f6844073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f684259c640 1 -- 172.21.15.33:0/1960890898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6844102980 msgr2=0x7f6844073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f684259c640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6844102980 0x7f6844073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f684259c640 1 -- 172.21.15.33:0/1960890898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 msgr2=0x7f6844074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f684259c640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 0x7f6844074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f684259c640 1 -- 172.21.15.33:0/1960890898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6844074900 con 0x7f6844101790 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f684259c640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f68440754f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f683400e9c0 tx=0x7f683400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f683b7fe640 1 -- 172.21.15.33:0/1960890898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683400cd80 con 0x7f6844101790 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68441a1b30 con 0x7f6844101790 2024-04-24T23:21:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f683b7fe640 1 -- 172.21.15.33:0/1960890898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68340035f0 con 0x7f6844101790 2024-04-24T23:21:53.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f683b7fe640 1 -- 172.21.15.33:0/1960890898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834010690 con 0x7f6844101790 2024-04-24T23:21:53.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.598+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68441a20d0 con 0x7f6844101790 2024-04-24T23:21:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.600+0000 7f683b7fe640 1 -- 172.21.15.33:0/1960890898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f68340037a0 con 0x7f6844101790 2024-04-24T23:21:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.600+0000 7f684bbdc740 1 Processor -- start 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 -- start start 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555704d6c2c0 con 0x555704e0e8f0 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555704d14910 con 0x555704e09780 2024-04-24T23:21:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684bbdc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555704d68280 con 0x555704e08d90 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e10d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56838/0 (socket says 172.21.15.33:56838) 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e0e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44280/0 (socket says 172.21.15.33:44280) 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 -- 172.21.15.33:0/2971081469 learned_addr learned my addr 172.21.15.33:0/2971081469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f6842d9d640 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683b7fe640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68300a2b90 0x7f68300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f6842d9d640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68300a2b90 0x7f68300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 -- 172.21.15.33:0/2971081469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 msgr2=0x555704e091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 -- 172.21.15.33:0/2971081469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 msgr2=0x555704e0e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e0e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f683bfff640 1 -- 172.21.15.33:0/2971081469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555704d25fb0 con 0x555704e0e8f0 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.601+0000 7f684259c640 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.602+0000 7f683b7fe640 1 -- 172.21.15.33:0/1960890898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68340cf870 con 0x7f6844101790 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.602+0000 7f683bfff640 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e10d80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f682c009c90 tx=0x7f682c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f356462b7cdbd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.602+0000 7f6842d9d640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68300a2b90 0x7f68300a5050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x555704e09fe0 tx=0x7f683c02a8b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.602+0000 7f6838ff9640 1 -- 172.21.15.33:0/2971081469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c01e070 con 0x555704e0e8f0 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.602+0000 7f6838ff9640 1 -- 172.21.15.33:0/2971081469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c019b10 con 0x555704e0e8f0 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.602+0000 7f6838ff9640 1 -- 172.21.15.33:0/2971081469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c00ea50 con 0x555704e0e8f0 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 -- 172.21.15.33:0/2971081469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 msgr2=0x555704e10d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e10d80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f682c009c90 tx=0x7f682c011810 comp rx=0 tx=0).stop 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 -- 172.21.15.33:0/2971081469 shutdown_connections 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e10d80 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2971081469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 -- 172.21.15.33:0/2971081469 >> 172.21.15.33:0/2971081469 conn(0x555704dd0950 msgr2=0x555704e04810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 -- 172.21.15.33:0/2971081469 shutdown_connections 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 -- 172.21.15.33:0/2971081469 wait complete. 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 -- start start 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.603+0000 7f684bbdc740 1 Processor -- start 2024-04-24T23:21:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 0x555704e131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e09780 0x555704e13720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555704e118a0 con 0x555704e0e8f0 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555704d27e90 con 0x555704e08d90 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555704e11a20 con 0x555704e09780 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684bbdc740 10 client.0 _ll_register_callbacks cb 0x555704d63970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 0x555704e131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 0x555704e131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44284/0 (socket says 172.21.15.33:44284) 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 -- 172.21.15.33:0/2877153677 learned_addr learned my addr 172.21.15.33:0/2877153677 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f683bfff640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e09780 0x555704e13720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f6842d9d640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 -- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e09780 msgr2=0x555704e13720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e09780 0x555704e13720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 -- 172.21.15.33:0/2877153677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 msgr2=0x555704e1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 -- 172.21.15.33:0/2877153677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555704b487e0 con 0x555704e08d90 2024-04-24T23:21:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f6842d9d640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f684259c640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 0x555704e131e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f68340058c0 tx=0x7f683409b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45eb0f737edbe034 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.604+0000 7f6829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:21:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f6829ffb640 1 -- 172.21.15.33:0/2877153677 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683409c3e0 con 0x555704e08d90 2024-04-24T23:21:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f6829ffb640 1 -- 172.21.15.33:0/2877153677 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6834003b30 con 0x555704e08d90 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c019b10 con 0x555704e08d90 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f6829ffb640 1 -- 172.21.15.33:0/2877153677 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68340a4950 con 0x555704e08d90 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 10 client.27053 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555704ba8f70 con 0x555704e08d90 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f6829ffb640 1 -- 172.21.15.33:0/2877153677 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c019b10 con 0x555704e08d90 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f6829ffb640 1 -- 172.21.15.33:0/2877153677 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f68340b3060 con 0x555704e08d90 2024-04-24T23:21:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 10 client.27053 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 20 client.27053 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e1ec50 0x555704e21110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 4 client.27053 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.605+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555704d405a0 con 0x555704e1ec50 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.606+0000 7f683bfff640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e1ec50 0x555704e21110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.606+0000 7f683bfff640 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e1ec50 0x555704e21110 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad939c6cb5ea770 server_cookie=8a0336b909f0d596 in_seq=0 out_seq=0 2024-04-24T23:21:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.606+0000 7f6829ffb640 10 client.27053 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f6829ffb640 1 -- 172.21.15.33:0/2877153677 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555704d405a0 con 0x555704e1ec50 2024-04-24T23:21:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f6829ffb640 10 client.27053 handle_command_reply: tid=0 2024-04-24T23:21:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 client.27053 shutdown 2024-04-24T23:21:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 msgr2=0x555704e131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 0x555704e131e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f68340058c0 tx=0x7f683409b700 comp rx=0 tx=0).stop 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 shutdown_connections 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e1ec50 0x555704e21110 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e09780 0x555704e13720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f6829ffb640 0 client.27053 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 --2- 172.21.15.33:0/2877153677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e08d90 0x555704e131e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 >> 172.21.15.33:0/2877153677 conn(0x555704dd0950 msgr2=0x555704e03950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 shutdown_connections 2024-04-24T23:21:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 1 -- 172.21.15.33:0/2877153677 wait complete. 2024-04-24T23:21:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.607+0000 7f684bbdc740 20 client.27053 trim_cache size 0 max 16384 2024-04-24T23:21:53.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 Processor -- start 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 -- start start 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68441aef00 con 0x555704e0e8f0 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68441ac7d0 con 0x555704e09780 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.626+0000 7f684bbdc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68441aa620 con 0x555704e08d90 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f6842d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44290/0 (socket says 172.21.15.33:44290) 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 -- 172.21.15.33:0/1198631615 learned_addr learned my addr 172.21.15.33:0/1198631615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f684259c640 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 -- 172.21.15.33:0/1198631615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 msgr2=0x555704e13460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e13460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 -- 172.21.15.33:0/1198631615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 msgr2=0x555704e259d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e259d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 -- 172.21.15.33:0/1198631615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555704a40170 con 0x555704e09780 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f683bfff640 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f682c005fd0 tx=0x7f682c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1bd382b148f678 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f682bfff640 1 -- 172.21.15.33:0/1198631615 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c01e040 con 0x555704e09780 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f682bfff640 1 -- 172.21.15.33:0/1198631615 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c005450 con 0x555704e09780 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.627+0000 7f682bfff640 1 -- 172.21.15.33:0/1198631615 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c003850 con 0x555704e09780 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 -- 172.21.15.33:0/1198631615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 msgr2=0x555704e1a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f682c005fd0 tx=0x7f682c005850 comp rx=0 tx=0).stop 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 -- 172.21.15.33:0/1198631615 shutdown_connections 2024-04-24T23:21:53.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e259d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a720 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 --2- 172.21.15.33:0/1198631615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e13460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 -- 172.21.15.33:0/1198631615 >> 172.21.15.33:0/1198631615 conn(0x555704dd0950 msgr2=0x555704e03950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 -- 172.21.15.33:0/1198631615 shutdown_connections 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 -- 172.21.15.33:0/1198631615 wait complete. 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 -- start start 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 Processor -- start 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.628+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e19f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68441aac90 con 0x555704e0e8f0 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68441ac100 con 0x555704e09780 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68441ab360 con 0x555704e08d90 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 10 client.0 _ll_register_callbacks cb 0x7f68340d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e18510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56854/0 (socket says 172.21.15.33:56854) 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 -- 172.21.15.33:0/4009784503 learned_addr learned my addr 172.21.15.33:0/4009784503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f683bfff640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684259c640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 -- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 msgr2=0x555704e19f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e19f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 -- 172.21.15.33:0/4009784503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 msgr2=0x555704e1a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 -- 172.21.15.33:0/4009784503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555704e18b50 con 0x555704e0e8f0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684259c640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e19f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f683bfff640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6842d9d640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e18510 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f683c035000 tx=0x7f683c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124479d2522242bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6829ffb640 1 -- 172.21.15.33:0/4009784503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c051070 con 0x555704e0e8f0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c005450 con 0x555704e0e8f0 2024-04-24T23:21:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6829ffb640 1 -- 172.21.15.33:0/4009784503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f683c004450 con 0x555704e0e8f0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f6829ffb640 1 -- 172.21.15.33:0/4009784503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f683c004650 con 0x555704e0e8f0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 10 client.17532 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.629+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555704e85c40 con 0x555704e0e8f0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f6829ffb640 1 -- 172.21.15.33:0/4009784503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c005450 con 0x555704e0e8f0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f6829ffb640 1 -- 172.21.15.33:0/4009784503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f683c04a040 con 0x555704e0e8f0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684bbdc740 10 client.17532 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684bbdc740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684bbdc740 20 client.17532 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684bbdc740 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e86650 0x555704e88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684bbdc740 4 client.17532 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555704d79800 con 0x555704e86650 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684259c640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e86650 0x555704e88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f684259c640 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e86650 0x555704e88a70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90de4cafa182595f server_cookie=1e87ffcc6b35f795 in_seq=0 out_seq=0 2024-04-24T23:21:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.630+0000 7f6829ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f6829ffb640 1 -- 172.21.15.33:0/4009784503 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555704d79800 con 0x555704e86650 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f6829ffb640 10 client.17532 handle_command_reply: tid=0 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 client.17532 shutdown 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 msgr2=0x555704e18510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e18510 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f683c035000 tx=0x7f683c0095c0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 shutdown_connections 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555704e86650 0x555704e88a70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555704e0e8f0 0x555704e18510 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f6829ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555704e09780 0x555704e1a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 --2- 172.21.15.33:0/4009784503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555704e08d90 0x555704e19f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 >> 172.21.15.33:0/4009784503 conn(0x555704dd0950 msgr2=0x555704e07700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 shutdown_connections 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 1 -- 172.21.15.33:0/4009784503 wait complete. 2024-04-24T23:21:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.632+0000 7f684bbdc740 20 client.17532 trim_cache size 0 max 16384 2024-04-24T23:21:53.644 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:21:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:21:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "b7e3c299-cf51-4e16-a781-2bb9012569a4", 2024-04-24T23:21:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:21:53.644 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68300a2b90 msgr2=0x7f68300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68300a2b90 0x7f68300a5050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x555704e09fe0 tx=0x7f683c02a8b0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 msgr2=0x7f68440754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f68440754f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f683400e9c0 tx=0x7f683400ee80 comp rx=0 tx=0).stop 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 shutdown_connections 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68300a2b90 0x7f68300a5050 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6844103340 0x7f6844074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6844102980 0x7f6844073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 --2- 172.21.15.33:0/1960890898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6844101790 0x7f68440754f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 >> 172.21.15.33:0/1960890898 conn(0x7f68440fcd20 msgr2=0x7f6844105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 shutdown_connections 2024-04-24T23:21:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.633+0000 7f6848ecd640 1 -- 172.21.15.33:0/1960890898 wait complete. 2024-04-24T23:21:53.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:21:53.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 -- 172.21.15.33:0/39894161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 msgr2=0x7fed70101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 --2- 172.21.15.33:0/39894161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70101bb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009900 tx=0x7fed6c02a410 comp rx=0 tx=0).stop 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 -- 172.21.15.33:0/39894161 shutdown_connections 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 --2- 172.21.15.33:0/39894161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 0x7fed70109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 --2- 172.21.15.33:0/39894161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed70102980 0x7fed70102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 --2- 172.21.15.33:0/39894161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70101bb0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 -- 172.21.15.33:0/39894161 >> 172.21.15.33:0/39894161 conn(0x7fed700fcd20 msgr2=0x7fed700ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 -- 172.21.15.33:0/39894161 shutdown_connections 2024-04-24T23:21:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.759+0000 7fed78110640 1 -- 172.21.15.33:0/39894161 wait complete. 2024-04-24T23:21:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 Processor -- start 2024-04-24T23:21:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 -- start start 2024-04-24T23:21:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed70102980 0x7fed70197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 0x7fed7019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010d410 con 0x7fed70101790 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010d290 con 0x7fed70102980 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed78110640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010d590 con 0x7fed70103340 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56872/0 (socket says 172.21.15.33:56872) 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 -- 172.21.15.33:0/3383699003 learned_addr learned my addr 172.21.15.33:0/3383699003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed756ab640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed70102980 0x7fed70197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed766ad640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 0x7fed7019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 -- 172.21.15.33:0/3383699003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 msgr2=0x7fed7019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 0x7fed7019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 -- 172.21.15.33:0/3383699003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed70102980 msgr2=0x7fed70197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed70102980 0x7fed70197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 -- 172.21.15.33:0/3383699003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7019f6b0 con 0x7fed70101790 2024-04-24T23:21:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed766ad640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 0x7fed7019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.760+0000 7fed75eac640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70197a90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fed6c031610 tx=0x7fed6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.761+0000 7fed66ffd640 1 -- 172.21.15.33:0/3383699003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c03f030 con 0x7fed70101790 2024-04-24T23:21:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.761+0000 7fed66ffd640 1 -- 172.21.15.33:0/3383699003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6c003ad0 con 0x7fed70101790 2024-04-24T23:21:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.761+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7019f940 con 0x7fed70101790 2024-04-24T23:21:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.761+0000 7fed66ffd640 1 -- 172.21.15.33:0/3383699003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c035070 con 0x7fed70101790 2024-04-24T23:21:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.761+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7019fe30 con 0x7fed70101790 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.763+0000 7fed66ffd640 1 -- 172.21.15.33:0/3383699003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fed6c042030 con 0x7fed70101790 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.763+0000 7fed7ae18740 1 Processor -- start 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.763+0000 7fed7ae18740 1 -- start start 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 0x55e71abf7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf8040 0x55e71abfcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71abff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed7ae18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e71ab576b0 con 0x55e71abf7650 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed7ae18740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e71aaff880 con 0x55e71abf8040 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed7ae18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e71ab53670 con 0x55e71abfd1b0 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 0x55e71abf7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed75eac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf8040 0x55e71abfcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 0x55e71abf7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56888/0 (socket says 172.21.15.33:56888) 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 -- 172.21.15.33:0/1446831244 learned_addr learned my addr 172.21.15.33:0/1446831244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed756ab640 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71abff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed66ffd640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed5c0a2b90 0x7fed5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 -- 172.21.15.33:0/1446831244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 msgr2=0x55e71abff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71abff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 -- 172.21.15.33:0/1446831244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf8040 msgr2=0x55e71abfcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf8040 0x55e71abfcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed766ad640 1 -- 172.21.15.33:0/1446831244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71a9357d0 con 0x55e71abf7650 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.764+0000 7fed756ab640 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71abff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.765+0000 7fed66ffd640 1 -- 172.21.15.33:0/3383699003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed6c0efeb0 con 0x7fed70101790 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.765+0000 7fed766ad640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed5c0a2b90 0x7fed5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.765+0000 7fed766ad640 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 0x55e71abf7a70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fed68009ea0 tx=0x7fed68009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84737f29e37751e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.765+0000 7fed5bfff640 1 -- 172.21.15.33:0/1446831244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68021070 con 0x55e71abf7650 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.765+0000 7fed5bfff640 1 -- 172.21.15.33:0/1446831244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6801bde0 con 0x55e71abf7650 2024-04-24T23:21:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.765+0000 7fed5bfff640 1 -- 172.21.15.33:0/1446831244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68010b60 con 0x55e71abf7650 2024-04-24T23:21:53.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 -- 172.21.15.33:0/1446831244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 msgr2=0x55e71abf7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 0x55e71abf7a70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fed68009ea0 tx=0x7fed68009f90 comp rx=0 tx=0).stop 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed766ad640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed5c0a2b90 0x7fed5c0a5050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed68010010 tx=0x7fed68021000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 -- 172.21.15.33:0/1446831244 shutdown_connections 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71abff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf8040 0x55e71abfcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 --2- 172.21.15.33:0/1446831244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf7650 0x55e71abf7a70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 -- 172.21.15.33:0/1446831244 >> 172.21.15.33:0/1446831244 conn(0x55e71abee8b0 msgr2=0x55e71abeecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 -- 172.21.15.33:0/1446831244 shutdown_connections 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 -- 172.21.15.33:0/1446831244 wait complete. 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 -- start start 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.766+0000 7fed7ae18740 1 Processor -- start 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf7650 0x55e71ac01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71ac08ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e71ac00160 con 0x55e71abf8040 2024-04-24T23:21:53.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e71ab12f50 con 0x55e71abf7650 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e71ac002e0 con 0x55e71abfd1b0 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed7ae18740 10 client.0 _ll_register_callbacks cb 0x55e71ab4e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed75eac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf7650 0x55e71ac01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56892/0 (socket says 172.21.15.33:56892) 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 -- 172.21.15.33:0/4188411096 learned_addr learned my addr 172.21.15.33:0/4188411096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed766ad640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71ac08ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 -- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 msgr2=0x55e71ac08ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71ac08ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 -- 172.21.15.33:0/4188411096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf7650 msgr2=0x55e71ac01a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf7650 0x55e71ac01a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 -- 172.21.15.33:0/4188411096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71a86b3f0 con 0x55e71abf8040 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed75eac640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf7650 0x55e71ac01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed756ab640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac01fb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed60012700 tx=0x7fed60012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136116aac8aa63f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.767+0000 7fed597fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed597fa640 1 -- 172.21.15.33:0/4188411096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60004020 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed597fa640 1 -- 172.21.15.33:0/4188411096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed600041c0 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6801bde0 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed597fa640 1 -- 172.21.15.33:0/4188411096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed60025050 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 10 client.17544 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e71a82b0f0 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed597fa640 1 -- 172.21.15.33:0/4188411096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6801bde0 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed597fa640 1 -- 172.21.15.33:0/4188411096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fed60003930 con 0x55e71abf8040 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 10 client.17544 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 20 client.17544 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac0d930 0x55e71ac0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 4 client.17544 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.768+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e71ab64bf0 con 0x55e71ac0d930 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.769+0000 7fed75eac640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac0d930 0x55e71ac0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.769+0000 7fed75eac640 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac0d930 0x55e71ac0fdd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ec5e17c07c41b6 server_cookie=81143e53abc10e95 in_seq=0 out_seq=0 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.769+0000 7fed597fa640 10 client.17544 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.769+0000 7fed597fa640 1 -- 172.21.15.33:0/4188411096 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e71ab64bf0 con 0x55e71ac0d930 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.769+0000 7fed597fa640 10 client.17544 handle_command_reply: tid=0 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 client.17544 shutdown 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 msgr2=0x55e71ac01fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac01fb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed60012700 tx=0x7fed60012bc0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 shutdown_connections 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac0d930 0x55e71ac0fdd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abfd1b0 0x55e71ac08ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac01fb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 --2- 172.21.15.33:0/4188411096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abf7650 0x55e71ac01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed597fa640 0 client.17544 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 >> 172.21.15.33:0/4188411096 conn(0x55e71abee8b0 msgr2=0x55e71abedaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 shutdown_connections 2024-04-24T23:21:53.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 1 -- 172.21.15.33:0/4188411096 wait complete. 2024-04-24T23:21:53.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.770+0000 7fed7ae18740 20 client.17544 trim_cache size 0 max 16384 2024-04-24T23:21:53.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.787+0000 7fed7ae18740 1 Processor -- start 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.787+0000 7fed7ae18740 1 -- start start 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71abf94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac08fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71abf99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed7ae18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed701aaa70 con 0x55e71abf8040 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed7ae18740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed701a8340 con 0x55e71abfd1b0 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed7ae18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed701a6190 con 0x55e71abf7650 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac08fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac08fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56906/0 (socket says 172.21.15.33:56906) 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 -- 172.21.15.33:0/3556041706 learned_addr learned my addr 172.21.15.33:0/3556041706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed766ad640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71abf99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed75eac640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71abf94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 -- 172.21.15.33:0/3556041706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 msgr2=0x55e71abf94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71abf94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 -- 172.21.15.33:0/3556041706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 msgr2=0x55e71abf99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71abf99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 -- 172.21.15.33:0/3556041706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71abee450 con 0x55e71abf8040 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed75eac640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71abf94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed766ad640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71abf99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed756ab640 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac08fb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fed60001eb0 tx=0x7fed60009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6b079759d62df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed5b7fe640 1 -- 172.21.15.33:0/3556041706 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60022040 con 0x55e71abf8040 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed5b7fe640 1 -- 172.21.15.33:0/3556041706 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed600055b0 con 0x55e71abf8040 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.788+0000 7fed5b7fe640 1 -- 172.21.15.33:0/3556041706 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6001c9e0 con 0x55e71abf8040 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 -- 172.21.15.33:0/3556041706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 msgr2=0x55e71ac08fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac08fb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fed60001eb0 tx=0x7fed60009d90 comp rx=0 tx=0).stop 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 -- 172.21.15.33:0/3556041706 shutdown_connections 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71abf99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac08fb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 --2- 172.21.15.33:0/3556041706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71abf94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 -- 172.21.15.33:0/3556041706 >> 172.21.15.33:0/3556041706 conn(0x55e71abee8b0 msgr2=0x55e71abeec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 -- 172.21.15.33:0/3556041706 shutdown_connections 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 -- 172.21.15.33:0/3556041706 wait complete. 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 -- start start 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.789+0000 7fed7ae18740 1 Processor -- start 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71ac06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac06670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71ac07680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed701a6800 con 0x55e71abf8040 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed701a7c70 con 0x55e71abfd1b0 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed701a6ed0 con 0x55e71abf7650 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed7ae18740 10 client.0 _ll_register_callbacks cb 0x7fed600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac06670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac06670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56920/0 (socket says 172.21.15.33:56920) 2024-04-24T23:21:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 -- 172.21.15.33:0/2359454461 learned_addr learned my addr 172.21.15.33:0/2359454461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed766ad640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71ac07680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed75eac640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71ac06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 -- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 msgr2=0x55e71ac06130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71ac06130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 -- 172.21.15.33:0/2359454461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 msgr2=0x55e71ac07680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71ac07680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 -- 172.21.15.33:0/2359454461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71ac07c70 con 0x55e71abf8040 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed756ab640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac06670 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fed60003880 tx=0x7fed600038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe585ca8bcf9dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed597fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.790+0000 7fed597fa640 1 -- 172.21.15.33:0/2359454461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600052e0 con 0x55e71abf8040 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed597fa640 1 -- 172.21.15.33:0/2359454461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed6001c780 con 0x55e71abf8040 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed600055b0 con 0x55e71abf8040 2024-04-24T23:21:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed597fa640 1 -- 172.21.15.33:0/2359454461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed6001c980 con 0x55e71abf8040 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 10 client.17550 fetch_fsmap learned FSMap version 27 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e71ac08ef0 con 0x55e71abf8040 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed597fa640 1 -- 172.21.15.33:0/2359454461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed600055b0 con 0x55e71abf8040 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed597fa640 1 -- 172.21.15.33:0/2359454461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fed6000fcd0 con 0x55e71abf8040 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 10 client.17550 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 20 client.17550 populate_metadata read hostname 'smithi033' 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac74320 0x55e71ac76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 4 client.17550 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e71ab5ef40 con 0x55e71ac74320 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.791+0000 7fed75eac640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac74320 0x55e71ac76740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed597fa640 10 client.17550 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed75eac640 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac74320 0x55e71ac76740 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd12c2c16118b347 server_cookie=9443bd1605594dfc in_seq=0 out_seq=0 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed597fa640 1 -- 172.21.15.33:0/2359454461 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e71ab5ef40 con 0x55e71ac74320 2024-04-24T23:21:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed597fa640 10 client.17550 handle_command_reply: tid=0 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 client.17550 shutdown 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 msgr2=0x55e71ac06670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac06670 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fed60003880 tx=0x7fed600038b0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 shutdown_connections 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e71ac74320 0x55e71ac76740 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e71abfd1b0 0x55e71ac07680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e71abf8040 0x55e71ac06670 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed597fa640 0 client.17550 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 --2- 172.21.15.33:0/2359454461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71abf7650 0x55e71ac06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 >> 172.21.15.33:0/2359454461 conn(0x55e71abee8b0 msgr2=0x55e71ac01a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 shutdown_connections 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 1 -- 172.21.15.33:0/2359454461 wait complete. 2024-04-24T23:21:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.792+0000 7fed7ae18740 20 client.17550 trim_cache size 0 max 16384 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: "b7e3c299-cf51-4e16-a781-2bb9012569a4": { 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "b7e3c299-cf51-4e16-a781-2bb9012569a4", 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:21:53.801 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed5c0a2b90 msgr2=0x7fed5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed5c0a2b90 0x7fed5c0a5050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed68010010 tx=0x7fed68021000 comp rx=0 tx=0).stop 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 msgr2=0x7fed70197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70197a90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fed6c031610 tx=0x7fed6c004270 comp rx=0 tx=0).stop 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 shutdown_connections 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed5c0a2b90 0x7fed5c0a5050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70103340 0x7fed7019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed70102980 0x7fed70197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 --2- 172.21.15.33:0/3383699003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed70101790 0x7fed70197a90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 >> 172.21.15.33:0/3383699003 conn(0x7fed700fcd20 msgr2=0x7fed70105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 shutdown_connections 2024-04-24T23:21:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:21:53.793+0000 7fed78110640 1 -- 172.21.15.33:0/3383699003 wait complete. 2024-04-24T23:21:53.805 INFO:tasks.cephfs.filesystem:scrub status for tag:b7e3c299-cf51-4e16-a781-2bb9012569a4 - {'path': '/', 'tag': 'b7e3c299-cf51-4e16-a781-2bb9012569a4', 'options': 'recursive,force'} 2024-04-24T23:21:54.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1131283 12.60 MB/sec execute 359 sec latency 41.187 ms 2024-04-24T23:21:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:21:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:21:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:21:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:55 smithi038 ceph-mon[32450]: pgmap v648: 129 pgs: 129 active+clean; 276 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-04-24T23:21:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:21:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:21:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:21:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:55 smithi117 ceph-mon[32529]: pgmap v648: 129 pgs: 129 active+clean; 276 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-04-24T23:21:55.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:21:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:21:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:21:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:21:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:55 smithi033 ceph-mon[29201]: pgmap v648: 129 pgs: 129 active+clean; 276 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-04-24T23:21:55.459 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1133654 12.60 MB/sec execute 360 sec latency 19.232 ms 2024-04-24T23:21:56.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1136374 12.61 MB/sec execute 361 sec latency 12.366 ms 2024-04-24T23:21:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:57 smithi038 ceph-mon[32450]: pgmap v649: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:21:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:57 smithi117 ceph-mon[32529]: pgmap v649: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:21:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:57 smithi033 ceph-mon[29201]: pgmap v649: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:21:57.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1137537 12.59 MB/sec execute 362 sec latency 98.164 ms 2024-04-24T23:21:58.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1140421 12.60 MB/sec execute 363 sec latency 21.313 ms 2024-04-24T23:21:59.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:21:59 smithi038 ceph-mon[32450]: pgmap v650: 129 pgs: 129 active+clean; 279 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:21:59.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:21:59 smithi117 ceph-mon[32529]: pgmap v650: 129 pgs: 129 active+clean; 279 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:21:59.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:21:59 smithi033 ceph-mon[29201]: pgmap v650: 129 pgs: 129 active+clean; 279 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-24T23:21:59.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1142585 12.59 MB/sec execute 364 sec latency 18.483 ms 2024-04-24T23:22:00.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1144546 12.59 MB/sec execute 365 sec latency 22.403 ms 2024-04-24T23:22:01.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:01 smithi038 ceph-mon[32450]: pgmap v651: 129 pgs: 129 active+clean; 279 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-04-24T23:22:01.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:01 smithi117 ceph-mon[32529]: pgmap v651: 129 pgs: 129 active+clean; 279 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-04-24T23:22:01.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:01 smithi033 ceph-mon[29201]: pgmap v651: 129 pgs: 129 active+clean; 279 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-04-24T23:22:01.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1147537 12.60 MB/sec execute 366 sec latency 12.317 ms 2024-04-24T23:22:02.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1148886 12.58 MB/sec execute 367 sec latency 52.530 ms 2024-04-24T23:22:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:03 smithi038 ceph-mon[32450]: pgmap v652: 129 pgs: 129 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-04-24T23:22:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:22:03.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:03 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:03 smithi117 ceph-mon[32529]: pgmap v652: 129 pgs: 129 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-04-24T23:22:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:22:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:03 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:03 smithi033 ceph-mon[29201]: pgmap v652: 129 pgs: 129 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-04-24T23:22:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:22:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:03 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:03.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1151379 12.59 MB/sec execute 368 sec latency 12.421 ms 2024-04-24T23:22:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:04 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:22:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:04.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:04 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:22:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:04 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:22:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:04 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:22:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:04 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:22:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:04 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:22:04.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1154037 12.59 MB/sec execute 369 sec latency 15.994 ms 2024-04-24T23:22:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:05 smithi038 ceph-mon[32450]: pgmap v653: 129 pgs: 129 active+clean; 277 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-04-24T23:22:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:05 smithi117 ceph-mon[32529]: pgmap v653: 129 pgs: 129 active+clean; 277 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-04-24T23:22:05.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:05 smithi033 ceph-mon[29201]: pgmap v653: 129 pgs: 129 active+clean; 277 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-04-24T23:22:05.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1155769 12.58 MB/sec execute 370 sec latency 18.191 ms 2024-04-24T23:22:06.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1158359 12.59 MB/sec execute 371 sec latency 19.327 ms 2024-04-24T23:22:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:07 smithi038 ceph-mon[32450]: pgmap v654: 129 pgs: 129 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-04-24T23:22:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:07 smithi117 ceph-mon[32529]: pgmap v654: 129 pgs: 129 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-04-24T23:22:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:07 smithi033 ceph-mon[29201]: pgmap v654: 129 pgs: 129 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-04-24T23:22:07.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1159818 12.57 MB/sec execute 372 sec latency 139.888 ms 2024-04-24T23:22:08.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1162135 12.57 MB/sec execute 373 sec latency 17.615 ms 2024-04-24T23:22:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:09 smithi038 ceph-mon[32450]: pgmap v655: 129 pgs: 129 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2024-04-24T23:22:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:09 smithi117 ceph-mon[32529]: pgmap v655: 129 pgs: 129 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2024-04-24T23:22:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:09 smithi033 ceph-mon[29201]: pgmap v655: 129 pgs: 129 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2024-04-24T23:22:09.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1164850 12.58 MB/sec execute 374 sec latency 22.171 ms 2024-04-24T23:22:10.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1166443 12.57 MB/sec execute 375 sec latency 20.850 ms 2024-04-24T23:22:11.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:11 smithi038 ceph-mon[32450]: pgmap v656: 129 pgs: 129 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-04-24T23:22:11.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:11 smithi117 ceph-mon[32529]: pgmap v656: 129 pgs: 129 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-04-24T23:22:11.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:11 smithi033 ceph-mon[29201]: pgmap v656: 129 pgs: 129 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-04-24T23:22:11.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1169035 12.57 MB/sec execute 376 sec latency 9.803 ms 2024-04-24T23:22:12.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1170963 12.56 MB/sec execute 377 sec latency 193.851 ms 2024-04-24T23:22:13.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:13 smithi038 ceph-mon[32450]: pgmap v657: 129 pgs: 129 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-24T23:22:13.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:13 smithi117 ceph-mon[32529]: pgmap v657: 129 pgs: 129 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-24T23:22:13.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:13 smithi033 ceph-mon[29201]: pgmap v657: 129 pgs: 129 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-24T23:22:13.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1172790 12.56 MB/sec execute 378 sec latency 40.754 ms 2024-04-24T23:22:14.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1175404 12.56 MB/sec execute 379 sec latency 15.187 ms 2024-04-24T23:22:15.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:15 smithi038 ceph-mon[32450]: pgmap v658: 129 pgs: 129 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-04-24T23:22:15.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:15 smithi117 ceph-mon[32529]: pgmap v658: 129 pgs: 129 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-04-24T23:22:15.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:15 smithi033 ceph-mon[29201]: pgmap v658: 129 pgs: 129 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-04-24T23:22:15.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1177067 12.55 MB/sec execute 380 sec latency 20.713 ms 2024-04-24T23:22:16.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1179671 12.56 MB/sec execute 381 sec latency 17.453 ms 2024-04-24T23:22:17.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:17 smithi038 ceph-mon[32450]: pgmap v659: 129 pgs: 129 active+clean; 285 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 251 op/s 2024-04-24T23:22:17.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:17 smithi117 ceph-mon[32529]: pgmap v659: 129 pgs: 129 active+clean; 285 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 251 op/s 2024-04-24T23:22:17.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:17 smithi033 ceph-mon[29201]: pgmap v659: 129 pgs: 129 active+clean; 285 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 251 op/s 2024-04-24T23:22:17.460 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1181968 12.55 MB/sec execute 382 sec latency 153.706 ms 2024-04-24T23:22:18.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1183675 12.55 MB/sec execute 383 sec latency 30.488 ms 2024-04-24T23:22:19.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:19 smithi038 ceph-mon[32450]: pgmap v660: 129 pgs: 129 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T23:22:19.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:19 smithi117 ceph-mon[32529]: pgmap v660: 129 pgs: 129 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T23:22:19.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:19 smithi033 ceph-mon[29201]: pgmap v660: 129 pgs: 129 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T23:22:19.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1186531 12.56 MB/sec execute 384 sec latency 14.171 ms 2024-04-24T23:22:20.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1188733 12.55 MB/sec execute 385 sec latency 15.744 ms 2024-04-24T23:22:21.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:21 smithi038 ceph-mon[32450]: pgmap v661: 129 pgs: 129 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-04-24T23:22:21.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:21 smithi117 ceph-mon[32529]: pgmap v661: 129 pgs: 129 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-04-24T23:22:21.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:21 smithi033 ceph-mon[29201]: pgmap v661: 129 pgs: 129 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-04-24T23:22:21.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1190711 12.55 MB/sec execute 386 sec latency 19.056 ms 2024-04-24T23:22:22.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1193593 12.56 MB/sec execute 387 sec latency 41.303 ms 2024-04-24T23:22:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:23 smithi038 ceph-mon[32450]: pgmap v662: 129 pgs: 129 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 288 op/s 2024-04-24T23:22:23.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:23 smithi117 ceph-mon[32529]: pgmap v662: 129 pgs: 129 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 288 op/s 2024-04-24T23:22:23.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:23 smithi033 ceph-mon[29201]: pgmap v662: 129 pgs: 129 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 288 op/s 2024-04-24T23:22:23.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1194853 12.54 MB/sec execute 388 sec latency 123.342 ms 2024-04-24T23:22:23.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:22:23.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:22:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 -- 172.21.15.33:0/1631697678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8102250 msgr2=0x7f5db8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/1631697678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8102250 0x7f5db8108ae0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009ca0 tx=0x7f5db002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 -- 172.21.15.33:0/1631697678 shutdown_connections 2024-04-24T23:22:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/1631697678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8102250 0x7f5db8108ae0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/1631697678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db8101890 0x7f5db8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/1631697678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db81006a0 0x7f5db8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 -- 172.21.15.33:0/1631697678 >> 172.21.15.33:0/1631697678 conn(0x7f5db80fbc30 msgr2=0x7f5db80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 -- 172.21.15.33:0/1631697678 shutdown_connections 2024-04-24T23:22:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.915+0000 7f5dbe54f640 1 -- 172.21.15.33:0/1631697678 wait complete. 2024-04-24T23:22:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 Processor -- start 2024-04-24T23:22:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 -- start start 2024-04-24T23:22:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 0x7f5db806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 0x7f5db819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8102250 0x7f5db819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810c300 con 0x7f5db8101890 2024-04-24T23:22:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810c180 con 0x7f5db8102250 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbe54f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810c480 con 0x7f5db81006a0 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 0x7f5db819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 0x7f5db806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 0x7f5db819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46600/0 (socket says 172.21.15.33:46600) 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5db77fe640 1 -- 172.21.15.33:0/807449018 learned_addr learned my addr 172.21.15.33:0/807449018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 0x7f5db806a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42460/0 (socket says 172.21.15.33:42460) 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.916+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8102250 0x7f5db819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db7fff640 1 -- 172.21.15.33:0/807449018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8102250 msgr2=0x7f5db819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db7fff640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8102250 0x7f5db819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db7fff640 1 -- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 msgr2=0x7f5db819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db7fff640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 0x7f5db819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db7fff640 1 -- 172.21.15.33:0/807449018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db819f650 con 0x7f5db81006a0 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db77fe640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 0x7f5db819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db7fff640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 0x7f5db806a8d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5da800e9c0 tx=0x7f5da800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db57fa640 1 -- 172.21.15.33:0/807449018 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da800cd80 con 0x7f5db81006a0 2024-04-24T23:22:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db57fa640 1 -- 172.21.15.33:0/807449018 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da8010910 con 0x7f5db81006a0 2024-04-24T23:22:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db819f940 con 0x7f5db81006a0 2024-04-24T23:22:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5db57fa640 1 -- 172.21.15.33:0/807449018 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8018980 con 0x7f5db81006a0 2024-04-24T23:22:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.917+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db819fee0 con 0x7f5db81006a0 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.920+0000 7f5db57fa640 1 -- 172.21.15.33:0/807449018 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5da801f020 con 0x7f5db81006a0 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.920+0000 7f5dc125b740 1 Processor -- start 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.920+0000 7f5dc125b740 1 -- start start 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.920+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 0x55c49ba03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03c30 0x55c49ba08730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba0b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dc125b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c49b9676b0 con 0x55c49ba03c30 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dc125b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c49b90f880 con 0x55c49ba08da0 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dc125b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c49b963670 con 0x55c49ba03240 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 0x55c49ba03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03c30 0x55c49ba08730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba0b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 0x55c49ba03660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42476/0 (socket says 172.21.15.33:42476) 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 -- 172.21.15.33:0/1811269066 learned_addr learned my addr 172.21.15.33:0/1811269066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5db57fa640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5da00a2c10 0x7f5da00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 -- 172.21.15.33:0/1811269066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 msgr2=0x55c49ba0b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba0b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 -- 172.21.15.33:0/1811269066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03c30 msgr2=0x55c49ba08730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03c30 0x55c49ba08730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 -- 172.21.15.33:0/1811269066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49b7457d0 con 0x55c49ba03240 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5db57fa640 1 -- 172.21.15.33:0/807449018 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5da80cf300 con 0x7f5db81006a0 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5da00a2c10 0x7f5da00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.921+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 0x55c49ba03660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009ca0 tx=0x7f5db0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de40016eff5849d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5d9e7fc640 1 -- 172.21.15.33:0/1811269066 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0031d30 con 0x55c49ba03240 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5d9e7fc640 1 -- 172.21.15.33:0/1811269066 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0036040 con 0x55c49ba03240 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5d9e7fc640 1 -- 172.21.15.33:0/1811269066 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0045400 con 0x55c49ba03240 2024-04-24T23:22:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 -- 172.21.15.33:0/1811269066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 msgr2=0x55c49ba03660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5da00a2c10 0x7f5da00a50d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009a60 tx=0x7f5db004b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 0x55c49ba03660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009ca0 tx=0x7f5db0004a40 comp rx=0 tx=0).stop 2024-04-24T23:22:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 -- 172.21.15.33:0/1811269066 shutdown_connections 2024-04-24T23:22:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba0b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03c30 0x55c49ba08730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1811269066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03240 0x55c49ba03660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 -- 172.21.15.33:0/1811269066 >> 172.21.15.33:0/1811269066 conn(0x55c49b9fe8b0 msgr2=0x55c49b9fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 -- 172.21.15.33:0/1811269066 shutdown_connections 2024-04-24T23:22:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.922+0000 7f5dc125b740 1 -- 172.21.15.33:0/1811269066 wait complete. 2024-04-24T23:22:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 -- start start 2024-04-24T23:22:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 Processor -- start 2024-04-24T23:22:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c49ba0bb70 con 0x55c49ba03240 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c49b922f50 con 0x55c49ba08da0 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c49ba0bcf0 con 0x55c49ba03c30 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dc125b740 10 client.0 _ll_register_callbacks cb 0x55c49b95e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba0d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42482/0 (socket says 172.21.15.33:42482) 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5db77fe640 1 -- 172.21.15.33:0/1376051016 learned_addr learned my addr 172.21.15.33:0/1376051016 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.923+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db77fe640 1 -- 172.21.15.33:0/1376051016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 msgr2=0x55c49ba14a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db77fe640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba14a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db77fe640 1 -- 172.21.15.33:0/1376051016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 msgr2=0x55c49ba0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db77fe640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db77fe640 1 -- 172.21.15.33:0/1376051016 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49b67b3f0 con 0x55c49ba03c30 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db7fff640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5db77fe640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba0d9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5dac012d20 tx=0x7f5dac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b939ce881f7b93d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5d977fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5d977fe640 1 -- 172.21.15.33:0/1376051016 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac012110 con 0x55c49ba03c30 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5d977fe640 1 -- 172.21.15.33:0/1376051016 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5dac004050 con 0x55c49ba03c30 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db0036040 con 0x55c49ba03c30 2024-04-24T23:22:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5d977fe640 1 -- 172.21.15.33:0/1376051016 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5dac025050 con 0x55c49ba03c30 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5dc125b740 10 client.27055 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c49b63b0f0 con 0x55c49ba03c30 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.924+0000 7f5d977fe640 1 -- 172.21.15.33:0/1376051016 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0036040 con 0x55c49ba03c30 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5d977fe640 1 -- 172.21.15.33:0/1376051016 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5dac010430 con 0x55c49ba03c30 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5dc125b740 10 client.27055 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5dc125b740 10 client.27055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5dc125b740 20 client.27055 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba19330 0x55c49ba1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5dc125b740 4 client.27055 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c49b974bf0 con 0x55c49ba19330 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5db7fff640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba19330 0x55c49ba1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5db7fff640 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba19330 0x55c49ba1b7f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b778ddc0e26ca62d server_cookie=ddba7b79851a518c in_seq=0 out_seq=0 2024-04-24T23:22:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.925+0000 7f5d977fe640 10 client.27055 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5d977fe640 1 -- 172.21.15.33:0/1376051016 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c49b974bf0 con 0x55c49ba19330 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5d977fe640 10 client.27055 handle_command_reply: tid=0 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 client.27055 shutdown 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 msgr2=0x55c49ba0d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba0d9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5dac012d20 tx=0x7f5dac00f790 comp rx=0 tx=0).stop 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 shutdown_connections 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba19330 0x55c49ba1b7f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5d977fe640 0 client.27055 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba0d9d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1376051016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 >> 172.21.15.33:0/1376051016 conn(0x55c49b9fe8b0 msgr2=0x55c49b9fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 shutdown_connections 2024-04-24T23:22:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 1 -- 172.21.15.33:0/1376051016 wait complete. 2024-04-24T23:22:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.926+0000 7f5dc125b740 20 client.27055 trim_cache size 0 max 16384 2024-04-24T23:22:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 Processor -- start 2024-04-24T23:22:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 -- start start 2024-04-24T23:22:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba03c30 0x55c49ba149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba08da0 0x55c49ba01630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81acc00 con 0x55c49ba03240 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81aa4d0 con 0x55c49ba03c30 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.947+0000 7f5dc125b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81a8320 con 0x55c49ba08da0 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46630/0 (socket says 172.21.15.33:46630) 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 -- 172.21.15.33:0/1552648533 learned_addr learned my addr 172.21.15.33:0/1552648533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba08da0 0x55c49ba01630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db77fe640 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba03c30 0x55c49ba149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 -- 172.21.15.33:0/1552648533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba08da0 msgr2=0x55c49ba01630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba08da0 0x55c49ba01630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 -- 172.21.15.33:0/1552648533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba03c30 msgr2=0x55c49ba149d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba03c30 0x55c49ba149d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 -- 172.21.15.33:0/1552648533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49b9fe450 con 0x55c49ba03240 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba08da0 0x55c49ba01630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5db7fff640 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5da8097c30 tx=0x7f5da80d2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557e7803edfa1b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5d9dffb640 1 -- 172.21.15.33:0/1552648533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da809b310 con 0x55c49ba03240 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5d9dffb640 1 -- 172.21.15.33:0/1552648533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da809bed0 con 0x55c49ba03240 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.948+0000 7f5d9dffb640 1 -- 172.21.15.33:0/1552648533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da809b130 con 0x55c49ba03240 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 -- 172.21.15.33:0/1552648533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 msgr2=0x55c49ba0d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5da8097c30 tx=0x7f5da80d2de0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 -- 172.21.15.33:0/1552648533 shutdown_connections 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba08da0 0x55c49ba01630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba03c30 0x55c49ba149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 --2- 172.21.15.33:0/1552648533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba0d490 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 -- 172.21.15.33:0/1552648533 >> 172.21.15.33:0/1552648533 conn(0x55c49b9fe8b0 msgr2=0x55c49ba0c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 -- 172.21.15.33:0/1552648533 shutdown_connections 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 -- 172.21.15.33:0/1552648533 wait complete. 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.949+0000 7f5dc125b740 1 -- start start 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 Processor -- start 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba11d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81a8990 con 0x55c49ba03240 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81a9e00 con 0x55c49ba08da0 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81a9060 con 0x55c49ba03c30 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dc125b740 10 client.0 _ll_register_callbacks cb 0x7f5dac009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba11d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba11d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46642/0 (socket says 172.21.15.33:46642) 2024-04-24T23:22:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db7fff640 1 -- 172.21.15.33:0/3744939676 learned_addr learned my addr 172.21.15.33:0/3744939676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5dbcaec640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 -- 172.21.15.33:0/3744939676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 msgr2=0x55c49ba132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 -- 172.21.15.33:0/3744939676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 msgr2=0x55c49ba11d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba11d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 -- 172.21.15.33:0/3744939676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49ba138d0 con 0x55c49ba03c30 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db7fff640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba11d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5db77fe640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba122d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5dac012cf0 tx=0x7f5dac012190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93dc8534961bfe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.950+0000 7f5d977fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5d977fe640 1 -- 172.21.15.33:0/3744939676 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac022040 con 0x55c49ba03c30 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da809bed0 con 0x55c49ba03c30 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5d977fe640 1 -- 172.21.15.33:0/3744939676 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5dac010ac0 con 0x55c49ba03c30 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5d977fe640 1 -- 172.21.15.33:0/3744939676 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5dac010640 con 0x55c49ba03c30 2024-04-24T23:22:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 10 client.27061 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c49ba14c50 con 0x55c49ba03c30 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5d977fe640 1 -- 172.21.15.33:0/3744939676 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da809bed0 con 0x55c49ba03c30 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5d977fe640 1 -- 172.21.15.33:0/3744939676 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5dac0039d0 con 0x55c49ba03c30 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 10 client.27061 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 10 client.27061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 20 client.27061 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba7ffd0 0x55c49ba823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 4 client.27061 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:22:23.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.951+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c49b96ef40 con 0x55c49ba7ffd0 2024-04-24T23:22:23.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.952+0000 7f5db7fff640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba7ffd0 0x55c49ba823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.952+0000 7f5d977fe640 10 client.27061 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.952+0000 7f5db7fff640 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba7ffd0 0x55c49ba823c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47df1efd735bfe19 server_cookie=f0bf3c38731c1aea in_seq=0 out_seq=0 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.952+0000 7f5d977fe640 1 -- 172.21.15.33:0/3744939676 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c49b96ef40 con 0x55c49ba7ffd0 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.952+0000 7f5d977fe640 10 client.27061 handle_command_reply: tid=0 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.952+0000 7f5dc125b740 1 client.27061 shutdown 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 msgr2=0x55c49ba122d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba122d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5dac012cf0 tx=0x7f5dac012190 comp rx=0 tx=0).stop 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 shutdown_connections 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49ba7ffd0 0x55c49ba823c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c49ba08da0 0x55c49ba132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49ba03c30 0x55c49ba122d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 --2- 172.21.15.33:0/3744939676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49ba03240 0x55c49ba11d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 >> 172.21.15.33:0/3744939676 conn(0x55c49b9fe8b0 msgr2=0x55c49b9fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5d977fe640 0 client.27061 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 shutdown_connections 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 1 -- 172.21.15.33:0/3744939676 wait complete. 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.953+0000 7f5dc125b740 20 client.27061 trim_cache size 0 max 16384 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:22:23.961 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:22:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5da00a2c10 msgr2=0x7f5da00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5da00a2c10 0x7f5da00a50d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009a60 tx=0x7f5db004b000 comp rx=0 tx=0).stop 2024-04-24T23:22:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 msgr2=0x7f5db806a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 0x7f5db806a8d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5da800e9c0 tx=0x7f5da800ee80 comp rx=0 tx=0).stop 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 shutdown_connections 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5da00a2c10 0x7f5da00a50d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8102250 0x7f5db819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8101890 0x7f5db819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 --2- 172.21.15.33:0/807449018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db81006a0 0x7f5db806a8d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 >> 172.21.15.33:0/807449018 conn(0x7f5db80fbc30 msgr2=0x7f5db8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 shutdown_connections 2024-04-24T23:22:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:23.954+0000 7f5dbe54f640 1 -- 172.21.15.33:0/807449018 wait complete. 2024-04-24T23:22:23.967 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:22:23.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.075+0000 7f8bc93b0640 1 -- 172.21.15.33:0/1398331699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc41017b0 msgr2=0x7f8bc4101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.075+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/1398331699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc41017b0 0x7f8bc4101bd0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009900 tx=0x7f8bb402a3f0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.075+0000 7f8bc93b0640 1 -- 172.21.15.33:0/1398331699 shutdown_connections 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.075+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/1398331699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4103360 0x7f8bc4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.075+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/1398331699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41029a0 0x7f8bc4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.075+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/1398331699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc41017b0 0x7f8bc4101bd0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- 172.21.15.33:0/1398331699 >> 172.21.15.33:0/1398331699 conn(0x7f8bc40fcd20 msgr2=0x7f8bc40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- 172.21.15.33:0/1398331699 shutdown_connections 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- 172.21.15.33:0/1398331699 wait complete. 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 Processor -- start 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- start start 2024-04-24T23:22:24.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 0x7f8bc40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc41029a0 0x7f8bc4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4103360 0x7f8bc4074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410d400 con 0x7f8bc4103360 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410d280 con 0x7f8bc41017b0 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.076+0000 7f8bc93b0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410d580 con 0x7f8bc41029a0 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 0x7f8bc40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4103360 0x7f8bc4074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 0x7f8bc40754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60050/0 (socket says 172.21.15.33:60050) 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 -- 172.21.15.33:0/3632698929 learned_addr learned my addr 172.21.15.33:0/3632698929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bbbfff640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc41029a0 0x7f8bc4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 -- 172.21.15.33:0/3632698929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc41029a0 msgr2=0x7f8bc4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc41029a0 0x7f8bc4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 -- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4103360 msgr2=0x7f8bc4074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4103360 0x7f8bc4074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 -- 172.21.15.33:0/3632698929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc4074930 con 0x7f8bc41017b0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bbbfff640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc41029a0 0x7f8bc4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc37fe640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4103360 0x7f8bc4074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc2ffd640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 0x7f8bc40754a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4031610 tx=0x7f8bb4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb403f030 con 0x7f8bc41017b0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4003ad0 con 0x7f8bc41017b0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc41a1bc0 con 0x7f8bc41017b0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4035070 con 0x7f8bc41017b0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.077+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc41a20d0 con 0x7f8bc41017b0 2024-04-24T23:22:24.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.078+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc4108ba0 con 0x7f8bc41017b0 2024-04-24T23:22:24.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.079+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8bb4042030 con 0x7f8bc41017b0 2024-04-24T23:22:24.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.080+0000 7f8bc0ff9640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bb00a2c10 0x7f8bb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.080+0000 7f8bbbfff640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bb00a2c10 0x7f8bb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.080+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bb40efcf0 con 0x7f8bc41017b0 2024-04-24T23:22:24.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.081+0000 7f8bbbfff640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bb00a2c10 0x7f8bb00a50d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4075210 tx=0x7f8bac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.084+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8bb40bc860 con 0x7f8bc41017b0 2024-04-24T23:22:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.319+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bc402d3c0 con 0x7f8bc41017b0 2024-04-24T23:22:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.319+0000 7f8bc0ff9640 1 -- 172.21.15.33:0/3632698929 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f8bb40bc160 con 0x7f8bc41017b0 2024-04-24T23:22:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:22:24.322 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:22:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bb00a2c10 msgr2=0x7f8bb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bb00a2c10 0x7f8bb00a50d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4075210 tx=0x7f8bac005f90 comp rx=0 tx=0).stop 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 msgr2=0x7f8bc40754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 0x7f8bc40754a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4031610 tx=0x7f8bb4004270 comp rx=0 tx=0).stop 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 shutdown_connections 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bb00a2c10 0x7f8bb00a50d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bc4103360 0x7f8bc4074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc41029a0 0x7f8bc4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 --2- 172.21.15.33:0/3632698929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc41017b0 0x7f8bc40754a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 >> 172.21.15.33:0/3632698929 conn(0x7f8bc40fcd20 msgr2=0x7f8bc4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 shutdown_connections 2024-04-24T23:22:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.322+0000 7f8bc93b0640 1 -- 172.21.15.33:0/3632698929 wait complete. 2024-04-24T23:22:24.335 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:22:24.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1197205 12.55 MB/sec execute 389 sec latency 137.827 ms 2024-04-24T23:22:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.460+0000 7fa509970640 1 -- 172.21.15.33:0/1838705125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504100680 msgr2=0x7fa504100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.460+0000 7fa509970640 1 --2- 172.21.15.33:0/1838705125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504100680 0x7fa504100aa0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009ca0 tx=0x7fa4f402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 -- 172.21.15.33:0/1838705125 shutdown_connections 2024-04-24T23:22:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 --2- 172.21.15.33:0/1838705125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa504102230 0x7fa504108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 --2- 172.21.15.33:0/1838705125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 --2- 172.21.15.33:0/1838705125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504100680 0x7fa504100aa0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 -- 172.21.15.33:0/1838705125 >> 172.21.15.33:0/1838705125 conn(0x7fa5040fbbf0 msgr2=0x7fa5040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 -- 172.21.15.33:0/1838705125 shutdown_connections 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 -- 172.21.15.33:0/1838705125 wait complete. 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 Processor -- start 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 -- start start 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa504100680 0x7fa504197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504102230 0x7fa50419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.461+0000 7fa509970640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410c300 con 0x7fa504102230 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa509970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410c180 con 0x7fa504100680 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa509970640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410c480 con 0x7fa504101870 2024-04-24T23:22:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa504100680 0x7fa504197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504102230 0x7fa50419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42544/0 (socket says 172.21.15.33:42544) 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 -- 172.21.15.33:0/3576083958 learned_addr learned my addr 172.21.15.33:0/3576083958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 -- 172.21.15.33:0/3576083958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa504100680 msgr2=0x7fa504197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa504100680 0x7fa504197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 -- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504102230 msgr2=0x7fa50419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504102230 0x7fa50419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 -- 172.21.15.33:0/3576083958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50419f720 con 0x7fa504101870 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504198040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800e9f0 tx=0x7fa4f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa50896e640 1 -- 172.21.15.33:0/3576083958 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800cd80 con 0x7fa504101870 2024-04-24T23:22:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50419fa10 con 0x7fa504101870 2024-04-24T23:22:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa50896e640 1 -- 172.21.15.33:0/3576083958 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8005500 con 0x7fa504101870 2024-04-24T23:22:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa50896e640 1 -- 172.21.15.33:0/3576083958 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8010690 con 0x7fa504101870 2024-04-24T23:22:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.462+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50419ff60 con 0x7fa504101870 2024-04-24T23:22:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50896e640 1 -- 172.21.15.33:0/3576083958 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa4f80046c0 con 0x7fa504101870 2024-04-24T23:22:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 Processor -- start 2024-04-24T23:22:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 -- start start 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4d02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d07720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d07d90 0x5568b4d0a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5568b4c66670 con 0x5568b4d07d90 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5568b4c0e970 con 0x5568b4d02c20 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50c322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568b4c62630 con 0x5568b4d02230 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa50896e640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4ec0a2b90 0x7fa4ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4d02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa5027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d07d90 0x5568b4d0a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4d02650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42554/0 (socket says 172.21.15.33:42554) 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa5037fe640 1 -- 172.21.15.33:0/1370250218 learned_addr learned my addr 172.21.15.33:0/1370250218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4ec0a2b90 0x7fa4ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.465+0000 7fa502ffd640 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d07720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa50896e640 1 -- 172.21.15.33:0/3576083958 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4f80cf7b0 con 0x7fa504101870 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 -- 172.21.15.33:0/1370250218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 msgr2=0x5568b4d07720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d07720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 -- 172.21.15.33:0/1370250218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d07d90 msgr2=0x5568b4d0a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d07d90 0x5568b4d0a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 -- 172.21.15.33:0/1370250218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b4c32400 con 0x5568b4d02230 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4d02650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0178e0 tx=0x7fa4fc017da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842a7fbb73a831d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4ec0a2b90 0x7fa4ec0a5050 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc011810 tx=0x7fa4fc013000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa4e9ffb640 1 -- 172.21.15.33:0/1370250218 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc009a90 con 0x5568b4d02230 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa4e9ffb640 1 -- 172.21.15.33:0/1370250218 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc032070 con 0x5568b4d02230 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.466+0000 7fa4e9ffb640 1 -- 172.21.15.33:0/1370250218 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00e620 con 0x5568b4d02230 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 -- 172.21.15.33:0/1370250218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 msgr2=0x5568b4d02650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4d02650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0178e0 tx=0x7fa4fc017da0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 -- 172.21.15.33:0/1370250218 shutdown_connections 2024-04-24T23:22:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d07d90 0x5568b4d0a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d07720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 --2- 172.21.15.33:0/1370250218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4d02650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 -- 172.21.15.33:0/1370250218 >> 172.21.15.33:0/1370250218 conn(0x5568b4cfd8a0 msgr2=0x5568b4cfdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 -- 172.21.15.33:0/1370250218 shutdown_connections 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 -- 172.21.15.33:0/1370250218 wait complete. 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 -- start start 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.467+0000 7fa50c322740 1 Processor -- start 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5568b4d0ace0 con 0x5568b4d02230 2024-04-24T23:22:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5568b4c22050 con 0x5568b4d02c20 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d0c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46714/0 (socket says 172.21.15.33:46714) 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa502ffd640 1 -- 172.21.15.33:0/3659525171 learned_addr learned my addr 172.21.15.33:0/3659525171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568b4d0ae60 con 0x5568b4d07d90 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa50c322740 10 client.0 _ll_register_callbacks cb 0x5568b4979be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5037fe640 1 -- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 msgr2=0x5568b4d0c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d0c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5037fe640 1 -- 172.21.15.33:0/3659525171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 msgr2=0x5568b4d0c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d0c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.468+0000 7fa5037fe640 1 -- 172.21.15.33:0/3659525171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b4bf3dc0 con 0x5568b4d07d90 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d138c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5568b4d03480 tx=0x7fa4fc005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3c82adac5e43de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa4e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:22:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3659525171 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc009a90 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3659525171 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4fc0050e0 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4fc032070 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3659525171 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc043050 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa50c322740 10 client.27079 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568b497a3f0 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.469+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3659525171 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc032070 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3659525171 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4fc0039c0 con 0x5568b4d07d90 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa50c322740 10 client.27079 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa50c322740 10 client.27079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa50c322740 20 client.27079 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa50c322740 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d17f60 0x5568b4d1a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa50c322740 4 client.27079 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b4c73bb0 con 0x5568b4d17f60 2024-04-24T23:22:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa502ffd640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d17f60 0x5568b4d1a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa4e2ffd640 10 client.27079 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.470+0000 7fa502ffd640 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d17f60 0x5568b4d1a3b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=418eb6aafabb9846 server_cookie=1e4b50d7cc24472a in_seq=0 out_seq=0 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3659525171 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568b4c73bb0 con 0x5568b4d17f60 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa4e2ffd640 10 client.27079 handle_command_reply: tid=0 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 client.27079 shutdown 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 msgr2=0x5568b4d138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d138c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5568b4d03480 tx=0x7fa4fc005fb0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 shutdown_connections 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d17f60 0x5568b4d1a3b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d138c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa4e2ffd640 0 client.27079 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 --2- 172.21.15.33:0/3659525171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 >> 172.21.15.33:0/3659525171 conn(0x5568b4cfd8a0 msgr2=0x5568b4cfcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 shutdown_connections 2024-04-24T23:22:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 1 -- 172.21.15.33:0/3659525171 wait complete. 2024-04-24T23:22:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.471+0000 7fa50c322740 20 client.27079 trim_cache size 0 max 16384 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 Processor -- start 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 -- start start 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4cfc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 0x5568b4d13880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d07d90 0x5568b4d03620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041aaab0 con 0x5568b4d02c20 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041a8380 con 0x5568b4d07d90 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa50c322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041a61d0 con 0x5568b4d02230 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4cfc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa5027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 0x5568b4d13880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4cfc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42582/0 (socket says 172.21.15.33:42582) 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa502ffd640 1 -- 172.21.15.33:0/1245989845 learned_addr learned my addr 172.21.15.33:0/1245989845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa5027fc640 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 0x5568b4d13880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46728/0 (socket says 172.21.15.33:46728) 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.485+0000 7fa5037fe640 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d07d90 0x5568b4d03620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa502ffd640 1 -- 172.21.15.33:0/1245989845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d07d90 msgr2=0x5568b4d03620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa502ffd640 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d07d90 0x5568b4d03620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa502ffd640 1 -- 172.21.15.33:0/1245989845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 msgr2=0x5568b4d13880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa502ffd640 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 0x5568b4d13880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa502ffd640 1 -- 172.21.15.33:0/1245989845 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b4a447d0 con 0x5568b4d02230 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa5027fc640 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 0x5568b4d13880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa502ffd640 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4cfc2b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4031ae0 tx=0x7fa4f4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8176c3d7b28e3322 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa4e97fa640 1 -- 172.21.15.33:0/1245989845 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4050020 con 0x5568b4d02230 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa4e97fa640 1 -- 172.21.15.33:0/1245989845 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4036040 con 0x5568b4d02230 2024-04-24T23:22:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.486+0000 7fa4e97fa640 1 -- 172.21.15.33:0/1245989845 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f404c940 con 0x5568b4d02230 2024-04-24T23:22:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 -- 172.21.15.33:0/1245989845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 msgr2=0x5568b4cfc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4cfc2b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4031ae0 tx=0x7fa4f4000f40 comp rx=0 tx=0).stop 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 -- 172.21.15.33:0/1245989845 shutdown_connections 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d07d90 0x5568b4d03620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02c20 0x5568b4d13880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 --2- 172.21.15.33:0/1245989845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d02230 0x5568b4cfc2b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 -- 172.21.15.33:0/1245989845 >> 172.21.15.33:0/1245989845 conn(0x5568b4cfd8a0 msgr2=0x5568b4d0ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 -- 172.21.15.33:0/1245989845 shutdown_connections 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 -- 172.21.15.33:0/1245989845 wait complete. 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 -- start start 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 Processor -- start 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.487+0000 7fa50c322740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d10d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa50c322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d7c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa50c322740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041a6840 con 0x5568b4d02230 2024-04-24T23:22:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa50c322740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041a7cb0 con 0x5568b4d02c20 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa50c322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041a6f10 con 0x5568b4d07d90 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa50c322740 10 client.0 _ll_register_callbacks cb 0x7fa4f8004ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa502ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d107e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46736/0 (socket says 172.21.15.33:46736) 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa502ffd640 1 -- 172.21.15.33:0/3231829535 learned_addr learned my addr 172.21.15.33:0/3231829535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5037fe640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d7c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 -- 172.21.15.33:0/3231829535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 msgr2=0x5568b4d7c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d7c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 -- 172.21.15.33:0/3231829535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 msgr2=0x5568b4d107e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d107e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 -- 172.21.15.33:0/3231829535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b4d7ca50 con 0x5568b4d02c20 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa502ffd640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa5027fc640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d10d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80ccf70 tx=0x7fa4f806acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50d589e0a3834c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa4e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3231829535 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80a4660 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3231829535 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4f80cca10 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.488+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f4036040 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3231829535 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4f809c380 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 10 client.27119 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568b4d7de20 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3231829535 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4036040 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3231829535 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4f809b0d0 con 0x5568b4d02c20 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 10 client.27119 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 10 client.27119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 20 client.27119 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d7e7f0 0x5568b4d80c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 4 client.27119 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:22:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568b4c6df00 con 0x5568b4d7e7f0 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa502ffd640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d7e7f0 0x5568b4d80c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa502ffd640 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d7e7f0 0x5568b4d80c10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f0c6ac6743c2b08 server_cookie=49e18a73c50dba9 in_seq=0 out_seq=0 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.489+0000 7fa4e2ffd640 10 client.27119 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa4e2ffd640 1 -- 172.21.15.33:0/3231829535 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568b4c6df00 con 0x5568b4d7e7f0 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa4e2ffd640 10 client.27119 handle_command_reply: tid=0 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 client.27119 shutdown 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 msgr2=0x5568b4d10d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d10d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80ccf70 tx=0x7fa4f806acc0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 shutdown_connections 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568b4d7e7f0 0x5568b4d80c10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568b4d07d90 0x5568b4d7c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa4e2ffd640 0 client.27119 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568b4d02c20 0x5568b4d10d20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 --2- 172.21.15.33:0/3231829535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568b4d02230 0x5568b4d107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 >> 172.21.15.33:0/3231829535 conn(0x5568b4cfd8a0 msgr2=0x5568b4d00580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 shutdown_connections 2024-04-24T23:22:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 1 -- 172.21.15.33:0/3231829535 wait complete. 2024-04-24T23:22:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa50c322740 20 client.27119 trim_cache size 0 max 16384 2024-04-24T23:22:24.499 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4ec0a2b90 msgr2=0x7fa4ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa509970640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4ec0a2b90 0x7fa4ec0a5050 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc011810 tx=0x7fa4fc013000 comp rx=0 tx=0).stop 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 msgr2=0x7fa504198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.490+0000 7fa509970640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504198040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800e9f0 tx=0x7fa4f800edf0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 shutdown_connections 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4ec0a2b90 0x7fa4ec0a5050 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa504102230 0x7fa50419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504101870 0x7fa504198040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 --2- 172.21.15.33:0/3576083958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa504100680 0x7fa504197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 >> 172.21.15.33:0/3576083958 conn(0x7fa5040fbbf0 msgr2=0x7fa5040fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 shutdown_connections 2024-04-24T23:22:24.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.491+0000 7fa509970640 1 -- 172.21.15.33:0/3576083958 wait complete. 2024-04-24T23:22:24.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:22:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 -- 172.21.15.33:0/2073560597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0107460 msgr2=0x7fcac01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 --2- 172.21.15.33:0/2073560597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0107460 0x7fcac01098b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009ca0 tx=0x7fcab802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 -- 172.21.15.33:0/2073560597 shutdown_connections 2024-04-24T23:22:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 --2- 172.21.15.33:0/2073560597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0107460 0x7fcac01098b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 --2- 172.21.15.33:0/2073560597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0104b30 0x7fcac0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 --2- 172.21.15.33:0/2073560597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac0102200 0x7fcac01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 -- 172.21.15.33:0/2073560597 >> 172.21.15.33:0/2073560597 conn(0x7fcac00fbbf0 msgr2=0x7fcac00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 -- 172.21.15.33:0/2073560597 shutdown_connections 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.614+0000 7fcac8570640 1 -- 172.21.15.33:0/2073560597 wait complete. 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 Processor -- start 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 -- start start 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 0x7fcac0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac0104b30 0x7fcac0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0107460 0x7fcac019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010d420 con 0x7fcac0102200 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010d2a0 con 0x7fcac0104b30 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac8570640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010d5a0 con 0x7fcac0107460 2024-04-24T23:22:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac630c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 0x7fcac0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac630c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 0x7fcac0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46754/0 (socket says 172.21.15.33:46754) 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac630c640 1 -- 172.21.15.33:0/3549856230 learned_addr learned my addr 172.21.15.33:0/3549856230 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0107460 0x7fcac019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.615+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac0104b30 0x7fcac0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3549856230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac0104b30 msgr2=0x7fcac0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac0104b30 0x7fcac0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 msgr2=0x7fcac0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 0x7fcac0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3549856230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019f6e0 con 0x7fcac0107460 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac630c640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 0x7fcac0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0107460 0x7fcac019f070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcab8031ae0 tx=0x7fcab8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcab77fe640 1 -- 172.21.15.33:0/3549856230 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8003b70 con 0x7fcac0107460 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcab77fe640 1 -- 172.21.15.33:0/3549856230 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8003d10 con 0x7fcac0107460 2024-04-24T23:22:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcab77fe640 1 -- 172.21.15.33:0/3549856230 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab802fc40 con 0x7fcac0107460 2024-04-24T23:22:24.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019f970 con 0x7fcac0107460 2024-04-24T23:22:24.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.616+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac019fe90 con 0x7fcac0107460 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcab77fe640 1 -- 172.21.15.33:0/3549856230 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcab8004050 con 0x7fcac0107460 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 Processor -- start 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 -- start start 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae484a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae489b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae48c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558bae3e4670 con 0x558bae485030 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558bae38c970 con 0x558bae48a1a0 2024-04-24T23:22:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcacb280740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558bae3e0630 con 0x558bae484640 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae489b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae489b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46756/0 (socket says 172.21.15.33:46756) 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 -- 172.21.15.33:0/954982432 learned_addr learned my addr 172.21.15.33:0/954982432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae484a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae48c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 -- 172.21.15.33:0/954982432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 msgr2=0x558bae484a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae484a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 -- 172.21.15.33:0/954982432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 msgr2=0x558bae48c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.619+0000 7fcac630c640 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae48c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcac630c640 1 -- 172.21.15.33:0/954982432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bae3b0400 con 0x558bae485030 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae48c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcab77fe640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcaac0a2b90 0x7fcaac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcac630c640 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae489b30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcabc011650 tx=0x7fcabc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed59740e6242bd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcaac0a2b90 0x7fcaac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcab77fe640 1 -- 172.21.15.33:0/3549856230 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcab80f0330 con 0x7fcac0107460 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcab4bfe640 1 -- 172.21.15.33:0/954982432 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc01e070 con 0x558bae485030 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcab4bfe640 1 -- 172.21.15.33:0/954982432 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc00fba0 con 0x558bae485030 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.620+0000 7fcab4bfe640 1 -- 172.21.15.33:0/954982432 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc019d80 con 0x558bae485030 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- 172.21.15.33:0/954982432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 msgr2=0x558bae489b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcaac0a2b90 0x7fcaac0a5050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x558bae485890 tx=0x7fcab0013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae489b30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcabc011650 tx=0x7fcabc009f90 comp rx=0 tx=0).stop 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- 172.21.15.33:0/954982432 shutdown_connections 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae48c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae489b30 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- 172.21.15.33:0/954982432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae484a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- 172.21.15.33:0/954982432 >> 172.21.15.33:0/954982432 conn(0x558bae47b8a0 msgr2=0x558bae47bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- 172.21.15.33:0/954982432 shutdown_connections 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- 172.21.15.33:0/954982432 wait complete. 2024-04-24T23:22:24.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- start start 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 Processor -- start 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae48ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 0x558bae495d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558bae48d150 con 0x558bae48a1a0 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558bae3a0050 con 0x558bae484640 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558bae48d2d0 con 0x558bae485030 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.621+0000 7fcacb280740 10 client.0 _ll_register_callbacks cb 0x558bae0f7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 0x558bae495d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 0x558bae495d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46762/0 (socket says 172.21.15.33:46762) 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3100815633 learned_addr learned my addr 172.21.15.33:0/3100815633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae48ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac630c640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 msgr2=0x558bae48ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae48ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3100815633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae484640 msgr2=0x558bae48e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 -- 172.21.15.33:0/3100815633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bae371dc0 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae48ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 0x558bae495d40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcab8031f80 tx=0x7fcab80bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f48635f7623bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcaa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/3100815633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80c0c50 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/3100815633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcab802fde0 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc00fba0 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/3100815633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcab80c5510 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcacb280740 10 client.17604 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.622+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bae0f83f0 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/3100815633 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc00fba0 con 0x558bae48a1a0 2024-04-24T23:22:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/3100815633 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcab80bcb00 con 0x558bae48a1a0 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcacb280740 10 client.17604 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcacb280740 10 client.17604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcacb280740 20 client.17604 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcacb280740 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae49a5f0 0x558bae49ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcacb280740 4 client.17604 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bae3f1bb0 con 0x558bae49a5f0 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcac630c640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae49a5f0 0x558bae49ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcac630c640 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae49a5f0 0x558bae49ca50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2daae60da7573d0 server_cookie=62421e39bc95d627 in_seq=0 out_seq=0 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.623+0000 7fcaa9ffb640 10 client.17604 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/3100815633 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bae3f1bb0 con 0x558bae49a5f0 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcaa9ffb640 10 client.17604 handle_command_reply: tid=0 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 client.17604 shutdown 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 msgr2=0x558bae495d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 0x558bae495d40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcab8031f80 tx=0x7fcab80bc200 comp rx=0 tx=0).stop 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 shutdown_connections 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae49a5f0 0x558bae49ca50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae48a1a0 0x558bae495d40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae48ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 --2- 172.21.15.33:0/3100815633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 >> 172.21.15.33:0/3100815633 conn(0x558bae47b8a0 msgr2=0x558bae47ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcaa9ffb640 0 client.17604 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 shutdown_connections 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 1 -- 172.21.15.33:0/3100815633 wait complete. 2024-04-24T23:22:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.624+0000 7fcacb280740 20 client.17604 trim_cache size 0 max 16384 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 Processor -- start 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 -- start start 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae495d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae47e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01aa9e0 con 0x558bae484640 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a82b0 con 0x558bae48a1a0 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.639+0000 7fcacb280740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a6100 con 0x558bae485030 2024-04-24T23:22:24.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46770/0 (socket says 172.21.15.33:46770) 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 -- 172.21.15.33:0/372684929 learned_addr learned my addr 172.21.15.33:0/372684929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae47e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae495d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 -- 172.21.15.33:0/372684929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 msgr2=0x558bae495d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae495d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 -- 172.21.15.33:0/372684929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 msgr2=0x558bae47e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae47e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 -- 172.21.15.33:0/372684929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bae1c27d0 con 0x558bae484640 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcac630c640 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 0x558bae48e7c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009f60 tx=0x7fcabc0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fea3c7996a95240 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcaabfff640 1 -- 172.21.15.33:0/372684929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc02f070 con 0x558bae484640 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcaabfff640 1 -- 172.21.15.33:0/372684929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc02a790 con 0x558bae484640 2024-04-24T23:22:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.640+0000 7fcaabfff640 1 -- 172.21.15.33:0/372684929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc01e040 con 0x558bae484640 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- 172.21.15.33:0/372684929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 msgr2=0x558bae48e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 0x558bae48e7c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009f60 tx=0x7fcabc0051d0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- 172.21.15.33:0/372684929 shutdown_connections 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae47e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae485030 0x558bae495d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- 172.21.15.33:0/372684929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae484640 0x558bae48e7c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- 172.21.15.33:0/372684929 >> 172.21.15.33:0/372684929 conn(0x558bae47b8a0 msgr2=0x558bae47ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- 172.21.15.33:0/372684929 shutdown_connections 2024-04-24T23:22:24.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- 172.21.15.33:0/372684929 wait complete. 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- start start 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 Processor -- start 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae4933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae493920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae494930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a6770 con 0x558bae485030 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a7be0 con 0x558bae48a1a0 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a6e40 con 0x558bae484640 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.641+0000 7fcacb280740 10 client.0 _ll_register_callbacks cb 0x7fcab8009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae493920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae493920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46786/0 (socket says 172.21.15.33:46786) 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 -- 172.21.15.33:0/4259065341 learned_addr learned my addr 172.21.15.33:0/4259065341 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac6b0d640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae494930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac630c640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae4933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 -- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 msgr2=0x558bae4933e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae4933e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 -- 172.21.15.33:0/4259065341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 msgr2=0x558bae494930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae494930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 -- 172.21.15.33:0/4259065341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bae494f20 con 0x558bae485030 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcac5b0b640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae493920 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcab00096a0 tx=0x7fcab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e42788923249b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcaa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/4259065341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004320 con 0x558bae485030 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/4259065341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcab001a070 con 0x558bae485030 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc02a790 con 0x558bae485030 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/4259065341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcab0036050 con 0x558bae485030 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcacb280740 10 client.17610 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:24.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.642+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bae500c70 con 0x558bae485030 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/4259065341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc02a790 con 0x558bae485030 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/4259065341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcab001f610 con 0x558bae485030 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcacb280740 10 client.17610 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcacb280740 10 client.17610 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcacb280740 20 client.17610 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcacb280740 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae501720 0x558bae503b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcacb280740 4 client.17610 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bae3ebf00 con 0x558bae501720 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcac630c640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae501720 0x558bae503b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcac630c640 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae501720 0x558bae503b40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee443329c5e8cd08 server_cookie=5d7e86bd458751c7 in_seq=0 out_seq=0 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.643+0000 7fcaa9ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcaa9ffb640 1 -- 172.21.15.33:0/4259065341 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bae3ebf00 con 0x558bae501720 2024-04-24T23:22:24.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcaa9ffb640 10 client.17610 handle_command_reply: tid=0 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 client.17610 shutdown 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 msgr2=0x558bae493920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae493920 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcab00096a0 tx=0x7fcab00046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 shutdown_connections 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558bae501720 0x558bae503b40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558bae48a1a0 0x558bae494930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558bae485030 0x558bae493920 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 --2- 172.21.15.33:0/4259065341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bae484640 0x558bae4933e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcaa9ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 >> 172.21.15.33:0/4259065341 conn(0x558bae47b8a0 msgr2=0x558bae485ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 shutdown_connections 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 1 -- 172.21.15.33:0/4259065341 wait complete. 2024-04-24T23:22:24.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcacb280740 20 client.17610 trim_cache size 0 max 16384 2024-04-24T23:22:24.653 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:22:24.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcaac0a2b90 msgr2=0x7fcaac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcaac0a2b90 0x7fcaac0a5050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x558bae485890 tx=0x7fcab0013040 comp rx=0 tx=0).stop 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0107460 msgr2=0x7fcac019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0107460 0x7fcac019f070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcab8031ae0 tx=0x7fcab8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 shutdown_connections 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcaac0a2b90 0x7fcaac0a5050 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac0107460 0x7fcac019f070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac0104b30 0x7fcac0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 --2- 172.21.15.33:0/3549856230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0102200 0x7fcac0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.644+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 >> 172.21.15.33:0/3549856230 conn(0x7fcac00fbbf0 msgr2=0x7fcac0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.645+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 shutdown_connections 2024-04-24T23:22:24.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.645+0000 7fcac8570640 1 -- 172.21.15.33:0/3549856230 wait complete. 2024-04-24T23:22:24.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 -- 172.21.15.33:0/1911574276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 msgr2=0x7f81e0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 --2- 172.21.15.33:0/1911574276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e0108ae0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009ca0 tx=0x7f81d802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 -- 172.21.15.33:0/1911574276 shutdown_connections 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 --2- 172.21.15.33:0/1911574276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e0108ae0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 --2- 172.21.15.33:0/1911574276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 0x7f81e0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 --2- 172.21.15.33:0/1911574276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e01006a0 0x7f81e0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 -- 172.21.15.33:0/1911574276 >> 172.21.15.33:0/1911574276 conn(0x7f81e00fbc30 msgr2=0x7f81e00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 -- 172.21.15.33:0/1911574276 shutdown_connections 2024-04-24T23:22:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.761+0000 7f81e74b3640 1 -- 172.21.15.33:0/1911574276 wait complete. 2024-04-24T23:22:24.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 Processor -- start 2024-04-24T23:22:24.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 -- start start 2024-04-24T23:22:24.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e01006a0 0x7f81e0199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 0x7f81e019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e01a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81e010c320 con 0x7f81e0102250 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81e010c1a0 con 0x7f81e0101890 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e74b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81e010c4a0 con 0x7f81e01006a0 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e01a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e01a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46816/0 (socket says 172.21.15.33:46816) 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e5a50640 1 -- 172.21.15.33:0/393172996 learned_addr learned my addr 172.21.15.33:0/393172996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 0x7f81e019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.762+0000 7f81e524f640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e01006a0 0x7f81e0199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e5a50640 1 -- 172.21.15.33:0/393172996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e01006a0 msgr2=0x7f81e0199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e5a50640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e01006a0 0x7f81e0199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e5a50640 1 -- 172.21.15.33:0/393172996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 msgr2=0x7f81e019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e5a50640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 0x7f81e019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e5a50640 1 -- 172.21.15.33:0/393172996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e01a18f0 con 0x7f81e0102250 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 0x7f81e019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e5a50640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e01a1250 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f81d8031ae0 tx=0x7f81d8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81d67fc640 1 -- 172.21.15.33:0/393172996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8003b70 con 0x7f81e0102250 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e01a1b80 con 0x7f81e0102250 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81d67fc640 1 -- 172.21.15.33:0/393172996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d8003d10 con 0x7f81e0102250 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81d67fc640 1 -- 172.21.15.33:0/393172996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d802fc40 con 0x7f81e0102250 2024-04-24T23:22:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.763+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e01a20d0 con 0x7f81e0102250 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81d67fc640 1 -- 172.21.15.33:0/393172996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f81d8004050 con 0x7f81e0102250 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 Processor -- start 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 -- start start 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81d67fc640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81cc0a2b90 0x7f81cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dad7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dad7c20 0x55a91dadc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 0x55a91dadf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a91da3b670 con 0x55a91dad7c20 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a91d9e3970 con 0x55a91dad7230 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.766+0000 7f81ea1be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a91da37630 con 0x55a91dadcd90 2024-04-24T23:22:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81d67fc640 1 -- 172.21.15.33:0/393172996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81d80f04a0 con 0x7f81e0102250 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81cc0a2b90 0x7f81cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 0x55a91dadf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 0x55a91dadf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42672/0 (socket says 172.21.15.33:42672) 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 -- 172.21.15.33:0/1831079663 learned_addr learned my addr 172.21.15.33:0/1831079663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dad7c20 0x55a91dadc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e524f640 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dad7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 -- 172.21.15.33:0/1831079663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 msgr2=0x55a91dad7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dad7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 -- 172.21.15.33:0/1831079663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dad7c20 msgr2=0x55a91dadc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dad7c20 0x55a91dadc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 -- 172.21.15.33:0/1831079663 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91da07400 con 0x55a91dadcd90 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81cc0a2b90 0x7f81cc0a5050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009fd0 tx=0x7f81d0009ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81e5a50640 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 0x55a91dadf220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81d80b8ef0 tx=0x7f81d8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48377dc4c56738f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81cb7fe640 1 -- 172.21.15.33:0/1831079663 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80bd190 con 0x55a91dadcd90 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81cb7fe640 1 -- 172.21.15.33:0/1831079663 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d8036040 con 0x55a91dadcd90 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.767+0000 7f81cb7fe640 1 -- 172.21.15.33:0/1831079663 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80bad90 con 0x55a91dadcd90 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 -- 172.21.15.33:0/1831079663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 msgr2=0x55a91dadf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 0x55a91dadf220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81d80b8ef0 tx=0x7f81d8005ef0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 -- 172.21.15.33:0/1831079663 shutdown_connections 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dadcd90 0x55a91dadf220 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dad7c20 0x55a91dadc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 --2- 172.21.15.33:0/1831079663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dad7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 -- 172.21.15.33:0/1831079663 >> 172.21.15.33:0/1831079663 conn(0x55a91dad28a0 msgr2=0x55a91dad2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 -- 172.21.15.33:0/1831079663 shutdown_connections 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 -- 172.21.15.33:0/1831079663 wait complete. 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.768+0000 7f81ea1be740 1 -- start start 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 Processor -- start 2024-04-24T23:22:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a91da37630 con 0x55a91dadcd90 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a91d9f7050 con 0x55a91dad7230 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a91dadfb00 con 0x55a91dad7c20 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81ea1be740 10 client.0 _ll_register_callbacks cb 0x55a91d74ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e524f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e4a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e524f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60220/0 (socket says 172.21.15.33:60220) 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e524f640 1 -- 172.21.15.33:0/2560350120 learned_addr learned my addr 172.21.15.33:0/2560350120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e4a4e640 1 -- 172.21.15.33:0/2560350120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 msgr2=0x55a91dae0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e4a4e640 1 -- 172.21.15.33:0/2560350120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 msgr2=0x55a91dae84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.769+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81e4a4e640 1 -- 172.21.15.33:0/2560350120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91d9c8dc0 con 0x55a91dad7c20 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81e5a50640 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae1460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55a91dad8020 tx=0x7f81d001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2d228231b8254 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2560350120 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0024070 con 0x55a91dad7c20 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2560350120 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81d0011d50 con 0x55a91dad7c20 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d8036040 con 0x55a91dad7c20 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2560350120 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81d00116a0 con 0x55a91dad7c20 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81ea1be740 10 client.27115 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a91d74f3f0 con 0x55a91dad7c20 2024-04-24T23:22:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.770+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2560350120 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d8036040 con 0x55a91dad7c20 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2560350120 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f81d002a9a0 con 0x55a91dad7c20 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81ea1be740 10 client.27115 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81ea1be740 10 client.27115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81ea1be740 20 client.27115 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91daecf20 0x55a91daef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81ea1be740 4 client.27115 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a91da48bb0 con 0x55a91daecf20 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81e524f640 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91daecf20 0x55a91daef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81c8ff9640 10 client.27115 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:24.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.771+0000 7f81e524f640 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91daecf20 0x55a91daef360 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1191c35eab540fe server_cookie=54ce9c3c3dad7c94 in_seq=0 out_seq=0 2024-04-24T23:22:24.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2560350120 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a91da48bb0 con 0x55a91daecf20 2024-04-24T23:22:24.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81c8ff9640 10 client.27115 handle_command_reply: tid=0 2024-04-24T23:22:24.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 client.27115 shutdown 2024-04-24T23:22:24.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 msgr2=0x55a91dae1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae1460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55a91dad8020 tx=0x7f81d001d000 comp rx=0 tx=0).stop 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 shutdown_connections 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91daecf20 0x55a91daef360 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae1460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2560350120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81c8ff9640 0 client.27115 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.772+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 >> 172.21.15.33:0/2560350120 conn(0x55a91dad28a0 msgr2=0x55a91dad1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.773+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 shutdown_connections 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.773+0000 7f81ea1be740 1 -- 172.21.15.33:0/2560350120 wait complete. 2024-04-24T23:22:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.773+0000 7f81ea1be740 20 client.27115 trim_cache size 0 max 16384 2024-04-24T23:22:24.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.788+0000 7f81ea1be740 1 Processor -- start 2024-04-24T23:22:24.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 -- start start 2024-04-24T23:22:24.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dad5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01aee80 con 0x55a91dadcd90 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01ac750 con 0x55a91dad7230 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81ea1be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01aa5a0 con 0x55a91dad7c20 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dad5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e4a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dad5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46840/0 (socket says 172.21.15.33:46840) 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e5a50640 1 -- 172.21.15.33:0/643530705 learned_addr learned my addr 172.21.15.33:0/643530705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 -- 172.21.15.33:0/643530705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 msgr2=0x55a91dae8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 -- 172.21.15.33:0/643530705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 msgr2=0x55a91dad5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dad5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 -- 172.21.15.33:0/643530705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91d8197d0 con 0x55a91dad7230 2024-04-24T23:22:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.789+0000 7f81e524f640 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55a91daef960 tx=0x7f81dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647ebcfee75a8c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81caffd640 1 -- 172.21.15.33:0/643530705 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc01f250 con 0x55a91dad7230 2024-04-24T23:22:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81caffd640 1 -- 172.21.15.33:0/643530705 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81dc004490 con 0x55a91dad7230 2024-04-24T23:22:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81caffd640 1 -- 172.21.15.33:0/643530705 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc010d90 con 0x55a91dad7230 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 -- 172.21.15.33:0/643530705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 msgr2=0x55a91dae0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55a91daef960 tx=0x7f81dc005f90 comp rx=0 tx=0).stop 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 -- 172.21.15.33:0/643530705 shutdown_connections 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dad5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7c20 0x55a91dae8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 --2- 172.21.15.33:0/643530705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7230 0x55a91dae0540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 -- 172.21.15.33:0/643530705 >> 172.21.15.33:0/643530705 conn(0x55a91dad28a0 msgr2=0x55a91dad1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.790+0000 7f81ea1be740 1 -- 172.21.15.33:0/643530705 shutdown_connections 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 -- 172.21.15.33:0/643530705 wait complete. 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 -- start start 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 Processor -- start 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 0x55a91dae5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7c20 0x55a91dae5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01aac10 con 0x55a91dadcd90 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01ac080 con 0x55a91dad7c20 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01ab2e0 con 0x55a91dad7230 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.791+0000 7f81ea1be740 10 client.0 _ll_register_callbacks cb 0x7f81d0011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 0x55a91dae5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 0x55a91dae5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42682/0 (socket says 172.21.15.33:42682) 2024-04-24T23:22:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e5a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 -- 172.21.15.33:0/2015019470 learned_addr learned my addr 172.21.15.33:0/2015019470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7c20 0x55a91dae5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 -- 172.21.15.33:0/2015019470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7c20 msgr2=0x55a91dae5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7c20 0x55a91dae5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 -- 172.21.15.33:0/2015019470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 msgr2=0x55a91dae6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 -- 172.21.15.33:0/2015019470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91dae7400 con 0x55a91dad7230 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81e524f640 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 0x55a91dae5980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009de0 tx=0x7f81dc004090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd9dd567143245 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2015019470 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc02f070 con 0x55a91dad7230 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.792+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81dc004490 con 0x55a91dad7230 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2015019470 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81dc010d10 con 0x55a91dad7230 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2015019470 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81dc036810 con 0x55a91dad7230 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 10 client.27121 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a91dae8750 con 0x55a91dad7230 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2015019470 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81dc004490 con 0x55a91dad7230 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2015019470 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f81dc01f020 con 0x55a91dad7230 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 10 client.27121 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 10 client.27121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 20 client.27121 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91db53b40 0x55a91db55f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 4 client.27121 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.793+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a91dae0d10 con 0x55a91db53b40 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91db53b40 0x55a91db55f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81c8ff9640 10 client.27121 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81e4a4e640 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91db53b40 0x55a91db55f60 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ea93408b2b57e2 server_cookie=b1c138e641fb29a7 in_seq=0 out_seq=0 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81c8ff9640 1 -- 172.21.15.33:0/2015019470 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a91dae0d10 con 0x55a91db53b40 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81c8ff9640 10 client.27121 handle_command_reply: tid=0 2024-04-24T23:22:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 client.27121 shutdown 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 msgr2=0x55a91dae5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 0x55a91dae5980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009de0 tx=0x7f81dc004090 comp rx=0 tx=0).stop 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 shutdown_connections 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a91db53b40 0x55a91db55f60 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a91dadcd90 0x55a91dae6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a91dad7c20 0x55a91dae5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81c8ff9640 0 client.27121 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 --2- 172.21.15.33:0/2015019470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a91dad7230 0x55a91dae5980 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.794+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 >> 172.21.15.33:0/2015019470 conn(0x55a91dad28a0 msgr2=0x55a91dad8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 shutdown_connections 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81ea1be740 1 -- 172.21.15.33:0/2015019470 wait complete. 2024-04-24T23:22:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81ea1be740 20 client.27121 trim_cache size 0 max 16384 2024-04-24T23:22:24.802 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:22:24.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81cc0a2b90 msgr2=0x7f81cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81cc0a2b90 0x7f81cc0a5050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009fd0 tx=0x7f81d0009ee0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 msgr2=0x7f81e01a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e01a1250 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f81d8031ae0 tx=0x7f81d8031ee0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 shutdown_connections 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81cc0a2b90 0x7f81cc0a5050 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e0102250 0x7f81e01a1250 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81e0101890 0x7f81e019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 --2- 172.21.15.33:0/393172996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e01006a0 0x7f81e0199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 >> 172.21.15.33:0/393172996 conn(0x7f81e00fbc30 msgr2=0x7f81e00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 shutdown_connections 2024-04-24T23:22:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:24.795+0000 7f81e74b3640 1 -- 172.21.15.33:0/393172996 wait complete. 2024-04-24T23:22:25.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:25 smithi038 ceph-mon[32450]: pgmap v663: 129 pgs: 129 active+clean; 269 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T23:22:25.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3632698929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:22:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:25 smithi117 ceph-mon[32529]: pgmap v663: 129 pgs: 129 active+clean; 269 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T23:22:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3632698929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:22:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:25 smithi033 ceph-mon[29201]: pgmap v663: 129 pgs: 129 active+clean; 269 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T23:22:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3632698929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:22:25.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1199511 12.53 MB/sec execute 390 sec latency 16.414 ms 2024-04-24T23:22:25.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:22:25.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:25.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:22:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 -- 172.21.15.33:0/3807756202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c104b90 msgr2=0x7fa70c106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 --2- 172.21.15.33:0/3807756202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c104b90 0x7fa70c106f80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009900 tx=0x7fa6f402a410 comp rx=0 tx=0).stop 2024-04-24T23:22:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 -- 172.21.15.33:0/3807756202 shutdown_connections 2024-04-24T23:22:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 --2- 172.21.15.33:0/3807756202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 0x7fa70c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 --2- 172.21.15.33:0/3807756202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c104b90 0x7fa70c106f80 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 --2- 172.21.15.33:0/3807756202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c102260 0x7fa70c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 -- 172.21.15.33:0/3807756202 >> 172.21.15.33:0/3807756202 conn(0x7fa70c0fbbf0 msgr2=0x7fa70c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 -- 172.21.15.33:0/3807756202 shutdown_connections 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.916+0000 7fa711c38640 1 -- 172.21.15.33:0/3807756202 wait complete. 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 Processor -- start 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 -- start start 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 0x7fa70c101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c104b90 0x7fa70c100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 0x7fa70c100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10d420 con 0x7fa70c102260 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10d2a0 con 0x7fa70c104b90 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa711c38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10d5a0 con 0x7fa70c1074c0 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 0x7fa70c101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 0x7fa70c101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39808/0 (socket says 172.21.15.33:39808) 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa70b7fe640 1 -- 172.21.15.33:0/3833434759 learned_addr learned my addr 172.21.15.33:0/3833434759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.917+0000 7fa70affd640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c104b90 0x7fa70c100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70bfff640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 0x7fa70c100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70b7fe640 1 -- 172.21.15.33:0/3833434759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 msgr2=0x7fa70c100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 0x7fa70c100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70b7fe640 1 -- 172.21.15.33:0/3833434759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c104b90 msgr2=0x7fa70c100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c104b90 0x7fa70c100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70b7fe640 1 -- 172.21.15.33:0/3833434759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c100fe0 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70bfff640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 0x7fa70c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 0x7fa70c101bd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00ebc0 tx=0x7fa6fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa708ff9640 1 -- 172.21.15.33:0/3833434759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc010630 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa708ff9640 1 -- 172.21.15.33:0/3833434759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6fc010cb0 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70c1a1c20 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa708ff9640 1 -- 172.21.15.33:0/3833434759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc018bd0 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.918+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70c1a2190 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.920+0000 7fa708ff9640 1 -- 172.21.15.33:0/3833434759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa6fc0107d0 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa708ff9640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6f80a2c10 0x7fa6f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa70affd640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6f80a2c10 0x7fa6f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa708ff9640 1 -- 172.21.15.33:0/3833434759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa6fc0cf9a0 con 0x7fa70c102260 2024-04-24T23:22:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 Processor -- start 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 -- start start 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e88f0 0x5575268ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575268462c0 con 0x5575268e3780 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5575267ee910 con 0x5575268e88f0 2024-04-24T23:22:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa714a24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557526842280 con 0x5575268e2d90 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.921+0000 7fa70affd640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6f80a2c10 0x7fa6f80a50d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa6f402aa00 tx=0x7fa6f40315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e88f0 0x5575268ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268e8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39818/0 (socket says 172.21.15.33:39818) 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268e31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:58008/0 (socket says 172.21.15.33:58008) 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70affd640 1 -- 172.21.15.33:0/10070587 learned_addr learned my addr 172.21.15.33:0/10070587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 -- 172.21.15.33:0/10070587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e88f0 msgr2=0x5575268ead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e88f0 0x5575268ead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 -- 172.21.15.33:0/10070587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 msgr2=0x5575268e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 -- 172.21.15.33:0/10070587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575267fffb0 con 0x5575268e2d90 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70affd640 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70bfff640 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e88f0 0x5575268ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268e31b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0cd110 tx=0x7fa6fc018670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55752a9228a86275 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa701ffb640 1 -- 172.21.15.33:0/10070587 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0a0220 con 0x5575268e2d90 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa701ffb640 1 -- 172.21.15.33:0/10070587 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6fc0a4ae0 con 0x5575268e2d90 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.922+0000 7fa701ffb640 1 -- 172.21.15.33:0/10070587 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc09b520 con 0x5575268e2d90 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 -- 172.21.15.33:0/10070587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 msgr2=0x5575268e31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268e31b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0cd110 tx=0x7fa6fc018670 comp rx=0 tx=0).stop 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 -- 172.21.15.33:0/10070587 shutdown_connections 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e88f0 0x5575268ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 --2- 172.21.15.33:0/10070587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268e31b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 -- 172.21.15.33:0/10070587 >> 172.21.15.33:0/10070587 conn(0x5575268aa950 msgr2=0x5575268de810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 -- 172.21.15.33:0/10070587 shutdown_connections 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 -- 172.21.15.33:0/10070587 wait complete. 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.923+0000 7fa714a24740 1 -- start start 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 Processor -- start 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 0x5575268f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575268eb8a0 con 0x5575268e88f0 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557526801e90 con 0x5575268e3780 2024-04-24T23:22:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5575268eba20 con 0x5575268e2d90 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa714a24740 10 client.0 _ll_register_callbacks cb 0x55752683d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 0x5575268f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 0x5575268f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39820/0 (socket says 172.21.15.33:39820) 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 -- 172.21.15.33:0/3342243922 learned_addr learned my addr 172.21.15.33:0/3342243922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70affd640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 -- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 msgr2=0x5575268ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 -- 172.21.15.33:0/3342243922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 msgr2=0x5575268ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 -- 172.21.15.33:0/3342243922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575266227e0 con 0x5575268e88f0 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70affd640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa70bfff640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 0x5575268f47a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa704009dd0 tx=0x7fa7040117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c918a927c2f945 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.924+0000 7fa6eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa6eaffd640 1 -- 172.21.15.33:0/3342243922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704004460 con 0x5575268e88f0 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa6eaffd640 1 -- 172.21.15.33:0/3342243922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa704005020 con 0x5575268e88f0 2024-04-24T23:22:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6fc0a4ae0 con 0x5575268e88f0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa6eaffd640 1 -- 172.21.15.33:0/3342243922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa704023050 con 0x5575268e88f0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 10 client.17640 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557526682f70 con 0x5575268e88f0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa6eaffd640 1 -- 172.21.15.33:0/3342243922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6fc0a4ae0 con 0x5575268e88f0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa6eaffd640 1 -- 172.21.15.33:0/3342243922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa70400f040 con 0x5575268e88f0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 10 client.17640 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 20 client.17640 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5575268f8df0 0x5575268fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 4 client.17640 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55752681a5a0 con 0x5575268f8df0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.925+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5575268f8df0 0x5575268fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.926+0000 7fa6eaffd640 10 client.17640 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.926+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5575268f8df0 0x5575268fb200 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9807f22e06fa5 server_cookie=e4f849b8bd8275d5 in_seq=0 out_seq=0 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.926+0000 7fa6eaffd640 1 -- 172.21.15.33:0/3342243922 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55752681a5a0 con 0x5575268f8df0 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.926+0000 7fa6eaffd640 10 client.17640 handle_command_reply: tid=0 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.926+0000 7fa714a24740 1 client.17640 shutdown 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 msgr2=0x5575268f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 0x5575268f47a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa704009dd0 tx=0x7fa7040117b0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 shutdown_connections 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5575268f8df0 0x5575268fb200 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e88f0 0x5575268f47a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa6eaffd640 0 client.17640 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 --2- 172.21.15.33:0/3342243922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e2d90 0x5575268ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 >> 172.21.15.33:0/3342243922 conn(0x5575268aa950 msgr2=0x5575268dd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 shutdown_connections 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 1 -- 172.21.15.33:0/3342243922 wait complete. 2024-04-24T23:22:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.927+0000 7fa714a24740 20 client.17640 trim_cache size 0 max 16384 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 Processor -- start 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 -- start start 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e2d90 0x5575268e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1aefa0 con 0x5575268e3780 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1ac870 con 0x5575268e2d90 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa714a24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1aa6c0 con 0x5575268e88f0 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39828/0 (socket says 172.21.15.33:39828) 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 -- 172.21.15.33:0/3156667051 learned_addr learned my addr 172.21.15.33:0/3156667051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e2d90 0x5575268e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70bfff640 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 -- 172.21.15.33:0/3156667051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 msgr2=0x5575268ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 -- 172.21.15.33:0/3156667051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e2d90 msgr2=0x5575268e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e2d90 0x5575268e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.945+0000 7fa70affd640 1 -- 172.21.15.33:0/3156667051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55752651a170 con 0x5575268e3780 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.946+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e2d90 0x5575268e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.946+0000 7fa70affd640 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268f4760 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009230 tx=0x7fa6f403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540909f86ab86821 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.946+0000 7fa7017fa640 1 -- 172.21.15.33:0/3156667051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4035040 con 0x5575268e3780 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.946+0000 7fa7017fa640 1 -- 172.21.15.33:0/3156667051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f404e070 con 0x5575268e3780 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.946+0000 7fa7017fa640 1 -- 172.21.15.33:0/3156667051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4032cc0 con 0x5575268e3780 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- 172.21.15.33:0/3156667051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 msgr2=0x5575268f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268f4760 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009230 tx=0x7fa6f403d000 comp rx=0 tx=0).stop 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- 172.21.15.33:0/3156667051 shutdown_connections 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e3780 0x5575268f4760 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- 172.21.15.33:0/3156667051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e2d90 0x5575268e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- 172.21.15.33:0/3156667051 >> 172.21.15.33:0/3156667051 conn(0x5575268aa950 msgr2=0x5575268e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- 172.21.15.33:0/3156667051 shutdown_connections 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- 172.21.15.33:0/3156667051 wait complete. 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- start start 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 Processor -- start 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 0x5575268f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1aad30 con 0x5575268e2d90 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1ac1a0 con 0x5575268e3780 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1ab400 con 0x5575268e88f0 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa714a24740 10 client.0 _ll_register_callbacks cb 0x7fa704009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 0x5575268f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 0x5575268f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39836/0 (socket says 172.21.15.33:39836) 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa70b7fe640 1 -- 172.21.15.33:0/2241146653 learned_addr learned my addr 172.21.15.33:0/2241146653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.947+0000 7fa70bfff640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70affd640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70b7fe640 1 -- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 msgr2=0x5575268f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70b7fe640 1 -- 172.21.15.33:0/2241146653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 msgr2=0x5575268f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70b7fe640 1 -- 172.21.15.33:0/2241146653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575268f3160 con 0x5575268e2d90 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa70b7fe640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 0x5575268f4200 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0cd480 tx=0x7fa6fc09c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77ca007cfed23b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa6eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:25.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa6eaffd640 1 -- 172.21.15.33:0/2241146653 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0b8070 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa6eaffd640 1 -- 172.21.15.33:0/2241146653 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa6fc09b2c0 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f404e070 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa6eaffd640 1 -- 172.21.15.33:0/2241146653 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6fc010e60 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa714a24740 10 client.17646 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55752695fff0 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.948+0000 7fa6eaffd640 1 -- 172.21.15.33:0/2241146653 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f404e070 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa6eaffd640 1 -- 172.21.15.33:0/2241146653 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa6fc07ace0 con 0x5575268e2d90 2024-04-24T23:22:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa714a24740 10 client.17646 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa714a24740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa714a24740 20 client.17646 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa714a24740 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557526960550 0x557526962940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa714a24740 4 client.17646 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557526853800 con 0x557526960550 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa70affd640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557526960550 0x557526962940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa6eaffd640 10 client.17646 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.949+0000 7fa70affd640 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557526960550 0x557526962940 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f72a67902551001 server_cookie=cbd6f18269baf8cd in_seq=0 out_seq=0 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa6eaffd640 1 -- 172.21.15.33:0/2241146653 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557526853800 con 0x557526960550 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa6eaffd640 10 client.17646 handle_command_reply: tid=0 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 client.17646 shutdown 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 msgr2=0x5575268f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 0x5575268f4200 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0cd480 tx=0x7fa6fc09c670 comp rx=0 tx=0).stop 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 shutdown_connections 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557526960550 0x557526962940 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575268e88f0 0x5575268f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5575268e3780 0x5575268f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa6eaffd640 0 client.17646 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.950+0000 7fa714a24740 1 --2- 172.21.15.33:0/2241146653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575268e2d90 0x5575268f4200 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 >> 172.21.15.33:0/2241146653 conn(0x5575268aa950 msgr2=0x5575268dd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 shutdown_connections 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa714a24740 1 -- 172.21.15.33:0/2241146653 wait complete. 2024-04-24T23:22:25.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa714a24740 20 client.17646 trim_cache size 0 max 16384 2024-04-24T23:22:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:22:25.965 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:22:25.965 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "2e8aceac-cd7d-41a7-8833-51abe94238ec", 2024-04-24T23:22:25.965 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:22:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6f80a2c10 msgr2=0x7fa6f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6f80a2c10 0x7fa6f80a50d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa6f402aa00 tx=0x7fa6f40315b0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 msgr2=0x7fa70c101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 0x7fa70c101bd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00ebc0 tx=0x7fa6fc00c6a0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 shutdown_connections 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa6f80a2c10 0x7fa6f80a50d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c1074c0 0x7fa70c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c104b90 0x7fa70c100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 --2- 172.21.15.33:0/3833434759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70c102260 0x7fa70c101bd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 >> 172.21.15.33:0/3833434759 conn(0x7fa70c0fbbf0 msgr2=0x7fa70c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:25.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 shutdown_connections 2024-04-24T23:22:25.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:25.951+0000 7fa711c38640 1 -- 172.21.15.33:0/3833434759 wait complete. 2024-04-24T23:22:25.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:22:25.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 -- 172.21.15.33:0/738599641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698102200 msgr2=0x7f26981045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 --2- 172.21.15.33:0/738599641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698102200 0x7f26981045f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2694009900 tx=0x7f269402a410 comp rx=0 tx=0).stop 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 -- 172.21.15.33:0/738599641 shutdown_connections 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 --2- 172.21.15.33:0/738599641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698107460 0x7f26981098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 --2- 172.21.15.33:0/738599641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2698104b30 0x7f2698106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 --2- 172.21.15.33:0/738599641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698102200 0x7f26981045f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 -- 172.21.15.33:0/738599641 >> 172.21.15.33:0/738599641 conn(0x7f26980fbbf0 msgr2=0x7f26980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 -- 172.21.15.33:0/738599641 shutdown_connections 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 -- 172.21.15.33:0/738599641 wait complete. 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 Processor -- start 2024-04-24T23:22:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 -- start start 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 0x7f2698199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.075+0000 7f269ff16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2698104b30 0x7f269819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269ff16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698107460 0x7f26981a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269ff16640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f269810d430 con 0x7f2698107460 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269ff16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f269810d2b0 con 0x7f2698102200 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269ff16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f269810d5b0 con 0x7f2698104b30 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269e4b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698107460 0x7f26981a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 0x7f2698199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 0x7f2698199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54330/0 (socket says 172.21.15.33:54330) 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 -- 172.21.15.33:0/4102752430 learned_addr learned my addr 172.21.15.33:0/4102752430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269e4b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698107460 0x7f26981a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39852/0 (socket says 172.21.15.33:39852) 2024-04-24T23:22:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269d4b1640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2698104b30 0x7f269819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 -- 172.21.15.33:0/4102752430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2698104b30 msgr2=0x7f269819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2698104b30 0x7f269819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 -- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698107460 msgr2=0x7f26981a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698107460 0x7f26981a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 -- 172.21.15.33:0/4102752430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26981a19b0 con 0x7f2698102200 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269dcb2640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 0x7f2698199da0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2694009230 tx=0x7f26940042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f268effd640 1 -- 172.21.15.33:0/4102752430 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269403f030 con 0x7f2698102200 2024-04-24T23:22:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26981a1c40 con 0x7f2698102200 2024-04-24T23:22:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f268effd640 1 -- 172.21.15.33:0/4102752430 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2694003ce0 con 0x7f2698102200 2024-04-24T23:22:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f268effd640 1 -- 172.21.15.33:0/4102752430 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694035070 con 0x7f2698102200 2024-04-24T23:22:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.076+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26981a2130 con 0x7f2698102200 2024-04-24T23:22:26.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.078+0000 7f268effd640 1 -- 172.21.15.33:0/4102752430 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2694042030 con 0x7f2698102200 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 Processor -- start 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 -- start start 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f268effd640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26840a2b90 0x7f26840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 0x560141435660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435c30 0x56014143a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x56014143d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601413996b0 con 0x56014143ada0 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560141341880 con 0x560141435c30 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560141395670 con 0x560141435240 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f269e4b3640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26840a2b90 0x7f26840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 0x560141435660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f269e4b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x56014143d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 0x560141435660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:58062/0 (socket says 172.21.15.33:58062) 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.079+0000 7f269dcb2640 1 -- 172.21.15.33:0/2236981771 learned_addr learned my addr 172.21.15.33:0/2236981771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269d4b1640 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435c30 0x56014143a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f268effd640 1 -- 172.21.15.33:0/4102752430 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f26940ef990 con 0x7f2698102200 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269dcb2640 1 -- 172.21.15.33:0/2236981771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435c30 msgr2=0x56014143a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269dcb2640 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435c30 0x56014143a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269dcb2640 1 -- 172.21.15.33:0/2236981771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 msgr2=0x56014143d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269dcb2640 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x56014143d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269dcb2640 1 -- 172.21.15.33:0/2236981771 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601411777d0 con 0x560141435240 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269e4b3640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26840a2b90 0x7f26840a5050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2690005fd0 tx=0x7f2690009f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f269dcb2640 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 0x560141435660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2694031b20 tx=0x7f2694040fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb871c3ed88534c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f2683fff640 1 -- 172.21.15.33:0/2236981771 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694049070 con 0x560141435240 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f2683fff640 1 -- 172.21.15.33:0/2236981771 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26940d00b0 con 0x560141435240 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.080+0000 7f2683fff640 1 -- 172.21.15.33:0/2236981771 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26940c01b0 con 0x560141435240 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 -- 172.21.15.33:0/2236981771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 msgr2=0x560141435660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 0x560141435660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2694031b20 tx=0x7f2694040fb0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 -- 172.21.15.33:0/2236981771 shutdown_connections 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x56014143d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435c30 0x56014143a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/2236981771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435240 0x560141435660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 -- 172.21.15.33:0/2236981771 >> 172.21.15.33:0/2236981771 conn(0x5601414308b0 msgr2=0x560141430cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 -- 172.21.15.33:0/2236981771 shutdown_connections 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 -- 172.21.15.33:0/2236981771 wait complete. 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 -- start start 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.081+0000 7f26a2c1a740 1 Processor -- start 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x56014143f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435c30 0x56014143f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x560141446910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56014143db10 con 0x56014143ada0 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560141354f50 con 0x560141435240 2024-04-24T23:22:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56014143dc90 con 0x560141435c30 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26a2c1a740 10 client.0 _ll_register_callbacks cb 0x560141390000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x560141446910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x56014143f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x56014143f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54340/0 (socket says 172.21.15.33:54340) 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269dcb2640 1 -- 172.21.15.33:0/3363354917 learned_addr learned my addr 172.21.15.33:0/3363354917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269d4b1640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435c30 0x56014143f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 -- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435c30 msgr2=0x56014143f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435c30 0x56014143f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 -- 172.21.15.33:0/3363354917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 msgr2=0x56014143f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x56014143f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 -- 172.21.15.33:0/3363354917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601410ad3f0 con 0x56014143ada0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269d4b1640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435c30 0x56014143f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f269e4b3640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x560141446910 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f269001e540 tx=0x7f269001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b045eababb5cf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.082+0000 7f26817fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26817fa640 1 -- 172.21.15.33:0/3363354917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690024070 con 0x56014143ada0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26817fa640 1 -- 172.21.15.33:0/3363354917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2690003970 con 0x56014143ada0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26940d00b0 con 0x56014143ada0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26817fa640 1 -- 172.21.15.33:0/3363354917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2690033050 con 0x56014143ada0 2024-04-24T23:22:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 10 client.17658 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56014106d0f0 con 0x56014143ada0 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26817fa640 1 -- 172.21.15.33:0/3363354917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26940d00b0 con 0x56014143ada0 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26817fa640 1 -- 172.21.15.33:0/3363354917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2690020600 con 0x56014143ada0 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 10 client.17658 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 20 client.17658 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x56014144d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 4 client.17658 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.083+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601413a6bf0 con 0x56014144b260 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.084+0000 7f269dcb2640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x56014144d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.084+0000 7f269dcb2640 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x56014144d6c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f640bb5bce9a0b53 server_cookie=455bfb8f9e5273b9 in_seq=0 out_seq=0 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.084+0000 7f26817fa640 10 client.17658 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26817fa640 1 -- 172.21.15.33:0/3363354917 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601413a6bf0 con 0x56014144b260 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26817fa640 10 client.17658 handle_command_reply: tid=0 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 client.17658 shutdown 2024-04-24T23:22:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 msgr2=0x560141446910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x560141446910 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f269001e540 tx=0x7f269001d000 comp rx=0 tx=0).stop 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 shutdown_connections 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x56014144d6c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56014143ada0 0x560141446910 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560141435c30 0x56014143f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26817fa640 0 client.17658 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/3363354917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x56014143f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 >> 172.21.15.33:0/3363354917 conn(0x5601414308b0 msgr2=0x56014142faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 shutdown_connections 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 1 -- 172.21.15.33:0/3363354917 wait complete. 2024-04-24T23:22:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.085+0000 7f26a2c1a740 20 client.17658 trim_cache size 0 max 16384 2024-04-24T23:22:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.103+0000 7f26a2c1a740 1 Processor -- start 2024-04-24T23:22:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 -- start start 2024-04-24T23:22:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x560141433860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x560141433da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26981aab50 con 0x560141435c30 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26981a8420 con 0x560141435240 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26981a6270 con 0x56014143ada0 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269d4b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x560141433860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269d4b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39892/0 (socket says 172.21.15.33:39892) 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269d4b1640 1 -- 172.21.15.33:0/343556617 learned_addr learned my addr 172.21.15.33:0/343556617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269e4b3640 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x560141433da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 -- 172.21.15.33:0/343556617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 msgr2=0x560141433da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x560141433da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 -- 172.21.15.33:0/343556617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 msgr2=0x5601414468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 -- 172.21.15.33:0/343556617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560141422090 con 0x560141435240 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269d4b1640 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.104+0000 7f269dcb2640 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x560141433860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f26940dc070 tx=0x7f269402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8657c4153eb7c26a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26837fe640 1 -- 172.21.15.33:0/343556617 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694049040 con 0x560141435240 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26837fe640 1 -- 172.21.15.33:0/343556617 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26940d00b0 con 0x560141435240 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26837fe640 1 -- 172.21.15.33:0/343556617 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269403b460 con 0x560141435240 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 -- 172.21.15.33:0/343556617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 msgr2=0x560141433860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x560141433860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f26940dc070 tx=0x7f269402a410 comp rx=0 tx=0).stop 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 -- 172.21.15.33:0/343556617 shutdown_connections 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x560141433da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/343556617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x560141433860 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 -- 172.21.15.33:0/343556617 >> 172.21.15.33:0/343556617 conn(0x5601414308b0 msgr2=0x56014143dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 -- 172.21.15.33:0/343556617 shutdown_connections 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.105+0000 7f26a2c1a740 1 -- 172.21.15.33:0/343556617 wait complete. 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 -- start start 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 Processor -- start 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x5601414442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x5601414457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26981a68e0 con 0x560141435c30 2024-04-24T23:22:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f26981a7d50 con 0x560141435240 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26981a6fb0 con 0x56014143ada0 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f26a2c1a740 10 client.0 _ll_register_callbacks cb 0x7f2690009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269d4b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x5601414442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269e4b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x5601414457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269dcb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x5601414442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54344/0 (socket says 172.21.15.33:54344) 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269d4b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39898/0 (socket says 172.21.15.33:39898) 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269dcb2640 1 -- 172.21.15.33:0/1652080618 learned_addr learned my addr 172.21.15.33:0/1652080618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.106+0000 7f269e4b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x5601414457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:58108/0 (socket says 172.21.15.33:58108) 2024-04-24T23:22:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269dcb2640 1 -- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 msgr2=0x5601414457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269dcb2640 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x5601414457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269dcb2640 1 -- 172.21.15.33:0/1652080618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 msgr2=0x5601414447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269dcb2640 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269dcb2640 1 -- 172.21.15.33:0/1652080618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560141445de0 con 0x560141435240 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269e4b3640 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x5601414457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f269dcb2640 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x5601414442a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26940bbe40 tx=0x7f26940bbc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158b74c597112957 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26817fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26817fa640 1 -- 172.21.15.33:0/1652080618 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694049070 con 0x560141435240 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26940d00b0 con 0x560141435240 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26817fa640 1 -- 172.21.15.33:0/1652080618 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f26940b8490 con 0x560141435240 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26817fa640 1 -- 172.21.15.33:0/1652080618 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26940c01b0 con 0x560141435240 2024-04-24T23:22:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26a2c1a740 10 client.27191 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601414b1ae0 con 0x560141435240 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.107+0000 7f26817fa640 1 -- 172.21.15.33:0/1652080618 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26940d00b0 con 0x560141435240 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26817fa640 1 -- 172.21.15.33:0/1652080618 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f26940bc110 con 0x560141435240 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 10 client.27191 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 10 client.27191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 20 client.27191 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x5601414b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 4 client.27191 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56014143f180 con 0x56014144b260 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f269d4b1640 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x5601414b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f269d4b1640 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x5601414b4270 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66062890248b1a85 server_cookie=7169042a857eb05c in_seq=0 out_seq=0 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26817fa640 10 client.27191 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26817fa640 1 -- 172.21.15.33:0/1652080618 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56014143f180 con 0x56014144b260 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26817fa640 10 client.27191 handle_command_reply: tid=0 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 1 client.27191 shutdown 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 msgr2=0x5601414442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.108+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x5601414442a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26940bbe40 tx=0x7f26940bbc80 comp rx=0 tx=0).stop 2024-04-24T23:22:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 shutdown_connections 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56014144b260 0x5601414b4270 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56014143ada0 0x5601414457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560141435c30 0x5601414447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 --2- 172.21.15.33:0/1652080618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560141435240 0x5601414442a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26817fa640 0 client.27191 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 >> 172.21.15.33:0/1652080618 conn(0x5601414308b0 msgr2=0x56014142f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 shutdown_connections 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 1 -- 172.21.15.33:0/1652080618 wait complete. 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f26a2c1a740 20 client.27191 trim_cache size 0 max 16384 2024-04-24T23:22:26.120 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: "2e8aceac-cd7d-41a7-8833-51abe94238ec": { 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "2e8aceac-cd7d-41a7-8833-51abe94238ec", 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:22:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:22:26.122 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26840a2b90 msgr2=0x7f26840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26840a2b90 0x7f26840a5050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2690005fd0 tx=0x7f2690009f10 comp rx=0 tx=0).stop 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 msgr2=0x7f2698199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 0x7f2698199da0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2694009230 tx=0x7f26940042f0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 shutdown_connections 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f26840a2b90 0x7f26840a5050 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2698107460 0x7f26981a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2698104b30 0x7f269819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 --2- 172.21.15.33:0/4102752430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2698102200 0x7f2698199da0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 >> 172.21.15.33:0/4102752430 conn(0x7f26980fbbf0 msgr2=0x7f2698101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 shutdown_connections 2024-04-24T23:22:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:26.109+0000 7f269ff16640 1 -- 172.21.15.33:0/4102752430 wait complete. 2024-04-24T23:22:26.124 INFO:tasks.cephfs.filesystem:scrub status for tag:2e8aceac-cd7d-41a7-8833-51abe94238ec - {'path': '/', 'tag': '2e8aceac-cd7d-41a7-8833-51abe94238ec', 'options': 'recursive,force'} 2024-04-24T23:22:26.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1201385 12.53 MB/sec execute 391 sec latency 12.342 ms 2024-04-24T23:22:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:22:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:22:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:22:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: pgmap v664: 129 pgs: 129 active+clean; 270 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2024-04-24T23:22:27.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:27 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: pgmap v664: 129 pgs: 129 active+clean; 270 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2024-04-24T23:22:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:27 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: pgmap v664: 129 pgs: 129 active+clean; 270 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2024-04-24T23:22:27.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:27 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:22:27.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1204296 12.55 MB/sec execute 392 sec latency 20.301 ms 2024-04-24T23:22:28.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1205828 12.53 MB/sec execute 393 sec latency 85.557 ms 2024-04-24T23:22:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:29 smithi038 ceph-mon[32450]: pgmap v665: 129 pgs: 129 active+clean; 273 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2024-04-24T23:22:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:29 smithi117 ceph-mon[32529]: pgmap v665: 129 pgs: 129 active+clean; 273 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2024-04-24T23:22:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:29 smithi033 ceph-mon[29201]: pgmap v665: 129 pgs: 129 active+clean; 273 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2024-04-24T23:22:29.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1208151 12.53 MB/sec execute 394 sec latency 17.724 ms 2024-04-24T23:22:30.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1210938 12.54 MB/sec execute 395 sec latency 17.444 ms 2024-04-24T23:22:31.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:31 smithi038 ceph-mon[32450]: pgmap v666: 129 pgs: 129 active+clean; 273 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-24T23:22:31.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:31 smithi117 ceph-mon[32529]: pgmap v666: 129 pgs: 129 active+clean; 273 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-24T23:22:31.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:31 smithi033 ceph-mon[29201]: pgmap v666: 129 pgs: 129 active+clean; 273 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-24T23:22:31.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1212735 12.53 MB/sec execute 396 sec latency 12.712 ms 2024-04-24T23:22:32.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1215216 12.53 MB/sec execute 397 sec latency 12.465 ms 2024-04-24T23:22:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:33 smithi038 ceph-mon[32450]: pgmap v667: 129 pgs: 129 active+clean; 277 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 289 op/s 2024-04-24T23:22:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:22:33.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:33 smithi117 ceph-mon[32529]: pgmap v667: 129 pgs: 129 active+clean; 277 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 289 op/s 2024-04-24T23:22:33.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:22:33.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:33 smithi033 ceph-mon[29201]: pgmap v667: 129 pgs: 129 active+clean; 277 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 289 op/s 2024-04-24T23:22:33.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:22:33.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1217141 12.52 MB/sec execute 398 sec latency 107.313 ms 2024-04-24T23:22:34.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:34 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:22:34.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:34 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:22:34.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:34 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:22:34.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1218995 12.52 MB/sec execute 399 sec latency 16.139 ms 2024-04-24T23:22:35.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:35 smithi038 ceph-mon[32450]: pgmap v668: 129 pgs: 129 active+clean; 275 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-04-24T23:22:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:35 smithi117 ceph-mon[32529]: pgmap v668: 129 pgs: 129 active+clean; 275 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-04-24T23:22:35.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:35 smithi033 ceph-mon[29201]: pgmap v668: 129 pgs: 129 active+clean; 275 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-04-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1221981 12.54 MB/sec execute 400 sec latency 18.716 ms 2024-04-24T23:22:36.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1224122 12.52 MB/sec execute 401 sec latency 15.337 ms 2024-04-24T23:22:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:37 smithi038 ceph-mon[32450]: pgmap v669: 129 pgs: 129 active+clean; 278 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-04-24T23:22:37.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:37 smithi117 ceph-mon[32529]: pgmap v669: 129 pgs: 129 active+clean; 278 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-04-24T23:22:37.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:37 smithi033 ceph-mon[29201]: pgmap v669: 129 pgs: 129 active+clean; 278 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-04-24T23:22:37.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1226035 12.52 MB/sec execute 402 sec latency 14.413 ms 2024-04-24T23:22:38.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1228508 12.52 MB/sec execute 403 sec latency 112.764 ms 2024-04-24T23:22:39.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:39 smithi038 ceph-mon[32450]: pgmap v670: 129 pgs: 129 active+clean; 279 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 250 op/s 2024-04-24T23:22:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:39 smithi117 ceph-mon[32529]: pgmap v670: 129 pgs: 129 active+clean; 279 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 250 op/s 2024-04-24T23:22:39.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:39 smithi033 ceph-mon[29201]: pgmap v670: 129 pgs: 129 active+clean; 279 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 250 op/s 2024-04-24T23:22:39.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1230026 12.51 MB/sec execute 404 sec latency 24.328 ms 2024-04-24T23:22:40.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1232807 12.52 MB/sec execute 405 sec latency 14.280 ms 2024-04-24T23:22:41.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:41 smithi038 ceph-mon[32450]: pgmap v671: 129 pgs: 129 active+clean; 279 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-04-24T23:22:41.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:41 smithi117 ceph-mon[32529]: pgmap v671: 129 pgs: 129 active+clean; 279 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-04-24T23:22:41.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:41 smithi033 ceph-mon[29201]: pgmap v671: 129 pgs: 129 active+clean; 279 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-04-24T23:22:41.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1235558 12.52 MB/sec execute 406 sec latency 14.414 ms 2024-04-24T23:22:42.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1237387 12.52 MB/sec execute 407 sec latency 12.835 ms 2024-04-24T23:22:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:43 smithi038 ceph-mon[32450]: pgmap v672: 129 pgs: 129 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-04-24T23:22:43.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:43 smithi117 ceph-mon[32529]: pgmap v672: 129 pgs: 129 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-04-24T23:22:43.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:43 smithi033 ceph-mon[29201]: pgmap v672: 129 pgs: 129 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-04-24T23:22:43.461 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1239784 12.52 MB/sec execute 408 sec latency 64.104 ms 2024-04-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1241931 12.51 MB/sec execute 409 sec latency 19.348 ms 2024-04-24T23:22:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:45 smithi038 ceph-mon[32450]: pgmap v673: 129 pgs: 129 active+clean; 284 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-04-24T23:22:45.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:45 smithi117 ceph-mon[32529]: pgmap v673: 129 pgs: 129 active+clean; 284 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-04-24T23:22:45.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:45 smithi033 ceph-mon[29201]: pgmap v673: 129 pgs: 129 active+clean; 284 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-04-24T23:22:45.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1243949 12.51 MB/sec execute 410 sec latency 14.722 ms 2024-04-24T23:22:46.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1246874 12.53 MB/sec execute 411 sec latency 31.786 ms 2024-04-24T23:22:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:47 smithi038 ceph-mon[32450]: pgmap v674: 129 pgs: 129 active+clean; 284 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-04-24T23:22:47.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:47 smithi117 ceph-mon[32529]: pgmap v674: 129 pgs: 129 active+clean; 284 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-04-24T23:22:47.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:47 smithi033 ceph-mon[29201]: pgmap v674: 129 pgs: 129 active+clean; 284 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-04-24T23:22:47.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1249292 12.52 MB/sec execute 412 sec latency 12.910 ms 2024-04-24T23:22:48.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1250858 12.51 MB/sec execute 413 sec latency 56.545 ms 2024-04-24T23:22:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:49 smithi038 ceph-mon[32450]: pgmap v675: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-04-24T23:22:49.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:49 smithi117 ceph-mon[32529]: pgmap v675: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-04-24T23:22:49.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:49 smithi033 ceph-mon[29201]: pgmap v675: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-04-24T23:22:49.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1253688 12.52 MB/sec execute 414 sec latency 15.867 ms 2024-04-24T23:22:50.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1255965 12.51 MB/sec execute 415 sec latency 12.941 ms 2024-04-24T23:22:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:51 smithi038 ceph-mon[32450]: pgmap v676: 129 pgs: 129 active+clean; 287 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 243 op/s 2024-04-24T23:22:51.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:51 smithi117 ceph-mon[32529]: pgmap v676: 129 pgs: 129 active+clean; 287 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 243 op/s 2024-04-24T23:22:51.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:51 smithi033 ceph-mon[29201]: pgmap v676: 129 pgs: 129 active+clean; 287 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 243 op/s 2024-04-24T23:22:51.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1257928 12.51 MB/sec execute 416 sec latency 17.465 ms 2024-04-24T23:22:52.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1260831 12.53 MB/sec execute 417 sec latency 27.109 ms 2024-04-24T23:22:53.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:53 smithi038 ceph-mon[32450]: pgmap v677: 129 pgs: 129 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 297 op/s 2024-04-24T23:22:53.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:53 smithi117 ceph-mon[32529]: pgmap v677: 129 pgs: 129 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 297 op/s 2024-04-24T23:22:53.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:53 smithi033 ceph-mon[29201]: pgmap v677: 129 pgs: 129 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 297 op/s 2024-04-24T23:22:53.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1261977 12.50 MB/sec execute 418 sec latency 112.265 ms 2024-04-24T23:22:54.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1264546 12.51 MB/sec execute 419 sec latency 21.930 ms 2024-04-24T23:22:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:55 smithi038 ceph-mon[32450]: pgmap v678: 129 pgs: 129 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-04-24T23:22:55.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:55 smithi117 ceph-mon[32529]: pgmap v678: 129 pgs: 129 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-04-24T23:22:55.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:55 smithi033 ceph-mon[29201]: pgmap v678: 129 pgs: 129 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-04-24T23:22:55.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1266752 12.50 MB/sec execute 420 sec latency 13.302 ms 2024-04-24T23:22:56.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:22:56.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:22:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3095504594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4104b90 msgr2=0x7f1cf4106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3095504594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4104b90 0x7f1cf4106f80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009900 tx=0x7f1ce802a410 comp rx=0 tx=0).stop 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3095504594 shutdown_connections 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3095504594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf41074c0 0x7f1cf41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3095504594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4104b90 0x7f1cf4106f80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3095504594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf4102260 0x7f1cf4104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.230+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3095504594 >> 172.21.15.33:0/3095504594 conn(0x7f1cf40fbbf0 msgr2=0x7f1cf40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3095504594 shutdown_connections 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3095504594 wait complete. 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 Processor -- start 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 -- start start 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 0x7f1cf4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf4104b90 0x7f1cf4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf41074c0 0x7f1cf419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410d440 con 0x7f1cf4102260 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410d2c0 con 0x7f1cf4104b90 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.231+0000 7f1cfa6ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410d5c0 con 0x7f1cf41074c0 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 0x7f1cf4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 0x7f1cf4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53090/0 (socket says 172.21.15.33:53090) 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3948706077 learned_addr learned my addr 172.21.15.33:0/3948706077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf4104b90 0x7f1cf4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf41074c0 0x7f1cf419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3948706077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf41074c0 msgr2=0x7f1cf419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf41074c0 0x7f1cf419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3948706077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf4104b90 msgr2=0x7f1cf4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf4104b90 0x7f1cf4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3948706077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf419f730 con 0x7f1cf4102260 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf4104b90 0x7f1cf4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf41074c0 0x7f1cf419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 0x7f1cf4197b10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9f0 tx=0x7f1ce400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf17fa640 1 -- 172.21.15.33:0/3948706077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400cd80 con 0x7f1cf4102260 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf17fa640 1 -- 172.21.15.33:0/3948706077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce4005500 con 0x7f1cf4102260 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf419fa20 con 0x7f1cf4102260 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cf17fa640 1 -- 172.21.15.33:0/3948706077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4010690 con 0x7f1cf4102260 2024-04-24T23:22:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.232+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf419fef0 con 0x7f1cf4102260 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf17fa640 1 -- 172.21.15.33:0/3948706077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1ce40046c0 con 0x7f1cf4102260 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 Processor -- start 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 -- start start 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599734f76b0 con 0x5599735931c0 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55997349f880 con 0x559973593bb0 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5599734f3670 con 0x559973596d10 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf8c8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf8c8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53100/0 (socket says 172.21.15.33:53100) 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf8c8b640 1 -- 172.21.15.33:0/2810049905 learned_addr learned my addr 172.21.15.33:0/2810049905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.235+0000 7f1cf17fa640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cdc0a2b90 0x7f1cdc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf37fe640 1 -- 172.21.15.33:0/2810049905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 msgr2=0x5599735966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf37fe640 1 -- 172.21.15.33:0/2810049905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 msgr2=0x5599735935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf37fe640 1 -- 172.21.15.33:0/2810049905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599732d57d0 con 0x559973596d10 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cdc0a2b90 0x7f1cdc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf17fa640 1 -- 172.21.15.33:0/3948706077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce40cf7f0 con 0x7f1cf4102260 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735a3b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009230 tx=0x7f1ce8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d1898d47f6ec80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cda7fc640 1 -- 172.21.15.33:0/2810049905 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8035040 con 0x559973596d10 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cda7fc640 1 -- 172.21.15.33:0/2810049905 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8004750 con 0x559973596d10 2024-04-24T23:22:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.236+0000 7f1cda7fc640 1 -- 172.21.15.33:0/2810049905 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8031ae0 con 0x559973596d10 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2810049905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 msgr2=0x5599735a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cdc0a2b90 0x7f1cdc0a5050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x559973594410 tx=0x7f1cec004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735a3b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009230 tx=0x7f1ce8005c00 comp rx=0 tx=0).stop 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2810049905 shutdown_connections 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735a3b80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2810049905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2810049905 >> 172.21.15.33:0/2810049905 conn(0x55997358e8b0 msgr2=0x55997358ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2810049905 shutdown_connections 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2810049905 wait complete. 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 -- start start 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.237+0000 7f1cfd3f9740 1 Processor -- start 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599735a4520 con 0x5599735931c0 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5599734b2f50 con 0x559973593bb0 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5599735a46a0 con 0x559973596d10 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 10 client.0 _ll_register_callbacks cb 0x5599734ee000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53114/0 (socket says 172.21.15.33:53114) 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3355250490 learned_addr learned my addr 172.21.15.33:0/3355250490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 msgr2=0x5599735ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735ad3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3355250490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 msgr2=0x5599735a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735a63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 -- 172.21.15.33:0/3355250490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55997320b3f0 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735a5e60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9c0 tx=0x7f1ce40056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168cd0c6938a45a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cd37fe640 1 -- 172.21.15.33:0/3355250490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce409b800 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cd37fe640 1 -- 172.21.15.33:0/3355250490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce40a46a0 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce8004750 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cd37fe640 1 -- 172.21.15.33:0/3355250490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ce409b620 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 10 client.17676 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.238+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599731cb0f0 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cd37fe640 1 -- 172.21.15.33:0/3355250490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8004750 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cd37fe640 1 -- 172.21.15.33:0/3355250490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1ce409b2a0 con 0x5599735931c0 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cfd3f9740 10 client.17676 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cfd3f9740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cfd3f9740 20 client.17676 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:56.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599735b1d70 0x5599735b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cfd3f9740 4 client.17676 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559973504bf0 con 0x5599735b1d70 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599735b1d70 0x5599735b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599735b1d70 0x5599735b4200 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce043dc4b8d8cc1c server_cookie=e2af4689b7c67ae7 in_seq=0 out_seq=0 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.239+0000 7f1cd37fe640 10 client.17676 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cd37fe640 1 -- 172.21.15.33:0/3355250490 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559973504bf0 con 0x5599735b1d70 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cd37fe640 10 client.17676 handle_command_reply: tid=0 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 client.17676 shutdown 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 msgr2=0x5599735a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735a5e60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9c0 tx=0x7f1ce40056b0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 shutdown_connections 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599735b1d70 0x5599735b4200 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973596d10 0x5599735ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cd37fe640 0 client.17676 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:56.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973593bb0 0x5599735a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/3355250490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735a5e60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 >> 172.21.15.33:0/3355250490 conn(0x55997358e8b0 msgr2=0x55997358daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 shutdown_connections 2024-04-24T23:22:56.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.240+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/3355250490 wait complete. 2024-04-24T23:22:56.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.241+0000 7f1cfd3f9740 20 client.17676 trim_cache size 0 max 16384 2024-04-24T23:22:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 Processor -- start 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 -- start start 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599735931c0 0x55997358d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 0x5599735ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41aa930 con 0x559973593bb0 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41a8200 con 0x559973596d10 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41a6050 con 0x5599735931c0 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 0x5599735ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 0x5599735ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53118/0 (socket says 172.21.15.33:53118) 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 -- 172.21.15.33:0/445416634 learned_addr learned my addr 172.21.15.33:0/445416634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf8c8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599735931c0 0x55997358d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 -- 172.21.15.33:0/445416634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599735931c0 msgr2=0x55997358d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599735931c0 0x55997358d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 -- 172.21.15.33:0/445416634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 msgr2=0x5599735b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf37fe640 1 -- 172.21.15.33:0/445416634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559973593f90 con 0x559973593bb0 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.259+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599735931c0 0x55997358d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.260+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 0x5599735ad3a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006010 tx=0x7f1ce8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c2d23ce3e9cef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.260+0000 7f1cd9ffb640 1 -- 172.21.15.33:0/445416634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8035040 con 0x559973593bb0 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.260+0000 7f1cd9ffb640 1 -- 172.21.15.33:0/445416634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8005520 con 0x559973593bb0 2024-04-24T23:22:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.260+0000 7f1cd9ffb640 1 -- 172.21.15.33:0/445416634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8004cb0 con 0x559973593bb0 2024-04-24T23:22:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.260+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/445416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 msgr2=0x5599735ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.260+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 0x5599735ad3a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006010 tx=0x7f1ce8005d50 comp rx=0 tx=0).stop 2024-04-24T23:22:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/445416634 shutdown_connections 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559973593bb0 0x5599735ad3a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/445416634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599735931c0 0x55997358d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/445416634 >> 172.21.15.33:0/445416634 conn(0x55997358e8b0 msgr2=0x55997358d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/445416634 shutdown_connections 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/445416634 wait complete. 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- start start 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 Processor -- start 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973593bb0 0x5599735aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41a66c0 con 0x5599735931c0 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41a7b30 con 0x559973596d10 2024-04-24T23:22:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41a6d90 con 0x559973593bb0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cfd3f9740 10 client.0 _ll_register_callbacks cb 0x7f1ce400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.261+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53134/0 (socket says 172.21.15.33:53134) 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 -- 172.21.15.33:0/2003470494 learned_addr learned my addr 172.21.15.33:0/2003470494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf8c8b640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973593bb0 0x5599735aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 -- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973593bb0 msgr2=0x5599735aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973593bb0 0x5599735aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 -- 172.21.15.33:0/2003470494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 msgr2=0x5599735abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 -- 172.21.15.33:0/2003470494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599735ac5e0 con 0x5599735931c0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cf3fff640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735aaaa0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4098120 tx=0x7f1ce4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be601a62354ad871 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cd37fe640 1 -- 172.21.15.33:0/2003470494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40a8070 con 0x5599735931c0 2024-04-24T23:22:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cd37fe640 1 -- 172.21.15.33:0/2003470494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce40a4b80 con 0x5599735931c0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce8005520 con 0x5599735931c0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cd37fe640 1 -- 172.21.15.33:0/2003470494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ce409b800 con 0x5599735931c0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cfd3f9740 10 client.17682 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559973618340 con 0x5599735931c0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.262+0000 7f1cd37fe640 1 -- 172.21.15.33:0/2003470494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8005520 con 0x5599735931c0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cd37fe640 1 -- 172.21.15.33:0/2003470494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1ce40ab020 con 0x5599735931c0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 10 client.17682 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 20 client.17682 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559973618ca0 0x55997361b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 4 client.17682 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599734fef40 con 0x559973618ca0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559973618ca0 0x55997361b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cf37fe640 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559973618ca0 0x55997361b0c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317fe2f9401dfeb2 server_cookie=a57ca0d11c54265a in_seq=0 out_seq=0 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cd37fe640 10 client.17682 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cd37fe640 1 -- 172.21.15.33:0/2003470494 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5599734fef40 con 0x559973618ca0 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cd37fe640 10 client.17682 handle_command_reply: tid=0 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.263+0000 7f1cfd3f9740 1 client.17682 shutdown 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 msgr2=0x5599735aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735aaaa0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4098120 tx=0x7f1ce4009310 comp rx=0 tx=0).stop 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 shutdown_connections 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559973618ca0 0x55997361b0c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559973596d10 0x5599735abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559973593bb0 0x5599735aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 --2- 172.21.15.33:0/2003470494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599735931c0 0x5599735aaaa0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cd37fe640 0 client.17682 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 >> 172.21.15.33:0/2003470494 conn(0x55997358e8b0 msgr2=0x5599735920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 shutdown_connections 2024-04-24T23:22:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 1 -- 172.21.15.33:0/2003470494 wait complete. 2024-04-24T23:22:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfd3f9740 20 client.17682 trim_cache size 0 max 16384 2024-04-24T23:22:56.272 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:22:56.272 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:22:56.272 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:22:56.272 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cdc0a2b90 msgr2=0x7f1cdc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cdc0a2b90 0x7f1cdc0a5050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x559973594410 tx=0x7f1cec004a40 comp rx=0 tx=0).stop 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 msgr2=0x7f1cf4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.264+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 0x7f1cf4197b10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9f0 tx=0x7f1ce400edf0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 shutdown_connections 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1cdc0a2b90 0x7f1cdc0a5050 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf41074c0 0x7f1cf419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf4104b90 0x7f1cf4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 --2- 172.21.15.33:0/3948706077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf4102260 0x7f1cf4197b10 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 >> 172.21.15.33:0/3948706077 conn(0x7f1cf40fbbf0 msgr2=0x7f1cf4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 shutdown_connections 2024-04-24T23:22:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.265+0000 7f1cfa6ee640 1 -- 172.21.15.33:0/3948706077 wait complete. 2024-04-24T23:22:56.277 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:22:56.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.388+0000 7fdc993fb640 1 -- 172.21.15.33:0/240854306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc941006a0 msgr2=0x7fdc94100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.388+0000 7fdc993fb640 1 --2- 172.21.15.33:0/240854306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc941006a0 0x7fdc94100ac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009900 tx=0x7fdc8402a410 comp rx=0 tx=0).stop 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- 172.21.15.33:0/240854306 shutdown_connections 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 --2- 172.21.15.33:0/240854306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc94102250 0x7fdc94108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 --2- 172.21.15.33:0/240854306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc94101890 0x7fdc94101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 --2- 172.21.15.33:0/240854306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc941006a0 0x7fdc94100ac0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- 172.21.15.33:0/240854306 >> 172.21.15.33:0/240854306 conn(0x7fdc940fbc30 msgr2=0x7fdc940fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- 172.21.15.33:0/240854306 shutdown_connections 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- 172.21.15.33:0/240854306 wait complete. 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 Processor -- start 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- start start 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc941006a0 0x7fdc94197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc94101890 0x7fdc94198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 0x7fdc9419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9410c320 con 0x7fdc94102250 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9410c1a0 con 0x7fdc94101890 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc993fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9410c4a0 con 0x7fdc941006a0 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 0x7fdc9419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc92ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc941006a0 0x7fdc94197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 0x7fdc9419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53152/0 (socket says 172.21.15.33:53152) 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc8bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc94101890 0x7fdc94198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc937fe640 1 -- 172.21.15.33:0/1426620503 learned_addr learned my addr 172.21.15.33:0/1426620503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.389+0000 7fdc92ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc941006a0 0x7fdc94197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:55472/0 (socket says 172.21.15.33:55472) 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc937fe640 1 -- 172.21.15.33:0/1426620503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc941006a0 msgr2=0x7fdc94197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc937fe640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc941006a0 0x7fdc94197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc937fe640 1 -- 172.21.15.33:0/1426620503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc94101890 msgr2=0x7fdc94198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc937fe640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc94101890 0x7fdc94198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc937fe640 1 -- 172.21.15.33:0/1426620503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9419f6c0 con 0x7fdc94102250 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc937fe640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 0x7fdc9419f050 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00eb10 tx=0x7fdc8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c004b20 con 0x7fdc94102250 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c004cc0 con 0x7fdc94102250 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c010760 con 0x7fdc94102250 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9419f9b0 con 0x7fdc94102250 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.390+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9419ff00 con 0x7fdc94102250 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.391+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc941a01f0 con 0x7fdc94102250 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.392+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdc8c005020 con 0x7fdc94102250 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.393+0000 7fdc90ff9640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdc800a2b90 0x7fdc800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.393+0000 7fdc92ffd640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdc800a2b90 0x7fdc800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.393+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc8c0cfad0 con 0x7fdc94102250 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.393+0000 7fdc92ffd640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdc800a2b90 0x7fdc800a5050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005ab0 tx=0x7fdc84000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.398+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc8c09c640 con 0x7fdc94102250 2024-04-24T23:22:56.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1268732 12.50 MB/sec execute 421 sec latency 16.227 ms 2024-04-24T23:22:56.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.673+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc941a03c0 con 0x7fdc94102250 2024-04-24T23:22:56.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.673+0000 7fdc90ff9640 1 -- 172.21.15.33:0/1426620503 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fdc8c09bf40 con 0x7fdc94102250 2024-04-24T23:22:56.675 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:22:56.675 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:22:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdc800a2b90 msgr2=0x7fdc800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdc800a2b90 0x7fdc800a5050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005ab0 tx=0x7fdc84000bc0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 msgr2=0x7fdc9419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 0x7fdc9419f050 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00eb10 tx=0x7fdc8c00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 shutdown_connections 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdc800a2b90 0x7fdc800a5050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc94102250 0x7fdc9419f050 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc94101890 0x7fdc94198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 --2- 172.21.15.33:0/1426620503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc941006a0 0x7fdc94197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 >> 172.21.15.33:0/1426620503 conn(0x7fdc940fbc30 msgr2=0x7fdc940fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 shutdown_connections 2024-04-24T23:22:56.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.676+0000 7fdc993fb640 1 -- 172.21.15.33:0/1426620503 wait complete. 2024-04-24T23:22:56.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/748100439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101870 msgr2=0x7f7b84101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/748100439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101870 0x7f7b84101cf0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009900 tx=0x7f7b7402a410 comp rx=0 tx=0).stop 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/748100439 shutdown_connections 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/748100439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84102230 0x7f7b84108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/748100439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101870 0x7f7b84101cf0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/748100439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b84100680 0x7f7b84100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/748100439 >> 172.21.15.33:0/748100439 conn(0x7f7b840fbbf0 msgr2=0x7f7b840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/748100439 shutdown_connections 2024-04-24T23:22:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.797+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/748100439 wait complete. 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 Processor -- start 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 -- start start 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 0x7f7b84069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b84101870 0x7f7b8419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84102230 0x7f7b8419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8410c350 con 0x7f7b84100680 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8410c1d0 con 0x7f7b84101870 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8bdc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8410c4d0 con 0x7f7b84102230 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 0x7f7b84069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 0x7f7b84069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53162/0 (socket says 172.21.15.33:53162) 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1527309420 learned_addr learned my addr 172.21.15.33:0/1527309420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8935c640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b84101870 0x7f7b8419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84102230 0x7f7b8419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1527309420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84102230 msgr2=0x7f7b8419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84102230 0x7f7b8419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.798+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1527309420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b84101870 msgr2=0x7f7b8419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b84101870 0x7f7b8419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1527309420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8419f710 con 0x7f7b84100680 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 0x7f7b84069440 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9c0 tx=0x7f7b8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b7affd640 1 -- 172.21.15.33:0/1527309420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000cd80 con 0x7f7b84100680 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b7affd640 1 -- 172.21.15.33:0/1527309420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b800035f0 con 0x7f7b84100680 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b7affd640 1 -- 172.21.15.33:0/1527309420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80010690 con 0x7f7b84100680 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8419f9a0 con 0x7f7b84100680 2024-04-24T23:22:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.799+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8419fe70 con 0x7f7b84100680 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.801+0000 7f7b7affd640 1 -- 172.21.15.33:0/1527309420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7b800037a0 con 0x7f7b84100680 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 Processor -- start 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 -- start start 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed56690650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed56695720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed56698220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ed565f4670 con 0x55ed56695d90 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ed5659c970 con 0x55ed56690c20 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ed565f0630 con 0x55ed56690230 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8935c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed56698220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b7affd640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b700a2b90 0x7f7b700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8935c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed56698220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53164/0 (socket says 172.21.15.33:53164) 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed56695720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.802+0000 7f7b8935c640 1 -- 172.21.15.33:0/4191673002 learned_addr learned my addr 172.21.15.33:0/4191673002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed56690650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b700a2b90 0x7f7b700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b7affd640 1 -- 172.21.15.33:0/1527309420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b800cf860 con 0x7f7b84100680 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b89b5d640 1 -- 172.21.15.33:0/4191673002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 msgr2=0x55ed56690650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed56690650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b89b5d640 1 -- 172.21.15.33:0/4191673002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 msgr2=0x55ed56698220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed56698220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b89b5d640 1 -- 172.21.15.33:0/4191673002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed565c0400 con 0x55ed56690c20 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b700a2b90 0x7f7b700a5050 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c005c40 tx=0x7f7b7c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed56695720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b800a4470 tx=0x7f7b800979a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba21f9fcb0c18640 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b6ffff640 1 -- 172.21.15.33:0/4191673002 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8009c430 con 0x55ed56690c20 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b6ffff640 1 -- 172.21.15.33:0/4191673002 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8009bd30 con 0x55ed56690c20 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.803+0000 7f7b6ffff640 1 -- 172.21.15.33:0/4191673002 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800a4510 con 0x55ed56690c20 2024-04-24T23:22:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4191673002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 msgr2=0x55ed56695720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed56695720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b800a4470 tx=0x7f7b800979a0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4191673002 shutdown_connections 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed56698220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed56695720 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4191673002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed56690650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4191673002 >> 172.21.15.33:0/4191673002 conn(0x55ed5668b8a0 msgr2=0x55ed5668bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4191673002 shutdown_connections 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4191673002 wait complete. 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 -- start start 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.804+0000 7f7b8ebac740 1 Processor -- start 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690230 0x55ed5669a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 0x55ed5669a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56695d90 0x55ed566a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ed56698ce0 con 0x55ed56690c20 2024-04-24T23:22:56.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ed565b0050 con 0x55ed56690230 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ed56698e60 con 0x55ed56695d90 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8ebac740 10 client.0 _ll_register_callbacks cb 0x55ed56307be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8935c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 0x55ed5669a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8935c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 0x55ed5669a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53166/0 (socket says 172.21.15.33:53166) 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8935c640 1 -- 172.21.15.33:0/4178178960 learned_addr learned my addr 172.21.15.33:0/4178178960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690230 0x55ed5669a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56695d90 0x55ed566a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 -- 172.21.15.33:0/4178178960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56695d90 msgr2=0x55ed566a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56695d90 0x55ed566a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 -- 172.21.15.33:0/4178178960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 msgr2=0x55ed5669a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 0x55ed5669a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 -- 172.21.15.33:0/4178178960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed56581dc0 con 0x55ed56690230 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b8935c640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 0x55ed5669a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690230 0x55ed5669a350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55ed56698990 tx=0x7f7b80097fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a379131435e126b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.805+0000 7f7b6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/4178178960 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8009a030 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/4178178960 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b8009b180 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8009bd30 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/4178178960 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b800a49a0 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 10 client.27227 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed563083f0 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/4178178960 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8009bd30 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/4178178960 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7b800961c0 con 0x55ed56690230 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 10 client.27227 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 10 client.27227 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 20 client.27227 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed566a5dc0 0x55ed566a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 4 client.27227 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed56601bb0 con 0x55ed566a5dc0 2024-04-24T23:22:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.806+0000 7f7b8935c640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed566a5dc0 0x55ed566a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.807+0000 7f7b8935c640 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed566a5dc0 0x55ed566a81b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f22da38cdbe6384f server_cookie=546d89c9752d4df2 in_seq=0 out_seq=0 2024-04-24T23:22:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.807+0000 7f7b6d7fa640 10 client.27227 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.808+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/4178178960 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed56601bb0 con 0x55ed566a5dc0 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.808+0000 7f7b6d7fa640 10 client.27227 handle_command_reply: tid=0 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 client.27227 shutdown 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690230 msgr2=0x55ed5669a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690230 0x55ed5669a350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55ed56698990 tx=0x7f7b80097fd0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 shutdown_connections 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed566a5dc0 0x55ed566a81b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56695d90 0x55ed566a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690c20 0x55ed5669a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/4178178960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690230 0x55ed5669a350 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b6d7fa640 0 client.27227 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 >> 172.21.15.33:0/4178178960 conn(0x55ed5668b8a0 msgr2=0x55ed5668ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 shutdown_connections 2024-04-24T23:22:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 1 -- 172.21.15.33:0/4178178960 wait complete. 2024-04-24T23:22:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.809+0000 7f7b8ebac740 20 client.27227 trim_cache size 0 max 16384 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.822+0000 7f7b8ebac740 1 Processor -- start 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.822+0000 7f7b8ebac740 1 -- start start 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 0x55ed5668e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690c20 0x55ed566a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56695d90 0x55ed5668ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841acc20 con 0x55ed56690230 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841aa4f0 con 0x55ed56695d90 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841a8340 con 0x55ed56690c20 2024-04-24T23:22:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 0x55ed5668e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 0x55ed5668e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53176/0 (socket says 172.21.15.33:53176) 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 -- 172.21.15.33:0/460562815 learned_addr learned my addr 172.21.15.33:0/460562815 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56695d90 0x55ed5668ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b8935c640 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690c20 0x55ed566a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 -- 172.21.15.33:0/460562815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690c20 msgr2=0x55ed566a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690c20 0x55ed566a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 -- 172.21.15.33:0/460562815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56695d90 msgr2=0x55ed5668ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56695d90 0x55ed5668ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 -- 172.21.15.33:0/460562815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed563d27d0 con 0x55ed56690230 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 0x55ed5668e780 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c6a0 tx=0x7f7b80003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0a6bde0e38daf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/460562815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8009c430 con 0x55ed56690230 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/460562815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8009bd30 con 0x55ed56690230 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.823+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/460562815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800ba740 con 0x55ed56690230 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 -- 172.21.15.33:0/460562815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 msgr2=0x55ed5668e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 0x55ed5668e780 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c6a0 tx=0x7f7b80003e30 comp rx=0 tx=0).stop 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 -- 172.21.15.33:0/460562815 shutdown_connections 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56695d90 0x55ed5668ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690c20 0x55ed566a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/460562815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56690230 0x55ed5668e780 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 -- 172.21.15.33:0/460562815 >> 172.21.15.33:0/460562815 conn(0x55ed5668b8a0 msgr2=0x55ed5668ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 -- 172.21.15.33:0/460562815 shutdown_connections 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 -- 172.21.15.33:0/460562815 wait complete. 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.824+0000 7f7b8ebac740 1 -- start start 2024-04-24T23:22:56.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 Processor -- start 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed5669ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed5669f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed566a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841a89b0 con 0x55ed56695d90 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841a9e20 con 0x55ed56690c20 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841a9080 con 0x55ed56690230 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8ebac740 10 client.0 _ll_register_callbacks cb 0x7f7b7c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:56.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8a35e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed566a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed5669ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed5669ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:55520/0 (socket says 172.21.15.33:55520) 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1423938986 learned_addr learned my addr 172.21.15.33:0/1423938986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8935c640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed5669f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 msgr2=0x55ed5669f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed5669f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1423938986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 msgr2=0x55ed566a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed566a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b89b5d640 1 -- 172.21.15.33:0/1423938986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed566a0710 con 0x55ed56690230 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.825+0000 7f7b8a35e640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed566a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b89b5d640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed5669ebd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7b800986b0 tx=0x7f7b8009b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d726ac8c7b7d334b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/1423938986 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80097fd0 con 0x55ed56690230 2024-04-24T23:22:56.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8009bd30 con 0x55ed56690230 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/1423938986 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b8009b180 con 0x55ed56690230 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/1423938986 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b8009c430 con 0x55ed56690230 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 10 client.27190 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed566a0710 con 0x55ed56690230 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/1423938986 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8009bd30 con 0x55ed56690230 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/1423938986 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7b800a1270 con 0x55ed56690230 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 10 client.27190 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 10 client.27190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 20 client.27190 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed5670c380 0x55ed5670e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 4 client.27190 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed565fbf00 con 0x55ed5670c380 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.826+0000 7f7b8935c640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed5670c380 0x55ed5670e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.827+0000 7f7b6d7fa640 10 client.27190 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.827+0000 7f7b8935c640 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed5670c380 0x55ed5670e770 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61687f4d215adf6b server_cookie=c80eedfdf81d231f in_seq=0 out_seq=0 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b6d7fa640 1 -- 172.21.15.33:0/1423938986 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed565fbf00 con 0x55ed5670c380 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b6d7fa640 10 client.27190 handle_command_reply: tid=0 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 client.27190 shutdown 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 msgr2=0x55ed5669ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed5669ebd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7b800986b0 tx=0x7f7b8009b930 comp rx=0 tx=0).stop 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 shutdown_connections 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ed5670c380 0x55ed5670e770 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed56695d90 0x55ed566a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed56690c20 0x55ed5669f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b6d7fa640 0 client.27190 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 --2- 172.21.15.33:0/1423938986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed56690230 0x55ed5669ebd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 >> 172.21.15.33:0/1423938986 conn(0x55ed5668b8a0 msgr2=0x55ed5668e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 shutdown_connections 2024-04-24T23:22:56.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 1 -- 172.21.15.33:0/1423938986 wait complete. 2024-04-24T23:22:56.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.828+0000 7f7b8ebac740 20 client.27190 trim_cache size 0 max 16384 2024-04-24T23:22:56.836 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b700a2b90 msgr2=0x7f7b700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b700a2b90 0x7f7b700a5050 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c005c40 tx=0x7f7b7c009500 comp rx=0 tx=0).stop 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 msgr2=0x7f7b84069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 0x7f7b84069440 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9c0 tx=0x7f7b8000ee80 comp rx=0 tx=0).stop 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 shutdown_connections 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b700a2b90 0x7f7b700a5050 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84102230 0x7f7b8419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b84101870 0x7f7b8419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 --2- 172.21.15.33:0/1527309420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84100680 0x7f7b84069440 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 >> 172.21.15.33:0/1527309420 conn(0x7f7b840fbbf0 msgr2=0x7f7b84104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 shutdown_connections 2024-04-24T23:22:56.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.829+0000 7f7b8bdc1640 1 -- 172.21.15.33:0/1527309420 wait complete. 2024-04-24T23:22:56.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 -- 172.21.15.33:0/2763448527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 msgr2=0x7f96a41046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 --2- 172.21.15.33:0/2763448527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a41046d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009900 tx=0x7f96a002a410 comp rx=0 tx=0).stop 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 -- 172.21.15.33:0/2763448527 shutdown_connections 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 --2- 172.21.15.33:0/2763448527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4107540 0x7f96a4109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 --2- 172.21.15.33:0/2763448527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96a4104c10 0x7f96a4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 --2- 172.21.15.33:0/2763448527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a41046d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 -- 172.21.15.33:0/2763448527 >> 172.21.15.33:0/2763448527 conn(0x7f96a40fbc30 msgr2=0x7f96a40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 -- 172.21.15.33:0/2763448527 shutdown_connections 2024-04-24T23:22:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.946+0000 7f96ac4da640 1 -- 172.21.15.33:0/2763448527 wait complete. 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 Processor -- start 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 -- start start 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a4101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4104c10 0x7f96a4100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96a4107540 0x7f96a41007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d420 con 0x7f96a41022e0 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d2a0 con 0x7f96a4107540 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96ac4da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d5a0 con 0x7f96a4104c10 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96aa276640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a4101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96aa276640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a4101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53184/0 (socket says 172.21.15.33:53184) 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96aa276640 1 -- 172.21.15.33:0/599459646 learned_addr learned my addr 172.21.15.33:0/599459646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96aaa77640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96a4107540 0x7f96a41007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.947+0000 7f96a9a75640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4104c10 0x7f96a4100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96aa276640 1 -- 172.21.15.33:0/599459646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4104c10 msgr2=0x7f96a4100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96aa276640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4104c10 0x7f96a4100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96aa276640 1 -- 172.21.15.33:0/599459646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96a4107540 msgr2=0x7f96a41007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96aa276640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96a4107540 0x7f96a41007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96aa276640 1 -- 172.21.15.33:0/599459646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a4101060 con 0x7f96a41022e0 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96aa276640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a4101c50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009230 tx=0x7f96a0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f969b7fe640 1 -- 172.21.15.33:0/599459646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a003f030 con 0x7f96a41022e0 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f969b7fe640 1 -- 172.21.15.33:0/599459646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a0003ad0 con 0x7f96a41022e0 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f969b7fe640 1 -- 172.21.15.33:0/599459646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0035070 con 0x7f96a41022e0 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41a1be0 con 0x7f96a41022e0 2024-04-24T23:22:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.948+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a20e0 con 0x7f96a41022e0 2024-04-24T23:22:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.950+0000 7f969b7fe640 1 -- 172.21.15.33:0/599459646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f96a0042030 con 0x7f96a41022e0 2024-04-24T23:22:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 Processor -- start 2024-04-24T23:22:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 -- start start 2024-04-24T23:22:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25350720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f25353220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557f252af670 con 0x557f2534bc20 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557f25257970 con 0x557f2534b230 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96af1ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f252ab630 con 0x557f25350d90 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f969b7fe640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96900a2b90 0x7f96900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96a9a75640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f25353220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96a9a75640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f25353220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:55540/0 (socket says 172.21.15.33:55540) 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.951+0000 7f96a9a75640 1 -- 172.21.15.33:0/1103379092 learned_addr learned my addr 172.21.15.33:0/1103379092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96aa276640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25350720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96aaa77640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96aaa77640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96900a2b90 0x7f96900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96a9a75640 1 -- 172.21.15.33:0/1103379092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 msgr2=0x557f2534b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96a9a75640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f969b7fe640 1 -- 172.21.15.33:0/599459646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96a00ef960 con 0x7f96a41022e0 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96a9a75640 1 -- 172.21.15.33:0/1103379092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 msgr2=0x557f25350720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96a9a75640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25350720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96a9a75640 1 -- 172.21.15.33:0/1103379092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2527b400 con 0x557f25350d90 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96aaa77640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96aa276640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25350720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96a9a75640 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f25353220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9694009bf0 tx=0x7f96940117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8a21ea794ca37d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f96aaa77640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96900a2b90 0x7f96900a5050 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f969c009fd0 tx=0x7f969c0096a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f9698bfe640 1 -- 172.21.15.33:0/1103379092 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694004030 con 0x557f25350d90 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f9698bfe640 1 -- 172.21.15.33:0/1103379092 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940041d0 con 0x557f25350d90 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.952+0000 7f9698bfe640 1 -- 172.21.15.33:0/1103379092 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940126b0 con 0x557f25350d90 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 -- 172.21.15.33:0/1103379092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 msgr2=0x557f25353220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f25353220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9694009bf0 tx=0x7f96940117b0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 -- 172.21.15.33:0/1103379092 shutdown_connections 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f25353220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25350720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1103379092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 -- 172.21.15.33:0/1103379092 >> 172.21.15.33:0/1103379092 conn(0x557f253468a0 msgr2=0x557f25346ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 -- 172.21.15.33:0/1103379092 shutdown_connections 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.953+0000 7f96af1ee740 1 -- 172.21.15.33:0/1103379092 wait complete. 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 -- start start 2024-04-24T23:22:56.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 Processor -- start 2024-04-24T23:22:56.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f253552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25355810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f25350d90 0x557f2535c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557f25353d40 con 0x557f2534bc20 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557f2526b050 con 0x557f25350d90 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f25353ec0 con 0x557f2534b230 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96af1ee740 10 client.0 _ll_register_callbacks cb 0x557f24fc2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25355810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96aaa77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f25350d90 0x557f2535c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25355810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53200/0 (socket says 172.21.15.33:53200) 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 -- 172.21.15.33:0/1675662770 learned_addr learned my addr 172.21.15.33:0/1675662770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96aa276640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f253552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 -- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 msgr2=0x557f253552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f253552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 -- 172.21.15.33:0/1675662770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f25350d90 msgr2=0x557f2535c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f25350d90 0x557f2535c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96a9a75640 1 -- 172.21.15.33:0/1675662770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2523cdc0 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.954+0000 7f96aaa77640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f25350d90 0x557f2535c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96a9a75640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25355810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9694000c00 tx=0x7f9694009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920d97d5aca28592 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f968dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f968dffb640 1 -- 172.21.15.33:0/1675662770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969401f040 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f968dffb640 1 -- 172.21.15.33:0/1675662770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96940037f0 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96940041d0 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f968dffb640 1 -- 172.21.15.33:0/1675662770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9694032050 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 10 client.17718 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f24fc33f0 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f968dffb640 1 -- 172.21.15.33:0/1675662770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940041d0 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f968dffb640 1 -- 172.21.15.33:0/1675662770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9694004b00 con 0x557f2534bc20 2024-04-24T23:22:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 10 client.17718 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 10 client.17718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 20 client.17718 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.955+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f25360d90 0x557f253631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.956+0000 7f96af1ee740 4 client.17718 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.956+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f252bcbb0 con 0x557f25360d90 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.956+0000 7f96aa276640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f25360d90 0x557f253631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.956+0000 7f96aa276640 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f25360d90 0x557f253631c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c8f381e8b204073 server_cookie=6c5270857f6a8e72 in_seq=0 out_seq=0 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.956+0000 7f968dffb640 10 client.17718 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f968dffb640 1 -- 172.21.15.33:0/1675662770 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f252bcbb0 con 0x557f25360d90 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f968dffb640 10 client.17718 handle_command_reply: tid=0 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 client.17718 shutdown 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 msgr2=0x557f25355810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25355810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9694000c00 tx=0x7f9694009910 comp rx=0 tx=0).stop 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 shutdown_connections 2024-04-24T23:22:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f25360d90 0x557f253631c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f25350d90 0x557f2535c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f968dffb640 0 client.17718 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f25355810 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 --2- 172.21.15.33:0/1675662770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f253552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 >> 172.21.15.33:0/1675662770 conn(0x557f253468a0 msgr2=0x557f25345b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 shutdown_connections 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 1 -- 172.21.15.33:0/1675662770 wait complete. 2024-04-24T23:22:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.957+0000 7f96af1ee740 20 client.17718 trim_cache size 0 max 16384 2024-04-24T23:22:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.971+0000 7f96af1ee740 1 Processor -- start 2024-04-24T23:22:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.971+0000 7f96af1ee740 1 -- start start 2024-04-24T23:22:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f2535c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f2534c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96af1ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41aeed0 con 0x557f2534bc20 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96af1ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ac7a0 con 0x557f2534b230 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96af1ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41aa5f0 con 0x557f25350d90 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39894/0 (socket says 172.21.15.33:39894) 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96a9a75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f2535c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 -- 172.21.15.33:0/4041547113 learned_addr learned my addr 172.21.15.33:0/4041547113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aaa77640 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f2534c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 -- 172.21.15.33:0/4041547113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 msgr2=0x557f2534c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f2534c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 -- 172.21.15.33:0/4041547113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 msgr2=0x557f2535c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f2535c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 -- 172.21.15.33:0/4041547113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2508d7d0 con 0x557f2534b230 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f96aa276640 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534c4b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b8290 tx=0x7f96a00f1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa122d27f65f4134 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f968ffff640 1 -- 172.21.15.33:0/4041547113 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0049040 con 0x557f2534b230 2024-04-24T23:22:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f968ffff640 1 -- 172.21.15.33:0/4041547113 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a00c01e0 con 0x557f2534b230 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.972+0000 7f968ffff640 1 -- 172.21.15.33:0/4041547113 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00b84a0 con 0x557f2534b230 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 -- 172.21.15.33:0/4041547113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 msgr2=0x557f2534c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534c4b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b8290 tx=0x7f96a00f1040 comp rx=0 tx=0).stop 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 -- 172.21.15.33:0/4041547113 shutdown_connections 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f25350d90 0x557f2534c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f2534bc20 0x557f2535c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 --2- 172.21.15.33:0/4041547113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534b230 0x557f2534c4b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 -- 172.21.15.33:0/4041547113 >> 172.21.15.33:0/4041547113 conn(0x557f253468a0 msgr2=0x557f25345b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 -- 172.21.15.33:0/4041547113 shutdown_connections 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 -- 172.21.15.33:0/4041547113 wait complete. 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.973+0000 7f96af1ee740 1 -- start start 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 Processor -- start 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f25359b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534bc20 0x557f2535a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 0x557f2535b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41aac60 con 0x557f25350d90 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ac0d0 con 0x557f2534bc20 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ab330 con 0x557f2534b230 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96af1ee740 10 client.0 _ll_register_callbacks cb 0x7f96a00b7a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 0x557f2535b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 0x557f2535b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53218/0 (socket says 172.21.15.33:53218) 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 -- 172.21.15.33:0/2995924820 learned_addr learned my addr 172.21.15.33:0/2995924820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aa276640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f25359b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96a9a75640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534bc20 0x557f2535a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 -- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 msgr2=0x557f25359b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f25359b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 -- 172.21.15.33:0/2995924820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534bc20 msgr2=0x557f2535a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534bc20 0x557f2535a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 -- 172.21.15.33:0/2995924820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2535b6a0 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96a9a75640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534bc20 0x557f2535a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aa276640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f25359b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f96aaa77640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 0x557f2535b0b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f969c0042b0 tx=0x7f969c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4231a1768830bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.974+0000 7f968dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f968dffb640 1 -- 172.21.15.33:0/2995924820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c018070 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f968dffb640 1 -- 172.21.15.33:0/2995924820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f969c00dad0 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a00c01e0 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f968dffb640 1 -- 172.21.15.33:0/2995924820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f969c00dcd0 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 10 client.17724 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f2535ca70 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f968dffb640 1 -- 172.21.15.33:0/2995924820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a00c01e0 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f968dffb640 1 -- 172.21.15.33:0/2995924820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f969c0104f0 con 0x557f25350d90 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 10 client.17724 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 10 client.17724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 20 client.17724 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f253c7a80 0x557f253c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 4 client.17724 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f253550c0 con 0x557f253c7a80 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.975+0000 7f96aa276640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f253c7a80 0x557f253c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f968dffb640 10 client.17724 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96aa276640 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f253c7a80 0x557f253c9ea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cefd7cb98c2244e server_cookie=388e07d0870eb114 in_seq=0 out_seq=0 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f968dffb640 1 -- 172.21.15.33:0/2995924820 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f253550c0 con 0x557f253c7a80 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f968dffb640 10 client.17724 handle_command_reply: tid=0 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 client.17724 shutdown 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 msgr2=0x557f2535b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 0x557f2535b0b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f969c0042b0 tx=0x7f969c004a40 comp rx=0 tx=0).stop 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 shutdown_connections 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x557f253c7a80 0x557f253c9ea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f25350d90 0x557f2535b0b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2534bc20 0x557f2535a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f968dffb640 0 client.17724 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 --2- 172.21.15.33:0/2995924820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f2534b230 0x557f25359b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 >> 172.21.15.33:0/2995924820 conn(0x557f253468a0 msgr2=0x557f25349780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 shutdown_connections 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 1 -- 172.21.15.33:0/2995924820 wait complete. 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.976+0000 7f96af1ee740 20 client.17724 trim_cache size 0 max 16384 2024-04-24T23:22:56.984 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96900a2b90 msgr2=0x7f96900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96900a2b90 0x7f96900a5050 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f969c009fd0 tx=0x7f969c0096a0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 msgr2=0x7f96a4101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a4101c50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009230 tx=0x7f96a0031b60 comp rx=0 tx=0).stop 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 shutdown_connections 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96900a2b90 0x7f96900a5050 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96a4107540 0x7f96a41007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4104c10 0x7f96a4100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 --2- 172.21.15.33:0/599459646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a41022e0 0x7f96a4101c50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 >> 172.21.15.33:0/599459646 conn(0x7f96a40fbc30 msgr2=0x7f96a4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 shutdown_connections 2024-04-24T23:22:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:56.977+0000 7f96ac4da640 1 -- 172.21.15.33:0/599459646 wait complete. 2024-04-24T23:22:56.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4211598093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 msgr2=0x7f2b58109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4211598093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58109be0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009ca0 tx=0x7f2b5002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4211598093 shutdown_connections 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4211598093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58109be0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4211598093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4211598093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b581017a0 0x7f2b58101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4211598093 >> 172.21.15.33:0/4211598093 conn(0x7f2b580fcd10 msgr2=0x7f2b580ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4211598093 shutdown_connections 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4211598093 wait complete. 2024-04-24T23:22:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.095+0000 7f2b5e15a640 1 Processor -- start 2024-04-24T23:22:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 -- start start 2024-04-24T23:22:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b581017a0 0x7f2b580754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5810d420 con 0x7f2b58103350 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5810d2a0 con 0x7f2b58102990 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b5e15a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5810d5a0 con 0x7f2b581017a0 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39920/0 (socket says 172.21.15.33:39920) 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53246/0 (socket says 172.21.15.33:53246) 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 -- 172.21.15.33:0/4187568735 learned_addr learned my addr 172.21.15.33:0/4187568735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b577fe640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b581017a0 0x7f2b580754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 -- 172.21.15.33:0/4187568735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b581017a0 msgr2=0x7f2b580754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b581017a0 0x7f2b580754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 -- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 msgr2=0x7f2b58074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 -- 172.21.15.33:0/4187568735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b58074900 con 0x7f2b58102990 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b57fff640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.096+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58073b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00e9f0 tx=0x7f2b4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.097+0000 7f2b54ff9640 1 -- 172.21.15.33:0/4187568735 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00cd80 con 0x7f2b58102990 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.097+0000 7f2b54ff9640 1 -- 172.21.15.33:0/4187568735 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c010910 con 0x7f2b58102990 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.097+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b581a1b80 con 0x7f2b58102990 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.097+0000 7f2b54ff9640 1 -- 172.21.15.33:0/4187568735 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c018990 con 0x7f2b58102990 2024-04-24T23:22:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.097+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581a2120 con 0x7f2b58102990 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b54ff9640 1 -- 172.21.15.33:0/4187568735 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2b4c010ac0 con 0x7f2b58102990 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 Processor -- start 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 -- start start 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44fee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44ff3d90 0x55bb44ff6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bb44f52670 con 0x55bb44fee230 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bb44efa970 con 0x55bb44feec20 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.099+0000 7f2b60e66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb44f4e630 con 0x55bb44ff3d90 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44fee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b54ff9640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2b400a2c10 0x7f2b400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44fee650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53250/0 (socket says 172.21.15.33:53250) 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b57fff640 1 -- 172.21.15.33:0/658885738 learned_addr learned my addr 172.21.15.33:0/658885738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44ff3d90 0x55bb44ff6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b57fff640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2b400a2c10 0x7f2b400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b54ff9640 1 -- 172.21.15.33:0/4187568735 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b4c0cf680 con 0x7f2b58102990 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 -- 172.21.15.33:0/658885738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44ff3d90 msgr2=0x55bb44ff6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44ff3d90 0x55bb44ff6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 -- 172.21.15.33:0/658885738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 msgr2=0x55bb44fee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44fee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 -- 172.21.15.33:0/658885738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb44f1e400 con 0x55bb44feec20 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b57fff640 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44fee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b577fe640 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff3720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009bd0 tx=0x7f2b48011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4192bda30b7683fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b57fff640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2b400a2c10 0x7f2b400a50d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55bb44fef480 tx=0x7f2b5002a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b3dffb640 1 -- 172.21.15.33:0/658885738 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004230 con 0x55bb44feec20 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b3dffb640 1 -- 172.21.15.33:0/658885738 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48009e40 con 0x55bb44feec20 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.100+0000 7f2b3dffb640 1 -- 172.21.15.33:0/658885738 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b480197e0 con 0x55bb44feec20 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 -- 172.21.15.33:0/658885738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 msgr2=0x55bb44ff3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff3720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009bd0 tx=0x7f2b48011810 comp rx=0 tx=0).stop 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 -- 172.21.15.33:0/658885738 shutdown_connections 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44ff3d90 0x55bb44ff6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff3720 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 --2- 172.21.15.33:0/658885738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44fee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 -- 172.21.15.33:0/658885738 >> 172.21.15.33:0/658885738 conn(0x55bb44fe98a0 msgr2=0x55bb44fe9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 -- 172.21.15.33:0/658885738 shutdown_connections 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 -- 172.21.15.33:0/658885738 wait complete. 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.101+0000 7f2b60e66740 1 -- start start 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 Processor -- start 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44fee230 0x55bb44ff8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 0x55bb44fff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bb44ff6d40 con 0x55bb44ff3d90 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bb44f0e050 con 0x55bb44feec20 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb44ff6ec0 con 0x55bb44fee230 2024-04-24T23:22:57.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b60e66740 10 client.0 _ll_register_callbacks cb 0x55bb44c65be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 0x55bb44fff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44fee230 0x55bb44ff8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 0x55bb44fff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53260/0 (socket says 172.21.15.33:53260) 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 -- 172.21.15.33:0/3282070264 learned_addr learned my addr 172.21.15.33:0/3282070264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 -- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44fee230 msgr2=0x55bb44ff8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44fee230 0x55bb44ff8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 -- 172.21.15.33:0/3282070264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 msgr2=0x55bb44ff8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b57fff640 1 -- 172.21.15.33:0/3282070264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb44edfdc0 con 0x55bb44ff3d90 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.102+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b57fff640 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 0x55bb44fff890 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2b50006010 tx=0x7f2b500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2946ad779932f8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b36ffd640 1 -- 172.21.15.33:0/3282070264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5004f070 con 0x55bb44ff3d90 2024-04-24T23:22:57.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b36ffd640 1 -- 172.21.15.33:0/3282070264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b500305c0 con 0x55bb44ff3d90 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb44f5fbb0 con 0x55bb44ff3d90 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b36ffd640 1 -- 172.21.15.33:0/3282070264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2b500307c0 con 0x55bb44ff3d90 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 10 client.17736 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb44c663f0 con 0x55bb44ff3d90 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b36ffd640 1 -- 172.21.15.33:0/3282070264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bb44f5fbb0 con 0x55bb44ff3d90 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b36ffd640 1 -- 172.21.15.33:0/3282070264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2b500056d0 con 0x55bb44ff3d90 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 10 client.17736 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 20 client.17736 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb45004160 0x55bb450065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 4 client.17736 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.103+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb44f59f00 con 0x55bb45004160 2024-04-24T23:22:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.104+0000 7f2b577fe640 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb45004160 0x55bb450065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.104+0000 7f2b577fe640 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb45004160 0x55bb450065c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=775fe3c39346a3b4 server_cookie=980160cea4f2504d in_seq=0 out_seq=0 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.104+0000 7f2b36ffd640 10 client.17736 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b36ffd640 1 -- 172.21.15.33:0/3282070264 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb44f59f00 con 0x55bb45004160 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b36ffd640 10 client.17736 handle_command_reply: tid=0 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 client.17736 shutdown 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 msgr2=0x55bb44fff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 0x55bb44fff890 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2b50006010 tx=0x7f2b500095c0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 shutdown_connections 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb45004160 0x55bb450065c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44ff3d90 0x55bb44fff890 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44feec20 0x55bb44ff8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b36ffd640 0 client.17736 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 --2- 172.21.15.33:0/3282070264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44fee230 0x55bb44ff8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 >> 172.21.15.33:0/3282070264 conn(0x55bb44fe98a0 msgr2=0x55bb44fe8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 shutdown_connections 2024-04-24T23:22:57.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 1 -- 172.21.15.33:0/3282070264 wait complete. 2024-04-24T23:22:57.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.105+0000 7f2b60e66740 20 client.17736 trim_cache size 0 max 16384 2024-04-24T23:22:57.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.119+0000 7f2b60e66740 1 Processor -- start 2024-04-24T23:22:57.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 -- start start 2024-04-24T23:22:57.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ff3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44fff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44fec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581aee70 con 0x55bb44fee230 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581ac740 con 0x55bb44ff3d90 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b60e66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581aa590 con 0x55bb44feec20 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ff3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ff3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53262/0 (socket says 172.21.15.33:53262) 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 -- 172.21.15.33:0/492190156 learned_addr learned my addr 172.21.15.33:0/492190156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b57fff640 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44fec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44fff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 -- 172.21.15.33:0/492190156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 msgr2=0x55bb44fff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44fff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 -- 172.21.15.33:0/492190156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 msgr2=0x55bb44fec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44fec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 -- 172.21.15.33:0/492190156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb44ff3ae0 con 0x55bb44fee230 2024-04-24T23:22:57.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.120+0000 7f2b577fe640 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ff3310 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2b480041a0 tx=0x7f2b48011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a7ed077cd7ad42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b3d7fa640 1 -- 172.21.15.33:0/492190156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48030020 con 0x55bb44fee230 2024-04-24T23:22:57.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b3d7fa640 1 -- 172.21.15.33:0/492190156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4802cb50 con 0x55bb44fee230 2024-04-24T23:22:57.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b3d7fa640 1 -- 172.21.15.33:0/492190156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4802c8b0 con 0x55bb44fee230 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 -- 172.21.15.33:0/492190156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 msgr2=0x55bb44ff3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ff3310 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2b480041a0 tx=0x7f2b48011810 comp rx=0 tx=0).stop 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 -- 172.21.15.33:0/492190156 shutdown_connections 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44fec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44fff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 --2- 172.21.15.33:0/492190156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ff3310 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 -- 172.21.15.33:0/492190156 >> 172.21.15.33:0/492190156 conn(0x55bb44fe98a0 msgr2=0x55bb44fe8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 -- 172.21.15.33:0/492190156 shutdown_connections 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.121+0000 7f2b60e66740 1 -- 172.21.15.33:0/492190156 wait complete. 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 -- start start 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 Processor -- start 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ffd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44ffd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44ffe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581aac00 con 0x55bb44fee230 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581ac070 con 0x55bb44ff3d90 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581ab2d0 con 0x55bb44feec20 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b60e66740 10 client.0 _ll_register_callbacks cb 0x7f2b50009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ffd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ffd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53278/0 (socket says 172.21.15.33:53278) 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b577fe640 1 -- 172.21.15.33:0/1824395698 learned_addr learned my addr 172.21.15.33:0/1824395698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b57fff640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44ffe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.122+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44ffd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b577fe640 1 -- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 msgr2=0x55bb44ffd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b577fe640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44ffd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b577fe640 1 -- 172.21.15.33:0/1824395698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 msgr2=0x55bb44ffe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b577fe640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44ffe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b577fe640 1 -- 172.21.15.33:0/1824395698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb44d307d0 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b57fff640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44ffe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b577fe640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ffd090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009bd0 tx=0x7f2b480096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e29fbde34081799 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b36ffd640 1 -- 172.21.15.33:0/1824395698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4803c070 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b36ffd640 1 -- 172.21.15.33:0/1824395698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b480052d0 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b4802cb50 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b36ffd640 1 -- 172.21.15.33:0/1824395698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2b48047050 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b60e66740 10 client.17742 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb4506a740 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.123+0000 7f2b36ffd640 1 -- 172.21.15.33:0/1824395698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4802cb50 con 0x55bb44fee230 2024-04-24T23:22:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b36ffd640 1 -- 172.21.15.33:0/1824395698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2b48035450 con 0x55bb44fee230 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 10 client.17742 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 20 client.17742 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb4506ade0 0x55bb4506d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 4 client.17742 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb44ff8100 con 0x55bb4506ade0 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb4506ade0 0x55bb4506d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b36ffd640 10 client.17742 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b56ffd640 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb4506ade0 0x55bb4506d1d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8593ade3322e03 server_cookie=5f8d28465a6b207d in_seq=0 out_seq=0 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b36ffd640 1 -- 172.21.15.33:0/1824395698 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb44ff8100 con 0x55bb4506ade0 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b36ffd640 10 client.17742 handle_command_reply: tid=0 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.124+0000 7f2b60e66740 1 client.17742 shutdown 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 msgr2=0x55bb44ffd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ffd090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009bd0 tx=0x7f2b480096d0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 shutdown_connections 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb4506ade0 0x55bb4506d1d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb44ff3d90 0x55bb44ffe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb44feec20 0x55bb44ffd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 --2- 172.21.15.33:0/1824395698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb44fee230 0x55bb44ffd090 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 >> 172.21.15.33:0/1824395698 conn(0x55bb44fe98a0 msgr2=0x55bb44fef6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b36ffd640 0 client.17742 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 shutdown_connections 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 1 -- 172.21.15.33:0/1824395698 wait complete. 2024-04-24T23:22:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b60e66740 20 client.17742 trim_cache size 0 max 16384 2024-04-24T23:22:57.134 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:22:57.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2b400a2c10 msgr2=0x7f2b400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2b400a2c10 0x7f2b400a50d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55bb44fef480 tx=0x7f2b5002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 msgr2=0x7f2b58073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58073b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00e9f0 tx=0x7f2b4c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 shutdown_connections 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2b400a2c10 0x7f2b400a50d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b58103350 0x7f2b58074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b58102990 0x7f2b58073b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 --2- 172.21.15.33:0/4187568735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b581017a0 0x7f2b580754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.125+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 >> 172.21.15.33:0/4187568735 conn(0x7f2b580fcd10 msgr2=0x7f2b58105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.126+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 shutdown_connections 2024-04-24T23:22:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:57.126+0000 7f2b5e15a640 1 -- 172.21.15.33:0/4187568735 wait complete. 2024-04-24T23:22:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:57 smithi038 ceph-mon[32450]: pgmap v679: 129 pgs: 129 active+clean; 273 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-24T23:22:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:57 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1426620503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:22:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:57 smithi117 ceph-mon[32529]: pgmap v679: 129 pgs: 129 active+clean; 273 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-24T23:22:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:57 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1426620503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:22:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:57 smithi033 ceph-mon[29201]: pgmap v679: 129 pgs: 129 active+clean; 273 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-24T23:22:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:57 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1426620503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:22:57.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1271508 12.51 MB/sec execute 422 sec latency 38.876 ms 2024-04-24T23:22:58.139 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:22:58.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:58.140 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 -- 172.21.15.33:0/368219658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c101870 msgr2=0x7f011c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 --2- 172.21.15.33:0/368219658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c101870 0x7f011c101cf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0110009900 tx=0x7f011002a410 comp rx=0 tx=0).stop 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 -- 172.21.15.33:0/368219658 shutdown_connections 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 --2- 172.21.15.33:0/368219658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f011c102230 0x7f011c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 --2- 172.21.15.33:0/368219658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c101870 0x7f011c101cf0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 --2- 172.21.15.33:0/368219658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f011c100680 0x7f011c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 -- 172.21.15.33:0/368219658 >> 172.21.15.33:0/368219658 conn(0x7f011c0fbbf0 msgr2=0x7f011c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 -- 172.21.15.33:0/368219658 shutdown_connections 2024-04-24T23:22:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 -- 172.21.15.33:0/368219658 wait complete. 2024-04-24T23:22:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.248+0000 7f0121f3b640 1 Processor -- start 2024-04-24T23:22:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 -- start start 2024-04-24T23:22:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 0x7f011c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f011c101870 0x7f011c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f011c102230 0x7f011c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f011c10c2e0 con 0x7f011c100680 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f011c10c160 con 0x7f011c102230 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f0121f3b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f011c10c460 con 0x7f011c101870 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 0x7f011c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 0x7f011c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53292/0 (socket says 172.21.15.33:53292) 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 -- 172.21.15.33:0/4054404639 learned_addr learned my addr 172.21.15.33:0/4054404639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011affd640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f011c101870 0x7f011c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011bfff640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f011c102230 0x7f011c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 -- 172.21.15.33:0/4054404639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f011c101870 msgr2=0x7f011c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f011c101870 0x7f011c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 -- 172.21.15.33:0/4054404639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f011c102230 msgr2=0x7f011c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f011c102230 0x7f011c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.249+0000 7f011b7fe640 1 -- 172.21.15.33:0/4054404639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c19f730 con 0x7f011c100680 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.250+0000 7f011b7fe640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 0x7f011c069440 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e9c0 tx=0x7f010c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.250+0000 7f0118ff9640 1 -- 172.21.15.33:0/4054404639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00cd80 con 0x7f011c100680 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.250+0000 7f0118ff9640 1 -- 172.21.15.33:0/4054404639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c0035f0 con 0x7f011c100680 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.250+0000 7f0118ff9640 1 -- 172.21.15.33:0/4054404639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c010690 con 0x7f011c100680 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.250+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c19f9c0 con 0x7f011c100680 2024-04-24T23:22:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.250+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c19fee0 con 0x7f011c100680 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.252+0000 7f0118ff9640 1 -- 172.21.15.33:0/4054404639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f010c0037a0 con 0x7f011c100680 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.252+0000 7f0124c46740 1 Processor -- start 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.252+0000 7f0124c46740 1 -- start start 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0124c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0124c46740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b06280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0124c46740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b08d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0124c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f718a642c0 con 0x55f718b00d90 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0124c46740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f718a0c910 con 0x55f718b068f0 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0124c46740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f718a60280 con 0x55f718b01780 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53308/0 (socket says 172.21.15.33:53308) 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 -- 172.21.15.33:0/3178284966 learned_addr learned my addr 172.21.15.33:0/3178284966 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011b7fe640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b06280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011affd640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b08d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0118ff9640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01040a2b90 0x7f01040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 -- 172.21.15.33:0/3178284966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 msgr2=0x55f718b06280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b06280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 -- 172.21.15.33:0/3178284966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 msgr2=0x55f718b08d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b08d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 -- 172.21.15.33:0/3178284966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f718a1dfb0 con 0x55f718b00d90 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011affd640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b08d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011b7fe640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b06280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0118ff9640 1 -- 172.21.15.33:0/4054404639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f010c0cf7b0 con 0x7f011c100680 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01040a2b90 0x7f01040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f011bfff640 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b011b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0114011590 tx=0x7f0114011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e5014dcfdf5bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.253+0000 7f0101ffb640 1 -- 172.21.15.33:0/3178284966 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114017050 con 0x55f718b00d90 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0101ffb640 1 -- 172.21.15.33:0/3178284966 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400fd20 con 0x55f718b00d90 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0101ffb640 1 -- 172.21.15.33:0/3178284966 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011401bb30 con 0x55f718b00d90 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f011bfff640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01040a2b90 0x7f01040a5050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0114022000 tx=0x7f0114023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 -- 172.21.15.33:0/3178284966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 msgr2=0x55f718b011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b011b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0114011590 tx=0x7f0114011a50 comp rx=0 tx=0).stop 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 -- 172.21.15.33:0/3178284966 shutdown_connections 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b08d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b06280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 --2- 172.21.15.33:0/3178284966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b011b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 -- 172.21.15.33:0/3178284966 >> 172.21.15.33:0/3178284966 conn(0x55f718ac8950 msgr2=0x55f718afc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 -- 172.21.15.33:0/3178284966 shutdown_connections 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.254+0000 7f0124c46740 1 -- 172.21.15.33:0/3178284966 wait complete. 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 -- start start 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 Processor -- start 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b00d90 0x55f718b0b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 0x55f718b0b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 0x55f718b12730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f718b098a0 con 0x55f718b068f0 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f718a1fe90 con 0x55f718b01780 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f718b09a20 con 0x55f718b00d90 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f0124c46740 10 client.0 _ll_register_callbacks cb 0x55f718a5b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f011bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 0x55f718b12730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f011affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 0x55f718b0b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b00d90 0x55f718b0b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f011affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 0x55f718b0b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39972/0 (socket says 172.21.15.33:39972) 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f011bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 0x55f718b12730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53312/0 (socket says 172.21.15.33:53312) 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.255+0000 7f011affd640 1 -- 172.21.15.33:0/2541730979 learned_addr learned my addr 172.21.15.33:0/2541730979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011affd640 1 -- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b00d90 msgr2=0x55f718b0b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011affd640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b00d90 0x55f718b0b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011affd640 1 -- 172.21.15.33:0/2541730979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 msgr2=0x55f718b12730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011affd640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 0x55f718b12730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011affd640 1 -- 172.21.15.33:0/2541730979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7188407e0 con 0x55f718b01780 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011b7fe640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b00d90 0x55f718b0b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011bfff640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 0x55f718b12730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f011affd640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 0x55f718b0b6f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f011002a930 tx=0x7f0110009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a37ece6b26612ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f00faffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:22:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f00faffd640 1 -- 172.21.15.33:0/2541730979 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035040 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f00faffd640 1 -- 172.21.15.33:0/2541730979 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0110004040 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011400fd20 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f00faffd640 1 -- 172.21.15.33:0/2541730979 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f011004f050 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f0124c46740 10 client.27281 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7188a0f70 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.256+0000 7f00faffd640 1 -- 172.21.15.33:0/2541730979 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400fd20 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f00faffd640 1 -- 172.21.15.33:0/2541730979 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0110004940 con 0x55f718b01780 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f0124c46740 10 client.27281 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f0124c46740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f0124c46740 20 client.27281 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f0124c46740 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b16cc0 0x55f718b190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f0124c46740 4 client.27281 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f718a385a0 con 0x55f718b16cc0 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f011b7fe640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b16cc0 0x55f718b190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f00faffd640 10 client.27281 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.257+0000 7f011b7fe640 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b16cc0 0x55f718b190d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfbcaf1efa4eb7f server_cookie=d9be567bd749e3af in_seq=0 out_seq=0 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f00faffd640 1 -- 172.21.15.33:0/2541730979 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f718a385a0 con 0x55f718b16cc0 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f00faffd640 10 client.27281 handle_command_reply: tid=0 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 client.27281 shutdown 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 msgr2=0x55f718b0b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 0x55f718b0b6f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f011002a930 tx=0x7f0110009670 comp rx=0 tx=0).stop 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 shutdown_connections 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b16cc0 0x55f718b190d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b068f0 0x55f718b12730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b01780 0x55f718b0b6f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 --2- 172.21.15.33:0/2541730979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b00d90 0x55f718b0b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f00faffd640 0 client.27281 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 >> 172.21.15.33:0/2541730979 conn(0x55f718ac8950 msgr2=0x55f718afb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 shutdown_connections 2024-04-24T23:22:58.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 1 -- 172.21.15.33:0/2541730979 wait complete. 2024-04-24T23:22:58.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.258+0000 7f0124c46740 20 client.27281 trim_cache size 0 max 16384 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.289+0000 7f0124c46740 1 Processor -- start 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.289+0000 7f0124c46740 1 -- start start 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.289+0000 7f0124c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b02be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.289+0000 7f0124c46740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b03120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.289+0000 7f0124c46740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f0124c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1acd00 con 0x55f718b00d90 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f0124c46740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1aa5d0 con 0x55f718b068f0 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f0124c46740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1a8420 con 0x55f718b01780 2024-04-24T23:22:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b02be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b02be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53314/0 (socket says 172.21.15.33:53314) 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 -- 172.21.15.33:0/18755468 learned_addr learned my addr 172.21.15.33:0/18755468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 -- 172.21.15.33:0/18755468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 msgr2=0x55f718b03120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b03120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 -- 172.21.15.33:0/18755468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 msgr2=0x55f718b126f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b126f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.290+0000 7f011b7fe640 1 -- 172.21.15.33:0/18755468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f718738170 con 0x55f718b00d90 2024-04-24T23:22:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.291+0000 7f011b7fe640 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b02be0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f010c09b980 tx=0x7f010c0cf9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75ca1874774511 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.291+0000 7f01017fa640 1 -- 172.21.15.33:0/18755468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c09bc80 con 0x55f718b00d90 2024-04-24T23:22:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.292+0000 7f01017fa640 1 -- 172.21.15.33:0/18755468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c0a0030 con 0x55f718b00d90 2024-04-24T23:22:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.292+0000 7f01017fa640 1 -- 172.21.15.33:0/18755468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c09b330 con 0x55f718b00d90 2024-04-24T23:22:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.293+0000 7f0124c46740 1 -- 172.21.15.33:0/18755468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 msgr2=0x55f718b02be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.293+0000 7f0124c46740 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b02be0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f010c09b980 tx=0x7f010c0cf9b0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.294+0000 7f0124c46740 1 -- 172.21.15.33:0/18755468 shutdown_connections 2024-04-24T23:22:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.294+0000 7f0124c46740 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b068f0 0x55f718b126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.294+0000 7f0124c46740 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b01780 0x55f718b03120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.294+0000 7f0124c46740 1 --2- 172.21.15.33:0/18755468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b00d90 0x55f718b02be0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.294+0000 7f0124c46740 1 -- 172.21.15.33:0/18755468 >> 172.21.15.33:0/18755468 conn(0x55f718ac8950 msgr2=0x55f718b0b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.301+0000 7f0124c46740 1 -- 172.21.15.33:0/18755468 shutdown_connections 2024-04-24T23:22:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.302+0000 7f0124c46740 1 -- 172.21.15.33:0/18755468 wait complete. 2024-04-24T23:22:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.302+0000 7f0124c46740 1 -- start start 2024-04-24T23:22:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.302+0000 7f0124c46740 1 Processor -- start 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b00d90 0x55f718b0fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b01780 0x55f718b10320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b068f0 0x55f718b11330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1a8a90 con 0x55f718b01780 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1a9f00 con 0x55f718b00d90 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1a9160 con 0x55f718b068f0 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f0124c46740 10 client.0 _ll_register_callbacks cb 0x7f0110009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b00d90 0x55f718b0fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b00d90 0x55f718b0fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39984/0 (socket says 172.21.15.33:39984) 2024-04-24T23:22:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.311+0000 7f011b7fe640 1 -- 172.21.15.33:0/2033147882 learned_addr learned my addr 172.21.15.33:0/2033147882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.312+0000 7f011affd640 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b01780 0x55f718b10320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.314+0000 7f011affd640 1 -- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b068f0 msgr2=0x55f718b11330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:22:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.314+0000 7f011affd640 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b068f0 0x55f718b11330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.314+0000 7f011affd640 1 -- 172.21.15.33:0/2033147882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b00d90 msgr2=0x55f718b0fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.314+0000 7f011affd640 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b00d90 0x55f718b0fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.314+0000 7f011affd640 1 -- 172.21.15.33:0/2033147882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f718b11920 con 0x55f718b01780 2024-04-24T23:22:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.315+0000 7f011affd640 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b01780 0x55f718b10320 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0110004d40 tx=0x7f0110004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1d7f048e86b453 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.319+0000 7f00faffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.319+0000 7f00faffd640 1 -- 172.21.15.33:0/2033147882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110058060 con 0x55f718b01780 2024-04-24T23:22:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.319+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010c0a0030 con 0x55f718b01780 2024-04-24T23:22:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.319+0000 7f00faffd640 1 -- 172.21.15.33:0/2033147882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0110031bd0 con 0x55f718b01780 2024-04-24T23:22:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.324+0000 7f00faffd640 1 -- 172.21.15.33:0/2033147882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0110035030 con 0x55f718b01780 2024-04-24T23:22:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.326+0000 7f0124c46740 10 client.17760 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:58.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.326+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f718b85a20 con 0x55f718b01780 2024-04-24T23:22:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.329+0000 7f00faffd640 1 -- 172.21.15.33:0/2033147882 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c0a0030 con 0x55f718b01780 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.330+0000 7f00faffd640 1 -- 172.21.15.33:0/2033147882 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f01100050c0 con 0x55f718b01780 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f0124c46740 10 client.17760 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f0124c46740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f0124c46740 20 client.17760 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f0124c46740 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b86280 0x55f718b886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f0124c46740 4 client.17760 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:22:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f718a71800 con 0x55f718b86280 2024-04-24T23:22:58.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.331+0000 7f011b7fe640 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b86280 0x55f718b886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.335+0000 7f011b7fe640 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b86280 0x55f718b886a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95dfb7f54dd032be server_cookie=4f3434611be61ee1 in_seq=0 out_seq=0 2024-04-24T23:22:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.335+0000 7f00faffd640 10 client.17760 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.339+0000 7f00faffd640 1 -- 172.21.15.33:0/2033147882 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f718a71800 con 0x55f718b86280 2024-04-24T23:22:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.339+0000 7f00faffd640 10 client.17760 handle_command_reply: tid=0 2024-04-24T23:22:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.339+0000 7f0124c46740 1 client.17760 shutdown 2024-04-24T23:22:58.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.343+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b01780 msgr2=0x55f718b10320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.343+0000 7f0124c46740 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b01780 0x55f718b10320 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0110004d40 tx=0x7f0110004f50 comp rx=0 tx=0).stop 2024-04-24T23:22:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 shutdown_connections 2024-04-24T23:22:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f0124c46740 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f718b86280 0x55f718b886a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f0124c46740 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f718b068f0 0x55f718b11330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f0124c46740 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f718b01780 0x55f718b10320 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f0124c46740 1 --2- 172.21.15.33:0/2033147882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f718b00d90 0x55f718b0fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 >> 172.21.15.33:0/2033147882 conn(0x55f718ac8950 msgr2=0x55f718b0b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.347+0000 7f00faffd640 0 client.17760 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.349+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 shutdown_connections 2024-04-24T23:22:58.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.350+0000 7f0124c46740 1 -- 172.21.15.33:0/2033147882 wait complete. 2024-04-24T23:22:58.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.350+0000 7f0124c46740 20 client.17760 trim_cache size 0 max 16384 2024-04-24T23:22:58.353 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:22:58.353 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:22:58.353 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "e05a96df-58d3-4eb4-89bd-4f91039c1496", 2024-04-24T23:22:58.353 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:22:58.353 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:22:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.352+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01040a2b90 msgr2=0x7f01040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.352+0000 7f0121f3b640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01040a2b90 0x7f01040a5050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0114022000 tx=0x7f0114023000 comp rx=0 tx=0).stop 2024-04-24T23:22:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.352+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 msgr2=0x7f011c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.352+0000 7f0121f3b640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 0x7f011c069440 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e9c0 tx=0x7f010c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 shutdown_connections 2024-04-24T23:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01040a2b90 0x7f01040a5050 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f011c102230 0x7f011c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f011c101870 0x7f011c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 --2- 172.21.15.33:0/4054404639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c100680 0x7f011c069440 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 >> 172.21.15.33:0/4054404639 conn(0x7f011c0fbbf0 msgr2=0x7f011c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.354+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 shutdown_connections 2024-04-24T23:22:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.355+0000 7f0121f3b640 1 -- 172.21.15.33:0/4054404639 wait complete. 2024-04-24T23:22:58.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:22:58.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:22:58.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1272640 12.49 MB/sec execute 423 sec latency 92.291 ms 2024-04-24T23:22:58.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.475+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3043700026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 msgr2=0x7f1e60100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.475+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3043700026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60100aa0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009900 tx=0x7f1e5802a410 comp rx=0 tx=0).stop 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3043700026 shutdown_connections 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3043700026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e60102230 0x7f1e60108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3043700026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60101870 0x7f1e60101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3043700026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60100aa0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3043700026 >> 172.21.15.33:0/3043700026 conn(0x7f1e600fbbf0 msgr2=0x7f1e600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3043700026 shutdown_connections 2024-04-24T23:22:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3043700026 wait complete. 2024-04-24T23:22:58.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 Processor -- start 2024-04-24T23:22:58.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.476+0000 7f1e64ed2640 1 -- start start 2024-04-24T23:22:58.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e64ed2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e64ed2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e60101870 0x7f1e601980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e64ed2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60102230 0x7f1e6019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e64ed2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010c340 con 0x7f1e60100680 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e64ed2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010c1c0 con 0x7f1e60101870 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e64ed2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010c4c0 con 0x7f1e60102230 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53336/0 (socket says 172.21.15.33:53336) 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 -- 172.21.15.33:0/3219298039 learned_addr learned my addr 172.21.15.33:0/3219298039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60102230 0x7f1e6019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e60101870 0x7f1e601980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 -- 172.21.15.33:0/3219298039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60102230 msgr2=0x7f1e6019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60102230 0x7f1e6019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 -- 172.21.15.33:0/3219298039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e60101870 msgr2=0x7f1e601980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e60101870 0x7f1e601980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 -- 172.21.15.33:0/3219298039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6019f780 con 0x7f1e60100680 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60102230 0x7f1e6019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:58.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.477+0000 7f1e5e59c640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60197b60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f1e58031440 tx=0x7f1e58032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.478+0000 7f1e577fe640 1 -- 172.21.15.33:0/3219298039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5803f030 con 0x7f1e60100680 2024-04-24T23:22:58.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.478+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6019fa10 con 0x7f1e60100680 2024-04-24T23:22:58.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.478+0000 7f1e577fe640 1 -- 172.21.15.33:0/3219298039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e58003ce0 con 0x7f1e60100680 2024-04-24T23:22:58.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.478+0000 7f1e577fe640 1 -- 172.21.15.33:0/3219298039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58035070 con 0x7f1e60100680 2024-04-24T23:22:58.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.478+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6019ff50 con 0x7f1e60100680 2024-04-24T23:22:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.480+0000 7f1e67bdd740 1 Processor -- start 2024-04-24T23:22:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 -- start start 2024-04-24T23:22:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 0x55d76dcb3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb8da0 0x55d76dcbb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d76dc176b0 con 0x55d76dcb3c30 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d76dbbf880 con 0x55d76dcb3240 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d76dc13670 con 0x55d76dcb8da0 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e577fe640 1 -- 172.21.15.33:0/3219298039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1e58042030 con 0x7f1e60100680 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 0x55d76dcb3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 0x55d76dcb3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40004/0 (socket says 172.21.15.33:40004) 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/1998017478 learned_addr learned my addr 172.21.15.33:0/1998017478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb8da0 0x55d76dcbb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5e59c640 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/1998017478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb8da0 msgr2=0x55d76dcbb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb8da0 0x55d76dcbb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/1998017478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 msgr2=0x55d76dcb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/1998017478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76d9f57d0 con 0x55d76dcb3240 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.481+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb8da0 0x55d76dcbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e577fe640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e4c0a2b90 0x7f1e4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 0x55d76dcb3660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e500107e0 tx=0x7f1e50010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8844a4e284473f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e5e59c640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e4c0a2b90 0x7f1e4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e54ff9640 1 -- 172.21.15.33:0/1998017478 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50003b70 con 0x55d76dcb3240 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e54ff9640 1 -- 172.21.15.33:0/1998017478 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5000ee10 con 0x55d76dcb3240 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e54ff9640 1 -- 172.21.15.33:0/1998017478 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50004770 con 0x55d76dcb3240 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.482+0000 7f1e577fe640 1 -- 172.21.15.33:0/3219298039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1e580ef2d0 con 0x7f1e60100680 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1998017478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 msgr2=0x55d76dcb3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e5e59c640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e4c0a2b90 0x7f1e4c0a5050 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e580ecdb0 tx=0x7f1e580f2db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 0x55d76dcb3660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e500107e0 tx=0x7f1e50010ca0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1998017478 shutdown_connections 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb8da0 0x55d76dcbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1998017478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb3240 0x55d76dcb3660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1998017478 >> 172.21.15.33:0/1998017478 conn(0x55d76dcae8b0 msgr2=0x55d76dcaecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1998017478 shutdown_connections 2024-04-24T23:22:58.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1998017478 wait complete. 2024-04-24T23:22:58.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 -- start start 2024-04-24T23:22:58.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.483+0000 7f1e67bdd740 1 Processor -- start 2024-04-24T23:22:58.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcbd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcbdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d76dcbbd50 con 0x55d76dcb3c30 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d76dbd2f50 con 0x55d76dcb8da0 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d76dcbbed0 con 0x55d76dcb3240 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e67bdd740 10 client.0 _ll_register_callbacks cb 0x55d76dc0e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcbd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcbdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcbd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:55694/0 (socket says 172.21.15.33:55694) 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/1367996915 learned_addr learned my addr 172.21.15.33:0/1367996915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 msgr2=0x55d76dcbd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcbd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/1367996915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 msgr2=0x55d76dcc4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.484+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/1367996915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76d92b3f0 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcbdbe0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1e60199070 tx=0x7f1e4800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f963fba840c85b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e45ffb640 1 -- 172.21.15.33:0/1367996915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48003d90 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e45ffb640 1 -- 172.21.15.33:0/1367996915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1e48005020 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e5000ee10 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e45ffb640 1 -- 172.21.15.33:0/1367996915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e48005220 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e67bdd740 10 client.17772 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d76d8eb0f0 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.485+0000 7f1e45ffb640 1 -- 172.21.15.33:0/1367996915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5000ee10 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e45ffb640 1 -- 172.21.15.33:0/1367996915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e4801b500 con 0x55d76dcb3c30 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e67bdd740 10 client.17772 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e67bdd740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e67bdd740 20 client.17772 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dcc9600 0x55d76dccba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e67bdd740 4 client.17772 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d76dc24bf0 con 0x55d76dcc9600 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dcc9600 0x55d76dccba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e45ffb640 10 client.17772 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.486+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dcc9600 0x55d76dccba20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3fd32af185e363 server_cookie=c510bac6866ceb82 in_seq=0 out_seq=0 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e45ffb640 1 -- 172.21.15.33:0/1367996915 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d76dc24bf0 con 0x55d76dcc9600 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e45ffb640 10 client.17772 handle_command_reply: tid=0 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 client.17772 shutdown 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 msgr2=0x55d76dcbdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcbdbe0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1e60199070 tx=0x7f1e4800bd80 comp rx=0 tx=0).stop 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 shutdown_connections 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dcc9600 0x55d76dccba20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcbdbe0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/1367996915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcbd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e45ffb640 0 client.17772 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 >> 172.21.15.33:0/1367996915 conn(0x55d76dcae8b0 msgr2=0x55d76dcadaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 shutdown_connections 2024-04-24T23:22:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 1 -- 172.21.15.33:0/1367996915 wait complete. 2024-04-24T23:22:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.487+0000 7f1e67bdd740 20 client.17772 trim_cache size 0 max 16384 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 Processor -- start 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 -- start start 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601accf0 con 0x55d76dcb3c30 2024-04-24T23:22:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aa5c0 con 0x55d76dcb8da0 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.506+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a8410 con 0x55d76dcb3240 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53352/0 (socket says 172.21.15.33:53352) 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/2548536000 learned_addr learned my addr 172.21.15.33:0/2548536000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/2548536000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 msgr2=0x55d76dcb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/2548536000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 msgr2=0x55d76dcc4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 -- 172.21.15.33:0/2548536000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76dca0090 con 0x55d76dcb3c30 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb5570 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001eb0 tx=0x7f1e480041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f04ba04e28eb5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e47fff640 1 -- 172.21.15.33:0/2548536000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4802c070 con 0x55d76dcb3c30 2024-04-24T23:22:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e47fff640 1 -- 172.21.15.33:0/2548536000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e48003e10 con 0x55d76dcb3c30 2024-04-24T23:22:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.507+0000 7f1e47fff640 1 -- 172.21.15.33:0/2548536000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48028930 con 0x55d76dcb3c30 2024-04-24T23:22:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2548536000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 msgr2=0x55d76dcb5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb5570 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001eb0 tx=0x7f1e480041d0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2548536000 shutdown_connections 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3c30 0x55d76dcb5570 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2548536000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3240 0x55d76dcb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2548536000 >> 172.21.15.33:0/2548536000 conn(0x55d76dcae8b0 msgr2=0x55d76dcbd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2548536000 shutdown_connections 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2548536000 wait complete. 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 -- start start 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.508+0000 7f1e67bdd740 1 Processor -- start 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 0x55d76dcc1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3c30 0x55d76dcc2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a8a80 con 0x55d76dcb3240 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a9ef0 con 0x55d76dcb8da0 2024-04-24T23:22:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a9150 con 0x55d76dcb3c30 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 0x55d76dcc1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e67bdd740 10 client.0 _ll_register_callbacks cb 0x7f1e48004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 0x55d76dcc1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53366/0 (socket says 172.21.15.33:53366) 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/2349309867 learned_addr learned my addr 172.21.15.33:0/2349309867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3c30 0x55d76dcc2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3c30 msgr2=0x55d76dcc2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3c30 0x55d76dcc2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/2349309867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 msgr2=0x55d76dcc3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 -- 172.21.15.33:0/2349309867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76dcc3b20 con 0x55d76dcb3240 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3c30 0x55d76dcc2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e5ed9d640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 0x55d76dcc1fe0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1e50010700 tx=0x7f1e48004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42baaea6d156faac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.509+0000 7f1e45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e45ffb640 1 -- 172.21.15.33:0/2349309867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50025040 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e45ffb640 1 -- 172.21.15.33:0/2349309867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1e50003b90 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e48003e10 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e45ffb640 1 -- 172.21.15.33:0/2349309867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e5003f050 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 10 client.17778 fetch_fsmap learned FSMap version 27 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d76dcc4e30 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e45ffb640 1 -- 172.21.15.33:0/2349309867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e48003e10 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e45ffb640 1 -- 172.21.15.33:0/2349309867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e5002f0c0 con 0x55d76dcb3240 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 10 client.17778 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 20 client.17778 populate_metadata read hostname 'smithi033' 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dd30140 0x55d76dd32560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 4 client.17778 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d76dc1ef40 con 0x55d76dd30140 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.510+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dd30140 0x55d76dd32560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e45ffb640 10 client.17778 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e5dd9b640 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dd30140 0x55d76dd32560 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995beecddfc8da66 server_cookie=350ee49f754c6e54 in_seq=0 out_seq=0 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e45ffb640 1 -- 172.21.15.33:0/2349309867 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d76dc1ef40 con 0x55d76dd30140 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e45ffb640 10 client.17778 handle_command_reply: tid=0 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 client.17778 shutdown 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 msgr2=0x55d76dcc1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 0x55d76dcc1fe0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1e50010700 tx=0x7f1e48004490 comp rx=0 tx=0).stop 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 shutdown_connections 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d76dd30140 0x55d76dd32560 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d76dcb8da0 0x55d76dcc3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d76dcb3c30 0x55d76dcc2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e45ffb640 0 client.17778 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:22:58.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 --2- 172.21.15.33:0/2349309867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d76dcb3240 0x55d76dcc1fe0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 >> 172.21.15.33:0/2349309867 conn(0x55d76dcae8b0 msgr2=0x55d76dcb5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 shutdown_connections 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 1 -- 172.21.15.33:0/2349309867 wait complete. 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.511+0000 7f1e67bdd740 20 client.17778 trim_cache size 0 max 16384 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stdout: "e05a96df-58d3-4eb4-89bd-4f91039c1496": { 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:22:58.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "e05a96df-58d3-4eb4-89bd-4f91039c1496", 2024-04-24T23:22:58.521 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:22:58.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:22:58.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:22:58.521 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e4c0a2b90 msgr2=0x7f1e4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e4c0a2b90 0x7f1e4c0a5050 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e580ecdb0 tx=0x7f1e580f2db0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 msgr2=0x7f1e60197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60197b60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f1e58031440 tx=0x7f1e58032e20 comp rx=0 tx=0).stop 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 shutdown_connections 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e4c0a2b90 0x7f1e4c0a5050 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e60102230 0x7f1e6019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e60101870 0x7f1e601980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 --2- 172.21.15.33:0/3219298039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e60100680 0x7f1e60197b60 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 >> 172.21.15.33:0/3219298039 conn(0x7f1e600fbbf0 msgr2=0x7f1e600fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.512+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 shutdown_connections 2024-04-24T23:22:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:22:58.513+0000 7f1e64ed2640 1 -- 172.21.15.33:0/3219298039 wait complete. 2024-04-24T23:22:58.533 INFO:tasks.cephfs.filesystem:scrub status for tag:e05a96df-58d3-4eb4-89bd-4f91039c1496 - {'path': '/', 'tag': 'e05a96df-58d3-4eb4-89bd-4f91039c1496', 'options': 'recursive,force'} 2024-04-24T23:22:59.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:22:59 smithi038 ceph-mon[32450]: pgmap v680: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 262 op/s 2024-04-24T23:22:59.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:22:59 smithi117 ceph-mon[32529]: pgmap v680: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 262 op/s 2024-04-24T23:22:59.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:22:59 smithi033 ceph-mon[29201]: pgmap v680: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 262 op/s 2024-04-24T23:22:59.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1275233 12.50 MB/sec execute 424 sec latency 24.257 ms 2024-04-24T23:23:00.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:00 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:23:00.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:00 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:00 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:23:00.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:00 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:23:00.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:00 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:00 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:00 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:23:00.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:00 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:00 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:23:00.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:00 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:23:00.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:00 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:00 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:00 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:23:00.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:00 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:00 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:23:00.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:00 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:23:00.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:00 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:00 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:00.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1277416 12.49 MB/sec execute 425 sec latency 14.884 ms 2024-04-24T23:23:01.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:01 smithi038 ceph-mon[32450]: pgmap v681: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-04-24T23:23:01.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:01 smithi117 ceph-mon[32529]: pgmap v681: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-04-24T23:23:01.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:01 smithi033 ceph-mon[29201]: pgmap v681: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-04-24T23:23:01.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1279239 12.49 MB/sec execute 426 sec latency 16.076 ms 2024-04-24T23:23:02.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1282036 12.50 MB/sec execute 427 sec latency 42.573 ms 2024-04-24T23:23:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:03 smithi038 ceph-mon[32450]: pgmap v682: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-04-24T23:23:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:03 smithi117 ceph-mon[32529]: pgmap v682: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-04-24T23:23:03.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:03 smithi033 ceph-mon[29201]: pgmap v682: 129 pgs: 129 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-04-24T23:23:03.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1283637 12.49 MB/sec execute 428 sec latency 75.055 ms 2024-04-24T23:23:04.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1285870 12.49 MB/sec execute 429 sec latency 176.226 ms 2024-04-24T23:23:05.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:05 smithi033 ceph-mon[29201]: pgmap v683: 129 pgs: 129 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-04-24T23:23:05.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:05 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:05 smithi038 ceph-mon[32450]: pgmap v683: 129 pgs: 129 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-04-24T23:23:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:05 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:05 smithi117 ceph-mon[32529]: pgmap v683: 129 pgs: 129 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-04-24T23:23:05.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:05 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:05.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1286981 12.47 MB/sec execute 430 sec latency 49.974 ms 2024-04-24T23:23:06.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1289419 12.48 MB/sec execute 431 sec latency 24.447 ms 2024-04-24T23:23:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: pgmap v684: 129 pgs: 129 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-04-24T23:23:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:07 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: pgmap v684: 129 pgs: 129 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:07 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: pgmap v684: 129 pgs: 129 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:07 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:07.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1291610 12.47 MB/sec execute 432 sec latency 18.154 ms 2024-04-24T23:23:08.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:08 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:23:08.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:08.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:08.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:08 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:23:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:08.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:08 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:23:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:23:08.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1293641 12.47 MB/sec execute 433 sec latency 16.391 ms 2024-04-24T23:23:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:09 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:23:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:09 smithi038 ceph-mon[32450]: pgmap v685: 129 pgs: 129 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T23:23:09.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:09 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:23:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:09 smithi117 ceph-mon[32529]: pgmap v685: 129 pgs: 129 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T23:23:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:09 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:23:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:09 smithi033 ceph-mon[29201]: pgmap v685: 129 pgs: 129 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T23:23:09.462 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1296334 12.48 MB/sec execute 434 sec latency 93.673 ms 2024-04-24T23:23:10.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1297943 12.46 MB/sec execute 435 sec latency 27.246 ms 2024-04-24T23:23:11.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:11 smithi038 ceph-mon[32450]: pgmap v686: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-04-24T23:23:11.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:11 smithi117 ceph-mon[32529]: pgmap v686: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-04-24T23:23:11.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:11 smithi033 ceph-mon[29201]: pgmap v686: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-04-24T23:23:11.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1300389 12.47 MB/sec execute 436 sec latency 11.903 ms 2024-04-24T23:23:12.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1303344 12.48 MB/sec execute 437 sec latency 17.144 ms 2024-04-24T23:23:13.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:13 smithi038 ceph-mon[32450]: pgmap v687: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-04-24T23:23:13.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:13 smithi117 ceph-mon[32529]: pgmap v687: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-04-24T23:23:13.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:13 smithi033 ceph-mon[29201]: pgmap v687: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-04-24T23:23:13.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1305258 12.46 MB/sec execute 438 sec latency 18.934 ms 2024-04-24T23:23:14.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1307387 12.47 MB/sec execute 439 sec latency 64.518 ms 2024-04-24T23:23:15.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:15 smithi038 ceph-mon[32450]: pgmap v688: 129 pgs: 129 active+clean; 279 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 253 op/s 2024-04-24T23:23:15.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:15 smithi117 ceph-mon[32529]: pgmap v688: 129 pgs: 129 active+clean; 279 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 253 op/s 2024-04-24T23:23:15.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:15 smithi033 ceph-mon[29201]: pgmap v688: 129 pgs: 129 active+clean; 279 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 253 op/s 2024-04-24T23:23:15.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1309509 12.46 MB/sec execute 440 sec latency 33.904 ms 2024-04-24T23:23:16.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1311447 12.46 MB/sec execute 441 sec latency 46.102 ms 2024-04-24T23:23:17.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:17 smithi038 ceph-mon[32450]: pgmap v689: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-04-24T23:23:17.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:17 smithi117 ceph-mon[32529]: pgmap v689: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-04-24T23:23:17.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:17 smithi033 ceph-mon[29201]: pgmap v689: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-04-24T23:23:17.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1314253 12.47 MB/sec execute 442 sec latency 11.372 ms 2024-04-24T23:23:18.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1315905 12.45 MB/sec execute 443 sec latency 41.346 ms 2024-04-24T23:23:19.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:19 smithi038 ceph-mon[32450]: pgmap v690: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:23:19.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:19 smithi117 ceph-mon[32529]: pgmap v690: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:23:19.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:19 smithi033 ceph-mon[29201]: pgmap v690: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-24T23:23:19.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1317928 12.46 MB/sec execute 444 sec latency 73.560 ms 2024-04-24T23:23:20.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1319881 12.44 MB/sec execute 445 sec latency 28.812 ms 2024-04-24T23:23:21.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:21 smithi038 ceph-mon[32450]: pgmap v691: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-04-24T23:23:21.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:21 smithi117 ceph-mon[32529]: pgmap v691: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-04-24T23:23:21.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:21 smithi033 ceph-mon[29201]: pgmap v691: 129 pgs: 129 active+clean; 282 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-04-24T23:23:21.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1321668 12.44 MB/sec execute 446 sec latency 23.362 ms 2024-04-24T23:23:22.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1324068 12.44 MB/sec execute 447 sec latency 14.276 ms 2024-04-24T23:23:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:23 smithi038 ceph-mon[32450]: pgmap v692: 129 pgs: 129 active+clean; 285 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-04-24T23:23:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:23 smithi117 ceph-mon[32529]: pgmap v692: 129 pgs: 129 active+clean; 285 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-04-24T23:23:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:23 smithi033 ceph-mon[29201]: pgmap v692: 129 pgs: 129 active+clean; 285 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-04-24T23:23:23.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1325617 12.43 MB/sec execute 448 sec latency 12.873 ms 2024-04-24T23:23:24.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1328185 12.44 MB/sec execute 449 sec latency 48.453 ms 2024-04-24T23:23:25.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:25 smithi038 ceph-mon[32450]: pgmap v693: 129 pgs: 129 active+clean; 285 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-04-24T23:23:25.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:25 smithi117 ceph-mon[32529]: pgmap v693: 129 pgs: 129 active+clean; 285 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-04-24T23:23:25.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:25 smithi033 ceph-mon[29201]: pgmap v693: 129 pgs: 129 active+clean; 285 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-04-24T23:23:25.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1329273 12.42 MB/sec execute 450 sec latency 23.655 ms 2024-04-24T23:23:26.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1331970 12.43 MB/sec execute 451 sec latency 23.271 ms 2024-04-24T23:23:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:27 smithi038 ceph-mon[32450]: pgmap v694: 129 pgs: 129 active+clean; 287 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-04-24T23:23:27.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:27 smithi117 ceph-mon[32529]: pgmap v694: 129 pgs: 129 active+clean; 287 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-04-24T23:23:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:27 smithi033 ceph-mon[29201]: pgmap v694: 129 pgs: 129 active+clean; 287 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-04-24T23:23:27.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1334126 12.42 MB/sec execute 452 sec latency 23.974 ms 2024-04-24T23:23:28.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1336139 12.42 MB/sec execute 453 sec latency 12.772 ms 2024-04-24T23:23:28.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:23:28.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:23:28.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 -- 172.21.15.33:0/2084648990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 msgr2=0x7f11d4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 --2- 172.21.15.33:0/2084648990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4102e00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c002a410 comp rx=0 tx=0).stop 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 -- 172.21.15.33:0/2084648990 shutdown_connections 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 --2- 172.21.15.33:0/2084648990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4103340 0x7f11d4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 --2- 172.21.15.33:0/2084648990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4102e00 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 --2- 172.21.15.33:0/2084648990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4101790 0x7f11d4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 -- 172.21.15.33:0/2084648990 >> 172.21.15.33:0/2084648990 conn(0x7f11d40fcd20 msgr2=0x7f11d40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 -- 172.21.15.33:0/2084648990 shutdown_connections 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 -- 172.21.15.33:0/2084648990 wait complete. 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 Processor -- start 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.666+0000 7f11db1f9640 1 -- start start 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4101790 0x7f11d4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103340 0x7f11d419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d3f0 con 0x7f11d4102980 2024-04-24T23:23:28.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d270 con 0x7f11d4103340 2024-04-24T23:23:28.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d570 con 0x7f11d4101790 2024-04-24T23:23:28.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42692/0 (socket says 172.21.15.33:42692) 2024-04-24T23:23:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 -- 172.21.15.33:0/667511596 learned_addr learned my addr 172.21.15.33:0/667511596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d8f95640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4101790 0x7f11d4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d9796640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103340 0x7f11d419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 -- 172.21.15.33:0/667511596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4101790 msgr2=0x7f11d4197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4101790 0x7f11d4197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 -- 172.21.15.33:0/667511596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103340 msgr2=0x7f11d419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103340 0x7f11d419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 -- 172.21.15.33:0/667511596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d419f680 con 0x7f11d4102980 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d8f95640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4101790 0x7f11d4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d9796640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103340 0x7f11d419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d3fff640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4197fa0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009230 tx=0x7f11c0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11d1ffb640 1 -- 172.21.15.33:0/667511596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c003f030 con 0x7f11d4102980 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.667+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d419f910 con 0x7f11d4102980 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.668+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d419fe50 con 0x7f11d4102980 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.668+0000 7f11d1ffb640 1 -- 172.21.15.33:0/667511596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c0003ad0 con 0x7f11d4102980 2024-04-24T23:23:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.668+0000 7f11d1ffb640 1 -- 172.21.15.33:0/667511596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0049060 con 0x7f11d4102980 2024-04-24T23:23:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.670+0000 7f11d1ffb640 1 -- 172.21.15.33:0/667511596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f11c0042030 con 0x7f11d4102980 2024-04-24T23:23:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.670+0000 7f11ddfe4740 1 Processor -- start 2024-04-24T23:23:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.670+0000 7f11ddfe4740 1 -- start start 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d1ffb640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f11c40a2c10 0x7f11c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd034660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd039730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd039da0 0x55c4bd03c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4bcf986b0 con 0x55c4bd034240 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c4bcf40880 con 0x55c4bd039da0 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c4bcf94670 con 0x55c4bd034c30 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d1ffb640 1 -- 172.21.15.33:0/667511596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11c00ef490 con 0x7f11d4102980 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d9796640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f11c40a2c10 0x7f11c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd034660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd034660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42698/0 (socket says 172.21.15.33:42698) 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 -- 172.21.15.33:0/4040682430 learned_addr learned my addr 172.21.15.33:0/4040682430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d9796640 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd039da0 0x55c4bd03c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d3fff640 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd039730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 -- 172.21.15.33:0/4040682430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 msgr2=0x55c4bd039730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd039730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 -- 172.21.15.33:0/4040682430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd039da0 msgr2=0x55c4bd03c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd039da0 0x55c4bd03c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 -- 172.21.15.33:0/4040682430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4bcd767d0 con 0x55c4bd034240 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d9796640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f11c40a2c10 0x7f11c40a50d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f11cc007fd0 tx=0x7f11cc015910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.671+0000 7f11d8f95640 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd034660 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f11c80114c0 tx=0x7f11c8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4bb627381af5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11beffd640 1 -- 172.21.15.33:0/4040682430 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8004020 con 0x55c4bd034240 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11beffd640 1 -- 172.21.15.33:0/4040682430 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c8005500 con 0x55c4bd034240 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11beffd640 1 -- 172.21.15.33:0/4040682430 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8004e60 con 0x55c4bd034240 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 -- 172.21.15.33:0/4040682430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 msgr2=0x55c4bd034660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd034660 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f11c80114c0 tx=0x7f11c8011980 comp rx=0 tx=0).stop 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 -- 172.21.15.33:0/4040682430 shutdown_connections 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd039da0 0x55c4bd03c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd039730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/4040682430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd034660 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 -- 172.21.15.33:0/4040682430 >> 172.21.15.33:0/4040682430 conn(0x55c4bd02f8b0 msgr2=0x55c4bd02fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.672+0000 7f11ddfe4740 1 -- 172.21.15.33:0/4040682430 shutdown_connections 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 -- 172.21.15.33:0/4040682430 wait complete. 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 -- start start 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 Processor -- start 2024-04-24T23:23:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd03e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd03eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd039da0 0x55c4bd045b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11d8f95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd03e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11d8f95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd03e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42700/0 (socket says 172.21.15.33:42700) 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11d8f95640 1 -- 172.21.15.33:0/3928789745 learned_addr learned my addr 172.21.15.33:0/3928789745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4bd03cd50 con 0x55c4bd034240 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c4bcf53f50 con 0x55c4bd034c30 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c4bd03ced0 con 0x55c4bd039da0 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11d9796640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd039da0 0x55c4bd045b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11ddfe4740 10 client.0 _ll_register_callbacks cb 0x55c4bcf8f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.673+0000 7f11d3fff640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd03eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11d8f95640 1 -- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd039da0 msgr2=0x55c4bd045b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11d8f95640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd039da0 0x55c4bd045b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11d8f95640 1 -- 172.21.15.33:0/3928789745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 msgr2=0x55c4bd03eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11d8f95640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd03eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11d8f95640 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4bccac3f0 con 0x55c4bd034240 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11d8f95640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd03e5b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11c8011e80 tx=0x7f11c8005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d79fa5050c177 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11b3fff640 1 -- 172.21.15.33:0/3928789745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c801d050 con 0x55c4bd034240 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11b3fff640 1 -- 172.21.15.33:0/3928789745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11c8024450 con 0x55c4bd034240 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c8005500 con 0x55c4bd034240 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11b3fff640 1 -- 172.21.15.33:0/3928789745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11c8032050 con 0x55c4bd034240 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11ddfe4740 10 client.17790 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4bcc6c0f0 con 0x55c4bd034240 2024-04-24T23:23:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.674+0000 7f11b3fff640 1 -- 172.21.15.33:0/3928789745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c8005500 con 0x55c4bd034240 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11b3fff640 1 -- 172.21.15.33:0/3928789745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f11c800f6b0 con 0x55c4bd034240 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11ddfe4740 10 client.17790 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11ddfe4740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11ddfe4740 20 client.17790 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd04a230 0x55c4bd04c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11ddfe4740 4 client.17790 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4bcfa5bf0 con 0x55c4bd04a230 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.675+0000 7f11d3fff640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd04a230 0x55c4bd04c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.676+0000 7f11d3fff640 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd04a230 0x55c4bd04c6f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4512769c6a0a713 server_cookie=17fbda3a4f795eaa in_seq=0 out_seq=0 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.676+0000 7f11b3fff640 10 client.17790 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.676+0000 7f11b3fff640 1 -- 172.21.15.33:0/3928789745 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4bcfa5bf0 con 0x55c4bd04a230 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.676+0000 7f11b3fff640 10 client.17790 handle_command_reply: tid=0 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.676+0000 7f11ddfe4740 1 client.17790 shutdown 2024-04-24T23:23:28.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 msgr2=0x55c4bd03e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd03e5b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11c8011e80 tx=0x7f11c8005240 comp rx=0 tx=0).stop 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 shutdown_connections 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd04a230 0x55c4bd04c6f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd039da0 0x55c4bd045b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd03eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11b3fff640 0 client.17790 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/3928789745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd034240 0x55c4bd03e5b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 >> 172.21.15.33:0/3928789745 conn(0x55c4bd02f8b0 msgr2=0x55c4bd02eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 shutdown_connections 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 1 -- 172.21.15.33:0/3928789745 wait complete. 2024-04-24T23:23:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.677+0000 7f11ddfe4740 20 client.17790 trim_cache size 0 max 16384 2024-04-24T23:23:28.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 Processor -- start 2024-04-24T23:23:28.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 -- start start 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034240 0x55c4bd02fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd045af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd02e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8012490 con 0x55c4bd039da0 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11b800fd60 con 0x55c4bd034240 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11b800dbb0 con 0x55c4bd034c30 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11d9796640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd02e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11d9796640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd02e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42710/0 (socket says 172.21.15.33:42710) 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11d9796640 1 -- 172.21.15.33:0/37757660 learned_addr learned my addr 172.21.15.33:0/37757660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.695+0000 7f11d8f95640 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034240 0x55c4bd02fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d3fff640 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd045af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d9796640 1 -- 172.21.15.33:0/37757660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 msgr2=0x55c4bd045af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d9796640 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd045af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d9796640 1 -- 172.21.15.33:0/37757660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034240 msgr2=0x55c4bd02fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d9796640 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034240 0x55c4bd02fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d9796640 1 -- 172.21.15.33:0/37757660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4bd036670 con 0x55c4bd039da0 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d8f95640 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034240 0x55c4bd02fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:28.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11d9796640 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd02e970 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0147e0 tx=0x7f11cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140319d3df1d4976 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11be7fc640 1 -- 172.21.15.33:0/37757660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc015400 con 0x55c4bd039da0 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11be7fc640 1 -- 172.21.15.33:0/37757660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11cc0155a0 con 0x55c4bd039da0 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.696+0000 7f11be7fc640 1 -- 172.21.15.33:0/37757660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc02c710 con 0x55c4bd039da0 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 -- 172.21.15.33:0/37757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 msgr2=0x55c4bd02e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd02e970 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0147e0 tx=0x7f11cc004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 -- 172.21.15.33:0/37757660 shutdown_connections 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd02e970 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034c30 0x55c4bd045af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/37757660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034240 0x55c4bd02fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 -- 172.21.15.33:0/37757660 >> 172.21.15.33:0/37757660 conn(0x55c4bd02f8b0 msgr2=0x55c4bd02e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 -- 172.21.15.33:0/37757660 shutdown_connections 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 -- 172.21.15.33:0/37757660 wait complete. 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 -- start start 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.697+0000 7f11ddfe4740 1 Processor -- start 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034240 0x55c4bd042a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd042f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11b800e220 con 0x55c4bd039da0 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11b800f690 con 0x55c4bd034c30 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11b800e8f0 con 0x55c4bd034240 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11ddfe4740 10 client.0 _ll_register_callbacks cb 0x7f11cc015e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd0ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42724/0 (socket says 172.21.15.33:42724) 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 -- 172.21.15.33:0/27368698 learned_addr learned my addr 172.21.15.33:0/27368698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd042f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d8f95640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034240 0x55c4bd042a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 -- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034240 msgr2=0x55c4bd042a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034240 0x55c4bd042a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 -- 172.21.15.33:0/27368698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 msgr2=0x55c4bd042f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd042f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 -- 172.21.15.33:0/27368698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4bd0aed90 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d8f95640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034240 0x55c4bd042a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11d9796640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd0ae820 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f11cc017010 tx=0x7f11cc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c7e30120f2cfe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.698+0000 7f11b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11b3fff640 1 -- 172.21.15.33:0/27368698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc03b060 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11b3fff640 1 -- 172.21.15.33:0/27368698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11cc029040 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11cc0155a0 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11b3fff640 1 -- 172.21.15.33:0/27368698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11cc042050 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 10 client.17796 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4bd0b0080 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11b3fff640 1 -- 172.21.15.33:0/27368698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11cc0155a0 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11b3fff640 1 -- 172.21.15.33:0/27368698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f11cc035070 con 0x55c4bd039da0 2024-04-24T23:23:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 10 client.17796 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 20 client.17796 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd0b0a30 0x55c4bd0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 4 client.17796 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4bcf9ff40 con 0x55c4bd0b0a30 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.699+0000 7f11d8f95640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd0b0a30 0x55c4bd0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11b3fff640 10 client.17796 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11d8f95640 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd0b0a30 0x55c4bd0b2e50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b776a25b3d628a54 server_cookie=3d67ee22eee78167 in_seq=0 out_seq=0 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11b3fff640 1 -- 172.21.15.33:0/27368698 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c4bcf9ff40 con 0x55c4bd0b0a30 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11b3fff640 10 client.17796 handle_command_reply: tid=0 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 client.17796 shutdown 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 msgr2=0x55c4bd0ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd0ae820 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f11cc017010 tx=0x7f11cc00cd20 comp rx=0 tx=0).stop 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 shutdown_connections 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c4bd0b0a30 0x55c4bd0b2e50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4bd039da0 0x55c4bd0ae820 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c4bd034c30 0x55c4bd042f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11b3fff640 0 client.17796 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 --2- 172.21.15.33:0/27368698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4bd034240 0x55c4bd042a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 >> 172.21.15.33:0/27368698 conn(0x55c4bd02f8b0 msgr2=0x55c4bd02eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 shutdown_connections 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.700+0000 7f11ddfe4740 1 -- 172.21.15.33:0/27368698 wait complete. 2024-04-24T23:23:28.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11ddfe4740 20 client.17796 trim_cache size 0 max 16384 2024-04-24T23:23:28.711 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:23:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:23:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:23:28.711 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f11c40a2c10 msgr2=0x7f11c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f11c40a2c10 0x7f11c40a50d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f11cc007fd0 tx=0x7f11cc015910 comp rx=0 tx=0).stop 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 msgr2=0x7f11d4197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4197fa0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009230 tx=0x7f11c0031b60 comp rx=0 tx=0).stop 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 shutdown_connections 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f11c40a2c10 0x7f11c40a50d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103340 0x7f11d419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4102980 0x7f11d4197fa0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 --2- 172.21.15.33:0/667511596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d4101790 0x7f11d4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 >> 172.21.15.33:0/667511596 conn(0x7f11d40fcd20 msgr2=0x7f11d4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 shutdown_connections 2024-04-24T23:23:28.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.701+0000 7f11db1f9640 1 -- 172.21.15.33:0/667511596 wait complete. 2024-04-24T23:23:28.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:23:28.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/4077961824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 msgr2=0x7f8bd8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/4077961824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd8101d00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009900 tx=0x7f8bcc02a410 comp rx=0 tx=0).stop 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/4077961824 shutdown_connections 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/4077961824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8102240 0x7f8bd8108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/4077961824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd8101d00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/4077961824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8100690 0x7f8bd8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/4077961824 >> 172.21.15.33:0/4077961824 conn(0x7f8bd80fbc20 msgr2=0x7f8bd80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/4077961824 shutdown_connections 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/4077961824 wait complete. 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 Processor -- start 2024-04-24T23:23:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.822+0000 7f8bdd7f4640 1 -- start start 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bdd7f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8100690 0x7f8bd806a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bdd7f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bdd7f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102240 0x7f8bd819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bdd7f4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810c2f0 con 0x7f8bd8101880 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bdd7f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810c170 con 0x7f8bd8100690 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bdd7f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810c470 con 0x7f8bd8102240 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8100690 0x7f8bd806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42746/0 (socket says 172.21.15.33:42746) 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 -- 172.21.15.33:0/1788596684 learned_addr learned my addr 172.21.15.33:0/1788596684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:28.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd77fe640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102240 0x7f8bd819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 -- 172.21.15.33:0/1788596684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102240 msgr2=0x7f8bd819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102240 0x7f8bd819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 -- 172.21.15.33:0/1788596684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8100690 msgr2=0x7f8bd806a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8100690 0x7f8bd806a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 -- 172.21.15.33:0/1788596684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd819f6c0 con 0x7f8bd8101880 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd6ffd640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8100690 0x7f8bd806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bd67fc640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd819c790 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009230 tx=0x7f8bcc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.823+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc03f030 con 0x7f8bd8101880 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.824+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd819f950 con 0x7f8bd8101880 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.824+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc003ce0 con 0x7f8bd8101880 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.824+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc035070 con 0x7f8bd8101880 2024-04-24T23:23:28.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.824+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd819fe90 con 0x7f8bd8101880 2024-04-24T23:23:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.825+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd81079b0 con 0x7f8bd8101880 2024-04-24T23:23:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.826+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8bcc042030 con 0x7f8bd8101880 2024-04-24T23:23:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.827+0000 7f8bbffff640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bc00a2b90 0x7f8bc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.827+0000 7f8bd6ffd640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bc00a2b90 0x7f8bc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.827+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bcc0efdc0 con 0x7f8bd8101880 2024-04-24T23:23:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.827+0000 7f8bd6ffd640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bc00a2b90 0x7f8bc00a5050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8005fd0 tx=0x7f8bc800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:28.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:28.831+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8bcc0bc2e0 con 0x7f8bd8101880 2024-04-24T23:23:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.128+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd8062340 con 0x7f8bd8101880 2024-04-24T23:23:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.128+0000 7f8bbffff640 1 -- 172.21.15.33:0/1788596684 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f8bcc0c05e0 con 0x7f8bd8101880 2024-04-24T23:23:29.130 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:23:29.130 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:23:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:23:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bc00a2b90 msgr2=0x7f8bc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bc00a2b90 0x7f8bc00a5050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8005fd0 tx=0x7f8bc800b410 comp rx=0 tx=0).stop 2024-04-24T23:23:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 msgr2=0x7f8bd819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd819c790 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009230 tx=0x7f8bcc004360 comp rx=0 tx=0).stop 2024-04-24T23:23:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 shutdown_connections 2024-04-24T23:23:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8bc00a2b90 0x7f8bc00a5050 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102240 0x7f8bd819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8101880 0x7f8bd819c790 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 --2- 172.21.15.33:0/1788596684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd8100690 0x7f8bd806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.130+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 >> 172.21.15.33:0/1788596684 conn(0x7f8bd80fbc20 msgr2=0x7f8bd8104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.131+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 shutdown_connections 2024-04-24T23:23:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.131+0000 7f8bdd7f4640 1 -- 172.21.15.33:0/1788596684 wait complete. 2024-04-24T23:23:29.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:23:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 -- 172.21.15.33:0/2835047524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10101880 msgr2=0x7f5c10101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 --2- 172.21.15.33:0/2835047524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10101880 0x7f5c10101d00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009900 tx=0x7f5c0402a410 comp rx=0 tx=0).stop 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 -- 172.21.15.33:0/2835047524 shutdown_connections 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 --2- 172.21.15.33:0/2835047524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10102240 0x7f5c10108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 --2- 172.21.15.33:0/2835047524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10101880 0x7f5c10101d00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 --2- 172.21.15.33:0/2835047524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10100690 0x7f5c10100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.247+0000 7f5c168ee640 1 -- 172.21.15.33:0/2835047524 >> 172.21.15.33:0/2835047524 conn(0x7f5c100fbc00 msgr2=0x7f5c100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.248+0000 7f5c168ee640 1 -- 172.21.15.33:0/2835047524 shutdown_connections 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.248+0000 7f5c168ee640 1 -- 172.21.15.33:0/2835047524 wait complete. 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.248+0000 7f5c168ee640 1 Processor -- start 2024-04-24T23:23:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.248+0000 7f5c168ee640 1 -- start start 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.248+0000 7f5c168ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10100690 0x7f5c10197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.248+0000 7f5c168ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10101880 0x7f5c10197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c168ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 0x7f5c1019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c168ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010c2f0 con 0x7f5c10102240 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c168ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010c170 con 0x7f5c10101880 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c168ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010c470 con 0x7f5c10100690 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 0x7f5c1019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10101880 0x7f5c10197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 0x7f5c1019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42764/0 (socket says 172.21.15.33:42764) 2024-04-24T23:23:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10101880 0x7f5c10197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33082/0 (socket says 172.21.15.33:33082) 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 -- 172.21.15.33:0/3694887443 learned_addr learned my addr 172.21.15.33:0/3694887443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10100690 0x7f5c10197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 -- 172.21.15.33:0/3694887443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10100690 msgr2=0x7f5c10197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10100690 0x7f5c10197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 -- 172.21.15.33:0/3694887443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10101880 msgr2=0x7f5c10197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10101880 0x7f5c10197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 -- 172.21.15.33:0/3694887443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1019f670 con 0x7f5c10102240 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c14e8b640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 0x7f5c1019f000 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800eaa0 tx=0x7f5c0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c0d7fa640 1 -- 172.21.15.33:0/3694887443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800cdb0 con 0x7f5c10102240 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.249+0000 7f5c0d7fa640 1 -- 172.21.15.33:0/3694887443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c08005020 con 0x7f5c10102240 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.250+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c10102b00 con 0x7f5c10102240 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.250+0000 7f5c0d7fa640 1 -- 172.21.15.33:0/3694887443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080116b0 con 0x7f5c10102240 2024-04-24T23:23:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.250+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c10103020 con 0x7f5c10102240 2024-04-24T23:23:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.252+0000 7f5c0d7fa640 1 -- 172.21.15.33:0/3694887443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5c08005560 con 0x7f5c10102240 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.252+0000 7f5c195f6740 1 Processor -- start 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 -- start start 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0d7fa640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5bf80a2c10 0x7f5bf80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557945217670 con 0x5579452b3230 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5579451bf970 con 0x5579452b8d90 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c195f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557945213630 con 0x5579452b3c20 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c14e8b640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5bf80a2c10 0x7f5bf80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42770/0 (socket says 172.21.15.33:42770) 2024-04-24T23:23:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3600377068 learned_addr learned my addr 172.21.15.33:0/3600377068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0d7fa640 1 -- 172.21.15.33:0/3694887443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c080cf510 con 0x7f5c10102240 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3600377068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 msgr2=0x5579452b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3600377068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 msgr2=0x5579452bb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452bb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3600377068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579451e3400 con 0x5579452b3230 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.253+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.254+0000 7f5c14e8b640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5bf80a2c10 0x7f5bf80a50d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5c0809ba20 tx=0x7f5c0806daa0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.254+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452b3650 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5c04031610 tx=0x7f5c04005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed4ae4a5eab345f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.254+0000 7f5bf67fc640 1 -- 172.21.15.33:0/3600377068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04035070 con 0x5579452b3230 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.254+0000 7f5bf67fc640 1 -- 172.21.15.33:0/3600377068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c04005510 con 0x5579452b3230 2024-04-24T23:23:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.254+0000 7f5bf67fc640 1 -- 172.21.15.33:0/3600377068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004c40 con 0x5579452b3230 2024-04-24T23:23:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- 172.21.15.33:0/3600377068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 msgr2=0x5579452b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452b3650 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5c04031610 tx=0x7f5c04005fd0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- 172.21.15.33:0/3600377068 shutdown_connections 2024-04-24T23:23:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452bb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3600377068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452b3650 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- 172.21.15.33:0/3600377068 >> 172.21.15.33:0/3600377068 conn(0x5579452ae8a0 msgr2=0x5579452aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- 172.21.15.33:0/3600377068 shutdown_connections 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- 172.21.15.33:0/3600377068 wait complete. 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- start start 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 Processor -- start 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 0x5579452bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452c4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5579452bbce0 con 0x5579452b3c20 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5579451d3050 con 0x5579452b8d90 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.255+0000 7f5c195f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5579452bbe60 con 0x5579452b3230 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c195f6740 10 client.0 _ll_register_callbacks cb 0x557944f2abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 0x5579452bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 0x5579452bd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42784/0 (socket says 172.21.15.33:42784) 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 -- 172.21.15.33:0/955429395 learned_addr learned my addr 172.21.15.33:0/955429395 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 -- 172.21.15.33:0/955429395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 msgr2=0x5579452bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452bd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 -- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 msgr2=0x5579452c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 -- 172.21.15.33:0/955429395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579451a4dc0 con 0x5579452b3c20 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 0x5579452bd7d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5c00010800 tx=0x7f5c00010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c8f38f16ba545f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5bef7fe640 1 -- 172.21.15.33:0/955429395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00004320 con 0x5579452b3c20 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5bef7fe640 1 -- 172.21.15.33:0/955429395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c0000f920 con 0x5579452b3c20 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c04005510 con 0x5579452b3c20 2024-04-24T23:23:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5bef7fe640 1 -- 172.21.15.33:0/955429395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c000044c0 con 0x5579452b3c20 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c195f6740 10 client.17814 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.256+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557944f2b3f0 con 0x5579452b3c20 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5bef7fe640 1 -- 172.21.15.33:0/955429395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c04005510 con 0x5579452b3c20 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5bef7fe640 1 -- 172.21.15.33:0/955429395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c0001a420 con 0x5579452b3c20 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c195f6740 10 client.17814 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c195f6740 10 client.17814 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c195f6740 20 client.17814 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c195f6740 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5579452c9240 0x5579452cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c195f6740 4 client.17814 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557945224bb0 con 0x5579452c9240 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5579452c9240 0x5579452cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5bef7fe640 10 client.17814 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.257+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5579452c9240 0x5579452cb680 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55b6ffec6bdd5e62 server_cookie=48e1a57d88ea6022 in_seq=0 out_seq=0 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5bef7fe640 1 -- 172.21.15.33:0/955429395 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557945224bb0 con 0x5579452c9240 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5bef7fe640 10 client.17814 handle_command_reply: tid=0 2024-04-24T23:23:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 client.17814 shutdown 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 msgr2=0x5579452bd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 0x5579452bd7d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5c00010800 tx=0x7f5c00010cc0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 shutdown_connections 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5579452c9240 0x5579452cb680 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3c20 0x5579452bd7d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5bef7fe640 0 client.17814 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 --2- 172.21.15.33:0/955429395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 >> 172.21.15.33:0/955429395 conn(0x5579452ae8a0 msgr2=0x5579452adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 shutdown_connections 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 1 -- 172.21.15.33:0/955429395 wait complete. 2024-04-24T23:23:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.258+0000 7f5c195f6740 20 client.17814 trim_cache size 0 max 16384 2024-04-24T23:23:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 Processor -- start 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 -- start start 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101aa9d0 con 0x5579452b3230 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101a82a0 con 0x5579452b8d90 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c195f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101a60f0 con 0x5579452b3c20 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42798/0 (socket says 172.21.15.33:42798) 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3289777725 learned_addr learned my addr 172.21.15.33:0/3289777725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.274+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3289777725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 msgr2=0x5579452c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3289777725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 msgr2=0x5579452b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3289777725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557944ff57d0 con 0x5579452b3230 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452ad9c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5c040319c0 tx=0x7f5c04005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b544aa60890b6885 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5bf5ffb640 1 -- 172.21.15.33:0/3289777725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04035040 con 0x5579452b3230 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5bf5ffb640 1 -- 172.21.15.33:0/3289777725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c04005040 con 0x5579452b3230 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.275+0000 7f5bf5ffb640 1 -- 172.21.15.33:0/3289777725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004ba0 con 0x5579452b3230 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 -- 172.21.15.33:0/3289777725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 msgr2=0x5579452ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452ad9c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5c040319c0 tx=0x7f5c04005fd0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 -- 172.21.15.33:0/3289777725 shutdown_connections 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b8d90 0x5579452b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3c20 0x5579452c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3289777725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b3230 0x5579452ad9c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 -- 172.21.15.33:0/3289777725 >> 172.21.15.33:0/3289777725 conn(0x5579452ae8a0 msgr2=0x5579452ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 -- 172.21.15.33:0/3289777725 shutdown_connections 2024-04-24T23:23:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 -- 172.21.15.33:0/3289777725 wait complete. 2024-04-24T23:23:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 -- start start 2024-04-24T23:23:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.276+0000 7f5c195f6740 1 Processor -- start 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b3c20 0x5579452c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b8d90 0x5579452c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101a6760 con 0x5579452b8d90 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101a7bd0 con 0x5579452b3c20 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101a6e30 con 0x5579452b3230 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c195f6740 10 client.0 _ll_register_callbacks cb 0x7f5c00010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b8d90 0x5579452c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452c2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53406/0 (socket says 172.21.15.33:53406) 2024-04-24T23:23:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3252554231 learned_addr learned my addr 172.21.15.33:0/3252554231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b3c20 0x5579452c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b3c20 msgr2=0x5579452c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b3c20 0x5579452c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3252554231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b8d90 msgr2=0x5579452c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b8d90 0x5579452c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 -- 172.21.15.33:0/3252554231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579452c3d40 con 0x5579452b3230 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5c0f7fe640 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452c2200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c040059d0 tx=0x7f5c04005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c9118d241d972d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.277+0000 7f5bef7fe640 1 -- 172.21.15.33:0/3252554231 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04054040 con 0x5579452b3230 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5bef7fe640 1 -- 172.21.15.33:0/3252554231 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c040053f0 con 0x5579452b3230 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c04005040 con 0x5579452b3230 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5bef7fe640 1 -- 172.21.15.33:0/3252554231 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c04035040 con 0x5579452b3230 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 10 client.27292 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55794532fa80 con 0x5579452b3230 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5bef7fe640 1 -- 172.21.15.33:0/3252554231 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c04005040 con 0x5579452b3230 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5bef7fe640 1 -- 172.21.15.33:0/3252554231 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c0403a8b0 con 0x5579452b3230 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 10 client.27292 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 10 client.27292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 20 client.27292 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55794532fe70 0x557945332290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 4 client.27292 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55794521ef00 con 0x55794532fe70 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.278+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55794532fe70 0x557945332290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.279+0000 7f5bef7fe640 10 client.27292 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.279+0000 7f5c0ffff640 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55794532fe70 0x557945332290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9aec432f2eaf24b0 server_cookie=77ad443541fedaf8 in_seq=0 out_seq=0 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5bef7fe640 1 -- 172.21.15.33:0/3252554231 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55794521ef00 con 0x55794532fe70 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5bef7fe640 10 client.27292 handle_command_reply: tid=0 2024-04-24T23:23:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 client.27292 shutdown 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 msgr2=0x5579452c2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452c2200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c040059d0 tx=0x7f5c04005c90 comp rx=0 tx=0).stop 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 shutdown_connections 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55794532fe70 0x557945332290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579452b8d90 0x5579452c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579452b3c20 0x5579452c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 --2- 172.21.15.33:0/3252554231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579452b3230 0x5579452c2200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5bef7fe640 0 client.27292 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 >> 172.21.15.33:0/3252554231 conn(0x5579452ae8a0 msgr2=0x5579452adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 shutdown_connections 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 1 -- 172.21.15.33:0/3252554231 wait complete. 2024-04-24T23:23:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c195f6740 20 client.27292 trim_cache size 0 max 16384 2024-04-24T23:23:29.288 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:23:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5bf80a2c10 msgr2=0x7f5bf80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c168ee640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5bf80a2c10 0x7f5bf80a50d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5c0809ba20 tx=0x7f5c0806daa0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 msgr2=0x7f5c1019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.280+0000 7f5c168ee640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 0x7f5c1019f000 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800eaa0 tx=0x7f5c0800ef60 comp rx=0 tx=0).stop 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 shutdown_connections 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5bf80a2c10 0x7f5bf80a50d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c10102240 0x7f5c1019f000 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c10101880 0x7f5c10197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 --2- 172.21.15.33:0/3694887443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10100690 0x7f5c10197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 >> 172.21.15.33:0/3694887443 conn(0x7f5c100fbc00 msgr2=0x7f5c100fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 shutdown_connections 2024-04-24T23:23:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.281+0000 7f5c168ee640 1 -- 172.21.15.33:0/3694887443 wait complete. 2024-04-24T23:23:29.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:23:29.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 -- 172.21.15.33:0/1349015034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 msgr2=0x7f17181045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 --2- 172.21.15.33:0/1349015034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f17181045f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1708009900 tx=0x7f170802a410 comp rx=0 tx=0).stop 2024-04-24T23:23:29.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 -- 172.21.15.33:0/1349015034 shutdown_connections 2024-04-24T23:23:29.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 --2- 172.21.15.33:0/1349015034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718107460 0x7f17181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 --2- 172.21.15.33:0/1349015034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718104b30 0x7f1718106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 --2- 172.21.15.33:0/1349015034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f17181045f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 -- 172.21.15.33:0/1349015034 >> 172.21.15.33:0/1349015034 conn(0x7f17180fbbf0 msgr2=0x7f17180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 -- 172.21.15.33:0/1349015034 shutdown_connections 2024-04-24T23:23:29.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.403+0000 7f171dca5640 1 -- 172.21.15.33:0/1349015034 wait complete. 2024-04-24T23:23:29.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.404+0000 7f171dca5640 1 Processor -- start 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.404+0000 7f171dca5640 1 -- start start 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.404+0000 7f171dca5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f1718101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.404+0000 7f171dca5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718104b30 0x7f17181001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.404+0000 7f171dca5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107460 0x7f17181006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f171dca5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f171810d450 con 0x7f1718107460 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f171dca5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f171810d2d0 con 0x7f1718104b30 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f171dca5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f171810d5d0 con 0x7f1718102200 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f1718101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f1718101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53418/0 (socket says 172.21.15.33:53418) 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 -- 172.21.15.33:0/3507886834 learned_addr learned my addr 172.21.15.33:0/3507886834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f1717fff640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107460 0x7f17181006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f1716ffd640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718104b30 0x7f17181001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 -- 172.21.15.33:0/3507886834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718104b30 msgr2=0x7f17181001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718104b30 0x7f17181001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 -- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107460 msgr2=0x7f17181006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107460 0x7f17181006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 -- 172.21.15.33:0/3507886834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1718100f80 con 0x7f1718102200 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f1717fff640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107460 0x7f17181006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f1716ffd640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718104b30 0x7f17181001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.405+0000 7f17177fe640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f1718101b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1708031440 tx=0x7f1708032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.406+0000 7f1714ff9640 1 -- 172.21.15.33:0/3507886834 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170803f030 con 0x7f1718102200 2024-04-24T23:23:29.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.406+0000 7f1714ff9640 1 -- 172.21.15.33:0/3507886834 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1708003ce0 con 0x7f1718102200 2024-04-24T23:23:29.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.406+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17181a1be0 con 0x7f1718102200 2024-04-24T23:23:29.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.406+0000 7f1714ff9640 1 -- 172.21.15.33:0/3507886834 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708035070 con 0x7f1718102200 2024-04-24T23:23:29.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.406+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181a2180 con 0x7f1718102200 2024-04-24T23:23:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.408+0000 7f1714ff9640 1 -- 172.21.15.33:0/3507886834 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1708042030 con 0x7f1718102200 2024-04-24T23:23:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.409+0000 7f1714ff9640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f17000a2c10 0x7f17000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.409+0000 7f1716ffd640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f17000a2c10 0x7f17000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.409+0000 7f1714ff9640 1 -- 172.21.15.33:0/3507886834 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f17080efce0 con 0x7f1718102200 2024-04-24T23:23:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.409+0000 7f17209b9740 1 Processor -- start 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.409+0000 7f17209b9740 1 -- start start 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17209b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5230 0x5601c9fb5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17209b9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5c20 0x5601c9fba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17209b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17209b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601c9f19670 con 0x5601c9fbad90 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17209b9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5601c9ec1970 con 0x5601c9fb5c20 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17209b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5601c9f15630 con 0x5601c9fb5230 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42842/0 (socket says 172.21.15.33:42842) 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 -- 172.21.15.33:0/3228794438 learned_addr learned my addr 172.21.15.33:0/3228794438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f17177fe640 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5230 0x5601c9fb5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 -- 172.21.15.33:0/3228794438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5230 msgr2=0x5601c9fb5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5230 0x5601c9fb5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1716ffd640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f17000a2c10 0x7f17000a50d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17181018e0 tx=0x7f170c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 -- 172.21.15.33:0/3228794438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5c20 msgr2=0x5601c9fba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1716ffd640 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5c20 0x5601c9fba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5c20 0x5601c9fba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1717fff640 1 -- 172.21.15.33:0/3228794438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c9ee5400 con 0x5601c9fbad90 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.410+0000 7f1716ffd640 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5c20 0x5601c9fba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.411+0000 7f1717fff640 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbd220 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1710009fd0 tx=0x7f17100108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26ea0a59032dabd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.411+0000 7f16fdffb640 1 -- 172.21.15.33:0/3228794438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710011760 con 0x5601c9fbad90 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.411+0000 7f16fdffb640 1 -- 172.21.15.33:0/3228794438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1710011de0 con 0x5601c9fbad90 2024-04-24T23:23:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.411+0000 7f16fdffb640 1 -- 172.21.15.33:0/3228794438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171000e3c0 con 0x5601c9fbad90 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 -- 172.21.15.33:0/3228794438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 msgr2=0x5601c9fbd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbd220 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1710009fd0 tx=0x7f17100108e0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 -- 172.21.15.33:0/3228794438 shutdown_connections 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbd220 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5c20 0x5601c9fba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 --2- 172.21.15.33:0/3228794438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5230 0x5601c9fb5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 -- 172.21.15.33:0/3228794438 >> 172.21.15.33:0/3228794438 conn(0x5601c9fb08a0 msgr2=0x5601c9fb0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 -- 172.21.15.33:0/3228794438 shutdown_connections 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 -- 172.21.15.33:0/3228794438 wait complete. 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.412+0000 7f17209b9740 1 -- start start 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 Processor -- start 2024-04-24T23:23:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fbf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1716ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fbf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42844/0 (socket says 172.21.15.33:42844) 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1717fff640 1 -- 172.21.15.33:0/724311832 learned_addr learned my addr 172.21.15.33:0/724311832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601c9fbdd40 con 0x5601c9fbad90 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5601c9ed5050 con 0x5601c9fb5230 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5601c9fbdec0 con 0x5601c9fb5c20 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f17177fe640 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1716ffd640 1 -- 172.21.15.33:0/724311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 msgr2=0x5601c9fbf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1716ffd640 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.413+0000 7f1716ffd640 1 -- 172.21.15.33:0/724311832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 msgr2=0x5601c9fc68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f1716ffd640 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f1716ffd640 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c9ea6dc0 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f17209b9740 10 client.0 _ll_register_callbacks cb 0x5601c9c2cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1710011de0 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f1716ffd640 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fbf8d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5601c9fb6690 tx=0x7f170c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da921595ef532eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f16f6ffd640 10 client.27304 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f16f6ffd640 1 -- 172.21.15.33:0/724311832 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c01ed10 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.414+0000 7f16f6ffd640 1 -- 172.21.15.33:0/724311832 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f170c01f420 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f16f6ffd640 1 -- 172.21.15.33:0/724311832 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f170c01f620 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 10 client.27304 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5601c9c2d3f0 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f16f6ffd640 1 -- 172.21.15.33:0/724311832 <== mon.1 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f170c01f7c0 con 0x5601c9fb5c20 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 10 client.27304 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 10 client.27304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 20 client.27304 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601c9fcaf20 0x5601c9fcd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 4 client.27304 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601c9f26bb0 con 0x5601c9fcaf20 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.415+0000 7f17177fe640 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601c9fcaf20 0x5601c9fcd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.416+0000 7f16f6ffd640 10 client.27304 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.416+0000 7f17177fe640 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601c9fcaf20 0x5601c9fcd390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=586bf8accf77ca4e server_cookie=6f238fdd52e7a8c6 in_seq=0 out_seq=0 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.416+0000 7f16f6ffd640 1 -- 172.21.15.33:0/724311832 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601c9f26bb0 con 0x5601c9fcaf20 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.416+0000 7f16f6ffd640 10 client.27304 handle_command_reply: tid=0 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.416+0000 7f17209b9740 1 client.27304 shutdown 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 msgr2=0x5601c9fbf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fbf8d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5601c9fb6690 tx=0x7f170c004730 comp rx=0 tx=0).stop 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 shutdown_connections 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601c9fcaf20 0x5601c9fcd390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f16f6ffd640 0 client.27304 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:23:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fbf8d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 --2- 172.21.15.33:0/724311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 >> 172.21.15.33:0/724311832 conn(0x5601c9fb08a0 msgr2=0x5601c9fafb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 shutdown_connections 2024-04-24T23:23:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 1 -- 172.21.15.33:0/724311832 wait complete. 2024-04-24T23:23:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.417+0000 7f17209b9740 20 client.27304 trim_cache size 0 max 16384 2024-04-24T23:23:29.423 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:29 smithi038 ceph-mon[32450]: pgmap v695: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T23:23:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:29 smithi117 ceph-mon[32529]: pgmap v695: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T23:23:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 Processor -- start 2024-04-24T23:23:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 -- start start 2024-04-24T23:23:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17181aefb0 con 0x5601c9fbad90 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17181ac880 con 0x5601c9fb5230 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17209b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17181aa6d0 con 0x5601c9fb5c20 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1716ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1716ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53438/0 (socket says 172.21.15.33:53438) 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1716ffd640 1 -- 172.21.15.33:0/2634575870 learned_addr learned my addr 172.21.15.33:0/2634575870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1717fff640 1 -- 172.21.15.33:0/2634575870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 msgr2=0x5601c9fc6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1717fff640 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1717fff640 1 -- 172.21.15.33:0/2634575870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 msgr2=0x5601c9fbe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1717fff640 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1717fff640 1 -- 172.21.15.33:0/2634575870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c9cf77d0 con 0x5601c9fbad90 2024-04-24T23:23:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f17177fe640 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.431+0000 7f1716ffd640 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.432+0000 7f1717fff640 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbebc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1710010fd0 tx=0x7f1710005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9217ec31f8ca3dda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.432+0000 7f16fd7fa640 1 -- 172.21.15.33:0/2634575870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171002b070 con 0x5601c9fbad90 2024-04-24T23:23:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.432+0000 7f16fd7fa640 1 -- 172.21.15.33:0/2634575870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1710004d00 con 0x5601c9fbad90 2024-04-24T23:23:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.432+0000 7f16fd7fa640 1 -- 172.21.15.33:0/2634575870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710022040 con 0x5601c9fbad90 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.432+0000 7f17209b9740 1 -- 172.21.15.33:0/2634575870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 msgr2=0x5601c9fbebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbebc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1710010fd0 tx=0x7f1710005c40 comp rx=0 tx=0).stop 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- 172.21.15.33:0/2634575870 shutdown_connections 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fbebc0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- 172.21.15.33:0/2634575870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- 172.21.15.33:0/2634575870 >> 172.21.15.33:0/2634575870 conn(0x5601c9fb08a0 msgr2=0x5601c9fb6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- 172.21.15.33:0/2634575870 shutdown_connections 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- 172.21.15.33:0/2634575870 wait complete. 2024-04-24T23:23:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- start start 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 Processor -- start 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fc3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17181aad40 con 0x5601c9fbad90 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17181ac1b0 con 0x5601c9fb5230 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17181ab410 con 0x5601c9fb5c20 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f17209b9740 10 client.0 _ll_register_callbacks cb 0x7f170c009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.433+0000 7f1717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42864/0 (socket says 172.21.15.33:42864) 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 -- 172.21.15.33:0/163613965 learned_addr learned my addr 172.21.15.33:0/163613965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f17177fe640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fc3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1716ffd640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 -- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 msgr2=0x5601c9fc4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 -- 172.21.15.33:0/163613965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 msgr2=0x5601c9fc3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fc3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 -- 172.21.15.33:0/163613965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c9fc5970 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f1717fff640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc5380 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f17100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514274bc7ef452f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f16f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f16f6ffd640 1 -- 172.21.15.33:0/163613965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171002b040 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f16f6ffd640 1 -- 172.21.15.33:0/163613965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f17100059a0 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1710004d00 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f16f6ffd640 1 -- 172.21.15.33:0/163613965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f171003e050 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f17209b9740 10 client.17838 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.434+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601ca0314d0 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f16f6ffd640 1 -- 172.21.15.33:0/163613965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1710004d00 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f16f6ffd640 1 -- 172.21.15.33:0/163613965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1710039030 con 0x5601c9fbad90 2024-04-24T23:23:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17209b9740 10 client.17838 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17209b9740 10 client.17838 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17209b9740 20 client.17838 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17209b9740 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601ca031bb0 0x5601ca033fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17209b9740 4 client.17838 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601c9f20f00 con 0x5601ca031bb0 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.435+0000 7f17177fe640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601ca031bb0 0x5601ca033fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f16f6ffd640 10 client.17838 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17177fe640 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601ca031bb0 0x5601ca033fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b17bf825bdd7041 server_cookie=6e99bb40a3466c6 in_seq=0 out_seq=0 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f16f6ffd640 1 -- 172.21.15.33:0/163613965 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601c9f20f00 con 0x5601ca031bb0 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f16f6ffd640 10 client.17838 handle_command_reply: tid=0 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 client.17838 shutdown 2024-04-24T23:23:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 msgr2=0x5601c9fc5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc5380 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f17100057b0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 shutdown_connections 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5601ca031bb0 0x5601ca033fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601c9fbad90 0x5601c9fc5380 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c9fb5c20 0x5601c9fc4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 --2- 172.21.15.33:0/163613965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5601c9fb5230 0x5601c9fc3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 >> 172.21.15.33:0/163613965 conn(0x5601c9fb08a0 msgr2=0x5601c9fb3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f16f6ffd640 0 client.17838 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 shutdown_connections 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 1 -- 172.21.15.33:0/163613965 wait complete. 2024-04-24T23:23:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.436+0000 7f17209b9740 20 client.17838 trim_cache size 0 max 16384 2024-04-24T23:23:29.444 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:23:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f17000a2c10 msgr2=0x7f17000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f17000a2c10 0x7f17000a50d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17181018e0 tx=0x7f170c005f90 comp rx=0 tx=0).stop 2024-04-24T23:23:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 msgr2=0x7f1718101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f1718101b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1708031440 tx=0x7f1708032ce0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 shutdown_connections 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f17000a2c10 0x7f17000a50d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107460 0x7f17181006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1718104b30 0x7f17181001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 --2- 172.21.15.33:0/3507886834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1718102200 0x7f1718101b70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 >> 172.21.15.33:0/3507886834 conn(0x7f17180fbbf0 msgr2=0x7f1718102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 shutdown_connections 2024-04-24T23:23:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.437+0000 7f171dca5640 1 -- 172.21.15.33:0/3507886834 wait complete. 2024-04-24T23:23:29.448 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:29 smithi033 ceph-mon[29201]: pgmap v695: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T23:23:29.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:23:29.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1338783 12.43 MB/sec execute 454 sec latency 60.700 ms 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3358435368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f81006a0 msgr2=0x7fb3f8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3358435368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f81006a0 0x7fb3f8100ac0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009900 tx=0x7fb3e802a410 comp rx=0 tx=0).stop 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3358435368 shutdown_connections 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3358435368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 0x7fb3f8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3358435368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f8101890 0x7fb3f8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3358435368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f81006a0 0x7fb3f8100ac0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3358435368 >> 172.21.15.33:0/3358435368 conn(0x7fb3f80fbc10 msgr2=0x7fb3f80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3358435368 shutdown_connections 2024-04-24T23:23:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.556+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3358435368 wait complete. 2024-04-24T23:23:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 Processor -- start 2024-04-24T23:23:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 -- start start 2024-04-24T23:23:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f81006a0 0x7fb3f8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 0x7fb3f819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 0x7fb3f81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f810c320 con 0x7fb3f8101890 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f810c1a0 con 0x7fb3f8102250 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fe41b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f810c4a0 con 0x7fb3f81006a0 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 0x7fb3f819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 0x7fb3f819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42892/0 (socket says 172.21.15.33:42892) 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3f77fe640 1 -- 172.21.15.33:0/3764070175 learned_addr learned my addr 172.21.15.33:0/3764070175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f81006a0 0x7fb3f8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.557+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 0x7fb3f81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f77fe640 1 -- 172.21.15.33:0/3764070175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f81006a0 msgr2=0x7fb3f8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f81006a0 0x7fb3f8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f77fe640 1 -- 172.21.15.33:0/3764070175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 msgr2=0x7fb3f81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 0x7fb3f81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f77fe640 1 -- 172.21.15.33:0/3764070175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f81a1940 con 0x7fb3f8101890 2024-04-24T23:23:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 0x7fb3f81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 0x7fb3f819a270 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e9f0 tx=0x7fb3ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f57fa640 1 -- 172.21.15.33:0/3764070175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00cd80 con 0x7fb3f8101890 2024-04-24T23:23:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f81a1c30 con 0x7fb3f8101890 2024-04-24T23:23:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f57fa640 1 -- 172.21.15.33:0/3764070175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ec005500 con 0x7fb3f8101890 2024-04-24T23:23:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3f57fa640 1 -- 172.21.15.33:0/3764070175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec010690 con 0x7fb3f8101890 2024-04-24T23:23:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.558+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81a2100 con 0x7fb3f8101890 2024-04-24T23:23:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.560+0000 7fb3f57fa640 1 -- 172.21.15.33:0/3764070175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb3ec0046c0 con 0x7fb3f8101890 2024-04-24T23:23:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb4011fd740 1 Processor -- start 2024-04-24T23:23:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb3f57fa640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3e00a2b90 0x7fb3e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb4011fd740 1 -- start start 2024-04-24T23:23:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3e00a2b90 0x7fb3e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb3f57fa640 1 -- 172.21.15.33:0/3764070175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3ec0cf840 con 0x7fb3f8101890 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.561+0000 7fb4011fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629aa621670 con 0x5629aa6bdc20 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb4011fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5629aa5c9970 con 0x5629aa6c2d90 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb4011fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5629aa61d630 con 0x5629aa6bd230 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3fc9b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42902/0 (socket says 172.21.15.33:42902) 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3fc9b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33182/0 (socket says 172.21.15.33:33182) 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f77fe640 1 -- 172.21.15.33:0/3635929422 learned_addr learned my addr 172.21.15.33:0/3635929422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 -- 172.21.15.33:0/3635929422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 msgr2=0x5629aa6c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 -- 172.21.15.33:0/3635929422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 msgr2=0x5629aa6c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3e00a2b90 0x7fb3e00a5050 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009fd0 tx=0x7fb3f000ff60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 -- 172.21.15.33:0/3635929422 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aa5ed400 con 0x5629aa6bd230 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6bd650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009230 tx=0x7fb3e8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886746f4c72bf7a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.562+0000 7fb3de7fc640 1 -- 172.21.15.33:0/3635929422 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8040070 con 0x5629aa6bd230 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb3de7fc640 1 -- 172.21.15.33:0/3635929422 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8004030 con 0x5629aa6bd230 2024-04-24T23:23:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb3de7fc640 1 -- 172.21.15.33:0/3635929422 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8035040 con 0x5629aa6bd230 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 -- 172.21.15.33:0/3635929422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 msgr2=0x5629aa6bd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6bd650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009230 tx=0x7fb3e8005e20 comp rx=0 tx=0).stop 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 -- 172.21.15.33:0/3635929422 shutdown_connections 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 --2- 172.21.15.33:0/3635929422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6bd650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 -- 172.21.15.33:0/3635929422 >> 172.21.15.33:0/3635929422 conn(0x5629aa6b88a0 msgr2=0x5629aa6b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.563+0000 7fb4011fd740 1 -- 172.21.15.33:0/3635929422 shutdown_connections 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 -- 172.21.15.33:0/3635929422 wait complete. 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 -- start start 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 Processor -- start 2024-04-24T23:23:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6bdc20 0x5629aa6c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6c2d90 0x5629aa6ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629aa6c5ce0 con 0x5629aa6bd230 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5629aa5dd050 con 0x5629aa6bdc20 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5629aa6c5e60 con 0x5629aa6c2d90 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.564+0000 7fb4011fd740 10 client.0 _ll_register_callbacks cb 0x5629aa334be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42910/0 (socket says 172.21.15.33:42910) 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 -- 172.21.15.33:0/2264803055 learned_addr learned my addr 172.21.15.33:0/2264803055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6c2d90 0x5629aa6ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6bdc20 0x5629aa6c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 -- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6c2d90 msgr2=0x5629aa6ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6c2d90 0x5629aa6ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 -- 172.21.15.33:0/2264803055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6bdc20 msgr2=0x5629aa6c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6bdc20 0x5629aa6c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 -- 172.21.15.33:0/2264803055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629aa5aedc0 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6c2d90 0x5629aa6ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6bdc20 0x5629aa6c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c73c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8005ab0 tx=0x7fb3e8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0689925649ce56c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2264803055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e804c030 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2264803055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3e80316b0 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e8004030 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2264803055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3e8059050 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb4011fd740 10 client.17850 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.565+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629aa3353f0 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2264803055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8004030 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2264803055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb3e8004ca0 con 0x5629aa6bd230 2024-04-24T23:23:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb4011fd740 10 client.17850 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb4011fd740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb4011fd740 20 client.17850 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb4011fd740 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa6d2dc0 0x5629aa6d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb4011fd740 4 client.17850 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629aa62ebb0 con 0x5629aa6d2dc0 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa6d2dc0 0x5629aa6d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa6d2dc0 0x5629aa6d51b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb5a9c79864da4d server_cookie=7f9dba999a1689aa in_seq=0 out_seq=0 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.566+0000 7fb3d77fe640 10 client.17850 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2264803055 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629aa62ebb0 con 0x5629aa6d2dc0 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb3d77fe640 10 client.17850 handle_command_reply: tid=0 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 client.17850 shutdown 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 msgr2=0x5629aa6c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c73c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8005ab0 tx=0x7fb3e8009590 comp rx=0 tx=0).stop 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 shutdown_connections 2024-04-24T23:23:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa6d2dc0 0x5629aa6d51b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6c2d90 0x5629aa6ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6bdc20 0x5629aa6c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb3d77fe640 0 client.17850 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 --2- 172.21.15.33:0/2264803055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c73c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 >> 172.21.15.33:0/2264803055 conn(0x5629aa6b88a0 msgr2=0x5629aa6b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.567+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 shutdown_connections 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.568+0000 7fb4011fd740 1 -- 172.21.15.33:0/2264803055 wait complete. 2024-04-24T23:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.568+0000 7fb4011fd740 20 client.17850 trim_cache size 0 max 16384 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.581+0000 7fb4011fd740 1 Processor -- start 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.581+0000 7fb4011fd740 1 -- start start 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bdc20 0x5629aa6ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb4011fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81aee00 con 0x5629aa6bd230 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb4011fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81ac6d0 con 0x5629aa6c2d90 2024-04-24T23:23:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb4011fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81aa520 con 0x5629aa6bdc20 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42912/0 (socket says 172.21.15.33:42912) 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 -- 172.21.15.33:0/936839364 learned_addr learned my addr 172.21.15.33:0/936839364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bdc20 0x5629aa6ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 -- 172.21.15.33:0/936839364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bdc20 msgr2=0x5629aa6ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bdc20 0x5629aa6ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 -- 172.21.15.33:0/936839364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 msgr2=0x5629aa6c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 -- 172.21.15.33:0/936839364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aa3ff7d0 con 0x5629aa6bd230 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c66b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006010 tx=0x7fb3e80097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3a0a9c4835834b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3ddffb640 1 -- 172.21.15.33:0/936839364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e804c070 con 0x5629aa6bd230 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3ddffb640 1 -- 172.21.15.33:0/936839364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e804d040 con 0x5629aa6bd230 2024-04-24T23:23:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.582+0000 7fb3ddffb640 1 -- 172.21.15.33:0/936839364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8040070 con 0x5629aa6bd230 2024-04-24T23:23:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 -- 172.21.15.33:0/936839364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 msgr2=0x5629aa6c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c66b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006010 tx=0x7fb3e80097c0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 -- 172.21.15.33:0/936839364 shutdown_connections 2024-04-24T23:23:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa6c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bdc20 0x5629aa6ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 --2- 172.21.15.33:0/936839364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bd230 0x5629aa6c66b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 -- 172.21.15.33:0/936839364 >> 172.21.15.33:0/936839364 conn(0x5629aa6b88a0 msgr2=0x5629aa6bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 -- 172.21.15.33:0/936839364 shutdown_connections 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.583+0000 7fb4011fd740 1 -- 172.21.15.33:0/936839364 wait complete. 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 -- start start 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 Processor -- start 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa737150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81aab90 con 0x5629aa6bdc20 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81ac000 con 0x5629aa6c2d90 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81ab260 con 0x5629aa6bd230 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb4011fd740 10 client.0 _ll_register_callbacks cb 0x7fb3ec00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb3f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb3f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42926/0 (socket says 172.21.15.33:42926) 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb3f77fe640 1 -- 172.21.15.33:0/410110886 learned_addr learned my addr 172.21.15.33:0/410110886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb3fc9b8640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa737150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.584+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3f77fe640 1 -- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 msgr2=0x5629aa6cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3f77fe640 1 -- 172.21.15.33:0/410110886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 msgr2=0x5629aa737150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa737150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3f77fe640 1 -- 172.21.15.33:0/410110886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629aa737710 con 0x5629aa6bdc20 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3f77fe640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6cbe60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec099ee0 tx=0x7fb3ec09b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb9d80b0748d28b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3d77fe640 1 -- 172.21.15.33:0/410110886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0a00c0 con 0x5629aa6bdc20 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3d77fe640 1 -- 172.21.15.33:0/410110886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3ec0a4bd0 con 0x5629aa6bdc20 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e804d040 con 0x5629aa6bdc20 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3d77fe640 1 -- 172.21.15.33:0/410110886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec0ac050 con 0x5629aa6bdc20 2024-04-24T23:23:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 10 client.17856 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629aa738b10 con 0x5629aa6bdc20 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3d77fe640 1 -- 172.21.15.33:0/410110886 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e804d040 con 0x5629aa6bdc20 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb3d77fe640 1 -- 172.21.15.33:0/410110886 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb3ec09b160 con 0x5629aa6bdc20 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 10 client.17856 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 20 client.17856 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa739540 0x5629aa73b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 4 client.17856 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.585+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629aa628f00 con 0x5629aa739540 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa739540 0x5629aa73b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb3d77fe640 10 client.17856 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb3f7fff640 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa739540 0x5629aa73b930 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bce1683ff1972ec server_cookie=80a7b25a80c02d2c in_seq=0 out_seq=0 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb3d77fe640 1 -- 172.21.15.33:0/410110886 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629aa628f00 con 0x5629aa739540 2024-04-24T23:23:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb3d77fe640 10 client.17856 handle_command_reply: tid=0 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 client.17856 shutdown 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 msgr2=0x5629aa6cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6cbe60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec099ee0 tx=0x7fb3ec09b790 comp rx=0 tx=0).stop 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 shutdown_connections 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5629aa739540 0x5629aa73b930 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629aa6c2d90 0x5629aa737150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb3d77fe640 0 client.17856 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aa6bdc20 0x5629aa6cbe60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 --2- 172.21.15.33:0/410110886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629aa6bd230 0x5629aa6cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.586+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 >> 172.21.15.33:0/410110886 conn(0x5629aa6b88a0 msgr2=0x5629aa6be460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 shutdown_connections 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb4011fd740 1 -- 172.21.15.33:0/410110886 wait complete. 2024-04-24T23:23:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb4011fd740 20 client.17856 trim_cache size 0 max 16384 2024-04-24T23:23:29.595 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:23:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3e00a2b90 msgr2=0x7fb3e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3e00a2b90 0x7fb3e00a5050 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009fd0 tx=0x7fb3f000ff60 comp rx=0 tx=0).stop 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 msgr2=0x7fb3f819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 0x7fb3f819a270 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e9f0 tx=0x7fb3ec00edf0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 shutdown_connections 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb3e00a2b90 0x7fb3e00a5050 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3f8102250 0x7fb3f81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f8101890 0x7fb3f819a270 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 --2- 172.21.15.33:0/3764070175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f81006a0 0x7fb3f8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 >> 172.21.15.33:0/3764070175 conn(0x7fb3f80fbc10 msgr2=0x7fb3f80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 shutdown_connections 2024-04-24T23:23:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:29.587+0000 7fb3fe41b640 1 -- 172.21.15.33:0/3764070175 wait complete. 2024-04-24T23:23:30.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:30 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1788596684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:23:30.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:30 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1788596684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:23:30.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:30 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1788596684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:23:30.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1340071 12.41 MB/sec execute 455 sec latency 22.516 ms 2024-04-24T23:23:30.602 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:23:30.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:30.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:23:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 -- 172.21.15.33:0/631883716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 msgr2=0x7f5c54101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 --2- 172.21.15.33:0/631883716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54101cf0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009900 tx=0x7f5c4802a410 comp rx=0 tx=0).stop 2024-04-24T23:23:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 -- 172.21.15.33:0/631883716 shutdown_connections 2024-04-24T23:23:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 --2- 172.21.15.33:0/631883716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 0x7f5c54108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 --2- 172.21.15.33:0/631883716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54101cf0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 --2- 172.21.15.33:0/631883716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100680 0x7f5c54100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 -- 172.21.15.33:0/631883716 >> 172.21.15.33:0/631883716 conn(0x7f5c540fbbf0 msgr2=0x7f5c540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 -- 172.21.15.33:0/631883716 shutdown_connections 2024-04-24T23:23:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.736+0000 7f5c59cee640 1 -- 172.21.15.33:0/631883716 wait complete. 2024-04-24T23:23:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 Processor -- start 2024-04-24T23:23:30.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 -- start start 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100680 0x7f5c54104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 0x7f5c54103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410c2e0 con 0x7f5c54101870 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410c160 con 0x7f5c54102230 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c59cee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410c460 con 0x7f5c54100680 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42936/0 (socket says 172.21.15.33:42936) 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c52ffd640 1 -- 172.21.15.33:0/3234789697 learned_addr learned my addr 172.21.15.33:0/3234789697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.737+0000 7f5c53fff640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 0x7f5c54103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c537fe640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100680 0x7f5c54104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c52ffd640 1 -- 172.21.15.33:0/3234789697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100680 msgr2=0x7f5c54104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100680 0x7f5c54104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c52ffd640 1 -- 172.21.15.33:0/3234789697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 msgr2=0x7f5c54103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 0x7f5c54103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c52ffd640 1 -- 172.21.15.33:0/3234789697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54103a20 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c53fff640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 0x7f5c54103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54102c50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009230 tx=0x7f5c480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c50ff9640 1 -- 172.21.15.33:0/3234789697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4803f030 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c541a1b60 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541a2100 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c50ff9640 1 -- 172.21.15.33:0/3234789697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c48003ce0 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.738+0000 7f5c50ff9640 1 -- 172.21.15.33:0/3234789697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48049060 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.740+0000 7f5c50ff9640 1 -- 172.21.15.33:0/3234789697 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5c48042030 con 0x7f5c54101870 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.740+0000 7f5c5c9f7740 1 Processor -- start 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 -- start start 2024-04-24T23:23:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b40301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b4035280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4037d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5654b3f932c0 con 0x5654b40358f0 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5654b3f3b910 con 0x5654b402fd90 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654b3f8f280 con 0x5654b4030780 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4037d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b40301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b40301b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33234/0 (socket says 172.21.15.33:33234) 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b4035280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c53fff640 1 -- 172.21.15.33:0/3629820641 learned_addr learned my addr 172.21.15.33:0/3629820641 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c50ff9640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c3c0a2b90 0x7f5c3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 -- 172.21.15.33:0/3629820641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 msgr2=0x5654b40301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b40301b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 -- 172.21.15.33:0/3629820641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 msgr2=0x5654b4037d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4037d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 -- 172.21.15.33:0/3629820641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b3f4cfb0 con 0x5654b4030780 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4037d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c53fff640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c3c0a2b90 0x7f5c3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c50ff9640 1 -- 172.21.15.33:0/3234789697 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c480ef600 con 0x7f5c54101870 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.741+0000 7f5c537fe640 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b4035280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009b40 tx=0x7f5c440113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b58b0d6ca2d4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c39ffb640 1 -- 172.21.15.33:0/3629820641 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44011e10 con 0x5654b4030780 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c39ffb640 1 -- 172.21.15.33:0/3629820641 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44012930 con 0x5654b4030780 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c39ffb640 1 -- 172.21.15.33:0/3629820641 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400fa60 con 0x5654b4030780 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/3629820641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 msgr2=0x5654b4035280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c53fff640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c3c0a2b90 0x7f5c3c0a5050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5654b4030fe0 tx=0x7f5c4c004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b4035280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009b40 tx=0x7f5c440113e0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/3629820641 shutdown_connections 2024-04-24T23:23:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4037d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b4035280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/3629820641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b40301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.742+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/3629820641 >> 172.21.15.33:0/3629820641 conn(0x5654b3ff7950 msgr2=0x5654b402b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/3629820641 shutdown_connections 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/3629820641 wait complete. 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 -- start start 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 Processor -- start 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b403a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4041750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5654b40388a0 con 0x5654b40358f0 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5654b3f4ee90 con 0x5654b402fd90 2024-04-24T23:23:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654b4038a20 con 0x5654b4030780 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.743+0000 7f5c5c9f7740 10 client.0 _ll_register_callbacks cb 0x5654b3f8a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4041750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b403a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33238/0 (socket says 172.21.15.33:33238) 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b403a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53552/0 (socket says 172.21.15.33:53552) 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4041750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42950/0 (socket says 172.21.15.33:42950) 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c537fe640 1 -- 172.21.15.33:0/852408880 learned_addr learned my addr 172.21.15.33:0/852408880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 -- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 msgr2=0x5654b403a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b403a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 -- 172.21.15.33:0/852408880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 msgr2=0x5654b403a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 -- 172.21.15.33:0/852408880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b3d6f7e0 con 0x5654b40358f0 2024-04-24T23:23:30.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b403a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c537fe640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c53fff640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4041750 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009fd0 tx=0x7f5c4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a6a35c9fc44477 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c32ffd640 1 -- 172.21.15.33:0/852408880 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c020960 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c44012930 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c32ffd640 1 -- 172.21.15.33:0/852408880 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c4c020b00 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c32ffd640 1 -- 172.21.15.33:0/852408880 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c4c020d00 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.744+0000 7f5c5c9f7740 10 client.17868 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654b3dcff70 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c32ffd640 1 -- 172.21.15.33:0/852408880 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44012930 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c32ffd640 1 -- 172.21.15.33:0/852408880 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c4c01e440 con 0x5654b40358f0 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 10 client.17868 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 20 client.17868 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b4046150 0x5654b40485b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 4 client.17868 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654b3f675a0 con 0x5654b4046150 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.745+0000 7f5c537fe640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b4046150 0x5654b40485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.746+0000 7f5c32ffd640 10 client.17868 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.746+0000 7f5c537fe640 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b4046150 0x5654b40485b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b258930960db2160 server_cookie=dc65bceac5184f7e in_seq=0 out_seq=0 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.746+0000 7f5c32ffd640 1 -- 172.21.15.33:0/852408880 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654b3f675a0 con 0x5654b4046150 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.746+0000 7f5c32ffd640 10 client.17868 handle_command_reply: tid=0 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.746+0000 7f5c5c9f7740 1 client.17868 shutdown 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 msgr2=0x5654b4041750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4041750 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009fd0 tx=0x7f5c4c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 shutdown_connections 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b4046150 0x5654b40485b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b40358f0 0x5654b4041750 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b4030780 0x5654b403a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c32ffd640 0 client.17868 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/852408880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 >> 172.21.15.33:0/852408880 conn(0x5654b3ff7950 msgr2=0x5654b402a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 shutdown_connections 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/852408880 wait complete. 2024-04-24T23:23:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.747+0000 7f5c5c9f7740 20 client.17868 trim_cache size 0 max 16384 2024-04-24T23:23:30.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 Processor -- start 2024-04-24T23:23:30.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 -- start start 2024-04-24T23:23:30.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b402fd90 0x5654b4031be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b4032120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b40358f0 0x5654b4041710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541aeea0 con 0x5654b4030780 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ac770 con 0x5654b40358f0 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541aa5c0 con 0x5654b402fd90 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b4032120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b402fd90 0x5654b4031be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b4032120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42962/0 (socket says 172.21.15.33:42962) 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 -- 172.21.15.33:0/4038719850 learned_addr learned my addr 172.21.15.33:0/4038719850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c53fff640 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b40358f0 0x5654b4041710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 -- 172.21.15.33:0/4038719850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b402fd90 msgr2=0x5654b4031be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b402fd90 0x5654b4031be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 -- 172.21.15.33:0/4038719850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b40358f0 msgr2=0x5654b4041710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b40358f0 0x5654b4041710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.767+0000 7f5c52ffd640 1 -- 172.21.15.33:0/4038719850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b3c67170 con 0x5654b4030780 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b4032120 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5c480f52f0 tx=0x7f5c480ef900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3170a17cf10908 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c397fa640 1 -- 172.21.15.33:0/4038719850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48035030 con 0x5654b4030780 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c397fa640 1 -- 172.21.15.33:0/4038719850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c480bc1d0 con 0x5654b4030780 2024-04-24T23:23:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c397fa640 1 -- 172.21.15.33:0/4038719850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c480b9c00 con 0x5654b4030780 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/4038719850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 msgr2=0x5654b4032120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b4032120 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5c480f52f0 tx=0x7f5c480ef900 comp rx=0 tx=0).stop 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/4038719850 shutdown_connections 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b40358f0 0x5654b4041710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b4032120 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/4038719850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b402fd90 0x5654b4031be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.768+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/4038719850 >> 172.21.15.33:0/4038719850 conn(0x5654b3ff7950 msgr2=0x5654b403a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/4038719850 shutdown_connections 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/4038719850 wait complete. 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 -- start start 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 Processor -- start 2024-04-24T23:23:30.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b403ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b40358f0 0x5654b403fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541aac30 con 0x5654b4030780 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ac0a0 con 0x5654b402fd90 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ab300 con 0x5654b40358f0 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c5c9f7740 10 client.0 _ll_register_callbacks cb 0x7f5c4c003c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b403ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b403ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42968/0 (socket says 172.21.15.33:42968) 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c52ffd640 1 -- 172.21.15.33:0/1941438111 learned_addr learned my addr 172.21.15.33:0/1941438111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c537fe640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.769+0000 7f5c53fff640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b40358f0 0x5654b403fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c52ffd640 1 -- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b40358f0 msgr2=0x5654b403fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b40358f0 0x5654b403fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c52ffd640 1 -- 172.21.15.33:0/1941438111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 msgr2=0x5654b403e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c52ffd640 1 -- 172.21.15.33:0/1941438111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b40404c0 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c52ffd640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b403ef10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ecdc0 tx=0x7f5c480bb680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e8982dcf1d9c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c32ffd640 1 -- 172.21.15.33:0/1941438111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48035070 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c32ffd640 1 -- 172.21.15.33:0/1941438111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c48003800 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c480bc1d0 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c32ffd640 1 -- 172.21.15.33:0/1941438111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c480bbad0 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c5c9f7740 10 client.17874 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654b40418c0 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c32ffd640 1 -- 172.21.15.33:0/1941438111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c480bc1d0 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.770+0000 7f5c32ffd640 1 -- 172.21.15.33:0/1941438111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c4804c060 con 0x5654b4030780 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c5c9f7740 10 client.17874 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c5c9f7740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c5c9f7740 20 client.17874 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b40accb0 0x5654b40af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c5c9f7740 4 client.17874 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:23:30.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5654b3fa0800 con 0x5654b40accb0 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c537fe640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b40accb0 0x5654b40af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c32ffd640 10 client.17874 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.771+0000 7f5c537fe640 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b40accb0 0x5654b40af0d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f5b81e3442a5d3 server_cookie=6346f83483884ac1 in_seq=0 out_seq=0 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c32ffd640 1 -- 172.21.15.33:0/1941438111 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5654b3fa0800 con 0x5654b40accb0 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c32ffd640 10 client.17874 handle_command_reply: tid=0 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 client.17874 shutdown 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 msgr2=0x5654b403ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b403ef10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ecdc0 tx=0x7f5c480bb680 comp rx=0 tx=0).stop 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 shutdown_connections 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654b40accb0 0x5654b40af0d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654b40358f0 0x5654b403fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654b4030780 0x5654b403ef10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 --2- 172.21.15.33:0/1941438111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654b402fd90 0x5654b403e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c32ffd640 0 client.17874 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.772+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 >> 172.21.15.33:0/1941438111 conn(0x5654b3ff7950 msgr2=0x5654b403a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 shutdown_connections 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c5c9f7740 1 -- 172.21.15.33:0/1941438111 wait complete. 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c5c9f7740 20 client.17874 trim_cache size 0 max 16384 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "a398e8cd-227c-471c-9928-bc953fdb1de3", 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:23:30.780 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:23:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c3c0a2b90 msgr2=0x7f5c3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c3c0a2b90 0x7f5c3c0a5050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5654b4030fe0 tx=0x7f5c4c004a40 comp rx=0 tx=0).stop 2024-04-24T23:23:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 msgr2=0x7f5c54102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54102c50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009230 tx=0x7f5c480042f0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 shutdown_connections 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c3c0a2b90 0x7f5c3c0a5050 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c54102230 0x7f5c54103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c54101870 0x7f5c54102c50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 --2- 172.21.15.33:0/3234789697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100680 0x7f5c54104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 >> 172.21.15.33:0/3234789697 conn(0x7f5c540fbbf0 msgr2=0x7f5c540fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 shutdown_connections 2024-04-24T23:23:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.773+0000 7f5c59cee640 1 -- 172.21.15.33:0/3234789697 wait complete. 2024-04-24T23:23:30.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:23:30.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:23:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 -- 172.21.15.33:0/2506729434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c1006a0 msgr2=0x7fb16c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 --2- 172.21.15.33:0/2506729434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c1006a0 0x7fb16c100ac0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a410 comp rx=0 tx=0).stop 2024-04-24T23:23:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 -- 172.21.15.33:0/2506729434 shutdown_connections 2024-04-24T23:23:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 --2- 172.21.15.33:0/2506729434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb16c102250 0x7fb16c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 --2- 172.21.15.33:0/2506729434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 0x7fb16c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 --2- 172.21.15.33:0/2506729434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c1006a0 0x7fb16c100ac0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 -- 172.21.15.33:0/2506729434 >> 172.21.15.33:0/2506729434 conn(0x7fb16c0fbc30 msgr2=0x7fb16c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 -- 172.21.15.33:0/2506729434 shutdown_connections 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.894+0000 7fb172c76640 1 -- 172.21.15.33:0/2506729434 wait complete. 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 Processor -- start 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 -- start start 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb16c1006a0 0x7fb16c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 0x7fb16c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 0x7fb16c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c10c320 con 0x7fb16c102250 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c10c1a0 con 0x7fb16c101890 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb172c76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c10c4a0 con 0x7fb16c1006a0 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb171213640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 0x7fb16c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 0x7fb16c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb171213640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 0x7fb16c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42998/0 (socket says 172.21.15.33:42998) 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb171213640 1 -- 172.21.15.33:0/2934595059 learned_addr learned my addr 172.21.15.33:0/2934595059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.895+0000 7fb170a12640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb16c1006a0 0x7fb16c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb16bfff640 1 -- 172.21.15.33:0/2934595059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb16c1006a0 msgr2=0x7fb16c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb16bfff640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb16c1006a0 0x7fb16c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb16bfff640 1 -- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 msgr2=0x7fb16c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb16bfff640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 0x7fb16c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb16bfff640 1 -- 172.21.15.33:0/2934595059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c19f720 con 0x7fb16c101890 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb171213640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 0x7fb16c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb16bfff640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 0x7fb16c198040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e9f0 tx=0x7fb15800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb169ffb640 1 -- 172.21.15.33:0/2934595059 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15800ccc0 con 0x7fb16c101890 2024-04-24T23:23:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c19fa10 con 0x7fb16c101890 2024-04-24T23:23:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb169ffb640 1 -- 172.21.15.33:0/2934595059 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb158010910 con 0x7fb16c101890 2024-04-24T23:23:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb169ffb640 1 -- 172.21.15.33:0/2934595059 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158018960 con 0x7fb16c101890 2024-04-24T23:23:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.896+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c19ffb0 con 0x7fb16c101890 2024-04-24T23:23:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.898+0000 7fb169ffb640 1 -- 172.21.15.33:0/2934595059 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb158010ac0 con 0x7fb16c101890 2024-04-24T23:23:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 Processor -- start 2024-04-24T23:23:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 -- start start 2024-04-24T23:23:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb169ffb640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb15c0a2b90 0x7fb15c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd578660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd57d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd580230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603cd4dc6b0 con 0x5603cd57dda0 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5603cd484880 con 0x5603cd578c30 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb175986740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5603cd4d8670 con 0x5603cd578240 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.899+0000 7fb169ffb640 1 -- 172.21.15.33:0/2934595059 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb1580cf7a0 con 0x7fb16c101890 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd578660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb171213640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb15c0a2b90 0x7fb15c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd578660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53594/0 (socket says 172.21.15.33:53594) 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 -- 172.21.15.33:0/2411097630 learned_addr learned my addr 172.21.15.33:0/2411097630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb171213640 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd580230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb16bfff640 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd57d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 -- 172.21.15.33:0/2411097630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 msgr2=0x5603cd57d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd57d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 -- 172.21.15.33:0/2411097630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 msgr2=0x5603cd580230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd580230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 -- 172.21.15.33:0/2411097630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cd2ba7d0 con 0x5603cd578240 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb171213640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb15c0a2b90 0x7fb15c0a5050 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb164009aa0 tx=0x7fb164011700 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb170a12640 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd578660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb160031610 tx=0x7fb160005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d0803ae9e3ed2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb156ffd640 1 -- 172.21.15.33:0/2411097630 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160040070 con 0x5603cd578240 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb156ffd640 1 -- 172.21.15.33:0/2411097630 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160004030 con 0x5603cd578240 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.900+0000 7fb156ffd640 1 -- 172.21.15.33:0/2411097630 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035040 con 0x5603cd578240 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 -- 172.21.15.33:0/2411097630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 msgr2=0x5603cd578660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd578660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb160031610 tx=0x7fb160005de0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 -- 172.21.15.33:0/2411097630 shutdown_connections 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd580230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd57d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 --2- 172.21.15.33:0/2411097630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd578660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 -- 172.21.15.33:0/2411097630 >> 172.21.15.33:0/2411097630 conn(0x5603cd5738b0 msgr2=0x5603cd573cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 -- 172.21.15.33:0/2411097630 shutdown_connections 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.901+0000 7fb175986740 1 -- 172.21.15.33:0/2411097630 wait complete. 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 -- start start 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 Processor -- start 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578240 0x5603cd586a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 0x5603cd586fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603cd585100 con 0x5603cd578c30 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5603cd497f50 con 0x5603cd578240 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5603cd585280 con 0x5603cd57dda0 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb175986740 10 client.0 _ll_register_callbacks cb 0x5603cd4d3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 0x5603cd586fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb170a12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578240 0x5603cd586a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb171213640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 0x5603cd586fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43026/0 (socket says 172.21.15.33:43026) 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.902+0000 7fb16bfff640 1 -- 172.21.15.33:0/3166722302 learned_addr learned my addr 172.21.15.33:0/3166722302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb16bfff640 1 -- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 msgr2=0x5603cd58df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb16bfff640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb16bfff640 1 -- 172.21.15.33:0/3166722302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578240 msgr2=0x5603cd586a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb16bfff640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578240 0x5603cd586a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb16bfff640 1 -- 172.21.15.33:0/3166722302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cd1f03f0 con 0x5603cd578c30 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb170a12640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578240 0x5603cd586a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb171213640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb16bfff640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 0x5603cd586fb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb15800c6a0 tx=0x7fb15809be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d52dbb36062c94e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb14bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb14bfff640 1 -- 172.21.15.33:0/3166722302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580cf450 con 0x5603cd578c30 2024-04-24T23:23:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb14bfff640 1 -- 172.21.15.33:0/3166722302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb15809b110 con 0x5603cd578c30 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160004030 con 0x5603cd578c30 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb14bfff640 1 -- 172.21.15.33:0/3166722302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb15809b310 con 0x5603cd578c30 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb175986740 10 client.17886 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603cd1b00f0 con 0x5603cd578c30 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.903+0000 7fb14bfff640 1 -- 172.21.15.33:0/3166722302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160004030 con 0x5603cd578c30 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb14bfff640 1 -- 172.21.15.33:0/3166722302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb158018b00 con 0x5603cd578c30 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb175986740 10 client.17886 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb175986740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb175986740 20 client.17886 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb175986740 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5928b0 0x5603cd594d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb175986740 4 client.17886 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603cd4e9bf0 con 0x5603cd5928b0 2024-04-24T23:23:30.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb170a12640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5928b0 0x5603cd594d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb170a12640 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5928b0 0x5603cd594d00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea81d24bb8272f01 server_cookie=30420a4e3e5d9958 in_seq=0 out_seq=0 2024-04-24T23:23:30.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.904+0000 7fb14bfff640 10 client.17886 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb14bfff640 1 -- 172.21.15.33:0/3166722302 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603cd4e9bf0 con 0x5603cd5928b0 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb14bfff640 10 client.17886 handle_command_reply: tid=0 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 client.17886 shutdown 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 msgr2=0x5603cd586fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 0x5603cd586fb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb15800c6a0 tx=0x7fb15809be00 comp rx=0 tx=0).stop 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 shutdown_connections 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5928b0 0x5603cd594d00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578c30 0x5603cd586fb0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb14bfff640 0 client.17886 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 --2- 172.21.15.33:0/3166722302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578240 0x5603cd586a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.905+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 >> 172.21.15.33:0/3166722302 conn(0x5603cd5738b0 msgr2=0x5603cd572af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.906+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 shutdown_connections 2024-04-24T23:23:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.906+0000 7fb175986740 1 -- 172.21.15.33:0/3166722302 wait complete. 2024-04-24T23:23:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.906+0000 7fb175986740 20 client.17886 trim_cache size 0 max 16384 2024-04-24T23:23:30.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 Processor -- start 2024-04-24T23:23:30.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 -- start start 2024-04-24T23:23:30.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd586090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd576860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1aaae0 con 0x5603cd57dda0 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a83b0 con 0x5603cd578c30 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb175986740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a6200 con 0x5603cd578240 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb171213640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd576860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb170a12640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd586090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33308/0 (socket says 172.21.15.33:33308) 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.925+0000 7fb16bfff640 1 -- 172.21.15.33:0/3755635566 learned_addr learned my addr 172.21.15.33:0/3755635566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb171213640 1 -- 172.21.15.33:0/3755635566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 msgr2=0x5603cd586090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb171213640 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd586090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb171213640 1 -- 172.21.15.33:0/3755635566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 msgr2=0x5603cd58df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb171213640 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb171213640 1 -- 172.21.15.33:0/3755635566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cd574620 con 0x5603cd57dda0 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb16bfff640 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb171213640 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd576860 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb1640049d0 tx=0x7fb164004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31357fac99cce323 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb1567fc640 1 -- 172.21.15.33:0/3755635566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400ed80 con 0x5603cd57dda0 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb1567fc640 1 -- 172.21.15.33:0/3755635566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16402a8e0 con 0x5603cd57dda0 2024-04-24T23:23:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.926+0000 7fb1567fc640 1 -- 172.21.15.33:0/3755635566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640217a0 con 0x5603cd57dda0 2024-04-24T23:23:30.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 -- 172.21.15.33:0/3755635566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 msgr2=0x5603cd576860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd576860 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb1640049d0 tx=0x7fb164004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 -- 172.21.15.33:0/3755635566 shutdown_connections 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd57dda0 0x5603cd576860 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 --2- 172.21.15.33:0/3755635566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd578240 0x5603cd586090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 -- 172.21.15.33:0/3755635566 >> 172.21.15.33:0/3755635566 conn(0x5603cd5738b0 msgr2=0x5603cd57a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 -- 172.21.15.33:0/3755635566 shutdown_connections 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 -- 172.21.15.33:0/3755635566 wait complete. 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 -- start start 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.927+0000 7fb175986740 1 Processor -- start 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578240 0x5603cd58b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a6870 con 0x5603cd578240 2024-04-24T23:23:30.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a7ce0 con 0x5603cd578c30 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a6f40 con 0x5603cd57dda0 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb175986740 10 client.0 _ll_register_callbacks cb 0x7fb15809ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb170a12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578240 0x5603cd58b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53610/0 (socket says 172.21.15.33:53610) 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 -- 172.21.15.33:0/823227846 learned_addr learned my addr 172.21.15.33:0/823227846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb16bfff640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 -- 172.21.15.33:0/823227846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 msgr2=0x5603cd58b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 -- 172.21.15.33:0/823227846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578240 msgr2=0x5603cd58b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578240 0x5603cd58b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 -- 172.21.15.33:0/823227846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cd58cf70 con 0x5603cd57dda0 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb170a12640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578240 0x5603cd58b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:30.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb171213640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58c980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb16400ecd0 tx=0x7fb164009a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835cd51379e0e46d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.928+0000 7fb14bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb14bfff640 1 -- 172.21.15.33:0/823227846 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164039060 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb14bfff640 1 -- 172.21.15.33:0/823227846 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb164027040 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16402a8e0 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb14bfff640 1 -- 172.21.15.33:0/823227846 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb164040050 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 10 client.27364 fetch_fsmap learned FSMap version 27 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603cd58e280 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb14bfff640 1 -- 172.21.15.33:0/823227846 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16402a8e0 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb14bfff640 1 -- 172.21.15.33:0/823227846 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb164020070 con 0x5603cd57dda0 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 10 client.27364 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 20 client.27364 populate_metadata read hostname 'smithi033' 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5f9640 0x5603cd5fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 4 client.27364 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603cd4e3f40 con 0x5603cd5f9640 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.929+0000 7fb170a12640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5f9640 0x5603cd5fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb14bfff640 10 client.27364 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb170a12640 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5f9640 0x5603cd5fba60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fda10788b78795e server_cookie=5bc890f22a292df in_seq=0 out_seq=0 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb14bfff640 1 -- 172.21.15.33:0/823227846 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5603cd4e3f40 con 0x5603cd5f9640 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb14bfff640 10 client.27364 handle_command_reply: tid=0 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 client.27364 shutdown 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 msgr2=0x5603cd58c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58c980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb16400ecd0 tx=0x7fb164009a30 comp rx=0 tx=0).stop 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 shutdown_connections 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5603cd5f9640 0x5603cd5fba60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603cd57dda0 0x5603cd58c980 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603cd578c30 0x5603cd58b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb14bfff640 0 client.27364 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:23:30.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 --2- 172.21.15.33:0/823227846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603cd578240 0x5603cd58b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 >> 172.21.15.33:0/823227846 conn(0x5603cd5738b0 msgr2=0x5603cd572260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 shutdown_connections 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 1 -- 172.21.15.33:0/823227846 wait complete. 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.930+0000 7fb175986740 20 client.27364 trim_cache size 0 max 16384 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "a398e8cd-227c-471c-9928-bc953fdb1de3": { 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:23:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "a398e8cd-227c-471c-9928-bc953fdb1de3", 2024-04-24T23:23:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:23:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:23:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:23:30.939 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:23:30.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb15c0a2b90 msgr2=0x7fb15c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb15c0a2b90 0x7fb15c0a5050 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb164009aa0 tx=0x7fb164011700 comp rx=0 tx=0).stop 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 msgr2=0x7fb16c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 0x7fb16c198040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e9f0 tx=0x7fb15800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 shutdown_connections 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb15c0a2b90 0x7fb15c0a5050 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16c102250 0x7fb16c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb16c101890 0x7fb16c198040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 --2- 172.21.15.33:0/2934595059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb16c1006a0 0x7fb16c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 >> 172.21.15.33:0/2934595059 conn(0x7fb16c0fbc30 msgr2=0x7fb16c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 shutdown_connections 2024-04-24T23:23:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:23:30.931+0000 7fb172c76640 1 -- 172.21.15.33:0/2934595059 wait complete. 2024-04-24T23:23:30.946 INFO:tasks.cephfs.filesystem:scrub status for tag:a398e8cd-227c-471c-9928-bc953fdb1de3 - {'path': '/', 'tag': 'a398e8cd-227c-471c-9928-bc953fdb1de3', 'options': 'recursive,force'} 2024-04-24T23:23:31.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:31 smithi033 ceph-mon[29201]: pgmap v696: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T23:23:31.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:31 smithi038 ceph-mon[32450]: pgmap v696: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T23:23:31.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:31 smithi117 ceph-mon[32529]: pgmap v696: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T23:23:31.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1342771 12.42 MB/sec execute 456 sec latency 15.985 ms 2024-04-24T23:23:32.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:32 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:23:32.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:32 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:32 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:23:32.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:32 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:23:32.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:32 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:32 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:32 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:23:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:32 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:32 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:23:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:32 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:23:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:32 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:32 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:32 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:23:32.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:32 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:32 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:23:32.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:32 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:23:32.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:32 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:32 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:23:32.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1345263 12.42 MB/sec execute 457 sec latency 27.416 ms 2024-04-24T23:23:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:33 smithi038 ceph-mon[32450]: pgmap v697: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:23:33.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:33 smithi117 ceph-mon[32529]: pgmap v697: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:23:33.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:33 smithi033 ceph-mon[29201]: pgmap v697: 129 pgs: 129 active+clean; 290 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-04-24T23:23:33.463 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1346854 12.41 MB/sec execute 458 sec latency 24.198 ms 2024-04-24T23:23:34.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1349478 12.42 MB/sec execute 459 sec latency 97.216 ms 2024-04-24T23:23:35.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:35 smithi038 ceph-mon[32450]: pgmap v698: 129 pgs: 129 active+clean; 284 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-04-24T23:23:35.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:35 smithi117 ceph-mon[32529]: pgmap v698: 129 pgs: 129 active+clean; 284 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-04-24T23:23:35.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:35 smithi033 ceph-mon[29201]: pgmap v698: 129 pgs: 129 active+clean; 284 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-04-24T23:23:35.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1350691 12.40 MB/sec execute 460 sec latency 22.527 ms 2024-04-24T23:23:36.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1353392 12.41 MB/sec execute 461 sec latency 17.426 ms 2024-04-24T23:23:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:37 smithi038 ceph-mon[32450]: pgmap v699: 129 pgs: 129 active+clean; 279 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-04-24T23:23:37.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:37 smithi117 ceph-mon[32529]: pgmap v699: 129 pgs: 129 active+clean; 279 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-04-24T23:23:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:37 smithi033 ceph-mon[29201]: pgmap v699: 129 pgs: 129 active+clean; 279 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-04-24T23:23:37.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1355560 12.40 MB/sec execute 462 sec latency 18.831 ms 2024-04-24T23:23:38.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:38 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:23:38.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:38 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:23:38.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:38 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:23:38.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1357353 12.40 MB/sec execute 463 sec latency 24.036 ms 2024-04-24T23:23:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:39 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:23:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:39 smithi038 ceph-mon[32450]: pgmap v700: 129 pgs: 129 active+clean; 271 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-04-24T23:23:39.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:39 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:23:39.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:39 smithi117 ceph-mon[32529]: pgmap v700: 129 pgs: 129 active+clean; 271 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-04-24T23:23:39.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:39 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:23:39.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:39 smithi033 ceph-mon[29201]: pgmap v700: 129 pgs: 129 active+clean; 271 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-04-24T23:23:39.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1359884 12.40 MB/sec execute 464 sec latency 80.885 ms 2024-04-24T23:23:40.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1361185 12.39 MB/sec execute 465 sec latency 28.277 ms 2024-04-24T23:23:41.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:41 smithi038 ceph-mon[32450]: pgmap v701: 129 pgs: 129 active+clean; 272 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 180 op/s 2024-04-24T23:23:41.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:41 smithi117 ceph-mon[32529]: pgmap v701: 129 pgs: 129 active+clean; 272 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 180 op/s 2024-04-24T23:23:41.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:41 smithi033 ceph-mon[29201]: pgmap v701: 129 pgs: 129 active+clean; 272 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 180 op/s 2024-04-24T23:23:41.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1363891 12.40 MB/sec execute 466 sec latency 18.459 ms 2024-04-24T23:23:42.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1366113 12.39 MB/sec execute 467 sec latency 12.906 ms 2024-04-24T23:23:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:43 smithi038 ceph-mon[32450]: pgmap v702: 129 pgs: 129 active+clean; 275 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T23:23:43.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:43 smithi117 ceph-mon[32529]: pgmap v702: 129 pgs: 129 active+clean; 275 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T23:23:43.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:43 smithi033 ceph-mon[29201]: pgmap v702: 129 pgs: 129 active+clean; 275 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T23:23:43.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1368164 12.39 MB/sec execute 468 sec latency 13.144 ms 2024-04-24T23:23:44.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1370944 12.40 MB/sec execute 469 sec latency 49.135 ms 2024-04-24T23:23:45.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:45 smithi033 ceph-mon[29201]: pgmap v703: 129 pgs: 129 active+clean; 275 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-04-24T23:23:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:45 smithi038 ceph-mon[32450]: pgmap v703: 129 pgs: 129 active+clean; 275 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-04-24T23:23:45.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:45 smithi117 ceph-mon[32529]: pgmap v703: 129 pgs: 129 active+clean; 275 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-04-24T23:23:45.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1372315 12.39 MB/sec execute 470 sec latency 61.181 ms 2024-04-24T23:23:46.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1374705 12.39 MB/sec execute 471 sec latency 31.011 ms 2024-04-24T23:23:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:47 smithi038 ceph-mon[32450]: pgmap v704: 129 pgs: 129 active+clean; 279 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-04-24T23:23:47.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:47 smithi117 ceph-mon[32529]: pgmap v704: 129 pgs: 129 active+clean; 279 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-04-24T23:23:47.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:47 smithi033 ceph-mon[29201]: pgmap v704: 129 pgs: 129 active+clean; 279 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-04-24T23:23:47.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1376073 12.41 MB/sec execute 472 sec latency 54.013 ms 2024-04-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1378832 12.42 MB/sec execute 473 sec latency 20.949 ms 2024-04-24T23:23:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:49 smithi038 ceph-mon[32450]: pgmap v705: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T23:23:49.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:49 smithi117 ceph-mon[32529]: pgmap v705: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T23:23:49.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:49 smithi033 ceph-mon[29201]: pgmap v705: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T23:23:49.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1380998 12.41 MB/sec execute 474 sec latency 19.158 ms 2024-04-24T23:23:50.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1382507 12.41 MB/sec execute 475 sec latency 120.640 ms 2024-04-24T23:23:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:51 smithi038 ceph-mon[32450]: pgmap v706: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-04-24T23:23:51.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:51 smithi117 ceph-mon[32529]: pgmap v706: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-04-24T23:23:51.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:51 smithi033 ceph-mon[29201]: pgmap v706: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-04-24T23:23:51.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1384602 12.40 MB/sec execute 476 sec latency 32.878 ms 2024-04-24T23:23:52.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1386554 12.40 MB/sec execute 477 sec latency 22.750 ms 2024-04-24T23:23:53.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:53 smithi038 ceph-mon[32450]: pgmap v707: 129 pgs: 129 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 285 op/s 2024-04-24T23:23:53.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:53 smithi117 ceph-mon[32529]: pgmap v707: 129 pgs: 129 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 285 op/s 2024-04-24T23:23:53.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:53 smithi033 ceph-mon[29201]: pgmap v707: 129 pgs: 129 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 285 op/s 2024-04-24T23:23:53.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1389531 12.41 MB/sec execute 478 sec latency 10.778 ms 2024-04-24T23:23:54.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1391680 12.40 MB/sec execute 479 sec latency 47.622 ms 2024-04-24T23:23:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:55 smithi038 ceph-mon[32450]: pgmap v708: 129 pgs: 129 active+clean; 268 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-04-24T23:23:55.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:55 smithi117 ceph-mon[32529]: pgmap v708: 129 pgs: 129 active+clean; 268 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-04-24T23:23:55.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:55 smithi033 ceph-mon[29201]: pgmap v708: 129 pgs: 129 active+clean; 268 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-04-24T23:23:55.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1393461 12.40 MB/sec execute 480 sec latency 77.516 ms 2024-04-24T23:23:56.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1396163 12.41 MB/sec execute 481 sec latency 29.277 ms 2024-04-24T23:23:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:57 smithi038 ceph-mon[32450]: pgmap v709: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-04-24T23:23:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:57 smithi117 ceph-mon[32529]: pgmap v709: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-04-24T23:23:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:57 smithi033 ceph-mon[29201]: pgmap v709: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-04-24T23:23:57.464 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1397965 12.40 MB/sec execute 482 sec latency 23.674 ms 2024-04-24T23:23:58.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1400585 12.41 MB/sec execute 483 sec latency 12.478 ms 2024-04-24T23:23:59.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:23:59 smithi038 ceph-mon[32450]: pgmap v710: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-04-24T23:23:59.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:23:59 smithi117 ceph-mon[32529]: pgmap v710: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-04-24T23:23:59.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:23:59 smithi033 ceph-mon[29201]: pgmap v710: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-04-24T23:23:59.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1403446 12.42 MB/sec execute 484 sec latency 18.583 ms 2024-04-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1404687 12.40 MB/sec execute 485 sec latency 42.173 ms 2024-04-24T23:24:00.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:24:00.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 -- 172.21.15.33:0/1245334460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 msgr2=0x7f3f84108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 --2- 172.21.15.33:0/1245334460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f84108ac0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009ca0 tx=0x7f3f7c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 -- 172.21.15.33:0/1245334460 shutdown_connections 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 --2- 172.21.15.33:0/1245334460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f84108ac0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 --2- 172.21.15.33:0/1245334460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84101870 0x7f3f84101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 --2- 172.21.15.33:0/1245334460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f84100680 0x7f3f84100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 -- 172.21.15.33:0/1245334460 >> 172.21.15.33:0/1245334460 conn(0x7f3f840fbbf0 msgr2=0x7f3f840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 -- 172.21.15.33:0/1245334460 shutdown_connections 2024-04-24T23:24:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.054+0000 7f3f8b815640 1 -- 172.21.15.33:0/1245334460 wait complete. 2024-04-24T23:24:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 Processor -- start 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 -- start start 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f84100680 0x7f3f84197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84101870 0x7f3f84197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f8419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8410d400 con 0x7f3f84102230 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8410d280 con 0x7f3f84100680 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f8b815640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8410d580 con 0x7f3f84101870 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f89db2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f8419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f89db2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f8419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59710/0 (socket says 172.21.15.33:59710) 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f89db2640 1 -- 172.21.15.33:0/666585818 learned_addr learned my addr 172.21.15.33:0/666585818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f895b1640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f84100680 0x7f3f84197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.055+0000 7f3f88db0640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84101870 0x7f3f84197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f89db2640 1 -- 172.21.15.33:0/666585818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84101870 msgr2=0x7f3f84197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f89db2640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84101870 0x7f3f84197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f89db2640 1 -- 172.21.15.33:0/666585818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f84100680 msgr2=0x7f3f84197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f89db2640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f84100680 0x7f3f84197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f89db2640 1 -- 172.21.15.33:0/666585818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8419f6a0 con 0x7f3f84102230 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f89db2640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f8419f030 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c031ae0 tx=0x7f3f7c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f7a7fc640 1 -- 172.21.15.33:0/666585818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c003b70 con 0x7f3f84102230 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8419f930 con 0x7f3f84102230 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f7a7fc640 1 -- 172.21.15.33:0/666585818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7c003d10 con 0x7f3f84102230 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f7a7fc640 1 -- 172.21.15.33:0/666585818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c02fc40 con 0x7f3f84102230 2024-04-24T23:24:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.056+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8419fe80 con 0x7f3f84102230 2024-04-24T23:24:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f7a7fc640 1 -- 172.21.15.33:0/666585818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3f7c004050 con 0x7f3f84102230 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 Processor -- start 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 -- start start 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 0x560a40d3f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 0x560a40d44730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d47230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a40ca36b0 con 0x560a40d44da0 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560a40c4b880 con 0x560a40d3f240 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.059+0000 7f3f8e603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560a40c9f670 con 0x560a40d3fc30 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f88db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 0x560a40d3f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f7a7fc640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f700a2b90 0x7f3f700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 0x560a40d44730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f88db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 0x560a40d3f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:45406/0 (socket says 172.21.15.33:45406) 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 0x560a40d44730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47418/0 (socket says 172.21.15.33:47418) 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f88db0640 1 -- 172.21.15.33:0/3198958570 learned_addr learned my addr 172.21.15.33:0/3198958570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f88db0640 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d47230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f89db2640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f700a2b90 0x7f3f700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f7a7fc640 1 -- 172.21.15.33:0/666585818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f7c0f0450 con 0x7f3f84102230 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 -- 172.21.15.33:0/3198958570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 msgr2=0x560a40d3f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 0x560a40d3f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 -- 172.21.15.33:0/3198958570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 msgr2=0x560a40d47230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d47230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 -- 172.21.15.33:0/3198958570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a40a817d0 con 0x560a40d3fc30 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f88db0640 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 0x560a40d3f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f895b1640 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 0x560a40d44730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009bf0 tx=0x7f3f80011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcffce438880f911 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f89db2640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f700a2b90 0x7f3f700a5050 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0edc10 tx=0x7f3f7c0877e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f6f7fe640 1 -- 172.21.15.33:0/3198958570 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80009da0 con 0x560a40d3fc30 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f6f7fe640 1 -- 172.21.15.33:0/3198958570 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f80005020 con 0x560a40d3fc30 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.060+0000 7f3f6f7fe640 1 -- 172.21.15.33:0/3198958570 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80004a50 con 0x560a40d3fc30 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 -- 172.21.15.33:0/3198958570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 msgr2=0x560a40d44730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 0x560a40d44730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009bf0 tx=0x7f3f80011810 comp rx=0 tx=0).stop 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 -- 172.21.15.33:0/3198958570 shutdown_connections 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d47230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3fc30 0x560a40d44730 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3198958570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3f240 0x560a40d3f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 -- 172.21.15.33:0/3198958570 >> 172.21.15.33:0/3198958570 conn(0x560a40d3a8b0 msgr2=0x560a40d3acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 -- 172.21.15.33:0/3198958570 shutdown_connections 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 -- 172.21.15.33:0/3198958570 wait complete. 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.061+0000 7f3f8e603740 1 -- start start 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 Processor -- start 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 0x560a40d49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d49ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d44da0 0x560a40d50be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a40d47d50 con 0x560a40d3f240 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560a40c5ef50 con 0x560a40d3fc30 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560a40d47ed0 con 0x560a40d44da0 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f8e603740 10 client.0 _ll_register_callbacks cb 0x560a40c9a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f88db0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 0x560a40d49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f895b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d49ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f88db0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 0x560a40d49660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59732/0 (socket says 172.21.15.33:59732) 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f88db0640 1 -- 172.21.15.33:0/3091905779 learned_addr learned my addr 172.21.15.33:0/3091905779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f88db0640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d44da0 0x560a40d50be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f895b1640 1 -- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d44da0 msgr2=0x560a40d50be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f895b1640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d44da0 0x560a40d50be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f895b1640 1 -- 172.21.15.33:0/3091905779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 msgr2=0x560a40d49660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f895b1640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 0x560a40d49660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f895b1640 1 -- 172.21.15.33:0/3091905779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a409b73f0 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.062+0000 7f3f88db0640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 0x560a40d49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f895b1640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d49ba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x560a40d3b820 tx=0x7f3f80004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f52bd6f44dc28b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f6cff9640 1 -- 172.21.15.33:0/3091905779 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80029020 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f80005020 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f6cff9640 1 -- 172.21.15.33:0/3091905779 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f80025470 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f6cff9640 1 -- 172.21.15.33:0/3091905779 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f80034050 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f8e603740 10 client.27422 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a409770f0 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.063+0000 7f3f6cff9640 1 -- 172.21.15.33:0/3091905779 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f80005020 con 0x560a40d3fc30 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f6cff9640 1 -- 172.21.15.33:0/3091905779 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3f80025ce0 con 0x560a40d3fc30 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f8e603740 10 client.27422 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f8e603740 10 client.27422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f8e603740 20 client.27422 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40d552a0 0x560a40d57720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f8e603740 4 client.27422 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a40cb0bf0 con 0x560a40d552a0 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f88db0640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40d552a0 0x560a40d57720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f6cff9640 10 client.27422 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.064+0000 7f3f88db0640 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40d552a0 0x560a40d57720 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22dd4911505e8cd server_cookie=d750ba338ca19abc in_seq=0 out_seq=0 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f6cff9640 1 -- 172.21.15.33:0/3091905779 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a40cb0bf0 con 0x560a40d552a0 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f6cff9640 10 client.27422 handle_command_reply: tid=0 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 client.27422 shutdown 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 msgr2=0x560a40d49ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d49ba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x560a40d3b820 tx=0x7f3f80004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 shutdown_connections 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40d552a0 0x560a40d57720 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d44da0 0x560a40d50be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f6cff9640 0 client.27422 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d49ba0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 --2- 172.21.15.33:0/3091905779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3f240 0x560a40d49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 >> 172.21.15.33:0/3091905779 conn(0x560a40d3a8b0 msgr2=0x560a40d39af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 shutdown_connections 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 1 -- 172.21.15.33:0/3091905779 wait complete. 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.065+0000 7f3f8e603740 20 client.27422 trim_cache size 0 max 16384 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.083+0000 7f3f8e603740 1 Processor -- start 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.083+0000 7f3f8e603740 1 -- start start 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d3d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d50ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d3dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f8e603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841accb0 con 0x560a40d44da0 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f8e603740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841aa580 con 0x560a40d3fc30 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f8e603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841a83d0 con 0x560a40d3f240 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f88db0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d3dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d50ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f88db0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d3dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59748/0 (socket says 172.21.15.33:59748) 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f88db0640 1 -- 172.21.15.33:0/581358905 learned_addr learned my addr 172.21.15.33:0/581358905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f88db0640 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d3d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 -- 172.21.15.33:0/581358905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 msgr2=0x560a40d3d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d3d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 -- 172.21.15.33:0/581358905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 msgr2=0x560a40d3dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d3dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 -- 172.21.15.33:0/581358905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a40d3b620 con 0x560a40d3fc30 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f895b1640 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d50ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3f800195a0 tx=0x7f3f80019450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fe0091fdb73c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f6effd640 1 -- 172.21.15.33:0/581358905 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80000e60 con 0x560a40d3fc30 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f6effd640 1 -- 172.21.15.33:0/581358905 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f80009a70 con 0x560a40d3fc30 2024-04-24T23:24:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.084+0000 7f3f6effd640 1 -- 172.21.15.33:0/581358905 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80029040 con 0x560a40d3fc30 2024-04-24T23:24:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 -- 172.21.15.33:0/581358905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 msgr2=0x560a40d50ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d50ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3f800195a0 tx=0x7f3f80019450 comp rx=0 tx=0).stop 2024-04-24T23:24:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 -- 172.21.15.33:0/581358905 shutdown_connections 2024-04-24T23:24:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d44da0 0x560a40d3dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d3fc30 0x560a40d50ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 --2- 172.21.15.33:0/581358905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d3d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 -- 172.21.15.33:0/581358905 >> 172.21.15.33:0/581358905 conn(0x560a40d3a8b0 msgr2=0x560a40d41090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 -- 172.21.15.33:0/581358905 shutdown_connections 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.085+0000 7f3f8e603740 1 -- 172.21.15.33:0/581358905 wait complete. 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 -- start start 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 Processor -- start 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d4e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 0x560a40d4e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d44da0 0x560a40d4f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841a8a40 con 0x560a40d3fc30 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841a9eb0 con 0x560a40d44da0 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841a9110 con 0x560a40d3f240 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f8e603740 10 client.0 _ll_register_callbacks cb 0x7f3f74009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 0x560a40d4e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 0x560a40d4e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59762/0 (socket says 172.21.15.33:59762) 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 -- 172.21.15.33:0/4177822296 learned_addr learned my addr 172.21.15.33:0/4177822296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f88db0640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d44da0 0x560a40d4f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f88db0640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 -- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 msgr2=0x560a40d4e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d4e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 -- 172.21.15.33:0/4177822296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d44da0 msgr2=0x560a40d4f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d44da0 0x560a40d4f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.086+0000 7f3f895b1640 1 -- 172.21.15.33:0/4177822296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a40d44af0 con 0x560a40d3fc30 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f88db0640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d44da0 0x560a40d4f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f895b1640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 0x560a40d4e540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f800092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0d27e5e339ef83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f6cff9640 1 -- 172.21.15.33:0/4177822296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80029070 con 0x560a40d3fc30 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f6cff9640 1 -- 172.21.15.33:0/4177822296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f80004e40 con 0x560a40d3fc30 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f80009a70 con 0x560a40d3fc30 2024-04-24T23:24:01.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f6cff9640 1 -- 172.21.15.33:0/4177822296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f80046050 con 0x560a40d3fc30 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 10 client.17910 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a40d50c80 con 0x560a40d3fc30 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f6cff9640 1 -- 172.21.15.33:0/4177822296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f80009a70 con 0x560a40d3fc30 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f6cff9640 1 -- 172.21.15.33:0/4177822296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3f80025a70 con 0x560a40d3fc30 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 10 client.17910 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 20 client.17910 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40dbbdc0 0x560a40dbe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 4 client.17910 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.087+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a40caaf40 con 0x560a40dbbdc0 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f88db0640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40dbbdc0 0x560a40dbe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f88db0640 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40dbbdc0 0x560a40dbe1e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b44e1c421712ded server_cookie=31a0d3baa4d6599d in_seq=0 out_seq=0 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f6cff9640 10 client.17910 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f6cff9640 1 -- 172.21.15.33:0/4177822296 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a40caaf40 con 0x560a40dbbdc0 2024-04-24T23:24:01.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f6cff9640 10 client.17910 handle_command_reply: tid=0 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 client.17910 shutdown 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 msgr2=0x560a40d4e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 0x560a40d4e540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f800092f0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 shutdown_connections 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560a40dbbdc0 0x560a40dbe1e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560a40d44da0 0x560a40d4f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f6cff9640 0 client.17910 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a40d3fc30 0x560a40d4e540 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 --2- 172.21.15.33:0/4177822296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a40d3f240 0x560a40d4e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 >> 172.21.15.33:0/4177822296 conn(0x560a40d3a8b0 msgr2=0x560a40d39260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 shutdown_connections 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.088+0000 7f3f8e603740 1 -- 172.21.15.33:0/4177822296 wait complete. 2024-04-24T23:24:01.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8e603740 20 client.17910 trim_cache size 0 max 16384 2024-04-24T23:24:01.097 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:24:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:24:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:24:01.097 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f700a2b90 msgr2=0x7f3f700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f700a2b90 0x7f3f700a5050 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0edc10 tx=0x7f3f7c0877e0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 msgr2=0x7f3f8419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f8419f030 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c031ae0 tx=0x7f3f7c031ee0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 shutdown_connections 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3f700a2b90 0x7f3f700a5050 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f84102230 0x7f3f8419f030 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84101870 0x7f3f84197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 --2- 172.21.15.33:0/666585818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f84100680 0x7f3f84197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 >> 172.21.15.33:0/666585818 conn(0x7f3f840fbbf0 msgr2=0x7f3f84104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 shutdown_connections 2024-04-24T23:24:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.089+0000 7f3f8b815640 1 -- 172.21.15.33:0/666585818 wait complete. 2024-04-24T23:24:01.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:24:01.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.204+0000 7f71f89e2640 1 -- 172.21.15.33:0/3207402489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4102200 msgr2=0x7f71f41045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.204+0000 7f71f89e2640 1 --2- 172.21.15.33:0/3207402489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4102200 0x7f71f41045f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009900 tx=0x7f71e402a410 comp rx=0 tx=0).stop 2024-04-24T23:24:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.204+0000 7f71f89e2640 1 -- 172.21.15.33:0/3207402489 shutdown_connections 2024-04-24T23:24:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 --2- 172.21.15.33:0/3207402489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 0x7f71f41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 --2- 172.21.15.33:0/3207402489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f4104b30 0x7f71f4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 --2- 172.21.15.33:0/3207402489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4102200 0x7f71f41045f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 -- 172.21.15.33:0/3207402489 >> 172.21.15.33:0/3207402489 conn(0x7f71f40fbbf0 msgr2=0x7f71f40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 -- 172.21.15.33:0/3207402489 shutdown_connections 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 -- 172.21.15.33:0/3207402489 wait complete. 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 Processor -- start 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 -- start start 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f4102200 0x7f71f4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 0x7f71f4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.205+0000 7f71f89e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 0x7f71f419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f89e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71f410d430 con 0x7f71f4104b30 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f89e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71f410d2b0 con 0x7f71f4107460 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f89e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71f410d5b0 con 0x7f71f4102200 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 0x7f71f4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 0x7f71f4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59768/0 (socket says 172.21.15.33:59768) 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f1d9b640 1 -- 172.21.15.33:0/1666268403 learned_addr learned my addr 172.21.15.33:0/1666268403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 0x7f71f419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f259c640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f4102200 0x7f71f4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 -- 172.21.15.33:0/1666268403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f4102200 msgr2=0x7f71f4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f4102200 0x7f71f4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 -- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 msgr2=0x7f71f4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 0x7f71f4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 -- 172.21.15.33:0/1666268403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f419f730 con 0x7f71f4107460 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f1d9b640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 0x7f71f4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f2d9d640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 0x7f71f419f0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009fd0 tx=0x7f71ec00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00ce20 con 0x7f71f4107460 2024-04-24T23:24:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f419fa20 con 0x7f71f4107460 2024-04-24T23:24:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ec005020 con 0x7f71f4107460 2024-04-24T23:24:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec010700 con 0x7f71f4107460 2024-04-24T23:24:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.206+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f419ff70 con 0x7f71f4107460 2024-04-24T23:24:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.207+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f40fd6e0 con 0x7f71f4107460 2024-04-24T23:24:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.208+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f71ec005540 con 0x7f71f4107460 2024-04-24T23:24:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.209+0000 7f71e37fe640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f71dc0a2b90 0x7f71dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.209+0000 7f71f259c640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f71dc0a2b90 0x7f71dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.209+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f71ec0cfa90 con 0x7f71f4107460 2024-04-24T23:24:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.209+0000 7f71f259c640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f71dc0a2b90 0x7f71dc0a5050 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71e40314e0 tx=0x7f71e4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.214+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f71ec09c600 con 0x7f71f4107460 2024-04-24T23:24:01.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:01 smithi038 ceph-mon[32450]: pgmap v711: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-04-24T23:24:01.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:01 smithi117 ceph-mon[32529]: pgmap v711: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-04-24T23:24:01.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.445+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71f402d170 con 0x7f71f4107460 2024-04-24T23:24:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.446+0000 7f71e37fe640 1 -- 172.21.15.33:0/1666268403 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7f71ec09bf00 con 0x7f71f4107460 2024-04-24T23:24:01.448 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:24:01.448 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:24:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:24:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.448+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f71dc0a2b90 msgr2=0x7f71dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.448+0000 7f71f89e2640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f71dc0a2b90 0x7f71dc0a5050 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71e40314e0 tx=0x7f71e4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.448+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 msgr2=0x7f71f419f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.448+0000 7f71f89e2640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 0x7f71f419f0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009fd0 tx=0x7f71ec00ef10 comp rx=0 tx=0).stop 2024-04-24T23:24:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 shutdown_connections 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f71dc0a2b90 0x7f71dc0a5050 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71f4107460 0x7f71f419f0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4104b30 0x7f71f4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 --2- 172.21.15.33:0/1666268403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f4102200 0x7f71f4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 >> 172.21.15.33:0/1666268403 conn(0x7f71f40fbbf0 msgr2=0x7f71f4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 shutdown_connections 2024-04-24T23:24:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.449+0000 7f71f89e2640 1 -- 172.21.15.33:0/1666268403 wait complete. 2024-04-24T23:24:01.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:01 smithi033 ceph-mon[29201]: pgmap v711: 129 pgs: 129 active+clean; 276 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-04-24T23:24:01.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:24:01.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1407382 12.41 MB/sec execute 486 sec latency 25.556 ms 2024-04-24T23:24:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 -- 172.21.15.33:0/1400982245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0101870 msgr2=0x7fcef0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 --2- 172.21.15.33:0/1400982245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0101870 0x7fcef0101cf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fced8009900 tx=0x7fced802a410 comp rx=0 tx=0).stop 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 -- 172.21.15.33:0/1400982245 shutdown_connections 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 --2- 172.21.15.33:0/1400982245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef0102230 0x7fcef0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 --2- 172.21.15.33:0/1400982245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0101870 0x7fcef0101cf0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 --2- 172.21.15.33:0/1400982245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef0100680 0x7fcef0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 -- 172.21.15.33:0/1400982245 >> 172.21.15.33:0/1400982245 conn(0x7fcef00fbbf0 msgr2=0x7fcef00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 -- 172.21.15.33:0/1400982245 shutdown_connections 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.566+0000 7fcef529a640 1 -- 172.21.15.33:0/1400982245 wait complete. 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 Processor -- start 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 -- start start 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 0x7fcef0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef0101870 0x7fcef019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef0102230 0x7fcef019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcef010c350 con 0x7fcef0100680 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcef010c1d0 con 0x7fcef0102230 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcef529a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcef010c4d0 con 0x7fcef0101870 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 0x7fcef0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 0x7fcef0069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59774/0 (socket says 172.21.15.33:59774) 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 -- 172.21.15.33:0/95072254 learned_addr learned my addr 172.21.15.33:0/95072254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fcee7fff640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef0101870 0x7fcef019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceef7fe640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef0102230 0x7fcef019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 -- 172.21.15.33:0/95072254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef0101870 msgr2=0x7fcef019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef0101870 0x7fcef019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 -- 172.21.15.33:0/95072254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef0102230 msgr2=0x7fcef019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef0102230 0x7fcef019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.567+0000 7fceeeffd640 1 -- 172.21.15.33:0/95072254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef019f6c0 con 0x7fcef0100680 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.568+0000 7fceeeffd640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 0x7fcef0069460 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcee000e9c0 tx=0x7fcee000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.568+0000 7fceecff9640 1 -- 172.21.15.33:0/95072254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee000cd80 con 0x7fcef0100680 2024-04-24T23:24:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.568+0000 7fceecff9640 1 -- 172.21.15.33:0/95072254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee00035f0 con 0x7fcef0100680 2024-04-24T23:24:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.568+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef019f9b0 con 0x7fcef0100680 2024-04-24T23:24:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.568+0000 7fceecff9640 1 -- 172.21.15.33:0/95072254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0010690 con 0x7fcef0100680 2024-04-24T23:24:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.568+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef019fed0 con 0x7fcef0100680 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.570+0000 7fceecff9640 1 -- 172.21.15.33:0/95072254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcee00037a0 con 0x7fcef0100680 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.570+0000 7fcef7fa6740 1 Processor -- start 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.570+0000 7fcef7fa6740 1 -- start start 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84df230 0x5582b84df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84e4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 0x5582b84e7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582b8443670 con 0x5582b84df230 2024-04-24T23:24:01.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5582b83eb970 con 0x5582b84e4d90 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5582b843f630 con 0x5582b84dfc20 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84df230 0x5582b84df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceecff9640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcedc0a2b90 0x7fcedc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 0x5582b84e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 0x5582b84e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:45462/0 (socket says 172.21.15.33:45462) 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84df230 0x5582b84df650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59780/0 (socket says 172.21.15.33:59780) 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 -- 172.21.15.33:0/613476596 learned_addr learned my addr 172.21.15.33:0/613476596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceeeffd640 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceef7fe640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcedc0a2b90 0x7fcedc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceecff9640 1 -- 172.21.15.33:0/95072254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcee00cf810 con 0x7fcef0100680 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 -- 172.21.15.33:0/613476596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 msgr2=0x5582b84e4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84e4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 -- 172.21.15.33:0/613476596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84df230 msgr2=0x5582b84df650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84df230 0x5582b84df650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 -- 172.21.15.33:0/613476596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b840f400 con 0x5582b84e4d90 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceeeffd640 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84e4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fceef7fe640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcedc0a2b90 0x7fcedc0a5050 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5582b84e0480 tx=0x7fcee8004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.571+0000 7fcee7fff640 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 0x5582b84e7220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302ce585e66e9f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcee57fa640 1 -- 172.21.15.33:0/613476596 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8035040 con 0x5582b84e4d90 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcee57fa640 1 -- 172.21.15.33:0/613476596 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced8005520 con 0x5582b84e4d90 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcee57fa640 1 -- 172.21.15.33:0/613476596 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80316b0 con 0x5582b84e4d90 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 -- 172.21.15.33:0/613476596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 msgr2=0x5582b84e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 0x5582b84e7220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004040 comp rx=0 tx=0).stop 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 -- 172.21.15.33:0/613476596 shutdown_connections 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84e4d90 0x5582b84e7220 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84e4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/613476596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84df230 0x5582b84df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 -- 172.21.15.33:0/613476596 >> 172.21.15.33:0/613476596 conn(0x5582b84da8a0 msgr2=0x5582b84daca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.572+0000 7fcef7fa6740 1 -- 172.21.15.33:0/613476596 shutdown_connections 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 -- 172.21.15.33:0/613476596 wait complete. 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 -- start start 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 Processor -- start 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582b84e7ce0 con 0x5582b84dfc20 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5582b83ff050 con 0x5582b84df230 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5582b84e7e60 con 0x5582b84e4d90 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcef7fa6740 10 client.0 _ll_register_callbacks cb 0x5582b8156be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59786/0 (socket says 172.21.15.33:59786) 2024-04-24T23:24:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fcee7fff640 1 -- 172.21.15.33:0/2484258581 learned_addr learned my addr 172.21.15.33:0/2484258581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.573+0000 7fceef7fe640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcee7fff640 1 -- 172.21.15.33:0/2484258581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 msgr2=0x5582b84f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcee7fff640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcee7fff640 1 -- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 msgr2=0x5582b84e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcee7fff640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcee7fff640 1 -- 172.21.15.33:0/2484258581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b83d0dc0 con 0x5582b84dfc20 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fceef7fe640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fceeeffd640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcee7fff640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84e97d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fced8006010 tx=0x7fced8004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991dc25b7d11698d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fced67fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fced67fc640 1 -- 172.21.15.33:0/2484258581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8053060 con 0x5582b84dfc20 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fced67fc640 1 -- 172.21.15.33:0/2484258581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fced8004980 con 0x5582b84dfc20 2024-04-24T23:24:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced8005520 con 0x5582b84dfc20 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fced67fc640 1 -- 172.21.15.33:0/2484258581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fced805a050 con 0x5582b84dfc20 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcef7fa6740 10 client.17928 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582b81573f0 con 0x5582b84dfc20 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.574+0000 7fced67fc640 1 -- 172.21.15.33:0/2484258581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced8005520 con 0x5582b84dfc20 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fced67fc640 1 -- 172.21.15.33:0/2484258581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fced8031c50 con 0x5582b84dfc20 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fcef7fa6740 10 client.17928 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fcef7fa6740 10 client.17928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fcef7fa6740 20 client.17928 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b84f4c70 0x5582b84f7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fcef7fa6740 4 client.17928 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b8450bb0 con 0x5582b84f4c70 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fceeeffd640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b84f4c70 0x5582b84f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fceeeffd640 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b84f4c70 0x5582b84f7060 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=538ca94fa14dfbc7 server_cookie=3a2a21f45f8ead23 in_seq=0 out_seq=0 2024-04-24T23:24:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.575+0000 7fced67fc640 10 client.17928 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fced67fc640 1 -- 172.21.15.33:0/2484258581 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582b8450bb0 con 0x5582b84f4c70 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fced67fc640 10 client.17928 handle_command_reply: tid=0 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 client.17928 shutdown 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 msgr2=0x5582b84e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84e97d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fced8006010 tx=0x7fced8004520 comp rx=0 tx=0).stop 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 shutdown_connections 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b84f4c70 0x5582b84f7060 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fced67fc640 0 client.17928 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84e97d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2484258581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 >> 172.21.15.33:0/2484258581 conn(0x5582b84da8a0 msgr2=0x5582b84d9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 shutdown_connections 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.576+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2484258581 wait complete. 2024-04-24T23:24:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.577+0000 7fcef7fa6740 20 client.17928 trim_cache size 0 max 16384 2024-04-24T23:24:01.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 Processor -- start 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 -- start start 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 0x5582b84dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01acbc0 con 0x5582b84e4d90 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01aa490 con 0x5582b84df230 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01a82e0 con 0x5582b84dfc20 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 0x5582b84dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 0x5582b84dd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59788/0 (socket says 172.21.15.33:59788) 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fceef7fe640 1 -- 172.21.15.33:0/3916234446 learned_addr learned my addr 172.21.15.33:0/3916234446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.590+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84d99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:45474/0 (socket says 172.21.15.33:45474) 2024-04-24T23:24:01.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceef7fe640 1 -- 172.21.15.33:0/3916234446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 msgr2=0x5582b84f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceef7fe640 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceef7fe640 1 -- 172.21.15.33:0/3916234446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 msgr2=0x5582b84d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceef7fe640 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceef7fe640 1 -- 172.21.15.33:0/3916234446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b82217d0 con 0x5582b84e4d90 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fcee7fff640 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceeeffd640 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fceef7fe640 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 0x5582b84dd580 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009ee0 tx=0x7fcee8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e2a015f96debbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fcee4ff9640 1 -- 172.21.15.33:0/3916234446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8013df0 con 0x5582b84e4d90 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fcee4ff9640 1 -- 172.21.15.33:0/3916234446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee800ecb0 con 0x5582b84e4d90 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.591+0000 7fcee4ff9640 1 -- 172.21.15.33:0/3916234446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8010880 con 0x5582b84e4d90 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- 172.21.15.33:0/3916234446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 msgr2=0x5582b84dd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 0x5582b84dd580 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009ee0 tx=0x7fcee8005bd0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- 172.21.15.33:0/3916234446 shutdown_connections 2024-04-24T23:24:01.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84e4d90 0x5582b84dd580 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84dfc20 0x5582b84f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/3916234446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- 172.21.15.33:0/3916234446 >> 172.21.15.33:0/3916234446 conn(0x5582b84da8a0 msgr2=0x5582b84d92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- 172.21.15.33:0/3916234446 shutdown_connections 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- 172.21.15.33:0/3916234446 wait complete. 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- start start 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 Processor -- start 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01a8950 con 0x5582b84dfc20 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01a9dc0 con 0x5582b84df230 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01a9020 con 0x5582b84e4d90 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcef7fa6740 10 client.0 _ll_register_callbacks cb 0x7fcee8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.592+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84ee0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59794/0 (socket says 172.21.15.33:59794) 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fceeeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 -- 172.21.15.33:0/2301845206 learned_addr learned my addr 172.21.15.33:0/2301845206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 -- 172.21.15.33:0/2301845206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 msgr2=0x5582b84ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 -- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 msgr2=0x5582b84edb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84edb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 -- 172.21.15.33:0/2301845206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b84ef640 con 0x5582b84dfc20 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcee7fff640 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84ee0b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fced802aa00 tx=0x7fced8003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba935769548cb35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fced67fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fced67fc640 1 -- 172.21.15.33:0/2301845206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8035050 con 0x5582b84dfc20 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fced67fc640 1 -- 172.21.15.33:0/2301845206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fced8003e10 con 0x5582b84dfc20 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee800ecb0 con 0x5582b84dfc20 2024-04-24T23:24:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fced67fc640 1 -- 172.21.15.33:0/2301845206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fced8066050 con 0x5582b84dfc20 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcef7fa6740 10 client.17934 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582b84f09b0 con 0x5582b84dfc20 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.593+0000 7fced67fc640 1 -- 172.21.15.33:0/2301845206 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee800ecb0 con 0x5582b84dfc20 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fced67fc640 1 -- 172.21.15.33:0/2301845206 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fced804e500 con 0x5582b84dfc20 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fcef7fa6740 10 client.17934 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fcef7fa6740 10 client.17934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fcef7fa6740 20 client.17934 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b855b6e0 0x5582b855db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fcef7fa6740 4 client.17934 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582b844af00 con 0x5582b855b6e0 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fceeeffd640 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b855b6e0 0x5582b855db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fced67fc640 10 client.17934 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.594+0000 7fceeeffd640 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b855b6e0 0x5582b855db00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbdd10112d230b93 server_cookie=5ef94b56859eb42b in_seq=0 out_seq=0 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fced67fc640 1 -- 172.21.15.33:0/2301845206 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582b844af00 con 0x5582b855b6e0 2024-04-24T23:24:01.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fced67fc640 10 client.17934 handle_command_reply: tid=0 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 client.17934 shutdown 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 msgr2=0x5582b84ee0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84ee0b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fced802aa00 tx=0x7fced8003a60 comp rx=0 tx=0).stop 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 shutdown_connections 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5582b855b6e0 0x5582b855db00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b84e4d90 0x5582b84ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fced67fc640 0 client.17934 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582b84dfc20 0x5582b84ee0b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 --2- 172.21.15.33:0/2301845206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582b84df230 0x5582b84edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 >> 172.21.15.33:0/2301845206 conn(0x5582b84da8a0 msgr2=0x5582b84d9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 shutdown_connections 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 1 -- 172.21.15.33:0/2301845206 wait complete. 2024-04-24T23:24:01.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.595+0000 7fcef7fa6740 20 client.17934 trim_cache size 0 max 16384 2024-04-24T23:24:01.603 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:24:01.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcedc0a2b90 msgr2=0x7fcedc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcedc0a2b90 0x7fcedc0a5050 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5582b84e0480 tx=0x7fcee8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 msgr2=0x7fcef0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 0x7fcef0069460 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcee000e9c0 tx=0x7fcee000ee80 comp rx=0 tx=0).stop 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 shutdown_connections 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcedc0a2b90 0x7fcedc0a5050 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef0102230 0x7fcef019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef0101870 0x7fcef019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 --2- 172.21.15.33:0/95072254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef0100680 0x7fcef0069460 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 >> 172.21.15.33:0/95072254 conn(0x7fcef00fbbf0 msgr2=0x7fcef0104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 shutdown_connections 2024-04-24T23:24:01.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.596+0000 7fcef529a640 1 -- 172.21.15.33:0/95072254 wait complete. 2024-04-24T23:24:01.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:24:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 -- 172.21.15.33:0/3788284753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 msgr2=0x7f06a8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 --2- 172.21.15.33:0/3788284753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8100aa0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f069c009900 tx=0x7f069c02a410 comp rx=0 tx=0).stop 2024-04-24T23:24:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 -- 172.21.15.33:0/3788284753 shutdown_connections 2024-04-24T23:24:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 --2- 172.21.15.33:0/3788284753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102230 0x7f06a8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 --2- 172.21.15.33:0/3788284753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8101870 0x7f06a8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 --2- 172.21.15.33:0/3788284753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8100aa0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 -- 172.21.15.33:0/3788284753 >> 172.21.15.33:0/3788284753 conn(0x7f06a80fbbf0 msgr2=0x7f06a80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 -- 172.21.15.33:0/3788284753 shutdown_connections 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 -- 172.21.15.33:0/3788284753 wait complete. 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 Processor -- start 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.755+0000 7f06ad761640 1 -- start start 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8101870 0x7f06a8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102230 0x7f06a819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810c300 con 0x7f06a8100680 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810c180 con 0x7f06a8101870 2024-04-24T23:24:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810c480 con 0x7f06a8102230 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59806/0 (socket says 172.21.15.33:59806) 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 -- 172.21.15.33:0/1209807487 learned_addr learned my addr 172.21.15.33:0/1209807487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a77fe640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102230 0x7f06a819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a67fc640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8101870 0x7f06a8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 -- 172.21.15.33:0/1209807487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102230 msgr2=0x7f06a819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102230 0x7f06a819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 -- 172.21.15.33:0/1209807487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8101870 msgr2=0x7f06a8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8101870 0x7f06a8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 -- 172.21.15.33:0/1209807487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a819f6f0 con 0x7f06a8100680 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8197b20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f069c031610 tx=0x7f069c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f068ffff640 1 -- 172.21.15.33:0/1209807487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c03f030 con 0x7f06a8100680 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f068ffff640 1 -- 172.21.15.33:0/1209807487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069c003ad0 con 0x7f06a8100680 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a819f980 con 0x7f06a8100680 2024-04-24T23:24:01.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f068ffff640 1 -- 172.21.15.33:0/1209807487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c035070 con 0x7f06a8100680 2024-04-24T23:24:01.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.756+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a819fe70 con 0x7f06a8100680 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f068ffff640 1 -- 172.21.15.33:0/1209807487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f069c042030 con 0x7f06a8100680 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 Processor -- start 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 -- start start 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 0x564875445a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x56487544ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56487544b1a0 0x56487544d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5648753a5670 con 0x564875445640 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56487534d970 con 0x56487544b1a0 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06b046d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5648753a1630 con 0x564875446030 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 0x564875445a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 0x564875445a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59818/0 (socket says 172.21.15.33:59818) 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06a77fe640 1 -- 172.21.15.33:0/2125374 learned_addr learned my addr 172.21.15.33:0/2125374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06a67fc640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56487544b1a0 0x56487544d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.759+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x56487544ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f068ffff640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06900a2b90 0x7f06900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 -- 172.21.15.33:0/2125374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 msgr2=0x56487544ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x56487544ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 -- 172.21.15.33:0/2125374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56487544b1a0 msgr2=0x56487544d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56487544b1a0 0x56487544d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 -- 172.21.15.33:0/2125374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564875371400 con 0x564875445640 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a67fc640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56487544b1a0 0x56487544d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x56487544ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f068ffff640 1 -- 172.21.15.33:0/1209807487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f069c0ef960 con 0x7f06a8100680 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06900a2b90 0x7f06900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 0x564875445a60 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f06a0011590 tx=0x7f06a0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18989c4d341931c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f068d7fa640 1 -- 172.21.15.33:0/2125374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0017050 con 0x564875445640 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f06a77fe640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06900a2b90 0x7f06900a5050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f06a0022000 tx=0x7f06a0023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f068d7fa640 1 -- 172.21.15.33:0/2125374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a000fd20 con 0x564875445640 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.760+0000 7f068d7fa640 1 -- 172.21.15.33:0/2125374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a001bbc0 con 0x564875445640 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 -- 172.21.15.33:0/2125374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 msgr2=0x564875445a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 0x564875445a60 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f06a0011590 tx=0x7f06a0011a50 comp rx=0 tx=0).stop 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 -- 172.21.15.33:0/2125374 shutdown_connections 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56487544b1a0 0x56487544d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x56487544ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 --2- 172.21.15.33:0/2125374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875445640 0x564875445a60 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 -- 172.21.15.33:0/2125374 >> 172.21.15.33:0/2125374 conn(0x56487543c8a0 msgr2=0x56487543cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 -- 172.21.15.33:0/2125374 shutdown_connections 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 -- 172.21.15.33:0/2125374 wait complete. 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 -- start start 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.761+0000 7f06b046d740 1 Processor -- start 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x56487544f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 0x56487544fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56487544b1a0 0x564875456d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56487544e150 con 0x564875446030 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564875361050 con 0x564875445640 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56487544e2d0 con 0x56487544b1a0 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 10 client.0 _ll_register_callbacks cb 0x5648750b8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 0x56487544fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x56487544f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 0x56487544fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59828/0 (socket says 172.21.15.33:59828) 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 -- 172.21.15.33:0/2947255986 learned_addr learned my addr 172.21.15.33:0/2947255986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a77fe640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56487544b1a0 0x564875456d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 -- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56487544b1a0 msgr2=0x564875456d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56487544b1a0 0x564875456d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 -- 172.21.15.33:0/2947255986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 msgr2=0x56487544f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x56487544f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 -- 172.21.15.33:0/2947255986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564875332dc0 con 0x564875446030 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x56487544f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06a67fc640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 0x56487544fcf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0698012700 tx=0x7f0698012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9adccfd7c26bb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f068aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f068aff5640 1 -- 172.21.15.33:0/2947255986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698004030 con 0x564875446030 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f068aff5640 1 -- 172.21.15.33:0/2947255986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06980041d0 con 0x564875446030 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.762+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06a000fd20 con 0x564875446030 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f068aff5640 1 -- 172.21.15.33:0/2947255986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0698025050 con 0x564875446030 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 10 client.17946 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648750b93f0 con 0x564875446030 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f068aff5640 1 -- 172.21.15.33:0/2947255986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a000fd20 con 0x564875446030 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f068aff5640 1 -- 172.21.15.33:0/2947255986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0698003a10 con 0x564875446030 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 10 client.17946 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 10 client.17946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 20 client.17946 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56487545b5f0 0x56487545da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 4 client.17946 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648753b2bb0 con 0x56487545b5f0 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.763+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56487545b5f0 0x56487545da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56487545b5f0 0x56487545da90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24dae27b292b11a8 server_cookie=93e2e2155948bdf4 in_seq=0 out_seq=0 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f068aff5640 10 client.17946 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f068aff5640 1 -- 172.21.15.33:0/2947255986 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648753b2bb0 con 0x56487545b5f0 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f068aff5640 10 client.17946 handle_command_reply: tid=0 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 client.17946 shutdown 2024-04-24T23:24:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 msgr2=0x56487544fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 0x56487544fcf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0698012700 tx=0x7f0698012bc0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 shutdown_connections 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56487545b5f0 0x56487545da90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56487544b1a0 0x564875456d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564875446030 0x56487544fcf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 --2- 172.21.15.33:0/2947255986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x56487544f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f068aff5640 0 client.17946 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.764+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 >> 172.21.15.33:0/2947255986 conn(0x56487543c8a0 msgr2=0x56487543bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.765+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 shutdown_connections 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.765+0000 7f06b046d740 1 -- 172.21.15.33:0/2947255986 wait complete. 2024-04-24T23:24:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.765+0000 7f06b046d740 20 client.17946 trim_cache size 0 max 16384 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 Processor -- start 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 -- start start 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 0x56487543b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875446030 0x564875456cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x56487543f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81aaa80 con 0x56487544b1a0 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81a8350 con 0x564875446030 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06b046d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81a61a0 con 0x564875445640 2024-04-24T23:24:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 0x56487543b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x56487543f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 0x56487543b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47580/0 (socket says 172.21.15.33:47580) 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 -- 172.21.15.33:0/3968394661 learned_addr learned my addr 172.21.15.33:0/3968394661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a67fc640 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875446030 0x564875456cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 -- 172.21.15.33:0/3968394661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875446030 msgr2=0x564875456cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875446030 0x564875456cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 -- 172.21.15.33:0/3968394661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 msgr2=0x56487543f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x56487543f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a6ffd640 1 -- 172.21.15.33:0/3968394661 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648751837d0 con 0x564875445640 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.779+0000 7f06a77fe640 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x56487543f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.780+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 0x56487543b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ed440 tx=0x7f069c0dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d863ab558deb29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.780+0000 7f068cff9640 1 -- 172.21.15.33:0/3968394661 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c049040 con 0x564875445640 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.780+0000 7f068cff9640 1 -- 172.21.15.33:0/3968394661 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069c0f6040 con 0x564875445640 2024-04-24T23:24:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.780+0000 7f068cff9640 1 -- 172.21.15.33:0/3968394661 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c03b460 con 0x564875445640 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- 172.21.15.33:0/3968394661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 msgr2=0x56487543b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 0x56487543b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ed440 tx=0x7f069c0dc040 comp rx=0 tx=0).stop 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- 172.21.15.33:0/3968394661 shutdown_connections 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x56487543f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875446030 0x564875456cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- 172.21.15.33:0/3968394661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875445640 0x56487543b9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- 172.21.15.33:0/3968394661 >> 172.21.15.33:0/3968394661 conn(0x56487543c8a0 msgr2=0x56487543b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- 172.21.15.33:0/3968394661 shutdown_connections 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- 172.21.15.33:0/3968394661 wait complete. 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- start start 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 Processor -- start 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x564875453d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x564875454280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x564875455290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81a6810 con 0x56487544b1a0 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81a7c80 con 0x564875445640 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81a6ee0 con 0x564875446030 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06b046d740 10 client.0 _ll_register_callbacks cb 0x7f0698009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.781+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x564875455290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x564875453d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x564875454280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x564875455290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59838/0 (socket says 172.21.15.33:59838) 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x564875453d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:45542/0 (socket says 172.21.15.33:45542) 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 -- 172.21.15.33:0/93266224 learned_addr learned my addr 172.21.15.33:0/93266224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 -- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 msgr2=0x564875454280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x564875454280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 -- 172.21.15.33:0/93266224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 msgr2=0x564875453d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x564875453d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 -- 172.21.15.33:0/93266224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564875455880 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x564875453d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06a77fe640 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x564875455290 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x564875446890 tx=0x7f06a000fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97627e372911495f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f068aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f068aff5640 1 -- 172.21.15.33:0/93266224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004df0 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f068aff5640 1 -- 172.21.15.33:0/93266224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06a0005050 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069c0f6040 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f068aff5640 1 -- 172.21.15.33:0/93266224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06a0017020 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06b046d740 10 client.17952 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.782+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564875456bf0 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f068aff5640 1 -- 172.21.15.33:0/93266224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069c0f6040 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f068aff5640 1 -- 172.21.15.33:0/93266224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f06a001a070 con 0x56487544b1a0 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06b046d740 10 client.17952 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06b046d740 10 client.17952 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06b046d740 20 client.17952 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06b046d740 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5648754c1f50 0x5648754c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06b046d740 4 client.17952 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648753acf00 con 0x5648754c1f50 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5648754c1f50 0x5648754c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f06a6ffd640 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5648754c1f50 0x5648754c4340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48f6e55beebdb209 server_cookie=3ad7d2d5f3473384 in_seq=0 out_seq=0 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.783+0000 7f068aff5640 10 client.17952 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f068aff5640 1 -- 172.21.15.33:0/93266224 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648753acf00 con 0x5648754c1f50 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f068aff5640 10 client.17952 handle_command_reply: tid=0 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 client.17952 shutdown 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 msgr2=0x564875455290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x564875455290 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x564875446890 tx=0x7f06a000fe70 comp rx=0 tx=0).stop 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 shutdown_connections 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5648754c1f50 0x5648754c4340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56487544b1a0 0x564875455290 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f068aff5640 0 client.17952 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:01.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564875446030 0x564875454280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 --2- 172.21.15.33:0/93266224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564875445640 0x564875453d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 >> 172.21.15.33:0/93266224 conn(0x56487543c8a0 msgr2=0x56487543bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 shutdown_connections 2024-04-24T23:24:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 1 -- 172.21.15.33:0/93266224 wait complete. 2024-04-24T23:24:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06b046d740 20 client.17952 trim_cache size 0 max 16384 2024-04-24T23:24:01.791 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06900a2b90 msgr2=0x7f06900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06ad761640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06900a2b90 0x7f06900a5050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f06a0022000 tx=0x7f06a0023000 comp rx=0 tx=0).stop 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 msgr2=0x7f06a8197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.784+0000 7f06ad761640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8197b20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f069c031610 tx=0x7f069c004270 comp rx=0 tx=0).stop 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 shutdown_connections 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f06900a2b90 0x7f06900a5050 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102230 0x7f06a819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8101870 0x7f06a8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 --2- 172.21.15.33:0/1209807487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06a8100680 0x7f06a8197b20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 >> 172.21.15.33:0/1209807487 conn(0x7f06a80fbbf0 msgr2=0x7f06a80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 shutdown_connections 2024-04-24T23:24:01.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.785+0000 7f06ad761640 1 -- 172.21.15.33:0/1209807487 wait complete. 2024-04-24T23:24:01.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 -- 172.21.15.33:0/4157331498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00301006a0 msgr2=0x7f0030100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 --2- 172.21.15.33:0/4157331498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00301006a0 0x7f0030100ac0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002002a410 comp rx=0 tx=0).stop 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 -- 172.21.15.33:0/4157331498 shutdown_connections 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 --2- 172.21.15.33:0/4157331498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0030102250 0x7f0030108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 --2- 172.21.15.33:0/4157331498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0030101890 0x7f0030101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 --2- 172.21.15.33:0/4157331498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00301006a0 0x7f0030100ac0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 -- 172.21.15.33:0/4157331498 >> 172.21.15.33:0/4157331498 conn(0x7f00300fbc30 msgr2=0x7f00300fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 -- 172.21.15.33:0/4157331498 shutdown_connections 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 -- 172.21.15.33:0/4157331498 wait complete. 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 Processor -- start 2024-04-24T23:24:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.911+0000 7f00366bd640 1 -- start start 2024-04-24T23:24:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f00366bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00301006a0 0x7f0030104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f00366bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0030101890 0x7f0030102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f00366bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 0x7f00301031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f00366bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f003010c2f0 con 0x7f0030102250 2024-04-24T23:24:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f00366bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f003010c170 con 0x7f00301006a0 2024-04-24T23:24:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f00366bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f003010c470 con 0x7f0030101890 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 0x7f00301031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 0x7f00301031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59858/0 (socket says 172.21.15.33:59858) 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f002ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00301006a0 0x7f0030104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 -- 172.21.15.33:0/2868212451 learned_addr learned my addr 172.21.15.33:0/2868212451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f002f7fe640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0030101890 0x7f0030102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 -- 172.21.15.33:0/2868212451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0030101890 msgr2=0x7f0030102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0030101890 0x7f0030102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 -- 172.21.15.33:0/2868212451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00301006a0 msgr2=0x7f0030104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00301006a0 0x7f0030104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 -- 172.21.15.33:0/2868212451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00301039b0 con 0x7f0030102250 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f002ffff640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00301006a0 0x7f0030104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.912+0000 7f0034c5a640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 0x7f00301031b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0028009fd0 tx=0x7f002800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.913+0000 7f002d7fa640 1 -- 172.21.15.33:0/2868212451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028017070 con 0x7f0030102250 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.913+0000 7f002d7fa640 1 -- 172.21.15.33:0/2868212451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0028003e50 con 0x7f0030102250 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.913+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301a1c60 con 0x7f0030102250 2024-04-24T23:24:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.913+0000 7f002d7fa640 1 -- 172.21.15.33:0/2868212451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280106c0 con 0x7f0030102250 2024-04-24T23:24:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.913+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301a2130 con 0x7f0030102250 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 Processor -- start 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002d7fa640 1 -- 172.21.15.33:0/2868212451 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0028003970 con 0x7f0030102250 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 -- start start 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a57720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c629b6670 con 0x563c62a52230 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563c6295e970 con 0x563c62a52c20 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f00393d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c629b2630 con 0x563c62a57d90 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a52650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59866/0 (socket says 172.21.15.33:59866) 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 -- 172.21.15.33:0/975545184 learned_addr learned my addr 172.21.15.33:0/975545184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002ffff640 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a57720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 -- 172.21.15.33:0/975545184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 msgr2=0x563c62a5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 -- 172.21.15.33:0/975545184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 msgr2=0x563c62a57720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a57720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002f7fe640 1 -- 172.21.15.33:0/975545184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c62982400 con 0x563c62a52230 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002d7fa640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f00180a2b90 0x7f00180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.916+0000 7f002ffff640 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a57720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.917+0000 7f002d7fa640 1 -- 172.21.15.33:0/2868212451 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00280cf090 con 0x7f0030102250 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.917+0000 7f002f7fe640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f00180a2b90 0x7f00180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.917+0000 7f002f7fe640 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a52650 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0024013630 tx=0x7f0024005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8792c44fcae3c416 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.917+0000 7f00167fc640 1 -- 172.21.15.33:0/975545184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002400f780 con 0x563c62a52230 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.917+0000 7f00167fc640 1 -- 172.21.15.33:0/975545184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024010e30 con 0x563c62a52230 2024-04-24T23:24:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.917+0000 7f00167fc640 1 -- 172.21.15.33:0/975545184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024023a20 con 0x563c62a52230 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 -- 172.21.15.33:0/975545184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 msgr2=0x563c62a52650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a52650 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0024013630 tx=0x7f0024005f90 comp rx=0 tx=0).stop 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f002f7fe640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f00180a2b90 0x7f00180a5050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0024023000 tx=0x7f0024024000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 -- 172.21.15.33:0/975545184 shutdown_connections 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a57720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 --2- 172.21.15.33:0/975545184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a52650 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 -- 172.21.15.33:0/975545184 >> 172.21.15.33:0/975545184 conn(0x563c62a4d8a0 msgr2=0x563c62a4dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 -- 172.21.15.33:0/975545184 shutdown_connections 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 -- 172.21.15.33:0/975545184 wait complete. 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 -- start start 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.918+0000 7f00393d3740 1 Processor -- start 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a5c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c62a5ad40 con 0x563c62a52230 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563c62972050 con 0x563c62a57d90 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c62a5aec0 con 0x563c62a52c20 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 10 client.0 _ll_register_callbacks cb 0x563c626c9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59882/0 (socket says 172.21.15.33:59882) 2024-04-24T23:24:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 -- 172.21.15.33:0/1204571532 learned_addr learned my addr 172.21.15.33:0/1204571532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f0034c5a640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a5c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 -- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 msgr2=0x563c62a5c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a5c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 -- 172.21.15.33:0/1204571532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 msgr2=0x563c62a63820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a63820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 -- 172.21.15.33:0/1204571532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c62943dc0 con 0x563c62a52230 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a5c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f002ffff640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5c2a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f0020031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f2cc8960dce938 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f000f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f000f7fe640 1 -- 172.21.15.33:0/1204571532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020035040 con 0x563c62a52230 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f000f7fe640 1 -- 172.21.15.33:0/1204571532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00200042c0 con 0x563c62a52230 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.919+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0024010e30 con 0x563c62a52230 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f000f7fe640 1 -- 172.21.15.33:0/1204571532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f002004f050 con 0x563c62a52230 2024-04-24T23:24:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 10 client.17964 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c626ca3f0 con 0x563c62a52230 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f000f7fe640 1 -- 172.21.15.33:0/1204571532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024010e30 con 0x563c62a52230 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f000f7fe640 1 -- 172.21.15.33:0/1204571532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f00200035f0 con 0x563c62a52230 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 10 client.17964 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 20 client.17964 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62a68150 0x563c62a6a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 4 client.17964 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c629c3bb0 con 0x563c62a68150 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.920+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62a68150 0x563c62a6a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62a68150 0x563c62a6a5e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2515cc80dff387fb server_cookie=b01dc9aa26d10afb in_seq=0 out_seq=0 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f000f7fe640 10 client.17964 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f000f7fe640 1 -- 172.21.15.33:0/1204571532 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c629c3bb0 con 0x563c62a68150 2024-04-24T23:24:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f000f7fe640 10 client.17964 handle_command_reply: tid=0 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f00393d3740 1 client.17964 shutdown 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 msgr2=0x563c62a5c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f00393d3740 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5c2a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f0020031a90 comp rx=0 tx=0).stop 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 shutdown_connections 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.921+0000 7f00393d3740 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62a68150 0x563c62a6a5e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a63820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a5c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f000f7fe640 0 client.17964 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 1 --2- 172.21.15.33:0/1204571532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5c2a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 >> 172.21.15.33:0/1204571532 conn(0x563c62a4d8a0 msgr2=0x563c62a4cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 shutdown_connections 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 1 -- 172.21.15.33:0/1204571532 wait complete. 2024-04-24T23:24:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.922+0000 7f00393d3740 20 client.17964 trim_cache size 0 max 16384 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 Processor -- start 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 -- start start 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00301aee10 con 0x563c62a52230 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ac6e0 con 0x563c62a52c20 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f00393d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00301aa530 con 0x563c62a57d90 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f002ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f002ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59884/0 (socket says 172.21.15.33:59884) 2024-04-24T23:24:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.935+0000 7f002ffff640 1 -- 172.21.15.33:0/594871901 learned_addr learned my addr 172.21.15.33:0/594871901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002f7fe640 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f0034c5a640 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002ffff640 1 -- 172.21.15.33:0/594871901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 msgr2=0x563c62a5be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002ffff640 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002ffff640 1 -- 172.21.15.33:0/594871901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 msgr2=0x563c62a637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002ffff640 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002ffff640 1 -- 172.21.15.33:0/594871901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c62a490c0 con 0x563c62a52230 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f002ffff640 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5b8c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0020033000 tx=0x7f002002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c048c7d4534e9501 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f0015ffb640 1 -- 172.21.15.33:0/594871901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020000e60 con 0x563c62a52230 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f0015ffb640 1 -- 172.21.15.33:0/594871901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00200037f0 con 0x563c62a52230 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.936+0000 7f0015ffb640 1 -- 172.21.15.33:0/594871901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020035030 con 0x563c62a52230 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- 172.21.15.33:0/594871901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 msgr2=0x563c62a5b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5b8c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0020033000 tx=0x7f002002a410 comp rx=0 tx=0).stop 2024-04-24T23:24:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- 172.21.15.33:0/594871901 shutdown_connections 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a57d90 0x563c62a5be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a52c20 0x563c62a637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- 172.21.15.33:0/594871901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a5b8c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- 172.21.15.33:0/594871901 >> 172.21.15.33:0/594871901 conn(0x563c62a4d8a0 msgr2=0x563c62a51100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- 172.21.15.33:0/594871901 shutdown_connections 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- 172.21.15.33:0/594871901 wait complete. 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- start start 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 Processor -- start 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a60c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00301aaba0 con 0x563c62a52230 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ac010 con 0x563c62a57d90 2024-04-24T23:24:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.937+0000 7f00393d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ab270 con 0x563c62a52c20 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f00393d3740 10 client.0 _ll_register_callbacks cb 0x7f0020009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a60c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a60c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59898/0 (socket says 172.21.15.33:59898) 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 -- 172.21.15.33:0/1855047247 learned_addr learned my addr 172.21.15.33:0/1855047247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f0034c5a640 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 -- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 msgr2=0x563c62a61190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a61190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 -- 172.21.15.33:0/1855047247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 msgr2=0x563c62a621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 -- 172.21.15.33:0/1855047247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c627947d0 con 0x563c62a52230 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f002ffff640 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a60c50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0020005770 tx=0x7f0020004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e2e1c88321314f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f000f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f000f7fe640 1 -- 172.21.15.33:0/1855047247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020035050 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f000f7fe640 1 -- 172.21.15.33:0/1855047247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00200043d0 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00200037f0 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f000f7fe640 1 -- 172.21.15.33:0/1855047247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0020065050 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f00393d3740 10 client.17970 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.938+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c62a638c0 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f000f7fe640 1 -- 172.21.15.33:0/1855047247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00200037f0 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f000f7fe640 1 -- 172.21.15.33:0/1855047247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f002003d450 con 0x563c62a52230 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f00393d3740 10 client.17970 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f00393d3740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f00393d3740 20 client.17970 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f00393d3740 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62aceb60 0x563c62ad0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f00393d3740 4 client.17970 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c629bdf00 con 0x563c62aceb60 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62aceb60 0x563c62ad0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62aceb60 0x563c62ad0f80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5de6d9f4490910 server_cookie=bd35ec24c97000f8 in_seq=0 out_seq=0 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.939+0000 7f000f7fe640 10 client.17970 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f000f7fe640 1 -- 172.21.15.33:0/1855047247 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c629bdf00 con 0x563c62aceb60 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f000f7fe640 10 client.17970 handle_command_reply: tid=0 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 client.17970 shutdown 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 msgr2=0x563c62a60c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a60c50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0020005770 tx=0x7f0020004050 comp rx=0 tx=0).stop 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 shutdown_connections 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563c62aceb60 0x563c62ad0f80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c62a57d90 0x563c62a621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f000f7fe640 0 client.17970 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c62a52c20 0x563c62a61190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 --2- 172.21.15.33:0/1855047247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c62a52230 0x563c62a60c50 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 >> 172.21.15.33:0/1855047247 conn(0x563c62a4d8a0 msgr2=0x563c62a534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 shutdown_connections 2024-04-24T23:24:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 1 -- 172.21.15.33:0/1855047247 wait complete. 2024-04-24T23:24:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00393d3740 20 client.17970 trim_cache size 0 max 16384 2024-04-24T23:24:01.948 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f00180a2b90 msgr2=0x7f00180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00366bd640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f00180a2b90 0x7f00180a5050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0024023000 tx=0x7f0024024000 comp rx=0 tx=0).stop 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 msgr2=0x7f00301031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.940+0000 7f00366bd640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 0x7f00301031b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0028009fd0 tx=0x7f002800b260 comp rx=0 tx=0).stop 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 shutdown_connections 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f00180a2b90 0x7f00180a5050 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030102250 0x7f00301031b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0030101890 0x7f0030102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 --2- 172.21.15.33:0/2868212451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00301006a0 0x7f0030104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 >> 172.21.15.33:0/2868212451 conn(0x7f00300fbc30 msgr2=0x7f00300fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 shutdown_connections 2024-04-24T23:24:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:01.941+0000 7f00366bd640 1 -- 172.21.15.33:0/2868212451 wait complete. 2024-04-24T23:24:02.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:02 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1666268403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:24:02.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:02 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1666268403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:24:02.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:02 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1666268403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:24:02.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1409994 12.41 MB/sec execute 487 sec latency 13.151 ms 2024-04-24T23:24:02.953 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:24:02.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:02.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 -- 172.21.15.33:0/673243703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 msgr2=0x7fd608100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 --2- 172.21.15.33:0/673243703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd608100aa0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009900 tx=0x7fd5f802a410 comp rx=0 tx=0).stop 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 -- 172.21.15.33:0/673243703 shutdown_connections 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 --2- 172.21.15.33:0/673243703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd608102230 0x7fd608108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 --2- 172.21.15.33:0/673243703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 0x7fd608101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 --2- 172.21.15.33:0/673243703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd608100aa0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 -- 172.21.15.33:0/673243703 >> 172.21.15.33:0/673243703 conn(0x7fd6080fbbf0 msgr2=0x7fd6080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 -- 172.21.15.33:0/673243703 shutdown_connections 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 -- 172.21.15.33:0/673243703 wait complete. 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 Processor -- start 2024-04-24T23:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.060+0000 7fd60e664640 1 -- start start 2024-04-24T23:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60e664640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd60810cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60e664640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 0x7fd60810b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60e664640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd608102230 0x7fd60810b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60e664640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810d420 con 0x7fd608100680 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60e664640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810d2a0 con 0x7fd608101870 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60e664640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810d5a0 con 0x7fd608102230 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd607fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd60810cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd607fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd60810cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59912/0 (socket says 172.21.15.33:59912) 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd607fff640 1 -- 172.21.15.33:0/1786395020 learned_addr learned my addr 172.21.15.33:0/1786395020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd6077fe640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 0x7fd60810b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd60cc01640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd608102230 0x7fd60810b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd6077fe640 1 -- 172.21.15.33:0/1786395020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd608102230 msgr2=0x7fd60810b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd6077fe640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd608102230 0x7fd60810b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd6077fe640 1 -- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 msgr2=0x7fd60810cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd6077fe640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd60810cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd6077fe640 1 -- 172.21.15.33:0/1786395020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60810bf30 con 0x7fd608101870 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.061+0000 7fd607fff640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd60810cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.062+0000 7fd6077fe640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 0x7fd60810b160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e9f0 tx=0x7fd5fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.062+0000 7fd6057fa640 1 -- 172.21.15.33:0/1786395020 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00ccc0 con 0x7fd608101870 2024-04-24T23:24:03.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.062+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6081a1bf0 con 0x7fd608101870 2024-04-24T23:24:03.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.062+0000 7fd6057fa640 1 -- 172.21.15.33:0/1786395020 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc010910 con 0x7fd608101870 2024-04-24T23:24:03.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.062+0000 7fd6057fa640 1 -- 172.21.15.33:0/1786395020 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc018960 con 0x7fd608101870 2024-04-24T23:24:03.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.062+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6081a2190 con 0x7fd608101870 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 Processor -- start 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 -- start start 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd6057fa640 1 -- 172.21.15.33:0/1786395020 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd5fc010ac0 con 0x7fd608101870 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447d7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b447332c0 con 0x561b447d0780 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561b446db910 con 0x561b447d58f0 2024-04-24T23:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd611373740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b4472f280 con 0x561b447cfd90 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.065+0000 7fd607fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd607fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447d5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59928/0 (socket says 172.21.15.33:59928) 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd607fff640 1 -- 172.21.15.33:0/206177631 learned_addr learned my addr 172.21.15.33:0/206177631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd6077fe640 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447d7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 -- 172.21.15.33:0/206177631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 msgr2=0x561b447d7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447d7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 -- 172.21.15.33:0/206177631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 msgr2=0x561b447d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 -- 172.21.15.33:0/206177631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b446ecfb0 con 0x561b447cfd90 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd6077fe640 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447d7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd6057fa640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5f00a2b90 0x7fd5f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd60cc01640 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447d01b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd600009f00 tx=0x7fd600011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e88f5ed4977a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd607fff640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5f00a2b90 0x7fd5f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd6057fa640 1 -- 172.21.15.33:0/1786395020 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd5fc0cf740 con 0x7fd608101870 2024-04-24T23:24:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd5ee7fc640 1 -- 172.21.15.33:0/206177631 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600004d20 con 0x561b447cfd90 2024-04-24T23:24:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd5ee7fc640 1 -- 172.21.15.33:0/206177631 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600004030 con 0x561b447cfd90 2024-04-24T23:24:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.066+0000 7fd5ee7fc640 1 -- 172.21.15.33:0/206177631 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600005260 con 0x561b447cfd90 2024-04-24T23:24:03.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 -- 172.21.15.33:0/206177631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 msgr2=0x561b447d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447d01b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd600009f00 tx=0x7fd600011810 comp rx=0 tx=0).stop 2024-04-24T23:24:03.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd607fff640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5f00a2b90 0x7fd5f00a5050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x561b447d0b80 tx=0x7fd5f80046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 -- 172.21.15.33:0/206177631 shutdown_connections 2024-04-24T23:24:03.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447d7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 --2- 172.21.15.33:0/206177631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447d01b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 -- 172.21.15.33:0/206177631 >> 172.21.15.33:0/206177631 conn(0x561b44797950 msgr2=0x561b447cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 -- 172.21.15.33:0/206177631 shutdown_connections 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 -- 172.21.15.33:0/206177631 wait complete. 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.067+0000 7fd611373740 1 -- start start 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 Processor -- start 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d0780 0x561b447da730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 0x561b447e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b447d88a0 con 0x561b447d58f0 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561b446eee90 con 0x561b447d0780 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b447d8a20 con 0x561b447cfd90 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd611373740 10 client.0 _ll_register_callbacks cb 0x561b4472a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:03.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 0x561b447e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd607fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 0x561b447e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59940/0 (socket says 172.21.15.33:59940) 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 -- 172.21.15.33:0/3747596167 learned_addr learned my addr 172.21.15.33:0/3747596167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d0780 0x561b447da730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 -- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 msgr2=0x561b447da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 -- 172.21.15.33:0/3747596167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d0780 msgr2=0x561b447da730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d0780 0x561b447da730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd60cc01640 1 -- 172.21.15.33:0/3747596167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b4450f7e0 con 0x561b447d58f0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.068+0000 7fd607fff640 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd60cc01640 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 0x561b447e1770 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd600005f40 tx=0x7fd6000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f965c4c32ea6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd5e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd5e77fe640 1 -- 172.21.15.33:0/3747596167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600027020 con 0x561b447d58f0 2024-04-24T23:24:03.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd5e77fe640 1 -- 172.21.15.33:0/3747596167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd600004c50 con 0x561b447d58f0 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd600004030 con 0x561b447d58f0 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd5e77fe640 1 -- 172.21.15.33:0/3747596167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd600032050 con 0x561b447d58f0 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd611373740 10 client.17982 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b4456ff70 con 0x561b447d58f0 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.069+0000 7fd5e77fe640 1 -- 172.21.15.33:0/3747596167 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600004030 con 0x561b447d58f0 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd5e77fe640 1 -- 172.21.15.33:0/3747596167 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd600018070 con 0x561b447d58f0 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd611373740 10 client.17982 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd611373740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd611373740 20 client.17982 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd611373740 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b447e5a90 0x561b447e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd611373740 4 client.17982 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b447075a0 con 0x561b447e5a90 2024-04-24T23:24:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd607fff640 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b447e5a90 0x561b447e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd607fff640 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b447e5a90 0x561b447e7ea0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c38bc2c70c8be3 server_cookie=dda7ae72e0c0ba18 in_seq=0 out_seq=0 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.070+0000 7fd5e77fe640 10 client.17982 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd5e77fe640 1 -- 172.21.15.33:0/3747596167 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b447075a0 con 0x561b447e5a90 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd5e77fe640 10 client.17982 handle_command_reply: tid=0 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 client.17982 shutdown 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 msgr2=0x561b447e1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 0x561b447e1770 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd600005f40 tx=0x7fd6000042f0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 shutdown_connections 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b447e5a90 0x561b447e7ea0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d58f0 0x561b447e1770 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d0780 0x561b447da730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd5e77fe640 0 client.17982 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 --2- 172.21.15.33:0/3747596167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 >> 172.21.15.33:0/3747596167 conn(0x561b44797950 msgr2=0x561b447ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 shutdown_connections 2024-04-24T23:24:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 1 -- 172.21.15.33:0/3747596167 wait complete. 2024-04-24T23:24:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.071+0000 7fd611373740 20 client.17982 trim_cache size 0 max 16384 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 Processor -- start 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 -- start start 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447e1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081aeea0 con 0x561b447d0780 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ac770 con 0x561b447d58f0 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd611373740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081aa5c0 con 0x561b447cfd90 2024-04-24T23:24:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447e1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59944/0 (socket says 172.21.15.33:59944) 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 -- 172.21.15.33:0/2433574427 learned_addr learned my addr 172.21.15.33:0/2433574427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd60cc01640 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd607fff640 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 -- 172.21.15.33:0/2433574427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 msgr2=0x561b447ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 -- 172.21.15.33:0/2433574427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 msgr2=0x561b447cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 -- 172.21.15.33:0/2433574427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b447d2260 con 0x561b447d0780 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd607fff640 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.090+0000 7fd6077fe640 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447e1730 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0cceb0 tx=0x7fd5fc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69957ac7830b48b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd5edffb640 1 -- 172.21.15.33:0/2433574427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0a4880 con 0x561b447d0780 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd5edffb640 1 -- 172.21.15.33:0/2433574427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc099fc0 con 0x561b447d0780 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd5edffb640 1 -- 172.21.15.33:0/2433574427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc018df0 con 0x561b447d0780 2024-04-24T23:24:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 -- 172.21.15.33:0/2433574427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 msgr2=0x561b447e1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447e1730 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0cceb0 tx=0x7fd5fc009700 comp rx=0 tx=0).stop 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 -- 172.21.15.33:0/2433574427 shutdown_connections 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447d58f0 0x561b447cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447e1730 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 --2- 172.21.15.33:0/2433574427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447cfd90 0x561b447ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 -- 172.21.15.33:0/2433574427 >> 172.21.15.33:0/2433574427 conn(0x561b44797950 msgr2=0x561b447da1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 -- 172.21.15.33:0/2433574427 shutdown_connections 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.091+0000 7fd611373740 1 -- 172.21.15.33:0/2433574427 wait complete. 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 -- start start 2024-04-24T23:24:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 Processor -- start 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447cfd90 0x561b447df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447d58f0 0x561b447e0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081aac30 con 0x561b447d0780 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ac0a0 con 0x561b447cfd90 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ab300 con 0x561b447d58f0 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd611373740 10 client.0 _ll_register_callbacks cb 0x7fd6000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447dfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59948/0 (socket says 172.21.15.33:59948) 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd6077fe640 1 -- 172.21.15.33:0/4100068586 learned_addr learned my addr 172.21.15.33:0/4100068586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd607fff640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447cfd90 0x561b447df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.092+0000 7fd60cc01640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447d58f0 0x561b447e0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd6077fe640 1 -- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447d58f0 msgr2=0x561b447e0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd6077fe640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447d58f0 0x561b447e0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd6077fe640 1 -- 172.21.15.33:0/4100068586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447cfd90 msgr2=0x561b447df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd6077fe640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447cfd90 0x561b447df5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd6077fe640 1 -- 172.21.15.33:0/4100068586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b447cba60 con 0x561b447d0780 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd6077fe640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447dfb30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c6a0 tx=0x7fd5fc0a00f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57324c22f1814b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd5e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd5e77fe640 1 -- 172.21.15.33:0/4100068586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0a4880 con 0x561b447d0780 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd5e77fe640 1 -- 172.21.15.33:0/4100068586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd5fc0d5070 con 0x561b447d0780 2024-04-24T23:24:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc099fc0 con 0x561b447d0780 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd5e77fe640 1 -- 172.21.15.33:0/4100068586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5fc09b110 con 0x561b447d0780 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 10 client.17988 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b44407170 con 0x561b447d0780 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd5e77fe640 1 -- 172.21.15.33:0/4100068586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc099fc0 con 0x561b447d0780 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd5e77fe640 1 -- 172.21.15.33:0/4100068586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd5fc0ba2a0 con 0x561b447d0780 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 10 client.17988 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 20 client.17988 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b4484cdd0 0x561b4484f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 4 client.17988 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.093+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b44740800 con 0x561b4484cdd0 2024-04-24T23:24:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.094+0000 7fd607fff640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b4484cdd0 0x561b4484f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.094+0000 7fd5e77fe640 10 client.17988 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.094+0000 7fd607fff640 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b4484cdd0 0x561b4484f1f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac036ce979d2886 server_cookie=ab9f579aa5e61f64 in_seq=0 out_seq=0 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.095+0000 7fd5e77fe640 1 -- 172.21.15.33:0/4100068586 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b44740800 con 0x561b4484cdd0 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.095+0000 7fd5e77fe640 10 client.17988 handle_command_reply: tid=0 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.095+0000 7fd611373740 1 client.17988 shutdown 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.095+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 msgr2=0x561b447dfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447dfb30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c6a0 tx=0x7fd5fc0a00f0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 shutdown_connections 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561b4484cdd0 0x561b4484f1f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b447d58f0 0x561b447e0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b447d0780 0x561b447dfb30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 --2- 172.21.15.33:0/4100068586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561b447cfd90 0x561b447df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 >> 172.21.15.33:0/4100068586 conn(0x561b44797950 msgr2=0x561b447ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd5e77fe640 0 client.17988 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 shutdown_connections 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 1 -- 172.21.15.33:0/4100068586 wait complete. 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.096+0000 7fd611373740 20 client.17988 trim_cache size 0 max 16384 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:24:03.103 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "2a51e823-3118-4c4b-9477-2588f26fac92", 2024-04-24T23:24:03.104 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:24:03.104 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:24:03.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5f00a2b90 msgr2=0x7fd5f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5f00a2b90 0x7fd5f00a5050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x561b447d0b80 tx=0x7fd5f80046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 msgr2=0x7fd60810b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 0x7fd60810b160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e9f0 tx=0x7fd5fc00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 shutdown_connections 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd5f00a2b90 0x7fd5f00a5050 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd608102230 0x7fd60810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd608101870 0x7fd60810b160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 --2- 172.21.15.33:0/1786395020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd608100680 0x7fd60810cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 >> 172.21.15.33:0/1786395020 conn(0x7fd6080fbbf0 msgr2=0x7fd608104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 shutdown_connections 2024-04-24T23:24:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.097+0000 7fd60e664640 1 -- 172.21.15.33:0/1786395020 wait complete. 2024-04-24T23:24:03.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:24:03.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:24:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 -- 172.21.15.33:0/3405538690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 msgr2=0x7f6734100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 --2- 172.21.15.33:0/3405538690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f6734100ac0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6730009900 tx=0x7f673002a410 comp rx=0 tx=0).stop 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 -- 172.21.15.33:0/3405538690 shutdown_connections 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 --2- 172.21.15.33:0/3405538690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6734102250 0x7f6734108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 --2- 172.21.15.33:0/3405538690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6734101890 0x7f6734101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 --2- 172.21.15.33:0/3405538690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f6734100ac0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 -- 172.21.15.33:0/3405538690 >> 172.21.15.33:0/3405538690 conn(0x7f67340fbc30 msgr2=0x7f67340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 -- 172.21.15.33:0/3405538690 shutdown_connections 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.226+0000 7f673acfc640 1 -- 172.21.15.33:0/3405538690 wait complete. 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 Processor -- start 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 -- start start 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f67341045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6734101890 0x7f6734102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6734102250 0x7f67341031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f673410c320 con 0x7f67341006a0 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f673410c1a0 con 0x7f6734101890 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f673acfc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f673410c4a0 con 0x7f6734102250 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f67341045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f67341045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59966/0 (socket says 172.21.15.33:59966) 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 -- 172.21.15.33:0/3326555596 learned_addr learned my addr 172.21.15.33:0/3326555596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f672bfff640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6734101890 0x7f6734102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6739299640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6734102250 0x7f67341031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 -- 172.21.15.33:0/3326555596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6734102250 msgr2=0x7f67341031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6734102250 0x7f67341031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 -- 172.21.15.33:0/3326555596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6734101890 msgr2=0x7f6734102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6734101890 0x7f6734102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 -- 172.21.15.33:0/3326555596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6734103a70 con 0x7f67341006a0 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.227+0000 7f6738a98640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f67341045e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6730005ab0 tx=0x7f6730004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.228+0000 7f6729ffb640 1 -- 172.21.15.33:0/3326555596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673003f030 con 0x7f67341006a0 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.228+0000 7f6729ffb640 1 -- 172.21.15.33:0/3326555596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6730003ad0 con 0x7f67341006a0 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.228+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67341a1c90 con 0x7f67341006a0 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.228+0000 7f6729ffb640 1 -- 172.21.15.33:0/3326555596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730035070 con 0x7f67341006a0 2024-04-24T23:24:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.228+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67341a21a0 con 0x7f67341006a0 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.230+0000 7f6729ffb640 1 -- 172.21.15.33:0/3326555596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6730042030 con 0x7f67341006a0 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.230+0000 7f673da07740 1 Processor -- start 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.230+0000 7f673da07740 1 -- start start 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.230+0000 7f673da07740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc189660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f673da07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f673da07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 0x556ddc191230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f673da07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ddc0ed6b0 con 0x556ddc18eda0 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f673da07740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ddc095880 con 0x556ddc189240 2024-04-24T23:24:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f673da07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ddc0e9670 con 0x556ddc189c30 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 0x556ddc191230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f6738a98640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f6739299640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc189660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 0x556ddc191230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59974/0 (socket says 172.21.15.33:59974) 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 -- 172.21.15.33:0/2065643205 learned_addr learned my addr 172.21.15.33:0/2065643205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f6739299640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc189660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:45692/0 (socket says 172.21.15.33:45692) 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 -- 172.21.15.33:0/2065643205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 msgr2=0x556ddc18e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 -- 172.21.15.33:0/2065643205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 msgr2=0x556ddc189660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc189660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 -- 172.21.15.33:0/2065643205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ddbecb7d0 con 0x556ddc18eda0 2024-04-24T23:24:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f6729ffb640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f671c0a2c10 0x7f671c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f672bfff640 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 0x556ddc191230 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6724009ca0 tx=0x7f6724011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626b0678b7413587 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f6739299640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f671c0a2c10 0x7f671c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f6729ffb640 1 -- 172.21.15.33:0/3326555596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67300ef9b0 con 0x7f67341006a0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f671affd640 1 -- 172.21.15.33:0/2065643205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672401e070 con 0x556ddc18eda0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f671affd640 1 -- 172.21.15.33:0/2065643205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724019b10 con 0x556ddc18eda0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.231+0000 7f671affd640 1 -- 172.21.15.33:0/2065643205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400ea80 con 0x556ddc18eda0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 -- 172.21.15.33:0/2065643205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 msgr2=0x556ddc191230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f6739299640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f671c0a2c10 0x7f671c0a50d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x556ddc18a490 tx=0x7f672c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 0x556ddc191230 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6724009ca0 tx=0x7f6724011810 comp rx=0 tx=0).stop 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 -- 172.21.15.33:0/2065643205 shutdown_connections 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc18eda0 0x556ddc191230 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 --2- 172.21.15.33:0/2065643205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc189660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 -- 172.21.15.33:0/2065643205 >> 172.21.15.33:0/2065643205 conn(0x556ddc1848b0 msgr2=0x556ddc184cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 -- 172.21.15.33:0/2065643205 shutdown_connections 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.232+0000 7f673da07740 1 -- 172.21.15.33:0/2065643205 wait complete. 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 -- start start 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 Processor -- start 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc193660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 0x556ddc193ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ddc191d50 con 0x556ddc189c30 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ddc0a8f50 con 0x556ddc189240 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ddc191ed0 con 0x556ddc18eda0 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f673da07740 10 client.0 _ll_register_callbacks cb 0x556ddc0e4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f672bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 0x556ddc193ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f672bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 0x556ddc193ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59980/0 (socket says 172.21.15.33:59980) 2024-04-24T23:24:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f672bfff640 1 -- 172.21.15.33:0/2405023289 learned_addr learned my addr 172.21.15.33:0/2405023289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f6738a98640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc193660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f6739299640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.233+0000 7f672bfff640 1 -- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 msgr2=0x556ddc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f672bfff640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f672bfff640 1 -- 172.21.15.33:0/2405023289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 msgr2=0x556ddc193660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f672bfff640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc193660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f672bfff640 1 -- 172.21.15.33:0/2405023289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ddbe013f0 con 0x556ddc189c30 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f672bfff640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 0x556ddc193ba0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6724009710 tx=0x7f6724009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7907cb9ff9a053f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f6713fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f6713fff640 1 -- 172.21.15.33:0/2405023289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724027020 con 0x556ddc189c30 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f6713fff640 1 -- 172.21.15.33:0/2405023289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6724005460 con 0x556ddc189c30 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724019b10 con 0x556ddc189c30 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f6713fff640 1 -- 172.21.15.33:0/2405023289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6724032050 con 0x556ddc189c30 2024-04-24T23:24:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f673da07740 10 client.18000 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ddbdc10f0 con 0x556ddc189c30 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f6713fff640 1 -- 172.21.15.33:0/2405023289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724019b10 con 0x556ddc189c30 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f6713fff640 1 -- 172.21.15.33:0/2405023289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6724004b30 con 0x556ddc189c30 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f673da07740 10 client.18000 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f673da07740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.234+0000 7f673da07740 20 client.18000 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.235+0000 7f673da07740 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc19f270 0x556ddc1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.235+0000 7f673da07740 4 client.18000 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.235+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ddc0fabf0 con 0x556ddc19f270 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.235+0000 7f6738a98640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc19f270 0x556ddc1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.235+0000 7f6738a98640 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc19f270 0x556ddc1a16e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cb0de19ece195e server_cookie=dceb0a9ba2625831 in_seq=0 out_seq=0 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.235+0000 7f6713fff640 10 client.18000 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f6713fff640 1 -- 172.21.15.33:0/2405023289 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ddc0fabf0 con 0x556ddc19f270 2024-04-24T23:24:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f6713fff640 10 client.18000 handle_command_reply: tid=0 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 client.18000 shutdown 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 msgr2=0x556ddc193ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 0x556ddc193ba0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6724009710 tx=0x7f6724009950 comp rx=0 tx=0).stop 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 shutdown_connections 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc19f270 0x556ddc1a16e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189c30 0x556ddc193ba0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 --2- 172.21.15.33:0/2405023289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189240 0x556ddc193660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f6713fff640 0 client.18000 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 >> 172.21.15.33:0/2405023289 conn(0x556ddc1848b0 msgr2=0x556ddc183af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 shutdown_connections 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 1 -- 172.21.15.33:0/2405023289 wait complete. 2024-04-24T23:24:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.236+0000 7f673da07740 20 client.18000 trim_cache size 0 max 16384 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.256+0000 7f673da07740 1 Processor -- start 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.256+0000 7f673da07740 1 -- start start 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f673da07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc18aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f673da07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f673da07740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc18eda0 0x556ddc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f673da07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67341aef30 con 0x556ddc189240 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f673da07740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f67341ac800 con 0x556ddc18eda0 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f673da07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67341aa650 con 0x556ddc189c30 2024-04-24T23:24:03.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f6738a98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc18aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f6738a98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc18aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59996/0 (socket says 172.21.15.33:59996) 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f6738a98640 1 -- 172.21.15.33:0/271787549 learned_addr learned my addr 172.21.15.33:0/271787549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f6739299640 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc18eda0 0x556ddc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 -- 172.21.15.33:0/271787549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc18eda0 msgr2=0x556ddc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc18eda0 0x556ddc19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 -- 172.21.15.33:0/271787549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 msgr2=0x556ddc18aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc18aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 -- 172.21.15.33:0/271787549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ddc176090 con 0x556ddc189c30 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f6738a98640 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc18aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f672bfff640 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18b530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67240234f0 tx=0x7f6724009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb40869ca590ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.257+0000 7f671a7fc640 1 -- 172.21.15.33:0/271787549 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724027070 con 0x556ddc189c30 2024-04-24T23:24:03.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.258+0000 7f671a7fc640 1 -- 172.21.15.33:0/271787549 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672401f040 con 0x556ddc189c30 2024-04-24T23:24:03.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.258+0000 7f671a7fc640 1 -- 172.21.15.33:0/271787549 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672401e070 con 0x556ddc189c30 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.258+0000 7f673da07740 1 -- 172.21.15.33:0/271787549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 msgr2=0x556ddc18b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.258+0000 7f673da07740 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18b530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67240234f0 tx=0x7f6724009f10 comp rx=0 tx=0).stop 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 -- 172.21.15.33:0/271787549 shutdown_connections 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc18eda0 0x556ddc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc189c30 0x556ddc18b530 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 --2- 172.21.15.33:0/271787549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc18aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 -- 172.21.15.33:0/271787549 >> 172.21.15.33:0/271787549 conn(0x556ddc1848b0 msgr2=0x556ddc184c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 -- 172.21.15.33:0/271787549 shutdown_connections 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 -- 172.21.15.33:0/271787549 wait complete. 2024-04-24T23:24:03.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 -- start start 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.259+0000 7f673da07740 1 Processor -- start 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc197860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189c30 0x556ddc197da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc203810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67341aacc0 con 0x556ddc189240 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f67341ac130 con 0x556ddc189c30 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67341ab390 con 0x556ddc18eda0 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 10 client.0 _ll_register_callbacks cb 0x7f672401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60012/0 (socket says 172.21.15.33:60012) 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 -- 172.21.15.33:0/1520148262 learned_addr learned my addr 172.21.15.33:0/1520148262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f672bfff640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189c30 0x556ddc197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6739299640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc203810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 -- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 msgr2=0x556ddc203810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc203810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 -- 172.21.15.33:0/1520148262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189c30 msgr2=0x556ddc197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189c30 0x556ddc197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 -- 172.21.15.33:0/1520148262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ddc1983e0 con 0x556ddc189240 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6738a98640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc197860 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f673003bfd0 tx=0x7f6730000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf1376ac3199a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6713fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6713fff640 1 -- 172.21.15.33:0/1520148262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67300efe60 con 0x556ddc189240 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f6713fff640 1 -- 172.21.15.33:0/1520148262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67300b9f20 con 0x556ddc189240 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.260+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672401f040 con 0x556ddc189240 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f6713fff640 1 -- 172.21.15.33:0/1520148262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6730049040 con 0x556ddc189240 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 10 client.18006 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:03.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ddc204ec0 con 0x556ddc189240 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f6713fff640 1 -- 172.21.15.33:0/1520148262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672401f040 con 0x556ddc189240 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f6713fff640 1 -- 172.21.15.33:0/1520148262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f67300b84f0 con 0x556ddc189240 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 10 client.18006 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 20 client.18006 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc205810 0x556ddc207c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 4 client.18006 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.261+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ddc0f4f40 con 0x556ddc205810 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f672bfff640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc205810 0x556ddc207c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f6713fff640 10 client.18006 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f672bfff640 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc205810 0x556ddc207c00 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334158aa414398a9 server_cookie=e6478ce12dca5f6 in_seq=0 out_seq=0 2024-04-24T23:24:03.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f6713fff640 1 -- 172.21.15.33:0/1520148262 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556ddc0f4f40 con 0x556ddc205810 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f6713fff640 10 client.18006 handle_command_reply: tid=0 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 client.18006 shutdown 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 msgr2=0x556ddc197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc197860 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f673003bfd0 tx=0x7f6730000bc0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 shutdown_connections 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556ddc205810 0x556ddc207c00 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ddc18eda0 0x556ddc203810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ddc189c30 0x556ddc197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f6713fff640 0 client.18006 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 --2- 172.21.15.33:0/1520148262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ddc189240 0x556ddc197860 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.262+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 >> 172.21.15.33:0/1520148262 conn(0x556ddc1848b0 msgr2=0x556ddc193640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 shutdown_connections 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673da07740 1 -- 172.21.15.33:0/1520148262 wait complete. 2024-04-24T23:24:03.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673da07740 20 client.18006 trim_cache size 0 max 16384 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: "2a51e823-3118-4c4b-9477-2588f26fac92": { 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "2a51e823-3118-4c4b-9477-2588f26fac92", 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:24:03.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:24:03.273 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:24:03.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f671c0a2c10 msgr2=0x7f671c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f671c0a2c10 0x7f671c0a50d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x556ddc18a490 tx=0x7f672c009500 comp rx=0 tx=0).stop 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 msgr2=0x7f67341045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f67341045e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6730005ab0 tx=0x7f6730004270 comp rx=0 tx=0).stop 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 shutdown_connections 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f671c0a2c10 0x7f671c0a50d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6734102250 0x7f67341031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6734101890 0x7f6734102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 --2- 172.21.15.33:0/3326555596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67341006a0 0x7f67341045e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 >> 172.21.15.33:0/3326555596 conn(0x7f67340fbc30 msgr2=0x7f67340fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 shutdown_connections 2024-04-24T23:24:03.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:03.263+0000 7f673acfc640 1 -- 172.21.15.33:0/3326555596 wait complete. 2024-04-24T23:24:03.275 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:03 smithi033 ceph-mon[29201]: pgmap v712: 129 pgs: 129 active+clean; 279 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-04-24T23:24:03.275 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:03 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:24:03.275 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:03 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:03.276 INFO:tasks.cephfs.filesystem:scrub status for tag:2a51e823-3118-4c4b-9477-2588f26fac92 - {'path': '/', 'tag': '2a51e823-3118-4c4b-9477-2588f26fac92', 'options': 'recursive,force'} 2024-04-24T23:24:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:03 smithi038 ceph-mon[32450]: pgmap v712: 129 pgs: 129 active+clean; 279 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-04-24T23:24:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:03 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:24:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:03 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:03 smithi117 ceph-mon[32529]: pgmap v712: 129 pgs: 129 active+clean; 279 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-04-24T23:24:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:03 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:24:03.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:03 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:03.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1412046 12.41 MB/sec execute 488 sec latency 12.442 ms 2024-04-24T23:24:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:04 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:24:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:04 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:04 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:24:04.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:04 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:04.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:04 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:24:04.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:04 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:04 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:24:04.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:04 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:04.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:04 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:24:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:04 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:04 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:24:04.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:04 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:04.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1414558 12.41 MB/sec execute 489 sec latency 23.760 ms 2024-04-24T23:24:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:05 smithi038 ceph-mon[32450]: pgmap v713: 129 pgs: 129 active+clean; 279 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-04-24T23:24:05.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:05 smithi117 ceph-mon[32529]: pgmap v713: 129 pgs: 129 active+clean; 279 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-04-24T23:24:05.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:05 smithi033 ceph-mon[29201]: pgmap v713: 129 pgs: 129 active+clean; 279 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-04-24T23:24:05.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1416698 12.40 MB/sec execute 490 sec latency 108.073 ms 2024-04-24T23:24:06.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1418542 12.40 MB/sec execute 491 sec latency 24.520 ms 2024-04-24T23:24:07.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:07 smithi117 ceph-mon[32529]: pgmap v714: 129 pgs: 129 active+clean; 280 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.4 MiB/s wr, 251 op/s 2024-04-24T23:24:07.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:07 smithi033 ceph-mon[29201]: pgmap v714: 129 pgs: 129 active+clean; 280 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.4 MiB/s wr, 251 op/s 2024-04-24T23:24:07.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1421564 12.41 MB/sec execute 492 sec latency 7.810 ms 2024-04-24T23:24:07.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:07 smithi038 ceph-mon[32450]: pgmap v714: 129 pgs: 129 active+clean; 280 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.4 MiB/s wr, 251 op/s 2024-04-24T23:24:08.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:08 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1423774 12.40 MB/sec execute 493 sec latency 14.256 ms 2024-04-24T23:24:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:08 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:08.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:08.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:08 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:09 smithi033 ceph-mon[29201]: pgmap v715: 129 pgs: 129 active+clean; 261 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-04-24T23:24:09.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:09.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:09.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:09.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:09.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1425657 12.40 MB/sec execute 494 sec latency 12.169 ms 2024-04-24T23:24:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:09 smithi038 ceph-mon[32450]: pgmap v715: 129 pgs: 129 active+clean; 261 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-04-24T23:24:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:09.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:09 smithi117 ceph-mon[32529]: pgmap v715: 129 pgs: 129 active+clean; 261 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-04-24T23:24:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:09.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:09.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:24:10.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1428219 12.41 MB/sec execute 495 sec latency 51.321 ms 2024-04-24T23:24:11.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:10 smithi033 ceph-mon[29201]: pgmap v716: 129 pgs: 129 active+clean; 255 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-04-24T23:24:11.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:10 smithi038 ceph-mon[32450]: pgmap v716: 129 pgs: 129 active+clean; 255 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-04-24T23:24:11.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:10 smithi117 ceph-mon[32529]: pgmap v716: 129 pgs: 129 active+clean; 255 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-04-24T23:24:11.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1429906 12.40 MB/sec execute 496 sec latency 19.954 ms 2024-04-24T23:24:12.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1432545 12.40 MB/sec execute 497 sec latency 12.562 ms 2024-04-24T23:24:13.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:13 smithi038 ceph-mon[32450]: pgmap v717: 129 pgs: 129 active+clean; 257 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-04-24T23:24:13.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:24:13.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:13 smithi117 ceph-mon[32529]: pgmap v717: 129 pgs: 129 active+clean; 257 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-04-24T23:24:13.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:24:13.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:13 smithi033 ceph-mon[29201]: pgmap v717: 129 pgs: 129 active+clean; 257 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-04-24T23:24:13.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:24:13.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1435627 12.41 MB/sec execute 498 sec latency 10.507 ms 2024-04-24T23:24:14.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:14 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:24:14.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:14 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:24:14.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:14 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:24:14.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1437943 12.41 MB/sec execute 499 sec latency 24.947 ms 2024-04-24T23:24:15.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:15 smithi038 ceph-mon[32450]: pgmap v718: 129 pgs: 129 active+clean; 257 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-04-24T23:24:15.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:15 smithi117 ceph-mon[32529]: pgmap v718: 129 pgs: 129 active+clean; 257 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-04-24T23:24:15.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:15 smithi033 ceph-mon[29201]: pgmap v718: 129 pgs: 129 active+clean; 257 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-04-24T23:24:15.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1439776 12.41 MB/sec execute 500 sec latency 85.236 ms 2024-04-24T23:24:16.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1442534 12.41 MB/sec execute 501 sec latency 34.107 ms 2024-04-24T23:24:17.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:17 smithi038 ceph-mon[32450]: pgmap v719: 129 pgs: 129 active+clean; 259 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-04-24T23:24:17.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:17 smithi117 ceph-mon[32529]: pgmap v719: 129 pgs: 129 active+clean; 259 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-04-24T23:24:17.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:17 smithi033 ceph-mon[29201]: pgmap v719: 129 pgs: 129 active+clean; 259 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-04-24T23:24:17.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1444701 12.40 MB/sec execute 502 sec latency 13.844 ms 2024-04-24T23:24:18.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1446817 12.41 MB/sec execute 503 sec latency 12.895 ms 2024-04-24T23:24:19.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:19 smithi038 ceph-mon[32450]: pgmap v720: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-04-24T23:24:19.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:19 smithi117 ceph-mon[32529]: pgmap v720: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-04-24T23:24:19.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:19 smithi033 ceph-mon[29201]: pgmap v720: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-04-24T23:24:19.465 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1449869 12.42 MB/sec execute 504 sec latency 10.799 ms 2024-04-24T23:24:20.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1450985 12.40 MB/sec execute 505 sec latency 142.799 ms 2024-04-24T23:24:21.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:21 smithi038 ceph-mon[32450]: pgmap v721: 129 pgs: 129 active+clean; 261 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-04-24T23:24:21.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:21 smithi117 ceph-mon[32529]: pgmap v721: 129 pgs: 129 active+clean; 261 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-04-24T23:24:21.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:21 smithi033 ceph-mon[29201]: pgmap v721: 129 pgs: 129 active+clean; 261 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-04-24T23:24:21.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1453494 12.41 MB/sec execute 506 sec latency 17.037 ms 2024-04-24T23:24:22.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1456088 12.41 MB/sec execute 507 sec latency 12.616 ms 2024-04-24T23:24:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:23 smithi038 ceph-mon[32450]: pgmap v722: 129 pgs: 129 active+clean; 271 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-04-24T23:24:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:23 smithi117 ceph-mon[32529]: pgmap v722: 129 pgs: 129 active+clean; 271 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-04-24T23:24:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:23 smithi033 ceph-mon[29201]: pgmap v722: 129 pgs: 129 active+clean; 271 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-04-24T23:24:23.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1457736 12.40 MB/sec execute 508 sec latency 14.195 ms 2024-04-24T23:24:24.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1460550 12.41 MB/sec execute 509 sec latency 17.616 ms 2024-04-24T23:24:25.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:25 smithi038 ceph-mon[32450]: pgmap v723: 129 pgs: 129 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.9 MiB/s wr, 218 op/s 2024-04-24T23:24:25.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:25 smithi117 ceph-mon[32529]: pgmap v723: 129 pgs: 129 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.9 MiB/s wr, 218 op/s 2024-04-24T23:24:25.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:25 smithi033 ceph-mon[29201]: pgmap v723: 129 pgs: 129 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.9 MiB/s wr, 218 op/s 2024-04-24T23:24:25.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1462609 12.40 MB/sec execute 510 sec latency 85.992 ms 2024-04-24T23:24:26.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1464354 12.40 MB/sec execute 511 sec latency 34.924 ms 2024-04-24T23:24:27.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:27 smithi038 ceph-mon[32450]: pgmap v724: 129 pgs: 129 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-04-24T23:24:27.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:27 smithi117 ceph-mon[32529]: pgmap v724: 129 pgs: 129 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-04-24T23:24:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:27 smithi033 ceph-mon[29201]: pgmap v724: 129 pgs: 129 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-04-24T23:24:27.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1467268 12.41 MB/sec execute 512 sec latency 17.025 ms 2024-04-24T23:24:28.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1468859 12.40 MB/sec execute 513 sec latency 35.851 ms 2024-04-24T23:24:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:29 smithi038 ceph-mon[32450]: pgmap v725: 129 pgs: 129 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-04-24T23:24:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:29 smithi117 ceph-mon[32529]: pgmap v725: 129 pgs: 129 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-04-24T23:24:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:29 smithi033 ceph-mon[29201]: pgmap v725: 129 pgs: 129 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-04-24T23:24:29.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1471428 12.40 MB/sec execute 514 sec latency 8.528 ms 2024-04-24T23:24:30.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1474117 12.40 MB/sec execute 515 sec latency 21.331 ms 2024-04-24T23:24:31.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:31 smithi038 ceph-mon[32450]: pgmap v726: 129 pgs: 129 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-04-24T23:24:31.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:31 smithi117 ceph-mon[32529]: pgmap v726: 129 pgs: 129 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-04-24T23:24:31.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:31 smithi033 ceph-mon[29201]: pgmap v726: 129 pgs: 129 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-04-24T23:24:31.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1475201 12.39 MB/sec execute 516 sec latency 86.611 ms 2024-04-24T23:24:32.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1478096 12.40 MB/sec execute 517 sec latency 19.608 ms 2024-04-24T23:24:33.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:24:33.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:24:33.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 -- 172.21.15.33:0/3540055803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f70102250 msgr2=0x7f0f70108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 --2- 172.21.15.33:0/3540055803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f70102250 0x7f0f70108ae0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009900 tx=0x7f0f6802a410 comp rx=0 tx=0).stop 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 -- 172.21.15.33:0/3540055803 shutdown_connections 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 --2- 172.21.15.33:0/3540055803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f70102250 0x7f0f70108ae0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 --2- 172.21.15.33:0/3540055803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f70101890 0x7f0f70101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 --2- 172.21.15.33:0/3540055803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f701006a0 0x7f0f70100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 -- 172.21.15.33:0/3540055803 >> 172.21.15.33:0/3540055803 conn(0x7f0f700fbc30 msgr2=0x7f0f700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 -- 172.21.15.33:0/3540055803 shutdown_connections 2024-04-24T23:24:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.393+0000 7f0f74838640 1 -- 172.21.15.33:0/3540055803 wait complete. 2024-04-24T23:24:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 Processor -- start 2024-04-24T23:24:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 -- start start 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 0x7f0f7006a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f70101890 0x7f0f7019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f70102250 0x7f0f7019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010c300 con 0x7f0f701006a0 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010c180 con 0x7f0f70101890 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.394+0000 7f0f74838640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010c480 con 0x7f0f70102250 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 0x7f0f7006a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 0x7f0f7006a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59604/0 (socket says 172.21.15.33:59604) 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2368677546 learned_addr learned my addr 172.21.15.33:0/2368677546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f70101890 0x7f0f7019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f70102250 0x7f0f7019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2368677546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f70102250 msgr2=0x7f0f7019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f70102250 0x7f0f7019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2368677546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f70101890 msgr2=0x7f0f7019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f70101890 0x7f0f7019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2368677546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7019f6f0 con 0x7f0f701006a0 2024-04-24T23:24:33.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.395+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 0x7f0f7006a8b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000ebc0 tx=0x7f0f6000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.396+0000 7f0f677fe640 1 -- 172.21.15.33:0/2368677546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60003b70 con 0x7f0f701006a0 2024-04-24T23:24:33.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.396+0000 7f0f677fe640 1 -- 172.21.15.33:0/2368677546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f60003d10 con 0x7f0f701006a0 2024-04-24T23:24:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.396+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f701965f0 con 0x7f0f701006a0 2024-04-24T23:24:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.396+0000 7f0f677fe640 1 -- 172.21.15.33:0/2368677546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60019980 con 0x7f0f701006a0 2024-04-24T23:24:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.396+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f70196b40 con 0x7f0f701006a0 2024-04-24T23:24:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.399+0000 7f0f677fe640 1 -- 172.21.15.33:0/2368677546 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0f60019b20 con 0x7f0f701006a0 2024-04-24T23:24:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.400+0000 7f0f7761d740 1 Processor -- start 2024-04-24T23:24:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.400+0000 7f0f677fe640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0f5c0a2ce0 0x7f0f5c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.400+0000 7f0f677fe640 1 -- 172.21.15.33:0/2368677546 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0f600cfac0 con 0x7f0f701006a0 2024-04-24T23:24:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0f5c0a2ce0 0x7f0f5c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 -- start start 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe23730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe23da0 0x55acffe26230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acffd826b0 con 0x55acffe1ec30 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55acffd2a880 con 0x55acffe1e240 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f7761d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55acffd7e670 con 0x55acffe23da0 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe1e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55810/0 (socket says 172.21.15.33:55810) 2024-04-24T23:24:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2027572439 learned_addr learned my addr 172.21.15.33:0/2027572439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.401+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe23730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe23da0 0x55acffe26230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2027572439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe23da0 msgr2=0x55acffe26230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe23da0 0x55acffe26230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2027572439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 msgr2=0x55acffe23730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0f5c0a2ce0 0x7f0f5c0a51a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0f68004b60 tx=0x7f0f68005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe23730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2027572439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acffb607d0 con 0x55acffe1e240 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe23730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe1e660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009270 tx=0x7f0f6000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c57c5b9db8e16f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f64ff9640 1 -- 172.21.15.33:0/2027572439 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f600b4020 con 0x55acffe1e240 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f64ff9640 1 -- 172.21.15.33:0/2027572439 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6009bf90 con 0x55acffe1e240 2024-04-24T23:24:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.402+0000 7f0f64ff9640 1 -- 172.21.15.33:0/2027572439 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f600babd0 con 0x55acffe1e240 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 -- 172.21.15.33:0/2027572439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 msgr2=0x55acffe1e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe1e660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009270 tx=0x7f0f6000cf90 comp rx=0 tx=0).stop 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 -- 172.21.15.33:0/2027572439 shutdown_connections 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe23da0 0x55acffe26230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe23730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2027572439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe1e660 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 -- 172.21.15.33:0/2027572439 >> 172.21.15.33:0/2027572439 conn(0x55acffe198b0 msgr2=0x55acffe19cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 -- 172.21.15.33:0/2027572439 shutdown_connections 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.403+0000 7f0f7761d740 1 -- 172.21.15.33:0/2027572439 wait complete. 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 -- start start 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 Processor -- start 2024-04-24T23:24:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe28c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acffe26d50 con 0x55acffe1ec30 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55acffd3df50 con 0x55acffe23da0 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55acffe26ed0 con 0x55acffe1e240 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.404+0000 7f0f7761d740 10 client.0 _ll_register_callbacks cb 0x55acffd79000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55818/0 (socket says 172.21.15.33:55818) 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 -- 172.21.15.33:0/4109316317 learned_addr learned my addr 172.21.15.33:0/4109316317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe28c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 -- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 msgr2=0x55acffe286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe286d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 -- 172.21.15.33:0/4109316317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 msgr2=0x55acffe28c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe28c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 -- 172.21.15.33:0/4109316317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acffa963f0 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2fc50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f68004fd0 tx=0x7f0f6802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f134395174f8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f55ffb640 1 -- 172.21.15.33:0/4109316317 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68055060 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f55ffb640 1 -- 172.21.15.33:0/4109316317 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0f680053c0 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.405+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6009bf90 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f55ffb640 1 -- 172.21.15.33:0/4109316317 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f6804c040 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 10 client.27536 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acffa560f0 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f55ffb640 1 -- 172.21.15.33:0/4109316317 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6009bf90 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f55ffb640 1 -- 172.21.15.33:0/4109316317 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0f68031440 con 0x55acffe23da0 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 10 client.27536 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 10 client.27536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:33.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 20 client.27536 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:33.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffe3ccf0 0x55acffe3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 4 client.27536 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:33.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.406+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acffd8fbf0 con 0x55acffe3ccf0 2024-04-24T23:24:33.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.407+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffe3ccf0 0x55acffe3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.407+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffe3ccf0 0x55acffe3f150 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9047b51ce66a90ec server_cookie=22926da26c6e3d96 in_seq=0 out_seq=0 2024-04-24T23:24:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.407+0000 7f0f55ffb640 10 client.27536 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f55ffb640 1 -- 172.21.15.33:0/4109316317 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0f600a4e50 con 0x55acffe3ccf0 2024-04-24T23:24:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f55ffb640 10 client.27536 handle_command_reply: tid=0 2024-04-24T23:24:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 client.27536 shutdown 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 msgr2=0x55acffe2fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2fc50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f68004fd0 tx=0x7f0f6802a410 comp rx=0 tx=0).stop 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 shutdown_connections 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffe3ccf0 0x55acffe3f150 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2fc50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe28c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 --2- 172.21.15.33:0/4109316317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 >> 172.21.15.33:0/4109316317 conn(0x55acffe198b0 msgr2=0x55acffe18af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f55ffb640 0 client.27536 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 shutdown_connections 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 1 -- 172.21.15.33:0/4109316317 wait complete. 2024-04-24T23:24:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.408+0000 7f0f7761d740 20 client.27536 trim_cache size 0 max 16384 2024-04-24T23:24:33.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:33 smithi038 ceph-mon[32450]: pgmap v727: 129 pgs: 129 active+clean; 288 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-04-24T23:24:33.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:33 smithi117 ceph-mon[32529]: pgmap v727: 129 pgs: 129 active+clean; 288 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-04-24T23:24:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 Processor -- start 2024-04-24T23:24:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 -- start start 2024-04-24T23:24:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe27cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1ec30 0x55acffe2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe23da0 0x55acffe1c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701aa9b0 con 0x55acffe23da0 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701a8280 con 0x55acffe1e240 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f7761d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701a60d0 con 0x55acffe1ec30 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe27cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe27cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55826/0 (socket says 172.21.15.33:55826) 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2950669307 learned_addr learned my addr 172.21.15.33:0/2950669307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe23da0 0x55acffe1c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.432+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1ec30 0x55acffe2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2950669307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1ec30 msgr2=0x55acffe2fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1ec30 0x55acffe2fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2950669307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe23da0 msgr2=0x55acffe1c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe23da0 0x55acffe1c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6e59c640 1 -- 172.21.15.33:0/2950669307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acffe0b090 con 0x55acffe1e240 2024-04-24T23:24:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe23da0 0x55acffe1c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe27cf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000eb90 tx=0x7f0f60009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49d97f5da539591 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f57fff640 1 -- 172.21.15.33:0/2950669307 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f600aa070 con 0x55acffe1e240 2024-04-24T23:24:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f57fff640 1 -- 172.21.15.33:0/2950669307 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f600a4e50 con 0x55acffe1e240 2024-04-24T23:24:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.433+0000 7f0f57fff640 1 -- 172.21.15.33:0/2950669307 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f600a6e60 con 0x55acffe1e240 2024-04-24T23:24:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- 172.21.15.33:0/2950669307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 msgr2=0x55acffe27cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe27cf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000eb90 tx=0x7f0f60009a20 comp rx=0 tx=0).stop 2024-04-24T23:24:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- 172.21.15.33:0/2950669307 shutdown_connections 2024-04-24T23:24:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe23da0 0x55acffe1c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1ec30 0x55acffe2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2950669307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe1e240 0x55acffe27cf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- 172.21.15.33:0/2950669307 >> 172.21.15.33:0/2950669307 conn(0x55acffe198b0 msgr2=0x55acffe20090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- 172.21.15.33:0/2950669307 shutdown_connections 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- 172.21.15.33:0/2950669307 wait complete. 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- start start 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 Processor -- start 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe2d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701a6740 con 0x55acffe1ec30 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701a7bb0 con 0x55acffe23da0 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.434+0000 7f0f7761d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701a6e10 con 0x55acffe1e240 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f7761d740 10 client.0 _ll_register_callbacks cb 0x7f0f600093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe2d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59650/0 (socket says 172.21.15.33:59650) 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe2d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 -- 172.21.15.33:0/2715585733 learned_addr learned my addr 172.21.15.33:0/2715585733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6ed9d640 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 -- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 msgr2=0x55acffe2d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe2d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 -- 172.21.15.33:0/2715585733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 msgr2=0x55acffe2e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 -- 172.21.15.33:0/2715585733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acffe2efe0 con 0x55acffe1ec30 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f6dd9b640 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe2d9e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0f7006a620 tx=0x7f0f58012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c1143a972a2060 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f55ffb640 1 -- 172.21.15.33:0/2715585733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58023060 con 0x55acffe1ec30 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f600a4e50 con 0x55acffe1ec30 2024-04-24T23:24:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f55ffb640 1 -- 172.21.15.33:0/2715585733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0f58004250 con 0x55acffe1ec30 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f55ffb640 1 -- 172.21.15.33:0/2715585733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f5802a050 con 0x55acffe1ec30 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f7761d740 10 client.18024 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.435+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acffea3500 con 0x55acffe1ec30 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f55ffb640 1 -- 172.21.15.33:0/2715585733 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f600a4e50 con 0x55acffe1ec30 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f55ffb640 1 -- 172.21.15.33:0/2715585733 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0f58003aa0 con 0x55acffe1ec30 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f7761d740 10 client.18024 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f7761d740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f7761d740 20 client.18024 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffea3f10 0x55acffea6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f7761d740 4 client.18024 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acffd89f40 con 0x55acffea3f10 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffea3f10 0x55acffea6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f55ffb640 10 client.18024 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.436+0000 7f0f6e59c640 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffea3f10 0x55acffea6330 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f3a62d3a7ad011 server_cookie=496ce16af0d43bd8 in_seq=0 out_seq=0 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f55ffb640 1 -- 172.21.15.33:0/2715585733 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55acffd89f40 con 0x55acffea3f10 2024-04-24T23:24:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f55ffb640 10 client.18024 handle_command_reply: tid=0 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 client.18024 shutdown 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 msgr2=0x55acffe2d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe2d9e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0f7006a620 tx=0x7f0f58012ca0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 shutdown_connections 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55acffea3f10 0x55acffea6330 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55acffe23da0 0x55acffe2e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acffe1ec30 0x55acffe2d9e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 --2- 172.21.15.33:0/2715585733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acffe1e240 0x55acffe2d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 >> 172.21.15.33:0/2715585733 conn(0x55acffe198b0 msgr2=0x55acffe1c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f55ffb640 0 client.18024 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 shutdown_connections 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 1 -- 172.21.15.33:0/2715585733 wait complete. 2024-04-24T23:24:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f7761d740 20 client.18024 trim_cache size 0 max 16384 2024-04-24T23:24:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:24:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:24:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:24:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0f5c0a2ce0 msgr2=0x7f0f5c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f74838640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0f5c0a2ce0 0x7f0f5c0a51a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0f68004b60 tx=0x7f0f68005b50 comp rx=0 tx=0).stop 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 msgr2=0x7f0f7006a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.437+0000 7f0f74838640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 0x7f0f7006a8b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000ebc0 tx=0x7f0f6000c6a0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 shutdown_connections 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0f5c0a2ce0 0x7f0f5c0a51a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f70102250 0x7f0f7019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f70101890 0x7f0f7019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 --2- 172.21.15.33:0/2368677546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f701006a0 0x7f0f7006a8b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 >> 172.21.15.33:0/2368677546 conn(0x7f0f700fbc30 msgr2=0x7f0f70104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 shutdown_connections 2024-04-24T23:24:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.438+0000 7f0f74838640 1 -- 172.21.15.33:0/2368677546 wait complete. 2024-04-24T23:24:33.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:33 smithi033 ceph-mon[29201]: pgmap v727: 129 pgs: 129 active+clean; 288 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-04-24T23:24:33.451 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:24:33.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:33.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1479646 12.39 MB/sec execute 518 sec latency 15.519 ms 2024-04-24T23:24:33.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 -- 172.21.15.33:0/3136213827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8101870 msgr2=0x7fe7d8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 --2- 172.21.15.33:0/3136213827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8101870 0x7fe7d8101cf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009900 tx=0x7fe7c802a410 comp rx=0 tx=0).stop 2024-04-24T23:24:33.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 -- 172.21.15.33:0/3136213827 shutdown_connections 2024-04-24T23:24:33.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 --2- 172.21.15.33:0/3136213827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d8102230 0x7fe7d8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 --2- 172.21.15.33:0/3136213827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8101870 0x7fe7d8101cf0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 --2- 172.21.15.33:0/3136213827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7d8100680 0x7fe7d8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 -- 172.21.15.33:0/3136213827 >> 172.21.15.33:0/3136213827 conn(0x7fe7d80fbbf0 msgr2=0x7fe7d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 -- 172.21.15.33:0/3136213827 shutdown_connections 2024-04-24T23:24:33.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.566+0000 7fe7e0409640 1 -- 172.21.15.33:0/3136213827 wait complete. 2024-04-24T23:24:33.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 Processor -- start 2024-04-24T23:24:33.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 -- start start 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7d8100680 0x7fe7d8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d8101870 0x7fe7d8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 0x7fe7d819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d810c300 con 0x7fe7d8102230 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d810c180 con 0x7fe7d8100680 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7e0409640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d810c480 con 0x7fe7d8101870 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7de9a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 0x7fe7d819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7de9a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 0x7fe7d819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59654/0 (socket says 172.21.15.33:59654) 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7de9a6640 1 -- 172.21.15.33:0/2004924980 learned_addr learned my addr 172.21.15.33:0/2004924980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7dd9a4640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d8101870 0x7fe7d8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.567+0000 7fe7de1a5640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7d8100680 0x7fe7d8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7de9a6640 1 -- 172.21.15.33:0/2004924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d8101870 msgr2=0x7fe7d8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7de9a6640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d8101870 0x7fe7d8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7de9a6640 1 -- 172.21.15.33:0/2004924980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7d8100680 msgr2=0x7fe7d8197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7de9a6640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7d8100680 0x7fe7d8197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7de9a6640 1 -- 172.21.15.33:0/2004924980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d819f6c0 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7de9a6640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 0x7fe7d819f050 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000eb10 tx=0x7fe7d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0004b20 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d819f9b0 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d0004cc0 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0010760 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.568+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d819ff20 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.569+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d81a0150 con 0x7fe7d8102230 2024-04-24T23:24:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.571+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe7d0005020 con 0x7fe7d8102230 2024-04-24T23:24:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.571+0000 7fe7cf7fe640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.572+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe7d00cfad0 con 0x7fe7d8102230 2024-04-24T23:24:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.572+0000 7fe7de1a5640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.572+0000 7fe7de1a5640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40095f0 tx=0x7fe7d4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.577+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7d009c640 con 0x7fe7d8102230 2024-04-24T23:24:33.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.846+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d81a0320 con 0x7fe7d8102230 2024-04-24T23:24:33.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.847+0000 7fe7cf7fe640 1 -- 172.21.15.33:0/2004924980 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fe7d009bf40 con 0x7fe7d8102230 2024-04-24T23:24:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:24:33.848 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:24:33.849 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe7c40a2c10 msgr2=0x7fe7c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40095f0 tx=0x7fe7d4005f90 comp rx=0 tx=0).stop 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 msgr2=0x7fe7d819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 0x7fe7d819f050 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000eb10 tx=0x7fe7d000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 shutdown_connections 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe7c40a2c10 0x7fe7c40a50d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d8102230 0x7fe7d819f050 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7d8101870 0x7fe7d8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 --2- 172.21.15.33:0/2004924980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7d8100680 0x7fe7d8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.849+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 >> 172.21.15.33:0/2004924980 conn(0x7fe7d80fbbf0 msgr2=0x7fe7d80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.850+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 shutdown_connections 2024-04-24T23:24:33.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:33.850+0000 7fe7e0409640 1 -- 172.21.15.33:0/2004924980 wait complete. 2024-04-24T23:24:33.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:24:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 -- 172.21.15.33:0/2958365915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 msgr2=0x7f35ac108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 --2- 172.21.15.33:0/2958365915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 0x7f35ac108ae0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f359c009ca0 tx=0x7f359c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 -- 172.21.15.33:0/2958365915 shutdown_connections 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 --2- 172.21.15.33:0/2958365915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 0x7f35ac108ae0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 --2- 172.21.15.33:0/2958365915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 0x7f35ac101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 --2- 172.21.15.33:0/2958365915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 -- 172.21.15.33:0/2958365915 >> 172.21.15.33:0/2958365915 conn(0x7f35ac0fbc10 msgr2=0x7f35ac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 -- 172.21.15.33:0/2958365915 shutdown_connections 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.018+0000 7f35b46dd640 1 -- 172.21.15.33:0/2958365915 wait complete. 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 Processor -- start 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 -- start start 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 0x7f35ac197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 0x7f35ac19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c320 con 0x7f35ac102250 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c1a0 con 0x7f35ac101890 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b46dd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c4a0 con 0x7f35ac1006a0 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b2479640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b2c7a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 0x7f35ac19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b1c78640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 0x7f35ac197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b2479640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:48752/0 (socket says 172.21.15.33:48752) 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.019+0000 7f35b2479640 1 -- 172.21.15.33:0/4141281816 learned_addr learned my addr 172.21.15.33:0/4141281816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b1c78640 1 -- 172.21.15.33:0/4141281816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 msgr2=0x7f35ac197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b1c78640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b1c78640 1 -- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 msgr2=0x7f35ac19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b1c78640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 0x7f35ac19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b1c78640 1 -- 172.21.15.33:0/4141281816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac19f6d0 con 0x7f35ac101890 2024-04-24T23:24:34.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b2479640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b1c78640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 0x7f35ac197ff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9f0 tx=0x7f35a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35ab7fe640 1 -- 172.21.15.33:0/4141281816 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a000ccc0 con 0x7f35ac101890 2024-04-24T23:24:34.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac19f9c0 con 0x7f35ac101890 2024-04-24T23:24:34.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35ab7fe640 1 -- 172.21.15.33:0/4141281816 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a0010910 con 0x7f35ac101890 2024-04-24T23:24:34.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35ab7fe640 1 -- 172.21.15.33:0/4141281816 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0018960 con 0x7f35ac101890 2024-04-24T23:24:34.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.020+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac19ff60 con 0x7f35ac101890 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 Processor -- start 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 -- start start 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff8ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff8d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35ab7fe640 1 -- 172.21.15.33:0/4141281816 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f35a0010ac0 con 0x7f35ac101890 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55831fee5670 con 0x55831ff8b1a0 2024-04-24T23:24:34.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55831fe8d970 con 0x55831ff85640 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.023+0000 7f35b73ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55831fee1630 con 0x55831ff86030 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b1c78640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff8d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff85a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55880/0 (socket says 172.21.15.33:55880) 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 -- 172.21.15.33:0/126439212 learned_addr learned my addr 172.21.15.33:0/126439212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2479640 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff8ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 -- 172.21.15.33:0/126439212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 msgr2=0x55831ff8ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff8ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 -- 172.21.15.33:0/126439212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 msgr2=0x55831ff8d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff8d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 -- 172.21.15.33:0/126439212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55831feb1400 con 0x55831ff85640 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2c7a640 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff85a60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f359c009ca0 tx=0x7f359c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59dec68279fb8c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35ab7fe640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f35940a2b90 0x7f35940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35a8ff9640 1 -- 172.21.15.33:0/126439212 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0098c0 con 0x55831ff85640 2024-04-24T23:24:34.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35b2479640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f35940a2b90 0x7f35940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.024+0000 7f35a8ff9640 1 -- 172.21.15.33:0/126439212 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c036040 con 0x55831ff85640 2024-04-24T23:24:34.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35a8ff9640 1 -- 172.21.15.33:0/126439212 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c004290 con 0x55831ff85640 2024-04-24T23:24:34.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35ab7fe640 1 -- 172.21.15.33:0/4141281816 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35a00cf7c0 con 0x7f35ac101890 2024-04-24T23:24:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 -- 172.21.15.33:0/126439212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 msgr2=0x55831ff85a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b2479640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f35940a2b90 0x7f35940a5050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55831ff86430 tx=0x7f35a4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff85a60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f359c009ca0 tx=0x7f359c004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 -- 172.21.15.33:0/126439212 shutdown_connections 2024-04-24T23:24:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff8d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff8ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 --2- 172.21.15.33:0/126439212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff85a60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 -- 172.21.15.33:0/126439212 >> 172.21.15.33:0/126439212 conn(0x55831ff7c8a0 msgr2=0x55831ff7cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 -- 172.21.15.33:0/126439212 shutdown_connections 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.025+0000 7f35b73ea740 1 -- 172.21.15.33:0/126439212 wait complete. 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 -- start start 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 Processor -- start 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 0x55831ff8fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff8b1a0 0x55831ff96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55831ff8df10 con 0x55831ff86030 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55831fea1050 con 0x55831ff8b1a0 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55831ff8e090 con 0x55831ff85640 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b73ea740 10 client.0 _ll_register_callbacks cb 0x55831fbf8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 0x55831ff8fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 0x55831ff8fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59684/0 (socket says 172.21.15.33:59684) 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 -- 172.21.15.33:0/2169249399 learned_addr learned my addr 172.21.15.33:0/2169249399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b2479640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b2c7a640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff8b1a0 0x55831ff96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 -- 172.21.15.33:0/2169249399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 msgr2=0x55831ff8f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff8f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 -- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff8b1a0 msgr2=0x55831ff96b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff8b1a0 0x55831ff96b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.026+0000 7f35b1c78640 1 -- 172.21.15.33:0/2169249399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55831fe72dc0 con 0x55831ff86030 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b1c78640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 0x55831ff8fac0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f35a00a4370 tx=0x7f35a0097f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8de302bc2e38d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f3591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f3591ffb640 1 -- 172.21.15.33:0/2169249399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a009b640 con 0x55831ff86030 2024-04-24T23:24:34.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f3591ffb640 1 -- 172.21.15.33:0/2169249399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35a00cfdb0 con 0x55831ff86030 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f359c036040 con 0x55831ff86030 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f3591ffb640 1 -- 172.21.15.33:0/2169249399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35a009bce0 con 0x55831ff86030 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 10 client.18042 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55831fbf93f0 con 0x55831ff86030 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f3591ffb640 1 -- 172.21.15.33:0/2169249399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c036040 con 0x55831ff86030 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f3591ffb640 1 -- 172.21.15.33:0/2169249399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f35a0018b00 con 0x55831ff86030 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 10 client.18042 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 10 client.18042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 20 client.18042 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55831ff9b3f0 0x55831ff9d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.027+0000 7f35b73ea740 4 client.18042 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.028+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55831fef2bb0 con 0x55831ff9b3f0 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.028+0000 7f35b2479640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55831ff9b3f0 0x55831ff9d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.028+0000 7f3591ffb640 10 client.18042 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:34.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.028+0000 7f35b2479640 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55831ff9b3f0 0x55831ff9d830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b5b4caee0a277c5 server_cookie=b21735cbdb3fb4d6 in_seq=0 out_seq=0 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f3591ffb640 1 -- 172.21.15.33:0/2169249399 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55831fef2bb0 con 0x55831ff9b3f0 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f3591ffb640 10 client.18042 handle_command_reply: tid=0 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 client.18042 shutdown 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 msgr2=0x55831ff8fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 0x55831ff8fac0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f35a00a4370 tx=0x7f35a0097f80 comp rx=0 tx=0).stop 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 shutdown_connections 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55831ff9b3f0 0x55831ff9d830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff8b1a0 0x55831ff96b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff86030 0x55831ff8fac0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 --2- 172.21.15.33:0/2169249399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff8f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f3591ffb640 0 client.18042 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:34.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 >> 172.21.15.33:0/2169249399 conn(0x55831ff7c8a0 msgr2=0x55831ff7bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 shutdown_connections 2024-04-24T23:24:34.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 1 -- 172.21.15.33:0/2169249399 wait complete. 2024-04-24T23:24:34.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.031+0000 7f35b73ea740 20 client.18042 trim_cache size 0 max 16384 2024-04-24T23:24:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 Processor -- start 2024-04-24T23:24:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 -- start start 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff8eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff96ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1aa9b0 con 0x55831ff8b1a0 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1a8280 con 0x55831ff85640 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b73ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1a60d0 con 0x55831ff86030 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b2c7a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b1c78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff96ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b2479640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff8eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b2479640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff8eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55908/0 (socket says 172.21.15.33:55908) 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b1c78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff96ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:48786/0 (socket says 172.21.15.33:48786) 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.047+0000 7f35b2479640 1 -- 172.21.15.33:0/3216294696 learned_addr learned my addr 172.21.15.33:0/3216294696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2479640 1 -- 172.21.15.33:0/3216294696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 msgr2=0x55831ff96ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2479640 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff96ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2479640 1 -- 172.21.15.33:0/3216294696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 msgr2=0x55831ff7f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2479640 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff7f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2479640 1 -- 172.21.15.33:0/3216294696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55831fcc37d0 con 0x55831ff85640 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2c7a640 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff7f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f35b2479640 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff8eba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009600 tx=0x7f35a4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d8198c258a3163c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f3593fff640 1 -- 172.21.15.33:0/3216294696 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a402c530 con 0x55831ff85640 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f3593fff640 1 -- 172.21.15.33:0/3216294696 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a4032070 con 0x55831ff85640 2024-04-24T23:24:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.048+0000 7f3593fff640 1 -- 172.21.15.33:0/3216294696 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40038e0 con 0x55831ff85640 2024-04-24T23:24:34.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 -- 172.21.15.33:0/3216294696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 msgr2=0x55831ff8eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff8eba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009600 tx=0x7f35a4017000 comp rx=0 tx=0).stop 2024-04-24T23:24:34.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 -- 172.21.15.33:0/3216294696 shutdown_connections 2024-04-24T23:24:34.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff7f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff86030 0x55831ff96ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3216294696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff85640 0x55831ff8eba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 -- 172.21.15.33:0/3216294696 >> 172.21.15.33:0/3216294696 conn(0x55831ff7c8a0 msgr2=0x55831ff7bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 -- 172.21.15.33:0/3216294696 shutdown_connections 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 -- 172.21.15.33:0/3216294696 wait complete. 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 -- start start 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.049+0000 7f35b73ea740 1 Processor -- start 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff93ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff86030 0x55831ff94400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff95410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1a6740 con 0x55831ff8b1a0 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1a7bb0 con 0x55831ff86030 2024-04-24T23:24:34.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1a6e10 con 0x55831ff85640 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b73ea740 10 client.0 _ll_register_callbacks cb 0x7f35a00d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2c7a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff95410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff93ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2c7a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff95410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59702/0 (socket says 172.21.15.33:59702) 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2c7a640 1 -- 172.21.15.33:0/3674723800 learned_addr learned my addr 172.21.15.33:0/3674723800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b1c78640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff86030 0x55831ff94400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 -- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff86030 msgr2=0x55831ff94400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff86030 0x55831ff94400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 -- 172.21.15.33:0/3674723800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 msgr2=0x55831ff95410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff95410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 -- 172.21.15.33:0/3674723800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55831ff95a00 con 0x55831ff85640 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b1c78640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff86030 0x55831ff94400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:34.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2c7a640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff95410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f35b2479640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff93ec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35a402cab0 tx=0x7f35a402c600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106b09d484f8e76b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f3591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.050+0000 7f3591ffb640 1 -- 172.21.15.33:0/3674723800 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4005050 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f3591ffb640 1 -- 172.21.15.33:0/3674723800 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35a40051f0 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a4032070 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f3591ffb640 1 -- 172.21.15.33:0/3674723800 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35a40053f0 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 10 client.27517 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55831ff96d70 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f3591ffb640 1 -- 172.21.15.33:0/3674723800 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a4032070 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f3591ffb640 1 -- 172.21.15.33:0/3674723800 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f35a4004a30 con 0x55831ff85640 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 10 client.27517 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 10 client.27517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 20 client.27517 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:34.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5583200020b0 0x5583200044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 4 client.27517 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55831ff8f370 con 0x5583200020b0 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.051+0000 7f35b1c78640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5583200020b0 0x5583200044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b1c78640 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5583200020b0 0x5583200044d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdbd4f27b83d2961 server_cookie=32f2846a53dd7dd8 in_seq=0 out_seq=0 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f3591ffb640 10 client.27517 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f3591ffb640 1 -- 172.21.15.33:0/3674723800 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55831ff8f370 con 0x5583200020b0 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f3591ffb640 10 client.27517 handle_command_reply: tid=0 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 client.27517 shutdown 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 msgr2=0x55831ff93ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff93ec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35a402cab0 tx=0x7f35a402c600 comp rx=0 tx=0).stop 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 shutdown_connections 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5583200020b0 0x5583200044d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55831ff8b1a0 0x55831ff95410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55831ff86030 0x55831ff94400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 --2- 172.21.15.33:0/3674723800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55831ff85640 0x55831ff93ec0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f3591ffb640 0 client.27517 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 >> 172.21.15.33:0/3674723800 conn(0x55831ff7c8a0 msgr2=0x55831ff86a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 shutdown_connections 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 1 -- 172.21.15.33:0/3674723800 wait complete. 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.052+0000 7f35b73ea740 20 client.27517 trim_cache size 0 max 16384 2024-04-24T23:24:34.060 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f35940a2b90 msgr2=0x7f35940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f35940a2b90 0x7f35940a5050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55831ff86430 tx=0x7f35a4005f90 comp rx=0 tx=0).stop 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 msgr2=0x7f35ac197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 0x7f35ac197ff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9f0 tx=0x7f35a000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 shutdown_connections 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f35940a2b90 0x7f35940a5050 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac102250 0x7f35ac19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101890 0x7f35ac197ff0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 --2- 172.21.15.33:0/4141281816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ac1006a0 0x7f35ac197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 >> 172.21.15.33:0/4141281816 conn(0x7f35ac0fbc10 msgr2=0x7f35ac0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 shutdown_connections 2024-04-24T23:24:34.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.053+0000 7f35b46dd640 1 -- 172.21.15.33:0/4141281816 wait complete. 2024-04-24T23:24:34.065 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:24:34.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 -- 172.21.15.33:0/495953179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 msgr2=0x7f4554100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 --2- 172.21.15.33:0/495953179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554100aa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4550009900 tx=0x7f455002a410 comp rx=0 tx=0).stop 2024-04-24T23:24:34.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 -- 172.21.15.33:0/495953179 shutdown_connections 2024-04-24T23:24:34.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 --2- 172.21.15.33:0/495953179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4554102230 0x7f4554108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 --2- 172.21.15.33:0/495953179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4554101870 0x7f4554101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 --2- 172.21.15.33:0/495953179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554100aa0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 -- 172.21.15.33:0/495953179 >> 172.21.15.33:0/495953179 conn(0x7f45540fbbf0 msgr2=0x7f45540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 -- 172.21.15.33:0/495953179 shutdown_connections 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 -- 172.21.15.33:0/495953179 wait complete. 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 Processor -- start 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.181+0000 7f455b9b0640 1 -- start start 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4554101870 0x7f455419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4554102230 0x7f455419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f455410c300 con 0x7f4554100680 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f455410c180 con 0x7f4554102230 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f455410c480 con 0x7f4554101870 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59726/0 (socket says 172.21.15.33:59726) 2024-04-24T23:24:34.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 -- 172.21.15.33:0/291282475 learned_addr learned my addr 172.21.15.33:0/291282475 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f4558f4b640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4554101870 0x7f455419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f4559f4d640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4554102230 0x7f455419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 -- 172.21.15.33:0/291282475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4554101870 msgr2=0x7f455419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4554101870 0x7f455419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 -- 172.21.15.33:0/291282475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4554102230 msgr2=0x7f455419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4554102230 0x7f455419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 -- 172.21.15.33:0/291282475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419f780 con 0x7f4554100680 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455974c640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554069440 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4550031610 tx=0x7f4550004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f454a7fc640 1 -- 172.21.15.33:0/291282475 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455003f030 con 0x7f4554100680 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f454a7fc640 1 -- 172.21.15.33:0/291282475 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4550003ad0 con 0x7f4554100680 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419f9b0 con 0x7f4554100680 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.182+0000 7f454a7fc640 1 -- 172.21.15.33:0/291282475 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550035070 con 0x7f4554100680 2024-04-24T23:24:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.183+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455419fef0 con 0x7f4554100680 2024-04-24T23:24:34.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.185+0000 7f454a7fc640 1 -- 172.21.15.33:0/291282475 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4550042030 con 0x7f4554100680 2024-04-24T23:24:34.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.185+0000 7f455e79f740 1 Processor -- start 2024-04-24T23:24:34.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.185+0000 7f455e79f740 1 -- start start 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455e79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455e79f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af5f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455e79f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5f4d90 0x5620af5f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455e79f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620af553670 con 0x5620af5f4d90 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455e79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5620af4fb970 con 0x5620af5ef230 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455e79f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5620af54f630 con 0x5620af5efc20 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f454a7fc640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45400a2c10 0x7f45400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f455974c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af5f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5ef650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55954/0 (socket says 172.21.15.33:55954) 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 -- 172.21.15.33:0/1577733780 learned_addr learned my addr 172.21.15.33:0/1577733780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4558f4b640 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5f4d90 0x5620af5f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45400a2c10 0x7f45400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f454a7fc640 1 -- 172.21.15.33:0/291282475 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f45500ef960 con 0x7f4554100680 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 -- 172.21.15.33:0/1577733780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 msgr2=0x5620af5f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af5f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 -- 172.21.15.33:0/1577733780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5f4d90 msgr2=0x5620af5f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5f4d90 0x5620af5f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4559f4d640 1 -- 172.21.15.33:0/1577733780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620af51f400 con 0x5620af5ef230 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.186+0000 7f4558f4b640 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5f4d90 0x5620af5f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.187+0000 7f4559f4d640 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5ef650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f454c011e00 tx=0x7f454c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63606d0a11ad5112 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.187+0000 7f453f7fe640 1 -- 172.21.15.33:0/1577733780 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c022070 con 0x5620af5ef230 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.187+0000 7f4559f4d640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45400a2c10 0x7f45400a50d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f454c00fc90 tx=0x7f454c018000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.187+0000 7f453f7fe640 1 -- 172.21.15.33:0/1577733780 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c003e50 con 0x5620af5ef230 2024-04-24T23:24:34.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.187+0000 7f453f7fe640 1 -- 172.21.15.33:0/1577733780 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0046b0 con 0x5620af5ef230 2024-04-24T23:24:34.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 -- 172.21.15.33:0/1577733780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 msgr2=0x5620af5ef650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5ef650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f454c011e00 tx=0x7f454c00f790 comp rx=0 tx=0).stop 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 -- 172.21.15.33:0/1577733780 shutdown_connections 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5f4d90 0x5620af5f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af5f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 --2- 172.21.15.33:0/1577733780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5ef650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 -- 172.21.15.33:0/1577733780 >> 172.21.15.33:0/1577733780 conn(0x5620af5ea8a0 msgr2=0x5620af5eaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 -- 172.21.15.33:0/1577733780 shutdown_connections 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 -- 172.21.15.33:0/1577733780 wait complete. 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 -- start start 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.188+0000 7f455e79f740 1 Processor -- start 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 0x5620af5f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af600920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620af5f7ce0 con 0x5620af5efc20 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5620af50f050 con 0x5620af5ef230 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5620af5f7e60 con 0x5620af5f4d90 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f4558f4b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 0x5620af5f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455e79f740 10 client.0 _ll_register_callbacks cb 0x5620af266be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f4558f4b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 0x5620af5f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59744/0 (socket says 172.21.15.33:59744) 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f4558f4b640 1 -- 172.21.15.33:0/3428271157 learned_addr learned my addr 172.21.15.33:0/3428271157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f4559f4d640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af600920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 -- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 msgr2=0x5620af600920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af600920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 -- 172.21.15.33:0/3428271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 msgr2=0x5620af5f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 0x5620af5f98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 -- 172.21.15.33:0/3428271157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620af4e0dc0 con 0x5620af5ef230 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f4558f4b640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 0x5620af5f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f455974c640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5f93a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45500b82e0 tx=0x7f45500bb9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0e0ac35f91877e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.189+0000 7f453cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f453cff9640 1 -- 172.21.15.33:0/3428271157 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500da050 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c003e50 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f453cff9640 1 -- 172.21.15.33:0/3428271157 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4550003c80 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f453cff9640 1 -- 172.21.15.33:0/3428271157 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4550049040 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 10 client.27578 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620af2673f0 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f453cff9640 1 -- 172.21.15.33:0/3428271157 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c003e50 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f453cff9640 1 -- 172.21.15.33:0/3428271157 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f45500d1030 con 0x5620af5ef230 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 10 client.27578 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 10 client.27578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 20 client.27578 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af605100 0x5620af607550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 4 client.27578 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620af560bb0 con 0x5620af605100 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.190+0000 7f4558f4b640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af605100 0x5620af607550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.191+0000 7f4558f4b640 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af605100 0x5620af607550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2d4bd9d6e89e130 server_cookie=26c17a64b58f799a in_seq=0 out_seq=0 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.191+0000 7f453cff9640 10 client.27578 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.191+0000 7f453cff9640 1 -- 172.21.15.33:0/3428271157 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620af560bb0 con 0x5620af605100 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.191+0000 7f453cff9640 10 client.27578 handle_command_reply: tid=0 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.191+0000 7f455e79f740 1 client.27578 shutdown 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 msgr2=0x5620af5f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5f93a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45500b82e0 tx=0x7f45500bb9c0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 shutdown_connections 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af605100 0x5620af607550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af600920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5efc20 0x5620af5f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 --2- 172.21.15.33:0/3428271157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5ef230 0x5620af5f93a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f453cff9640 0 client.27578 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 >> 172.21.15.33:0/3428271157 conn(0x5620af5ea8a0 msgr2=0x5620af5e9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 shutdown_connections 2024-04-24T23:24:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 1 -- 172.21.15.33:0/3428271157 wait complete. 2024-04-24T23:24:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.192+0000 7f455e79f740 20 client.27578 trim_cache size 0 max 16384 2024-04-24T23:24:34.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:34 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2004924980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:24:34.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 Processor -- start 2024-04-24T23:24:34.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 -- start start 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af6008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5f4d90 0x5620af5e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45541accd0 con 0x5620af5ef230 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45541aa5a0 con 0x5620af5f4d90 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455e79f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a83f0 con 0x5620af5efc20 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455974c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f4559f4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5f4d90 0x5620af5e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455974c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5f9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59748/0 (socket says 172.21.15.33:59748) 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.206+0000 7f455974c640 1 -- 172.21.15.33:0/1386659614 learned_addr learned my addr 172.21.15.33:0/1386659614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f4558f4b640 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af6008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f455974c640 1 -- 172.21.15.33:0/1386659614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 msgr2=0x5620af6008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f455974c640 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af6008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f455974c640 1 -- 172.21.15.33:0/1386659614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5f4d90 msgr2=0x5620af5e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f455974c640 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5f4d90 0x5620af5e99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f455974c640 1 -- 172.21.15.33:0/1386659614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620af3317d0 con 0x5620af5ef230 2024-04-24T23:24:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f455974c640 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5f9840 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4550031440 tx=0x7f45500bbaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600bddf3d6036da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f453effd640 1 -- 172.21.15.33:0/1386659614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500da070 con 0x5620af5ef230 2024-04-24T23:24:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f453effd640 1 -- 172.21.15.33:0/1386659614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4550003760 con 0x5620af5ef230 2024-04-24T23:24:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.207+0000 7f453effd640 1 -- 172.21.15.33:0/1386659614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500c45b0 con 0x5620af5ef230 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 -- 172.21.15.33:0/1386659614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 msgr2=0x5620af5f9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5f9840 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4550031440 tx=0x7f45500bbaf0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 -- 172.21.15.33:0/1386659614 shutdown_connections 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5f4d90 0x5620af5e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5efc20 0x5620af6008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 --2- 172.21.15.33:0/1386659614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5f9840 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 -- 172.21.15.33:0/1386659614 >> 172.21.15.33:0/1386659614 conn(0x5620af5ea8a0 msgr2=0x5620af5e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 -- 172.21.15.33:0/1386659614 shutdown_connections 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 -- 172.21.15.33:0/1386659614 wait complete. 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 -- start start 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.208+0000 7f455e79f740 1 Processor -- start 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5efc20 0x5620af5fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af5ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a8a60 con 0x5620af5ef230 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a9ed0 con 0x5620af5efc20 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a9130 con 0x5620af5f4d90 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 10 client.0 _ll_register_callbacks cb 0x7f45500b7a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5fdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59766/0 (socket says 172.21.15.33:59766) 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 -- 172.21.15.33:0/2119351182 learned_addr learned my addr 172.21.15.33:0/2119351182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f4559f4d640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af5ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f4558f4b640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5efc20 0x5620af5fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 -- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 msgr2=0x5620af5ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af5ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 -- 172.21.15.33:0/2119351182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5efc20 msgr2=0x5620af5fe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5efc20 0x5620af5fe1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 -- 172.21.15.33:0/2119351182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620af5ff790 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455974c640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5fdca0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4550005c40 tx=0x7f455003b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1514a17626966562 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f453cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f453cff9640 1 -- 172.21.15.33:0/2119351182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455009e070 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f453cff9640 1 -- 172.21.15.33:0/2119351182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f45500c45b0 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5620af55af00 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.209+0000 7f453cff9640 1 -- 172.21.15.33:0/2119351182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45500da040 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 10 client.18066 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620af5ff790 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f453cff9640 1 -- 172.21.15.33:0/2119351182 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5620af55af00 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f453cff9640 1 -- 172.21.15.33:0/2119351182 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f45500a1020 con 0x5620af5ef230 2024-04-24T23:24:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 10 client.18066 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 10 client.18066 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 20 client.18066 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af66b8b0 0x5620af66dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 4 client.18066 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620af4d8820 con 0x5620af66b8b0 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.210+0000 7f4558f4b640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af66b8b0 0x5620af66dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f453cff9640 10 client.18066 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f4558f4b640 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af66b8b0 0x5620af66dca0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c92a37174068107b server_cookie=34d20cb04136b600 in_seq=0 out_seq=0 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f453cff9640 1 -- 172.21.15.33:0/2119351182 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620af4d8820 con 0x5620af66b8b0 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f453cff9640 10 client.18066 handle_command_reply: tid=0 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 client.18066 shutdown 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 msgr2=0x5620af5fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5fdca0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4550005c40 tx=0x7f455003b700 comp rx=0 tx=0).stop 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 shutdown_connections 2024-04-24T23:24:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5620af66b8b0 0x5620af66dca0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620af5f4d90 0x5620af5ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f453cff9640 0 client.18066 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620af5efc20 0x5620af5fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 --2- 172.21.15.33:0/2119351182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620af5ef230 0x5620af5fdca0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 >> 172.21.15.33:0/2119351182 conn(0x5620af5ea8a0 msgr2=0x5620af5f9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 shutdown_connections 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 1 -- 172.21.15.33:0/2119351182 wait complete. 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.211+0000 7f455e79f740 20 client.18066 trim_cache size 0 max 16384 2024-04-24T23:24:34.218 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:24:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45400a2c10 msgr2=0x7f45400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45400a2c10 0x7f45400a50d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f454c00fc90 tx=0x7f454c018000 comp rx=0 tx=0).stop 2024-04-24T23:24:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 msgr2=0x7f4554069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554069440 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4550031610 tx=0x7f4550004270 comp rx=0 tx=0).stop 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 shutdown_connections 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45400a2c10 0x7f45400a50d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4554102230 0x7f455419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4554101870 0x7f455419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 --2- 172.21.15.33:0/291282475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4554100680 0x7f4554069440 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 >> 172.21.15.33:0/291282475 conn(0x7f45540fbbf0 msgr2=0x7f4554104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 shutdown_connections 2024-04-24T23:24:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.212+0000 7f455b9b0640 1 -- 172.21.15.33:0/291282475 wait complete. 2024-04-24T23:24:34.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.338+0000 7f431a06b640 1 -- 172.21.15.33:0/4209006487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314102230 msgr2=0x7f4314108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.338+0000 7f431a06b640 1 --2- 172.21.15.33:0/4209006487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314102230 0x7f4314108ac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f430c009be0 tx=0x7f430c02a6c0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.338+0000 7f431a06b640 1 -- 172.21.15.33:0/4209006487 shutdown_connections 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 --2- 172.21.15.33:0/4209006487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314102230 0x7f4314108ac0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 --2- 172.21.15.33:0/4209006487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314101870 0x7f4314101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 --2- 172.21.15.33:0/4209006487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314100680 0x7f4314100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 -- 172.21.15.33:0/4209006487 >> 172.21.15.33:0/4209006487 conn(0x7f43140fbbf0 msgr2=0x7f43140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 -- 172.21.15.33:0/4209006487 shutdown_connections 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 -- 172.21.15.33:0/4209006487 wait complete. 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 Processor -- start 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 -- start start 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.339+0000 7f431a06b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 0x7f4314197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f431a06b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314101870 0x7f4314197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f431a06b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314102230 0x7f431419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f431a06b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f431410c2e0 con 0x7f4314100680 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f431a06b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f431410c160 con 0x7f4314101870 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f431a06b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f431410c460 con 0x7f4314102230 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 0x7f4314197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 0x7f4314197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59782/0 (socket says 172.21.15.33:59782) 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 -- 172.21.15.33:0/3523880604 learned_addr learned my addr 172.21.15.33:0/3523880604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314101870 0x7f4314197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f4313fff640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314102230 0x7f431419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 -- 172.21.15.33:0/3523880604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314102230 msgr2=0x7f431419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314102230 0x7f431419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 -- 172.21.15.33:0/3523880604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314101870 msgr2=0x7f4314197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314101870 0x7f4314197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 -- 172.21.15.33:0/3523880604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419f6a0 con 0x7f4314100680 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f4313fff640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314102230 0x7f431419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f43137fe640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 0x7f4314197a80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9c0 tx=0x7f430400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f4310ff9640 1 -- 172.21.15.33:0/3523880604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400ccc0 con 0x7f4314100680 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4314102bc0 con 0x7f4314100680 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f4310ff9640 1 -- 172.21.15.33:0/3523880604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43040035f0 con 0x7f4314100680 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.340+0000 7f4310ff9640 1 -- 172.21.15.33:0/3523880604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400ce60 con 0x7f4314100680 2024-04-24T23:24:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.341+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4314103160 con 0x7f4314100680 2024-04-24T23:24:34.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f4310ff9640 1 -- 172.21.15.33:0/3523880604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f43040037a0 con 0x7f4314100680 2024-04-24T23:24:34.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 Processor -- start 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 -- start start 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5242650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f5247720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f524a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f0f51a6670 con 0x55f0f5242230 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f0f514e970 con 0x55f0f5247d90 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.343+0000 7f431cd7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0f51a2630 con 0x55f0f5242c20 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5242650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f524a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5242650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59790/0 (socket says 172.21.15.33:59790) 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4313fff640 1 -- 172.21.15.33:0/3991338817 learned_addr learned my addr 172.21.15.33:0/3991338817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f43137fe640 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f5247720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4310ff9640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42fc0a2b90 0x7f42fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 -- 172.21.15.33:0/3991338817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 msgr2=0x55f0f5247720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f5247720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 -- 172.21.15.33:0/3991338817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 msgr2=0x55f0f5242650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5242650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 -- 172.21.15.33:0/3991338817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f5172400 con 0x55f0f5247d90 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4313fff640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42fc0a2b90 0x7f42fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4310ff9640 1 -- 172.21.15.33:0/3523880604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43040cf7a0 con 0x7f4314100680 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.344+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f524a220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4308005fd0 tx=0x7f4308009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd8f600069c7fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.345+0000 7f4313fff640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42fc0a2b90 0x7f42fc0a5050 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.345+0000 7f42f9ffb640 1 -- 172.21.15.33:0/3991338817 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308004050 con 0x55f0f5247d90 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.345+0000 7f42f9ffb640 1 -- 172.21.15.33:0/3991338817 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43080041f0 con 0x55f0f5247d90 2024-04-24T23:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.345+0000 7f42f9ffb640 1 -- 172.21.15.33:0/3991338817 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430801a660 con 0x55f0f5247d90 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 -- 172.21.15.33:0/3991338817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 msgr2=0x55f0f524a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f524a220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4308005fd0 tx=0x7f4308009260 comp rx=0 tx=0).stop 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 -- 172.21.15.33:0/3991338817 shutdown_connections 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f524a220 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f5247720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3991338817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5242650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 -- 172.21.15.33:0/3991338817 >> 172.21.15.33:0/3991338817 conn(0x55f0f523d8a0 msgr2=0x55f0f523dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 -- 172.21.15.33:0/3991338817 shutdown_connections 2024-04-24T23:24:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 -- 172.21.15.33:0/3991338817 wait complete. 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 -- start start 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.346+0000 7f431cd7a740 1 Processor -- start 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 0x55f0f524c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f524c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f5253880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f0f524ad40 con 0x55f0f5247d90 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f0f5162050 con 0x55f0f5242230 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0f524aec0 con 0x55f0f5242c20 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f431cd7a740 10 client.0 _ll_register_callbacks cb 0x55f0f4eb9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 0x55f0f524c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f4313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f5253880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 0x55f0f524c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56000/0 (socket says 172.21.15.33:56000) 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 -- 172.21.15.33:0/3215146425 learned_addr learned my addr 172.21.15.33:0/3215146425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f524c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 -- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 msgr2=0x55f0f524c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f524c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 -- 172.21.15.33:0/3215146425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 msgr2=0x55f0f5253880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f5253880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 -- 172.21.15.33:0/3215146425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f5133dc0 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f43137fe640 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 0x55f0f524c300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f43040058c0 tx=0x7f430409b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91328691d379fcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.347+0000 7f42f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f42f2ffd640 1 -- 172.21.15.33:0/3215146425 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040a0020 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f42f2ffd640 1 -- 172.21.15.33:0/3215146425 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43040979c0 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43080041f0 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f42f2ffd640 1 -- 172.21.15.33:0/3215146425 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43040a48e0 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 10 client.27596 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0f4eba3f0 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f42f2ffd640 1 -- 172.21.15.33:0/3215146425 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43080041f0 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f42f2ffd640 1 -- 172.21.15.33:0/3215146425 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f43040a1240 con 0x55f0f5242230 2024-04-24T23:24:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 10 client.27596 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 20 client.27596 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f525a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 4 client.27596 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0f51b3bb0 con 0x55f0f5258320 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.348+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f525a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.349+0000 7f4312ffd640 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f525a7b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9394fbede639bfd server_cookie=6126574533bf34d7 in_seq=0 out_seq=0 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.349+0000 7f42f2ffd640 10 client.27596 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.349+0000 7f42f2ffd640 1 -- 172.21.15.33:0/3215146425 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0f51b3bb0 con 0x55f0f5258320 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.349+0000 7f42f2ffd640 10 client.27596 handle_command_reply: tid=0 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.349+0000 7f431cd7a740 1 client.27596 shutdown 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 msgr2=0x55f0f524c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 0x55f0f524c300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f43040058c0 tx=0x7f430409b5d0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 shutdown_connections 2024-04-24T23:24:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f525a7b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f5253880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f42f2ffd640 0 client.27596 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f524c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 --2- 172.21.15.33:0/3215146425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242230 0x55f0f524c300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 >> 172.21.15.33:0/3215146425 conn(0x55f0f523d8a0 msgr2=0x55f0f523cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 shutdown_connections 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 1 -- 172.21.15.33:0/3215146425 wait complete. 2024-04-24T23:24:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.350+0000 7f431cd7a740 20 client.27596 trim_cache size 0 max 16384 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 Processor -- start 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 -- start start 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242230 0x55f0f52434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 0x55f0f5253840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f52439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43141aaac0 con 0x55f0f5247d90 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43141a8390 con 0x55f0f5242c20 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f431cd7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43141a61e0 con 0x55f0f5242230 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.365+0000 7f4313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f52439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 0x55f0f5253840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 0x55f0f5253840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56016/0 (socket says 172.21.15.33:56016) 2024-04-24T23:24:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f52439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59802/0 (socket says 172.21.15.33:59802) 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 -- 172.21.15.33:0/2837881727 learned_addr learned my addr 172.21.15.33:0/2837881727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f43137fe640 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242230 0x55f0f52434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 -- 172.21.15.33:0/2837881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242230 msgr2=0x55f0f52434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242230 0x55f0f52434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 -- 172.21.15.33:0/2837881727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 msgr2=0x55f0f52439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f52439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 -- 172.21.15.33:0/2837881727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f4f847d0 con 0x55f0f5242c20 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4313fff640 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f52439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f4312ffd640 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 0x55f0f5253840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55f0f523e770 tx=0x7f4308009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e088a138d27b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f42f97fa640 1 -- 172.21.15.33:0/2837881727 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430802dbc0 con 0x55f0f5242c20 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f42f97fa640 1 -- 172.21.15.33:0/2837881727 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308005520 con 0x55f0f5242c20 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.366+0000 7f42f97fa640 1 -- 172.21.15.33:0/2837881727 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430802d770 con 0x55f0f5242c20 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 -- 172.21.15.33:0/2837881727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 msgr2=0x55f0f5253840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 0x55f0f5253840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55f0f523e770 tx=0x7f4308009f90 comp rx=0 tx=0).stop 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 -- 172.21.15.33:0/2837881727 shutdown_connections 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5247d90 0x55f0f52439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5242c20 0x55f0f5253840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 --2- 172.21.15.33:0/2837881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242230 0x55f0f52434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 -- 172.21.15.33:0/2837881727 >> 172.21.15.33:0/2837881727 conn(0x55f0f523d8a0 msgr2=0x55f0f523cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 -- 172.21.15.33:0/2837881727 shutdown_connections 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 -- 172.21.15.33:0/2837881727 wait complete. 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.367+0000 7f431cd7a740 1 -- start start 2024-04-24T23:24:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 Processor -- start 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5251090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f52515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f52525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43141a6850 con 0x55f0f5242230 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43141a7cc0 con 0x55f0f5247d90 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43141a6f20 con 0x55f0f5242c20 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f431cd7a740 10 client.0 _ll_register_callbacks cb 0x7f430409ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5251090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5251090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59814/0 (socket says 172.21.15.33:59814) 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 -- 172.21.15.33:0/841803005 learned_addr learned my addr 172.21.15.33:0/841803005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f4312ffd640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f52515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f4313fff640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f52525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 -- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 msgr2=0x55f0f52515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f52515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 -- 172.21.15.33:0/841803005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 msgr2=0x55f0f52525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.368+0000 7f43137fe640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f52525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f43137fe640 1 -- 172.21.15.33:0/841803005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f5252bd0 con 0x55f0f5242230 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f4312ffd640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f52515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f4313fff640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f52525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f43137fe640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5251090 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4304099f20 tx=0x7f430409a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976047c386902fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f42f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f42f2ffd640 1 -- 172.21.15.33:0/841803005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430409b2c0 con 0x55f0f5242230 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308005520 con 0x55f0f5242230 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f42f2ffd640 1 -- 172.21.15.33:0/841803005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f430409ab30 con 0x55f0f5242230 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f42f2ffd640 1 -- 172.21.15.33:0/841803005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43040a13c0 con 0x55f0f5242230 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f431cd7a740 10 client.18084 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:34.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0f52be820 con 0x55f0f5242230 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.369+0000 7f42f2ffd640 1 -- 172.21.15.33:0/841803005 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308005520 con 0x55f0f5242230 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f42f2ffd640 1 -- 172.21.15.33:0/841803005 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f430409ad30 con 0x55f0f5242230 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f431cd7a740 10 client.18084 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f431cd7a740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f431cd7a740 20 client.18084 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f431cd7a740 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f52c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f431cd7a740 4 client.18084 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0f51adf00 con 0x55f0f5258320 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.370+0000 7f4312ffd640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f52c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f4312ffd640 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f52c0ff0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5afc5b01714b1b server_cookie=99a8af0dbc592d35 in_seq=0 out_seq=0 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f42f2ffd640 10 client.18084 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f42f2ffd640 1 -- 172.21.15.33:0/841803005 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0f51adf00 con 0x55f0f5258320 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f42f2ffd640 10 client.18084 handle_command_reply: tid=0 2024-04-24T23:24:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 client.18084 shutdown 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 msgr2=0x55f0f5251090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5251090 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4304099f20 tx=0x7f430409a180 comp rx=0 tx=0).stop 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 shutdown_connections 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f0f5258320 0x55f0f52c0ff0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f0f5247d90 0x55f0f52525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f42f2ffd640 0 client.18084 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f5242c20 0x55f0f52515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 --2- 172.21.15.33:0/841803005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0f5242230 0x55f0f5251090 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 >> 172.21.15.33:0/841803005 conn(0x55f0f523d8a0 msgr2=0x55f0f523c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 shutdown_connections 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 1 -- 172.21.15.33:0/841803005 wait complete. 2024-04-24T23:24:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.371+0000 7f431cd7a740 20 client.18084 trim_cache size 0 max 16384 2024-04-24T23:24:34.379 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:24:34.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42fc0a2b90 msgr2=0x7f42fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42fc0a2b90 0x7f42fc0a5050 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c009500 comp rx=0 tx=0).stop 2024-04-24T23:24:34.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 msgr2=0x7f4314197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 0x7f4314197a80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9c0 tx=0x7f430400edc0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 shutdown_connections 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f42fc0a2b90 0x7f42fc0a5050 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4314102230 0x7f431419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4314101870 0x7f4314197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 --2- 172.21.15.33:0/3523880604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4314100680 0x7f4314197a80 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 >> 172.21.15.33:0/3523880604 conn(0x7f43140fbbf0 msgr2=0x7f43140688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 shutdown_connections 2024-04-24T23:24:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:34.372+0000 7f431a06b640 1 -- 172.21.15.33:0/3523880604 wait complete. 2024-04-24T23:24:34.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:34 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2004924980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:24:34.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:34 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2004924980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:24:34.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1481960 12.39 MB/sec execute 519 sec latency 12.312 ms 2024-04-24T23:24:35.385 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:24:35.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:35.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:24:35.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:35 smithi033 ceph-mon[29201]: pgmap v728: 129 pgs: 129 active+clean; 288 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-04-24T23:24:35.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:35 smithi038 ceph-mon[32450]: pgmap v728: 129 pgs: 129 active+clean; 288 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-04-24T23:24:35.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:35 smithi117 ceph-mon[32529]: pgmap v728: 129 pgs: 129 active+clean; 288 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-04-24T23:24:35.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1484086 12.38 MB/sec execute 520 sec latency 20.947 ms 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 -- 172.21.15.33:0/1278521109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 msgr2=0x7f5f181097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 --2- 172.21.15.33:0/1278521109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 0x7f5f181097b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f1002a410 comp rx=0 tx=0).stop 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 -- 172.21.15.33:0/1278521109 shutdown_connections 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 --2- 172.21.15.33:0/1278521109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 0x7f5f181097b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 --2- 172.21.15.33:0/1278521109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18104a30 0x7f5f18106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 --2- 172.21.15.33:0/1278521109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f18102100 0x7f5f181044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 -- 172.21.15.33:0/1278521109 >> 172.21.15.33:0/1278521109 conn(0x7f5f180fbaf0 msgr2=0x7f5f180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 -- 172.21.15.33:0/1278521109 shutdown_connections 2024-04-24T23:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.501+0000 7f5f1759e640 1 -- 172.21.15.33:0/1278521109 wait complete. 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 Processor -- start 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 -- start start 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 0x7f5f18197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f18104a30 0x7f5f18197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 0x7f5f1819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1810d320 con 0x7f5f18107360 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1810d1a0 con 0x7f5f18102100 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1759e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1810d4a0 con 0x7f5f18104a30 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 0x7f5f18197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 0x7f5f18197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56046/0 (socket says 172.21.15.33:56046) 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f1659c640 1 -- 172.21.15.33:0/3695968804 learned_addr learned my addr 172.21.15.33:0/3695968804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.502+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 0x7f5f1819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1659c640 1 -- 172.21.15.33:0/3695968804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f18104a30 msgr2=0x7f5f18197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1659c640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f18104a30 0x7f5f18197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1659c640 1 -- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 msgr2=0x7f5f1819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1659c640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 0x7f5f1819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1659c640 1 -- 172.21.15.33:0/3695968804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819f540 con 0x7f5f18102100 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1659c640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 0x7f5f18197970 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800eb90 tx=0x7f5f0800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f0f7fe640 1 -- 172.21.15.33:0/3695968804 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08004d20 con 0x7f5f18102100 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1819f830 con 0x7f5f18102100 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f0f7fe640 1 -- 172.21.15.33:0/3695968804 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0800ee90 con 0x7f5f18102100 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f0f7fe640 1 -- 172.21.15.33:0/3695968804 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08010850 con 0x7f5f18102100 2024-04-24T23:24:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.503+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1819fd30 con 0x7f5f18102100 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f0f7fe640 1 -- 172.21.15.33:0/3695968804 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5f080109f0 con 0x7f5f18102100 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 Processor -- start 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 -- start start 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44201d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a442022880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a442025380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a441f7c2c0 con 0x55a442022ef0 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a441f24910 con 0x55a44201dd80 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a441f78280 con 0x55a44201d390 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a442022880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a442025380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44201d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44201d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:48942/0 (socket says 172.21.15.33:48942) 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.506+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2062416416 learned_addr learned my addr 172.21.15.33:0/2062416416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2062416416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 msgr2=0x55a442022880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a442022880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2062416416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 msgr2=0x55a442025380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a442025380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f0f7fe640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5f040a2b90 0x7f5f040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2062416416 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a441f35fb0 con 0x55a44201d390 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f15d9b640 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a442025380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f0f7fe640 1 -- 172.21.15.33:0/3695968804 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5f080cfac0 con 0x7f5f18102100 2024-04-24T23:24:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5f040a2b90 0x7f5f040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44201d7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f10005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eff1d7521d21653 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f0cff9640 1 -- 172.21.15.33:0/2062416416 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1004c070 con 0x55a44201d390 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f0cff9640 1 -- 172.21.15.33:0/2062416416 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f10003ad0 con 0x55a44201d390 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.507+0000 7f5f0cff9640 1 -- 172.21.15.33:0/2062416416 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10040070 con 0x55a44201d390 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2062416416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 msgr2=0x55a44201d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5f040a2b90 0x7f5f040a5050 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f10049000 tx=0x7f5f1004c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44201d7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f10005c00 comp rx=0 tx=0).stop 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2062416416 shutdown_connections 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a442025380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a442022880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2062416416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44201d7b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2062416416 >> 172.21.15.33:0/2062416416 conn(0x55a441fe0950 msgr2=0x55a442018e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2062416416 shutdown_connections 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2062416416 wait complete. 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 -- start start 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.508+0000 7f5f1f9ec740 1 Processor -- start 2024-04-24T23:24:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a442027810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201dd80 0x55a442027d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 0x55a44202ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a442025ea0 con 0x55a44201dd80 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a441f37e90 con 0x55a442022ef0 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a442026020 con 0x55a44201d390 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 0x55a44202ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201dd80 0x55a442027d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 0x55a44202ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:56064/0 (socket says 172.21.15.33:56064) 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 -- 172.21.15.33:0/4200375932 learned_addr learned my addr 172.21.15.33:0/4200375932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1f9ec740 10 client.0 _ll_register_callbacks cb 0x55a441f73970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f1659c640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a442027810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 -- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 msgr2=0x55a442027810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a442027810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 -- 172.21.15.33:0/4200375932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201dd80 msgr2=0x55a442027d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201dd80 0x55a442027d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f16d9d640 1 -- 172.21.15.33:0/4200375932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a441d587e0 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.509+0000 7f5f15d9b640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201dd80 0x55a442027d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 0x55a44202ed90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f10006010 tx=0x7f5f10031a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5973d2e820695310 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5efdffb640 1 -- 172.21.15.33:0/4200375932 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10058070 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f10003ad0 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5efdffb640 1 -- 172.21.15.33:0/4200375932 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5f100035f0 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5efdffb640 1 -- 172.21.15.33:0/4200375932 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f10064050 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5f1f9ec740 10 client.27614 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.510+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a441db8f70 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5efdffb640 1 -- 172.21.15.33:0/4200375932 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f10003ad0 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5efdffb640 1 -- 172.21.15.33:0/4200375932 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5f1005d480 con 0x55a442022ef0 2024-04-24T23:24:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1f9ec740 10 client.27614 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1f9ec740 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1f9ec740 20 client.27614 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a442032ef0 0x55a4420352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1f9ec740 4 client.27614 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a441f505a0 con 0x55a442032ef0 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.511+0000 7f5f1659c640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a442032ef0 0x55a4420352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5efdffb640 10 client.27614 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1659c640 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a442032ef0 0x55a4420352e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ba62f936896d91 server_cookie=bf39a9c7488efb82 in_seq=0 out_seq=0 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5efdffb640 1 -- 172.21.15.33:0/4200375932 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a441f505a0 con 0x55a442032ef0 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5efdffb640 10 client.27614 handle_command_reply: tid=0 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 client.27614 shutdown 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 msgr2=0x55a44202ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 0x55a44202ed90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f10006010 tx=0x7f5f10031a90 comp rx=0 tx=0).stop 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 shutdown_connections 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a442032ef0 0x55a4420352e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a442022ef0 0x55a44202ed90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201dd80 0x55a442027d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/4200375932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a442027810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5efdffb640 0 client.27614 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.512+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 >> 172.21.15.33:0/4200375932 conn(0x55a441fe0950 msgr2=0x55a442013b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.513+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 shutdown_connections 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.513+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/4200375932 wait complete. 2024-04-24T23:24:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.513+0000 7f5f1f9ec740 20 client.27614 trim_cache size 0 max 16384 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.531+0000 7f5f1f9ec740 1 Processor -- start 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.531+0000 7f5f1f9ec740 1 -- start start 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.531+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 0x55a44201bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a442022ef0 0x55a44201c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181aa930 con 0x55a44201d390 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181a8200 con 0x55a44201dd80 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181a6050 con 0x55a442022ef0 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 0x55a44201bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 0x55a44201bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59862/0 (socket says 172.21.15.33:59862) 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 -- 172.21.15.33:0/273727791 learned_addr learned my addr 172.21.15.33:0/273727791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a442022ef0 0x55a44201c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 -- 172.21.15.33:0/273727791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a442022ef0 msgr2=0x55a44201c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a442022ef0 0x55a44201c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 -- 172.21.15.33:0/273727791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 msgr2=0x55a44202ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 -- 172.21.15.33:0/273727791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a441c50170 con 0x55a44201d390 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a442022ef0 0x55a44201c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f15d9b640 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5f1659c640 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 0x55a44201bd00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5f08005a30 tx=0x7f5f080cd100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d551de418e692ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5efffff640 1 -- 172.21.15.33:0/273727791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08098230 con 0x55a44201d390 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5efffff640 1 -- 172.21.15.33:0/273727791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0809c690 con 0x55a44201d390 2024-04-24T23:24:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.532+0000 7f5efffff640 1 -- 172.21.15.33:0/273727791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0809b8f0 con 0x55a44201d390 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/273727791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 msgr2=0x55a44201bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 0x55a44201bd00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5f08005a30 tx=0x7f5f080cd100 comp rx=0 tx=0).stop 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/273727791 shutdown_connections 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a442022ef0 0x55a44201c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/273727791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a44201d390 0x55a44201bd00 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/273727791 >> 172.21.15.33:0/273727791 conn(0x55a441fe0950 msgr2=0x55a442018780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/273727791 shutdown_connections 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.533+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/273727791 wait complete. 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 -- start start 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 Processor -- start 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44202c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a44202de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181a66c0 con 0x55a442022ef0 2024-04-24T23:24:35.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181a7b30 con 0x55a44201dd80 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181a6d90 con 0x55a44201d390 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1f9ec740 10 client.0 _ll_register_callbacks cb 0x7f5f1004a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a44202de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44202c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a44202de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59876/0 (socket says 172.21.15.33:59876) 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2006314748 learned_addr learned my addr 172.21.15.33:0/2006314748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44202c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:48970/0 (socket says 172.21.15.33:48970) 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 msgr2=0x55a44202c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44202c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2006314748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 msgr2=0x55a44202cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 -- 172.21.15.33:0/2006314748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44202e3f0 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f1659c640 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44202c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5f16d9d640 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a44202de00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5f10005ab0 tx=0x7f5f10005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f5987fb00aafed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.534+0000 7f5efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5efdffb640 1 -- 172.21.15.33:0/2006314748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10058050 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f0809c690 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5efdffb640 1 -- 172.21.15.33:0/2006314748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5f1005d640 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5efdffb640 1 -- 172.21.15.33:0/2006314748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f10071050 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 10 client.18102 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a442099740 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5efdffb640 1 -- 172.21.15.33:0/2006314748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0809c690 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5efdffb640 1 -- 172.21.15.33:0/2006314748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5f10003820 con 0x55a442022ef0 2024-04-24T23:24:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 10 client.18102 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 20 client.18102 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a44209a020 0x55a44209c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 4 client.18102 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a441f89800 con 0x55a44209a020 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.535+0000 7f5f1659c640 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a44209a020 0x55a44209c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.536+0000 7f5f1659c640 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a44209a020 0x55a44209c440 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799a052377286bbd server_cookie=56ebcd80ca53ca0c in_seq=0 out_seq=0 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.536+0000 7f5efdffb640 10 client.18102 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5efdffb640 1 -- 172.21.15.33:0/2006314748 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a441f89800 con 0x55a44209a020 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5efdffb640 10 client.18102 handle_command_reply: tid=0 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 client.18102 shutdown 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 msgr2=0x55a44202de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a44202de00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5f10005ab0 tx=0x7f5f10005c00 comp rx=0 tx=0).stop 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 shutdown_connections 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a44209a020 0x55a44209c440 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a442022ef0 0x55a44202de00 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a44201dd80 0x55a44202cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 --2- 172.21.15.33:0/2006314748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a44201d390 0x55a44202c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 >> 172.21.15.33:0/2006314748 conn(0x55a441fe0950 msgr2=0x55a4420132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5efdffb640 0 client.18102 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 shutdown_connections 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 1 -- 172.21.15.33:0/2006314748 wait complete. 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.537+0000 7f5f1f9ec740 20 client.18102 trim_cache size 0 max 16384 2024-04-24T23:24:35.545 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:24:35.546 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:24:35.546 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "c435e545-118d-45bd-bdcb-641fb124fcdc", 2024-04-24T23:24:35.546 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:24:35.546 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5f040a2b90 msgr2=0x7f5f040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5f040a2b90 0x7f5f040a5050 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f10049000 tx=0x7f5f1004c000 comp rx=0 tx=0).stop 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 msgr2=0x7f5f18197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 0x7f5f18197970 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800eb90 tx=0x7f5f0800c6a0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 shutdown_connections 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5f040a2b90 0x7f5f040a5050 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18107360 0x7f5f1819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f18104a30 0x7f5f18197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 --2- 172.21.15.33:0/3695968804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f18102100 0x7f5f18197970 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 >> 172.21.15.33:0/3695968804 conn(0x7f5f180fbaf0 msgr2=0x7f5f18102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 shutdown_connections 2024-04-24T23:24:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.538+0000 7f5f1759e640 1 -- 172.21.15.33:0/3695968804 wait complete. 2024-04-24T23:24:35.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:24:35.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 -- 172.21.15.33:0/2950032285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 msgr2=0x7f4f88109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 --2- 172.21.15.33:0/2950032285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f88109be0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009ca0 tx=0x7f4f8002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 -- 172.21.15.33:0/2950032285 shutdown_connections 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 --2- 172.21.15.33:0/2950032285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f88109be0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 --2- 172.21.15.33:0/2950032285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f88102990 0x7f4f88102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 --2- 172.21.15.33:0/2950032285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f881017a0 0x7f4f88101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 -- 172.21.15.33:0/2950032285 >> 172.21.15.33:0/2950032285 conn(0x7f4f880fcd10 msgr2=0x7f4f880ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 -- 172.21.15.33:0/2950032285 shutdown_connections 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.744+0000 7f4f902f5640 1 -- 172.21.15.33:0/2950032285 wait complete. 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 Processor -- start 2024-04-24T23:24:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 -- start start 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f881017a0 0x7f4f88197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f88102990 0x7f4f88197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f8819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810d420 con 0x7f4f88103350 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810d2a0 con 0x7f4f881017a0 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f902f5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810d5a0 con 0x7f4f88102990 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f8819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f8819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57862/0 (socket says 172.21.15.33:57862) 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 -- 172.21.15.33:0/3611991139 learned_addr learned my addr 172.21.15.33:0/3611991139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e091640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f881017a0 0x7f4f88197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8d890640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f88102990 0x7f4f88197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 -- 172.21.15.33:0/3611991139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f88102990 msgr2=0x7f4f88197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f88102990 0x7f4f88197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 -- 172.21.15.33:0/3611991139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f881017a0 msgr2=0x7f4f88197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f881017a0 0x7f4f88197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 -- 172.21.15.33:0/3611991139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8819f660 con 0x7f4f88103350 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.745+0000 7f4f8e892640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f8819eff0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4f80031ae0 tx=0x7f4f80031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.746+0000 7f4f7f7fe640 1 -- 172.21.15.33:0/3611991139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80003c30 con 0x7f4f88103350 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.746+0000 7f4f7f7fe640 1 -- 172.21.15.33:0/3611991139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f80003dd0 con 0x7f4f88103350 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.746+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8819f8f0 con 0x7f4f88103350 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.746+0000 7f4f7f7fe640 1 -- 172.21.15.33:0/3611991139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8002fcb0 con 0x7f4f88103350 2024-04-24T23:24:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.746+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8819fe90 con 0x7f4f88103350 2024-04-24T23:24:35.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.748+0000 7f4f7f7fe640 1 -- 172.21.15.33:0/3611991139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4f80004050 con 0x7f4f88103350 2024-04-24T23:24:35.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.748+0000 7f4f93002740 1 Processor -- start 2024-04-24T23:24:35.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 -- start start 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e2610660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e2615730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2618230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9e25746b0 con 0x55e9e2610c30 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e9e251c880 con 0x55e9e2615da0 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f93002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9e2570670 con 0x55e9e2610240 2024-04-24T23:24:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8e091640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e2615730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8e091640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e2615730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57878/0 (socket says 172.21.15.33:57878) 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8e091640 1 -- 172.21.15.33:0/7935733 learned_addr learned my addr 172.21.15.33:0/7935733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2618230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f7f7fe640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f740a2c10 0x7f4f740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e2610660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8e892640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f740a2c10 0x7f4f740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 -- 172.21.15.33:0/7935733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 msgr2=0x55e9e2610660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e2610660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f7f7fe640 1 -- 172.21.15.33:0/3611991139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4f800f0570 con 0x7f4f88103350 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 -- 172.21.15.33:0/7935733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 msgr2=0x55e9e2615730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e2615730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8d890640 1 -- 172.21.15.33:0/7935733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e23527d0 con 0x55e9e2615da0 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.749+0000 7f4f8e091640 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e2615730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.750+0000 7f4f8d890640 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2618230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800afd0 tx=0x7f4f7800a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d180c873f5d0b4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.750+0000 7f4f8e892640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f740a2c10 0x7f4f740a50d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f800b9530 tx=0x7f4f80087da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.750+0000 7f4f7cbfe640 1 -- 172.21.15.33:0/7935733 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78004020 con 0x55e9e2615da0 2024-04-24T23:24:35.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.750+0000 7f4f7cbfe640 1 -- 172.21.15.33:0/7935733 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f78005500 con 0x55e9e2615da0 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.750+0000 7f4f7cbfe640 1 -- 172.21.15.33:0/7935733 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78004cb0 con 0x55e9e2615da0 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- 172.21.15.33:0/7935733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 msgr2=0x55e9e2618230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2618230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800afd0 tx=0x7f4f7800a640 comp rx=0 tx=0).stop 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- 172.21.15.33:0/7935733 shutdown_connections 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2618230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e2615730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- 172.21.15.33:0/7935733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e2610660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- 172.21.15.33:0/7935733 >> 172.21.15.33:0/7935733 conn(0x55e9e260b8b0 msgr2=0x55e9e260bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- 172.21.15.33:0/7935733 shutdown_connections 2024-04-24T23:24:35.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- 172.21.15.33:0/7935733 wait complete. 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- start start 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 Processor -- start 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2615da0 0x55e9e2621c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9e2618d50 con 0x55e9e2610240 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e9e252ff50 con 0x55e9e2610c30 2024-04-24T23:24:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9e2618ed0 con 0x55e9e2615da0 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.751+0000 7f4f93002740 10 client.0 _ll_register_callbacks cb 0x55e9e256b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8d890640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34596/0 (socket says 172.21.15.33:34596) 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 -- 172.21.15.33:0/1615649560 learned_addr learned my addr 172.21.15.33:0/1615649560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8d890640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2615da0 0x55e9e2621c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 -- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2615da0 msgr2=0x55e9e2621c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2615da0 0x55e9e2621c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 -- 172.21.15.33:0/1615649560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610240 msgr2=0x55e9e261a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 -- 172.21.15.33:0/1615649560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e22883f0 con 0x55e9e2610c30 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8d890640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f8e091640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f840126e0 tx=0x7f4f84012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed3f99021fb20612 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f71ffb640 1 -- 172.21.15.33:0/1615649560 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84012190 con 0x55e9e2610c30 2024-04-24T23:24:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f78005500 con 0x55e9e2610c30 2024-04-24T23:24:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f71ffb640 1 -- 172.21.15.33:0/1615649560 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4f84004030 con 0x55e9e2610c30 2024-04-24T23:24:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f71ffb640 1 -- 172.21.15.33:0/1615649560 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f84025050 con 0x55e9e2610c30 2024-04-24T23:24:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f93002740 10 client.27632 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.752+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9e22480f0 con 0x55e9e2610c30 2024-04-24T23:24:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f71ffb640 1 -- 172.21.15.33:0/1615649560 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f78005500 con 0x55e9e2610c30 2024-04-24T23:24:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f71ffb640 1 -- 172.21.15.33:0/1615649560 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4f84003850 con 0x55e9e2610c30 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f93002740 10 client.27632 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f93002740 10 client.27632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f93002740 20 client.27632 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f93002740 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e2626490 0x55e9e2628910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f93002740 4 client.27632 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e2581bf0 con 0x55e9e2626490 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f8d890640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e2626490 0x55e9e2628910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f8d890640 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e2626490 0x55e9e2628910 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4090c7dac049aa3 server_cookie=2647ccacce7e19ed in_seq=0 out_seq=0 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.753+0000 7f4f71ffb640 10 client.27632 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f71ffb640 1 -- 172.21.15.33:0/1615649560 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9e2581bf0 con 0x55e9e2626490 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f71ffb640 10 client.27632 handle_command_reply: tid=0 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 client.27632 shutdown 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 msgr2=0x55e9e261abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f840126e0 tx=0x7f4f84012ba0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 shutdown_connections 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e2626490 0x55e9e2628910 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2615da0 0x55e9e2621c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f71ffb640 0 client.27632 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 --2- 172.21.15.33:0/1615649560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 >> 172.21.15.33:0/1615649560 conn(0x55e9e260b8b0 msgr2=0x55e9e260aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 shutdown_connections 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 1 -- 172.21.15.33:0/1615649560 wait complete. 2024-04-24T23:24:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.754+0000 7f4f93002740 20 client.27632 trim_cache size 0 max 16384 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 Processor -- start 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 -- start start 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2615da0 0x55e9e2621bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881acbf0 con 0x55e9e2615da0 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881aa4c0 con 0x55e9e2610c30 2024-04-24T23:24:35.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f93002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881a8310 con 0x55e9e2610240 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:45884/0 (socket says 172.21.15.33:45884) 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 -- 172.21.15.33:0/2082656706 learned_addr learned my addr 172.21.15.33:0/2082656706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8e091640 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2615da0 0x55e9e2621bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 -- 172.21.15.33:0/2082656706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 msgr2=0x55e9e261abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 -- 172.21.15.33:0/2082656706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2615da0 msgr2=0x55e9e2621bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2615da0 0x55e9e2621bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.775+0000 7f4f8d890640 1 -- 172.21.15.33:0/2082656706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e2611010 con 0x55e9e2610240 2024-04-24T23:24:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.776+0000 7f4f8d890640 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261a690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f780058c0 tx=0x7f4f7800a450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e2ac6ec9c516af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.776+0000 7f4f73fff640 1 -- 172.21.15.33:0/2082656706 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78032070 con 0x55e9e2610240 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.776+0000 7f4f73fff640 1 -- 172.21.15.33:0/2082656706 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f780054c0 con 0x55e9e2610240 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.776+0000 7f4f73fff640 1 -- 172.21.15.33:0/2082656706 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800f640 con 0x55e9e2610240 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.776+0000 7f4f93002740 1 -- 172.21.15.33:0/2082656706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 msgr2=0x55e9e261a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.776+0000 7f4f93002740 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261a690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f780058c0 tx=0x7f4f7800a450 comp rx=0 tx=0).stop 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- 172.21.15.33:0/2082656706 shutdown_connections 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2615da0 0x55e9e2621bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2610c30 0x55e9e261abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 --2- 172.21.15.33:0/2082656706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261a690 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- 172.21.15.33:0/2082656706 >> 172.21.15.33:0/2082656706 conn(0x55e9e260b8b0 msgr2=0x55e9e2612090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- 172.21.15.33:0/2082656706 shutdown_connections 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- 172.21.15.33:0/2082656706 wait complete. 2024-04-24T23:24:35.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- start start 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 Processor -- start 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e261f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2620460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881a8980 con 0x55e9e2610c30 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881a9df0 con 0x55e9e2615da0 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881a9050 con 0x55e9e2610240 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f93002740 10 client.0 _ll_register_callbacks cb 0x7f4f84009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f8e091640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e261f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f8d890640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f8e091640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e261f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57894/0 (socket says 172.21.15.33:57894) 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f8e091640 1 -- 172.21.15.33:0/590143518 learned_addr learned my addr 172.21.15.33:0/590143518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.777+0000 7f4f8d890640 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2620460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f8e091640 1 -- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 msgr2=0x55e9e261ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f8e091640 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f8e091640 1 -- 172.21.15.33:0/590143518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 msgr2=0x55e9e2620460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f8e091640 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2620460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f8e091640 1 -- 172.21.15.33:0/590143518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e2620a50 con 0x55e9e2610c30 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f8e091640 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e261f450 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x55e9e261bb70 tx=0x7f4f840123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2dbc78b6ab7a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f71ffb640 1 -- 172.21.15.33:0/590143518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8402e060 con 0x55e9e2610c30 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f71ffb640 1 -- 172.21.15.33:0/590143518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4f84003880 con 0x55e9e2610c30 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f780054c0 con 0x55e9e2610c30 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f71ffb640 1 -- 172.21.15.33:0/590143518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f84035050 con 0x55e9e2610c30 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f93002740 10 client.18120 fetch_fsmap learned FSMap version 27 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.778+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9e2621e00 con 0x55e9e2610c30 2024-04-24T23:24:35.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f71ffb640 1 -- 172.21.15.33:0/590143518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f780054c0 con 0x55e9e2610c30 2024-04-24T23:24:35.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f71ffb640 1 -- 172.21.15.33:0/590143518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4f8401c460 con 0x55e9e2610c30 2024-04-24T23:24:35.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f93002740 10 client.18120 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f93002740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f93002740 20 client.18120 populate_metadata read hostname 'smithi033' 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f93002740 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e268d020 0x55e9e268f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f93002740 4 client.18120 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9e257bf40 con 0x55e9e268d020 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.779+0000 7f4f8d890640 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e268d020 0x55e9e268f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f71ffb640 10 client.18120 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f8d890640 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e268d020 0x55e9e268f440 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f66b9f1c4d949e0 server_cookie=2d5458eac3e26514 in_seq=0 out_seq=0 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f71ffb640 1 -- 172.21.15.33:0/590143518 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e9e257bf40 con 0x55e9e268d020 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f71ffb640 10 client.18120 handle_command_reply: tid=0 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 client.18120 shutdown 2024-04-24T23:24:35.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 msgr2=0x55e9e261f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e261f450 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x55e9e261bb70 tx=0x7f4f840123a0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 shutdown_connections 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e9e268d020 0x55e9e268f440 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9e2615da0 0x55e9e2620460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9e2610c30 0x55e9e261f450 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 --2- 172.21.15.33:0/590143518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e2610240 0x55e9e261ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 >> 172.21.15.33:0/590143518 conn(0x55e9e260b8b0 msgr2=0x55e9e261a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f71ffb640 0 client.18120 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 shutdown_connections 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 1 -- 172.21.15.33:0/590143518 wait complete. 2024-04-24T23:24:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.780+0000 7f4f93002740 20 client.18120 trim_cache size 0 max 16384 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "c435e545-118d-45bd-bdcb-641fb124fcdc": { 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "c435e545-118d-45bd-bdcb-641fb124fcdc", 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:24:35.789 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:24:35.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f740a2c10 msgr2=0x7f4f740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f740a2c10 0x7f4f740a50d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f800b9530 tx=0x7f4f80087da0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 msgr2=0x7f4f8819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f8819eff0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4f80031ae0 tx=0x7f4f80031ee0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 shutdown_connections 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4f740a2c10 0x7f4f740a50d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f88103350 0x7f4f8819eff0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f88102990 0x7f4f88197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 --2- 172.21.15.33:0/3611991139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f881017a0 0x7f4f88197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 >> 172.21.15.33:0/3611991139 conn(0x7f4f880fcd10 msgr2=0x7f4f88105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 shutdown_connections 2024-04-24T23:24:35.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:24:35.781+0000 7f4f902f5640 1 -- 172.21.15.33:0/3611991139 wait complete. 2024-04-24T23:24:35.794 INFO:tasks.cephfs.filesystem:scrub status for tag:c435e545-118d-45bd-bdcb-641fb124fcdc - {'path': '/', 'tag': 'c435e545-118d-45bd-bdcb-641fb124fcdc', 'options': 'recursive,force'} 2024-04-24T23:24:36.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1485425 12.38 MB/sec execute 521 sec latency 112.008 ms 2024-04-24T23:24:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:24:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:24:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:24:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:37 smithi038 ceph-mon[32450]: pgmap v729: 129 pgs: 129 active+clean; 287 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 210 op/s 2024-04-24T23:24:37.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:24:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:24:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:24:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:37 smithi117 ceph-mon[32529]: pgmap v729: 129 pgs: 129 active+clean; 287 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 210 op/s 2024-04-24T23:24:37.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:24:37.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:24:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:24:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:24:37.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:37 smithi033 ceph-mon[29201]: pgmap v729: 129 pgs: 129 active+clean; 287 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 210 op/s 2024-04-24T23:24:37.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1487449 12.37 MB/sec execute 522 sec latency 21.162 ms 2024-04-24T23:24:38.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1489298 12.37 MB/sec execute 523 sec latency 12.649 ms 2024-04-24T23:24:39.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:39 smithi038 ceph-mon[32450]: pgmap v730: 129 pgs: 129 active+clean; 286 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-04-24T23:24:39.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:39 smithi117 ceph-mon[32529]: pgmap v730: 129 pgs: 129 active+clean; 286 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-04-24T23:24:39.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:39 smithi033 ceph-mon[29201]: pgmap v730: 129 pgs: 129 active+clean; 286 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-04-24T23:24:39.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1492191 12.38 MB/sec execute 524 sec latency 20.353 ms 2024-04-24T23:24:40.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1494101 12.37 MB/sec execute 525 sec latency 12.925 ms 2024-04-24T23:24:41.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:41 smithi038 ceph-mon[32450]: pgmap v731: 129 pgs: 129 active+clean; 282 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 232 op/s 2024-04-24T23:24:41.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:41 smithi117 ceph-mon[32529]: pgmap v731: 129 pgs: 129 active+clean; 282 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 232 op/s 2024-04-24T23:24:41.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:41 smithi033 ceph-mon[29201]: pgmap v731: 129 pgs: 129 active+clean; 282 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 232 op/s 2024-04-24T23:24:41.466 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1496137 12.37 MB/sec execute 526 sec latency 57.383 ms 2024-04-24T23:24:42.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1498272 12.36 MB/sec execute 527 sec latency 32.652 ms 2024-04-24T23:24:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:43 smithi038 ceph-mon[32450]: pgmap v732: 129 pgs: 129 active+clean; 268 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-04-24T23:24:43.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:24:43.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:43 smithi117 ceph-mon[32529]: pgmap v732: 129 pgs: 129 active+clean; 268 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-04-24T23:24:43.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:24:43.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:43 smithi033 ceph-mon[29201]: pgmap v732: 129 pgs: 129 active+clean; 268 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-04-24T23:24:43.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:24:43.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1500307 12.37 MB/sec execute 528 sec latency 13.491 ms 2024-04-24T23:24:44.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:44 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:24:44.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:44 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:24:44.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:44 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:24:44.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1503046 12.37 MB/sec execute 529 sec latency 12.243 ms 2024-04-24T23:24:45.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:45 smithi038 ceph-mon[32450]: pgmap v733: 129 pgs: 129 active+clean; 268 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2024-04-24T23:24:45.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:45 smithi117 ceph-mon[32529]: pgmap v733: 129 pgs: 129 active+clean; 268 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2024-04-24T23:24:45.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:45 smithi033 ceph-mon[29201]: pgmap v733: 129 pgs: 129 active+clean; 268 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2024-04-24T23:24:45.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1505476 12.37 MB/sec execute 530 sec latency 23.053 ms 2024-04-24T23:24:46.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1506985 12.36 MB/sec execute 531 sec latency 70.260 ms 2024-04-24T23:24:47.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:47 smithi038 ceph-mon[32450]: pgmap v734: 129 pgs: 129 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-04-24T23:24:47.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:47 smithi117 ceph-mon[32529]: pgmap v734: 129 pgs: 129 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-04-24T23:24:47.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:47 smithi033 ceph-mon[29201]: pgmap v734: 129 pgs: 129 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-04-24T23:24:47.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1509530 12.37 MB/sec execute 532 sec latency 23.814 ms 2024-04-24T23:24:48.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1511103 12.36 MB/sec execute 533 sec latency 18.643 ms 2024-04-24T23:24:49.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:49 smithi038 ceph-mon[32450]: pgmap v735: 129 pgs: 129 active+clean; 273 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-04-24T23:24:49.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:49 smithi117 ceph-mon[32529]: pgmap v735: 129 pgs: 129 active+clean; 273 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-04-24T23:24:49.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:49 smithi033 ceph-mon[29201]: pgmap v735: 129 pgs: 129 active+clean; 273 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-04-24T23:24:49.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1513864 12.37 MB/sec execute 534 sec latency 65.015 ms 2024-04-24T23:24:50.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1516400 12.37 MB/sec execute 535 sec latency 14.595 ms 2024-04-24T23:24:51.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:51 smithi038 ceph-mon[32450]: pgmap v736: 129 pgs: 129 active+clean; 273 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-24T23:24:51.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:51 smithi033 ceph-mon[29201]: pgmap v736: 129 pgs: 129 active+clean; 273 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-24T23:24:51.506 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:51 smithi117 ceph-mon[32529]: pgmap v736: 129 pgs: 129 active+clean; 273 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-24T23:24:51.507 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1517732 12.36 MB/sec execute 536 sec latency 81.124 ms 2024-04-24T23:24:52.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1520449 12.37 MB/sec execute 537 sec latency 20.994 ms 2024-04-24T23:24:53.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:53 smithi038 ceph-mon[32450]: pgmap v737: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 306 op/s 2024-04-24T23:24:53.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:53 smithi117 ceph-mon[32529]: pgmap v737: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 306 op/s 2024-04-24T23:24:53.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:53 smithi033 ceph-mon[29201]: pgmap v737: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 306 op/s 2024-04-24T23:24:53.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1522085 12.36 MB/sec execute 538 sec latency 40.430 ms 2024-04-24T23:24:54.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1524588 12.36 MB/sec execute 539 sec latency 16.221 ms 2024-04-24T23:24:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:55 smithi038 ceph-mon[32450]: pgmap v738: 129 pgs: 129 active+clean; 277 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-04-24T23:24:55.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:55 smithi117 ceph-mon[32529]: pgmap v738: 129 pgs: 129 active+clean; 277 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-04-24T23:24:55.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:55 smithi033 ceph-mon[29201]: pgmap v738: 129 pgs: 129 active+clean; 277 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-04-24T23:24:55.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1527038 12.36 MB/sec execute 540 sec latency 17.399 ms 2024-04-24T23:24:56.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1528341 12.35 MB/sec execute 541 sec latency 73.903 ms 2024-04-24T23:24:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:57 smithi038 ceph-mon[32450]: pgmap v739: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-04-24T23:24:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:57 smithi117 ceph-mon[32529]: pgmap v739: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-04-24T23:24:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:57 smithi033 ceph-mon[29201]: pgmap v739: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-04-24T23:24:57.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1530601 12.35 MB/sec execute 542 sec latency 25.842 ms 2024-04-24T23:24:58.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1532197 12.34 MB/sec execute 543 sec latency 23.234 ms 2024-04-24T23:24:59.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:24:59 smithi038 ceph-mon[32450]: pgmap v740: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-04-24T23:24:59.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:24:59 smithi117 ceph-mon[32529]: pgmap v740: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-04-24T23:24:59.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:24:59 smithi033 ceph-mon[29201]: pgmap v740: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-04-24T23:24:59.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1534910 12.35 MB/sec execute 544 sec latency 14.895 ms 2024-04-24T23:25:00.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1537018 12.34 MB/sec execute 545 sec latency 19.087 ms 2024-04-24T23:25:01.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:01 smithi038 ceph-mon[32450]: pgmap v741: 129 pgs: 129 active+clean; 278 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-04-24T23:25:01.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:01 smithi117 ceph-mon[32529]: pgmap v741: 129 pgs: 129 active+clean; 278 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-04-24T23:25:01.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:01 smithi033 ceph-mon[29201]: pgmap v741: 129 pgs: 129 active+clean; 278 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-04-24T23:25:01.467 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1538747 12.34 MB/sec execute 546 sec latency 70.909 ms 2024-04-24T23:25:02.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1541009 12.34 MB/sec execute 547 sec latency 22.570 ms 2024-04-24T23:25:03.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:03 smithi038 ceph-mon[32450]: pgmap v742: 129 pgs: 129 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-04-24T23:25:03.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:03 smithi117 ceph-mon[32529]: pgmap v742: 129 pgs: 129 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-04-24T23:25:03.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:03 smithi033 ceph-mon[29201]: pgmap v742: 129 pgs: 129 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-04-24T23:25:03.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1542823 12.33 MB/sec execute 548 sec latency 16.939 ms 2024-04-24T23:25:04.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1545592 12.34 MB/sec execute 549 sec latency 14.945 ms 2024-04-24T23:25:05.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:05 smithi038 ceph-mon[32450]: pgmap v743: 129 pgs: 129 active+clean; 283 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-04-24T23:25:05.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:05 smithi117 ceph-mon[32529]: pgmap v743: 129 pgs: 129 active+clean; 283 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-04-24T23:25:05.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:05 smithi033 ceph-mon[29201]: pgmap v743: 129 pgs: 129 active+clean; 283 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-04-24T23:25:05.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1547796 12.33 MB/sec execute 550 sec latency 24.406 ms 2024-04-24T23:25:05.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:25:05.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:25:05.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.910+0000 7ff9f54cd640 1 -- 172.21.15.33:0/3993113766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f010b0b0 msgr2=0x7ff9f010d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.910+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/3993113766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f010b0b0 0x7ff9f010d540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009860 tx=0x7ff9e402a340 comp rx=0 tx=0).stop 2024-04-24T23:25:05.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.911+0000 7ff9f54cd640 1 -- 172.21.15.33:0/3993113766 shutdown_connections 2024-04-24T23:25:05.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.911+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/3993113766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 0x7ff9f010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.911+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/3993113766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f010b0b0 0x7ff9f010d540 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.911+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/3993113766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f0108740 0x7ff9f010ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.911+0000 7ff9f54cd640 1 -- 172.21.15.33:0/3993113766 >> 172.21.15.33:0/3993113766 conn(0x7ff9f006e5d0 msgr2=0x7ff9f0070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:05.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.912+0000 7ff9f54cd640 1 -- 172.21.15.33:0/3993113766 shutdown_connections 2024-04-24T23:25:05.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.912+0000 7ff9f54cd640 1 -- 172.21.15.33:0/3993113766 wait complete. 2024-04-24T23:25:05.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 Processor -- start 2024-04-24T23:25:05.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 -- start start 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f0108740 0x7ff9f01a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f010b0b0 0x7ff9f01a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 0x7ff9f01adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f011c240 con 0x7ff9f010da80 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f011c0c0 con 0x7ff9f0108740 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.913+0000 7ff9f54cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f011c3c0 con 0x7ff9f010b0b0 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f010b0b0 0x7ff9f01a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f010b0b0 0x7ff9f01a6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34368/0 (socket says 172.21.15.33:34368) 2024-04-24T23:25:05.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/828440532 learned_addr learned my addr 172.21.15.33:0/828440532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:05.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 0x7ff9f01adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f0108740 0x7ff9f01a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/828440532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f010b0b0 msgr2=0x7ff9f01a6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f010b0b0 0x7ff9f01a6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/828440532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f0108740 msgr2=0x7ff9f01a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f0108740 0x7ff9f01a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.914+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/828440532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01ae530 con 0x7ff9f010da80 2024-04-24T23:25:05.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.915+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 0x7ff9f01adec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8005fd0 tx=0x7ff9e800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:05.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.916+0000 7ff9d7fff640 1 -- 172.21.15.33:0/828440532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8004b20 con 0x7ff9f010da80 2024-04-24T23:25:05.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.916+0000 7ff9d7fff640 1 -- 172.21.15.33:0/828440532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e8003e50 con 0x7ff9f010da80 2024-04-24T23:25:05.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.916+0000 7ff9d7fff640 1 -- 172.21.15.33:0/828440532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800b570 con 0x7ff9f010da80 2024-04-24T23:25:05.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.916+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f01ae820 con 0x7ff9f010da80 2024-04-24T23:25:05.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.916+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01aed20 con 0x7ff9f010da80 2024-04-24T23:25:05.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.920+0000 7ff9f81db740 1 Processor -- start 2024-04-24T23:25:05.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.921+0000 7ff9d7fff640 1 -- 172.21.15.33:0/828440532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff9e801d020 con 0x7ff9f010da80 2024-04-24T23:25:05.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.921+0000 7ff9d7fff640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9d80a2ce0 0x7ff9d80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9d7fff640 1 -- 172.21.15.33:0/828440532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9e80cf2d0 con 0x7ff9f010da80 2024-04-24T23:25:05.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 -- start start 2024-04-24T23:25:05.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 0x5574f57a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57aa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5574f57066b0 con 0x5574f57a7da0 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5574f56ae880 con 0x5574f57a2240 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.922+0000 7ff9f81db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574f5702670 con 0x5574f57a2c30 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9d80a2ce0 0x7ff9d80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 0x5574f57a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 0x5574f57a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34372/0 (socket says 172.21.15.33:34372) 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/2838944487 learned_addr learned my addr 172.21.15.33:0/2838944487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/2838944487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 msgr2=0x5574f57a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/2838944487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 msgr2=0x5574f57aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.923+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/2838944487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f54e47d0 con 0x5574f57a2c30 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.924+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 0x5574f57a7730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000c00 tx=0x7ff9e4030930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e63c67d976ba450 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.924+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9d80a2ce0 0x7ff9d80a51a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40057c0 tx=0x7ff9e403a000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.924+0000 7ff9d57fa640 1 -- 172.21.15.33:0/2838944487 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004480 con 0x5574f57a2c30 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.925+0000 7ff9d57fa640 1 -- 172.21.15.33:0/2838944487 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e404f070 con 0x5574f57a2c30 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.925+0000 7ff9d57fa640 1 -- 172.21.15.33:0/2838944487 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40326a0 con 0x5574f57a2c30 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.926+0000 7ff9f81db740 1 -- 172.21.15.33:0/2838944487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 msgr2=0x5574f57a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.926+0000 7ff9f81db740 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 0x5574f57a7730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000c00 tx=0x7ff9e4030930 comp rx=0 tx=0).stop 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.926+0000 7ff9f81db740 1 -- 172.21.15.33:0/2838944487 shutdown_connections 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2c30 0x5574f57a7730 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 --2- 172.21.15.33:0/2838944487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 -- 172.21.15.33:0/2838944487 >> 172.21.15.33:0/2838944487 conn(0x5574f579d8b0 msgr2=0x5574f579dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 -- 172.21.15.33:0/2838944487 shutdown_connections 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 -- 172.21.15.33:0/2838944487 wait complete. 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.927+0000 7ff9f81db740 1 -- start start 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 Processor -- start 2024-04-24T23:25:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5574f57aacf0 con 0x5574f57a2c30 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5574f56c1f50 con 0x5574f57a2240 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574f57aae70 con 0x5574f57a7da0 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9f81db740 10 client.0 _ll_register_callbacks cb 0x5574f56fd000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34378/0 (socket says 172.21.15.33:34378) 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/509802091 learned_addr learned my addr 172.21.15.33:0/509802091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.928+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.929+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/509802091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 msgr2=0x5574f57ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.929+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.929+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/509802091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 msgr2=0x5574f57acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.929+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.929+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/509802091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f541a3f0 con 0x5574f57a7da0 2024-04-24T23:25:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.929+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.930+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57b3d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80a3e80 tx=0x7ff9e801ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619c7f72cc761af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.930+0000 7ff9d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.931+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/509802091 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80b4060 con 0x5574f57a7da0 2024-04-24T23:25:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.931+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/509802091 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9e8004000 con 0x5574f57a7da0 2024-04-24T23:25:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.931+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e404f070 con 0x5574f57a7da0 2024-04-24T23:25:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.932+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/509802091 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9e80b9050 con 0x5574f57a7da0 2024-04-24T23:25:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.932+0000 7ff9f81db740 10 client.27595 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.932+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574f53da0f0 con 0x5574f57a7da0 2024-04-24T23:25:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/509802091 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e404f070 con 0x5574f57a7da0 2024-04-24T23:25:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/509802091 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff9e800b040 con 0x5574f57a7da0 2024-04-24T23:25:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9f81db740 10 client.27595 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9f81db740 10 client.27595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9f81db740 20 client.27595 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9f81db740 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f57ba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9f81db740 4 client.27595 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.933+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574f5713bf0 con 0x5574f57b8520 2024-04-24T23:25:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.934+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f57ba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.935+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f57ba970 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d79d8ee5f57975 server_cookie=9ad66818f9414c98 in_seq=0 out_seq=0 2024-04-24T23:25:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.935+0000 7ff9d2ff5640 10 client.27595 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/509802091 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574f5713bf0 con 0x5574f57b8520 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9d2ff5640 10 client.27595 handle_command_reply: tid=0 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 client.27595 shutdown 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 msgr2=0x5574f57b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57b3d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80a3e80 tx=0x7ff9e801ee80 comp rx=0 tx=0).stop 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 shutdown_connections 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f57ba970 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9d2ff5640 0 client.27595 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57b3d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 --2- 172.21.15.33:0/509802091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.936+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 >> 172.21.15.33:0/509802091 conn(0x5574f579d8b0 msgr2=0x5574f579caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.937+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 shutdown_connections 2024-04-24T23:25:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.937+0000 7ff9f81db740 1 -- 172.21.15.33:0/509802091 wait complete. 2024-04-24T23:25:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.937+0000 7ff9f81db740 20 client.27595 trim_cache size 0 max 16384 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.955+0000 7ff9f81db740 1 Processor -- start 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 -- start start 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01bba30 con 0x5574f57a2c30 2024-04-24T23:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b9300 con 0x5574f57a2240 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9f81db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b7150 con 0x5574f57a7da0 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54022/0 (socket says 172.21.15.33:54022) 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/1534475862 learned_addr learned my addr 172.21.15.33:0/1534475862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/1534475862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 msgr2=0x5574f57a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/1534475862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 msgr2=0x5574f57ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 -- 172.21.15.33:0/1534475862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f57baf50 con 0x5574f57a2c30 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.956+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57b3d00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x5574f57a36a0 tx=0x7ff9e4030670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ac1980b4ce754d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9d4ff9640 1 -- 172.21.15.33:0/1534475862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004480 con 0x5574f57a2c30 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9d4ff9640 1 -- 172.21.15.33:0/1534475862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e4058070 con 0x5574f57a2c30 2024-04-24T23:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9d4ff9640 1 -- 172.21.15.33:0/1534475862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004ab0 con 0x5574f57a2c30 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 -- 172.21.15.33:0/1534475862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 msgr2=0x5574f57b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57b3d00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x5574f57a36a0 tx=0x7ff9e4030670 comp rx=0 tx=0).stop 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 -- 172.21.15.33:0/1534475862 shutdown_connections 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a7da0 0x5574f57a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a2c30 0x5574f57b3d00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1534475862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2240 0x5574f57ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 -- 172.21.15.33:0/1534475862 >> 172.21.15.33:0/1534475862 conn(0x5574f579d8b0 msgr2=0x5574f57ac7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 -- 172.21.15.33:0/1534475862 shutdown_connections 2024-04-24T23:25:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.957+0000 7ff9f81db740 1 -- 172.21.15.33:0/1534475862 wait complete. 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 -- start start 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 Processor -- start 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2240 0x5574f57b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2c30 0x5574f57b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b77c0 con 0x5574f57a7da0 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b8c30 con 0x5574f57a2c30 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b7e90 con 0x5574f57a2240 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9f81db740 10 client.0 _ll_register_callbacks cb 0x7ff9e809b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2240 0x5574f57b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54026/0 (socket says 172.21.15.33:54026) 2024-04-24T23:25:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/1734871595 learned_addr learned my addr 172.21.15.33:0/1734871595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ee7fc640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2c30 0x5574f57b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2240 msgr2=0x5574f57b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2240 0x5574f57b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/1734871595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2c30 msgr2=0x5574f57b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2c30 0x5574f57b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9ef7fe640 1 -- 172.21.15.33:0/1734871595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f57aac30 con 0x5574f57a7da0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.958+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2240 0x5574f57b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9ef7fe640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57b2650 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff9e809b440 tx=0x7ff9e8004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b96f60179bbb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/1734871595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80b4040 con 0x5574f57a7da0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/1734871595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9e8003970 con 0x5574f57a7da0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e4058070 con 0x5574f57a7da0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/1734871595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9e801a040 con 0x5574f57a7da0 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 10 client.18135 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574f579e620 con 0x5574f57a7da0 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/1734871595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e4058070 con 0x5574f57a7da0 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/1734871595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff9e809a600 con 0x5574f57a7da0 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 10 client.18135 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 20 client.18135 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f5820ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 4 client.18135 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574f570df40 con 0x5574f57b8520 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.959+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f5820ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9d2ff5640 10 client.18135 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9eeffd640 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f5820ea0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af85344566436dd2 server_cookie=373fc9571c78741e in_seq=0 out_seq=0 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9d2ff5640 1 -- 172.21.15.33:0/1734871595 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574f570df40 con 0x5574f57b8520 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9d2ff5640 10 client.18135 handle_command_reply: tid=0 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 client.18135 shutdown 2024-04-24T23:25:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 msgr2=0x5574f57b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57b2650 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff9e809b440 tx=0x7ff9e8004420 comp rx=0 tx=0).stop 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 shutdown_connections 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5574f57b8520 0x5574f5820ea0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5574f57a7da0 0x5574f57b2650 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5574f57a2c30 0x5574f57b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 --2- 172.21.15.33:0/1734871595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574f57a2240 0x5574f57b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 >> 172.21.15.33:0/1734871595 conn(0x5574f579d8b0 msgr2=0x5574f579c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9d2ff5640 0 client.18135 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 shutdown_connections 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 1 -- 172.21.15.33:0/1734871595 wait complete. 2024-04-24T23:25:05.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.960+0000 7ff9f81db740 20 client.18135 trim_cache size 0 max 16384 2024-04-24T23:25:05.969 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:25:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:25:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:25:05.969 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:25:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9d80a2ce0 msgr2=0x7ff9d80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9d80a2ce0 0x7ff9d80a51a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40057c0 tx=0x7ff9e403a000 comp rx=0 tx=0).stop 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 msgr2=0x7ff9f01adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 0x7ff9f01adec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8005fd0 tx=0x7ff9e800c740 comp rx=0 tx=0).stop 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 shutdown_connections 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9d80a2ce0 0x7ff9d80a51a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f010da80 0x7ff9f01adec0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9f010b0b0 0x7ff9f01a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 --2- 172.21.15.33:0/828440532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9f0108740 0x7ff9f01a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 >> 172.21.15.33:0/828440532 conn(0x7ff9f006e5d0 msgr2=0x7ff9f0108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 shutdown_connections 2024-04-24T23:25:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:05.961+0000 7ff9f54cd640 1 -- 172.21.15.33:0/828440532 wait complete. 2024-04-24T23:25:05.973 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:25:05.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 -- 172.21.15.33:0/4248155184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 msgr2=0x7fa338101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 --2- 172.21.15.33:0/4248155184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa338101d10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009900 tx=0x7fa32c02a410 comp rx=0 tx=0).stop 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 -- 172.21.15.33:0/4248155184 shutdown_connections 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 --2- 172.21.15.33:0/4248155184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa338102250 0x7fa338108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 --2- 172.21.15.33:0/4248155184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa338101d10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 --2- 172.21.15.33:0/4248155184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 0x7fa338100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 -- 172.21.15.33:0/4248155184 >> 172.21.15.33:0/4248155184 conn(0x7fa3380fbc30 msgr2=0x7fa3380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 -- 172.21.15.33:0/4248155184 shutdown_connections 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.078+0000 7fa33ec64640 1 -- 172.21.15.33:0/4248155184 wait complete. 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 Processor -- start 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 -- start start 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 0x7fa338199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa33819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa338102250 0x7fa3381a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810c320 con 0x7fa338101890 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810c1a0 con 0x7fa338102250 2024-04-24T23:25:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ec64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810c4a0 con 0x7fa3381006a0 2024-04-24T23:25:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa337fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa33819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa337fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa33819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54048/0 (socket says 172.21.15.33:54048) 2024-04-24T23:25:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa337fff640 1 -- 172.21.15.33:0/968806128 learned_addr learned my addr 172.21.15.33:0/968806128 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33ca00640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 0x7fa338199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.079+0000 7fa33d201640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa338102250 0x7fa3381a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ca00640 1 -- 172.21.15.33:0/968806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa338102250 msgr2=0x7fa3381a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ca00640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa338102250 0x7fa3381a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ca00640 1 -- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 msgr2=0x7fa33819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ca00640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa33819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ca00640 1 -- 172.21.15.33:0/968806128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3381a1920 con 0x7fa3381006a0 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ca00640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 0x7fa338199d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa3381016f0 tx=0x7fa32400dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324017070 con 0x7fa3381006a0 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3381a1c10 con 0x7fa3381006a0 2024-04-24T23:25:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324010460 con 0x7fa3381006a0 2024-04-24T23:25:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32400f620 con 0x7fa3381006a0 2024-04-24T23:25:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.080+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a2110 con 0x7fa3381006a0 2024-04-24T23:25:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.081+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3381a22e0 con 0x7fa3381006a0 2024-04-24T23:25:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.082+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa32401a070 con 0x7fa3381006a0 2024-04-24T23:25:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.083+0000 7fa335ffb640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3280a2b90 0x7fa3280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.083+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa3240ced70 con 0x7fa3381006a0 2024-04-24T23:25:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.083+0000 7fa337fff640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3280a2b90 0x7fa3280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.084+0000 7fa337fff640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3280a2b90 0x7fa3280a5050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa32c031440 tx=0x7fa32c0315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.087+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa32409b2d0 con 0x7fa3381006a0 2024-04-24T23:25:06.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.373+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3381a24b0 con 0x7fa3381006a0 2024-04-24T23:25:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.375+0000 7fa335ffb640 1 -- 172.21.15.33:0/968806128 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7fa32409f5d0 con 0x7fa3381006a0 2024-04-24T23:25:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:25:06.376 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:25:06.377 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3280a2b90 msgr2=0x7fa3280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3280a2b90 0x7fa3280a5050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa32c031440 tx=0x7fa32c0315b0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 msgr2=0x7fa338199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 0x7fa338199d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa3381016f0 tx=0x7fa32400dbb0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 shutdown_connections 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa3280a2b90 0x7fa3280a5050 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa338102250 0x7fa3381a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa338101890 0x7fa33819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 --2- 172.21.15.33:0/968806128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3381006a0 0x7fa338199d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.380+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 >> 172.21.15.33:0/968806128 conn(0x7fa3380fbc30 msgr2=0x7fa338069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.381+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 shutdown_connections 2024-04-24T23:25:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.381+0000 7fa33ec64640 1 -- 172.21.15.33:0/968806128 wait complete. 2024-04-24T23:25:06.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:25:06.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1549521 12.33 MB/sec execute 551 sec latency 35.990 ms 2024-04-24T23:25:06.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 -- 172.21.15.33:0/2774980030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98640ffe10 msgr2=0x7f9864100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 --2- 172.21.15.33:0/2774980030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98640ffe10 0x7f9864100230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a410 comp rx=0 tx=0).stop 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 -- 172.21.15.33:0/2774980030 shutdown_connections 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 --2- 172.21.15.33:0/2774980030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9864101920 0x7f9864108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 --2- 172.21.15.33:0/2774980030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9864100f60 0x7f98641013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 --2- 172.21.15.33:0/2774980030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98640ffe10 0x7f9864100230 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 -- 172.21.15.33:0/2774980030 >> 172.21.15.33:0/2774980030 conn(0x7f98640fb380 msgr2=0x7f98640fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 -- 172.21.15.33:0/2774980030 shutdown_connections 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 -- 172.21.15.33:0/2774980030 wait complete. 2024-04-24T23:25:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.503+0000 7f986a04d640 1 Processor -- start 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 -- start start 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640ffe10 0x7f9864199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 0x7f986419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9864101920 0x7f98641a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f986410b220 con 0x7f9864100f60 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f986410b0a0 con 0x7f9864101920 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f986a04d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f986410b3a0 con 0x7f98640ffe10 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 0x7f986419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 0x7f986419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54068/0 (socket says 172.21.15.33:54068) 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 -- 172.21.15.33:0/3147201520 learned_addr learned my addr 172.21.15.33:0/3147201520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f98637fe640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640ffe10 0x7f9864199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9863fff640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9864101920 0x7f98641a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 -- 172.21.15.33:0/3147201520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640ffe10 msgr2=0x7f9864199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640ffe10 0x7f9864199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 -- 172.21.15.33:0/3147201520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9864101920 msgr2=0x7f98641a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9864101920 0x7f98641a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.504+0000 7f9862ffd640 1 -- 172.21.15.33:0/3147201520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98641a18f0 con 0x7f9864100f60 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.505+0000 7f9862ffd640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 0x7f986419a1f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f985400e9c0 tx=0x7f985400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.505+0000 7f9860ff9640 1 -- 172.21.15.33:0/3147201520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400cd80 con 0x7f9864100f60 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.505+0000 7f9860ff9640 1 -- 172.21.15.33:0/3147201520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98540035f0 con 0x7f9864100f60 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.505+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98641a1be0 con 0x7f9864100f60 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.505+0000 7f9860ff9640 1 -- 172.21.15.33:0/3147201520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854010690 con 0x7f9864100f60 2024-04-24T23:25:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.505+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a20e0 con 0x7f9864100f60 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.507+0000 7f986c9fd740 1 Processor -- start 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 -- start start 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c630e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 0x55cd0c635f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 0x55cd0c638a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9860ff9640 1 -- 172.21.15.33:0/3147201520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f98540037a0 con 0x7f9864100f60 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd0c58c670 con 0x55cd0c6365a0 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cd0c534970 con 0x55cd0c631430 2024-04-24T23:25:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f986c9fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cd0c588630 con 0x55cd0c630a40 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 0x55cd0c638a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f98637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 0x55cd0c635f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 0x55cd0c638a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54070/0 (socket says 172.21.15.33:54070) 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 -- 172.21.15.33:0/1952827044 learned_addr learned my addr 172.21.15.33:0/1952827044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f98637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 0x55cd0c635f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36418/0 (socket says 172.21.15.33:36418) 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9863fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c630e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 -- 172.21.15.33:0/1952827044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 msgr2=0x55cd0c630e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c630e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 -- 172.21.15.33:0/1952827044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 msgr2=0x55cd0c635f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 0x55cd0c635f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9862ffd640 1 -- 172.21.15.33:0/1952827044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0c558400 con 0x55cd0c6365a0 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f9863fff640 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c630e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.508+0000 7f98637fe640 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 0x55cd0c635f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9862ffd640 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 0x55cd0c638a30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f98540d5a80 tx=0x7f98540d5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4925c4c3f89c7e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9860ff9640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f984c0a2b90 0x7f984c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9863fff640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f984c0a2b90 0x7f984c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9849ffb640 1 -- 172.21.15.33:0/1952827044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540de070 con 0x55cd0c6365a0 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9860ff9640 1 -- 172.21.15.33:0/3147201520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f98540cf720 con 0x7f9864100f60 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9849ffb640 1 -- 172.21.15.33:0/1952827044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9854012eb0 con 0x55cd0c6365a0 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.509+0000 7f9849ffb640 1 -- 172.21.15.33:0/1952827044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985409b730 con 0x55cd0c6365a0 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 -- 172.21.15.33:0/1952827044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 msgr2=0x55cd0c638a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f9863fff640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f984c0a2b90 0x7f984c0a5050 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55cd0c631c90 tx=0x7f985c010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 0x55cd0c638a30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f98540d5a80 tx=0x7f98540d5b70 comp rx=0 tx=0).stop 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 -- 172.21.15.33:0/1952827044 shutdown_connections 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c6365a0 0x55cd0c638a30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c631430 0x55cd0c635f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1952827044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c630e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 -- 172.21.15.33:0/1952827044 >> 172.21.15.33:0/1952827044 conn(0x55cd0c6238a0 msgr2=0x55cd0c623ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 -- 172.21.15.33:0/1952827044 shutdown_connections 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 -- 172.21.15.33:0/1952827044 wait complete. 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.510+0000 7f986c9fd740 1 -- start start 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 Processor -- start 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 0x55cd0c63ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c631430 0x55cd0c63b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f98637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 0x55cd0c63ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f98637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 0x55cd0c63ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54080/0 (socket says 172.21.15.33:54080) 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c642080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f98637fe640 1 -- 172.21.15.33:0/2184396768 learned_addr learned my addr 172.21.15.33:0/2184396768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd0c639550 con 0x55cd0c630a40 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cd0c548050 con 0x55cd0c6365a0 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cd0c6396d0 con 0x55cd0c631430 2024-04-24T23:25:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9862ffd640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c631430 0x55cd0c63b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f986c9fd740 10 client.0 _ll_register_callbacks cb 0x55cd0c29fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9863fff640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c642080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9862ffd640 1 -- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 msgr2=0x55cd0c642080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9862ffd640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c642080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9862ffd640 1 -- 172.21.15.33:0/2184396768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 msgr2=0x55cd0c63ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9862ffd640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 0x55cd0c63ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f9862ffd640 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0c519dc0 con 0x55cd0c631430 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.511+0000 7f98637fe640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 0x55cd0c63ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f9862ffd640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c631430 0x55cd0c63b040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98540044e0 tx=0x7f98540096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ba848445bb88b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f9842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f9842ffd640 1 -- 172.21.15.33:0/2184396768 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540de030 con 0x55cd0c631430 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f9842ffd640 1 -- 172.21.15.33:0/2184396768 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9854003b10 con 0x55cd0c631430 2024-04-24T23:25:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd0c599bb0 con 0x55cd0c631430 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f9842ffd640 1 -- 172.21.15.33:0/2184396768 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f985401a0c0 con 0x55cd0c631430 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f986c9fd740 10 client.27619 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd0c2a03f0 con 0x55cd0c631430 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.512+0000 7f9842ffd640 1 -- 172.21.15.33:0/2184396768 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cd0c599bb0 con 0x55cd0c631430 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f9842ffd640 1 -- 172.21.15.33:0/2184396768 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9854012ad0 con 0x55cd0c631430 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f986c9fd740 10 client.27619 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f986c9fd740 10 client.27619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f986c9fd740 20 client.27619 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f986c9fd740 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c646550 0x55cd0c648940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f986c9fd740 4 client.27619 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd0c593f00 con 0x55cd0c646550 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.513+0000 7f98637fe640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c646550 0x55cd0c648940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.514+0000 7f9842ffd640 10 client.27619 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.514+0000 7f98637fe640 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c646550 0x55cd0c648940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4612fb39f840eb9 server_cookie=7fe1f7f219348f05 in_seq=0 out_seq=0 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.514+0000 7f9842ffd640 1 -- 172.21.15.33:0/2184396768 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd0c593f00 con 0x55cd0c646550 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.514+0000 7f9842ffd640 10 client.27619 handle_command_reply: tid=0 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.514+0000 7f986c9fd740 1 client.27619 shutdown 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c631430 msgr2=0x55cd0c63b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c631430 0x55cd0c63b040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98540044e0 tx=0x7f98540096e0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 shutdown_connections 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c646550 0x55cd0c648940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c642080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c631430 0x55cd0c63b040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 --2- 172.21.15.33:0/2184396768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c630a40 0x55cd0c63ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 >> 172.21.15.33:0/2184396768 conn(0x55cd0c6238a0 msgr2=0x55cd0c622b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f9842ffd640 0 client.27619 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 shutdown_connections 2024-04-24T23:25:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 1 -- 172.21.15.33:0/2184396768 wait complete. 2024-04-24T23:25:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.515+0000 7f986c9fd740 20 client.27619 trim_cache size 0 max 16384 2024-04-24T23:25:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 Processor -- start 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 -- start start 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c6229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c642040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c631cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98641aaa60 con 0x55cd0c631430 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98641a8330 con 0x55cd0c6365a0 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f986c9fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98641a6180 con 0x55cd0c630a40 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c642040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c642040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54094/0 (socket says 172.21.15.33:54094) 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.530+0000 7f9863fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c631cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 -- 172.21.15.33:0/804813957 learned_addr learned my addr 172.21.15.33:0/804813957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f98637fe640 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c6229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 -- 172.21.15.33:0/804813957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 msgr2=0x55cd0c6229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c6229c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 -- 172.21.15.33:0/804813957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 msgr2=0x55cd0c631cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c631cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 -- 172.21.15.33:0/804813957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0c36a7d0 con 0x55cd0c631430 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9863fff640 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c631cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f9862ffd640 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c642040 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f985400c6a0 tx=0x7f9854009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258f3644242b257b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f98497fa640 1 -- 172.21.15.33:0/804813957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540de070 con 0x55cd0c631430 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f98497fa640 1 -- 172.21.15.33:0/804813957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9854086040 con 0x55cd0c631430 2024-04-24T23:25:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.531+0000 7f98497fa640 1 -- 172.21.15.33:0/804813957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985409b550 con 0x55cd0c631430 2024-04-24T23:25:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 -- 172.21.15.33:0/804813957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 msgr2=0x55cd0c642040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c642040 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f985400c6a0 tx=0x7f9854009750 comp rx=0 tx=0).stop 2024-04-24T23:25:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 -- 172.21.15.33:0/804813957 shutdown_connections 2024-04-24T23:25:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c631cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c642040 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 --2- 172.21.15.33:0/804813957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c6229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 -- 172.21.15.33:0/804813957 >> 172.21.15.33:0/804813957 conn(0x55cd0c6238a0 msgr2=0x55cd0c6222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 -- 172.21.15.33:0/804813957 shutdown_connections 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 -- 172.21.15.33:0/804813957 wait complete. 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.532+0000 7f986c9fd740 1 -- start start 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 Processor -- start 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c63ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c63f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c6aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98641a67f0 con 0x55cd0c631430 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f98641a7c60 con 0x55cd0c6365a0 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98641a6ec0 con 0x55cd0c630a40 2024-04-24T23:25:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f986c9fd740 10 client.0 _ll_register_callbacks cb 0x7f9854005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c63f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c63f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54106/0 (socket says 172.21.15.33:54106) 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f98637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c63ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 -- 172.21.15.33:0/1139109725 learned_addr learned my addr 172.21.15.33:0/1139109725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9863fff640 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c6aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 -- 172.21.15.33:0/1139109725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 msgr2=0x55cd0c63ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c63ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 -- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 msgr2=0x55cd0c6aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c6aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.533+0000 7f9862ffd640 1 -- 172.21.15.33:0/1139109725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0c6422e0 con 0x55cd0c631430 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9862ffd640 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c63f280 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f98540d5840 tx=0x7f98540d5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6038b30f54061a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9842ffd640 1 -- 172.21.15.33:0/1139109725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540de070 con 0x55cd0c631430 2024-04-24T23:25:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9842ffd640 1 -- 172.21.15.33:0/1139109725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f985409be50 con 0x55cd0c631430 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9854086040 con 0x55cd0c631430 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9842ffd640 1 -- 172.21.15.33:0/1139109725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f985409b550 con 0x55cd0c631430 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 10 client.18159 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd0c6422e0 con 0x55cd0c631430 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9842ffd640 1 -- 172.21.15.33:0/1139109725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9854086040 con 0x55cd0c631430 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f9842ffd640 1 -- 172.21.15.33:0/1139109725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f98540cfb40 con 0x55cd0c631430 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 10 client.18159 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 10 client.18159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 20 client.18159 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c6ac4f0 0x55cd0c6ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 4 client.18159 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd0c511820 con 0x55cd0c6ac4f0 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.534+0000 7f98637fe640 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c6ac4f0 0x55cd0c6ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.535+0000 7f98637fe640 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c6ac4f0 0x55cd0c6ae8e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e52a80e8fe6af48 server_cookie=8706c4f7fec22dbc in_seq=0 out_seq=0 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.535+0000 7f9842ffd640 10 client.18159 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f9842ffd640 1 -- 172.21.15.33:0/1139109725 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd0c511820 con 0x55cd0c6ac4f0 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f9842ffd640 10 client.18159 handle_command_reply: tid=0 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 client.18159 shutdown 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 msgr2=0x55cd0c63f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c63f280 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f98540d5840 tx=0x7f98540d5b70 comp rx=0 tx=0).stop 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 shutdown_connections 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cd0c6ac4f0 0x55cd0c6ae8e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd0c6365a0 0x55cd0c6aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f9842ffd640 0 client.18159 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0c631430 0x55cd0c63f280 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 --2- 172.21.15.33:0/1139109725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd0c630a40 0x55cd0c63ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.536+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 >> 172.21.15.33:0/1139109725 conn(0x55cd0c6238a0 msgr2=0x55cd0c622b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 shutdown_connections 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986c9fd740 1 -- 172.21.15.33:0/1139109725 wait complete. 2024-04-24T23:25:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986c9fd740 20 client.18159 trim_cache size 0 max 16384 2024-04-24T23:25:06.543 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f984c0a2b90 msgr2=0x7f984c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f984c0a2b90 0x7f984c0a5050 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55cd0c631c90 tx=0x7f985c010040 comp rx=0 tx=0).stop 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 msgr2=0x7f986419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 0x7f986419a1f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f985400e9c0 tx=0x7f985400ee80 comp rx=0 tx=0).stop 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 shutdown_connections 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f984c0a2b90 0x7f984c0a5050 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9864101920 0x7f98641a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9864100f60 0x7f986419a1f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 --2- 172.21.15.33:0/3147201520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640ffe10 0x7f9864199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 >> 172.21.15.33:0/3147201520 conn(0x7f98640fb380 msgr2=0x7f98641042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.537+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 shutdown_connections 2024-04-24T23:25:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.538+0000 7f986a04d640 1 -- 172.21.15.33:0/3147201520 wait complete. 2024-04-24T23:25:06.550 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:25:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 -- 172.21.15.33:0/445873366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 msgr2=0x7f1390109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 --2- 172.21.15.33:0/445873366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f1390109bd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1388009ca0 tx=0x7f138802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 -- 172.21.15.33:0/445873366 shutdown_connections 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 --2- 172.21.15.33:0/445873366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f1390109bd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 --2- 172.21.15.33:0/445873366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390102980 0x7f1390102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 --2- 172.21.15.33:0/445873366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390101790 0x7f1390101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 -- 172.21.15.33:0/445873366 >> 172.21.15.33:0/445873366 conn(0x7f13900fcd20 msgr2=0x7f13900ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 -- 172.21.15.33:0/445873366 shutdown_connections 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.667+0000 7f139531d640 1 -- 172.21.15.33:0/445873366 wait complete. 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 Processor -- start 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 -- start start 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 0x7f1390197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 0x7f1390197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f139019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f139010d3f0 con 0x7f1390101790 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f139010d270 con 0x7f1390103340 2024-04-24T23:25:06.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f139531d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f139010d570 con 0x7f1390102980 2024-04-24T23:25:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f138effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 0x7f1390197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f1387fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 0x7f1390197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f1387fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 0x7f1390197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34464/0 (socket says 172.21.15.33:34464) 2024-04-24T23:25:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f1387fff640 1 -- 172.21.15.33:0/87046418 learned_addr learned my addr 172.21.15.33:0/87046418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f138effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 0x7f1390197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54116/0 (socket says 172.21.15.33:54116) 2024-04-24T23:25:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.668+0000 7f138f7fe640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f139019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138f7fe640 1 -- 172.21.15.33:0/87046418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 msgr2=0x7f1390197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138f7fe640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 0x7f1390197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138f7fe640 1 -- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 msgr2=0x7f1390197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138f7fe640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 0x7f1390197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138f7fe640 1 -- 172.21.15.33:0/87046418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139019f650 con 0x7f1390103340 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f1387fff640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 0x7f1390197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138effd640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 0x7f1390197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138f7fe640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f139019efe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1388031ae0 tx=0x7f1388004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138cff9640 1 -- 172.21.15.33:0/87046418 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138804b070 con 0x7f1390103340 2024-04-24T23:25:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138cff9640 1 -- 172.21.15.33:0/87046418 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1388003d80 con 0x7f1390103340 2024-04-24T23:25:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1390073970 con 0x7f1390103340 2024-04-24T23:25:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f138cff9640 1 -- 172.21.15.33:0/87046418 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388004810 con 0x7f1390103340 2024-04-24T23:25:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.669+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1390073e90 con 0x7f1390103340 2024-04-24T23:25:06.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f138cff9640 1 -- 172.21.15.33:0/87046418 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f138804e070 con 0x7f1390103340 2024-04-24T23:25:06.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 Processor -- start 2024-04-24T23:25:06.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 -- start start 2024-04-24T23:25:06.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b44f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b44fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b4501220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d7b445d670 con 0x55d7b44fed90 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d7b4405970 con 0x55d7b44f9230 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.672+0000 7f1398020740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7b4459630 con 0x55d7b44f9c20 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b4501220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b4501220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54130/0 (socket says 172.21.15.33:54130) 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 -- 172.21.15.33:0/2216213686 learned_addr learned my addr 172.21.15.33:0/2216213686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f138effd640 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b44fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b44f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f138cff9640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f137c0a2b90 0x7f137c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f138f7fe640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f137c0a2b90 0x7f137c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 -- 172.21.15.33:0/2216213686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 msgr2=0x55d7b44fe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b44fe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f138cff9640 1 -- 172.21.15.33:0/87046418 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f13880f3be0 con 0x7f1390103340 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 -- 172.21.15.33:0/2216213686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 msgr2=0x55d7b44f9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b44f9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 -- 172.21.15.33:0/2216213686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b4429400 con 0x55d7b44fed90 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f138effd640 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b44fe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f1387fff640 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b4501220 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1378011c70 tx=0x7f1378009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69a812138bfee9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.673+0000 7f138f7fe640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f137c0a2b90 0x7f137c0a5050 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1388009ca0 tx=0x7f13880b5740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f13857fa640 1 -- 172.21.15.33:0/2216213686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137800fc50 con 0x55d7b44fed90 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f13857fa640 1 -- 172.21.15.33:0/2216213686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137800fdf0 con 0x55d7b44fed90 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f13857fa640 1 -- 172.21.15.33:0/2216213686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137801d5a0 con 0x55d7b44fed90 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f1398020740 1 -- 172.21.15.33:0/2216213686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 msgr2=0x55d7b4501220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f1398020740 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b4501220 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1378011c70 tx=0x7f1378009c60 comp rx=0 tx=0).stop 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f1398020740 1 -- 172.21.15.33:0/2216213686 shutdown_connections 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.674+0000 7f1398020740 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b4501220 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b44fe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 --2- 172.21.15.33:0/2216213686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b44f9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- 172.21.15.33:0/2216213686 >> 172.21.15.33:0/2216213686 conn(0x55d7b44f48a0 msgr2=0x55d7b44f4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- 172.21.15.33:0/2216213686 shutdown_connections 2024-04-24T23:25:06.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- 172.21.15.33:0/2216213686 wait complete. 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- start start 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 Processor -- start 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4503390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b450a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d7b4501d40 con 0x55d7b44fed90 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d7b4419050 con 0x55d7b44f9230 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.675+0000 7f1398020740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7b4501ec0 con 0x55d7b44f9c20 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f1398020740 10 client.0 _ll_register_callbacks cb 0x55d7b4170be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45038d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34486/0 (socket says 172.21.15.33:34486) 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f1387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b450a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 -- 172.21.15.33:0/2139443975 learned_addr learned my addr 172.21.15.33:0/2139443975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f1387fff640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4503390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 -- 172.21.15.33:0/2139443975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 msgr2=0x55d7b4503390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4503390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 -- 172.21.15.33:0/2139443975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 msgr2=0x55d7b450a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b450a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 -- 172.21.15.33:0/2139443975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b43eadc0 con 0x55d7b44f9c20 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f1387fff640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b450a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f138effd640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45038d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1380012d00 tx=0x7f138000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4826a0351f26beed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.676+0000 7f13767fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.677+0000 7f13767fc640 1 -- 172.21.15.33:0/2139443975 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380009e50 con 0x55d7b44f9c20 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.677+0000 7f13767fc640 1 -- 172.21.15.33:0/2139443975 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1380012110 con 0x55d7b44f9c20 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.677+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f137800fdf0 con 0x55d7b44f9c20 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.677+0000 7f13767fc640 1 -- 172.21.15.33:0/2139443975 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1380025050 con 0x55d7b44f9c20 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.677+0000 7f1398020740 10 client.27637 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.677+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7b41713f0 con 0x55d7b44f9c20 2024-04-24T23:25:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f13767fc640 1 -- 172.21.15.33:0/2139443975 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137800fdf0 con 0x55d7b44f9c20 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f13767fc640 1 -- 172.21.15.33:0/2139443975 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1380003cb0 con 0x55d7b44f9c20 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1398020740 10 client.27637 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1398020740 10 client.27637 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1398020740 20 client.27637 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1398020740 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4520290 0x55d7b45226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1398020740 4 client.27637 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7b446abb0 con 0x55d7b4520290 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.678+0000 7f1387fff640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4520290 0x55d7b45226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.679+0000 7f1387fff640 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4520290 0x55d7b45226f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1095be156bc6b01 server_cookie=315fd99b52102f65 in_seq=0 out_seq=0 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.679+0000 7f13767fc640 10 client.27637 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.679+0000 7f13767fc640 1 -- 172.21.15.33:0/2139443975 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7b446abb0 con 0x55d7b4520290 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.679+0000 7f13767fc640 10 client.27637 handle_command_reply: tid=0 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.679+0000 7f1398020740 1 client.27637 shutdown 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 msgr2=0x55d7b45038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45038d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1380012d00 tx=0x7f138000f790 comp rx=0 tx=0).stop 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 shutdown_connections 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4520290 0x55d7b45226f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b450a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45038d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 --2- 172.21.15.33:0/2139443975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4503390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f13767fc640 0 client.27637 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 >> 172.21.15.33:0/2139443975 conn(0x55d7b44f48a0 msgr2=0x55d7b44f3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 shutdown_connections 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 1 -- 172.21.15.33:0/2139443975 wait complete. 2024-04-24T23:25:06.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.680+0000 7f1398020740 20 client.27637 trim_cache size 0 max 16384 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.694+0000 7f1398020740 1 Processor -- start 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.694+0000 7f1398020740 1 -- start start 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1398020740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9230 0x55d7b45029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1398020740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 0x55d7b450a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1398020740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b44fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1398020740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13901a2120 con 0x55d7b44fed90 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1398020740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f139019f9f0 con 0x55d7b44f9c20 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1398020740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1390071930 con 0x55d7b44f9230 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b44fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f138effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 0x55d7b450a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f138effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 0x55d7b450a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36516/0 (socket says 172.21.15.33:36516) 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b44fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54158/0 (socket says 172.21.15.33:54158) 2024-04-24T23:25:06.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f138effd640 1 -- 172.21.15.33:0/1629640211 learned_addr learned my addr 172.21.15.33:0/1629640211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9230 0x55d7b45029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 -- 172.21.15.33:0/1629640211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9230 msgr2=0x55d7b45029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9230 0x55d7b45029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 -- 172.21.15.33:0/1629640211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 msgr2=0x55d7b450a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 0x55d7b450a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 -- 172.21.15.33:0/1629640211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b423b7d0 con 0x55d7b44fed90 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f138effd640 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 0x55d7b450a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1387fff640 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b44fa4e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1378011880 tx=0x7f1378009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fbb36c5593ccf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.695+0000 7f1384ff9640 1 -- 172.21.15.33:0/1629640211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137800fe00 con 0x55d7b44fed90 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1384ff9640 1 -- 172.21.15.33:0/1629640211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137802d890 con 0x55d7b44fed90 2024-04-24T23:25:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1384ff9640 1 -- 172.21.15.33:0/1629640211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378004c30 con 0x55d7b44fed90 2024-04-24T23:25:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 -- 172.21.15.33:0/1629640211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 msgr2=0x55d7b44fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b44fa4e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1378011880 tx=0x7f1378009780 comp rx=0 tx=0).stop 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 -- 172.21.15.33:0/1629640211 shutdown_connections 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b44fa4e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9c20 0x55d7b450a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 --2- 172.21.15.33:0/1629640211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9230 0x55d7b45029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 -- 172.21.15.33:0/1629640211 >> 172.21.15.33:0/1629640211 conn(0x55d7b44f48a0 msgr2=0x55d7b44f3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 -- 172.21.15.33:0/1629640211 shutdown_connections 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.696+0000 7f1398020740 1 -- 172.21.15.33:0/1629640211 wait complete. 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 -- start start 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 Processor -- start 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4507d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b45092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1390071fa0 con 0x55d7b44fed90 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1390073410 con 0x55d7b44f9230 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1390072670 con 0x55d7b44f9c20 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1398020740 10 client.0 _ll_register_callbacks cb 0x7f1380009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1387fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4507d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34506/0 (socket says 172.21.15.33:34506) 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 -- 172.21.15.33:0/3066222184 learned_addr learned my addr 172.21.15.33:0/3066222184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f1387fff640 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b45092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 -- 172.21.15.33:0/3066222184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 msgr2=0x55d7b4507d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4507d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 -- 172.21.15.33:0/3066222184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 msgr2=0x55d7b45092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b45092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.697+0000 7f138effd640 1 -- 172.21.15.33:0/3066222184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b45098a0 con 0x55d7b44f9c20 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f138effd640 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45082a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1380012d00 tx=0x7f1380009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8be0efaa89e96d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f13767fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f13767fc640 1 -- 172.21.15.33:0/3066222184 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138002d070 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f137802d890 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f13767fc640 1 -- 172.21.15.33:0/3066222184 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f13800050e0 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f13767fc640 1 -- 172.21.15.33:0/3066222184 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1380039050 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f1398020740 10 client.27643 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.698+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7b450ac40 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f13767fc640 1 -- 172.21.15.33:0/3066222184 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137802d890 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f13767fc640 1 -- 172.21.15.33:0/3066222184 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f13800035f0 con 0x55d7b44f9c20 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 10 client.27643 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 10 client.27643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 20 client.27643 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4587070 0x55d7b4589490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 4 client.27643 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7b4503180 con 0x55d7b4587070 2024-04-24T23:25:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1387fff640 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4587070 0x55d7b4589490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f13767fc640 10 client.27643 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1387fff640 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4587070 0x55d7b4589490 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f2c9f2b985a4ad2 server_cookie=e4a2d88cebe63ac5 in_seq=0 out_seq=0 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f13767fc640 1 -- 172.21.15.33:0/3066222184 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7b4503180 con 0x55d7b4587070 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f13767fc640 10 client.27643 handle_command_reply: tid=0 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.699+0000 7f1398020740 1 client.27643 shutdown 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 msgr2=0x55d7b45082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45082a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1380012d00 tx=0x7f1380009ec0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 shutdown_connections 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d7b4587070 0x55d7b4589490 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7b44fed90 0x55d7b45092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f13767fc640 0 client.27643 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7b44f9c20 0x55d7b45082a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 --2- 172.21.15.33:0/3066222184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7b44f9230 0x55d7b4507d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 >> 172.21.15.33:0/3066222184 conn(0x55d7b44f48a0 msgr2=0x55d7b44f7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 shutdown_connections 2024-04-24T23:25:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 1 -- 172.21.15.33:0/3066222184 wait complete. 2024-04-24T23:25:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f1398020740 20 client.27643 trim_cache size 0 max 16384 2024-04-24T23:25:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f137c0a2b90 msgr2=0x7f137c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f137c0a2b90 0x7f137c0a5050 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1388009ca0 tx=0x7f13880b5740 comp rx=0 tx=0).stop 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 msgr2=0x7f139019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f139019efe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1388031ae0 tx=0x7f1388004ab0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 shutdown_connections 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f137c0a2b90 0x7f137c0a5050 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1390103340 0x7f139019efe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1390102980 0x7f1390197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 --2- 172.21.15.33:0/87046418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1390101790 0x7f1390197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 >> 172.21.15.33:0/87046418 conn(0x7f13900fcd20 msgr2=0x7f1390105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.700+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 shutdown_connections 2024-04-24T23:25:06.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.701+0000 7f139531d640 1 -- 172.21.15.33:0/87046418 wait complete. 2024-04-24T23:25:06.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:25:06.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.824+0000 7f3d148bd640 1 -- 172.21.15.33:0/387924535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 msgr2=0x7f3d10101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.824+0000 7f3d148bd640 1 --2- 172.21.15.33:0/387924535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d10101d10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009900 tx=0x7f3cf802a410 comp rx=0 tx=0).stop 2024-04-24T23:25:06.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 -- 172.21.15.33:0/387924535 shutdown_connections 2024-04-24T23:25:06.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 --2- 172.21.15.33:0/387924535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d10102250 0x7f3d10108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 --2- 172.21.15.33:0/387924535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d10101d10 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 --2- 172.21.15.33:0/387924535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d101006a0 0x7f3d10100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 -- 172.21.15.33:0/387924535 >> 172.21.15.33:0/387924535 conn(0x7f3d100fbc30 msgr2=0x7f3d100fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 -- 172.21.15.33:0/387924535 shutdown_connections 2024-04-24T23:25:06.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 -- 172.21.15.33:0/387924535 wait complete. 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 Processor -- start 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.825+0000 7f3d148bd640 1 -- start start 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d101006a0 0x7f3d10199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d1019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10102250 0x7f3d101a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010f670 con 0x7f3d10101890 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010f4f0 con 0x7f3d101006a0 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010f7f0 con 0x7f3d10102250 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d1019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d1019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54190/0 (socket says 172.21.15.33:54190) 2024-04-24T23:25:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d101006a0 0x7f3d10199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0dd9b640 1 -- 172.21.15.33:0/3602530387 learned_addr learned my addr 172.21.15.33:0/3602530387 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10102250 0x7f3d101a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 -- 172.21.15.33:0/3602530387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10102250 msgr2=0x7f3d101a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10102250 0x7f3d101a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 -- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 msgr2=0x7f3d1019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d1019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 -- 172.21.15.33:0/3602530387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d101a18f0 con 0x7f3d101006a0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d101006a0 0x7f3d10199cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d00005fd0 tx=0x7f3d0000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d077fe640 1 -- 172.21.15.33:0/3602530387 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00003970 con 0x7f3d101006a0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.826+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d101a1be0 con 0x7f3d101006a0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.827+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a2130 con 0x7f3d101006a0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.827+0000 7f3d077fe640 1 -- 172.21.15.33:0/3602530387 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00004020 con 0x7f3d101006a0 2024-04-24T23:25:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.827+0000 7f3d077fe640 1 -- 172.21.15.33:0/3602530387 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00010690 con 0x7f3d101006a0 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 Processor -- start 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 -- start start 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c55720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c58220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580f3bb4670 con 0x5580f3c50230 2024-04-24T23:25:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5580f3b5c970 con 0x5580f3c55d90 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.829+0000 7f3d175d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5580f3bb0630 con 0x5580f3c50c20 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d077fe640 1 -- 172.21.15.33:0/3602530387 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3d00003b10 con 0x7f3d101006a0 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c50650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54204/0 (socket says 172.21.15.33:54204) 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/2936006659 learned_addr learned my addr 172.21.15.33:0/2936006659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c58220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c55720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 -- 172.21.15.33:0/2936006659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 msgr2=0x5580f3c55720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c55720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 -- 172.21.15.33:0/2936006659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 msgr2=0x5580f3c50650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c50650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 -- 172.21.15.33:0/2936006659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580f3b80400 con 0x5580f3c55d90 2024-04-24T23:25:06.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c55720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c58220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8006010 tx=0x7f3cf8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a65817a529ae2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.830+0000 7f3d077fe640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cfc0a2b70 0x7f3cfc0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d04ff9640 1 -- 172.21.15.33:0/2936006659 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8040070 con 0x5580f3c55d90 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d04ff9640 1 -- 172.21.15.33:0/2936006659 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8005500 con 0x5580f3c55d90 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d04ff9640 1 -- 172.21.15.33:0/2936006659 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8035040 con 0x5580f3c55d90 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cfc0a2b70 0x7f3cfc0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d077fe640 1 -- 172.21.15.33:0/3602530387 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3d000cfa80 con 0x7f3d101006a0 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d175d5740 1 -- 172.21.15.33:0/2936006659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 msgr2=0x5580f3c58220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cfc0a2b70 0x7f3cfc0a5030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x5580f3c51480 tx=0x7f3d08005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.831+0000 7f3d175d5740 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c58220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8006010 tx=0x7f3cf8004040 comp rx=0 tx=0).stop 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- 172.21.15.33:0/2936006659 shutdown_connections 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c58220 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c55720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 --2- 172.21.15.33:0/2936006659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- 172.21.15.33:0/2936006659 >> 172.21.15.33:0/2936006659 conn(0x5580f3c4b8a0 msgr2=0x5580f3c4bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- 172.21.15.33:0/2936006659 shutdown_connections 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- 172.21.15.33:0/2936006659 wait complete. 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- start start 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 Processor -- start 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50230 0x5580f3c5a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 0x5580f3c618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580f3c58d30 con 0x5580f3c55d90 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5580f3b70050 con 0x5580f3c50c20 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5580f3c58eb0 con 0x5580f3c50230 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d175d5740 10 client.0 _ll_register_callbacks cb 0x5580f38c7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 0x5580f3c618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 0x5580f3c618a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54210/0 (socket says 172.21.15.33:54210) 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/1552881460 learned_addr learned my addr 172.21.15.33:0/1552881460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.832+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50230 0x5580f3c5a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50230 msgr2=0x5580f3c5a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50230 0x5580f3c5a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/1552881460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 msgr2=0x5580f3c5a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/1552881460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580f3b41dc0 con 0x5580f3c55d90 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50230 0x5580f3c5a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 0x5580f3c618a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800f9d0 tx=0x7f3d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d494942b7d57b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/1552881460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08005020 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf8005500 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/1552881460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3d08018030 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/1552881460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d08033050 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d175d5740 10 client.18189 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5580f38c83f0 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.833+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/1552881460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8005500 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/1552881460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3d0801d620 con 0x5580f3c55d90 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d175d5740 10 client.18189 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d175d5740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d175d5740 20 client.18189 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d175d5740 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3c662c0 0x5580f3c68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d175d5740 4 client.18189 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580f3bc1bb0 con 0x5580f3c662c0 2024-04-24T23:25:06.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3c662c0 0x5580f3c68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3cf5ffb640 10 client.18189 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.834+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3c662c0 0x5580f3c68760 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f7bb1fd25fef16 server_cookie=47ffea6544ecc0a5 in_seq=0 out_seq=0 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/1552881460 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580f3bc1bb0 con 0x5580f3c662c0 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3cf5ffb640 10 client.18189 handle_command_reply: tid=0 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 client.18189 shutdown 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 msgr2=0x5580f3c618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 0x5580f3c618a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800f9d0 tx=0x7f3d080046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 shutdown_connections 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3c662c0 0x5580f3c68760 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c55d90 0x5580f3c618a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3cf5ffb640 0 client.18189 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 --2- 172.21.15.33:0/1552881460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50230 0x5580f3c5a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 >> 172.21.15.33:0/1552881460 conn(0x5580f3c4b8a0 msgr2=0x5580f3c4ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 shutdown_connections 2024-04-24T23:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.835+0000 7f3d175d5740 1 -- 172.21.15.33:0/1552881460 wait complete. 2024-04-24T23:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.836+0000 7f3d175d5740 20 client.18189 trim_cache size 0 max 16384 2024-04-24T23:25:06.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 Processor -- start 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 -- start start 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0012490 con 0x5580f3c50230 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf000fd60 con 0x5580f3c55d90 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.850+0000 7f3d175d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf000dbb0 con 0x5580f3c50c20 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c4e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54214/0 (socket says 172.21.15.33:54214) 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0e59c640 1 -- 172.21.15.33:0/3537287040 learned_addr learned my addr 172.21.15.33:0/3537287040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/3537287040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 msgr2=0x5580f3c61860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c61860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/3537287040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 msgr2=0x5580f3c4e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c4e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 -- 172.21.15.33:0/3537287040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580f39927d0 con 0x5580f3c55d90 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c4ecf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d08019040 tx=0x7f3d0800fe90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16c28b4821bdb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3cf7fff640 1 -- 172.21.15.33:0/3537287040 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0800f790 con 0x5580f3c55d90 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3cf7fff640 1 -- 172.21.15.33:0/3537287040 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d08004c10 con 0x5580f3c55d90 2024-04-24T23:25:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.851+0000 7f3cf7fff640 1 -- 172.21.15.33:0/3537287040 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08005680 con 0x5580f3c55d90 2024-04-24T23:25:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 -- 172.21.15.33:0/3537287040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 msgr2=0x5580f3c4ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c4ecf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d08019040 tx=0x7f3d0800fe90 comp rx=0 tx=0).stop 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 -- 172.21.15.33:0/3537287040 shutdown_connections 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c55d90 0x5580f3c4ecf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c50c20 0x5580f3c61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 --2- 172.21.15.33:0/3537287040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 -- 172.21.15.33:0/3537287040 >> 172.21.15.33:0/3537287040 conn(0x5580f3c4b8a0 msgr2=0x5580f3c4ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 -- 172.21.15.33:0/3537287040 shutdown_connections 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.852+0000 7f3d175d5740 1 -- 172.21.15.33:0/3537287040 wait complete. 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 -- start start 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 Processor -- start 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c55d90 0x5580f3c605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf000e220 con 0x5580f3c50230 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf000f690 con 0x5580f3c50c20 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf000e8f0 con 0x5580f3c55d90 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d175d5740 10 client.0 _ll_register_callbacks cb 0x7f3d08009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c5f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54224/0 (socket says 172.21.15.33:54224) 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 -- 172.21.15.33:0/619977290 learned_addr learned my addr 172.21.15.33:0/619977290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0ed9d640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c55d90 0x5580f3c605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 -- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c55d90 msgr2=0x5580f3c605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c55d90 0x5580f3c605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 -- 172.21.15.33:0/619977290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 msgr2=0x5580f3c5f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 -- 172.21.15.33:0/619977290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580f3c60be0 con 0x5580f3c50230 2024-04-24T23:25:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3d0e59c640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c5f0f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3d000cd560 tx=0x7f3d000a0390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32bad84a46dbac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.853+0000 7f3cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/619977290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0009bf50 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/619977290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3d0009a250 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d08004c10 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/619977290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d000985c0 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 10 client.18195 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5580f3cccb00 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/619977290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d08004c10 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/619977290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3d0009b3a0 con 0x5580f3c50230 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 10 client.18195 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 20 client.18195 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3ccd4d0 0x5580f3ccf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 4 client.18195 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580f3c55b00 con 0x5580f3ccd4d0 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.854+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3ccd4d0 0x5580f3ccf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3cf5ffb640 10 client.18195 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d0dd9b640 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3ccd4d0 0x5580f3ccf8f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c59834b40105787 server_cookie=df73263083d792dc in_seq=0 out_seq=0 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3cf5ffb640 1 -- 172.21.15.33:0/619977290 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580f3c55b00 con 0x5580f3ccd4d0 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3cf5ffb640 10 client.18195 handle_command_reply: tid=0 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 client.18195 shutdown 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 msgr2=0x5580f3c5f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c5f0f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3d000cd560 tx=0x7f3d000a0390 comp rx=0 tx=0).stop 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 shutdown_connections 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580f3ccd4d0 0x5580f3ccf8f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580f3c55d90 0x5580f3c605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580f3c50c20 0x5580f3c5f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 --2- 172.21.15.33:0/619977290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580f3c50230 0x5580f3c5f0f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3cf5ffb640 0 client.18195 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 >> 172.21.15.33:0/619977290 conn(0x5580f3c4b8a0 msgr2=0x5580f3c4ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 shutdown_connections 2024-04-24T23:25:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 1 -- 172.21.15.33:0/619977290 wait complete. 2024-04-24T23:25:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.855+0000 7f3d175d5740 20 client.18195 trim_cache size 0 max 16384 2024-04-24T23:25:06.863 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cfc0a2b70 msgr2=0x7f3cfc0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cfc0a2b70 0x7f3cfc0a5030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x5580f3c51480 tx=0x7f3d08005f90 comp rx=0 tx=0).stop 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d101006a0 msgr2=0x7f3d10199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d101006a0 0x7f3d10199cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d00005fd0 tx=0x7f3d0000ef90 comp rx=0 tx=0).stop 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 shutdown_connections 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3cfc0a2b70 0x7f3cfc0a5030 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d10102250 0x7f3d101a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d10101890 0x7f3d1019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 --2- 172.21.15.33:0/3602530387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d101006a0 0x7f3d10199cb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 >> 172.21.15.33:0/3602530387 conn(0x7f3d100fbc30 msgr2=0x7f3d10104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 shutdown_connections 2024-04-24T23:25:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:06.856+0000 7f3d148bd640 1 -- 172.21.15.33:0/3602530387 wait complete. 2024-04-24T23:25:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:07 smithi038 ceph-mon[32450]: pgmap v744: 129 pgs: 129 active+clean; 283 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-04-24T23:25:07.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:07 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/968806128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:25:07.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:07 smithi117 ceph-mon[32529]: pgmap v744: 129 pgs: 129 active+clean; 283 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-04-24T23:25:07.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:07 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/968806128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:25:07.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:07 smithi033 ceph-mon[29201]: pgmap v744: 129 pgs: 129 active+clean; 283 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-04-24T23:25:07.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:07 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/968806128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:25:07.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1551749 12.33 MB/sec execute 552 sec latency 57.422 ms 2024-04-24T23:25:07.868 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:25:07.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:07.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:25:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 -- 172.21.15.33:0/2370675902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074100680 msgr2=0x7f2074100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 --2- 172.21.15.33:0/2370675902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074100680 0x7f2074100aa0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2070009900 tx=0x7f207002a410 comp rx=0 tx=0).stop 2024-04-24T23:25:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 -- 172.21.15.33:0/2370675902 shutdown_connections 2024-04-24T23:25:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 --2- 172.21.15.33:0/2370675902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074102230 0x7f2074108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 --2- 172.21.15.33:0/2370675902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074101870 0x7f2074101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 --2- 172.21.15.33:0/2370675902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074100680 0x7f2074100aa0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 -- 172.21.15.33:0/2370675902 >> 172.21.15.33:0/2370675902 conn(0x7f20740fbbf0 msgr2=0x7f20740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 -- 172.21.15.33:0/2370675902 shutdown_connections 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.991+0000 7f207b438640 1 -- 172.21.15.33:0/2370675902 wait complete. 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 Processor -- start 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 -- start start 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 0x7f2074069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074101870 0x7f207419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 0x7f207419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f207410c2d0 con 0x7f2074102230 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f207410c150 con 0x7f2074100680 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f207b438640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f207410c450 con 0x7f2074101870 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20799d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 0x7f207419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 0x7f2074069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 0x7f2074069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36594/0 (socket says 172.21.15.33:36594) 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20799d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 0x7f207419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54230/0 (socket says 172.21.15.33:54230) 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 -- 172.21.15.33:0/418024430 learned_addr learned my addr 172.21.15.33:0/418024430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20789d3640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074101870 0x7f207419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 -- 172.21.15.33:0/418024430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074101870 msgr2=0x7f207419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074101870 0x7f207419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 -- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 msgr2=0x7f207419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 0x7f207419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20791d4640 1 -- 172.21.15.33:0/418024430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207419f760 con 0x7f2074100680 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.992+0000 7f20799d5640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 0x7f207419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.993+0000 7f20791d4640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 0x7f2074069440 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2070031610 tx=0x7f2070004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.993+0000 7f206a7fc640 1 -- 172.21.15.33:0/418024430 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207003f030 con 0x7f2074100680 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.993+0000 7f206a7fc640 1 -- 172.21.15.33:0/418024430 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070003ad0 con 0x7f2074100680 2024-04-24T23:25:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.993+0000 7f206a7fc640 1 -- 172.21.15.33:0/418024430 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070035070 con 0x7f2074100680 2024-04-24T23:25:07.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.993+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207419f990 con 0x7f2074100680 2024-04-24T23:25:07.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.993+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207419fe30 con 0x7f2074100680 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.995+0000 7f207e229740 1 Processor -- start 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 -- start start 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b74681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b746d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b746fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5645b73cb2c0 con 0x5645b7468780 2024-04-24T23:25:08.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5645b7373910 con 0x5645b7467d90 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f207e229740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5645b73c7280 con 0x5645b746d8f0 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f206a7fc640 1 -- 172.21.15.33:0/418024430 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2070042030 con 0x7f2074100680 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b74681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20791d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b746d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b74681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36598/0 (socket says 172.21.15.33:36598) 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 -- 172.21.15.33:0/2823676658 learned_addr learned my addr 172.21.15.33:0/2823676658 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20789d3640 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b746fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 -- 172.21.15.33:0/2823676658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 msgr2=0x5645b746fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b746fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 -- 172.21.15.33:0/2823676658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 msgr2=0x5645b746d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b746d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20799d5640 1 -- 172.21.15.33:0/2823676658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b7384fb0 con 0x5645b7467d90 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20791d4640 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b746d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.996+0000 7f20789d3640 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b746fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f206a7fc640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20600a2b90 0x7f20600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f20799d5640 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b74681b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f206c0107e0 tx=0x7f206c010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a13a2b6f62486 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f20791d4640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20600a2b90 0x7f20600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f205f7fe640 1 -- 172.21.15.33:0/2823676658 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c003b70 con 0x5645b7467d90 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f206a7fc640 1 -- 172.21.15.33:0/418024430 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20700efa50 con 0x7f2074100680 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f205f7fe640 1 -- 172.21.15.33:0/2823676658 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c00ee10 con 0x5645b7467d90 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.997+0000 7f205f7fe640 1 -- 172.21.15.33:0/2823676658 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c011440 con 0x5645b7467d90 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- 172.21.15.33:0/2823676658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 msgr2=0x5645b74681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f20791d4640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20600a2b90 0x7f20600a5050 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f20700ed530 tx=0x7f20700f4040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b74681b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f206c0107e0 tx=0x7f206c010ca0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- 172.21.15.33:0/2823676658 shutdown_connections 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b746fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b746d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- 172.21.15.33:0/2823676658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b74681b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- 172.21.15.33:0/2823676658 >> 172.21.15.33:0/2823676658 conn(0x5645b742f950 msgr2=0x5645b7463810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- 172.21.15.33:0/2823676658 shutdown_connections 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- 172.21.15.33:0/2823676658 wait complete. 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- start start 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 Processor -- start 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b7467d90 0x5645b74722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7468780 0x5645b7472820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 0x5645b7479860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5645b74708a0 con 0x5645b746d8f0 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5645b7386e90 con 0x5645b7468780 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.998+0000 7f207e229740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5645b7470a20 con 0x5645b7467d90 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f207e229740 10 client.0 _ll_register_callbacks cb 0x5645b73c2970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 0x5645b7479860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 0x5645b7479860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54244/0 (socket says 172.21.15.33:54244) 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 -- 172.21.15.33:0/1928101173 learned_addr learned my addr 172.21.15.33:0/1928101173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20789d3640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7468780 0x5645b7472820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b7467d90 0x5645b74722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 -- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b7467d90 msgr2=0x5645b74722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b7467d90 0x5645b74722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 -- 172.21.15.33:0/1928101173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7468780 msgr2=0x5645b7472820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7468780 0x5645b7472820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 -- 172.21.15.33:0/1928101173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b71a77e0 con 0x5645b746d8f0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f20799d5640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 0x5645b7479860 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f206c004060 tx=0x7f206c0040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b580822539c196b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f205cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f205cff9640 1 -- 172.21.15.33:0/1928101173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c025040 con 0x5645b746d8f0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f205cff9640 1 -- 172.21.15.33:0/1928101173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f206c0047e0 con 0x5645b746d8f0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f206c00ee10 con 0x5645b746d8f0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f205cff9640 1 -- 172.21.15.33:0/1928101173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f206c031050 con 0x5645b746d8f0 2024-04-24T23:25:08.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f207e229740 10 client.18207 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:07.999+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645b7207f70 con 0x5645b746d8f0 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f205cff9640 1 -- 172.21.15.33:0/1928101173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c00ee10 con 0x5645b746d8f0 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f205cff9640 1 -- 172.21.15.33:0/1928101173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f206c01a5a0 con 0x5645b746d8f0 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f207e229740 10 client.18207 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f207e229740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f207e229740 20 client.18207 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f207e229740 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b747db90 0x5645b747ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f207e229740 4 client.18207 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b739f5a0 con 0x5645b747db90 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f20789d3640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b747db90 0x5645b747ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f20789d3640 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b747db90 0x5645b747ffb0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90a771a77fc9d9 server_cookie=bfb8acc86bcf5d4e in_seq=0 out_seq=0 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.000+0000 7f205cff9640 10 client.18207 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f205cff9640 1 -- 172.21.15.33:0/1928101173 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645b739f5a0 con 0x5645b747db90 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f205cff9640 10 client.18207 handle_command_reply: tid=0 2024-04-24T23:25:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 client.18207 shutdown 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 msgr2=0x5645b7479860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 0x5645b7479860 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f206c004060 tx=0x7f206c0040d0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 shutdown_connections 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b747db90 0x5645b747ffb0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b746d8f0 0x5645b7479860 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7468780 0x5645b7472820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 --2- 172.21.15.33:0/1928101173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b7467d90 0x5645b74722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f205cff9640 0 client.18207 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.001+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 >> 172.21.15.33:0/1928101173 conn(0x5645b742f950 msgr2=0x5645b7462950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.002+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 shutdown_connections 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.002+0000 7f207e229740 1 -- 172.21.15.33:0/1928101173 wait complete. 2024-04-24T23:25:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.002+0000 7f207e229740 20 client.18207 trim_cache size 0 max 16384 2024-04-24T23:25:08.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.021+0000 7f207e229740 1 Processor -- start 2024-04-24T23:25:08.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.021+0000 7f207e229740 1 -- start start 2024-04-24T23:25:08.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.021+0000 7f207e229740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7466e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f207e229740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7479780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f207e229740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7484870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f207e229740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20741accb0 con 0x5645b7468780 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f207e229740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20741aa580 con 0x5645b7467d90 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f207e229740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20741a83d0 con 0x5645b746d8f0 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7479780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7479780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54258/0 (socket says 172.21.15.33:54258) 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 -- 172.21.15.33:0/3731527635 learned_addr learned my addr 172.21.15.33:0/3731527635 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20799d5640 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7466e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20799d5640 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7484870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 -- 172.21.15.33:0/3731527635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 msgr2=0x5645b7484870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7484870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 -- 172.21.15.33:0/3731527635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 msgr2=0x5645b7466e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7466e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 -- 172.21.15.33:0/3731527635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b709f170 con 0x5645b7468780 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20799d5640 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7484870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f20789d3640 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7479780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x5645b7480820 tx=0x7f2064005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aead16211065ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f205effd640 1 -- 172.21.15.33:0/3731527635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064020070 con 0x5645b7468780 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f205effd640 1 -- 172.21.15.33:0/3731527635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20640206f0 con 0x5645b7468780 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.022+0000 7f205effd640 1 -- 172.21.15.33:0/3731527635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064010d00 con 0x5645b7468780 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 -- 172.21.15.33:0/3731527635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 msgr2=0x5645b7479780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7479780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x5645b7480820 tx=0x7f2064005f90 comp rx=0 tx=0).stop 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 -- 172.21.15.33:0/3731527635 shutdown_connections 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7484870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7479780 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 --2- 172.21.15.33:0/3731527635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7466e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 -- 172.21.15.33:0/3731527635 >> 172.21.15.33:0/3731527635 conn(0x5645b742f950 msgr2=0x5645b7466700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 -- 172.21.15.33:0/3731527635 shutdown_connections 2024-04-24T23:25:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.023+0000 7f207e229740 1 -- 172.21.15.33:0/3731527635 wait complete. 2024-04-24T23:25:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 -- start start 2024-04-24T23:25:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 Processor -- start 2024-04-24T23:25:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7477610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7477b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7478b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20741a8a40 con 0x5645b7468780 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20741a9eb0 con 0x5645b7467d90 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20741a9110 con 0x5645b746d8f0 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f207e229740 10 client.0 _ll_register_callbacks cb 0x7f20700f3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f20799d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7477610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f20789d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7477b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f20789d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7477b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54274/0 (socket says 172.21.15.33:54274) 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f20789d3640 1 -- 172.21.15.33:0/3413995450 learned_addr learned my addr 172.21.15.33:0/3413995450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.024+0000 7f20799d5640 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7478b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f20799d5640 1 -- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 msgr2=0x5645b7478b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f20799d5640 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7478b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f20799d5640 1 -- 172.21.15.33:0/3413995450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 msgr2=0x5645b7477b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f20799d5640 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7477b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f20799d5640 1 -- 172.21.15.33:0/3413995450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b7479150 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f20799d5640 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7477610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5645b745ea40 tx=0x7f206c03a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2833f292b425c262 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f205cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f205cff9640 1 -- 172.21.15.33:0/3413995450 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c01a690 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20640206f0 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f205cff9640 1 -- 172.21.15.33:0/3413995450 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f206c003ad0 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f205cff9640 1 -- 172.21.15.33:0/3413995450 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f206c047050 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f207e229740 10 client.27728 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.025+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645b74e4710 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f205cff9640 1 -- 172.21.15.33:0/3413995450 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20640206f0 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f205cff9640 1 -- 172.21.15.33:0/3413995450 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f206c00e720 con 0x5645b7467d90 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f207e229740 10 client.27728 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f207e229740 10 client.27728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f207e229740 20 client.27728 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f207e229740 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b74e5060 0x5645b74e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f207e229740 4 client.27728 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5645b73d8800 con 0x5645b74e5060 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f20789d3640 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b74e5060 0x5645b74e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f205cff9640 10 client.27728 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.026+0000 7f20789d3640 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b74e5060 0x5645b74e7480 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83798d7e348cb2d3 server_cookie=deb0fdd871129eff in_seq=0 out_seq=0 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f205cff9640 1 -- 172.21.15.33:0/3413995450 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5645b73d8800 con 0x5645b74e5060 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f205cff9640 10 client.27728 handle_command_reply: tid=0 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 client.27728 shutdown 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 msgr2=0x5645b7477610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7477610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5645b745ea40 tx=0x7f206c03a040 comp rx=0 tx=0).stop 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 shutdown_connections 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5645b74e5060 0x5645b74e7480 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5645b746d8f0 0x5645b7478b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645b7468780 0x5645b7477b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 --2- 172.21.15.33:0/3413995450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5645b7467d90 0x5645b7477610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f205cff9640 0 client.27728 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.027+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 >> 172.21.15.33:0/3413995450 conn(0x5645b742f950 msgr2=0x5645b74620c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 shutdown_connections 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207e229740 1 -- 172.21.15.33:0/3413995450 wait complete. 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207e229740 20 client.27728 trim_cache size 0 max 16384 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d07b6d32-c888-471f-8896-168e1346caee", 2024-04-24T23:25:08.034 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20600a2b90 msgr2=0x7f20600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20600a2b90 0x7f20600a5050 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f20700ed530 tx=0x7f20700f4040 comp rx=0 tx=0).stop 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 msgr2=0x7f2074069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 0x7f2074069440 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2070031610 tx=0x7f2070004270 comp rx=0 tx=0).stop 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 shutdown_connections 2024-04-24T23:25:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20600a2b90 0x7f20600a5050 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074102230 0x7f207419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074101870 0x7f207419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 --2- 172.21.15.33:0/418024430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2074100680 0x7f2074069440 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 >> 172.21.15.33:0/418024430 conn(0x7f20740fbbf0 msgr2=0x7f2074104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 shutdown_connections 2024-04-24T23:25:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.028+0000 7f207b438640 1 -- 172.21.15.33:0/418024430 wait complete. 2024-04-24T23:25:08.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:25:08.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:25:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 -- 172.21.15.33:0/2041071941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300104bd0 msgr2=0x7f2300106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 --2- 172.21.15.33:0/2041071941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300104bd0 0x7f2300106fc0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009900 tx=0x7f22f402a410 comp rx=0 tx=0).stop 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 -- 172.21.15.33:0/2041071941 shutdown_connections 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 --2- 172.21.15.33:0/2041071941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2300107500 0x7f23001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 --2- 172.21.15.33:0/2041071941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300104bd0 0x7f2300106fc0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 --2- 172.21.15.33:0/2041071941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23001022a0 0x7f2300104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 -- 172.21.15.33:0/2041071941 >> 172.21.15.33:0/2041071941 conn(0x7f23000fbc10 msgr2=0x7f23000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 -- 172.21.15.33:0/2041071941 shutdown_connections 2024-04-24T23:25:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.150+0000 7f230697e640 1 -- 172.21.15.33:0/2041071941 wait complete. 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 Processor -- start 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 -- start start 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23001022a0 0x7f2300101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2300104bd0 0x7f2300100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 0x7f2300100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f230010d3f0 con 0x7f2300107500 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f230010d270 con 0x7f23001022a0 2024-04-24T23:25:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f230697e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f230010d570 con 0x7f2300104bd0 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f2304f1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 0x7f2300100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f2304f1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 0x7f2300100790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54302/0 (socket says 172.21.15.33:54302) 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f2304f1b640 1 -- 172.21.15.33:0/1085820215 learned_addr learned my addr 172.21.15.33:0/1085820215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f22fffff640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23001022a0 0x7f2300101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2300104bd0 0x7f2300100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f2304f1b640 1 -- 172.21.15.33:0/1085820215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2300104bd0 msgr2=0x7f2300100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.151+0000 7f2304f1b640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2300104bd0 0x7f2300100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f2304f1b640 1 -- 172.21.15.33:0/1085820215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23001022a0 msgr2=0x7f2300101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f2304f1b640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23001022a0 0x7f2300101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f2304f1b640 1 -- 172.21.15.33:0/1085820215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2300101020 con 0x7f2300107500 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f2304f1b640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 0x7f2300100790 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f22f800eaa0 tx=0x7f22f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f22fd7fa640 1 -- 172.21.15.33:0/1085820215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800cdb0 con 0x7f2300107500 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f22fd7fa640 1 -- 172.21.15.33:0/1085820215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f8005020 con 0x7f2300107500 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23001a1c20 con 0x7f2300107500 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f22fd7fa640 1 -- 172.21.15.33:0/1085820215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80116b0 con 0x7f2300107500 2024-04-24T23:25:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.152+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23001a20f0 con 0x7f2300107500 2024-04-24T23:25:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.154+0000 7f22fd7fa640 1 -- 172.21.15.33:0/1085820215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f22f8005560 con 0x7f2300107500 2024-04-24T23:25:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.154+0000 7f230976d740 1 Processor -- start 2024-04-24T23:25:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.154+0000 7f230976d740 1 -- start start 2024-04-24T23:25:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f230976d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01240 0x5637cbd01660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f230976d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01c30 0x5637cbd06730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f230976d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd09230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f230976d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637cbc656b0 con 0x5637cbd01240 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f230976d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5637cbc0d880 con 0x5637cbd01c30 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f230976d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637cbc61670 con 0x5637cbd06da0 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fd7fa640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22e80a2b90 0x7f22e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01240 0x5637cbd01660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01240 0x5637cbd01660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54312/0 (socket says 172.21.15.33:54312) 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22ff7fe640 1 -- 172.21.15.33:0/2595355494 learned_addr learned my addr 172.21.15.33:0/2595355494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01c30 0x5637cbd06730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f2304f1b640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22e80a2b90 0x7f22e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd09230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fd7fa640 1 -- 172.21.15.33:0/1085820215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22f80cf670 con 0x7f2300107500 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 -- 172.21.15.33:0/2595355494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 msgr2=0x5637cbd09230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd09230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 -- 172.21.15.33:0/2595355494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01240 msgr2=0x5637cbd01660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01240 0x5637cbd01660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 -- 172.21.15.33:0/2595355494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637cba437d0 con 0x5637cbd01c30 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd09230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22fffff640 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01c30 0x5637cbd06730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009c20 tx=0x7f22f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc8a7b8d5d5dd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f2304f1b640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22e80a2b90 0x7f22e80a5050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f22f80a4220 tx=0x7f22f801eb40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.155+0000 7f22e67fc640 1 -- 172.21.15.33:0/2595355494 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0004050 con 0x5637cbd01c30 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f22e67fc640 1 -- 172.21.15.33:0/2595355494 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f0005500 con 0x5637cbd01c30 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f22e67fc640 1 -- 172.21.15.33:0/2595355494 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0004d90 con 0x5637cbd01c30 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 -- 172.21.15.33:0/2595355494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01c30 msgr2=0x5637cbd06730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01c30 0x5637cbd06730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009c20 tx=0x7f22f0011810 comp rx=0 tx=0).stop 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 -- 172.21.15.33:0/2595355494 shutdown_connections 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd09230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01c30 0x5637cbd06730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 --2- 172.21.15.33:0/2595355494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01240 0x5637cbd01660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 -- 172.21.15.33:0/2595355494 >> 172.21.15.33:0/2595355494 conn(0x5637cbcfc8b0 msgr2=0x5637cbcfccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 -- 172.21.15.33:0/2595355494 shutdown_connections 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.156+0000 7f230976d740 1 -- 172.21.15.33:0/2595355494 wait complete. 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 -- start start 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 Processor -- start 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd0bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd12bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637cbd09d50 con 0x5637cbd01c30 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5637cbc20f50 con 0x5637cbd01240 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637cbd09ed0 con 0x5637cbd06da0 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f230976d740 10 client.0 _ll_register_callbacks cb 0x5637cbc5c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd0bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd0bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54326/0 (socket says 172.21.15.33:54326) 2024-04-24T23:25:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 -- 172.21.15.33:0/3541989025 learned_addr learned my addr 172.21.15.33:0/3541989025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd12bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 -- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 msgr2=0x5637cbd12bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd12bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 -- 172.21.15.33:0/3541989025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 msgr2=0x5637cbd0b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22fffff640 1 -- 172.21.15.33:0/3541989025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637cb9793f0 con 0x5637cbd01c30 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.157+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22fffff640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd0bb70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f22f001b040 tx=0x7f22f00045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b975fcc6fc8bb29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22df7fe640 1 -- 172.21.15.33:0/3541989025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0029070 con 0x5637cbd01c30 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22df7fe640 1 -- 172.21.15.33:0/3541989025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22f0004f20 con 0x5637cbd01c30 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f0005500 con 0x5637cbd01c30 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22df7fe640 1 -- 172.21.15.33:0/3541989025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22f0035050 con 0x5637cbd01c30 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 10 client.18225 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:08.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637cb9390f0 con 0x5637cbd01c30 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22df7fe640 1 -- 172.21.15.33:0/3541989025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f0005500 con 0x5637cbd01c30 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f22df7fe640 1 -- 172.21.15.33:0/3541989025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f22f0023430 con 0x5637cbd01c30 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 10 client.18225 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 10 client.18225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 20 client.18225 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd171f0 0x5637cbd19670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 4 client.18225 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.158+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637cbc72bf0 con 0x5637cbd171f0 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.159+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd171f0 0x5637cbd19670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.159+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd171f0 0x5637cbd19670 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824b882250811105 server_cookie=e032cee44e923203 in_seq=0 out_seq=0 2024-04-24T23:25:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.159+0000 7f22df7fe640 10 client.18225 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.159+0000 7f22df7fe640 1 -- 172.21.15.33:0/3541989025 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637cbc72bf0 con 0x5637cbd171f0 2024-04-24T23:25:08.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f22df7fe640 10 client.18225 handle_command_reply: tid=0 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 client.18225 shutdown 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 msgr2=0x5637cbd0bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd0bb70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f22f001b040 tx=0x7f22f00045b0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 shutdown_connections 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd171f0 0x5637cbd19670 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd12bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd0bb70 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f22df7fe640 0 client.18225 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 --2- 172.21.15.33:0/3541989025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 >> 172.21.15.33:0/3541989025 conn(0x5637cbcfc8b0 msgr2=0x5637cbcfbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 shutdown_connections 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 1 -- 172.21.15.33:0/3541989025 wait complete. 2024-04-24T23:25:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.160+0000 7f230976d740 20 client.18225 trim_cache size 0 max 16384 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 Processor -- start 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 -- start start 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd12b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbcfb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23001aee70 con 0x5637cbd01c30 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23001ac740 con 0x5637cbd01240 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f230976d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23001aa590 con 0x5637cbd06da0 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd12b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd12b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54334/0 (socket says 172.21.15.33:54334) 2024-04-24T23:25:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 -- 172.21.15.33:0/910480050 learned_addr learned my addr 172.21.15.33:0/910480050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbcfb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 -- 172.21.15.33:0/910480050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 msgr2=0x5637cbcfb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbcfb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 -- 172.21.15.33:0/910480050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 msgr2=0x5637cbd0a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 -- 172.21.15.33:0/910480050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637cbd03690 con 0x5637cbd01c30 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbcfb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.179+0000 7f22fffff640 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd12b70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009c20 tx=0x7f22f00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebdb6e9f7730c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f22e5ffb640 1 -- 172.21.15.33:0/910480050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0029070 con 0x5637cbd01c30 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f22e5ffb640 1 -- 172.21.15.33:0/910480050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f0009dd0 con 0x5637cbd01c30 2024-04-24T23:25:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f22e5ffb640 1 -- 172.21.15.33:0/910480050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0005460 con 0x5637cbd01c30 2024-04-24T23:25:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 -- 172.21.15.33:0/910480050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 msgr2=0x5637cbd12b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd12b70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009c20 tx=0x7f22f00096a0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 -- 172.21.15.33:0/910480050 shutdown_connections 2024-04-24T23:25:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbcfb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd12b70 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 --2- 172.21.15.33:0/910480050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd0a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 -- 172.21.15.33:0/910480050 >> 172.21.15.33:0/910480050 conn(0x5637cbcfc8b0 msgr2=0x5637cbcfb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 -- 172.21.15.33:0/910480050 shutdown_connections 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.180+0000 7f230976d740 1 -- 172.21.15.33:0/910480050 wait complete. 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 -- start start 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 Processor -- start 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd11e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23001aac00 con 0x5637cbd01c30 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23001ac070 con 0x5637cbd01240 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23001ab2d0 con 0x5637cbd06da0 2024-04-24T23:25:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f230976d740 10 client.0 _ll_register_callbacks cb 0x7f22f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f22fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f22fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd10eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54344/0 (socket says 172.21.15.33:54344) 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd10970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36704/0 (socket says 172.21.15.33:36704) 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f22fffff640 1 -- 172.21.15.33:0/1915253122 learned_addr learned my addr 172.21.15.33:0/1915253122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.181+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd11e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22fffff640 1 -- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 msgr2=0x5637cbd11e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22fffff640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd11e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22fffff640 1 -- 172.21.15.33:0/1915253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 msgr2=0x5637cbd10970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22fffff640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd10970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22fffff640 1 -- 172.21.15.33:0/1915253122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637cbd12460 con 0x5637cbd01c30 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd10970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22fffff640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd10eb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f22f0023a10 tx=0x7f22f0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85028c04c0a19fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22df7fe640 1 -- 172.21.15.33:0/1915253122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0029070 con 0x5637cbd01c30 2024-04-24T23:25:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22df7fe640 1 -- 172.21.15.33:0/1915253122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22f0004450 con 0x5637cbd01c30 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f0009dd0 con 0x5637cbd01c30 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22df7fe640 1 -- 172.21.15.33:0/1915253122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22f0045050 con 0x5637cbd01c30 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f230976d740 10 client.18231 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637cbd7de20 con 0x5637cbd01c30 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.182+0000 7f22df7fe640 1 -- 172.21.15.33:0/1915253122 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f0009dd0 con 0x5637cbd01c30 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f22df7fe640 1 -- 172.21.15.33:0/1915253122 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f22f00035f0 con 0x5637cbd01c30 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f230976d740 10 client.18231 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f230976d740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f230976d740 20 client.18231 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f230976d740 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd7e760 0x5637cbd80b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f230976d740 4 client.18231 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637cbc6cf40 con 0x5637cbd7e760 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.183+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd7e760 0x5637cbd80b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f22df7fe640 10 client.18231 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f22ff7fe640 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd7e760 0x5637cbd80b80 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8bebf6003154309 server_cookie=3d98fdef182f910 in_seq=0 out_seq=0 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f22df7fe640 1 -- 172.21.15.33:0/1915253122 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5637cbc6cf40 con 0x5637cbd7e760 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f22df7fe640 10 client.18231 handle_command_reply: tid=0 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 client.18231 shutdown 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 msgr2=0x5637cbd10eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd10eb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f22f0023a10 tx=0x7f22f0005dc0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 shutdown_connections 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5637cbd7e760 0x5637cbd80b80 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637cbd06da0 0x5637cbd11e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637cbd01c30 0x5637cbd10eb0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 --2- 172.21.15.33:0/1915253122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5637cbd01240 0x5637cbd10970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f22df7fe640 0 client.18231 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 >> 172.21.15.33:0/1915253122 conn(0x5637cbcfc8b0 msgr2=0x5637cbcfb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 shutdown_connections 2024-04-24T23:25:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 1 -- 172.21.15.33:0/1915253122 wait complete. 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.184+0000 7f230976d740 20 client.18231 trim_cache size 0 max 16384 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: "d07b6d32-c888-471f-8896-168e1346caee": { 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d07b6d32-c888-471f-8896-168e1346caee", 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:25:08.191 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22e80a2b90 msgr2=0x7f22e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22e80a2b90 0x7f22e80a5050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f22f80a4220 tx=0x7f22f801eb40 comp rx=0 tx=0).stop 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 msgr2=0x7f2300100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 0x7f2300100790 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f22f800eaa0 tx=0x7f22f800ef60 comp rx=0 tx=0).stop 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 shutdown_connections 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f22e80a2b90 0x7f22e80a5050 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2300107500 0x7f2300100790 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2300104bd0 0x7f2300100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 --2- 172.21.15.33:0/1085820215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23001022a0 0x7f2300101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 >> 172.21.15.33:0/1085820215 conn(0x7f23000fbc10 msgr2=0x7f2300102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 shutdown_connections 2024-04-24T23:25:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:08.185+0000 7f230697e640 1 -- 172.21.15.33:0/1085820215 wait complete. 2024-04-24T23:25:08.197 INFO:tasks.cephfs.filesystem:scrub status for tag:d07b6d32-c888-471f-8896-168e1346caee - {'path': '/', 'tag': 'd07b6d32-c888-471f-8896-168e1346caee', 'options': 'recursive,force'} 2024-04-24T23:25:08.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1553342 12.32 MB/sec execute 553 sec latency 20.744 ms 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: pgmap v745: 129 pgs: 129 active+clean; 285 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:09 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:09.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: pgmap v745: 129 pgs: 129 active+clean; 285 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-04-24T23:25:09.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:25:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:25:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:25:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:09 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: pgmap v745: 129 pgs: 129 active+clean; 285 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-04-24T23:25:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:25:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:25:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:25:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:09.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:09 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:09.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1556174 12.33 MB/sec execute 554 sec latency 16.781 ms 2024-04-24T23:25:10.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:10.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:10.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:25:10.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:10.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:10.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:25:10.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:10.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:10.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:25:10.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1557834 12.32 MB/sec execute 555 sec latency 24.523 ms 2024-04-24T23:25:11.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:11 smithi038 ceph-mon[32450]: pgmap v746: 129 pgs: 129 active+clean; 285 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-04-24T23:25:11.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:11 smithi117 ceph-mon[32529]: pgmap v746: 129 pgs: 129 active+clean; 285 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-04-24T23:25:11.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:11 smithi033 ceph-mon[29201]: pgmap v746: 129 pgs: 129 active+clean; 285 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-04-24T23:25:11.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1560025 12.32 MB/sec execute 556 sec latency 23.135 ms 2024-04-24T23:25:12.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1562040 12.32 MB/sec execute 557 sec latency 56.423 ms 2024-04-24T23:25:13.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:13 smithi038 ceph-mon[32450]: pgmap v747: 129 pgs: 129 active+clean; 289 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-04-24T23:25:13.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:13 smithi117 ceph-mon[32529]: pgmap v747: 129 pgs: 129 active+clean; 289 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-04-24T23:25:13.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:13 smithi033 ceph-mon[29201]: pgmap v747: 129 pgs: 129 active+clean; 289 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-04-24T23:25:13.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1563982 12.32 MB/sec execute 558 sec latency 13.549 ms 2024-04-24T23:25:14.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1566938 12.33 MB/sec execute 559 sec latency 10.195 ms 2024-04-24T23:25:15.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:15 smithi038 ceph-mon[32450]: pgmap v748: 129 pgs: 129 active+clean; 286 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-04-24T23:25:15.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:15 smithi117 ceph-mon[32529]: pgmap v748: 129 pgs: 129 active+clean; 286 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-04-24T23:25:15.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:15 smithi033 ceph-mon[29201]: pgmap v748: 129 pgs: 129 active+clean; 286 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-04-24T23:25:15.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1569348 12.32 MB/sec execute 560 sec latency 15.355 ms 2024-04-24T23:25:16.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1571187 12.32 MB/sec execute 561 sec latency 18.427 ms 2024-04-24T23:25:17.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:17 smithi038 ceph-mon[32450]: pgmap v749: 129 pgs: 129 active+clean; 275 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 238 op/s 2024-04-24T23:25:17.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:17 smithi117 ceph-mon[32529]: pgmap v749: 129 pgs: 129 active+clean; 275 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 238 op/s 2024-04-24T23:25:17.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:17 smithi033 ceph-mon[29201]: pgmap v749: 129 pgs: 129 active+clean; 275 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 238 op/s 2024-04-24T23:25:17.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1573635 12.33 MB/sec execute 562 sec latency 79.342 ms 2024-04-24T23:25:18.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:18 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:25:18.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:18 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:25:18.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:18 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:25:18.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:18 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:25:18.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:18 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:25:18.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:18 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:25:18.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1575119 12.32 MB/sec execute 563 sec latency 15.563 ms 2024-04-24T23:25:19.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:19 smithi038 ceph-mon[32450]: pgmap v750: 129 pgs: 129 active+clean; 266 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-24T23:25:19.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:19 smithi117 ceph-mon[32529]: pgmap v750: 129 pgs: 129 active+clean; 266 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-24T23:25:19.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:19 smithi033 ceph-mon[29201]: pgmap v750: 129 pgs: 129 active+clean; 266 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-24T23:25:19.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1577650 12.32 MB/sec execute 564 sec latency 15.334 ms 2024-04-24T23:25:20.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1579827 12.31 MB/sec execute 565 sec latency 19.892 ms 2024-04-24T23:25:21.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:21 smithi038 ceph-mon[32450]: pgmap v751: 129 pgs: 129 active+clean; 269 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 217 op/s 2024-04-24T23:25:21.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:21 smithi117 ceph-mon[32529]: pgmap v751: 129 pgs: 129 active+clean; 269 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 217 op/s 2024-04-24T23:25:21.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:21 smithi033 ceph-mon[29201]: pgmap v751: 129 pgs: 129 active+clean; 269 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 217 op/s 2024-04-24T23:25:21.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1581581 12.31 MB/sec execute 566 sec latency 14.157 ms 2024-04-24T23:25:22.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1583653 12.31 MB/sec execute 567 sec latency 73.488 ms 2024-04-24T23:25:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:23 smithi038 ceph-mon[32450]: pgmap v752: 129 pgs: 129 active+clean; 272 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-04-24T23:25:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:23 smithi117 ceph-mon[32529]: pgmap v752: 129 pgs: 129 active+clean; 272 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-04-24T23:25:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:23 smithi033 ceph-mon[29201]: pgmap v752: 129 pgs: 129 active+clean; 272 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-04-24T23:25:23.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1585197 12.30 MB/sec execute 568 sec latency 45.460 ms 2024-04-24T23:25:24.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1588075 12.31 MB/sec execute 569 sec latency 12.820 ms 2024-04-24T23:25:25.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:25 smithi038 ceph-mon[32450]: pgmap v753: 129 pgs: 129 active+clean; 271 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:25:25.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:25 smithi117 ceph-mon[32529]: pgmap v753: 129 pgs: 129 active+clean; 271 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:25:25.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:25 smithi033 ceph-mon[29201]: pgmap v753: 129 pgs: 129 active+clean; 271 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T23:25:25.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1590046 12.30 MB/sec execute 570 sec latency 13.353 ms 2024-04-24T23:25:26.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1592121 12.30 MB/sec execute 571 sec latency 19.043 ms 2024-04-24T23:25:27.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:27 smithi117 ceph-mon[32529]: pgmap v754: 129 pgs: 129 active+clean; 273 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-04-24T23:25:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:27 smithi033 ceph-mon[29201]: pgmap v754: 129 pgs: 129 active+clean; 273 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-04-24T23:25:27.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1593857 12.29 MB/sec execute 572 sec latency 77.009 ms 2024-04-24T23:25:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:27 smithi038 ceph-mon[32450]: pgmap v754: 129 pgs: 129 active+clean; 273 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-04-24T23:25:28.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1595778 12.29 MB/sec execute 573 sec latency 13.578 ms 2024-04-24T23:25:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:29 smithi117 ceph-mon[32529]: pgmap v755: 129 pgs: 129 active+clean; 274 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-24T23:25:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:29 smithi033 ceph-mon[29201]: pgmap v755: 129 pgs: 129 active+clean; 274 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-24T23:25:29.468 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1598605 12.30 MB/sec execute 574 sec latency 16.146 ms 2024-04-24T23:25:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:29 smithi038 ceph-mon[32450]: pgmap v755: 129 pgs: 129 active+clean; 274 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-24T23:25:30.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1600406 12.29 MB/sec execute 575 sec latency 19.448 ms 2024-04-24T23:25:31.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:31 smithi117 ceph-mon[32529]: pgmap v756: 129 pgs: 129 active+clean; 274 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-04-24T23:25:31.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:31 smithi033 ceph-mon[29201]: pgmap v756: 129 pgs: 129 active+clean; 274 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-04-24T23:25:31.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1602754 12.29 MB/sec execute 576 sec latency 15.220 ms 2024-04-24T23:25:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:31 smithi038 ceph-mon[32450]: pgmap v756: 129 pgs: 129 active+clean; 274 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-04-24T23:25:32.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1604542 12.29 MB/sec execute 577 sec latency 145.561 ms 2024-04-24T23:25:33.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:33 smithi117 ceph-mon[32529]: pgmap v757: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-04-24T23:25:33.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:33 smithi033 ceph-mon[29201]: pgmap v757: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-04-24T23:25:33.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1606358 12.29 MB/sec execute 578 sec latency 28.420 ms 2024-04-24T23:25:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:33 smithi038 ceph-mon[32450]: pgmap v757: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-04-24T23:25:34.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1609302 12.29 MB/sec execute 579 sec latency 8.019 ms 2024-04-24T23:25:35.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:35 smithi033 ceph-mon[29201]: pgmap v758: 129 pgs: 129 active+clean; 279 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-04-24T23:25:35.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1610986 12.29 MB/sec execute 580 sec latency 31.764 ms 2024-04-24T23:25:35.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:35 smithi038 ceph-mon[32450]: pgmap v758: 129 pgs: 129 active+clean; 279 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-04-24T23:25:35.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:35 smithi117 ceph-mon[32529]: pgmap v758: 129 pgs: 129 active+clean; 279 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-04-24T23:25:36.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1613450 12.29 MB/sec execute 581 sec latency 13.506 ms 2024-04-24T23:25:37.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:37 smithi033 ceph-mon[29201]: pgmap v759: 129 pgs: 129 active+clean; 282 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:25:37.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1616085 12.29 MB/sec execute 582 sec latency 112.439 ms 2024-04-24T23:25:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:37 smithi038 ceph-mon[32450]: pgmap v759: 129 pgs: 129 active+clean; 282 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:25:37.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:37 smithi117 ceph-mon[32529]: pgmap v759: 129 pgs: 129 active+clean; 282 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-04-24T23:25:38.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:25:38.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 -- 172.21.15.33:0/3752489212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468102230 msgr2=0x7f1468108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 --2- 172.21.15.33:0/3752489212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468102230 0x7f1468108ac0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1460009ca0 tx=0x7f146002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 -- 172.21.15.33:0/3752489212 shutdown_connections 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 --2- 172.21.15.33:0/3752489212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468102230 0x7f1468108ac0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 --2- 172.21.15.33:0/3752489212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1468101870 0x7f1468101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 --2- 172.21.15.33:0/3752489212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 -- 172.21.15.33:0/3752489212 >> 172.21.15.33:0/3752489212 conn(0x7f14680fbbf0 msgr2=0x7f14680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 -- 172.21.15.33:0/3752489212 shutdown_connections 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.325+0000 7f146fcca640 1 -- 172.21.15.33:0/3752489212 wait complete. 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 Processor -- start 2024-04-24T23:25:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 -- start start 2024-04-24T23:25:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 0x7f146819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1468102230 0x7f146819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f146810c330 con 0x7f1468101870 2024-04-24T23:25:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f146810c1b0 con 0x7f1468102230 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146fcca640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f146810c4b0 con 0x7f1468100680 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146d265640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 0x7f146819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146da66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146da66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42432/0 (socket says 172.21.15.33:42432) 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146da66640 1 -- 172.21.15.33:0/2098008186 learned_addr learned my addr 172.21.15.33:0/2098008186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146d265640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 0x7f146819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47376/0 (socket says 172.21.15.33:47376) 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.326+0000 7f146e267640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1468102230 0x7f146819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146d265640 1 -- 172.21.15.33:0/2098008186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 msgr2=0x7f1468069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146d265640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146d265640 1 -- 172.21.15.33:0/2098008186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1468102230 msgr2=0x7f146819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146d265640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1468102230 0x7f146819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146d265640 1 -- 172.21.15.33:0/2098008186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146819f6d0 con 0x7f1468101870 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146da66640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146d265640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 0x7f146819c7a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f145800e9f0 tx=0x7f145800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f145effd640 1 -- 172.21.15.33:0/2098008186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145800ccc0 con 0x7f1468101870 2024-04-24T23:25:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f145effd640 1 -- 172.21.15.33:0/2098008186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1458010910 con 0x7f1468101870 2024-04-24T23:25:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146819f960 con 0x7f1468101870 2024-04-24T23:25:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f145effd640 1 -- 172.21.15.33:0/2098008186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458018960 con 0x7f1468101870 2024-04-24T23:25:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.327+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146819feb0 con 0x7f1468101870 2024-04-24T23:25:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.330+0000 7f145effd640 1 -- 172.21.15.33:0/2098008186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1458010ac0 con 0x7f1468101870 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.331+0000 7f145effd640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14540a2b90 0x7f14540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.331+0000 7f146da66640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14540a2b90 0x7f14540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.331+0000 7f145effd640 1 -- 172.21.15.33:0/2098008186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f14580cf7a0 con 0x7f1468101870 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.331+0000 7f14729d6740 1 Processor -- start 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.331+0000 7f146da66640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14540a2b90 0x7f14540a5050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f14640058c0 tx=0x7f1464005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.331+0000 7f14729d6740 1 -- start start 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f14729d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 0x55de2026da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f14729d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de20272b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f14729d6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de20275640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f14729d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55de201cd6b0 con 0x55de2026d650 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f14729d6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55de20175880 con 0x55de202731b0 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f14729d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de201c9670 con 0x55de2026e040 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146da66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 0x55de2026da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de20272b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146da66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 0x55de2026da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47386/0 (socket says 172.21.15.33:47386) 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146da66640 1 -- 172.21.15.33:0/1558893030 learned_addr learned my addr 172.21.15.33:0/1558893030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146e267640 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de20275640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 -- 172.21.15.33:0/1558893030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 msgr2=0x55de20275640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de20275640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 -- 172.21.15.33:0/1558893030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 msgr2=0x55de2026da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 0x55de2026da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 -- 172.21.15.33:0/1558893030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de1ffab7d0 con 0x55de2026e040 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146da66640 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 0x55de2026da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f146d265640 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de20272b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f145809b9c0 tx=0x7f145809bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22c01eb65b68277 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.332+0000 7f1453fff640 1 -- 172.21.15.33:0/1558893030 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14580a4880 con 0x55de2026e040 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f1453fff640 1 -- 172.21.15.33:0/1558893030 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1458099d60 con 0x55de2026e040 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f1453fff640 1 -- 172.21.15.33:0/1558893030 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145809b370 con 0x55de2026e040 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 -- 172.21.15.33:0/1558893030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 msgr2=0x55de20272b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de20272b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f145809b9c0 tx=0x7f145809bcc0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 -- 172.21.15.33:0/1558893030 shutdown_connections 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de20275640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de20272b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 --2- 172.21.15.33:0/1558893030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026d650 0x55de2026da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.333+0000 7f14729d6740 1 -- 172.21.15.33:0/1558893030 >> 172.21.15.33:0/1558893030 conn(0x55de202648b0 msgr2=0x55de20264cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 -- 172.21.15.33:0/1558893030 shutdown_connections 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 -- 172.21.15.33:0/1558893030 wait complete. 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 -- start start 2024-04-24T23:25:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 Processor -- start 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 0x55de202779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de20277f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de2027ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55de20276100 con 0x55de2026e040 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55de20188f50 con 0x55de202731b0 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de20276280 con 0x55de2026d650 2024-04-24T23:25:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f14729d6740 10 client.0 _ll_register_callbacks cb 0x55de201c4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f146da66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 0x55de202779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.334+0000 7f146d265640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de20277f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146d265640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de20277f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47390/0 (socket says 172.21.15.33:47390) 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146e267640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de2027ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 0x55de202779e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42438/0 (socket says 172.21.15.33:42438) 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146d265640 1 -- 172.21.15.33:0/3055257173 learned_addr learned my addr 172.21.15.33:0/3055257173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 -- 172.21.15.33:0/3055257173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 msgr2=0x55de2027ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de2027ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 -- 172.21.15.33:0/3055257173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 msgr2=0x55de20277f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de20277f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 -- 172.21.15.33:0/3055257173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de1fee13f0 con 0x55de2026d650 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f146da66640 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 0x55de202779e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1464009c10 tx=0x7f1464013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c000c86d66cd0bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f14517fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f14517fa640 1 -- 172.21.15.33:0/3055257173 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146401ccf0 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1458099d60 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f14517fa640 1 -- 172.21.15.33:0/3055257173 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1464004020 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.335+0000 7f14517fa640 1 -- 172.21.15.33:0/3055257173 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1464004220 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 10 client.27700 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de1fea10f0 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14517fa640 1 -- 172.21.15.33:0/3055257173 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1458099d60 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14517fa640 1 -- 172.21.15.33:0/3055257173 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14640048a0 con 0x55de2026d650 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 10 client.27700 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 10 client.27700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 20 client.27700 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202838e0 0x55de20285d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 4 client.27700 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de201dabf0 con 0x55de202838e0 2024-04-24T23:25:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.336+0000 7f146d265640 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202838e0 0x55de20285d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f14517fa640 10 client.27700 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f146d265640 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202838e0 0x55de20285d50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f23586356733b53 server_cookie=dbc25092f569f566 in_seq=0 out_seq=0 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f14517fa640 1 -- 172.21.15.33:0/3055257173 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de201dabf0 con 0x55de202838e0 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f14517fa640 10 client.27700 handle_command_reply: tid=0 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f14729d6740 1 client.27700 shutdown 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 msgr2=0x55de202779e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.337+0000 7f14729d6740 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 0x55de202779e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1464009c10 tx=0x7f1464013040 comp rx=0 tx=0).stop 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 shutdown_connections 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202838e0 0x55de20285d50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de202731b0 0x55de2027ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de20277f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 --2- 172.21.15.33:0/3055257173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026d650 0x55de202779e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14517fa640 0 client.27700 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 >> 172.21.15.33:0/3055257173 conn(0x55de202648b0 msgr2=0x55de20263af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 shutdown_connections 2024-04-24T23:25:38.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 1 -- 172.21.15.33:0/3055257173 wait complete. 2024-04-24T23:25:38.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.338+0000 7f14729d6740 20 client.27700 trim_cache size 0 max 16384 2024-04-24T23:25:38.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.355+0000 7f14729d6740 1 Processor -- start 2024-04-24T23:25:38.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.355+0000 7f14729d6740 1 -- start start 2024-04-24T23:25:38.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f14729d6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de20277d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f14729d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de2027eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f14729d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de202731b0 0x55de20263260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f14729d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14681aaaf0 con 0x55de2026e040 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f14729d6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14681a83c0 con 0x55de2026d650 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f14729d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14681a6210 con 0x55de202731b0 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de20277d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146e267640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de202731b0 0x55de20263260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de20277d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43838/0 (socket says 172.21.15.33:43838) 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 -- 172.21.15.33:0/2848713099 learned_addr learned my addr 172.21.15.33:0/2848713099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146d265640 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de2027eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 -- 172.21.15.33:0/2848713099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de202731b0 msgr2=0x55de20263260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de202731b0 0x55de20263260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 -- 172.21.15.33:0/2848713099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 msgr2=0x55de2027eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de2027eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.356+0000 7f146da66640 1 -- 172.21.15.33:0/2848713099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de20272e90 con 0x55de2026d650 2024-04-24T23:25:38.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f146da66640 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de20277d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f146400a010 tx=0x7f146401c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6424fa2a5eb57d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14537fe640 1 -- 172.21.15.33:0/2848713099 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464039020 con 0x55de2026d650 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14537fe640 1 -- 172.21.15.33:0/2848713099 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464004660 con 0x55de2026d650 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14537fe640 1 -- 172.21.15.33:0/2848713099 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640054d0 con 0x55de2026d650 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14729d6740 1 -- 172.21.15.33:0/2848713099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 msgr2=0x55de20277d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14729d6740 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de20277d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f146400a010 tx=0x7f146401c860 comp rx=0 tx=0).stop 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14729d6740 1 -- 172.21.15.33:0/2848713099 shutdown_connections 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.357+0000 7f14729d6740 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de202731b0 0x55de20263260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de2026e040 0x55de2027eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 --2- 172.21.15.33:0/2848713099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de20277d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- 172.21.15.33:0/2848713099 >> 172.21.15.33:0/2848713099 conn(0x55de202648b0 msgr2=0x55de202775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- 172.21.15.33:0/2848713099 shutdown_connections 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- 172.21.15.33:0/2848713099 wait complete. 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- start start 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 Processor -- start 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de2027c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de2027cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de202731b0 0x55de2027dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14681a6880 con 0x55de202731b0 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14681a7cf0 con 0x55de2026d650 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.358+0000 7f14729d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14681a6f50 con 0x55de2026e040 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f14729d6740 10 client.0 _ll_register_callbacks cb 0x7f146400fb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de2027c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de2027c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43842/0 (socket says 172.21.15.33:43842) 2024-04-24T23:25:38.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146e267640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de202731b0 0x55de2027dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 -- 172.21.15.33:0/4222519821 learned_addr learned my addr 172.21.15.33:0/4222519821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146d265640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de2027cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 -- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 msgr2=0x55de2027cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de2027cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 -- 172.21.15.33:0/4222519821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de202731b0 msgr2=0x55de2027dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de202731b0 0x55de2027dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 -- 172.21.15.33:0/4222519821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2027e260 con 0x55de2026d650 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146d265640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de2027cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f146da66640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de2027c720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1464004e30 tx=0x7f146400fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4a4a4b017e10512 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f14517fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f14517fa640 1 -- 172.21.15.33:0/4222519821 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464005250 con 0x55de2026d650 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f14517fa640 1 -- 172.21.15.33:0/4222519821 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f14640053f0 con 0x55de2026d650 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.359+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1464004660 con 0x55de2026d650 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14517fa640 1 -- 172.21.15.33:0/4222519821 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f146404c050 con 0x55de2026d650 2024-04-24T23:25:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 10 client.27764 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de202e9fb0 con 0x55de2026d650 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14517fa640 1 -- 172.21.15.33:0/4222519821 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464004660 con 0x55de2026d650 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14517fa640 1 -- 172.21.15.33:0/4222519821 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1464035500 con 0x55de2026d650 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 10 client.27764 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 10 client.27764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 20 client.27764 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202ea890 0x55de202eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 4 client.27764 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de20275ac0 con 0x55de202ea890 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.360+0000 7f146d265640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202ea890 0x55de202eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f146d265640 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202ea890 0x55de202eccb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c3c99658c2b13f server_cookie=989185d791905ed5 in_seq=0 out_seq=0 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14517fa640 10 client.27764 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14517fa640 1 -- 172.21.15.33:0/4222519821 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55de20275ac0 con 0x55de202ea890 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14517fa640 10 client.27764 handle_command_reply: tid=0 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 client.27764 shutdown 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 msgr2=0x55de2027c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de2027c720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1464004e30 tx=0x7f146400fef0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 shutdown_connections 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de202ea890 0x55de202eccb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de202731b0 0x55de2027dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de2026e040 0x55de2027cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14517fa640 0 client.27764 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 --2- 172.21.15.33:0/4222519821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de2026d650 0x55de2027c720 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 >> 172.21.15.33:0/4222519821 conn(0x55de202648b0 msgr2=0x55de20277df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 shutdown_connections 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 1 -- 172.21.15.33:0/4222519821 wait complete. 2024-04-24T23:25:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.361+0000 7f14729d6740 20 client.27764 trim_cache size 0 max 16384 2024-04-24T23:25:38.368 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:25:38.368 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:25:38.368 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:25:38.368 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14540a2b90 msgr2=0x7f14540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14540a2b90 0x7f14540a5050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f14640058c0 tx=0x7f1464005f90 comp rx=0 tx=0).stop 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 msgr2=0x7f146819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 0x7f146819c7a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f145800e9f0 tx=0x7f145800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 shutdown_connections 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f14540a2b90 0x7f14540a5050 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1468102230 0x7f146819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1468101870 0x7f146819c7a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 --2- 172.21.15.33:0/2098008186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1468100680 0x7f1468069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 >> 172.21.15.33:0/2098008186 conn(0x7f14680fbbf0 msgr2=0x7f1468104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 shutdown_connections 2024-04-24T23:25:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.362+0000 7f146fcca640 1 -- 172.21.15.33:0/2098008186 wait complete. 2024-04-24T23:25:38.374 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:25:38.375 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:38.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1617509 12.28 MB/sec execute 583 sec latency 19.442 ms 2024-04-24T23:25:38.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.475+0000 7ff480e6d640 1 -- 172.21.15.33:0/1947085082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 msgr2=0x7ff47c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.475+0000 7ff480e6d640 1 --2- 172.21.15.33:0/1947085082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c108ac0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff474009ca0 tx=0x7ff47402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 -- 172.21.15.33:0/1947085082 shutdown_connections 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 --2- 172.21.15.33:0/1947085082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c108ac0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 --2- 172.21.15.33:0/1947085082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c101870 0x7ff47c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 --2- 172.21.15.33:0/1947085082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff47c100680 0x7ff47c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 -- 172.21.15.33:0/1947085082 >> 172.21.15.33:0/1947085082 conn(0x7ff47c0fbbf0 msgr2=0x7ff47c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 -- 172.21.15.33:0/1947085082 shutdown_connections 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 -- 172.21.15.33:0/1947085082 wait complete. 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 Processor -- start 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 -- start start 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.476+0000 7ff480e6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c100680 0x7ff47c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff47c101870 0x7ff47c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10c2e0 con 0x7ff47c102230 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10c160 con 0x7ff47c101870 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10c460 con 0x7ff47c100680 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c100680 0x7ff47c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47418/0 (socket says 172.21.15.33:47418) 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 -- 172.21.15.33:0/3084893414 learned_addr learned my addr 172.21.15.33:0/3084893414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff479d9b640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff47c101870 0x7ff47c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 -- 172.21.15.33:0/3084893414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c100680 msgr2=0x7ff47c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c100680 0x7ff47c0694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 -- 172.21.15.33:0/3084893414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff47c101870 msgr2=0x7ff47c19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff47c101870 0x7ff47c19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 -- 172.21.15.33:0/3084893414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c19f610 con 0x7ff47c102230 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff47ad9d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c19ed50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff474031ae0 tx=0x7ff474031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474003c30 con 0x7ff47c102230 2024-04-24T23:25:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c19f8a0 con 0x7ff47c102230 2024-04-24T23:25:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff474003dd0 con 0x7ff47c102230 2024-04-24T23:25:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47402fcb0 con 0x7ff47c102230 2024-04-24T23:25:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.477+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c19fe40 con 0x7ff47c102230 2024-04-24T23:25:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.479+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47c107940 con 0x7ff47c102230 2024-04-24T23:25:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.480+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff474004050 con 0x7ff47c102230 2024-04-24T23:25:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.480+0000 7ff46b7fe640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4640a2b90 0x7ff4640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.480+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff4740f0610 con 0x7ff47c102230 2024-04-24T23:25:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.480+0000 7ff47a59c640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4640a2b90 0x7ff4640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.481+0000 7ff47a59c640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4640a2b90 0x7ff4640a5050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff47c1016d0 tx=0x7ff46c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.485+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff4740bd300 con 0x7ff47c102230 2024-04-24T23:25:38.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.715+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff47c062340 con 0x7ff47c102230 2024-04-24T23:25:38.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.716+0000 7ff46b7fe640 1 -- 172.21.15.33:0/3084893414 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11859 (secure 0 0 0) 0x7ff4740bcc00 con 0x7ff47c102230 2024-04-24T23:25:38.719 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:25:38.719 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:16:22.201419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4640a2b90 msgr2=0x7ff4640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4640a2b90 0x7ff4640a5050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff47c1016d0 tx=0x7ff46c005f90 comp rx=0 tx=0).stop 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 msgr2=0x7ff47c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c19ed50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff474031ae0 tx=0x7ff474031ee0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 shutdown_connections 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff4640a2b90 0x7ff4640a5050 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff47c102230 0x7ff47c19ed50 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff47c101870 0x7ff47c19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 --2- 172.21.15.33:0/3084893414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c100680 0x7ff47c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 >> 172.21.15.33:0/3084893414 conn(0x7ff47c0fbbf0 msgr2=0x7ff47c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.718+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 shutdown_connections 2024-04-24T23:25:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.719+0000 7ff480e6d640 1 -- 172.21.15.33:0/3084893414 wait complete. 2024-04-24T23:25:38.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 -- 172.21.15.33:0/2645250560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98101870 msgr2=0x7f6a98101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 --2- 172.21.15.33:0/2645250560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98101870 0x7f6a98101cf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009900 tx=0x7f6a8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 -- 172.21.15.33:0/2645250560 shutdown_connections 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 --2- 172.21.15.33:0/2645250560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a98102230 0x7f6a98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 --2- 172.21.15.33:0/2645250560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98101870 0x7f6a98101cf0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 --2- 172.21.15.33:0/2645250560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98100680 0x7f6a98100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 -- 172.21.15.33:0/2645250560 >> 172.21.15.33:0/2645250560 conn(0x7f6a980fbbf0 msgr2=0x7f6a980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 -- 172.21.15.33:0/2645250560 shutdown_connections 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.836+0000 7f6a9e909640 1 -- 172.21.15.33:0/2645250560 wait complete. 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 Processor -- start 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 -- start start 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 0x7f6a98197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98101870 0x7f6a98197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a98102230 0x7f6a9819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9810d480 con 0x7f6a98100680 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9810d300 con 0x7f6a98101870 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.837+0000 7f6a9e909640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9810d600 con 0x7f6a98102230 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 0x7f6a98197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 0x7f6a98197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47432/0 (socket says 172.21.15.33:47432) 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 -- 172.21.15.33:0/208128894 learned_addr learned my addr 172.21.15.33:0/208128894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a977fe640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98101870 0x7f6a98197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a98102230 0x7f6a9819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 -- 172.21.15.33:0/208128894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a98102230 msgr2=0x7f6a9819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a98102230 0x7f6a9819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 -- 172.21.15.33:0/208128894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98101870 msgr2=0x7f6a98197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98101870 0x7f6a98197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 -- 172.21.15.33:0/208128894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9819f6d0 con 0x7f6a98100680 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a97fff640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 0x7f6a98197ab0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e9c0 tx=0x7f6a8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a957fa640 1 -- 172.21.15.33:0/208128894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800cd80 con 0x7f6a98100680 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a957fa640 1 -- 172.21.15.33:0/208128894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a880035f0 con 0x7f6a98100680 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9819f9c0 con 0x7f6a98100680 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a957fa640 1 -- 172.21.15.33:0/208128894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88010690 con 0x7f6a98100680 2024-04-24T23:25:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.838+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9819fee0 con 0x7f6a98100680 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a957fa640 1 -- 172.21.15.33:0/208128894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6a880037a0 con 0x7f6a98100680 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 Processor -- start 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 -- start start 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a09070f670 con 0x55a0907ab230 2024-04-24T23:25:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0906b7970 con 0x55a0907b0d90 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6aa1615740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a09070b630 con 0x55a0907abc20 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a9cea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a9cea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47446/0 (socket says 172.21.15.33:47446) 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a9cea6640 1 -- 172.21.15.33:0/3013128342 learned_addr learned my addr 172.21.15.33:0/3013128342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a977fe640 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a97fff640 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a957fa640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6a800a2b90 0x7f6a800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a977fe640 1 -- 172.21.15.33:0/3013128342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 msgr2=0x55a0907b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a977fe640 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a977fe640 1 -- 172.21.15.33:0/3013128342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 msgr2=0x55a0907ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a977fe640 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.841+0000 7f6a977fe640 1 -- 172.21.15.33:0/3013128342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0906db400 con 0x55a0907b0d90 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6a800a2b90 0x7f6a800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a957fa640 1 -- 172.21.15.33:0/208128894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6a880cf7b0 con 0x7f6a98100680 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a977fe640 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907b3220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02a930 tx=0x7f6a8c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5dc38fb02d45569 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a7e7fc640 1 -- 172.21.15.33:0/3013128342 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c035040 con 0x55a0907b0d90 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a7e7fc640 1 -- 172.21.15.33:0/3013128342 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c005520 con 0x55a0907b0d90 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a7e7fc640 1 -- 172.21.15.33:0/3013128342 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0316b0 con 0x55a0907b0d90 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6aa1615740 1 -- 172.21.15.33:0/3013128342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 msgr2=0x55a0907b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6a800a2b90 0x7f6a800a5050 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009fd0 tx=0x7f6a90004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.842+0000 7f6aa1615740 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907b3220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02a930 tx=0x7f6a8c005c00 comp rx=0 tx=0).stop 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- 172.21.15.33:0/3013128342 shutdown_connections 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907b3220 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 --2- 172.21.15.33:0/3013128342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- 172.21.15.33:0/3013128342 >> 172.21.15.33:0/3013128342 conn(0x55a0907a68a0 msgr2=0x55a0907a6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- 172.21.15.33:0/3013128342 shutdown_connections 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- 172.21.15.33:0/3013128342 wait complete. 2024-04-24T23:25:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- start start 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 Processor -- start 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907ab230 0x55a0907b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907abc20 0x55a0907b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 0x55a0907bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0907b3ce0 con 0x55a0907b0d90 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0906cb050 con 0x55a0907abc20 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0907b3e60 con 0x55a0907ab230 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6aa1615740 10 client.0 _ll_register_callbacks cb 0x55a090422be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6a9cea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 0x55a0907bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6a9cea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 0x55a0907bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47448/0 (socket says 172.21.15.33:47448) 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6a9cea6640 1 -- 172.21.15.33:0/821753983 learned_addr learned my addr 172.21.15.33:0/821753983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6a97fff640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907ab230 0x55a0907b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.843+0000 7f6a977fe640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907abc20 0x55a0907b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a9cea6640 1 -- 172.21.15.33:0/821753983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907ab230 msgr2=0x55a0907b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907ab230 0x55a0907b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a9cea6640 1 -- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907abc20 msgr2=0x55a0907b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907abc20 0x55a0907b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a9cea6640 1 -- 172.21.15.33:0/821753983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09069cdc0 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a97fff640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907ab230 0x55a0907b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 0x55a0907bc8e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6a90020950 tx=0x7f6a90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769e65be57dba67c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a777fe640 1 -- 172.21.15.33:0/821753983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90010dc0 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a777fe640 1 -- 172.21.15.33:0/821753983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6a9001b440 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c005520 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a777fe640 1 -- 172.21.15.33:0/821753983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a90020e10 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6aa1615740 10 client.18267 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0904233f0 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.844+0000 7f6a777fe640 1 -- 172.21.15.33:0/821753983 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c005520 con 0x55a0907b0d90 2024-04-24T23:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6a777fe640 1 -- 172.21.15.33:0/821753983 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6a90027020 con 0x55a0907b0d90 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6aa1615740 10 client.18267 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6aa1615740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6aa1615740 20 client.18267 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6aa1615740 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a0907c1280 0x55a0907c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6aa1615740 4 client.18267 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a09071cbb0 con 0x55a0907c1280 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6a97fff640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a0907c1280 0x55a0907c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6a97fff640 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a0907c1280 0x55a0907c3700 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67024cbff3f22168 server_cookie=86e548ec529a2e21 in_seq=0 out_seq=0 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.845+0000 7f6a777fe640 10 client.18267 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6a777fe640 1 -- 172.21.15.33:0/821753983 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a09071cbb0 con 0x55a0907c1280 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6a777fe640 10 client.18267 handle_command_reply: tid=0 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 client.18267 shutdown 2024-04-24T23:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 msgr2=0x55a0907bc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 0x55a0907bc8e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6a90020950 tx=0x7f6a90004ab0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 shutdown_connections 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a0907c1280 0x55a0907c3700 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907b0d90 0x55a0907bc8e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907abc20 0x55a0907b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 --2- 172.21.15.33:0/821753983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907ab230 0x55a0907b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6a777fe640 0 client.18267 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 >> 172.21.15.33:0/821753983 conn(0x55a0907a68a0 msgr2=0x55a0907a5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 shutdown_connections 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 1 -- 172.21.15.33:0/821753983 wait complete. 2024-04-24T23:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.846+0000 7f6aa1615740 20 client.18267 trim_cache size 0 max 16384 2024-04-24T23:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 Processor -- start 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 -- start start 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907ab230 0x55a0907a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 0x55a0907bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907b0d90 0x55a0907aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981acc60 con 0x55a0907abc20 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981aa530 con 0x55a0907ab230 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.860+0000 7f6aa1615740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981a8380 con 0x55a0907b0d90 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 0x55a0907bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 0x55a0907bc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47460/0 (socket says 172.21.15.33:47460) 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 -- 172.21.15.33:0/1146141305 learned_addr learned my addr 172.21.15.33:0/1146141305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907ab230 0x55a0907a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907b0d90 0x55a0907aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 -- 172.21.15.33:0/1146141305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907b0d90 msgr2=0x55a0907aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907b0d90 0x55a0907aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 -- 172.21.15.33:0/1146141305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907ab230 msgr2=0x55a0907a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907ab230 0x55a0907a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 -- 172.21.15.33:0/1146141305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0904ed7d0 con 0x55a0907abc20 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a97fff640 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907ab230 0x55a0907a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a977fe640 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 0x55a0907bc8a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c005ec0 tx=0x7f6a8c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749de69c82a87e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a7dffb640 1 -- 172.21.15.33:0/1146141305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c053070 con 0x55a0907abc20 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a7dffb640 1 -- 172.21.15.33:0/1146141305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c005520 con 0x55a0907abc20 2024-04-24T23:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.861+0000 7f6a7dffb640 1 -- 172.21.15.33:0/1146141305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c035070 con 0x55a0907abc20 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 -- 172.21.15.33:0/1146141305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 msgr2=0x55a0907bc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 0x55a0907bc8a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c005ec0 tx=0x7f6a8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 -- 172.21.15.33:0/1146141305 shutdown_connections 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907b0d90 0x55a0907aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907abc20 0x55a0907bc8a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 --2- 172.21.15.33:0/1146141305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907ab230 0x55a0907a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 -- 172.21.15.33:0/1146141305 >> 172.21.15.33:0/1146141305 conn(0x55a0907a68a0 msgr2=0x55a0907a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 -- 172.21.15.33:0/1146141305 shutdown_connections 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 -- 172.21.15.33:0/1146141305 wait complete. 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 -- start start 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.862+0000 7f6aa1615740 1 Processor -- start 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907bab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907bbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981a89f0 con 0x55a0907ab230 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981a9e60 con 0x55a0907b0d90 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a981a90c0 con 0x55a0907abc20 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 10 client.0 _ll_register_callbacks cb 0x7f6a90009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47466/0 (socket says 172.21.15.33:47466) 2024-04-24T23:25:38.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 -- 172.21.15.33:0/622674758 learned_addr learned my addr 172.21.15.33:0/622674758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907bbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a977fe640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 -- 172.21.15.33:0/622674758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 msgr2=0x55a0907bab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907bab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 -- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 msgr2=0x55a0907bbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907bbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 -- 172.21.15.33:0/622674758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0907bc110 con 0x55a0907ab230 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a9cea6640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907bbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a97fff640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ba5d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6a8809b980 tx=0x7f6a8809bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4a8c5e8962530e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a777fe640 1 -- 172.21.15.33:0/622674758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880a0030 con 0x55a0907ab230 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6a777fe640 1 -- 172.21.15.33:0/622674758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6a88003b30 con 0x55a0907ab230 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.863+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c005520 con 0x55a0907ab230 2024-04-24T23:25:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6a777fe640 1 -- 172.21.15.33:0/622674758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a880a48f0 con 0x55a0907ab230 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 10 client.18273 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a090827f50 con 0x55a0907ab230 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6a777fe640 1 -- 172.21.15.33:0/622674758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c005520 con 0x55a0907ab230 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6a777fe640 1 -- 172.21.15.33:0/622674758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6a8809b160 con 0x55a0907ab230 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 10 client.18273 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 10 client.18273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 20 client.18273 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a090828880 0x55a09082aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 4 client.18273 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a090716f00 con 0x55a090828880 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.864+0000 7f6a977fe640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a090828880 0x55a09082aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6a977fe640 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a090828880 0x55a09082aca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ef514715c506be7 server_cookie=fe7dc5a76c83b019 in_seq=0 out_seq=0 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6a777fe640 10 client.18273 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:38.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6a777fe640 1 -- 172.21.15.33:0/622674758 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a090716f00 con 0x55a090828880 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6a777fe640 10 client.18273 handle_command_reply: tid=0 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 client.18273 shutdown 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 msgr2=0x55a0907ba5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ba5d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6a8809b980 tx=0x7f6a8809bb80 comp rx=0 tx=0).stop 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 shutdown_connections 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55a090828880 0x55a09082aca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0907b0d90 0x55a0907bbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0907abc20 0x55a0907bab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 --2- 172.21.15.33:0/622674758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0907ab230 0x55a0907ba5d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6a777fe640 0 client.18273 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 >> 172.21.15.33:0/622674758 conn(0x55a0907a68a0 msgr2=0x55a0907aa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.865+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 shutdown_connections 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6aa1615740 1 -- 172.21.15.33:0/622674758 wait complete. 2024-04-24T23:25:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6aa1615740 20 client.18273 trim_cache size 0 max 16384 2024-04-24T23:25:38.873 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:25:38.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6a800a2b90 msgr2=0x7f6a800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6a800a2b90 0x7f6a800a5050 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009fd0 tx=0x7f6a90004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 msgr2=0x7f6a98197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 0x7f6a98197ab0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e9c0 tx=0x7f6a8800ee80 comp rx=0 tx=0).stop 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 shutdown_connections 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6a800a2b90 0x7f6a800a5050 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a98102230 0x7f6a9819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a98101870 0x7f6a98197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 --2- 172.21.15.33:0/208128894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a98100680 0x7f6a98197ab0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 >> 172.21.15.33:0/208128894 conn(0x7f6a980fbbf0 msgr2=0x7f6a98104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 shutdown_connections 2024-04-24T23:25:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.866+0000 7f6a9e909640 1 -- 172.21.15.33:0/208128894 wait complete. 2024-04-24T23:25:38.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:25:38.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.984+0000 7f99ac5b7640 1 -- 172.21.15.33:0/819819709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a4100ff0 msgr2=0x7f99a4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.984+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/819819709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a4100ff0 0x7f99a4101470 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9994009900 tx=0x7f999402a410 comp rx=0 tx=0).stop 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- 172.21.15.33:0/819819709 shutdown_connections 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/819819709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a41019b0 0x7f99a4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/819819709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a4100ff0 0x7f99a4101470 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/819819709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 0x7f99a4100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- 172.21.15.33:0/819819709 >> 172.21.15.33:0/819819709 conn(0x7f99a40fb370 msgr2=0x7f99a40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- 172.21.15.33:0/819819709 shutdown_connections 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- 172.21.15.33:0/819819709 wait complete. 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 Processor -- start 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- start start 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 0x7f99a419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a4100ff0 0x7f99a419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 0x7f99a41a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a410c310 con 0x7f99a41019b0 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99a410c190 con 0x7f99a40ffe00 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99ac5b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99a410c490 con 0x7f99a4100ff0 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99aab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 0x7f99a41a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99aa353640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 0x7f99a419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99aab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 0x7f99a41a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47474/0 (socket says 172.21.15.33:47474) 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.985+0000 7f99aab54640 1 -- 172.21.15.33:0/3827479169 learned_addr learned my addr 172.21.15.33:0/3827479169 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a4100ff0 0x7f99a419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aa353640 1 -- 172.21.15.33:0/3827479169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a4100ff0 msgr2=0x7f99a419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aa353640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a4100ff0 0x7f99a419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aa353640 1 -- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 msgr2=0x7f99a41a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aa353640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 0x7f99a41a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aa353640 1 -- 172.21.15.33:0/3827479169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a41a5db0 con 0x7f99a40ffe00 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a4100ff0 0x7f99a419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aab54640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 0x7f99a41a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99aa353640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 0x7f99a419e830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f999b7fe640 1 -- 172.21.15.33:0/3827479169 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ccc0 con 0x7f99a40ffe00 2024-04-24T23:25:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41a5fe0 con 0x7f99a40ffe00 2024-04-24T23:25:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f999b7fe640 1 -- 172.21.15.33:0/3827479169 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a00035f0 con 0x7f99a40ffe00 2024-04-24T23:25:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41a64f0 con 0x7f99a40ffe00 2024-04-24T23:25:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.986+0000 7f999b7fe640 1 -- 172.21.15.33:0/3827479169 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ce60 con 0x7f99a40ffe00 2024-04-24T23:25:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 Processor -- start 2024-04-24T23:25:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f999b7fe640 1 -- 172.21.15.33:0/3827479169 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f99a00037a0 con 0x7f99a40ffe00 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 -- start start 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 0x565541432650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432c20 0x565541437720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x56554143a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565541396670 con 0x565541432230 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56554133e970 con 0x565541432c20 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99af3a5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565541392630 con 0x565541437d90 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 0x565541432650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 0x565541432650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47488/0 (socket says 172.21.15.33:47488) 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aa353640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432c20 0x565541437720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 -- 172.21.15.33:0/3756248640 learned_addr learned my addr 172.21.15.33:0/3756248640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x56554143a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 -- 172.21.15.33:0/3756248640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 msgr2=0x56554143a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f999b7fe640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x56554143a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 -- 172.21.15.33:0/3756248640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432c20 msgr2=0x565541437720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432c20 0x565541437720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aab54640 1 -- 172.21.15.33:0/3756248640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565541362400 con 0x565541432230 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.989+0000 7f99aa353640 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432c20 0x565541437720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.990+0000 7f999b7fe640 1 -- 172.21.15.33:0/3827479169 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99a00cf8e0 con 0x7f99a40ffe00 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.990+0000 7f99aab54640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.990+0000 7f99aab54640 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 0x565541432650 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f999c0118b0 tx=0x7f999c011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d70f5817048edd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.990+0000 7f9998bfe640 1 -- 172.21.15.33:0/3756248640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c00fa80 con 0x565541432230 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.990+0000 7f9998bfe640 1 -- 172.21.15.33:0/3756248640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c00fca0 con 0x565541432230 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.990+0000 7f9998bfe640 1 -- 172.21.15.33:0/3756248640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c019400 con 0x565541432230 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 -- 172.21.15.33:0/3756248640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 msgr2=0x565541432650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99aab54640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99900a2c10 0x7f99900a50d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f999c012fd0 tx=0x7f999c021000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 0x565541432650 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f999c0118b0 tx=0x7f999c011d70 comp rx=0 tx=0).stop 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 -- 172.21.15.33:0/3756248640 shutdown_connections 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x56554143a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432c20 0x565541437720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3756248640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432230 0x565541432650 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 -- 172.21.15.33:0/3756248640 >> 172.21.15.33:0/3756248640 conn(0x56554142d8a0 msgr2=0x56554142dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 -- 172.21.15.33:0/3756248640 shutdown_connections 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 -- 172.21.15.33:0/3756248640 wait complete. 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 -- start start 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.991+0000 7f99af3a5740 1 Processor -- start 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x56554143c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 0x56554143c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x5655414437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56554143ad40 con 0x565541432c20 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x565541352050 con 0x565541437d90 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56554143aec0 con 0x565541432230 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99af3a5740 10 client.0 _ll_register_callbacks cb 0x5655410a9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 0x56554143c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 0x56554143c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47498/0 (socket says 172.21.15.33:47498) 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 -- 172.21.15.33:0/3350497182 learned_addr learned my addr 172.21.15.33:0/3350497182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99aab54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x5655414437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99aa353640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x56554143c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 -- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 msgr2=0x56554143c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x56554143c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 -- 172.21.15.33:0/3350497182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 msgr2=0x5655414437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x5655414437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99a9b52640 1 -- 172.21.15.33:0/3350497182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565541323dc0 con 0x565541432c20 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99aa353640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x56554143c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.992+0000 7f99aab54640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x5655414437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 0x56554143c790 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f999402a930 tx=0x7f9994031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba7b3f84f9393f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f998dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f998dffb640 1 -- 172.21.15.33:0/3350497182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994035070 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999c00fca0 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f998dffb640 1 -- 172.21.15.33:0/3350497182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9994004240 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f998dffb640 1 -- 172.21.15.33:0/3350497182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f999404d050 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f99af3a5740 10 client.18285 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5655410aa3f0 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f998dffb640 1 -- 172.21.15.33:0/3350497182 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c00fca0 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.993+0000 7f998dffb640 1 -- 172.21.15.33:0/3350497182 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f99940049b0 con 0x565541432c20 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99af3a5740 10 client.18285 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99af3a5740 10 client.18285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99af3a5740 20 client.18285 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x565541448090 0x56554144a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99af3a5740 4 client.18285 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655413a3bb0 con 0x565541448090 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99aa353640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x565541448090 0x56554144a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f99aa353640 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x565541448090 0x56554144a550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=628beebb979ea207 server_cookie=4f4ad12234a83f0 in_seq=0 out_seq=0 2024-04-24T23:25:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.994+0000 7f998dffb640 10 client.18285 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f998dffb640 1 -- 172.21.15.33:0/3350497182 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5655413a3bb0 con 0x565541448090 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f998dffb640 10 client.18285 handle_command_reply: tid=0 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 client.18285 shutdown 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 msgr2=0x56554143c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 0x56554143c790 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f999402a930 tx=0x7f9994031960 comp rx=0 tx=0).stop 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 shutdown_connections 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x565541448090 0x56554144a550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x5655414437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541432c20 0x56554143c790 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3350497182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x56554143c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 >> 172.21.15.33:0/3350497182 conn(0x56554142d8a0 msgr2=0x56554142cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f998dffb640 0 client.18285 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 shutdown_connections 2024-04-24T23:25:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 1 -- 172.21.15.33:0/3350497182 wait complete. 2024-04-24T23:25:39.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:38.995+0000 7f99af3a5740 20 client.18285 trim_cache size 0 max 16384 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.018+0000 7f99af3a5740 1 Processor -- start 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.018+0000 7f99af3a5740 1 -- start start 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x5655414336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x565541443790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x5655414340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99af3a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41b3370 con 0x565541433c00 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99af3a5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41b0c40 con 0x565541437d90 2024-04-24T23:25:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99af3a5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41aea90 con 0x565541432230 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99aab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x5655414340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x565541443790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99aab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x5655414340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47500/0 (socket says 172.21.15.33:47500) 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99aa353640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x5655414336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99aab54640 1 -- 172.21.15.33:0/3117813224 learned_addr learned my addr 172.21.15.33:0/3117813224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 -- 172.21.15.33:0/3117813224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 msgr2=0x5655414336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x5655414336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 -- 172.21.15.33:0/3117813224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 msgr2=0x5655414340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x5655414340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 -- 172.21.15.33:0/3117813224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655411747d0 con 0x565541437d90 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99aab54640 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x5655414340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.019+0000 7f99a9b52640 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x565541443790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f999403e000 tx=0x7f9994001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c388493143038ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f998ffff640 1 -- 172.21.15.33:0/3117813224 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994056070 con 0x565541437d90 2024-04-24T23:25:39.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f998ffff640 1 -- 172.21.15.33:0/3117813224 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99940035f0 con 0x565541437d90 2024-04-24T23:25:39.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f998ffff640 1 -- 172.21.15.33:0/3117813224 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994035070 con 0x565541437d90 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 -- 172.21.15.33:0/3117813224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 msgr2=0x565541443790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x565541443790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f999403e000 tx=0x7f9994001e70 comp rx=0 tx=0).stop 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 -- 172.21.15.33:0/3117813224 shutdown_connections 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x5655414340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541437d90 0x565541443790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 --2- 172.21.15.33:0/3117813224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541432230 0x5655414336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.020+0000 7f99af3a5740 1 -- 172.21.15.33:0/3117813224 >> 172.21.15.33:0/3117813224 conn(0x56554142d8a0 msgr2=0x56554143c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 -- 172.21.15.33:0/3117813224 shutdown_connections 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 -- 172.21.15.33:0/3117813224 wait complete. 2024-04-24T23:25:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 -- start start 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 Processor -- start 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432230 0x565541440790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x565541440cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x5655414ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41af100 con 0x565541433c00 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41b0570 con 0x565541432230 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41af7d0 con 0x565541437d90 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99af3a5740 10 client.0 _ll_register_callbacks cb 0x7f9994009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99a9b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x565541440cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99aa353640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432230 0x565541440790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99a9b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x565541440cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47512/0 (socket says 172.21.15.33:47512) 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99a9b52640 1 -- 172.21.15.33:0/232791702 learned_addr learned my addr 172.21.15.33:0/232791702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99aab54640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x5655414ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.021+0000 7f99a9b52640 1 -- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 msgr2=0x5655414ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99a9b52640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x5655414ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99a9b52640 1 -- 172.21.15.33:0/232791702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432230 msgr2=0x565541440790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99a9b52640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432230 0x565541440790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99a9b52640 1 -- 172.21.15.33:0/232791702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655414acc40 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99aa353640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432230 0x565541440790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99a9b52640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x565541440cd0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9994004e00 tx=0x7f9994031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdd5f3395b6e855 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f998dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f998dffb640 1 -- 172.21.15.33:0/232791702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994035070 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99940035f0 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f998dffb640 1 -- 172.21.15.33:0/232791702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99940048c0 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f998dffb640 1 -- 172.21.15.33:0/232791702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9994056070 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 10 client.18291 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5655414adf40 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f998dffb640 1 -- 172.21.15.33:0/232791702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99940035f0 con 0x565541433c00 2024-04-24T23:25:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f998dffb640 1 -- 172.21.15.33:0/232791702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9994051430 con 0x565541433c00 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 10 client.18291 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 10 client.18291 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 20 client.18291 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5655414ae7d0 0x5655414b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 4 client.18291 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.022+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56554139df00 con 0x5655414ae7d0 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99aa353640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5655414ae7d0 0x5655414b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f998dffb640 10 client.18291 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99aa353640 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5655414ae7d0 0x5655414b0bf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7439758d7fc522dd server_cookie=c3a740f6a49a43ef in_seq=0 out_seq=0 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f998dffb640 1 -- 172.21.15.33:0/232791702 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56554139df00 con 0x5655414ae7d0 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f998dffb640 10 client.18291 handle_command_reply: tid=0 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 client.18291 shutdown 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 msgr2=0x565541440cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x565541440cd0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9994004e00 tx=0x7f9994031400 comp rx=0 tx=0).stop 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 shutdown_connections 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5655414ae7d0 0x5655414b0bf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565541437d90 0x5655414ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565541433c00 0x565541440cd0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 --2- 172.21.15.33:0/232791702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565541432230 0x565541440790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 >> 172.21.15.33:0/232791702 conn(0x56554142d8a0 msgr2=0x56554142c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.023+0000 7f998dffb640 0 client.18291 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 shutdown_connections 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99af3a5740 1 -- 172.21.15.33:0/232791702 wait complete. 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99af3a5740 20 client.18291 trim_cache size 0 max 16384 2024-04-24T23:25:39.031 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:25:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99900a2c10 msgr2=0x7f99900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99900a2c10 0x7f99900a50d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f999c012fd0 tx=0x7f999c021000 comp rx=0 tx=0).stop 2024-04-24T23:25:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 msgr2=0x7f99a419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 0x7f99a419e830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).stop 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 shutdown_connections 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a41019b0 0x7f99a41a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a4100ff0 0x7f99a419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 --2- 172.21.15.33:0/3827479169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99a40ffe00 0x7f99a419e830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 >> 172.21.15.33:0/3827479169 conn(0x7f99a40fb370 msgr2=0x7f99a40fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 shutdown_connections 2024-04-24T23:25:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.024+0000 7f99ac5b7640 1 -- 172.21.15.33:0/3827479169 wait complete. 2024-04-24T23:25:39.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:25:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 -- 172.21.15.33:0/2748064319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94101c90 msgr2=0x7f7e941020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/2748064319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94101c90 0x7f7e941020b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009930 tx=0x7f7e8402a420 comp rx=0 tx=0).stop 2024-04-24T23:25:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 -- 172.21.15.33:0/2748064319 shutdown_connections 2024-04-24T23:25:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/2748064319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e941012f0 0x7f7e94108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/2748064319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e94100930 0x7f7e94100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/2748064319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94101c90 0x7f7e941020b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 -- 172.21.15.33:0/2748064319 >> 172.21.15.33:0/2748064319 conn(0x7f7e940fb380 msgr2=0x7f7e940fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 -- 172.21.15.33:0/2748064319 shutdown_connections 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.143+0000 7f7e9a75a640 1 -- 172.21.15.33:0/2748064319 wait complete. 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 Processor -- start 2024-04-24T23:25:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 -- start start 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 0x7f7e94197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e941012f0 0x7f7e94198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e94101c90 0x7f7e9419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9410c320 con 0x7f7e94100930 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9410c1a0 con 0x7f7e941012f0 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.144+0000 7f7e9a75a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9410c4a0 con 0x7f7e94101c90 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 0x7f7e94197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 0x7f7e94197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47532/0 (socket says 172.21.15.33:47532) 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 -- 172.21.15.33:0/3004436265 learned_addr learned my addr 172.21.15.33:0/3004436265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e937fe640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e941012f0 0x7f7e94198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e98cf7640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e94101c90 0x7f7e9419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 -- 172.21.15.33:0/3004436265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e94101c90 msgr2=0x7f7e9419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e94101c90 0x7f7e9419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 -- 172.21.15.33:0/3004436265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e941012f0 msgr2=0x7f7e94198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e941012f0 0x7f7e94198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 -- 172.21.15.33:0/3004436265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9419f760 con 0x7f7e94100930 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 0x7f7e94197b40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005ab0 tx=0x7f7e84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e917fa640 1 -- 172.21.15.33:0/3004436265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8403f070 con 0x7f7e94100930 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9419f9f0 con 0x7f7e94100930 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e917fa640 1 -- 172.21.15.33:0/3004436265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84003cd0 con 0x7f7e94100930 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e917fa640 1 -- 172.21.15.33:0/3004436265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84035070 con 0x7f7e94100930 2024-04-24T23:25:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.145+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9419ff30 con 0x7f7e94100930 2024-04-24T23:25:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e917fa640 1 -- 172.21.15.33:0/3004436265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7e8403b560 con 0x7f7e94100930 2024-04-24T23:25:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 Processor -- start 2024-04-24T23:25:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 -- start start 2024-04-24T23:25:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab88791650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab88796720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab88799220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab886f5670 con 0x55ab88791c20 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ab8869d970 con 0x55ab88791230 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e9d465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab886f1630 con 0x55ab88796d90 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab88796720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e98cf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab88791650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab88796720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47536/0 (socket says 172.21.15.33:47536) 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e93fff640 1 -- 172.21.15.33:0/1673659654 learned_addr learned my addr 172.21.15.33:0/1673659654 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.148+0000 7f7e98cf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab88791650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43952/0 (socket says 172.21.15.33:43952) 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e917fa640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7e7c0a2b90 0x7f7e7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e937fe640 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab88799220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e98cf7640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7e7c0a2b90 0x7f7e7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e917fa640 1 -- 172.21.15.33:0/3004436265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7e8403be00 con 0x7f7e94100930 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e93fff640 1 -- 172.21.15.33:0/1673659654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 msgr2=0x55ab88799220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e93fff640 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab88799220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e93fff640 1 -- 172.21.15.33:0/1673659654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 msgr2=0x55ab88791650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e93fff640 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab88791650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e93fff640 1 -- 172.21.15.33:0/1673659654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab886c1400 con 0x55ab88791c20 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e937fe640 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab88799220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e98cf7640 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab88791650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e93fff640 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab88796720 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7e840ed040 tx=0x7f7e84009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6309aeef15528beb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e7a7fc640 1 -- 172.21.15.33:0/1673659654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84048070 con 0x55ab88791c20 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e98cf7640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7e7c0a2b90 0x7f7e7c0a5050 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55ab88792480 tx=0x7f7e8c004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e7a7fc640 1 -- 172.21.15.33:0/1673659654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e840bbf10 con 0x55ab88791c20 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.149+0000 7f7e7a7fc640 1 -- 172.21.15.33:0/1673659654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e840b7c60 con 0x55ab88791c20 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 -- 172.21.15.33:0/1673659654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 msgr2=0x55ab88796720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab88796720 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7e840ed040 tx=0x7f7e84009260 comp rx=0 tx=0).stop 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 -- 172.21.15.33:0/1673659654 shutdown_connections 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab88799220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab88796720 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 --2- 172.21.15.33:0/1673659654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab88791650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 -- 172.21.15.33:0/1673659654 >> 172.21.15.33:0/1673659654 conn(0x55ab8878c8a0 msgr2=0x55ab8878cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 -- 172.21.15.33:0/1673659654 shutdown_connections 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 -- 172.21.15.33:0/1673659654 wait complete. 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 -- start start 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.150+0000 7f7e9d465740 1 Processor -- start 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab8879b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab8879b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab887a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab88799b00 con 0x55ab88791c20 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ab886b1050 con 0x55ab88791230 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab88799c80 con 0x55ab88796d90 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e9d465740 10 client.0 _ll_register_callbacks cb 0x55ab88408be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab8879b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab8879b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab8879b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47550/0 (socket says 172.21.15.33:47550) 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e98cf7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab887a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab8879b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:43968/0 (socket says 172.21.15.33:43968) 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e937fe640 1 -- 172.21.15.33:0/3852240539 learned_addr learned my addr 172.21.15.33:0/3852240539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 -- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 msgr2=0x55ab887a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab887a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 -- 172.21.15.33:0/3852240539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 msgr2=0x55ab8879b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab8879b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 -- 172.21.15.33:0/3852240539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab88682dc0 con 0x55ab88791230 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e937fe640 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab8879b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab8879b030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e840c03d0 tx=0x7f7e84009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e2bfa8afa0a512 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e737fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.151+0000 7f7e737fe640 1 -- 172.21.15.33:0/3852240539 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84048070 con 0x55ab88791230 2024-04-24T23:25:39.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e840bbf10 con 0x55ab88791230 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e737fe640 1 -- 172.21.15.33:0/3852240539 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7e840c4640 con 0x55ab88791230 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e737fe640 1 -- 172.21.15.33:0/3852240539 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e840b81b0 con 0x55ab88791230 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 10 client.27818 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab884093f0 con 0x55ab88791230 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e737fe640 1 -- 172.21.15.33:0/3852240539 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e840bbf10 con 0x55ab88791230 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e737fe640 1 -- 172.21.15.33:0/3852240539 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e8404b060 con 0x55ab88791230 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 10 client.27818 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 10 client.27818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 20 client.27818 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab887a6c70 0x55ab887a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 4 client.27818 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab88702bb0 con 0x55ab887a6c70 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.152+0000 7f7e937fe640 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab887a6c70 0x55ab887a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.153+0000 7f7e937fe640 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab887a6c70 0x55ab887a90a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520c219e6cb72e13 server_cookie=1c1ead1ad373394d in_seq=0 out_seq=0 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.153+0000 7f7e737fe640 10 client.27818 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.153+0000 7f7e737fe640 1 -- 172.21.15.33:0/3852240539 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab88702bb0 con 0x55ab887a6c70 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.153+0000 7f7e737fe640 10 client.27818 handle_command_reply: tid=0 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.153+0000 7f7e9d465740 1 client.27818 shutdown 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 msgr2=0x55ab8879b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab8879b030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e840c03d0 tx=0x7f7e84009560 comp rx=0 tx=0).stop 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 shutdown_connections 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab887a6c70 0x55ab887a90a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88796d90 0x55ab887a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88791c20 0x55ab8879b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3852240539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791230 0x55ab8879b030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e737fe640 0 client.27818 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 >> 172.21.15.33:0/3852240539 conn(0x55ab8878c8a0 msgr2=0x55ab8878bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 shutdown_connections 2024-04-24T23:25:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 1 -- 172.21.15.33:0/3852240539 wait complete. 2024-04-24T23:25:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.154+0000 7f7e9d465740 20 client.27818 trim_cache size 0 max 16384 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 Processor -- start 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 -- start start 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8878f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab887a2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8878fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941aab10 con 0x55ab88796d90 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941a83e0 con 0x55ab88791c20 2024-04-24T23:25:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.167+0000 7f7e9d465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941a6230 con 0x55ab88791230 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e98cf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8878fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8878f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab887a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8878f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42654/0 (socket says 172.21.15.33:42654) 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 -- 172.21.15.33:0/3482020560 learned_addr learned my addr 172.21.15.33:0/3482020560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 -- 172.21.15.33:0/3482020560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 msgr2=0x55ab887a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab887a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 -- 172.21.15.33:0/3482020560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 msgr2=0x55ab8878fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8878fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 -- 172.21.15.33:0/3482020560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab884d37d0 con 0x55ab88791230 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e98cf7640 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8878fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e93fff640 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8878f580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7e840314e0 tx=0x7f7e840b8580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9ec14fe2d22ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e79ffb640 1 -- 172.21.15.33:0/3482020560 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84048070 con 0x55ab88791230 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e79ffb640 1 -- 172.21.15.33:0/3482020560 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e840f5070 con 0x55ab88791230 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.168+0000 7f7e79ffb640 1 -- 172.21.15.33:0/3482020560 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e840bbf10 con 0x55ab88791230 2024-04-24T23:25:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 -- 172.21.15.33:0/3482020560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 msgr2=0x55ab8878f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8878f580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7e840314e0 tx=0x7f7e840b8580 comp rx=0 tx=0).stop 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 -- 172.21.15.33:0/3482020560 shutdown_connections 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8878fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab887a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 --2- 172.21.15.33:0/3482020560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8878f580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 -- 172.21.15.33:0/3482020560 >> 172.21.15.33:0/3482020560 conn(0x55ab8878c8a0 msgr2=0x55ab8878bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 -- 172.21.15.33:0/3482020560 shutdown_connections 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 -- 172.21.15.33:0/3482020560 wait complete. 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 -- start start 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.169+0000 7f7e9d465740 1 Processor -- start 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8879f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab8879f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8880b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941a68a0 con 0x55ab88796d90 2024-04-24T23:25:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941a7d10 con 0x55ab88791c20 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941a6f70 con 0x55ab88791230 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 10 client.0 _ll_register_callbacks cb 0x7f7e8c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e98cf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8880b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8879f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e98cf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8880b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47576/0 (socket says 172.21.15.33:47576) 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e98cf7640 1 -- 172.21.15.33:0/2365316923 learned_addr learned my addr 172.21.15.33:0/2365316923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e937fe640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab8879f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 -- 172.21.15.33:0/2365316923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 msgr2=0x55ab8879f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab8879f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 -- 172.21.15.33:0/2365316923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 msgr2=0x55ab8880b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8880b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 -- 172.21.15.33:0/2365316923 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab8879fed0 con 0x55ab88791230 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e98cf7640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8880b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e93fff640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8879f2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ab8878c870 tx=0x7f7e840b7fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4333bf5ce80fd24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e737fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e737fe640 1 -- 172.21.15.33:0/2365316923 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84048070 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e840f5070 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.170+0000 7f7e737fe640 1 -- 172.21.15.33:0/2365316923 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7e840c4640 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e737fe640 1 -- 172.21.15.33:0/2365316923 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e840bb870 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 10 client.27766 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab8880c810 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e737fe640 1 -- 172.21.15.33:0/2365316923 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e840f5070 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e737fe640 1 -- 172.21.15.33:0/2365316923 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e84031b20 con 0x55ab88791230 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 10 client.27766 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 10 client.27766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 20 client.27766 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab8880ce70 0x55ab8880f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 4 client.27766 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab886fcf00 con 0x55ab8880ce70 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.171+0000 7f7e937fe640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab8880ce70 0x55ab8880f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e937fe640 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab8880ce70 0x55ab8880f260 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ecc6c9ca2dffc server_cookie=963711f41a1d7228 in_seq=0 out_seq=0 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e737fe640 10 client.27766 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e737fe640 1 -- 172.21.15.33:0/2365316923 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab886fcf00 con 0x55ab8880ce70 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e737fe640 10 client.27766 handle_command_reply: tid=0 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 client.27766 shutdown 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 msgr2=0x55ab8879f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8879f2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ab8878c870 tx=0x7f7e840b7fe0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 shutdown_connections 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ab8880ce70 0x55ab8880f260 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab88796d90 0x55ab8880b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e737fe640 0 client.27766 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab88791c20 0x55ab8879f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 --2- 172.21.15.33:0/2365316923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab88791230 0x55ab8879f2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 >> 172.21.15.33:0/2365316923 conn(0x55ab8878c8a0 msgr2=0x55ab8878f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 shutdown_connections 2024-04-24T23:25:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 1 -- 172.21.15.33:0/2365316923 wait complete. 2024-04-24T23:25:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.172+0000 7f7e9d465740 20 client.27766 trim_cache size 0 max 16384 2024-04-24T23:25:39.181 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7e7c0a2b90 msgr2=0x7f7e7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7e7c0a2b90 0x7f7e7c0a5050 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55ab88792480 tx=0x7f7e8c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 msgr2=0x7f7e94197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 0x7f7e94197b40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005ab0 tx=0x7f7e84004270 comp rx=0 tx=0).stop 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 shutdown_connections 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7e7c0a2b90 0x7f7e7c0a5050 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e94101c90 0x7f7e9419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e941012f0 0x7f7e94198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 --2- 172.21.15.33:0/3004436265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e94100930 0x7f7e94197b40 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 >> 172.21.15.33:0/3004436265 conn(0x7f7e940fb380 msgr2=0x7f7e940ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 shutdown_connections 2024-04-24T23:25:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:39.173+0000 7f7e9a75a640 1 -- 172.21.15.33:0/3004436265 wait complete. 2024-04-24T23:25:39.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:39 smithi033 ceph-mon[29201]: pgmap v760: 129 pgs: 129 active+clean; 282 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T23:25:39.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:39 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3084893414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:25:39.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1620242 12.29 MB/sec execute 584 sec latency 11.680 ms 2024-04-24T23:25:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:39 smithi038 ceph-mon[32450]: pgmap v760: 129 pgs: 129 active+clean; 282 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T23:25:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:39 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3084893414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:25:39.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:39 smithi117 ceph-mon[32529]: pgmap v760: 129 pgs: 129 active+clean; 282 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T23:25:39.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:39 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3084893414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:25:40.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:25:40.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:40.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 -- 172.21.15.33:0/3329205890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0102230 msgr2=0x7feaa0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 --2- 172.21.15.33:0/3329205890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0102230 0x7feaa0108ac0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fea98009ca0 tx=0x7fea9802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 -- 172.21.15.33:0/3329205890 shutdown_connections 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 --2- 172.21.15.33:0/3329205890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0102230 0x7feaa0108ac0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 --2- 172.21.15.33:0/3329205890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 0x7feaa0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 --2- 172.21.15.33:0/3329205890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0100680 0x7feaa0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 -- 172.21.15.33:0/3329205890 >> 172.21.15.33:0/3329205890 conn(0x7feaa00fbbf0 msgr2=0x7feaa00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 -- 172.21.15.33:0/3329205890 shutdown_connections 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.295+0000 7feaa64a5640 1 -- 172.21.15.33:0/3329205890 wait complete. 2024-04-24T23:25:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 Processor -- start 2024-04-24T23:25:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 -- start start 2024-04-24T23:25:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 0x7feaa0196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 0x7feaa0196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0102230 0x7feaa019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feaa010c2e0 con 0x7feaa0100680 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feaa010c160 con 0x7feaa0101870 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7feaa64a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaa010c460 con 0x7feaa0102230 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 0x7feaa0196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 0x7feaa0196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.296+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 0x7feaa0196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47588/0 (socket says 172.21.15.33:47588) 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 -- 172.21.15.33:0/4080726572 learned_addr learned my addr 172.21.15.33:0/4080726572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7feaa4a42640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0102230 0x7feaa019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 -- 172.21.15.33:0/4080726572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0102230 msgr2=0x7feaa019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0102230 0x7feaa019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 -- 172.21.15.33:0/4080726572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 msgr2=0x7feaa0196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 0x7feaa0196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 -- 172.21.15.33:0/4080726572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa019e570 con 0x7feaa0100680 2024-04-24T23:25:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7feaa4a42640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0102230 0x7feaa019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 0x7feaa0196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9ffff640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 0x7feaa0196950 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fea9000e9c0 tx=0x7fea9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9d7fa640 1 -- 172.21.15.33:0/4080726572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9000cd80 con 0x7feaa0100680 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9d7fa640 1 -- 172.21.15.33:0/4080726572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea900035f0 con 0x7feaa0100680 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa019e860 con 0x7feaa0100680 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7fea9d7fa640 1 -- 172.21.15.33:0/4080726572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90010690 con 0x7feaa0100680 2024-04-24T23:25:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.297+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa019ee00 con 0x7feaa0100680 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.299+0000 7fea9d7fa640 1 -- 172.21.15.33:0/4080726572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea900037a0 con 0x7feaa0100680 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 Processor -- start 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 -- start start 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1df0e0 0x558d6b1df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e90f0 0x558d6b1eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558d6b1422c0 con 0x558d6b1e0450 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558d6b0ea910 con 0x558d6b1df0e0 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa91b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d6b13e280 con 0x558d6b1e90f0 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa4a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa4a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1deba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47602/0 (socket says 172.21.15.33:47602) 2024-04-24T23:25:40.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa4a42640 1 -- 172.21.15.33:0/365205611 learned_addr learned my addr 172.21.15.33:0/365205611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9ffff640 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1df0e0 0x558d6b1df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e90f0 0x558d6b1eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9d7fa640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fea880a2c10 0x7fea880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9ffff640 1 -- 172.21.15.33:0/365205611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e90f0 msgr2=0x558d6b1eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9ffff640 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e90f0 0x558d6b1eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9ffff640 1 -- 172.21.15.33:0/365205611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 msgr2=0x558d6b1deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9ffff640 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1deba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9ffff640 1 -- 172.21.15.33:0/365205611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6b0fbfb0 con 0x558d6b1df0e0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa4a42640 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7feaa4a42640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fea880a2c10 0x7fea880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.300+0000 7fea9d7fa640 1 -- 172.21.15.33:0/4080726572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea900cf7e0 con 0x7feaa0100680 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.301+0000 7fea9ffff640 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1df0e0 0x558d6b1df560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea900ccf20 tx=0x7fea9005bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0617abf7e3b05e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.301+0000 7feaa4a42640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fea880a2c10 0x7fea880a50d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fea98006010 tx=0x7fea980095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.301+0000 7fea867fc640 1 -- 172.21.15.33:0/365205611 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9009ba80 con 0x558d6b1df0e0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.301+0000 7fea867fc640 1 -- 172.21.15.33:0/365205611 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9009c3b0 con 0x558d6b1df0e0 2024-04-24T23:25:40.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.301+0000 7fea867fc640 1 -- 172.21.15.33:0/365205611 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9009b610 con 0x558d6b1df0e0 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- 172.21.15.33:0/365205611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1df0e0 msgr2=0x558d6b1df560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1df0e0 0x558d6b1df560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea900ccf20 tx=0x7fea9005bda0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- 172.21.15.33:0/365205611 shutdown_connections 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e90f0 0x558d6b1eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1df0e0 0x558d6b1df560 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- 172.21.15.33:0/365205611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- 172.21.15.33:0/365205611 >> 172.21.15.33:0/365205611 conn(0x558d6b1a6950 msgr2=0x558d6b1da810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- 172.21.15.33:0/365205611 shutdown_connections 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- 172.21.15.33:0/365205611 wait complete. 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- start start 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 Processor -- start 2024-04-24T23:25:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 0x558d6b1ed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e0450 0x558d6b1eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558d6b1ebc00 con 0x558d6b1df0e0 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558d6b0fde90 con 0x558d6b1e90f0 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d6b1ebd80 con 0x558d6b1e0450 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.302+0000 7feaa91b8740 10 client.0 _ll_register_callbacks cb 0x558d6b139970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 0x558d6b1ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e0450 0x558d6b1eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 0x558d6b1ed510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47608/0 (socket says 172.21.15.33:47608) 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9ffff640 1 -- 172.21.15.33:0/1265001788 learned_addr learned my addr 172.21.15.33:0/1265001788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7feaa4a42640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 -- 172.21.15.33:0/1265001788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 msgr2=0x558d6b1f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 -- 172.21.15.33:0/1265001788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 msgr2=0x558d6b1ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 0x558d6b1ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 -- 172.21.15.33:0/1265001788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6af1e7e0 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9ffff640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 0x558d6b1ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e0450 0x558d6b1eda50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x558d6b1dfb80 tx=0x7fea9400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5062d4d64f7ca28f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1265001788 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94012110 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9009c3b0 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1265001788 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea94004050 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1265001788 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea94025050 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7feaa91b8740 10 client.27778 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.303+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d6af7ef70 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1265001788 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9009c3b0 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1265001788 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fea94010430 con 0x558d6b1e0450 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7feaa91b8740 10 client.27778 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7feaa91b8740 10 client.27778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7feaa91b8740 20 client.27778 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b1f9030 0x558d6b1fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7feaa91b8740 4 client.27778 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d6b1165a0 con 0x558d6b1f9030 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7fea9ffff640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b1f9030 0x558d6b1fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7fea9ffff640 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b1f9030 0x558d6b1fb4d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4477de85218631a server_cookie=2436262a2d2f2e95 in_seq=0 out_seq=0 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.304+0000 7fea7f7fe640 10 client.27778 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.305+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1265001788 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d6b1165a0 con 0x558d6b1f9030 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.305+0000 7fea7f7fe640 10 client.27778 handle_command_reply: tid=0 2024-04-24T23:25:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.305+0000 7feaa91b8740 1 client.27778 shutdown 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e0450 msgr2=0x558d6b1eda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e0450 0x558d6b1eda50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x558d6b1dfb80 tx=0x7fea9400f790 comp rx=0 tx=0).stop 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 shutdown_connections 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b1f9030 0x558d6b1fb4d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1e0450 0x558d6b1eda50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7fea7f7fe640 0 client.27778 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1265001788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1df0e0 0x558d6b1ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 >> 172.21.15.33:0/1265001788 conn(0x558d6b1a6950 msgr2=0x558d6b1d9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 shutdown_connections 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 1 -- 172.21.15.33:0/1265001788 wait complete. 2024-04-24T23:25:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.306+0000 7feaa91b8740 20 client.27778 trim_cache size 0 max 16384 2024-04-24T23:25:40.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.325+0000 7feaa91b8740 1 Processor -- start 2024-04-24T23:25:40.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.325+0000 7feaa91b8740 1 -- start start 2024-04-24T23:25:40.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa91b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01aab00 con 0x558d6b1e0450 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa91b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01a83d0 con 0x558d6b1e90f0 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa91b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01a6220 con 0x558d6b1df0e0 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47614/0 (socket says 172.21.15.33:47614) 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 -- 172.21.15.33:0/678417671 learned_addr learned my addr 172.21.15.33:0/678417671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9ffff640 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7feaa4a42640 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 -- 172.21.15.33:0/678417671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 msgr2=0x558d6b1dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 -- 172.21.15.33:0/678417671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 msgr2=0x558d6b1dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 -- 172.21.15.33:0/678417671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6ae16170 con 0x558d6b1e0450 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1f4a50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fea940059f0 tx=0x7fea94012340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32392b5fc5e4f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.326+0000 7fea85ffb640 1 -- 172.21.15.33:0/678417671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94000e60 con 0x558d6b1e0450 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7fea85ffb640 1 -- 172.21.15.33:0/678417671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea94005590 con 0x558d6b1e0450 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7fea85ffb640 1 -- 172.21.15.33:0/678417671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94010e60 con 0x558d6b1e0450 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 -- 172.21.15.33:0/678417671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 msgr2=0x558d6b1f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1f4a50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fea940059f0 tx=0x7fea94012340 comp rx=0 tx=0).stop 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 -- 172.21.15.33:0/678417671 shutdown_connections 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e90f0 0x558d6b1dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e0450 0x558d6b1f4a50 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 --2- 172.21.15.33:0/678417671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 -- 172.21.15.33:0/678417671 >> 172.21.15.33:0/678417671 conn(0x558d6b1a6950 msgr2=0x558d6b1e5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 -- 172.21.15.33:0/678417671 shutdown_connections 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.327+0000 7feaa91b8740 1 -- 172.21.15.33:0/678417671 wait complete. 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 -- start start 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 Processor -- start 2024-04-24T23:25:40.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e0450 0x558d6b1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 0x558d6b1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01a6890 con 0x558d6b1e90f0 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01a7d00 con 0x558d6b1e0450 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01a6f60 con 0x558d6b1df0e0 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa91b8740 10 client.0 _ll_register_callbacks cb 0x7fea94009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa4a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 0x558d6b1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa4a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 0x558d6b1f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47618/0 (socket says 172.21.15.33:47618) 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7feaa4a42640 1 -- 172.21.15.33:0/1314137777 learned_addr learned my addr 172.21.15.33:0/1314137777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42720/0 (socket says 172.21.15.33:42720) 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.328+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e0450 0x558d6b1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea9ffff640 1 -- 172.21.15.33:0/1314137777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e0450 msgr2=0x558d6b1f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea9ffff640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e0450 0x558d6b1f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea9ffff640 1 -- 172.21.15.33:0/1314137777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 msgr2=0x558d6b1f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea9ffff640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 0x558d6b1f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea9ffff640 1 -- 172.21.15.33:0/1314137777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6b1f4090 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7feaa4a42640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 0x558d6b1f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea9ffff640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1f2550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea900cd2c0 tx=0x7fea900cf9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=206d4d3e4e1fb27b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1314137777 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9009b7f0 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea94005590 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1314137777 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea90099d50 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1314137777 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea9009c3b0 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7feaa91b8740 10 client.27784 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d6b25fad0 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.329+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1314137777 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea94005590 con 0x558d6b1df0e0 2024-04-24T23:25:40.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1314137777 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fea90098320 con 0x558d6b1df0e0 2024-04-24T23:25:40.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7feaa91b8740 10 client.27784 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:40.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7feaa91b8740 10 client.27784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:40.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7feaa91b8740 20 client.27784 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:40.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b260100 0x558d6b2624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7feaa91b8740 4 client.27784 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:25:40.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558d6b14f800 con 0x558d6b260100 2024-04-24T23:25:40.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b260100 0x558d6b2624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7fea7f7fe640 10 client.27784 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.330+0000 7fea9f7fe640 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b260100 0x558d6b2624f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5265430da3412e server_cookie=264df3c07644a1bc in_seq=0 out_seq=0 2024-04-24T23:25:40.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7fea7f7fe640 1 -- 172.21.15.33:0/1314137777 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558d6b14f800 con 0x558d6b260100 2024-04-24T23:25:40.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7fea7f7fe640 10 client.27784 handle_command_reply: tid=0 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 client.27784 shutdown 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 msgr2=0x558d6b1f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1f2550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea900cd2c0 tx=0x7fea900cf9e0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 shutdown_connections 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x558d6b260100 0x558d6b2624f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d6b1e90f0 0x558d6b1f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d6b1e0450 0x558d6b1f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7fea7f7fe640 0 client.27784 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 --2- 172.21.15.33:0/1314137777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d6b1df0e0 0x558d6b1f2550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 >> 172.21.15.33:0/1314137777 conn(0x558d6b1a6950 msgr2=0x558d6b1d90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 shutdown_connections 2024-04-24T23:25:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.331+0000 7feaa91b8740 1 -- 172.21.15.33:0/1314137777 wait complete. 2024-04-24T23:25:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa91b8740 20 client.27784 trim_cache size 0 max 16384 2024-04-24T23:25:40.341 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:25:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:25:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "24a9e2b9-ad7d-4387-b8a1-eb2a7bb7b725", 2024-04-24T23:25:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:25:40.342 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:25:40.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fea880a2c10 msgr2=0x7fea880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fea880a2c10 0x7fea880a50d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fea98006010 tx=0x7fea980095c0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 msgr2=0x7feaa0196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 0x7feaa0196950 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fea9000e9c0 tx=0x7fea9000ee80 comp rx=0 tx=0).stop 2024-04-24T23:25:40.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 shutdown_connections 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fea880a2c10 0x7fea880a50d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaa0102230 0x7feaa019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaa0101870 0x7feaa0196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 --2- 172.21.15.33:0/4080726572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa0100680 0x7feaa0196950 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 >> 172.21.15.33:0/4080726572 conn(0x7feaa00fbbf0 msgr2=0x7feaa0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 shutdown_connections 2024-04-24T23:25:40.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.332+0000 7feaa64a5640 1 -- 172.21.15.33:0/4080726572 wait complete. 2024-04-24T23:25:40.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:25:40.345 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:25:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 -- 172.21.15.33:0/2667581721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801006a0 msgr2=0x7fad80100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 --2- 172.21.15.33:0/2667581721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801006a0 0x7fad80100ac0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009900 tx=0x7fad7c02a410 comp rx=0 tx=0).stop 2024-04-24T23:25:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 -- 172.21.15.33:0/2667581721 shutdown_connections 2024-04-24T23:25:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 --2- 172.21.15.33:0/2667581721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad80102250 0x7fad80108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 --2- 172.21.15.33:0/2667581721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80101890 0x7fad80101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 --2- 172.21.15.33:0/2667581721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801006a0 0x7fad80100ac0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 -- 172.21.15.33:0/2667581721 >> 172.21.15.33:0/2667581721 conn(0x7fad800fbc30 msgr2=0x7fad800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 -- 172.21.15.33:0/2667581721 shutdown_connections 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 -- 172.21.15.33:0/2667581721 wait complete. 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 Processor -- start 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.452+0000 7fad87d16640 1 -- start start 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 0x7fad8006a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80102250 0x7fad8019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8019cfc0 0x7fad8019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad8010c2f0 con 0x7fad80101890 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad8010c170 con 0x7fad8019cfc0 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad8010c470 con 0x7fad80102250 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 0x7fad8006a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 0x7fad8006a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47630/0 (socket says 172.21.15.33:47630) 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 -- 172.21.15.33:0/1049150957 learned_addr learned my addr 172.21.15.33:0/1049150957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad862b3640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8019cfc0 0x7fad8019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad852b1640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80102250 0x7fad8019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 -- 172.21.15.33:0/1049150957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80102250 msgr2=0x7fad8019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80102250 0x7fad8019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 -- 172.21.15.33:0/1049150957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8019cfc0 msgr2=0x7fad8019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8019cfc0 0x7fad8019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 -- 172.21.15.33:0/1049150957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8019fcf0 con 0x7fad80101890 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad85ab2640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 0x7fad8006a860 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009230 tx=0x7fad7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad76ffd640 1 -- 172.21.15.33:0/1049150957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c03f030 con 0x7fad80101890 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad76ffd640 1 -- 172.21.15.33:0/1049150957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c031a20 con 0x7fad80101890 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad76ffd640 1 -- 172.21.15.33:0/1049150957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c035070 con 0x7fad80101890 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8019ff80 con 0x7fad80101890 2024-04-24T23:25:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.453+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a0390 con 0x7fad80101890 2024-04-24T23:25:40.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 Processor -- start 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 -- start start 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad76ffd640 1 -- 172.21.15.33:0/1049150957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fad7c042030 con 0x7fad80101890 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4d8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6ab4306b0 con 0x55c6ab4d1040 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c6ab3d8880 con 0x55c6ab4d0650 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6ab42c670 con 0x55c6ab4d61b0 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad862b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad852b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4d8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4d5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47632/0 (socket says 172.21.15.33:47632) 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad862b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44076/0 (socket says 172.21.15.33:44076) 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.456+0000 7fad85ab2640 1 -- 172.21.15.33:0/2253574038 learned_addr learned my addr 172.21.15.33:0/2253574038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad852b1640 1 -- 172.21.15.33:0/2253574038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 msgr2=0x55c6ab4d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad852b1640 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad852b1640 1 -- 172.21.15.33:0/2253574038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 msgr2=0x55c6ab4d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad852b1640 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad852b1640 1 -- 172.21.15.33:0/2253574038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ab20e7d0 con 0x55c6ab4d61b0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad862b3640 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad76ffd640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad6c0a2b90 0x7fad6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad862b3640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad6c0a2b90 0x7fad6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad852b1640 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4d8640 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad8006ae20 tx=0x7fad70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8a07fcadfdd78b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad76ffd640 1 -- 172.21.15.33:0/1049150957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad7c0ef6d0 con 0x7fad80101890 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad6bfff640 1 -- 172.21.15.33:0/2253574038 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70004050 con 0x55c6ab4d61b0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad6bfff640 1 -- 172.21.15.33:0/2253574038 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70005500 con 0x55c6ab4d61b0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.457+0000 7fad6bfff640 1 -- 172.21.15.33:0/2253574038 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70004d90 con 0x55c6ab4d61b0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2253574038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 msgr2=0x55c6ab4d8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4d8640 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad8006ae20 tx=0x7fad70011810 comp rx=0 tx=0).stop 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad862b3640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad6c0a2b90 0x7fad6c0a5050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55c6ab4d18a0 tx=0x7fad78009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2253574038 shutdown_connections 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4d8640 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2253574038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2253574038 >> 172.21.15.33:0/2253574038 conn(0x55c6ab4c78b0 msgr2=0x55c6ab4c7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2253574038 shutdown_connections 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.458+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2253574038 wait complete. 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 -- start start 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 Processor -- start 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d0650 0x55c6ab4daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6ab4d9160 con 0x55c6ab4d1040 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c6ab3ebf50 con 0x55c6ab4d61b0 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6ab4d92e0 con 0x55c6ab4d0650 2024-04-24T23:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad8aa1c740 10 client.0 _ll_register_callbacks cb 0x55c6ab427000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad852b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad852b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4dafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47638/0 (socket says 172.21.15.33:47638) 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad852b1640 1 -- 172.21.15.33:0/3783208017 learned_addr learned my addr 172.21.15.33:0/3783208017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad862b3640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.459+0000 7fad85ab2640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d0650 0x55c6ab4daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad852b1640 1 -- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d0650 msgr2=0x55c6ab4daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad852b1640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d0650 0x55c6ab4daa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad852b1640 1 -- 172.21.15.33:0/3783208017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d61b0 msgr2=0x55c6ab4e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad852b1640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad852b1640 1 -- 172.21.15.33:0/3783208017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ab1443f0 con 0x55c6ab4d1040 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad862b3640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad852b1640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4dafb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fad70011d10 tx=0x7fad70009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cee2770bb079f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad697fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad697fa640 1 -- 172.21.15.33:0/3783208017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70029070 con 0x55c6ab4d1040 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad697fa640 1 -- 172.21.15.33:0/3783208017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad70004f20 con 0x55c6ab4d1040 2024-04-24T23:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad70005500 con 0x55c6ab4d1040 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad697fa640 1 -- 172.21.15.33:0/3783208017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad70035050 con 0x55c6ab4d1040 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad8aa1c740 10 client.18339 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6ab1040f0 con 0x55c6ab4d1040 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.460+0000 7fad697fa640 1 -- 172.21.15.33:0/3783208017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70005500 con 0x55c6ab4d1040 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad697fa640 1 -- 172.21.15.33:0/3783208017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad70003bf0 con 0x55c6ab4d1040 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad8aa1c740 10 client.18339 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad8aa1c740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad8aa1c740 20 client.18339 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab4e6730 0x55c6ab4e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad8aa1c740 4 client.18339 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6ab43dbf0 con 0x55c6ab4e6730 2024-04-24T23:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad85ab2640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab4e6730 0x55c6ab4e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad697fa640 10 client.18339 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.461+0000 7fad85ab2640 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab4e6730 0x55c6ab4e8b90 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f6ff962c30ec98 server_cookie=441d8f00d0b9e41c in_seq=0 out_seq=0 2024-04-24T23:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad697fa640 1 -- 172.21.15.33:0/3783208017 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6ab43dbf0 con 0x55c6ab4e6730 2024-04-24T23:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad697fa640 10 client.18339 handle_command_reply: tid=0 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 client.18339 shutdown 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 msgr2=0x55c6ab4dafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4dafb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fad70011d10 tx=0x7fad70009640 comp rx=0 tx=0).stop 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 shutdown_connections 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab4e6730 0x55c6ab4e8b90 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d1040 0x55c6ab4dafb0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad697fa640 0 client.18339 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/3783208017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d0650 0x55c6ab4daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 >> 172.21.15.33:0/3783208017 conn(0x55c6ab4c78b0 msgr2=0x55c6ab4c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 shutdown_connections 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 1 -- 172.21.15.33:0/3783208017 wait complete. 2024-04-24T23:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.462+0000 7fad8aa1c740 20 client.18339 trim_cache size 0 max 16384 2024-04-24T23:25:40.472 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1622586 12.29 MB/sec execute 585 sec latency 65.378 ms 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.482+0000 7fad8aa1c740 1 Processor -- start 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.482+0000 7fad8aa1c740 1 -- start start 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ab020 con 0x55c6ab4d0650 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a88f0 con 0x55c6ab4d1040 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a6740 con 0x55c6ab4d61b0 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47642/0 (socket says 172.21.15.33:47642) 2024-04-24T23:25:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 -- 172.21.15.33:0/2772209590 learned_addr learned my addr 172.21.15.33:0/2772209590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad852b1640 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad862b3640 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 -- 172.21.15.33:0/2772209590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 msgr2=0x55c6ab4c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 -- 172.21.15.33:0/2772209590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d1040 msgr2=0x55c6ab4e1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 -- 172.21.15.33:0/2772209590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ab4c8320 con 0x55c6ab4d0650 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad852b1640 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad85ab2640 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d96b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0314e0 tx=0x7fad7c0f5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8320237dd975ccd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad6b7fe640 1 -- 172.21.15.33:0/2772209590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0bb5b0 con 0x55c6ab4d0650 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.483+0000 7fad6b7fe640 1 -- 172.21.15.33:0/2772209590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c003e00 con 0x55c6ab4d0650 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad6b7fe640 1 -- 172.21.15.33:0/2772209590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0b9f50 con 0x55c6ab4d0650 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2772209590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 msgr2=0x55c6ab4d96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d96b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0314e0 tx=0x7fad7c0f5550 comp rx=0 tx=0).stop 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2772209590 shutdown_connections 2024-04-24T23:25:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2772209590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d0650 0x55c6ab4d96b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2772209590 >> 172.21.15.33:0/2772209590 conn(0x55c6ab4c78b0 msgr2=0x55c6ab4c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2772209590 shutdown_connections 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.484+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2772209590 wait complete. 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 -- start start 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 Processor -- start 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4dfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a6db0 con 0x55c6ab4d61b0 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a8220 con 0x55c6ab4d0650 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a7480 con 0x55c6ab4d1040 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad8aa1c740 10 client.0 _ll_register_callbacks cb 0x7fad7c0f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47646/0 (socket says 172.21.15.33:47646) 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 -- 172.21.15.33:0/2528221767 learned_addr learned my addr 172.21.15.33:0/2528221767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad85ab2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4dfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad852b1640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 -- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d1040 msgr2=0x55c6ab4e0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 -- 172.21.15.33:0/2528221767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 msgr2=0x55c6ab4dfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4dfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad862b3640 1 -- 172.21.15.33:0/2528221767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ab4e15c0 con 0x55c6ab4d61b0 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.486+0000 7fad85ab2640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4dfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad862b3640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e0fd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fad78005fd0 tx=0x7fad78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c85bf460fa0ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 1 -- 172.21.15.33:0/2528221767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780045c0 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 1 -- 172.21.15.33:0/2528221767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad78004760 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6ab4da860 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 1 -- 172.21.15.33:0/2528221767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad78034050 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 10 client.18345 fetch_fsmap learned FSMap version 27 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6ab54d0d0 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 1 -- 172.21.15.33:0/2528221767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c6ab4da860 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 1 -- 172.21.15.33:0/2528221767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad78005780 con 0x55c6ab4d61b0 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 10 client.18345 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 20 client.18345 populate_metadata read hostname 'smithi033' 2024-04-24T23:25:40.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab54db30 0x55c6ab54ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 4 client.18345 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6ab437f40 con 0x55c6ab54db30 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad85ab2640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab54db30 0x55c6ab54ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad85ab2640 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab54db30 0x55c6ab54ff50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d13a428f4b0db84 server_cookie=7765509adb8a6085 in_seq=0 out_seq=0 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 10 client.18345 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 1 -- 172.21.15.33:0/2528221767 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c6ab437f40 con 0x55c6ab54db30 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 10 client.18345 handle_command_reply: tid=0 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 client.18345 shutdown 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 msgr2=0x55c6ab4e0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e0fd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fad78005fd0 tx=0x7fad78004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 shutdown_connections 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c6ab54db30 0x55c6ab54ff50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6ab4d61b0 0x55c6ab4e0fd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6ab4d1040 0x55c6ab4e0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad697fa640 0 client.18345 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:25:40.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 --2- 172.21.15.33:0/2528221767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c6ab4d0650 0x55c6ab4dfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.487+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 >> 172.21.15.33:0/2528221767 conn(0x55c6ab4c78b0 msgr2=0x55c6ab4c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 shutdown_connections 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad8aa1c740 1 -- 172.21.15.33:0/2528221767 wait complete. 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad8aa1c740 20 client.18345 trim_cache size 0 max 16384 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: "24a9e2b9-ad7d-4387-b8a1-eb2a7bb7b725": { 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "24a9e2b9-ad7d-4387-b8a1-eb2a7bb7b725", 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:25:40.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:25:40.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:25:40.497 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:25:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad6c0a2b90 msgr2=0x7fad6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad6c0a2b90 0x7fad6c0a5050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55c6ab4d18a0 tx=0x7fad78009500 comp rx=0 tx=0).stop 2024-04-24T23:25:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 msgr2=0x7fad8006a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 0x7fad8006a860 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009230 tx=0x7fad7c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 shutdown_connections 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fad6c0a2b90 0x7fad6c0a5050 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8019cfc0 0x7fad8019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80102250 0x7fad8019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 --2- 172.21.15.33:0/1049150957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad80101890 0x7fad8006a860 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 >> 172.21.15.33:0/1049150957 conn(0x7fad800fbc30 msgr2=0x7fad80104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 shutdown_connections 2024-04-24T23:25:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:25:40.488+0000 7fad87d16640 1 -- 172.21.15.33:0/1049150957 wait complete. 2024-04-24T23:25:40.500 INFO:tasks.cephfs.filesystem:scrub status for tag:24a9e2b9-ad7d-4387-b8a1-eb2a7bb7b725 - {'path': '/', 'tag': '24a9e2b9-ad7d-4387-b8a1-eb2a7bb7b725', 'options': 'recursive,force'} 2024-04-24T23:25:41.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:41 smithi033 ceph-mon[29201]: pgmap v761: 129 pgs: 129 active+clean; 285 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 219 op/s 2024-04-24T23:25:41.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:41 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:25:41.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:41 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:41.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1624330 12.28 MB/sec execute 586 sec latency 16.067 ms 2024-04-24T23:25:41.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:41 smithi038 ceph-mon[32450]: pgmap v761: 129 pgs: 129 active+clean; 285 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 219 op/s 2024-04-24T23:25:41.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:41 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:25:41.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:41 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:41.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:41 smithi117 ceph-mon[32529]: pgmap v761: 129 pgs: 129 active+clean; 285 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 219 op/s 2024-04-24T23:25:41.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:41 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:25:41.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:41 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:42.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:42 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:25:42.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:42 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:42.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:42 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:25:42.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:42 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:42.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1627028 12.29 MB/sec execute 587 sec latency 89.243 ms 2024-04-24T23:25:42.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:42 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:25:42.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:42 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:42.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:42 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:25:42.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:42 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:42.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:42 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:25:42.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:42 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:42.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:42 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:25:42.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:42 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:25:43.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:43 smithi033 ceph-mon[29201]: pgmap v762: 129 pgs: 129 active+clean; 285 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-04-24T23:25:43.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1628552 12.28 MB/sec execute 588 sec latency 29.705 ms 2024-04-24T23:25:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:43 smithi038 ceph-mon[32450]: pgmap v762: 129 pgs: 129 active+clean; 285 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-04-24T23:25:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:43 smithi117 ceph-mon[32529]: pgmap v762: 129 pgs: 129 active+clean; 285 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 257 op/s 2024-04-24T23:25:44.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1631014 12.28 MB/sec execute 589 sec latency 9.757 ms 2024-04-24T23:25:45.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:45 smithi033 ceph-mon[29201]: pgmap v763: 129 pgs: 129 active+clean; 287 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-04-24T23:25:45.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1633373 12.28 MB/sec execute 590 sec latency 15.736 ms 2024-04-24T23:25:45.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:45 smithi038 ceph-mon[32450]: pgmap v763: 129 pgs: 129 active+clean; 287 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-04-24T23:25:45.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:45 smithi117 ceph-mon[32529]: pgmap v763: 129 pgs: 129 active+clean; 287 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-04-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1634981 12.28 MB/sec execute 591 sec latency 18.736 ms 2024-04-24T23:25:47.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:47 smithi033 ceph-mon[29201]: pgmap v764: 129 pgs: 129 active+clean; 290 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-04-24T23:25:47.469 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1637634 12.28 MB/sec execute 592 sec latency 50.451 ms 2024-04-24T23:25:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:47 smithi038 ceph-mon[32450]: pgmap v764: 129 pgs: 129 active+clean; 290 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-04-24T23:25:47.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:47 smithi117 ceph-mon[32529]: pgmap v764: 129 pgs: 129 active+clean; 290 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-04-24T23:25:48.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1639016 12.27 MB/sec execute 593 sec latency 23.998 ms 2024-04-24T23:25:48.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:48 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:25:48.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:48 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:25:48.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:48 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:25:48.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:48 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:25:48.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:48 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:25:48.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:48 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:25:49.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1641537 12.28 MB/sec execute 594 sec latency 21.525 ms 2024-04-24T23:25:49.576 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:49 smithi117 ceph-mon[32529]: pgmap v765: 129 pgs: 129 active+clean; 291 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-04-24T23:25:49.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:49 smithi038 ceph-mon[32450]: pgmap v765: 129 pgs: 129 active+clean; 291 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-04-24T23:25:49.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:49 smithi033 ceph-mon[29201]: pgmap v765: 129 pgs: 129 active+clean; 291 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-04-24T23:25:50.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1644065 12.28 MB/sec execute 595 sec latency 15.673 ms 2024-04-24T23:25:51.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1645777 12.27 MB/sec execute 596 sec latency 14.095 ms 2024-04-24T23:25:51.472 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:51 smithi038 ceph-mon[32450]: pgmap v766: 129 pgs: 129 active+clean; 289 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-24T23:25:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:51 smithi117 ceph-mon[32529]: pgmap v766: 129 pgs: 129 active+clean; 289 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-24T23:25:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:51 smithi033 ceph-mon[29201]: pgmap v766: 129 pgs: 129 active+clean; 289 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-24T23:25:52.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1648408 12.28 MB/sec execute 597 sec latency 69.802 ms 2024-04-24T23:25:53.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1650079 12.27 MB/sec execute 598 sec latency 105.244 ms 2024-04-24T23:25:53.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:53 smithi038 ceph-mon[32450]: pgmap v767: 129 pgs: 129 active+clean; 270 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2024-04-24T23:25:53.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:53 smithi117 ceph-mon[32529]: pgmap v767: 129 pgs: 129 active+clean; 270 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2024-04-24T23:25:53.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:53 smithi033 ceph-mon[29201]: pgmap v767: 129 pgs: 129 active+clean; 270 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2024-04-24T23:25:54.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1652449 12.27 MB/sec execute 599 sec latency 22.676 ms 2024-04-24T23:25:55.470 INFO:tasks.workunit.client.0.smithi033.stdout: 1 cleanup 600 sec 2024-04-24T23:25:55.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:55 smithi038 ceph-mon[32450]: pgmap v768: 129 pgs: 129 active+clean; 269 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2024-04-24T23:25:55.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:55 smithi117 ceph-mon[32529]: pgmap v768: 129 pgs: 129 active+clean; 269 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2024-04-24T23:25:55.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:55 smithi033 ceph-mon[29201]: pgmap v768: 129 pgs: 129 active+clean; 269 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: 0 cleanup 600 sec 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Operation Count AvgLat MaxLat 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: ---------------------------------------- 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: NTCreateX 234607 0.938 152.711 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Close 172331 0.003 0.088 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Rename 9933 4.312 142.789 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Unlink 47377 1.618 100.662 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Deltree 6 76.018 92.880 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Mkdir 3 0.015 0.016 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Qpathinfo 212645 0.711 176.219 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Qfileinfo 37268 0.001 0.045 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Qfsinfo 38991 0.178 14.306 2024-04-24T23:25:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: Sfileinfo 19112 0.008 0.812 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: Find 82217 0.078 85.228 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: WriteX 116967 0.015 27.231 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: ReadX 367768 0.026 50.442 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: LockX 764 0.845 16.138 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: UnlockX 764 0.737 7.328 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: Flush 16444 4.823 193.842 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stdout:Throughput 12.2698 MB/sec 1 clients 1 procs max_latency=193.851 ms 2024-04-24T23:25:55.776 INFO:tasks.workunit.client.0.smithi033.stderr:+ dbench 10 2024-04-24T23:25:55.777 INFO:tasks.workunit.client.0.smithi033.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-24T23:25:55.777 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-04-24T23:25:55.777 INFO:tasks.workunit.client.0.smithi033.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-24T23:25:55.777 INFO:tasks.workunit.client.0.smithi033.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-24T23:25:55.877 INFO:tasks.workunit.client.0.smithi033.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-24T23:25:55.877 INFO:tasks.workunit.client.0.smithi033.stdout:releasing clients 2024-04-24T23:25:56.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 465 152.60 MB/sec warmup 1 sec latency 75.946 ms 2024-04-24T23:25:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:57 smithi038 ceph-mon[32450]: pgmap v769: 129 pgs: 129 active+clean; 269 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 232 op/s 2024-04-24T23:25:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:57 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:25:57.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:57 smithi117 ceph-mon[32529]: pgmap v769: 129 pgs: 129 active+clean; 269 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 232 op/s 2024-04-24T23:25:57.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:57 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:25:57.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:57 smithi033 ceph-mon[29201]: pgmap v769: 129 pgs: 129 active+clean; 269 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 232 op/s 2024-04-24T23:25:57.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:57 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:25:57.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 966 111.96 MB/sec warmup 2 sec latency 382.082 ms 2024-04-24T23:25:58.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:58 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:25:58.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:58 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:25:58.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:58 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:25:58.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 1259 80.99 MB/sec warmup 3 sec latency 888.030 ms 2024-04-24T23:25:59.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:25:59 smithi038 ceph-mon[32450]: pgmap v770: 129 pgs: 129 active+clean; 272 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-04-24T23:25:59.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:25:59 smithi117 ceph-mon[32529]: pgmap v770: 129 pgs: 129 active+clean; 272 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-04-24T23:25:59.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:25:59 smithi033 ceph-mon[29201]: pgmap v770: 129 pgs: 129 active+clean; 272 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-04-24T23:25:59.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 1548 63.82 MB/sec warmup 4 sec latency 214.948 ms 2024-04-24T23:26:00.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 1867 54.74 MB/sec warmup 5 sec latency 94.467 ms 2024-04-24T23:26:01.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:01 smithi038 ceph-mon[32450]: pgmap v771: 129 pgs: 129 active+clean; 288 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-04-24T23:26:01.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:01 smithi117 ceph-mon[32529]: pgmap v771: 129 pgs: 129 active+clean; 288 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-04-24T23:26:01.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:01 smithi033 ceph-mon[29201]: pgmap v771: 129 pgs: 129 active+clean; 288 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-04-24T23:26:01.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 2649 50.19 MB/sec warmup 6 sec latency 85.187 ms 2024-04-24T23:26:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:02 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:26:02.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:02 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:26:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:02 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:26:02.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 3174 47.38 MB/sec warmup 7 sec latency 282.481 ms 2024-04-24T23:26:03.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:03 smithi033 ceph-mon[29201]: pgmap v772: 129 pgs: 129 active+clean; 330 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 419 op/s 2024-04-24T23:26:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:03 smithi038 ceph-mon[32450]: pgmap v772: 129 pgs: 129 active+clean; 330 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 419 op/s 2024-04-24T23:26:03.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:03 smithi117 ceph-mon[32529]: pgmap v772: 129 pgs: 129 active+clean; 330 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 419 op/s 2024-04-24T23:26:03.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 3644 46.48 MB/sec warmup 8 sec latency 352.050 ms 2024-04-24T23:26:04.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 4312 46.50 MB/sec warmup 9 sec latency 176.541 ms 2024-04-24T23:26:05.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:05 smithi038 ceph-mon[32450]: pgmap v773: 129 pgs: 129 active+clean; 347 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 375 op/s 2024-04-24T23:26:05.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:05 smithi117 ceph-mon[32529]: pgmap v773: 129 pgs: 129 active+clean; 347 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 375 op/s 2024-04-24T23:26:05.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:05 smithi033 ceph-mon[29201]: pgmap v773: 129 pgs: 129 active+clean; 347 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 375 op/s 2024-04-24T23:26:05.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 4755 44.26 MB/sec warmup 10 sec latency 80.639 ms 2024-04-24T23:26:06.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 5042 41.47 MB/sec warmup 11 sec latency 80.511 ms 2024-04-24T23:26:07.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:07 smithi038 ceph-mon[32450]: pgmap v774: 129 pgs: 129 active+clean; 430 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 503 op/s 2024-04-24T23:26:07.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:07 smithi117 ceph-mon[32529]: pgmap v774: 129 pgs: 129 active+clean; 430 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 503 op/s 2024-04-24T23:26:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:07 smithi033 ceph-mon[29201]: pgmap v774: 129 pgs: 129 active+clean; 430 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 503 op/s 2024-04-24T23:26:07.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 5219 38.21 MB/sec warmup 12 sec latency 231.648 ms 2024-04-24T23:26:08.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 5583 36.41 MB/sec warmup 13 sec latency 60.730 ms 2024-04-24T23:26:09.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:09 smithi038 ceph-mon[32450]: pgmap v775: 129 pgs: 129 active+clean; 465 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 533 op/s 2024-04-24T23:26:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:09 smithi117 ceph-mon[32529]: pgmap v775: 129 pgs: 129 active+clean; 465 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 533 op/s 2024-04-24T23:26:09.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:09 smithi033 ceph-mon[29201]: pgmap v775: 129 pgs: 129 active+clean; 465 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 533 op/s 2024-04-24T23:26:09.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 6217 35.98 MB/sec warmup 14 sec latency 71.542 ms 2024-04-24T23:26:10.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:26:10.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:26:10.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 -- 172.21.15.33:0/1574732089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374101880 msgr2=0x7f4374101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 --2- 172.21.15.33:0/1574732089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374101880 0x7f4374101d00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4368009900 tx=0x7f436802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 -- 172.21.15.33:0/1574732089 shutdown_connections 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 --2- 172.21.15.33:0/1574732089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4374102240 0x7f4374108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 --2- 172.21.15.33:0/1574732089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374101880 0x7f4374101d00 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 --2- 172.21.15.33:0/1574732089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 -- 172.21.15.33:0/1574732089 >> 172.21.15.33:0/1574732089 conn(0x7f43740fbc00 msgr2=0x7f43740fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 -- 172.21.15.33:0/1574732089 shutdown_connections 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 -- 172.21.15.33:0/1574732089 wait complete. 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.619+0000 7f4379cc1640 1 Processor -- start 2024-04-24T23:26:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 -- start start 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374104620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4374101880 0x7f4374102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 0x7f43741031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f437410c340 con 0x7f4374102240 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f437410c1c0 con 0x7f4374101880 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4379cc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f437410c4c0 con 0x7f4374100690 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f43737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 0x7f43741031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f43737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374104620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46410/0 (socket says 172.21.15.33:46410) 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f43737fe640 1 -- 172.21.15.33:0/4276613890 learned_addr learned my addr 172.21.15.33:0/4276613890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 0x7f43741031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57066/0 (socket says 172.21.15.33:57066) 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4372ffd640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4374101880 0x7f4374102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 -- 172.21.15.33:0/4276613890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 msgr2=0x7f4374104620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374104620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 -- 172.21.15.33:0/4276613890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4374101880 msgr2=0x7f4374102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4374101880 0x7f4374102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f4373fff640 1 -- 172.21.15.33:0/4276613890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4374103a30 con 0x7f4374102240 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.620+0000 7f43737fe640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.621+0000 7f4373fff640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 0x7f43741031a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f436c009fd0 tx=0x7f436c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.621+0000 7f4370ff9640 1 -- 172.21.15.33:0/4276613890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00ce20 con 0x7f4374102240 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.621+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741a1bd0 con 0x7f4374102240 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.621+0000 7f4370ff9640 1 -- 172.21.15.33:0/4276613890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c005020 con 0x7f4374102240 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.621+0000 7f4370ff9640 1 -- 172.21.15.33:0/4276613890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c010720 con 0x7f4374102240 2024-04-24T23:26:10.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.621+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741a20a0 con 0x7f4374102240 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.624+0000 7f437c9cd740 1 Processor -- start 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.624+0000 7f4370ff9640 1 -- 172.21.15.33:0/4276613890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f436c005540 con 0x7f4374102240 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.624+0000 7f437c9cd740 1 -- start start 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23ba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f437c9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbe231e6b0 con 0x55bbe23bac30 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f437c9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bbe22c6880 con 0x55bbe23ba240 2024-04-24T23:26:10.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f437c9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bbe231a670 con 0x55bbe23bfda0 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23bf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57078/0 (socket says 172.21.15.33:57078) 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 -- 172.21.15.33:0/3391907836 learned_addr learned my addr 172.21.15.33:0/3391907836 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f4372ffd640 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23ba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f4372ffd640 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 -- 172.21.15.33:0/3391907836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 msgr2=0x55bbe23c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 -- 172.21.15.33:0/3391907836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 msgr2=0x55bbe23ba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23ba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 -- 172.21.15.33:0/3391907836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe20fc7d0 con 0x55bbe23bac30 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f4370ff9640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f435c0a2b90 0x7f435c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f4372ffd640 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f4372ffd640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f435c0a2b90 0x7f435c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f4370ff9640 1 -- 172.21.15.33:0/4276613890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f436c0cfb70 con 0x7f4374102240 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.625+0000 7f43737fe640 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23bf730 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f43640115c0 tx=0x7f4364011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51c0d6dd4fe44d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f4359ffb640 1 -- 172.21.15.33:0/3391907836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364005020 con 0x55bbe23bac30 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f4359ffb640 1 -- 172.21.15.33:0/3391907836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43640047a0 con 0x55bbe23bac30 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f4359ffb640 1 -- 172.21.15.33:0/3391907836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436400fa00 con 0x55bbe23bac30 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f437c9cd740 1 -- 172.21.15.33:0/3391907836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 msgr2=0x55bbe23bf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f4372ffd640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f435c0a2b90 0x7f435c0a5050 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4368031440 tx=0x7f4368041040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f437c9cd740 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23bf730 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f43640115c0 tx=0x7f4364011a80 comp rx=0 tx=0).stop 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f437c9cd740 1 -- 172.21.15.33:0/3391907836 shutdown_connections 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f437c9cd740 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f437c9cd740 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23bf730 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.626+0000 7f437c9cd740 1 --2- 172.21.15.33:0/3391907836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23ba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 -- 172.21.15.33:0/3391907836 >> 172.21.15.33:0/3391907836 conn(0x55bbe23b58b0 msgr2=0x55bbe23b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 -- 172.21.15.33:0/3391907836 shutdown_connections 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 -- 172.21.15.33:0/3391907836 wait complete. 2024-04-24T23:26:10.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 -- start start 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 Processor -- start 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 0x55bbe23c4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bac30 0x55bbe23c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbe23c2d50 con 0x55bbe23ba240 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.627+0000 7f437c9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bbe22d9f50 con 0x55bbe23bac30 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f437c9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bbe23c2ed0 con 0x55bbe23bfda0 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 0x55bbe23c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f437c9cd740 10 client.0 _ll_register_callbacks cb 0x55bbe2315000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 0x55bbe23c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57080/0 (socket says 172.21.15.33:57080) 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 -- 172.21.15.33:0/1643742567 learned_addr learned my addr 172.21.15.33:0/1643742567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f4372ffd640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bac30 0x55bbe23c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f4373fff640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 -- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 msgr2=0x55bbe23cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 -- 172.21.15.33:0/1643742567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bac30 msgr2=0x55bbe23c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bac30 0x55bbe23c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 -- 172.21.15.33:0/1643742567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe20323f0 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f43737fe640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 0x55bbe23c4650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f436401b040 tx=0x7f43640052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2335b39ca86fa827 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f4352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.628+0000 7f4352ffd640 1 -- 172.21.15.33:0/1643742567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436401d050 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f4352ffd640 1 -- 172.21.15.33:0/1643742567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43640054a0 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43640047a0 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f4352ffd640 1 -- 172.21.15.33:0/1643742567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4364032050 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 10 client.18357 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbe1ff20f0 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f4352ffd640 1 -- 172.21.15.33:0/1643742567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43640047a0 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f4352ffd640 1 -- 172.21.15.33:0/1643742567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f43640035f0 con 0x55bbe23ba240 2024-04-24T23:26:10.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 10 client.18357 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 20 client.18357 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe23d02e0 0x55bbe23d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 4 client.18357 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe232bbf0 con 0x55bbe23d02e0 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.629+0000 7f4372ffd640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe23d02e0 0x55bbe23d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.630+0000 7f4372ffd640 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe23d02e0 0x55bbe23d2780 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3488e0193a415f4 server_cookie=91d8f807afa5e9d0 in_seq=0 out_seq=0 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.630+0000 7f4352ffd640 10 client.18357 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.631+0000 7f4352ffd640 1 -- 172.21.15.33:0/1643742567 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbe232bbf0 con 0x55bbe23d02e0 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.631+0000 7f4352ffd640 10 client.18357 handle_command_reply: tid=0 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.631+0000 7f437c9cd740 1 client.18357 shutdown 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.631+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 msgr2=0x55bbe23c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.631+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 0x55bbe23c4650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f436401b040 tx=0x7f43640052b0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 shutdown_connections 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe23d02e0 0x55bbe23d2780 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bfda0 0x55bbe23cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bac30 0x55bbe23c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1643742567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23ba240 0x55bbe23c4650 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f4352ffd640 0 client.18357 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 >> 172.21.15.33:0/1643742567 conn(0x55bbe23b58b0 msgr2=0x55bbe23b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 shutdown_connections 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 1 -- 172.21.15.33:0/1643742567 wait complete. 2024-04-24T23:26:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.632+0000 7f437c9cd740 20 client.18357 trim_cache size 0 max 16384 2024-04-24T23:26:10.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.657+0000 7f437c9cd740 1 Processor -- start 2024-04-24T23:26:10.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 -- start start 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bac30 0x55bbe23cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bfda0 0x55bbe23c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43741aef40 con 0x55bbe23bfda0 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43741ac7b0 con 0x55bbe23ba240 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f437c9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43741aa600 con 0x55bbe23bac30 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f43737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f4373fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bfda0 0x55bbe23c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f43737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35956/0 (socket says 172.21.15.33:35956) 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bac30 0x55bbe23cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.658+0000 7f43737fe640 1 -- 172.21.15.33:0/928819637 learned_addr learned my addr 172.21.15.33:0/928819637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:10.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43737fe640 1 -- 172.21.15.33:0/928819637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bac30 msgr2=0x55bbe23cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43737fe640 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bac30 0x55bbe23cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43737fe640 1 -- 172.21.15.33:0/928819637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bfda0 msgr2=0x55bbe23c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43737fe640 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bfda0 0x55bbe23c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43737fe640 1 -- 172.21.15.33:0/928819637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe23b53f0 con 0x55bbe23ba240 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f4373fff640 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bfda0 0x55bbe23c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43737fe640 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23c32d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43640039e0 tx=0x7f4364005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744f1406161c9b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43597fa640 1 -- 172.21.15.33:0/928819637 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436403b070 con 0x55bbe23ba240 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43597fa640 1 -- 172.21.15.33:0/928819637 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364029070 con 0x55bbe23ba240 2024-04-24T23:26:10.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.659+0000 7f43597fa640 1 -- 172.21.15.33:0/928819637 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436401d040 con 0x55bbe23ba240 2024-04-24T23:26:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 -- 172.21.15.33:0/928819637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 msgr2=0x55bbe23c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23c32d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43640039e0 tx=0x7f4364005970 comp rx=0 tx=0).stop 2024-04-24T23:26:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 -- 172.21.15.33:0/928819637 shutdown_connections 2024-04-24T23:26:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bfda0 0x55bbe23c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23bac30 0x55bbe23cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 --2- 172.21.15.33:0/928819637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23ba240 0x55bbe23c32d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 -- 172.21.15.33:0/928819637 >> 172.21.15.33:0/928819637 conn(0x55bbe23b58b0 msgr2=0x55bbe23b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 -- 172.21.15.33:0/928819637 shutdown_connections 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 -- 172.21.15.33:0/928819637 wait complete. 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.660+0000 7f437c9cd740 1 -- start start 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 Processor -- start 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23ba240 0x55bbe23c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bfda0 0x55bbe23ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43741aac70 con 0x55bbe23bac30 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43741ac0e0 con 0x55bbe23bfda0 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43741ab340 con 0x55bbe23ba240 2024-04-24T23:26:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f437c9cd740 10 client.0 _ll_register_callbacks cb 0x7f4364005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4373fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bfda0 0x55bbe23ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57096/0 (socket says 172.21.15.33:57096) 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f43737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23ba240 0x55bbe23c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 -- 172.21.15.33:0/1213861196 learned_addr learned my addr 172.21.15.33:0/1213861196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 -- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23ba240 msgr2=0x55bbe23c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23ba240 0x55bbe23c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 -- 172.21.15.33:0/1213861196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bfda0 msgr2=0x55bbe23ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bfda0 0x55bbe23ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4372ffd640 1 -- 172.21.15.33:0/1213861196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe23b5200 con 0x55bbe23bac30 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f4373fff640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bfda0 0x55bbe23ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.661+0000 7f43737fe640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23ba240 0x55bbe23c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f4372ffd640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23c9550 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4368006010 tx=0x7f436802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283084e1f206bbda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f4352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f4352ffd640 1 -- 172.21.15.33:0/1213861196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368046070 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f4352ffd640 1 -- 172.21.15.33:0/1213861196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43680563a0 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4364029070 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f4352ffd640 1 -- 172.21.15.33:0/1213861196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4368066050 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f437c9cd740 10 client.18363 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbe23b6620 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.662+0000 7f4352ffd640 1 -- 172.21.15.33:0/1213861196 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364029070 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f4352ffd640 1 -- 172.21.15.33:0/1213861196 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f4368003d20 con 0x55bbe23bac30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f437c9cd740 10 client.18363 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f437c9cd740 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f437c9cd740 20 client.18363 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe2436d90 0x55bbe24391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f437c9cd740 4 client.18363 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:10.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbe2325f40 con 0x55bbe2436d90 2024-04-24T23:26:10.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f43737fe640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe2436d90 0x55bbe24391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f4352ffd640 10 client.18363 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:10.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.663+0000 7f43737fe640 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe2436d90 0x55bbe24391b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b9ecef37ff1cbf server_cookie=45a2a87f5d3effbc in_seq=0 out_seq=0 2024-04-24T23:26:10.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:10 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:10.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.666+0000 7f4352ffd640 1 -- 172.21.15.33:0/1213861196 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbe2325f40 con 0x55bbe2436d90 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.666+0000 7f4352ffd640 10 client.18363 handle_command_reply: tid=0 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.666+0000 7f437c9cd740 1 client.18363 shutdown 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 msgr2=0x55bbe23c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23c9550 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4368006010 tx=0x7f436802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 shutdown_connections 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bbe2436d90 0x55bbe24391b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bbe23bfda0 0x55bbe23ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbe23bac30 0x55bbe23c9550 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 --2- 172.21.15.33:0/1213861196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bbe23ba240 0x55bbe23c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 >> 172.21.15.33:0/1213861196 conn(0x55bbe23b58b0 msgr2=0x55bbe23b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f4352ffd640 0 client.18363 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 shutdown_connections 2024-04-24T23:26:10.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 1 -- 172.21.15.33:0/1213861196 wait complete. 2024-04-24T23:26:10.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.667+0000 7f437c9cd740 20 client.18363 trim_cache size 0 max 16384 2024-04-24T23:26:10.673 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:26:10.673 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:26:10.673 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:26:10.673 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:26:10.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f435c0a2b90 msgr2=0x7f435c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f435c0a2b90 0x7f435c0a5050 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4368031440 tx=0x7f4368041040 comp rx=0 tx=0).stop 2024-04-24T23:26:10.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 msgr2=0x7f43741031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 0x7f43741031a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f436c009fd0 tx=0x7f436c00ef10 comp rx=0 tx=0).stop 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 shutdown_connections 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f435c0a2b90 0x7f435c0a5050 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4374102240 0x7f43741031a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4374101880 0x7f4374102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 --2- 172.21.15.33:0/4276613890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4374100690 0x7f4374104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 >> 172.21.15.33:0/4276613890 conn(0x7f43740fbc00 msgr2=0x7f43740fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 shutdown_connections 2024-04-24T23:26:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.668+0000 7f4379cc1640 1 -- 172.21.15.33:0/4276613890 wait complete. 2024-04-24T23:26:10.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:10 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:10.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:10 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:10.684 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:10.684 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:10.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.802+0000 7f322fc38640 1 -- 172.21.15.33:0/3495625098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 msgr2=0x7f3228100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.802+0000 7f322fc38640 1 --2- 172.21.15.33:0/3495625098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228100ac0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3224009900 tx=0x7f322402a410 comp rx=0 tx=0).stop 2024-04-24T23:26:10.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.803+0000 7f322fc38640 1 -- 172.21.15.33:0/3495625098 shutdown_connections 2024-04-24T23:26:10.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.803+0000 7f322fc38640 1 --2- 172.21.15.33:0/3495625098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 0x7f3228108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.803+0000 7f322fc38640 1 --2- 172.21.15.33:0/3495625098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228101890 0x7f3228101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.803+0000 7f322fc38640 1 --2- 172.21.15.33:0/3495625098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228100ac0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.803+0000 7f322fc38640 1 -- 172.21.15.33:0/3495625098 >> 172.21.15.33:0/3495625098 conn(0x7f32280fbc30 msgr2=0x7f32280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.803+0000 7f322fc38640 1 -- 172.21.15.33:0/3495625098 shutdown_connections 2024-04-24T23:26:10.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.804+0000 7f322fc38640 1 -- 172.21.15.33:0/3495625098 wait complete. 2024-04-24T23:26:10.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 Processor -- start 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 -- start start 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228101890 0x7f322819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 0x7f32281a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f322810b220 con 0x7f32281006a0 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f322810b0a0 con 0x7f3228102250 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.805+0000 7f322fc38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f322810b3a0 con 0x7f3228101890 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57118/0 (socket says 172.21.15.33:57118) 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 -- 172.21.15.33:0/51543058 learned_addr learned my addr 172.21.15.33:0/51543058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d1d3640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228101890 0x7f322819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 -- 172.21.15.33:0/51543058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228101890 msgr2=0x7f322819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228101890 0x7f322819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 -- 172.21.15.33:0/51543058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 msgr2=0x7f32281a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322e1d5640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 0x7f32281a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 0x7f32281a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 -- 172.21.15.33:0/51543058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32281a19b0 con 0x7f32281006a0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322e1d5640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 0x7f32281a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f322d9d4640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228199df0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3224031610 tx=0x7f3224004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.806+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322403f030 con 0x7f32281006a0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.807+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32281a1c40 con 0x7f32281006a0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.807+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3224003ad0 con 0x7f32281006a0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.807+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224035070 con 0x7f32281006a0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.807+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281a2180 con 0x7f32281006a0 2024-04-24T23:26:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.808+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3228193710 con 0x7f32281006a0 2024-04-24T23:26:10.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.809+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3224042030 con 0x7f32281006a0 2024-04-24T23:26:10.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.810+0000 7f321effd640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32140a2b70 0x7f32140a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.810+0000 7f322d1d3640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32140a2b70 0x7f32140a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.810+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f32240ef9e0 con 0x7f32281006a0 2024-04-24T23:26:10.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.810+0000 7f322d1d3640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32140a2b70 0x7f32140a5030 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3218009bd0 tx=0x7f3218005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:10.815+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32240bc5b0 con 0x7f32281006a0 2024-04-24T23:26:10.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 7127 36.15 MB/sec warmup 15 sec latency 54.269 ms 2024-04-24T23:26:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.070+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3228062340 con 0x7f32281006a0 2024-04-24T23:26:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.070+0000 7f321effd640 1 -- 172.21.15.33:0/51543058 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11863 (secure 0 0 0) 0x7f32240bbeb0 con 0x7f32281006a0 2024-04-24T23:26:11.073 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:26:11.073 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:26:01.133927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:26:11.074 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.072+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32140a2b70 msgr2=0x7f32140a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32140a2b70 0x7f32140a5030 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3218009bd0 tx=0x7f3218005f90 comp rx=0 tx=0).stop 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 msgr2=0x7f3228199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228199df0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3224031610 tx=0x7f3224004270 comp rx=0 tx=0).stop 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 shutdown_connections 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32140a2b70 0x7f32140a5030 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3228102250 0x7f32281a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228101890 0x7f322819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 --2- 172.21.15.33:0/51543058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32281006a0 0x7f3228199df0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 >> 172.21.15.33:0/51543058 conn(0x7f32280fbc30 msgr2=0x7f32281908e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 shutdown_connections 2024-04-24T23:26:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.073+0000 7f322fc38640 1 -- 172.21.15.33:0/51543058 wait complete. 2024-04-24T23:26:11.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:26:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 -- 172.21.15.33:0/4275772842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10de30 msgr2=0x7ffa6c1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 --2- 172.21.15.33:0/4275772842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1102f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007380 tx=0x7ffa5c02afd0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 -- 172.21.15.33:0/4275772842 shutdown_connections 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 --2- 172.21.15.33:0/4275772842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1102f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 --2- 172.21.15.33:0/4275772842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c10b370 0x7ffa6c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 --2- 172.21.15.33:0/4275772842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c1089b0 0x7ffa6c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 -- 172.21.15.33:0/4275772842 >> 172.21.15.33:0/4275772842 conn(0x7ffa6c06e970 msgr2=0x7ffa6c070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 -- 172.21.15.33:0/4275772842 shutdown_connections 2024-04-24T23:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.210+0000 7ffa718cd640 1 -- 172.21.15.33:0/4275772842 wait complete. 2024-04-24T23:26:11.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.211+0000 7ffa718cd640 1 Processor -- start 2024-04-24T23:26:11.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.211+0000 7ffa718cd640 1 -- start start 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.211+0000 7ffa718cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c1089b0 0x7ffa6c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa718cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10b370 0x7ffa6c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa718cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa718cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c113a50 con 0x7ffa6c10de30 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa718cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1138d0 con 0x7ffa6c10b370 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa718cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c113bd0 con 0x7ffa6c1089b0 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1a7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57126/0 (socket says 172.21.15.33:57126) 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.212+0000 7ffa6b7fe640 1 -- 172.21.15.33:0/2091391178 learned_addr learned my addr 172.21.15.33:0/2091391178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.213+0000 7ffa6b7fe640 1 -- 172.21.15.33:0/2091391178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c1089b0 msgr2=0x7ffa6c1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.213+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c1089b0 0x7ffa6c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.213+0000 7ffa6b7fe640 1 -- 172.21.15.33:0/2091391178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10b370 msgr2=0x7ffa6c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.213+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10b370 0x7ffa6c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.213+0000 7ffa6b7fe640 1 -- 172.21.15.33:0/2091391178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c1a7ec0 con 0x7ffa6c10de30 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.213+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1a7850 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00d090 tx=0x7ffa5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.214+0000 7ffa708cb640 1 -- 172.21.15.33:0/2091391178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c03f070 con 0x7ffa6c10de30 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.214+0000 7ffa708cb640 1 -- 172.21.15.33:0/2091391178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa5c033910 con 0x7ffa6c10de30 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.214+0000 7ffa708cb640 1 -- 172.21.15.33:0/2091391178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0035f0 con 0x7ffa6c10de30 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.214+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c108d90 con 0x7ffa6c10de30 2024-04-24T23:26:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.214+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c109310 con 0x7ffa6c10de30 2024-04-24T23:26:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.218+0000 7ffa745dc740 1 Processor -- start 2024-04-24T23:26:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.218+0000 7ffa708cb640 1 -- 172.21.15.33:0/2091391178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffa5c033ac0 con 0x7ffa6c10de30 2024-04-24T23:26:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa708cb640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa540a2c90 0x7ffa540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 -- start start 2024-04-24T23:26:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa708cb640 1 -- 172.21.15.33:0/2091391178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa5c0f2f00 con 0x7ffa6c10de30 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d20a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a6430 0x5636d20aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636d2001670 con 0x5636d20a6430 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5636d1fa9970 con 0x5636d20a5a40 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa745dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5636d1ffd630 con 0x5636d20ab5a0 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a6430 0x5636d20aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d20a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d20a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36014/0 (socket says 172.21.15.33:36014) 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa6affd640 1 -- 172.21.15.33:0/1749076322 learned_addr learned my addr 172.21.15.33:0/1749076322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.219+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa540a2c90 0x7ffa540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6affd640 1 -- 172.21.15.33:0/1749076322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 msgr2=0x5636d20ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6affd640 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6affd640 1 -- 172.21.15.33:0/1749076322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a6430 msgr2=0x5636d20aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6affd640 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a6430 0x5636d20aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6affd640 1 -- 172.21.15.33:0/1749076322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d1fcd400 con 0x5636d20a5a40 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.220+0000 7ffa6affd640 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d20a5e60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400e470 tx=0x7ffa6400e870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54f30500ea04201 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.221+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa540a2c90 0x7ffa540a5150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5636d20a6830 tx=0x7ffa60006960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.221+0000 7ffa51ffb640 1 -- 172.21.15.33:0/1749076322 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004b20 con 0x5636d20a5a40 2024-04-24T23:26:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.222+0000 7ffa51ffb640 1 -- 172.21.15.33:0/1749076322 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64003e50 con 0x5636d20a5a40 2024-04-24T23:26:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.222+0000 7ffa51ffb640 1 -- 172.21.15.33:0/1749076322 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400c640 con 0x5636d20a5a40 2024-04-24T23:26:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.223+0000 7ffa745dc740 1 -- 172.21.15.33:0/1749076322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 msgr2=0x5636d20a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.223+0000 7ffa745dc740 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d20a5e60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400e470 tx=0x7ffa6400e870 comp rx=0 tx=0).stop 2024-04-24T23:26:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.224+0000 7ffa745dc740 1 -- 172.21.15.33:0/1749076322 shutdown_connections 2024-04-24T23:26:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.224+0000 7ffa745dc740 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.224+0000 7ffa745dc740 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a6430 0x5636d20aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.224+0000 7ffa745dc740 1 --2- 172.21.15.33:0/1749076322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d20a5e60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.224+0000 7ffa745dc740 1 -- 172.21.15.33:0/1749076322 >> 172.21.15.33:0/1749076322 conn(0x5636d20988a0 msgr2=0x5636d2098ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.225+0000 7ffa745dc740 1 -- 172.21.15.33:0/1749076322 shutdown_connections 2024-04-24T23:26:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.225+0000 7ffa745dc740 1 -- 172.21.15.33:0/1749076322 wait complete. 2024-04-24T23:26:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 -- start start 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 Processor -- start 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d209ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 0x5636d209ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20ab5a0 0x5636d20a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636d209d380 con 0x5636d20ab5a0 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5636d1fbd050 con 0x5636d20a5a40 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5636d209d500 con 0x5636d20a6430 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.226+0000 7ffa745dc740 10 client.0 _ll_register_callbacks cb 0x5636d1d14be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 0x5636d209ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 0x5636d209ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46450/0 (socket says 172.21.15.33:46450) 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2732588039 learned_addr learned my addr 172.21.15.33:0/2732588039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20ab5a0 0x5636d20a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 msgr2=0x5636d209ea40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d209ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2732588039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20ab5a0 msgr2=0x5636d20a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20ab5a0 0x5636d20a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.227+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2732588039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d1f8edc0 con 0x5636d20a6430 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.228+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 0x5636d209ef80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009090 tx=0x7ffa600068f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53f42cda37c3530 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.228+0000 7ffa4affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.228+0000 7ffa4affd640 1 -- 172.21.15.33:0/2732588039 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60042070 con 0x5636d20a6430 2024-04-24T23:26:11.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.228+0000 7ffa4affd640 1 -- 172.21.15.33:0/2732588039 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa6000b9e0 con 0x5636d20a6430 2024-04-24T23:26:11.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.229+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa64003e50 con 0x5636d20a6430 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.231+0000 7ffa4affd640 1 -- 172.21.15.33:0/2732588039 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa60014d60 con 0x5636d20a6430 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.231+0000 7ffa745dc740 10 client.27832 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.231+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636d1d153f0 con 0x5636d20a6430 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa4affd640 1 -- 172.21.15.33:0/2732588039 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64003e50 con 0x5636d20a6430 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa4affd640 1 -- 172.21.15.33:0/2732588039 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ffa600193f0 con 0x5636d20a6430 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa745dc740 10 client.27832 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa745dc740 10 client.27832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa745dc740 20 client.27832 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d20c4370 0x5636d20c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa745dc740 4 client.27832 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:11.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636d200ebb0 con 0x5636d20c4370 2024-04-24T23:26:11.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.232+0000 7ffa6affd640 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d20c4370 0x5636d20c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.233+0000 7ffa4affd640 10 client.27832 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:11.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.233+0000 7ffa6affd640 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d20c4370 0x5636d20c67e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=367299f461e95953 server_cookie=28634f5ffe53be7a in_seq=0 out_seq=0 2024-04-24T23:26:11.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.236+0000 7ffa4affd640 1 -- 172.21.15.33:0/2732588039 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636d200ebb0 con 0x5636d20c4370 2024-04-24T23:26:11.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.236+0000 7ffa4affd640 10 client.27832 handle_command_reply: tid=0 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.236+0000 7ffa745dc740 1 client.27832 shutdown 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.236+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 msgr2=0x5636d209ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.236+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 0x5636d209ef80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009090 tx=0x7ffa600068f0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.237+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 shutdown_connections 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.237+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d20c4370 0x5636d20c67e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.237+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20ab5a0 0x5636d20a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.237+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20a6430 0x5636d209ef80 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009090 tx=0x7ffa600068f0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.237+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2732588039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20a5a40 0x5636d209ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.237+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 >> 172.21.15.33:0/2732588039 conn(0x5636d20988a0 msgr2=0x5636d2097b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.238+0000 7ffa4affd640 0 client.27832 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:11.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.238+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 shutdown_connections 2024-04-24T23:26:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.239+0000 7ffa745dc740 1 -- 172.21.15.33:0/2732588039 wait complete. 2024-04-24T23:26:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.239+0000 7ffa745dc740 20 client.27832 trim_cache size 0 max 16384 2024-04-24T23:26:11.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 Processor -- start 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 -- start start 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d209b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20ab5a0 0x5636d209bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d209c230 0x5636d20a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c07ecb0 con 0x5636d20a5a40 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c07c520 con 0x5636d20ab5a0 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.260+0000 7ffa745dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c07a370 con 0x5636d209c230 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d209b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d209b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57138/0 (socket says 172.21.15.33:57138) 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 -- 172.21.15.33:0/217447943 learned_addr learned my addr 172.21.15.33:0/217447943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d209c230 0x5636d20a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20ab5a0 0x5636d209bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 -- 172.21.15.33:0/217447943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d209c230 msgr2=0x5636d20a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d209c230 0x5636d20a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 -- 172.21.15.33:0/217447943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20ab5a0 msgr2=0x5636d209bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20ab5a0 0x5636d209bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 -- 172.21.15.33:0/217447943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d1ddf7d0 con 0x5636d20a5a40 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa6affd640 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d209b7b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ffa64000f80 tx=0x7ffa64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25052e405a55597 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa517fa640 1 -- 172.21.15.33:0/217447943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6402c070 con 0x5636d20a5a40 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa517fa640 1 -- 172.21.15.33:0/217447943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64003e50 con 0x5636d20a5a40 2024-04-24T23:26:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.261+0000 7ffa517fa640 1 -- 172.21.15.33:0/217447943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64026e60 con 0x5636d20a5a40 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 -- 172.21.15.33:0/217447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 msgr2=0x5636d209b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d209b7b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ffa64000f80 tx=0x7ffa64004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 -- 172.21.15.33:0/217447943 shutdown_connections 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d209c230 0x5636d20a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d20ab5a0 0x5636d209bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 --2- 172.21.15.33:0/217447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d209b7b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 -- 172.21.15.33:0/217447943 >> 172.21.15.33:0/217447943 conn(0x5636d20988a0 msgr2=0x5636d209ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 -- 172.21.15.33:0/217447943 shutdown_connections 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 -- 172.21.15.33:0/217447943 wait complete. 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.262+0000 7ffa745dc740 1 -- start start 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 Processor -- start 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d209c230 0x5636d20d56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d20d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c07a9e0 con 0x5636d20a5a40 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d20d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d20d5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57154/0 (socket says 172.21.15.33:57154) 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2739237585 learned_addr learned my addr 172.21.15.33:0/2739237585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6affd640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d209c230 0x5636d20d56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c07be50 con 0x5636d209c230 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c07b0b0 con 0x5636d20ab5a0 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6b7fe640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa745dc740 10 client.0 _ll_register_callbacks cb 0x7ffa60004e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2739237585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 msgr2=0x5636d20d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20d6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d209c230 msgr2=0x5636d20d56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d209c230 0x5636d20d56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.263+0000 7ffa6a7fc640 1 -- 172.21.15.33:0/2739237585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d20d71f0 con 0x5636d20a5a40 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa6affd640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d209c230 0x5636d20d56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa6a7fc640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d20d5bf0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ffa600198b0 tx=0x7ffa6000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e649a50140bcf3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa4affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa4affd640 1 -- 172.21.15.33:0/2739237585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60054040 con 0x5636d20a5a40 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa4affd640 1 -- 172.21.15.33:0/2739237585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa6000d710 con 0x5636d20a5a40 2024-04-24T23:26:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa64003e50 con 0x5636d20a5a40 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa4affd640 1 -- 172.21.15.33:0/2739237585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa600193f0 con 0x5636d20a5a40 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa745dc740 10 client.18387 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636d212a770 con 0x5636d20a5a40 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa4affd640 1 -- 172.21.15.33:0/2739237585 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64003e50 con 0x5636d20a5a40 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.264+0000 7ffa4affd640 1 -- 172.21.15.33:0/2739237585 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ffa60046400 con 0x5636d20a5a40 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa745dc740 10 client.18387 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa745dc740 10 client.18387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa745dc740 20 client.18387 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d212b1c0 0x5636d212d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa745dc740 4 client.18387 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636d2008f00 con 0x5636d212b1c0 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa6affd640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d212b1c0 0x5636d212d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa4affd640 10 client.18387 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.265+0000 7ffa6affd640 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d212b1c0 0x5636d212d5e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea54988e0688858c server_cookie=be9e307e0c8d90c9 in_seq=0 out_seq=0 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa4affd640 1 -- 172.21.15.33:0/2739237585 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636d2008f00 con 0x5636d212b1c0 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa4affd640 10 client.18387 handle_command_reply: tid=0 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 client.18387 shutdown 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 msgr2=0x5636d20d5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d20d5bf0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ffa600198b0 tx=0x7ffa6000b760 comp rx=0 tx=0).stop 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 shutdown_connections 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5636d212b1c0 0x5636d212d5e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636d20ab5a0 0x5636d20d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636d20a5a40 0x5636d20d5bf0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 --2- 172.21.15.33:0/2739237585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636d209c230 0x5636d20d56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 >> 172.21.15.33:0/2739237585 conn(0x5636d20988a0 msgr2=0x5636d209ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa4affd640 0 client.18387 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 shutdown_connections 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 1 -- 172.21.15.33:0/2739237585 wait complete. 2024-04-24T23:26:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.267+0000 7ffa745dc740 20 client.18387 trim_cache size 0 max 16384 2024-04-24T23:26:11.274 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa540a2c90 msgr2=0x7ffa540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa540a2c90 0x7ffa540a5150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5636d20a6830 tx=0x7ffa60006960 comp rx=0 tx=0).stop 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 msgr2=0x7ffa6c1a7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1a7850 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00d090 tx=0x7ffa5c02b910 comp rx=0 tx=0).stop 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 shutdown_connections 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa540a2c90 0x7ffa540a5150 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c10de30 0x7ffa6c1a7850 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa6c10b370 0x7ffa6c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 --2- 172.21.15.33:0/2091391178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c1089b0 0x7ffa6c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 >> 172.21.15.33:0/2091391178 conn(0x7ffa6c06e970 msgr2=0x7ffa6c070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 shutdown_connections 2024-04-24T23:26:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.268+0000 7ffa718cd640 1 -- 172.21.15.33:0/2091391178 wait complete. 2024-04-24T23:26:11.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 -- 172.21.15.33:0/3765033272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34101870 msgr2=0x7f8d34101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 --2- 172.21.15.33:0/3765033272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34101870 0x7f8d34101cf0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 -- 172.21.15.33:0/3765033272 shutdown_connections 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 --2- 172.21.15.33:0/3765033272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34102230 0x7f8d34108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 --2- 172.21.15.33:0/3765033272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34101870 0x7f8d34101cf0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 --2- 172.21.15.33:0/3765033272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34100680 0x7f8d34100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.391+0000 7f8d38812640 1 -- 172.21.15.33:0/3765033272 >> 172.21.15.33:0/3765033272 conn(0x7f8d340fbbf0 msgr2=0x7f8d340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.392+0000 7f8d38812640 1 -- 172.21.15.33:0/3765033272 shutdown_connections 2024-04-24T23:26:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.392+0000 7f8d38812640 1 -- 172.21.15.33:0/3765033272 wait complete. 2024-04-24T23:26:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.392+0000 7f8d38812640 1 Processor -- start 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.392+0000 7f8d38812640 1 -- start start 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d38812640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 0x7f8d34199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d38812640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34101870 0x7f8d3419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d38812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 0x7f8d341a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d38812640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410c300 con 0x7f8d34102230 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d38812640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410c180 con 0x7f8d34100680 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d38812640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410c480 con 0x7f8d34101870 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d32d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 0x7f8d341a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 0x7f8d34199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d32d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 0x7f8d341a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57166/0 (socket says 172.21.15.33:57166) 2024-04-24T23:26:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d32d9d640 1 -- 172.21.15.33:0/2117645792 learned_addr learned my addr 172.21.15.33:0/2117645792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 0x7f8d34199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36064/0 (socket says 172.21.15.33:36064) 2024-04-24T23:26:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34101870 0x7f8d3419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 -- 172.21.15.33:0/2117645792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34101870 msgr2=0x7f8d3419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34101870 0x7f8d3419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 -- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 msgr2=0x7f8d341a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 0x7f8d341a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 -- 172.21.15.33:0/2117645792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d341a1930 con 0x7f8d34100680 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d32d9d640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 0x7f8d341a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34101870 0x7f8d3419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.393+0000 7f8d3259c640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 0x7f8d34199d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e9c0 tx=0x7f8d2400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.394+0000 7f8d2bfff640 1 -- 172.21.15.33:0/2117645792 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400ccc0 con 0x7f8d34100680 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.394+0000 7f8d2bfff640 1 -- 172.21.15.33:0/2117645792 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d240035f0 con 0x7f8d34100680 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.394+0000 7f8d2bfff640 1 -- 172.21.15.33:0/2117645792 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400ce60 con 0x7f8d34100680 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.394+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341a1c20 con 0x7f8d34100680 2024-04-24T23:26:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.394+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341a2170 con 0x7f8d34100680 2024-04-24T23:26:11.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.396+0000 7f8d2bfff640 1 -- 172.21.15.33:0/2117645792 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d240037a0 con 0x7f8d34100680 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 Processor -- start 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 -- start start 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d2bfff640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d140a2c10 0x7f8d140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3feece670 con 0x55e3fef6ac20 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3fee76970 con 0x55e3fef6a230 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e3feeca630 con 0x55e3fef6fd90 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d32d9d640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d140a2c10 0x7f8d140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d2bfff640 1 -- 172.21.15.33:0/2117645792 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8d240cf900 con 0x7f8d34100680 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57178/0 (socket says 172.21.15.33:57178) 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3135256485 learned_addr learned my addr 172.21.15.33:0/3135256485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.397+0000 7f8d32d9d640 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3135256485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 msgr2=0x55e3fef72220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef72220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3135256485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 msgr2=0x55e3fef6a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3135256485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fee9a400 con 0x55e3fef6ac20 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d3259c640 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d32d9d640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d140a2c10 0x7f8d140a50d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009fd0 tx=0x7f8d2c011810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6f720 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02a930 tx=0x7f8d1c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d20450f2ee7ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d28d99640 1 -- 172.21.15.33:0/3135256485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c040070 con 0x55e3fef6ac20 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d28d99640 1 -- 172.21.15.33:0/3135256485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c031930 con 0x55e3fef6ac20 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.398+0000 7f8d28d99640 1 -- 172.21.15.33:0/3135256485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035040 con 0x55e3fef6ac20 2024-04-24T23:26:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3135256485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 msgr2=0x55e3fef6f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6f720 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02a930 tx=0x7f8d1c009680 comp rx=0 tx=0).stop 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3135256485 shutdown_connections 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef72220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6f720 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3135256485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3135256485 >> 172.21.15.33:0/3135256485 conn(0x55e3fef658a0 msgr2=0x55e3fef65ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3135256485 shutdown_connections 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3135256485 wait complete. 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 -- start start 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.399+0000 7f8d3b5f9740 1 Processor -- start 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef74320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef74860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3fef72ce0 con 0x55e3fef6ac20 2024-04-24T23:26:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3fee8a050 con 0x55e3fef6a230 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e3fef72e60 con 0x55e3fef6fd90 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 10 client.0 _ll_register_callbacks cb 0x55e3febe1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef74860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef74320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef74860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57190/0 (socket says 172.21.15.33:57190) 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 -- 172.21.15.33:0/2675069692 learned_addr learned my addr 172.21.15.33:0/2675069692 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d32d9d640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 -- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 msgr2=0x55e3fef7b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 -- 172.21.15.33:0/2675069692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 msgr2=0x55e3fef74320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef74320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 -- 172.21.15.33:0/2675069692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fee5bdc0 con 0x55e3fef6ac20 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3259c640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef74320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef74860 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c000c00 tx=0x7f8d1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192f0c1d5accf483 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d21ffb640 1 -- 172.21.15.33:0/2675069692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c04b020 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d21ffb640 1 -- 172.21.15.33:0/2675069692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8d1c0037f0 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.400+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1c031930 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d21ffb640 1 -- 172.21.15.33:0/2675069692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d1c056050 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 10 client.18399 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3febe23f0 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d21ffb640 1 -- 172.21.15.33:0/2675069692 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c031930 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d21ffb640 1 -- 172.21.15.33:0/2675069692 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8d1c0524b0 con 0x55e3fef6ac20 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 10 client.18399 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 10 client.18399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 20 client.18399 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fef7fd20 0x55e3fef82110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 4 client.18399 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3feedbbb0 con 0x55e3fef7fd20 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.401+0000 7f8d3259c640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fef7fd20 0x55e3fef82110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d21ffb640 10 client.18399 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3259c640 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fef7fd20 0x55e3fef82110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a446850d127ad274 server_cookie=ba71ba78974e9e6a in_seq=0 out_seq=0 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d21ffb640 1 -- 172.21.15.33:0/2675069692 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3feedbbb0 con 0x55e3fef7fd20 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d21ffb640 10 client.18399 handle_command_reply: tid=0 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 client.18399 shutdown 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 msgr2=0x55e3fef74860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef74860 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c000c00 tx=0x7f8d1c004270 comp rx=0 tx=0).stop 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 shutdown_connections 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fef7fd20 0x55e3fef82110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d21ffb640 0 client.18399 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef74860 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/2675069692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef74320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.402+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 >> 172.21.15.33:0/2675069692 conn(0x55e3fef658a0 msgr2=0x55e3fef64b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.403+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 shutdown_connections 2024-04-24T23:26:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.403+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/2675069692 wait complete. 2024-04-24T23:26:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.403+0000 7f8d3b5f9740 20 client.18399 trim_cache size 0 max 16384 2024-04-24T23:26:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 Processor -- start 2024-04-24T23:26:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 -- start start 2024-04-24T23:26:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341aee80 con 0x55e3fef6ac20 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ac6f0 con 0x55e3fef6a230 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.418+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341aa540 con 0x55e3fef6fd90 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57200/0 (socket says 172.21.15.33:57200) 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 -- 172.21.15.33:0/822439623 learned_addr learned my addr 172.21.15.33:0/822439623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d32d9d640 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 -- 172.21.15.33:0/822439623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 msgr2=0x55e3fef7b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 -- 172.21.15.33:0/822439623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 msgr2=0x55e3fef6bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 -- 172.21.15.33:0/822439623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fecac7d0 con 0x55e3fef6ac20 2024-04-24T23:26:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6c2b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c052920 tx=0x7f8d1c009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134aa755c735b64d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d23fff640 1 -- 172.21.15.33:0/822439623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c04b070 con 0x55e3fef6ac20 2024-04-24T23:26:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d23fff640 1 -- 172.21.15.33:0/822439623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c052d10 con 0x55e3fef6ac20 2024-04-24T23:26:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.419+0000 7f8d23fff640 1 -- 172.21.15.33:0/822439623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035070 con 0x55e3fef6ac20 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/822439623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 msgr2=0x55e3fef6c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6c2b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c052920 tx=0x7f8d1c009610 comp rx=0 tx=0).stop 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/822439623 shutdown_connections 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef6c2b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/822439623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef6bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/822439623 >> 172.21.15.33:0/822439623 conn(0x55e3fef658a0 msgr2=0x55e3fef6b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/822439623 shutdown_connections 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/822439623 wait complete. 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.420+0000 7f8d3b5f9740 1 -- start start 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 Processor -- start 2024-04-24T23:26:11.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef79320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef79860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341aabb0 con 0x55e3fef6ac20 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ac020 con 0x55e3fef6a230 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ab280 con 0x55e3fef6fd90 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3b5f9740 10 client.0 _ll_register_callbacks cb 0x7f8d240978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef79860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef79860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57202/0 (socket says 172.21.15.33:57202) 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef79320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3890523157 learned_addr learned my addr 172.21.15.33:0/3890523157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 msgr2=0x55e3fef7a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3890523157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 msgr2=0x55e3fef79320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef79320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 -- 172.21.15.33:0/3890523157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fef7ae60 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d3259c640 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef79320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d29d9b640 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef79860 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0314e0 tx=0x7f8d1c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a47ff4c709bcc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.421+0000 7f8d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d21ffb640 1 -- 172.21.15.33:0/3890523157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c04b070 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d21ffb640 1 -- 172.21.15.33:0/3890523157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8d1c031800 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1c052d10 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d21ffb640 1 -- 172.21.15.33:0/3890523157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d1c035040 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 10 client.18405 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3fefe6450 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d21ffb640 1 -- 172.21.15.33:0/3890523157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c052d10 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d21ffb640 1 -- 172.21.15.33:0/3890523157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8d1c0052a0 con 0x55e3fef6ac20 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 10 client.18405 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 10 client.18405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 20 client.18405 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fefe6ba0 0x55e3fefe8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 4 client.18405 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.422+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3feed5f00 con 0x55e3fefe6ba0 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3259c640 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fefe6ba0 0x55e3fefe8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3259c640 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fefe6ba0 0x55e3fefe8f90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a59d6e5e6efef9e3 server_cookie=6feee62e1ad9ce5b in_seq=0 out_seq=0 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d21ffb640 10 client.18405 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d21ffb640 1 -- 172.21.15.33:0/3890523157 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3feed5f00 con 0x55e3fefe6ba0 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d21ffb640 10 client.18405 handle_command_reply: tid=0 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 client.18405 shutdown 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 msgr2=0x55e3fef79860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef79860 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0314e0 tx=0x7f8d1c009680 comp rx=0 tx=0).stop 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 shutdown_connections 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e3fefe6ba0 0x55e3fefe8f90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3fef6fd90 0x55e3fef7a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3fef6ac20 0x55e3fef79860 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d21ffb640 0 client.18405 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 --2- 172.21.15.33:0/3890523157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3fef6a230 0x55e3fef79320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 >> 172.21.15.33:0/3890523157 conn(0x55e3fef658a0 msgr2=0x55e3fef6bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.423+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 shutdown_connections 2024-04-24T23:26:11.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d3b5f9740 1 -- 172.21.15.33:0/3890523157 wait complete. 2024-04-24T23:26:11.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d3b5f9740 20 client.18405 trim_cache size 0 max 16384 2024-04-24T23:26:11.430 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d140a2c10 msgr2=0x7f8d140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d140a2c10 0x7f8d140a50d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009fd0 tx=0x7f8d2c011810 comp rx=0 tx=0).stop 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 msgr2=0x7f8d34199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 0x7f8d34199d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e9c0 tx=0x7f8d2400ee80 comp rx=0 tx=0).stop 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 shutdown_connections 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d140a2c10 0x7f8d140a50d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d34102230 0x7f8d341a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d34101870 0x7f8d3419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 --2- 172.21.15.33:0/2117645792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d34100680 0x7f8d34199d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 >> 172.21.15.33:0/2117645792 conn(0x7f8d340fbbf0 msgr2=0x7f8d340fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 shutdown_connections 2024-04-24T23:26:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.424+0000 7f8d38812640 1 -- 172.21.15.33:0/2117645792 wait complete. 2024-04-24T23:26:11.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:11 smithi033 ceph-mon[29201]: pgmap v776: 129 pgs: 129 active+clean; 465 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 24 MiB/s wr, 462 op/s 2024-04-24T23:26:11.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:11.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:11.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:26:11.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:11 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/51543058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:11.436 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:26:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 -- 172.21.15.33:0/1078487527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880100680 msgr2=0x7f8880100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 --2- 172.21.15.33:0/1078487527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880100680 0x7f8880100aa0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8870009ca0 tx=0x7f887002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 -- 172.21.15.33:0/1078487527 shutdown_connections 2024-04-24T23:26:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 --2- 172.21.15.33:0/1078487527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 0x7f8880108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 --2- 172.21.15.33:0/1078487527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880101870 0x7f8880101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 --2- 172.21.15.33:0/1078487527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880100680 0x7f8880100aa0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 -- 172.21.15.33:0/1078487527 >> 172.21.15.33:0/1078487527 conn(0x7f88800fbbf0 msgr2=0x7f88800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 -- 172.21.15.33:0/1078487527 shutdown_connections 2024-04-24T23:26:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 -- 172.21.15.33:0/1078487527 wait complete. 2024-04-24T23:26:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 Processor -- start 2024-04-24T23:26:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.547+0000 7f8885888640 1 -- start start 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8885888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880100680 0x7f8880197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8885888640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 0x7f8880198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8885888640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 0x7f888019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8885888640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f888010c300 con 0x7f8880101870 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8885888640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f888010c180 con 0x7f8880102230 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8885888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f888010c480 con 0x7f8880100680 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 0x7f8880198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880100680 0x7f8880197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 0x7f8880198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57212/0 (socket says 172.21.15.33:57212) 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887e7fc640 1 -- 172.21.15.33:0/70700191 learned_addr learned my addr 172.21.15.33:0/70700191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 0x7f888019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 -- 172.21.15.33:0/70700191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 msgr2=0x7f888019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 0x7f888019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 -- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 msgr2=0x7f8880198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 0x7f8880198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 -- 172.21.15.33:0/70700191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888019f720 con 0x7f8880100680 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887e7fc640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 0x7f8880198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887f7fe640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 0x7f888019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f887effd640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880100680 0x7f8880197b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8870009bf0 tx=0x7f8870031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.548+0000 7f8884886640 1 -- 172.21.15.33:0/70700191 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887002fa20 con 0x7f8880100680 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.549+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888019f9b0 con 0x7f8880100680 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.549+0000 7f8884886640 1 -- 172.21.15.33:0/70700191 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f887002fbc0 con 0x7f8880100680 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.549+0000 7f8884886640 1 -- 172.21.15.33:0/70700191 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870046750 con 0x7f8880100680 2024-04-24T23:26:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.549+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888019ff00 con 0x7f8880100680 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.551+0000 7f8884886640 1 -- 172.21.15.33:0/70700191 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f887002fd70 con 0x7f8880100680 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.551+0000 7f8888591740 1 Processor -- start 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.551+0000 7f8888591740 1 -- start start 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.551+0000 7f8884886640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88680a2b90 0x7f88680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.551+0000 7f8888591740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 0x55beaf52d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f8888591740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf532720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f8888591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf532d90 0x55beaf535220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f8888591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55beaf491670 con 0x55beaf532d90 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f8888591740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55beaf439970 con 0x55beaf52d230 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f8888591740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55beaf48d630 con 0x55beaf52dc20 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f8884886640 1 -- 172.21.15.33:0/70700191 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f88700f0630 con 0x7f8880100680 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887f7fe640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88680a2b90 0x7f88680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 0x55beaf52d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf532720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf532720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46556/0 (socket says 172.21.15.33:46556) 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887e7fc640 1 -- 172.21.15.33:0/2695554419 learned_addr learned my addr 172.21.15.33:0/2695554419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 0x55beaf52d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36122/0 (socket says 172.21.15.33:36122) 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887f7fe640 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf532d90 0x55beaf535220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 -- 172.21.15.33:0/2695554419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 msgr2=0x55beaf532720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf532720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 -- 172.21.15.33:0/2695554419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf532d90 msgr2=0x55beaf535220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf532d90 0x55beaf535220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.552+0000 7f887effd640 1 -- 172.21.15.33:0/2695554419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beaf45d400 con 0x55beaf52d230 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.553+0000 7f887effd640 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 0x55beaf52d650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8870006010 tx=0x7f887002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873a3ee3e5073029 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.553+0000 7f887f7fe640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88680a2b90 0x7f88680a5050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8878004650 tx=0x7f8878009bf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.553+0000 7f8865ffb640 1 -- 172.21.15.33:0/2695554419 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700882f0 con 0x55beaf52d230 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.553+0000 7f8865ffb640 1 -- 172.21.15.33:0/2695554419 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8870036070 con 0x55beaf52d230 2024-04-24T23:26:11.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.553+0000 7f8865ffb640 1 -- 172.21.15.33:0/2695554419 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700bae00 con 0x55beaf52d230 2024-04-24T23:26:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 -- 172.21.15.33:0/2695554419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 msgr2=0x55beaf52d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 0x55beaf52d650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8870006010 tx=0x7f887002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 -- 172.21.15.33:0/2695554419 shutdown_connections 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf532d90 0x55beaf535220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf532720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 --2- 172.21.15.33:0/2695554419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52d230 0x55beaf52d650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 -- 172.21.15.33:0/2695554419 >> 172.21.15.33:0/2695554419 conn(0x55beaf5288a0 msgr2=0x55beaf528ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 -- 172.21.15.33:0/2695554419 shutdown_connections 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 -- 172.21.15.33:0/2695554419 wait complete. 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 -- start start 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.554+0000 7f8888591740 1 Processor -- start 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf537140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf537680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf53eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55beaf439970 con 0x55beaf53c6b0 2024-04-24T23:26:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55beaf44d050 con 0x55beaf52dc20 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55beaf535b00 con 0x55beaf532d90 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f8888591740 10 client.0 _ll_register_callbacks cb 0x55beaf1a4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf537680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf53eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf537140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf537140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36128/0 (socket says 172.21.15.33:36128) 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf537680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46568/0 (socket says 172.21.15.33:46568) 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 -- 172.21.15.33:0/2390764147 learned_addr learned my addr 172.21.15.33:0/2390764147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf53eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57226/0 (socket says 172.21.15.33:57226) 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 -- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 msgr2=0x55beaf537680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf537680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 -- 172.21.15.33:0/2390764147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 msgr2=0x55beaf53eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf53eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 -- 172.21.15.33:0/2390764147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beaf41edc0 con 0x55beaf52dc20 2024-04-24T23:26:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f887effd640 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf537140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88700b8f60 tx=0x7f88700c0f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81b43770b6a1f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.555+0000 7f885effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f885effd640 1 -- 172.21.15.33:0/2390764147 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700bcb00 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8870036070 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f885effd640 1 -- 172.21.15.33:0/2390764147 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f88700c52e0 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f885effd640 1 -- 172.21.15.33:0/2390764147 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88700c5520 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 10 client.27923 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beaf1a53f0 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f885effd640 1 -- 172.21.15.33:0/2390764147 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8870036070 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f885effd640 1 -- 172.21.15.33:0/2390764147 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f887004e030 con 0x55beaf52dc20 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 10 client.27923 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 10 client.27923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 20 client.27923 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf543180 0x55beaf545600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 4 client.27923 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beaf49ebb0 con 0x55beaf543180 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.556+0000 7f887e7fc640 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf543180 0x55beaf545600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f885effd640 10 client.27923 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f887e7fc640 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf543180 0x55beaf545600 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09807bb57e9f108 server_cookie=98ec6c4aafc41716 in_seq=0 out_seq=0 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f885effd640 1 -- 172.21.15.33:0/2390764147 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55beaf49ebb0 con 0x55beaf543180 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f885effd640 10 client.27923 handle_command_reply: tid=0 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f8888591740 1 client.27923 shutdown 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 msgr2=0x55beaf537140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.557+0000 7f8888591740 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf537140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88700b8f60 tx=0x7f88700c0f20 comp rx=0 tx=0).stop 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 shutdown_connections 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf543180 0x55beaf545600 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf53eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f885effd640 0 client.27923 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf537680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 --2- 172.21.15.33:0/2390764147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf537140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 >> 172.21.15.33:0/2390764147 conn(0x55beaf5288a0 msgr2=0x55beaf527b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 shutdown_connections 2024-04-24T23:26:11.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 1 -- 172.21.15.33:0/2390764147 wait complete. 2024-04-24T23:26:11.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.558+0000 7f8888591740 20 client.27923 trim_cache size 0 max 16384 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 Processor -- start 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 -- start start 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf5375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf52bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf52e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88801acc20 con 0x55beaf53c6b0 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f88801aa490 con 0x55beaf52dc20 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f8888591740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a82e0 con 0x55beaf532d90 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf52e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf52e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57242/0 (socket says 172.21.15.33:57242) 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 -- 172.21.15.33:0/87642294 learned_addr learned my addr 172.21.15.33:0/87642294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf52bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887effd640 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf5375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 -- 172.21.15.33:0/87642294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 msgr2=0x55beaf52bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf52bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 -- 172.21.15.33:0/87642294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 msgr2=0x55beaf5375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf5375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 -- 172.21.15.33:0/87642294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beaf26f7d0 con 0x55beaf53c6b0 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887effd640 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf5375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.572+0000 7f887f7fe640 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf52e760 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f88780137f0 tx=0x7f8878020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ed1b15b7667cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f88657fa640 1 -- 172.21.15.33:0/87642294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878013cd0 con 0x55beaf53c6b0 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f88657fa640 1 -- 172.21.15.33:0/87642294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878013e70 con 0x55beaf53c6b0 2024-04-24T23:26:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f88657fa640 1 -- 172.21.15.33:0/87642294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878022840 con 0x55beaf53c6b0 2024-04-24T23:26:11.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 -- 172.21.15.33:0/87642294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 msgr2=0x55beaf52e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf52e760 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f88780137f0 tx=0x7f8878020000 comp rx=0 tx=0).stop 2024-04-24T23:26:11.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 -- 172.21.15.33:0/87642294 shutdown_connections 2024-04-24T23:26:11.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf52e760 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf532d90 0x55beaf52bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 --2- 172.21.15.33:0/87642294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf52dc20 0x55beaf5375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 -- 172.21.15.33:0/87642294 >> 172.21.15.33:0/87642294 conn(0x55beaf5288a0 msgr2=0x55beaf52b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 -- 172.21.15.33:0/87642294 shutdown_connections 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.573+0000 7f8888591740 1 -- 172.21.15.33:0/87642294 wait complete. 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 -- start start 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 Processor -- start 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf5544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf532d90 0x55beaf5549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf555a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a8950 con 0x55beaf53c6b0 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a9dc0 con 0x55beaf532d90 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a9020 con 0x55beaf52dc20 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f8888591740 10 client.0 _ll_register_callbacks cb 0x7f8878013820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf555a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f887f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf555a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57256/0 (socket says 172.21.15.33:57256) 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f887f7fe640 1 -- 172.21.15.33:0/611111969 learned_addr learned my addr 172.21.15.33:0/611111969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f887e7fc640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf532d90 0x55beaf5549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.574+0000 7f887effd640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf5544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f887f7fe640 1 -- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 msgr2=0x55beaf5544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f887f7fe640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf5544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f887f7fe640 1 -- 172.21.15.33:0/611111969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf532d90 msgr2=0x55beaf5549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f887f7fe640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf532d90 0x55beaf5549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f887f7fe640 1 -- 172.21.15.33:0/611111969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beaf555ff0 con 0x55beaf53c6b0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f887f7fe640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf555a00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8878011810 tx=0x7f8878011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8c201a2b187e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f885effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f885effd640 1 -- 172.21.15.33:0/611111969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878013cd0 con 0x55beaf53c6b0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f885effd640 1 -- 172.21.15.33:0/611111969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8878005050 con 0x55beaf53c6b0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8878013e70 con 0x55beaf53c6b0 2024-04-24T23:26:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f885effd640 1 -- 172.21.15.33:0/611111969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8878022410 con 0x55beaf53c6b0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f8888591740 10 client.18423 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beaf5a9580 con 0x55beaf53c6b0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.575+0000 7f885effd640 1 -- 172.21.15.33:0/611111969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878013e70 con 0x55beaf53c6b0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f885effd640 1 -- 172.21.15.33:0/611111969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f887801f040 con 0x55beaf53c6b0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 10 client.18423 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 20 client.18423 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf5a9fc0 0x55beaf5ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 4 client.18423 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55beaf498f00 con 0x55beaf5a9fc0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f887effd640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf5a9fc0 0x55beaf5ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f885effd640 10 client.18423 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f887effd640 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf5a9fc0 0x55beaf5ac3e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a746d75f9abc55e server_cookie=48d6ce4779a7887b in_seq=0 out_seq=0 2024-04-24T23:26:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f885effd640 1 -- 172.21.15.33:0/611111969 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55beaf498f00 con 0x55beaf5a9fc0 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f885effd640 10 client.18423 handle_command_reply: tid=0 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 1 client.18423 shutdown 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.576+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 msgr2=0x55beaf555a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf555a00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8878011810 tx=0x7f8878011af0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 shutdown_connections 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55beaf5a9fc0 0x55beaf5ac3e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55beaf53c6b0 0x55beaf555a00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55beaf532d90 0x55beaf5549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 --2- 172.21.15.33:0/611111969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beaf52dc20 0x55beaf5544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f885effd640 0 client.18423 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 >> 172.21.15.33:0/611111969 conn(0x55beaf5288a0 msgr2=0x55beaf537460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 shutdown_connections 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 1 -- 172.21.15.33:0/611111969 wait complete. 2024-04-24T23:26:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8888591740 20 client.18423 trim_cache size 0 max 16384 2024-04-24T23:26:11.584 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88680a2b90 msgr2=0x7f88680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88680a2b90 0x7f88680a5050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8878004650 tx=0x7f8878009bf0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880100680 msgr2=0x7f8880197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880100680 0x7f8880197b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8870009bf0 tx=0x7f8870031ee0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 shutdown_connections 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f88680a2b90 0x7f88680a5050 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102230 0x7f888019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8880101870 0x7f8880198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 --2- 172.21.15.33:0/70700191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8880100680 0x7f8880197b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 >> 172.21.15.33:0/70700191 conn(0x7f88800fbbf0 msgr2=0x7f8880068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.577+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 shutdown_connections 2024-04-24T23:26:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:11.578+0000 7f8885888640 1 -- 172.21.15.33:0/70700191 wait complete. 2024-04-24T23:26:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:11 smithi038 ceph-mon[32450]: pgmap v776: 129 pgs: 129 active+clean; 465 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 24 MiB/s wr, 462 op/s 2024-04-24T23:26:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:26:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:11 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/51543058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:11 smithi117 ceph-mon[32529]: pgmap v776: 129 pgs: 129 active+clean; 465 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 24 MiB/s wr, 462 op/s 2024-04-24T23:26:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:26:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:11 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/51543058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:11.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 7598 36.46 MB/sec warmup 16 sec latency 63.309 ms 2024-04-24T23:26:12.590 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:26:12.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:12.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:26:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.712+0000 7f3ee6d14640 1 -- 172.21.15.33:0/1572917531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 msgr2=0x7f3ee0102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.712+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/1572917531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0102e00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009900 tx=0x7f3ecc02a410 comp rx=0 tx=0).stop 2024-04-24T23:26:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 -- 172.21.15.33:0/1572917531 shutdown_connections 2024-04-24T23:26:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/1572917531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0103340 0x7f3ee0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/1572917531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0102e00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/1572917531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee0101790 0x7f3ee0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 -- 172.21.15.33:0/1572917531 >> 172.21.15.33:0/1572917531 conn(0x7f3ee00fcd20 msgr2=0x7f3ee00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 -- 172.21.15.33:0/1572917531 shutdown_connections 2024-04-24T23:26:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.713+0000 7f3ee6d14640 1 -- 172.21.15.33:0/1572917531 wait complete. 2024-04-24T23:26:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 Processor -- start 2024-04-24T23:26:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 -- start start 2024-04-24T23:26:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 0x7f3ee0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee0103340 0x7f3ee019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee010d3f0 con 0x7f3ee0102980 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee010d270 con 0x7f3ee0101790 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee6d14640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee010d570 con 0x7f3ee0103340 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3edffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 0x7f3ee0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3edffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57278/0 (socket says 172.21.15.33:57278) 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3edffff640 1 -- 172.21.15.33:0/518441088 learned_addr learned my addr 172.21.15.33:0/518441088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee0103340 0x7f3ee019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.714+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 0x7f3ee0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36162/0 (socket says 172.21.15.33:36162) 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3edffff640 1 -- 172.21.15.33:0/518441088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee0103340 msgr2=0x7f3ee019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3edffff640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee0103340 0x7f3ee019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3edffff640 1 -- 172.21.15.33:0/518441088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 msgr2=0x7f3ee0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3edffff640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 0x7f3ee0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3edffff640 1 -- 172.21.15.33:0/518441088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee019f6e0 con 0x7f3ee0102980 2024-04-24T23:26:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 0x7f3ee0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3edffff640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0198000 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009230 tx=0x7f3ecc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3eddffb640 1 -- 172.21.15.33:0/518441088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc03f030 con 0x7f3ee0102980 2024-04-24T23:26:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3eddffb640 1 -- 172.21.15.33:0/518441088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ecc003ce0 con 0x7f3ee0102980 2024-04-24T23:26:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee019f970 con 0x7f3ee0102980 2024-04-24T23:26:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3eddffb640 1 -- 172.21.15.33:0/518441088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc035070 con 0x7f3ee0102980 2024-04-24T23:26:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.715+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee019fe60 con 0x7f3ee0102980 2024-04-24T23:26:12.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3eddffb640 1 -- 172.21.15.33:0/518441088 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3ecc042030 con 0x7f3ee0102980 2024-04-24T23:26:12.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 Processor -- start 2024-04-24T23:26:12.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 -- start start 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa229501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa22955280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22957d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aa228b32c0 con 0x55aa22950780 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aa2285b910 con 0x55aa2294fd90 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.718+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aa228af280 con 0x55aa229558f0 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa22955280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa229501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3edffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22957d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa22955280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57292/0 (socket says 172.21.15.33:57292) 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee4ab0640 1 -- 172.21.15.33:0/3847204948 learned_addr learned my addr 172.21.15.33:0/3847204948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3eddffb640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3ed00a2b90 0x7f3ed00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 -- 172.21.15.33:0/3847204948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 msgr2=0x55aa22957d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22957d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 -- 172.21.15.33:0/3847204948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 msgr2=0x55aa22955280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa22955280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 -- 172.21.15.33:0/3847204948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa2286cfb0 con 0x55aa2294fd90 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa22955280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3edffff640 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22957d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3eddffb640 1 -- 172.21.15.33:0/518441088 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ecc0ef900 con 0x7f3ee0102980 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3ed00a2b90 0x7f3ed00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.719+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa229501b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009f00 tx=0x7f3ed8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e84e51313a3f591 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ecaffd640 1 -- 172.21.15.33:0/3847204948 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80045c0 con 0x55aa2294fd90 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ecaffd640 1 -- 172.21.15.33:0/3847204948 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed8004760 con 0x55aa2294fd90 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ecaffd640 1 -- 172.21.15.33:0/3847204948 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed801c660 con 0x55aa2294fd90 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3847204948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 msgr2=0x55aa229501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3ed00a2b90 0x7f3ed00a5050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8020000 tx=0x7f3ed8021000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa229501b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009f00 tx=0x7f3ed8011810 comp rx=0 tx=0).stop 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3847204948 shutdown_connections 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22957d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa22955280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3847204948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa229501b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.720+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3847204948 >> 172.21.15.33:0/3847204948 conn(0x55aa22917950 msgr2=0x55aa2294b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.721+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3847204948 shutdown_connections 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.721+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3847204948 wait complete. 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.721+0000 7f3ee9a26740 1 -- start start 2024-04-24T23:26:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.721+0000 7f3ee9a26740 1 Processor -- start 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa2295a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22961760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aa229588a0 con 0x55aa22950780 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aa2286ee90 con 0x55aa2294fd90 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aa22958a20 con 0x55aa229558f0 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee9a26740 10 client.0 _ll_register_callbacks cb 0x55aa228aa970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa2295a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa2295a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57298/0 (socket says 172.21.15.33:57298) 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 -- 172.21.15.33:0/4187134026 learned_addr learned my addr 172.21.15.33:0/4187134026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3ee52b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22961760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 -- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 msgr2=0x55aa22961760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22961760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 -- 172.21.15.33:0/4187134026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 msgr2=0x55aa2295a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.722+0000 7f3edffff640 1 -- 172.21.15.33:0/4187134026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa2268f7e0 con 0x55aa22950780 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3edffff640 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa2295a720 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0bbdb0 tx=0x7f3ecc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b534704cb6a2a452 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ebffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ebffff640 1 -- 172.21.15.33:0/4187134026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc049070 con 0x55aa22950780 2024-04-24T23:26:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ebffff640 1 -- 172.21.15.33:0/4187134026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ecc0b8490 con 0x55aa22950780 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ed8004760 con 0x55aa22950780 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ebffff640 1 -- 172.21.15.33:0/4187134026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ecc0bbbf0 con 0x55aa22950780 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ee9a26740 10 client.18435 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aa226eff70 con 0x55aa22950780 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.723+0000 7f3ebffff640 1 -- 172.21.15.33:0/4187134026 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed8004760 con 0x55aa22950780 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ebffff640 1 -- 172.21.15.33:0/4187134026 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3ecc0d2070 con 0x55aa22950780 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee9a26740 10 client.18435 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee9a26740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee9a26740 20 client.18435 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa22965bb0 0x55aa22968050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee9a26740 4 client.18435 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa228875a0 con 0x55aa22965bb0 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa22965bb0 0x55aa22968050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa22965bb0 0x55aa22968050 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd99c0d3bc5daed server_cookie=4e425e799588d27e in_seq=0 out_seq=0 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.724+0000 7f3ebffff640 10 client.18435 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.725+0000 7f3ebffff640 1 -- 172.21.15.33:0/4187134026 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa228875a0 con 0x55aa22965bb0 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.725+0000 7f3ebffff640 10 client.18435 handle_command_reply: tid=0 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.725+0000 7f3ee9a26740 1 client.18435 shutdown 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 msgr2=0x55aa2295a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa2295a720 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0bbdb0 tx=0x7f3ecc0092e0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 shutdown_connections 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa22965bb0 0x55aa22968050 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa229558f0 0x55aa22961760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa22950780 0x55aa2295a720 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/4187134026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ebffff640 0 client.18435 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 >> 172.21.15.33:0/4187134026 conn(0x55aa22917950 msgr2=0x55aa2294a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 shutdown_connections 2024-04-24T23:26:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 1 -- 172.21.15.33:0/4187134026 wait complete. 2024-04-24T23:26:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.726+0000 7f3ee9a26740 20 client.18435 trim_cache size 0 max 16384 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.745+0000 7f3ee9a26740 1 Processor -- start 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 -- start start 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 0x55aa2294eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa22961720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa229558f0 0x55aa2296c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01acc90 con 0x55aa2294fd90 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01aa500 con 0x55aa229558f0 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01a8350 con 0x55aa22950780 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 0x55aa2294eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 0x55aa2294eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57302/0 (socket says 172.21.15.33:57302) 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 -- 172.21.15.33:0/3794808358 learned_addr learned my addr 172.21.15.33:0/3794808358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3edffff640 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa22961720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa229558f0 0x55aa2296c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 -- 172.21.15.33:0/3794808358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 msgr2=0x55aa22961720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa22961720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 -- 172.21.15.33:0/3794808358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa229558f0 msgr2=0x55aa2296c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa229558f0 0x55aa2296c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 -- 172.21.15.33:0/3794808358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa22587170 con 0x55aa2294fd90 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.746+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 0x55aa2294eed0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55aa2294c6c0 tx=0x7f3ed4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f14f0c8937aa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3eca7fc640 1 -- 172.21.15.33:0/3794808358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4005220 con 0x55aa2294fd90 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3eca7fc640 1 -- 172.21.15.33:0/3794808358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed4004c80 con 0x55aa2294fd90 2024-04-24T23:26:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3eca7fc640 1 -- 172.21.15.33:0/3794808358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4020730 con 0x55aa2294fd90 2024-04-24T23:26:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3794808358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 msgr2=0x55aa2294eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 0x55aa2294eed0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55aa2294c6c0 tx=0x7f3ed4012b00 comp rx=0 tx=0).stop 2024-04-24T23:26:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3794808358 shutdown_connections 2024-04-24T23:26:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa229558f0 0x55aa2296c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa22961720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/3794808358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa2294fd90 0x55aa2294eed0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3794808358 >> 172.21.15.33:0/3794808358 conn(0x55aa22917950 msgr2=0x55aa2294e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3794808358 shutdown_connections 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.747+0000 7f3ee9a26740 1 -- 172.21.15.33:0/3794808358 wait complete. 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 -- start start 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 Processor -- start 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa2295f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 0x55aa22960690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01a89c0 con 0x55aa229558f0 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01a9e30 con 0x55aa2294fd90 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01a9090 con 0x55aa22950780 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee9a26740 10 client.0 _ll_register_callbacks cb 0x7f3ecc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee52b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 0x55aa22960690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee4ab0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36208/0 (socket says 172.21.15.33:36208) 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee4ab0640 1 -- 172.21.15.33:0/1081868668 learned_addr learned my addr 172.21.15.33:0/1081868668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3ee52b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 0x55aa22960690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57310/0 (socket says 172.21.15.33:57310) 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.748+0000 7f3edffff640 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa2295f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee52b1640 1 -- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 msgr2=0x55aa2295f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa2295f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee52b1640 1 -- 172.21.15.33:0/1081868668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 msgr2=0x55aa2295f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee52b1640 1 -- 172.21.15.33:0/1081868668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa22960c80 con 0x55aa229558f0 2024-04-24T23:26:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee52b1640 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 0x55aa22960690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8011d10 tx=0x7f3ed8009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1a1841e5f83949 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ebffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ebffff640 1 -- 172.21.15.33:0/1081868668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8003970 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ebffff640 1 -- 172.21.15.33:0/1081868668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ed8004a00 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ed4004c80 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ebffff640 1 -- 172.21.15.33:0/1081868668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ed8041050 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee9a26740 10 client.18441 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aa229cc4e0 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.749+0000 7f3ebffff640 1 -- 172.21.15.33:0/1081868668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed4004c80 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ebffff640 1 -- 172.21.15.33:0/1081868668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3ed80287e0 con 0x55aa229558f0 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee9a26740 10 client.18441 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee9a26740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee9a26740 20 client.18441 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa229ccd70 0x55aa229cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee9a26740 4 client.18441 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:26:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aa228c0800 con 0x55aa229ccd70 2024-04-24T23:26:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa229ccd70 0x55aa229cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ee4ab0640 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa229ccd70 0x55aa229cf190 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce3dd3ad415801e server_cookie=bc936afdf25fb60b in_seq=0 out_seq=0 2024-04-24T23:26:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.750+0000 7f3ebffff640 10 client.18441 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ebffff640 1 -- 172.21.15.33:0/1081868668 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aa228c0800 con 0x55aa229ccd70 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ebffff640 10 client.18441 handle_command_reply: tid=0 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 client.18441 shutdown 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 msgr2=0x55aa22960690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 0x55aa22960690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8011d10 tx=0x7f3ed8009760 comp rx=0 tx=0).stop 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 shutdown_connections 2024-04-24T23:26:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aa229ccd70 0x55aa229cf190 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa229558f0 0x55aa22960690 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ebffff640 0 client.18441 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa22950780 0x55aa2295f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 --2- 172.21.15.33:0/1081868668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa2294fd90 0x55aa2295f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 >> 172.21.15.33:0/1081868668 conn(0x55aa22917950 msgr2=0x55aa2294a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 shutdown_connections 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 1 -- 172.21.15.33:0/1081868668 wait complete. 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.753+0000 7f3ee9a26740 20 client.18441 trim_cache size 0 max 16384 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:26:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "b5a6f49e-fa76-484a-8bd5-1bc579ceb519", 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3ed00a2b90 msgr2=0x7f3ed00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3ed00a2b90 0x7f3ed00a5050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8020000 tx=0x7f3ed8021000 comp rx=0 tx=0).stop 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 msgr2=0x7f3ee0198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0198000 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009230 tx=0x7f3ecc0042f0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 shutdown_connections 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3ed00a2b90 0x7f3ed00a5050 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee0103340 0x7f3ee019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ee0102980 0x7f3ee0198000 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 --2- 172.21.15.33:0/518441088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ee0101790 0x7f3ee0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 >> 172.21.15.33:0/518441088 conn(0x7f3ee00fcd20 msgr2=0x7f3ee0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 shutdown_connections 2024-04-24T23:26:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.754+0000 7f3ee6d14640 1 -- 172.21.15.33:0/518441088 wait complete. 2024-04-24T23:26:12.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:26:12.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:26:12.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 8184 36.45 MB/sec warmup 17 sec latency 53.360 ms 2024-04-24T23:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 -- 172.21.15.33:0/3788225750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 msgr2=0x7f4ba0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 --2- 172.21.15.33:0/3788225750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 0x7f4ba0108ac0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009ca0 tx=0x7f4b9802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 -- 172.21.15.33:0/3788225750 shutdown_connections 2024-04-24T23:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 --2- 172.21.15.33:0/3788225750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 0x7f4ba0108ac0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 --2- 172.21.15.33:0/3788225750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba0101870 0x7f4ba0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 --2- 172.21.15.33:0/3788225750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 -- 172.21.15.33:0/3788225750 >> 172.21.15.33:0/3788225750 conn(0x7f4ba00fbbf0 msgr2=0x7f4ba00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 -- 172.21.15.33:0/3788225750 shutdown_connections 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.878+0000 7f4ba6022640 1 -- 172.21.15.33:0/3788225750 wait complete. 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 Processor -- start 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 -- start start 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba0101870 0x7f4ba0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 0x7f4ba019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba010c2e0 con 0x7f4ba0102230 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba010c160 con 0x7f4ba0100680 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4ba6022640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba010c460 con 0x7f4ba0101870 2024-04-24T23:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 0x7f4ba019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:36216/0 (socket says 172.21.15.33:36216) 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/235053744 learned_addr learned my addr 172.21.15.33:0/235053744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.879+0000 7f4b97fff640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba0101870 0x7f4ba0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/235053744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba0101870 msgr2=0x7f4ba0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba0101870 0x7f4ba0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 msgr2=0x7f4ba019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 0x7f4ba019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/235053744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba019f6d0 con 0x7f4ba0100680 2024-04-24T23:26:12.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0197ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000e9c0 tx=0x7f4b9000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9d7fa640 1 -- 172.21.15.33:0/235053744 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000cde0 con 0x7f4ba0100680 2024-04-24T23:26:12.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9d7fa640 1 -- 172.21.15.33:0/235053744 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b90010910 con 0x7f4ba0100680 2024-04-24T23:26:12.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba019f9c0 con 0x7f4ba0100680 2024-04-24T23:26:12.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4b9d7fa640 1 -- 172.21.15.33:0/235053744 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90018a00 con 0x7f4ba0100680 2024-04-24T23:26:12.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.880+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba019ff10 con 0x7f4ba0100680 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.883+0000 7f4b9d7fa640 1 -- 172.21.15.33:0/235053744 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4b9001f020 con 0x7f4ba0100680 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.883+0000 7f4ba8e05740 1 Processor -- start 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.883+0000 7f4ba8e05740 1 -- start start 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.883+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceea4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceea9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c8cee046b0 con 0x55c8ceea5040 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8cedac880 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c8cee00670 con 0x55c8ceea4650 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceea4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b97fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceea9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceea4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46674/0 (socket says 172.21.15.33:46674) 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 -- 172.21.15.33:0/2404895549 learned_addr learned my addr 172.21.15.33:0/2404895549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9d7fa640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b8c0a2b90 0x7f4b8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 -- 172.21.15.33:0/2404895549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 msgr2=0x55c8ceeac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 -- 172.21.15.33:0/2404895549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 msgr2=0x55c8ceea9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceea9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 -- 172.21.15.33:0/2404895549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8cebe27d0 con 0x55c8ceea4650 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9d7fa640 1 -- 172.21.15.33:0/235053744 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b900cf5c0 con 0x7f4ba0100680 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.884+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b8c0a2b90 0x7f4b8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.885+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceea4a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009ca0 tx=0x7f4b980093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1487b573613dec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.885+0000 7f4b95ffb640 1 -- 172.21.15.33:0/2404895549 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98042040 con 0x55c8ceea4650 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.885+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b8c0a2b90 0x7f4b8c0a5050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b9804d000 tx=0x7f4b9804e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.885+0000 7f4b95ffb640 1 -- 172.21.15.33:0/2404895549 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b98044070 con 0x55c8ceea4650 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.885+0000 7f4b95ffb640 1 -- 172.21.15.33:0/2404895549 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98048430 con 0x55c8ceea4650 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 -- 172.21.15.33:0/2404895549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 msgr2=0x55c8ceea4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceea4a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009ca0 tx=0x7f4b980093b0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 -- 172.21.15.33:0/2404895549 shutdown_connections 2024-04-24T23:26:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceea9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/2404895549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceea4a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 -- 172.21.15.33:0/2404895549 >> 172.21.15.33:0/2404895549 conn(0x55c8cee9fcc0 msgr2=0x55c8ceea00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 -- 172.21.15.33:0/2404895549 shutdown_connections 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 -- 172.21.15.33:0/2404895549 wait complete. 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.886+0000 7f4ba8e05740 1 -- start start 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 Processor -- start 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceeaea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea5040 0x55c8ceeaefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c8ceead160 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8cedbff50 con 0x55c8ceea5040 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c8ceead2e0 con 0x55c8ceea4650 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4ba8e05740 10 client.0 _ll_register_callbacks cb 0x55c8cedfb000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b97fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea5040 0x55c8ceeaefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57350/0 (socket says 172.21.15.33:57350) 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceeaea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 -- 172.21.15.33:0/3894389637 learned_addr learned my addr 172.21.15.33:0/3894389637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 -- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 msgr2=0x55c8ceeaea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceeaea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 -- 172.21.15.33:0/3894389637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea5040 msgr2=0x55c8ceeaefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea5040 0x55c8ceeaefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.887+0000 7f4b9ffff640 1 -- 172.21.15.33:0/3894389637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ceb183f0 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb6000 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009600 tx=0x7f4b9802ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751e5b21e36a7e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b7effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b7effd640 1 -- 172.21.15.33:0/3894389637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98059020 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b98044070 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b7effd640 1 -- 172.21.15.33:0/3894389637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b980307c0 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b7effd640 1 -- 172.21.15.33:0/3894389637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b98042030 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 10 client.18453 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8cead80f0 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b7effd640 1 -- 172.21.15.33:0/3894389637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b98044070 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4b7effd640 1 -- 172.21.15.33:0/3894389637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f4b98048c00 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 10 client.18453 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 20 client.18453 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8ceebc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 4 client.18453 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.888+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8cee11bf0 con 0x55c8ceeba4f0 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.889+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8ceebc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.889+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8ceebc8e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b2151dde4ae613 server_cookie=71c63659d11d80f7 in_seq=0 out_seq=0 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.889+0000 7f4b7effd640 10 client.18453 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4b7effd640 1 -- 172.21.15.33:0/3894389637 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8cee11bf0 con 0x55c8ceeba4f0 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4b7effd640 10 client.18453 handle_command_reply: tid=0 2024-04-24T23:26:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 client.18453 shutdown 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 msgr2=0x55c8ceeb6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb6000 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009600 tx=0x7f4b9802ff40 comp rx=0 tx=0).stop 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 shutdown_connections 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8ceebc8e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb6000 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea5040 0x55c8ceeaefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4b7effd640 0 client.18453 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/3894389637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea4650 0x55c8ceeaea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 >> 172.21.15.33:0/3894389637 conn(0x55c8cee9fcc0 msgr2=0x55c8cee9aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 shutdown_connections 2024-04-24T23:26:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 1 -- 172.21.15.33:0/3894389637 wait complete. 2024-04-24T23:26:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.890+0000 7f4ba8e05740 20 client.18453 trim_cache size 0 max 16384 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 Processor -- start 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 -- start start 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 0x55c8ceeae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea5040 0x55c8ceeb5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceea2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01aaaf0 con 0x55c8ceea4650 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01a8360 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01a61b0 con 0x55c8ceea5040 2024-04-24T23:26:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.909+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 0x55c8ceeae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 0x55c8ceeae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57358/0 (socket says 172.21.15.33:57358) 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/705874315 learned_addr learned my addr 172.21.15.33:0/705874315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b97fff640 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea5040 0x55c8ceeb5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceea2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/705874315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea5040 msgr2=0x55c8ceeb5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea5040 0x55c8ceeb5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/705874315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 msgr2=0x55c8ceea2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceea2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 -- 172.21.15.33:0/705874315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ceead440 con 0x55c8ceea4650 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b97fff640 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea5040 0x55c8ceeb5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 0x55c8ceeae480 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4b900058c0 tx=0x7f4b900d1ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54fa67e4b3bf54d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b957fa640 1 -- 172.21.15.33:0/705874315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9009c190 con 0x55c8ceea4650 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b957fa640 1 -- 172.21.15.33:0/705874315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b900d2d90 con 0x55c8ceea4650 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.910+0000 7f4b957fa640 1 -- 172.21.15.33:0/705874315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90010430 con 0x55c8ceea4650 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 -- 172.21.15.33:0/705874315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 msgr2=0x55c8ceeae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 0x55c8ceeae480 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4b900058c0 tx=0x7f4b900d1ad0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 -- 172.21.15.33:0/705874315 shutdown_connections 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceea2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceea5040 0x55c8ceeb5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/705874315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea4650 0x55c8ceeae480 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 -- 172.21.15.33:0/705874315 >> 172.21.15.33:0/705874315 conn(0x55c8cee9fcc0 msgr2=0x55c8ceeadd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 -- 172.21.15.33:0/705874315 shutdown_connections 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 -- 172.21.15.33:0/705874315 wait complete. 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.911+0000 7f4ba8e05740 1 -- start start 2024-04-24T23:26:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 Processor -- start 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea4650 0x55c8ceeb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceeb4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01a6820 con 0x55c8ceea5040 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01a7c90 con 0x55c8ceea4650 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01a6ef0 con 0x55c8ceeaa1b0 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4ba8e05740 10 client.0 _ll_register_callbacks cb 0x7f4b90097d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceeb4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceeb4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57368/0 (socket says 172.21.15.33:57368) 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 -- 172.21.15.33:0/1414590764 learned_addr learned my addr 172.21.15.33:0/1414590764 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea4650 0x55c8ceeb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 -- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceeaa1b0 msgr2=0x55c8ceeb52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 -- 172.21.15.33:0/1414590764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea4650 msgr2=0x55c8ceeb3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea4650 0x55c8ceeb3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b97fff640 1 -- 172.21.15.33:0/1414590764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ceeb58e0 con 0x55c8ceea5040 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.912+0000 7f4b9ffff640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b97fff640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceeb4350 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4b88012810 tx=0x7f4b88012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfd3979eb9ccc19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b7effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b7effd640 1 -- 172.21.15.33:0/1414590764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88023060 con 0x55c8ceea5040 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b7effd640 1 -- 172.21.15.33:0/1414590764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b88004250 con 0x55c8ceea5040 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b900d2d90 con 0x55c8ceea5040 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b7effd640 1 -- 172.21.15.33:0/1414590764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b8802a050 con 0x55c8ceea5040 2024-04-24T23:26:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 10 client.18459 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8cef21160 con 0x55c8ceea5040 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b7effd640 1 -- 172.21.15.33:0/1414590764 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b900d2d90 con 0x55c8ceea5040 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b7effd640 1 -- 172.21.15.33:0/1414590764 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f4b88003a30 con 0x55c8ceea5040 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 10 client.18459 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 20 client.18459 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8cef23c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 4 client.18459 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8cee0bf40 con 0x55c8ceeba4f0 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.913+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8cef23c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.914+0000 7f4b9f7fe640 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8cef23c30 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0364c17f8a736f server_cookie=3fa1a6319cc66506 in_seq=0 out_seq=0 2024-04-24T23:26:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.914+0000 7f4b7effd640 10 client.18459 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4b7effd640 1 -- 172.21.15.33:0/1414590764 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c8cee0bf40 con 0x55c8ceeba4f0 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4b7effd640 10 client.18459 handle_command_reply: tid=0 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 client.18459 shutdown 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 msgr2=0x55c8ceeb4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceeb4350 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4b88012810 tx=0x7f4b88012c10 comp rx=0 tx=0).stop 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 shutdown_connections 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c8ceeba4f0 0x55c8cef23c30 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8ceeaa1b0 0x55c8ceeb52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ceea5040 0x55c8ceeb4350 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4b7effd640 0 client.18459 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 --2- 172.21.15.33:0/1414590764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8ceea4650 0x55c8ceeb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 >> 172.21.15.33:0/1414590764 conn(0x55c8cee9fcc0 msgr2=0x55c8cee9a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 shutdown_connections 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 1 -- 172.21.15.33:0/1414590764 wait complete. 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba8e05740 20 client.18459 trim_cache size 0 max 16384 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:26:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "b5a6f49e-fa76-484a-8bd5-1bc579ceb519": { 2024-04-24T23:26:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:26:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "b5a6f49e-fa76-484a-8bd5-1bc579ceb519", 2024-04-24T23:26:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:26:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:26:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:26:12.926 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b8c0a2b90 msgr2=0x7f4b8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba6022640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b8c0a2b90 0x7f4b8c0a5050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b9804d000 tx=0x7f4b9804e000 comp rx=0 tx=0).stop 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 msgr2=0x7f4ba0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.916+0000 7f4ba6022640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0197ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000e9c0 tx=0x7f4b9000ee80 comp rx=0 tx=0).stop 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 shutdown_connections 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f4b8c0a2b90 0x7f4b8c0a5050 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ba0102230 0x7f4ba019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba0101870 0x7f4ba0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 --2- 172.21.15.33:0/235053744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0100680 0x7f4ba0197ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 >> 172.21.15.33:0/235053744 conn(0x7f4ba00fbbf0 msgr2=0x7f4ba00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 shutdown_connections 2024-04-24T23:26:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:12.917+0000 7f4ba6022640 1 -- 172.21.15.33:0/235053744 wait complete. 2024-04-24T23:26:12.929 INFO:tasks.cephfs.filesystem:scrub status for tag:b5a6f49e-fa76-484a-8bd5-1bc579ceb519 - {'path': '/', 'tag': 'b5a6f49e-fa76-484a-8bd5-1bc579ceb519', 'options': 'recursive,force'} 2024-04-24T23:26:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:13 smithi038 ceph-mon[32450]: pgmap v777: 129 pgs: 129 active+clean; 470 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-04-24T23:26:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:13 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:13 smithi117 ceph-mon[32529]: pgmap v777: 129 pgs: 129 active+clean; 470 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-04-24T23:26:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:13 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:13 smithi033 ceph-mon[29201]: pgmap v777: 129 pgs: 129 active+clean; 470 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-04-24T23:26:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:13 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:13.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 8639 36.33 MB/sec warmup 18 sec latency 92.074 ms 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:14 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:26:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:14 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:26:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:14 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:14.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 8873 35.01 MB/sec warmup 19 sec latency 60.831 ms 2024-04-24T23:26:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:15 smithi038 ceph-mon[32450]: pgmap v778: 129 pgs: 129 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 15 MiB/s wr, 411 op/s 2024-04-24T23:26:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:15 smithi117 ceph-mon[32529]: pgmap v778: 129 pgs: 129 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 15 MiB/s wr, 411 op/s 2024-04-24T23:26:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:15 smithi033 ceph-mon[29201]: pgmap v778: 129 pgs: 129 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 15 MiB/s wr, 411 op/s 2024-04-24T23:26:15.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 9276 33.84 MB/sec warmup 20 sec latency 108.519 ms 2024-04-24T23:26:16.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 9708 33.14 MB/sec warmup 21 sec latency 201.076 ms 2024-04-24T23:26:17.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:17 smithi038 ceph-mon[32450]: pgmap v779: 129 pgs: 129 active+clean; 471 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 367 op/s 2024-04-24T23:26:17.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:17 smithi117 ceph-mon[32529]: pgmap v779: 129 pgs: 129 active+clean; 471 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 367 op/s 2024-04-24T23:26:17.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:17 smithi033 ceph-mon[29201]: pgmap v779: 129 pgs: 129 active+clean; 471 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 367 op/s 2024-04-24T23:26:17.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 10234 32.99 MB/sec warmup 22 sec latency 104.468 ms 2024-04-24T23:26:18.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 10791 32.48 MB/sec warmup 23 sec latency 101.884 ms 2024-04-24T23:26:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:19 smithi038 ceph-mon[32450]: pgmap v780: 129 pgs: 129 active+clean; 460 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 283 op/s 2024-04-24T23:26:19.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:19 smithi117 ceph-mon[32529]: pgmap v780: 129 pgs: 129 active+clean; 460 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 283 op/s 2024-04-24T23:26:19.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:19 smithi033 ceph-mon[29201]: pgmap v780: 129 pgs: 129 active+clean; 460 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 283 op/s 2024-04-24T23:26:19.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 11428 32.38 MB/sec warmup 24 sec latency 46.379 ms 2024-04-24T23:26:20.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 12354 33.44 MB/sec warmup 25 sec latency 52.575 ms 2024-04-24T23:26:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:21 smithi038 ceph-mon[32450]: pgmap v781: 129 pgs: 129 active+clean; 463 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-04-24T23:26:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:21 smithi117 ceph-mon[32529]: pgmap v781: 129 pgs: 129 active+clean; 463 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-04-24T23:26:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:21 smithi033 ceph-mon[29201]: pgmap v781: 129 pgs: 129 active+clean; 463 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-04-24T23:26:21.877 INFO:tasks.workunit.client.0.smithi033.stdout: 10 12823 33.79 MB/sec warmup 26 sec latency 89.619 ms 2024-04-24T23:26:22.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 13095 32.99 MB/sec warmup 27 sec latency 81.456 ms 2024-04-24T23:26:23.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:23 smithi038 ceph-mon[32450]: pgmap v782: 129 pgs: 129 active+clean; 471 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 348 op/s 2024-04-24T23:26:23.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:23 smithi117 ceph-mon[32529]: pgmap v782: 129 pgs: 129 active+clean; 471 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 348 op/s 2024-04-24T23:26:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:23 smithi033 ceph-mon[29201]: pgmap v782: 129 pgs: 129 active+clean; 471 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 348 op/s 2024-04-24T23:26:23.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 13503 32.47 MB/sec warmup 28 sec latency 125.980 ms 2024-04-24T23:26:24.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 13755 31.69 MB/sec warmup 29 sec latency 129.945 ms 2024-04-24T23:26:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:25 smithi038 ceph-mon[32450]: pgmap v783: 129 pgs: 129 active+clean; 476 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-24T23:26:25.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:25 smithi117 ceph-mon[32529]: pgmap v783: 129 pgs: 129 active+clean; 476 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-24T23:26:25.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:25 smithi033 ceph-mon[29201]: pgmap v783: 129 pgs: 129 active+clean; 476 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-24T23:26:25.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 14139 31.10 MB/sec warmup 30 sec latency 63.225 ms 2024-04-24T23:26:26.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 14848 31.30 MB/sec warmup 31 sec latency 78.382 ms 2024-04-24T23:26:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:27 smithi038 ceph-mon[32450]: pgmap v784: 129 pgs: 129 active+clean; 481 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-24T23:26:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:27 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:26:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:27 smithi117 ceph-mon[32529]: pgmap v784: 129 pgs: 129 active+clean; 481 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-24T23:26:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:27 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:26:27.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:27 smithi033 ceph-mon[29201]: pgmap v784: 129 pgs: 129 active+clean; 481 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-24T23:26:27.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:27 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:26:27.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 15668 31.41 MB/sec warmup 32 sec latency 59.084 ms 2024-04-24T23:26:28.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:28 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:26:28.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:28 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:26:28.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:28 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:26:28.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:28 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:26:28.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:28 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:26:28.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:28 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:26:28.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 16168 31.79 MB/sec warmup 33 sec latency 67.906 ms 2024-04-24T23:26:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:29 smithi038 ceph-mon[32450]: pgmap v785: 129 pgs: 129 active+clean; 482 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 324 op/s 2024-04-24T23:26:29.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:29 smithi117 ceph-mon[32529]: pgmap v785: 129 pgs: 129 active+clean; 482 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 324 op/s 2024-04-24T23:26:29.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:29 smithi033 ceph-mon[29201]: pgmap v785: 129 pgs: 129 active+clean; 482 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 324 op/s 2024-04-24T23:26:29.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 16687 31.76 MB/sec warmup 34 sec latency 127.004 ms 2024-04-24T23:26:30.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 17077 31.47 MB/sec warmup 35 sec latency 186.251 ms 2024-04-24T23:26:31.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:31 smithi038 ceph-mon[32450]: pgmap v786: 129 pgs: 129 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-04-24T23:26:31.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:31 smithi117 ceph-mon[32529]: pgmap v786: 129 pgs: 129 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-04-24T23:26:31.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:31 smithi033 ceph-mon[29201]: pgmap v786: 129 pgs: 129 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-04-24T23:26:31.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 17539 31.30 MB/sec warmup 36 sec latency 75.860 ms 2024-04-24T23:26:32.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 17878 30.94 MB/sec warmup 37 sec latency 136.421 ms 2024-04-24T23:26:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:33 smithi038 ceph-mon[32450]: pgmap v787: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-04-24T23:26:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:33 smithi117 ceph-mon[32529]: pgmap v787: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-04-24T23:26:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:33 smithi033 ceph-mon[29201]: pgmap v787: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-04-24T23:26:33.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 18064 30.25 MB/sec warmup 38 sec latency 437.806 ms 2024-04-24T23:26:34.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 18451 29.91 MB/sec warmup 39 sec latency 255.427 ms 2024-04-24T23:26:35.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:35 smithi038 ceph-mon[32450]: pgmap v788: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-04-24T23:26:35.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:35 smithi117 ceph-mon[32529]: pgmap v788: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-04-24T23:26:35.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:35 smithi033 ceph-mon[29201]: pgmap v788: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-04-24T23:26:35.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 19071 30.06 MB/sec warmup 40 sec latency 59.583 ms 2024-04-24T23:26:36.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 19642 29.74 MB/sec warmup 41 sec latency 62.965 ms 2024-04-24T23:26:37.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:37 smithi038 ceph-mon[32450]: pgmap v789: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.9 MiB/s wr, 361 op/s 2024-04-24T23:26:37.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:37 smithi117 ceph-mon[32529]: pgmap v789: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.9 MiB/s wr, 361 op/s 2024-04-24T23:26:37.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:37 smithi033 ceph-mon[29201]: pgmap v789: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.9 MiB/s wr, 361 op/s 2024-04-24T23:26:37.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 20244 30.08 MB/sec warmup 42 sec latency 43.753 ms 2024-04-24T23:26:38.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 20862 30.33 MB/sec warmup 43 sec latency 128.030 ms 2024-04-24T23:26:39.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:39 smithi038 ceph-mon[32450]: pgmap v790: 129 pgs: 129 active+clean; 489 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-04-24T23:26:39.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:39 smithi117 ceph-mon[32529]: pgmap v790: 129 pgs: 129 active+clean; 489 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-04-24T23:26:39.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:39 smithi033 ceph-mon[29201]: pgmap v790: 129 pgs: 129 active+clean; 489 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-04-24T23:26:39.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 21286 30.34 MB/sec warmup 44 sec latency 88.307 ms 2024-04-24T23:26:40.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 21567 29.89 MB/sec warmup 45 sec latency 67.916 ms 2024-04-24T23:26:41.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:41 smithi038 ceph-mon[32450]: pgmap v791: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 375 op/s 2024-04-24T23:26:41.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:41 smithi117 ceph-mon[32529]: pgmap v791: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 375 op/s 2024-04-24T23:26:41.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:41 smithi033 ceph-mon[29201]: pgmap v791: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 375 op/s 2024-04-24T23:26:41.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 21856 29.49 MB/sec warmup 46 sec latency 146.938 ms 2024-04-24T23:26:42.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 22323 29.30 MB/sec warmup 47 sec latency 88.308 ms 2024-04-24T23:26:42.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:26:42.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:26:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 -- 172.21.15.33:0/3177855441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c104b30 msgr2=0x7fca9c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/3177855441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c104b30 0x7fca9c106f20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009900 tx=0x7fca8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:26:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 -- 172.21.15.33:0/3177855441 shutdown_connections 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/3177855441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca9c107460 0x7fca9c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/3177855441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c104b30 0x7fca9c106f20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/3177855441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c102200 0x7fca9c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 -- 172.21.15.33:0/3177855441 >> 172.21.15.33:0/3177855441 conn(0x7fca9c0fbbf0 msgr2=0x7fca9c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 -- 172.21.15.33:0/3177855441 shutdown_connections 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 -- 172.21.15.33:0/3177855441 wait complete. 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.053+0000 7fcaa35a9640 1 Processor -- start 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 -- start start 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 0x7fca9c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c104b30 0x7fca9c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca9c107460 0x7fca9c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10d440 con 0x7fca9c102200 2024-04-24T23:26:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10d2c0 con 0x7fca9c107460 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa35a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10d5c0 con 0x7fca9c104b30 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 0x7fca9c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c104b30 0x7fca9c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 0x7fca9c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37926/0 (socket says 172.21.15.33:37926) 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1b46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca9c107460 0x7fca9c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 -- 172.21.15.33:0/2783910299 learned_addr learned my addr 172.21.15.33:0/2783910299 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 -- 172.21.15.33:0/2783910299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c104b30 msgr2=0x7fca9c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c104b30 0x7fca9c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 -- 172.21.15.33:0/2783910299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca9c107460 msgr2=0x7fca9c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca9c107460 0x7fca9c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 -- 172.21.15.33:0/2783910299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c19f730 con 0x7fca9c102200 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa0b44640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c104b30 0x7fca9c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.054+0000 7fcaa1345640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 0x7fca9c197b10 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fca9800eab0 tx=0x7fca9800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.055+0000 7fca927fc640 1 -- 172.21.15.33:0/2783910299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9800ce40 con 0x7fca9c102200 2024-04-24T23:26:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.055+0000 7fca927fc640 1 -- 172.21.15.33:0/2783910299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca98011950 con 0x7fca9c102200 2024-04-24T23:26:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.055+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c19fa20 con 0x7fca9c102200 2024-04-24T23:26:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.055+0000 7fca927fc640 1 -- 172.21.15.33:0/2783910299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98010ae0 con 0x7fca9c102200 2024-04-24T23:26:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.055+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c19ff90 con 0x7fca9c102200 2024-04-24T23:26:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.057+0000 7fca927fc640 1 -- 172.21.15.33:0/2783910299 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fca98011b00 con 0x7fca9c102200 2024-04-24T23:26:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 Processor -- start 2024-04-24T23:26:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 -- start start 2024-04-24T23:26:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 0x565286280a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286285b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652862861b0 0x565286288640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fca927fc640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca880a2c10 0x7fca880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652861e06b0 con 0x5652862861b0 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x565286188880 con 0x565286280650 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa6397740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5652861dc670 con 0x565286281040 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa1b46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 0x565286280a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652862861b0 0x565286288640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa1b46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 0x565286280a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40118/0 (socket says 172.21.15.33:40118) 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.058+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286285b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 -- 172.21.15.33:0/2707466473 learned_addr learned my addr 172.21.15.33:0/2707466473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fca927fc640 1 -- 172.21.15.33:0/2783910299 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca980cf990 con 0x7fca9c102200 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca880a2c10 0x7fca880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 -- 172.21.15.33:0/2707466473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 msgr2=0x565286285b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286285b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 -- 172.21.15.33:0/2707466473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652862861b0 msgr2=0x565286288640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652862861b0 0x565286288640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 -- 172.21.15.33:0/2707466473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565285fbe7d0 con 0x565286280650 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa0b44640 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652862861b0 0x565286288640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.059+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 0x565286280a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca94009fd0 tx=0x7fca94011ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72699bd19d44fca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.060+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca880a2c10 0x7fca880a50d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fca9401d000 tx=0x7fca9401f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.060+0000 7fca877fe640 1 -- 172.21.15.33:0/2707466473 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9400fcb0 con 0x565286280650 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.060+0000 7fca877fe640 1 -- 172.21.15.33:0/2707466473 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca94022070 con 0x565286280650 2024-04-24T23:26:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.060+0000 7fca877fe640 1 -- 172.21.15.33:0/2707466473 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94027b90 con 0x565286280650 2024-04-24T23:26:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 -- 172.21.15.33:0/2707466473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 msgr2=0x565286280a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 0x565286280a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca94009fd0 tx=0x7fca94011ad0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 -- 172.21.15.33:0/2707466473 shutdown_connections 2024-04-24T23:26:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652862861b0 0x565286288640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286285b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2707466473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286280650 0x565286280a70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 -- 172.21.15.33:0/2707466473 >> 172.21.15.33:0/2707466473 conn(0x56528627bcc0 msgr2=0x56528627c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 -- 172.21.15.33:0/2707466473 shutdown_connections 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 -- 172.21.15.33:0/2707466473 wait complete. 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 -- start start 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.061+0000 7fcaa6397740 1 Processor -- start 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x5652862787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286278d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286279d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565286289100 con 0x565286280650 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56528619bf50 con 0x5652862861b0 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565286289280 con 0x565286281040 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286278d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa6397740 10 client.0 _ll_register_callbacks cb 0x5652861d7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286278d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47032/0 (socket says 172.21.15.33:47032) 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 -- 172.21.15.33:0/851690446 learned_addr learned my addr 172.21.15.33:0/851690446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x5652862787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286279d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 -- 172.21.15.33:0/851690446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 msgr2=0x565286279d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286279d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 -- 172.21.15.33:0/851690446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 msgr2=0x5652862787d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x5652862787d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa0b44640 1 -- 172.21.15.33:0/851690446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565285ef43f0 con 0x565286281040 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286279d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.062+0000 7fcaa1345640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x5652862787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa0b44640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286278d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca8c031440 tx=0x7fca8c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b773e53f55d5e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fca84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fca84ff9640 1 -- 172.21.15.33:0/851690446 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c035040 con 0x565286281040 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fca84ff9640 1 -- 172.21.15.33:0/851690446 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca8c004950 con 0x565286281040 2024-04-24T23:26:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca94022070 con 0x565286281040 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fca84ff9640 1 -- 172.21.15.33:0/851690446 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca8c04f050 con 0x565286281040 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 10 client.27919 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565285eb40f0 con 0x565286281040 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fca84ff9640 1 -- 172.21.15.33:0/851690446 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca94022070 con 0x565286281040 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fca84ff9640 1 -- 172.21.15.33:0/851690446 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fca8c044020 con 0x565286281040 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 10 client.27919 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 10 client.27919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 20 client.27919 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56528629ad00 0x56528629d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 4 client.27919 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.063+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652861edbf0 con 0x56528629ad00 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.064+0000 7fcaa1345640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56528629ad00 0x56528629d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.064+0000 7fcaa1345640 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56528629ad00 0x56528629d180 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18210d7963736c06 server_cookie=ea39b0e33a350d02 in_seq=0 out_seq=0 2024-04-24T23:26:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.064+0000 7fca84ff9640 10 client.27919 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fca84ff9640 1 -- 172.21.15.33:0/851690446 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652861edbf0 con 0x56528629ad00 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fca84ff9640 10 client.27919 handle_command_reply: tid=0 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 client.27919 shutdown 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 msgr2=0x565286278d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286278d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca8c031440 tx=0x7fca8c005270 comp rx=0 tx=0).stop 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 shutdown_connections 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56528629ad00 0x56528629d180 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286279d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286281040 0x565286278d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 --2- 172.21.15.33:0/851690446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x5652862787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fca84ff9640 0 client.27919 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 >> 172.21.15.33:0/851690446 conn(0x56528627bcc0 msgr2=0x56528627cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 shutdown_connections 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 1 -- 172.21.15.33:0/851690446 wait complete. 2024-04-24T23:26:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.065+0000 7fcaa6397740 20 client.27919 trim_cache size 0 max 16384 2024-04-24T23:26:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 Processor -- start 2024-04-24T23:26:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 -- start start 2024-04-24T23:26:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 0x565286282da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286281040 0x5652862832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286276260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1aab20 con 0x565286281040 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1a8390 con 0x5652862861b0 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa6397740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1a61e0 con 0x565286280650 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286281040 0x5652862832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 0x565286282da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.085+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 0x565286282da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47044/0 (socket says 172.21.15.33:47044) 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286281040 0x5652862832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37946/0 (socket says 172.21.15.33:37946) 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 -- 172.21.15.33:0/2752616708 learned_addr learned my addr 172.21.15.33:0/2752616708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286276260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 -- 172.21.15.33:0/2752616708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 msgr2=0x565286276260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286276260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 -- 172.21.15.33:0/2752616708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286281040 msgr2=0x5652862832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286281040 0x5652862832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 -- 172.21.15.33:0/2752616708 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652862797c0 con 0x565286280650 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fcaa1345640 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 0x565286282da0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca980cd100 tx=0x7fca9806ade0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed70f105edff836d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fca86ffd640 1 -- 172.21.15.33:0/2752616708 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980a0210 con 0x565286280650 2024-04-24T23:26:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fca86ffd640 1 -- 172.21.15.33:0/2752616708 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca980a4ad0 con 0x565286280650 2024-04-24T23:26:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.086+0000 7fca86ffd640 1 -- 172.21.15.33:0/2752616708 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98066db0 con 0x565286280650 2024-04-24T23:26:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 -- 172.21.15.33:0/2752616708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 msgr2=0x565286282da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 0x565286282da0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca980cd100 tx=0x7fca9806ade0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 -- 172.21.15.33:0/2752616708 shutdown_connections 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652862861b0 0x565286276260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286281040 0x5652862832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 --2- 172.21.15.33:0/2752616708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565286280650 0x565286282da0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.087+0000 7fcaa6397740 1 -- 172.21.15.33:0/2752616708 >> 172.21.15.33:0/2752616708 conn(0x56528627bcc0 msgr2=0x5652862787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 -- 172.21.15.33:0/2752616708 shutdown_connections 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 -- 172.21.15.33:0/2752616708 wait complete. 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 -- start start 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 Processor -- start 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x565286294de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286281040 0x565286295320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652862861b0 0x565286296330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1a6850 con 0x565286280650 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1a7cc0 con 0x565286281040 2024-04-24T23:26:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.088+0000 7fcaa6397740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1a6f20 con 0x5652862861b0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa6397740 10 client.0 _ll_register_callbacks cb 0x7fca8c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x565286294de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa0b44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286281040 0x565286295320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652862861b0 0x565286296330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x565286294de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37952/0 (socket says 172.21.15.33:37952) 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1345640 1 -- 172.21.15.33:0/711191890 learned_addr learned my addr 172.21.15.33:0/711191890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 -- 172.21.15.33:0/711191890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286281040 msgr2=0x565286295320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286281040 0x565286295320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 -- 172.21.15.33:0/711191890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 msgr2=0x565286294de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x565286294de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 -- 172.21.15.33:0/711191890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565286296920 con 0x5652862861b0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1345640 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x565286294de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fcaa1b46640 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652862861b0 0x565286296330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca94011fd0 tx=0x7fca94005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8753060f1a1bc3b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.089+0000 7fca84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fca84ff9640 1 -- 172.21.15.33:0/711191890 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94035060 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fca84ff9640 1 -- 172.21.15.33:0/711191890 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca940055d0 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca980a4ad0 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fca84ff9640 1 -- 172.21.15.33:0/711191890 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca9403c050 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 10 client.27925 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565286297ca0 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fca84ff9640 1 -- 172.21.15.33:0/711191890 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca980a4ad0 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fca84ff9640 1 -- 172.21.15.33:0/711191890 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fca94003bc0 con 0x5652862861b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 10 client.27925 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 10 client.27925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 20 client.27925 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5652863018b0 0x565286303cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 4 client.27925 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652861e7f40 con 0x5652863018b0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.090+0000 7fcaa1345640 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5652863018b0 0x565286303cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa1345640 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5652863018b0 0x565286303cd0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d089ef385d4c763b server_cookie=a4f20a84895b8e2f in_seq=0 out_seq=0 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fca84ff9640 10 client.27925 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fca84ff9640 1 -- 172.21.15.33:0/711191890 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652861e7f40 con 0x5652863018b0 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fca84ff9640 10 client.27925 handle_command_reply: tid=0 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 client.27925 shutdown 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652862861b0 msgr2=0x565286296330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652862861b0 0x565286296330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca94011fd0 tx=0x7fca94005050 comp rx=0 tx=0).stop 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 shutdown_connections 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5652863018b0 0x565286303cd0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652862861b0 0x565286296330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fca84ff9640 0 client.27925 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565286281040 0x565286295320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 --2- 172.21.15.33:0/711191890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565286280650 0x565286294de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 >> 172.21.15.33:0/711191890 conn(0x56528627bcc0 msgr2=0x565286278790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 shutdown_connections 2024-04-24T23:26:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 1 -- 172.21.15.33:0/711191890 wait complete. 2024-04-24T23:26:43.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.091+0000 7fcaa6397740 20 client.27925 trim_cache size 0 max 16384 2024-04-24T23:26:43.097 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:26:43.097 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:26:43.097 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:26:43.097 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca880a2c10 msgr2=0x7fca880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca880a2c10 0x7fca880a50d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fca9401d000 tx=0x7fca9401f000 comp rx=0 tx=0).stop 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 msgr2=0x7fca9c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 0x7fca9c197b10 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fca9800eab0 tx=0x7fca9800ef70 comp rx=0 tx=0).stop 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 shutdown_connections 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fca880a2c10 0x7fca880a50d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca9c107460 0x7fca9c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c104b30 0x7fca9c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 --2- 172.21.15.33:0/2783910299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9c102200 0x7fca9c197b10 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 >> 172.21.15.33:0/2783910299 conn(0x7fca9c0fbbf0 msgr2=0x7fca9c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 shutdown_connections 2024-04-24T23:26:43.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.092+0000 7fcaa35a9640 1 -- 172.21.15.33:0/2783910299 wait complete. 2024-04-24T23:26:43.106 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:43.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 -- 172.21.15.33:0/3315388547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01074c0 msgr2=0x7f57e01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 --2- 172.21.15.33:0/3315388547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01074c0 0x7f57e01098b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009ca0 tx=0x7f57d802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 -- 172.21.15.33:0/3315388547 shutdown_connections 2024-04-24T23:26:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 --2- 172.21.15.33:0/3315388547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01074c0 0x7f57e01098b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 --2- 172.21.15.33:0/3315388547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57e0104b90 0x7f57e0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 --2- 172.21.15.33:0/3315388547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57e0102260 0x7f57e0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 -- 172.21.15.33:0/3315388547 >> 172.21.15.33:0/3315388547 conn(0x7f57e00fbbf0 msgr2=0x7f57e00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 -- 172.21.15.33:0/3315388547 shutdown_connections 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.231+0000 7f57e85ce640 1 -- 172.21.15.33:0/3315388547 wait complete. 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 Processor -- start 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 -- start start 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57e0102260 0x7f57e0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 0x7f57e0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57e01074c0 0x7f57e019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010d420 con 0x7f57e0104b90 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010d2a0 con 0x7f57e0102260 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e85ce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010d5a0 con 0x7f57e01074c0 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 0x7f57e0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 0x7f57e0197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37964/0 (socket says 172.21.15.33:37964) 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e636a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57e0102260 0x7f57e0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 -- 172.21.15.33:0/4275300854 learned_addr learned my addr 172.21.15.33:0/4275300854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e6b6b640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57e01074c0 0x7f57e019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 -- 172.21.15.33:0/4275300854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57e01074c0 msgr2=0x7f57e019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57e01074c0 0x7f57e019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 -- 172.21.15.33:0/4275300854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57e0102260 msgr2=0x7f57e0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57e0102260 0x7f57e0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.232+0000 7f57e5b69640 1 -- 172.21.15.33:0/4275300854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e019f6b0 con 0x7f57e0104b90 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.233+0000 7f57e5b69640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 0x7f57e0197fd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9f0 tx=0x7f57dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.233+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00ccc0 con 0x7f57e0104b90 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.233+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57dc005500 con 0x7f57e0104b90 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.233+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e019f9a0 con 0x7f57e0104b90 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.233+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc010690 con 0x7f57e0104b90 2024-04-24T23:26:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.233+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e019fef0 con 0x7f57e0104b90 2024-04-24T23:26:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.234+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e00fd6e0 con 0x7f57e0104b90 2024-04-24T23:26:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.235+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f57dc0046c0 con 0x7f57e0104b90 2024-04-24T23:26:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.236+0000 7f57d77fe640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f57cc0a2c10 0x7f57cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.236+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f57dc0cf8c0 con 0x7f57e0104b90 2024-04-24T23:26:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.237+0000 7f57e636a640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f57cc0a2c10 0x7f57cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.237+0000 7f57e636a640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f57cc0a2c10 0x7f57cc0a50d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f57d00095f0 tx=0x7f57d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.241+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f57dc053740 con 0x7f57e0104b90 2024-04-24T23:26:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.544+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57e002d380 con 0x7f57e0104b90 2024-04-24T23:26:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.545+0000 7f57d77fe640 1 -- 172.21.15.33:0/4275300854 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11863 (secure 0 0 0) 0x7f57dc09c490 con 0x7f57e0104b90 2024-04-24T23:26:43.546 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:26:43.546 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:26:01.133927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:26:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f57cc0a2c10 msgr2=0x7f57cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f57cc0a2c10 0x7f57cc0a50d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f57d00095f0 tx=0x7f57d0005f90 comp rx=0 tx=0).stop 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 msgr2=0x7f57e0197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 0x7f57e0197fd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9f0 tx=0x7f57dc00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 shutdown_connections 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f57cc0a2c10 0x7f57cc0a50d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.547+0000 7f57e85ce640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57e01074c0 0x7f57e019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.548+0000 7f57e85ce640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e0104b90 0x7f57e0197fd0 secure :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9f0 tx=0x7f57dc00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.548+0000 7f57e85ce640 1 --2- 172.21.15.33:0/4275300854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57e0102260 0x7f57e0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.548+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 >> 172.21.15.33:0/4275300854 conn(0x7f57e00fbbf0 msgr2=0x7f57e0102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.548+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 shutdown_connections 2024-04-24T23:26:43.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.548+0000 7f57e85ce640 1 -- 172.21.15.33:0/4275300854 wait complete. 2024-04-24T23:26:43.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:26:43.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:43 smithi033 ceph-mon[29201]: pgmap v792: 129 pgs: 129 active+clean; 481 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 479 op/s 2024-04-24T23:26:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:43 smithi038 ceph-mon[32450]: pgmap v792: 129 pgs: 129 active+clean; 481 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 479 op/s 2024-04-24T23:26:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:43 smithi117 ceph-mon[32529]: pgmap v792: 129 pgs: 129 active+clean; 481 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 479 op/s 2024-04-24T23:26:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.782+0000 7fb66dd5a640 1 -- 172.21.15.33:0/1704287210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6681007b0 msgr2=0x7fb668100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.782+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/1704287210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6681007b0 0x7fb668100c30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb660009ca0 tx=0x7fb66002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.784+0000 7fb66dd5a640 1 -- 172.21.15.33:0/1704287210 shutdown_connections 2024-04-24T23:26:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.784+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/1704287210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 0x7fb668105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.784+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/1704287210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6681007b0 0x7fb668100c30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.784+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/1704287210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680ffa50 0x7fb6680ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.784+0000 7fb66dd5a640 1 -- 172.21.15.33:0/1704287210 >> 172.21.15.33:0/1704287210 conn(0x7fb6680fb380 msgr2=0x7fb6680fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.787+0000 7fb66dd5a640 1 -- 172.21.15.33:0/1704287210 shutdown_connections 2024-04-24T23:26:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.789+0000 7fb66dd5a640 1 -- 172.21.15.33:0/1704287210 wait complete. 2024-04-24T23:26:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.790+0000 7fb66dd5a640 1 Processor -- start 2024-04-24T23:26:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 -- start start 2024-04-24T23:26:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6680ffa50 0x7fb66810bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6681007b0 0x7fb66810c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 0x7fb66806f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb668113b70 con 0x7fb668101170 2024-04-24T23:26:43.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681139f0 con 0x7fb6681007b0 2024-04-24T23:26:43.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.794+0000 7fb66dd5a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb668113cf0 con 0x7fb6680ffa50 2024-04-24T23:26:43.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.797+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6681007b0 0x7fb66810c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.797+0000 7fb6677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6680ffa50 0x7fb66810bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.797+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6681007b0 0x7fb66810c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40198/0 (socket says 172.21.15.33:40198) 2024-04-24T23:26:43.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.797+0000 7fb667fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 0x7fb66806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.797+0000 7fb666ffd640 1 -- 172.21.15.33:0/3965220638 learned_addr learned my addr 172.21.15.33:0/3965220638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.799+0000 7fb667fff640 1 -- 172.21.15.33:0/3965220638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6680ffa50 msgr2=0x7fb66810bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.799+0000 7fb667fff640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6680ffa50 0x7fb66810bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.799+0000 7fb667fff640 1 -- 172.21.15.33:0/3965220638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6681007b0 msgr2=0x7fb66810c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.799+0000 7fb667fff640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6681007b0 0x7fb66810c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.799+0000 7fb667fff640 1 -- 172.21.15.33:0/3965220638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66806f6c0 con 0x7fb668101170 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.799+0000 7fb667fff640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 0x7fb66806f180 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00b410 tx=0x7fb65c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.800+0000 7fb664ff9640 1 -- 172.21.15.33:0/3965220638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c015020 con 0x7fb668101170 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.800+0000 7fb664ff9640 1 -- 172.21.15.33:0/3965220638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb65c011d10 con 0x7fb668101170 2024-04-24T23:26:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.800+0000 7fb664ff9640 1 -- 172.21.15.33:0/3965220638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00f2e0 con 0x7fb668101170 2024-04-24T23:26:43.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.803+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66806f9b0 con 0x7fb668101170 2024-04-24T23:26:43.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.803+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66806fe80 con 0x7fb668101170 2024-04-24T23:26:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.808+0000 7fb670a64740 1 Processor -- start 2024-04-24T23:26:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.810+0000 7fb664ff9640 1 -- 172.21.15.33:0/3965220638 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb65c018020 con 0x7fb668101170 2024-04-24T23:26:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 -- start start 2024-04-24T23:26:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec97a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec97f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec982220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563cec8de670 con 0x563cec97ac20 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563cec886970 con 0x563cec97fd90 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.812+0000 7fb670a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563cec8da630 con 0x563cec97a230 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.814+0000 7fb664ff9640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6500a2d90 0x7fb6500a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.814+0000 7fb667fff640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6500a2d90 0x7fb6500a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.814+0000 7fb664ff9640 1 -- 172.21.15.33:0/3965220638 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb65c0cea90 con 0x7fb668101170 2024-04-24T23:26:43.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.815+0000 7fb667fff640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6500a2d90 0x7fb6500a5250 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0959a0 tx=0x7fb65c0c5a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.818+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec97a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.818+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec97a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47104/0 (socket says 172.21.15.33:47104) 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.818+0000 7fb666ffd640 1 -- 172.21.15.33:0/841256214 learned_addr learned my addr 172.21.15.33:0/841256214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.818+0000 7fb666ffd640 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec982220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.818+0000 7fb6677fe640 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec97f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.818+0000 7fb666ffd640 1 -- 172.21.15.33:0/841256214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 msgr2=0x563cec982220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.819+0000 7fb666ffd640 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec982220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.819+0000 7fb666ffd640 1 -- 172.21.15.33:0/841256214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 msgr2=0x563cec97f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.819+0000 7fb666ffd640 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec97f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.819+0000 7fb666ffd640 1 -- 172.21.15.33:0/841256214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cec8aa400 con 0x563cec97a230 2024-04-24T23:26:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.819+0000 7fb666ffd640 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec97a650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fe0 tx=0x7fb660031a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc38b2ecebf6d45c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.820+0000 7fb64dffb640 1 -- 172.21.15.33:0/841256214 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660003b70 con 0x563cec97a230 2024-04-24T23:26:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.820+0000 7fb64dffb640 1 -- 172.21.15.33:0/841256214 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb660036040 con 0x563cec97a230 2024-04-24T23:26:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 -- 172.21.15.33:0/841256214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 msgr2=0x563cec97a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec97a650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fe0 tx=0x7fb660031a00 comp rx=0 tx=0).stop 2024-04-24T23:26:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 -- 172.21.15.33:0/841256214 shutdown_connections 2024-04-24T23:26:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec982220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec97f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 --2- 172.21.15.33:0/841256214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec97a650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.821+0000 7fb670a64740 1 -- 172.21.15.33:0/841256214 >> 172.21.15.33:0/841256214 conn(0x563cec9758a0 msgr2=0x563cec975ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.828+0000 7fb670a64740 1 -- 172.21.15.33:0/841256214 shutdown_connections 2024-04-24T23:26:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.833+0000 7fb670a64740 1 -- 172.21.15.33:0/841256214 wait complete. 2024-04-24T23:26:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.833+0000 7fb670a64740 1 -- start start 2024-04-24T23:26:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.833+0000 7fb670a64740 1 Processor -- start 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec984380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec9848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec98b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563cec982d30 con 0x563cec97a230 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563cec89a050 con 0x563cec97ac20 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563cec982eb0 con 0x563cec97fd90 2024-04-24T23:26:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.835+0000 7fb670a64740 10 client.0 _ll_register_callbacks cb 0x563cec5f1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:43.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.838+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec98b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.838+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec984380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.838+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec984380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38002/0 (socket says 172.21.15.33:38002) 2024-04-24T23:26:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.838+0000 7fb666ffd640 1 -- 172.21.15.33:0/3341126788 learned_addr learned my addr 172.21.15.33:0/3341126788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.841+0000 7fb6677fe640 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec9848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.842+0000 7fb666ffd640 1 -- 172.21.15.33:0/3341126788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 msgr2=0x563cec98b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.842+0000 7fb666ffd640 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec98b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.842+0000 7fb666ffd640 1 -- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 msgr2=0x563cec9848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.842+0000 7fb666ffd640 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec9848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.842+0000 7fb666ffd640 1 -- 172.21.15.33:0/3341126788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cec86bdc0 con 0x563cec97a230 2024-04-24T23:26:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.850+0000 7fb666ffd640 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec984380 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb660006010 tx=0x7fb660000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fb76ea9c8ab989 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.850+0000 7fb646ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.855+0000 7fb646ffd640 1 -- 172.21.15.33:0/3341126788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660003d80 con 0x563cec97a230 2024-04-24T23:26:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.855+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb660036040 con 0x563cec97a230 2024-04-24T23:26:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.855+0000 7fb646ffd640 1 -- 172.21.15.33:0/3341126788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb660003fe0 con 0x563cec97a230 2024-04-24T23:26:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.858+0000 7fb646ffd640 1 -- 172.21.15.33:0/3341126788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6600041e0 con 0x563cec97a230 2024-04-24T23:26:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.859+0000 7fb670a64740 10 client.18492 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.859+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cec5f23f0 con 0x563cec97a230 2024-04-24T23:26:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.860+0000 7fb646ffd640 1 -- 172.21.15.33:0/3341126788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb660036040 con 0x563cec97a230 2024-04-24T23:26:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb646ffd640 1 -- 172.21.15.33:0/3341126788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb66003ac10 con 0x563cec97a230 2024-04-24T23:26:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb670a64740 10 client.18492 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb670a64740 10 client.18492 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:26:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb670a64740 20 client.18492 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb670a64740 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9900e0 0x563cec992520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb670a64740 4 client.18492 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.861+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cec8ebbb0 con 0x563cec9900e0 2024-04-24T23:26:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.862+0000 7fb6677fe640 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9900e0 0x563cec992520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.865+0000 7fb6677fe640 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9900e0 0x563cec992520 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be2ea5545818d832 server_cookie=4a00789021d5dc21 in_seq=0 out_seq=0 2024-04-24T23:26:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.865+0000 7fb646ffd640 10 client.18492 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.869+0000 7fb646ffd640 1 -- 172.21.15.33:0/3341126788 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563cec8ebbb0 con 0x563cec9900e0 2024-04-24T23:26:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.869+0000 7fb646ffd640 10 client.18492 handle_command_reply: tid=0 2024-04-24T23:26:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.869+0000 7fb670a64740 1 client.18492 shutdown 2024-04-24T23:26:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.873+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 msgr2=0x563cec984380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.873+0000 7fb670a64740 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec984380 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb660006010 tx=0x7fb660000f40 comp rx=0 tx=0).stop 2024-04-24T23:26:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.874+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 shutdown_connections 2024-04-24T23:26:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.874+0000 7fb670a64740 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9900e0 0x563cec992520 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.874+0000 7fb670a64740 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec98b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.874+0000 7fb670a64740 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec9848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.874+0000 7fb670a64740 1 --2- 172.21.15.33:0/3341126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec984380 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.874+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 >> 172.21.15.33:0/3341126788 conn(0x563cec9758a0 msgr2=0x563cec974b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 22612 29.05 MB/sec warmup 48 sec latency 327.658 ms 2024-04-24T23:26:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.878+0000 7fb646ffd640 0 client.18492 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.878+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 shutdown_connections 2024-04-24T23:26:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.879+0000 7fb670a64740 1 -- 172.21.15.33:0/3341126788 wait complete. 2024-04-24T23:26:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.880+0000 7fb670a64740 20 client.18492 trim_cache size 0 max 16384 2024-04-24T23:26:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 Processor -- start 2024-04-24T23:26:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 -- start start 2024-04-24T23:26:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec984820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec98b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec97f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681c4190 con 0x563cec97ac20 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681c1a00 con 0x563cec97fd90 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.912+0000 7fb670a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681bf900 con 0x563cec97a230 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.913+0000 7fb6677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec98b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.913+0000 7fb6677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec98b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38008/0 (socket says 172.21.15.33:38008) 2024-04-24T23:26:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.913+0000 7fb6677fe640 1 -- 172.21.15.33:0/1829568504 learned_addr learned my addr 172.21.15.33:0/1829568504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.914+0000 7fb666ffd640 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec984820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.914+0000 7fb666ffd640 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec97f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.915+0000 7fb666ffd640 1 -- 172.21.15.33:0/1829568504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 msgr2=0x563cec97f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.915+0000 7fb666ffd640 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec97f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.915+0000 7fb666ffd640 1 -- 172.21.15.33:0/1829568504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 msgr2=0x563cec98b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.915+0000 7fb666ffd640 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec98b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.915+0000 7fb666ffd640 1 -- 172.21.15.33:0/1829568504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cec6bc7d0 con 0x563cec97a230 2024-04-24T23:26:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.918+0000 7fb666ffd640 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec984820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb660056040 tx=0x7fb6600099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1500c8a4f8e6c5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.919+0000 7fb64d7fa640 1 -- 172.21.15.33:0/1829568504 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66003db00 con 0x563cec97a230 2024-04-24T23:26:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.919+0000 7fb64d7fa640 1 -- 172.21.15.33:0/1829568504 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb660036070 con 0x563cec97a230 2024-04-24T23:26:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.920+0000 7fb64d7fa640 1 -- 172.21.15.33:0/1829568504 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600045e0 con 0x563cec97a230 2024-04-24T23:26:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.920+0000 7fb670a64740 1 -- 172.21.15.33:0/1829568504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 msgr2=0x563cec984820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.920+0000 7fb670a64740 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec984820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb660056040 tx=0x7fb6600099e0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.923+0000 7fb670a64740 1 -- 172.21.15.33:0/1829568504 shutdown_connections 2024-04-24T23:26:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.923+0000 7fb670a64740 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97fd90 0x563cec97f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.923+0000 7fb670a64740 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97ac20 0x563cec98b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.923+0000 7fb670a64740 1 --2- 172.21.15.33:0/1829568504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97a230 0x563cec984820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.923+0000 7fb670a64740 1 -- 172.21.15.33:0/1829568504 >> 172.21.15.33:0/1829568504 conn(0x563cec9758a0 msgr2=0x563cec9742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.926+0000 7fb670a64740 1 -- 172.21.15.33:0/1829568504 shutdown_connections 2024-04-24T23:26:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.927+0000 7fb670a64740 1 -- 172.21.15.33:0/1829568504 wait complete. 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.927+0000 7fb670a64740 1 -- start start 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 Processor -- start 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec98b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec988330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec988960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681bfec0 con 0x563cec97a230 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681c1330 con 0x563cec97ac20 2024-04-24T23:26:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681c0590 con 0x563cec97fd90 2024-04-24T23:26:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb670a64740 10 client.0 _ll_register_callbacks cb 0x7fb6600315d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.928+0000 7fb6677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec988330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.929+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec98b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.929+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec98b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38016/0 (socket says 172.21.15.33:38016) 2024-04-24T23:26:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.929+0000 7fb666ffd640 1 -- 172.21.15.33:0/2978026642 learned_addr learned my addr 172.21.15.33:0/2978026642 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.929+0000 7fb666ffd640 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec988960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.932+0000 7fb666ffd640 1 -- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 msgr2=0x563cec988330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.932+0000 7fb666ffd640 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec988330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.932+0000 7fb666ffd640 1 -- 172.21.15.33:0/2978026642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 msgr2=0x563cec98b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.932+0000 7fb666ffd640 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec98b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.932+0000 7fb666ffd640 1 -- 172.21.15.33:0/2978026642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cec9f4740 con 0x563cec97fd90 2024-04-24T23:26:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.932+0000 7fb666ffd640 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec988960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb66003d5e0 tx=0x7fb66002a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d3ddbf6dd898 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.936+0000 7fb646ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.937+0000 7fb646ffd640 1 -- 172.21.15.33:0/2978026642 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66003d610 con 0x563cec97fd90 2024-04-24T23:26:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.937+0000 7fb646ffd640 1 -- 172.21.15.33:0/2978026642 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb660004590 con 0x563cec97fd90 2024-04-24T23:26:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.937+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb660036070 con 0x563cec97fd90 2024-04-24T23:26:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.938+0000 7fb646ffd640 1 -- 172.21.15.33:0/2978026642 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb660004790 con 0x563cec97fd90 2024-04-24T23:26:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.938+0000 7fb670a64740 10 client.27949 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.938+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cec9f5a40 con 0x563cec97fd90 2024-04-24T23:26:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.939+0000 7fb646ffd640 1 -- 172.21.15.33:0/2978026642 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb660036070 con 0x563cec97fd90 2024-04-24T23:26:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.939+0000 7fb646ffd640 1 -- 172.21.15.33:0/2978026642 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb66004cb80 con 0x563cec97fd90 2024-04-24T23:26:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.939+0000 7fb670a64740 10 client.27949 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.939+0000 7fb670a64740 10 client.27949 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:26:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.939+0000 7fb670a64740 20 client.27949 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.939+0000 7fb670a64740 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9f62b0 0x563cec9f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.940+0000 7fb670a64740 4 client.27949 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:26:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.940+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cec8e5f00 con 0x563cec9f62b0 2024-04-24T23:26:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.940+0000 7fb6677fe640 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9f62b0 0x563cec9f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.944+0000 7fb6677fe640 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9f62b0 0x563cec9f86d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f92560c425565800 server_cookie=adcc96726e22d410 in_seq=0 out_seq=0 2024-04-24T23:26:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.944+0000 7fb646ffd640 10 client.27949 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.948+0000 7fb646ffd640 1 -- 172.21.15.33:0/2978026642 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cec8e5f00 con 0x563cec9f62b0 2024-04-24T23:26:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.948+0000 7fb646ffd640 10 client.27949 handle_command_reply: tid=0 2024-04-24T23:26:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.948+0000 7fb670a64740 1 client.27949 shutdown 2024-04-24T23:26:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.951+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 msgr2=0x563cec988960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.951+0000 7fb670a64740 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec988960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb66003d5e0 tx=0x7fb66002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 shutdown_connections 2024-04-24T23:26:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb670a64740 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563cec9f62b0 0x563cec9f86d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb670a64740 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cec97fd90 0x563cec988960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb670a64740 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563cec97ac20 0x563cec988330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb670a64740 1 --2- 172.21.15.33:0/2978026642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cec97a230 0x563cec98b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 >> 172.21.15.33:0/2978026642 conn(0x563cec9758a0 msgr2=0x563cec9742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.952+0000 7fb646ffd640 0 client.27949 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:26:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.953+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 shutdown_connections 2024-04-24T23:26:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.953+0000 7fb670a64740 1 -- 172.21.15.33:0/2978026642 wait complete. 2024-04-24T23:26:43.956 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:26:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.954+0000 7fb670a64740 20 client.27949 trim_cache size 0 max 16384 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6500a2d90 msgr2=0x7fb6500a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6500a2d90 0x7fb6500a5250 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0959a0 tx=0x7fb65c0c5a90 comp rx=0 tx=0).stop 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 msgr2=0x7fb66806f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 0x7fb66806f180 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00b410 tx=0x7fb65c00b8d0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 shutdown_connections 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6500a2d90 0x7fb6500a5250 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb668101170 0x7fb66806f180 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6681007b0 0x7fb66810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 --2- 172.21.15.33:0/3965220638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6680ffa50 0x7fb66810bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.956+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 >> 172.21.15.33:0/3965220638 conn(0x7fb6680fb380 msgr2=0x7fb6681019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.958+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 shutdown_connections 2024-04-24T23:26:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:43.958+0000 7fb66dd5a640 1 -- 172.21.15.33:0/3965220638 wait complete. 2024-04-24T23:26:43.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:26:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 -- 172.21.15.33:0/1669500464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81017b0 msgr2=0x7f52a8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1669500464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81017b0 0x7f52a8101bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5298009900 tx=0x7f529802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 -- 172.21.15.33:0/1669500464 shutdown_connections 2024-04-24T23:26:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1669500464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 0x7f52a8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1669500464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a81029a0 0x7f52a8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1669500464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81017b0 0x7f52a8101bd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 -- 172.21.15.33:0/1669500464 >> 172.21.15.33:0/1669500464 conn(0x7f52a80fcd20 msgr2=0x7f52a80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 -- 172.21.15.33:0/1669500464 shutdown_connections 2024-04-24T23:26:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.115+0000 7f52ae05d640 1 -- 172.21.15.33:0/1669500464 wait complete. 2024-04-24T23:26:44.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 Processor -- start 2024-04-24T23:26:44.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 -- start start 2024-04-24T23:26:44.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a81017b0 0x7f52a8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 0x7f52a8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 0x7f52a819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52a810d470 con 0x7f52a8103360 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f52a810d2f0 con 0x7f52a81029a0 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.116+0000 7f52ae05d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52a810d5f0 con 0x7f52a81017b0 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 0x7f52a8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 0x7f52a8198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40262/0 (socket says 172.21.15.33:40262) 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 -- 172.21.15.33:0/1441815456 learned_addr learned my addr 172.21.15.33:0/1441815456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 0x7f52a819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a77fe640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a81017b0 0x7f52a8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 -- 172.21.15.33:0/1441815456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a81017b0 msgr2=0x7f52a8197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a81017b0 0x7f52a8197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 -- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 msgr2=0x7f52a819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 0x7f52a819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a6ffd640 1 -- 172.21.15.33:0/1441815456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a819f730 con 0x7f52a81029a0 2024-04-24T23:26:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.117+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 0x7f52a819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.118+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 0x7f52a8198050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f529c00d770 tx=0x7f529c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.118+0000 7f52a4ff9640 1 -- 172.21.15.33:0/1441815456 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c017070 con 0x7f52a81029a0 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.118+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a819fa20 con 0x7f52a81029a0 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.118+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a819ff70 con 0x7f52a81029a0 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.118+0000 7f52a4ff9640 1 -- 172.21.15.33:0/1441815456 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c005500 con 0x7f52a81029a0 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.118+0000 7f52a4ff9640 1 -- 172.21.15.33:0/1441815456 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c004e00 con 0x7f52a81029a0 2024-04-24T23:26:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.121+0000 7f52a4ff9640 1 -- 172.21.15.33:0/1441815456 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f529c009c30 con 0x7f52a81029a0 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.121+0000 7f52b0d6e740 1 Processor -- start 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.121+0000 7f52b0d6e740 1 -- start start 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c5c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c61d90 0x556b73c64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a4ff9640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52900a2b90 0x7f52900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c5c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c5c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47152/0 (socket says 172.21.15.33:47152) 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a4ff9640 1 -- 172.21.15.33:0/1441815456 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f529c0ceca0 con 0x7f52a81029a0 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c61720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38034/0 (socket says 172.21.15.33:38034) 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a7fff640 1 -- 172.21.15.33:0/2697218828 learned_addr learned my addr 172.21.15.33:0/2697218828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b73bc0670 con 0x556b73c5cc20 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556b73b68970 con 0x556b73c61d90 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b73bbc630 con 0x556b73c5c230 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52900a2b90 0x7f52900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.122+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c61d90 0x556b73c64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 -- 172.21.15.33:0/2697218828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c61d90 msgr2=0x556b73c64220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c61d90 0x556b73c64220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 -- 172.21.15.33:0/2697218828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 msgr2=0x556b73c61720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c61720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 -- 172.21.15.33:0/2697218828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b73b8c400 con 0x556b73c5c230 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52900a2b90 0x7f52900a5050 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52a00099e0 tx=0x7f52a001e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f52a7fff640 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c5c650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f52a000f5c0 tx=0x7f52a000fa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be8216f5a361702 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f528dffb640 1 -- 172.21.15.33:0/2697218828 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a002b070 con 0x556b73c5c230 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f528dffb640 1 -- 172.21.15.33:0/2697218828 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a0003e50 con 0x556b73c5c230 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.123+0000 7f528dffb640 1 -- 172.21.15.33:0/2697218828 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0022070 con 0x556b73c5c230 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 msgr2=0x556b73c5c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c5c650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f52a000f5c0 tx=0x7f52a000fa80 comp rx=0 tx=0).stop 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 shutdown_connections 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c61d90 0x556b73c64220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c61720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2697218828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c5c650 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 >> 172.21.15.33:0/2697218828 conn(0x556b73c578a0 msgr2=0x556b73c57ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 shutdown_connections 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2697218828 wait complete. 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.124+0000 7f52b0d6e740 1 -- start start 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 Processor -- start 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 0x556b73c66270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c61d90 0x556b73c6d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b73c64ce0 con 0x556b73c5cc20 2024-04-24T23:26:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556b73b7c050 con 0x556b73c5c230 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b73c64e60 con 0x556b73c61d90 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52b0d6e740 10 client.0 _ll_register_callbacks cb 0x556b738d3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 0x556b73c66270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 0x556b73c66270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40280/0 (socket says 172.21.15.33:40280) 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52a77fe640 1 -- 172.21.15.33:0/107888000 learned_addr learned my addr 172.21.15.33:0/107888000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.125+0000 7f52a7fff640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c61d90 0x556b73c6d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a77fe640 1 -- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c61d90 msgr2=0x556b73c6d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a77fe640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c61d90 0x556b73c6d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a77fe640 1 -- 172.21.15.33:0/107888000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 msgr2=0x556b73c667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a77fe640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a77fe640 1 -- 172.21.15.33:0/107888000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b73b4ddc0 con 0x556b73c5c230 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52a77fe640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 0x556b73c66270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5298000c00 tx=0x7f5298004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11b48b40819d27a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f5286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f5286ffd640 1 -- 172.21.15.33:0/107888000 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298035040 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f5286ffd640 1 -- 172.21.15.33:0/107888000 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f529803f2f0 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a0003e50 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f5286ffd640 1 -- 172.21.15.33:0/107888000 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5298057050 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52b0d6e740 10 client.28016 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b738d43f0 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.126+0000 7f5286ffd640 1 -- 172.21.15.33:0/107888000 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a0003e50 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f5286ffd640 1 -- 172.21.15.33:0/107888000 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f5298004a70 con 0x556b73c5c230 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52b0d6e740 10 client.28016 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52b0d6e740 10 client.28016 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52b0d6e740 20 client.28016 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73c744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52b0d6e740 4 client.28016 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b73bcdbb0 con 0x556b73c72030 2024-04-24T23:26:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73c744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f5286ffd640 10 client.28016 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.127+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73c744b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0766d6e1c3eb496 server_cookie=35690073ab96379e in_seq=0 out_seq=0 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f5286ffd640 1 -- 172.21.15.33:0/107888000 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b73bcdbb0 con 0x556b73c72030 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f5286ffd640 10 client.28016 handle_command_reply: tid=0 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 client.28016 shutdown 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 msgr2=0x556b73c66270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 0x556b73c66270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5298000c00 tx=0x7f5298004440 comp rx=0 tx=0).stop 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 shutdown_connections 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73c744b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f5286ffd640 0 client.28016 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c61d90 0x556b73c6d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c5cc20 0x556b73c667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/107888000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5c230 0x556b73c66270 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 >> 172.21.15.33:0/107888000 conn(0x556b73c578a0 msgr2=0x556b73c56b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 shutdown_connections 2024-04-24T23:26:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.128+0000 7f52b0d6e740 1 -- 172.21.15.33:0/107888000 wait complete. 2024-04-24T23:26:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.129+0000 7f52b0d6e740 20 client.28016 trim_cache size 0 max 16384 2024-04-24T23:26:44.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.146+0000 7f52b0d6e740 1 Processor -- start 2024-04-24T23:26:44.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 -- start start 2024-04-24T23:26:44.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c78db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52940124f0 con 0x556b73c61d90 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f529400fd60 con 0x556b73c5cc20 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f529400dbb0 con 0x556b73c5c230 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c78db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c78db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38046/0 (socket says 172.21.15.33:38046) 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 -- 172.21.15.33:0/1051766850 learned_addr learned my addr 172.21.15.33:0/1051766850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 -- 172.21.15.33:0/1051766850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 msgr2=0x556b73c562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 -- 172.21.15.33:0/1051766850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 msgr2=0x556b73c6d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.147+0000 7f52a7fff640 1 -- 172.21.15.33:0/1051766850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7399e7d0 con 0x556b73c61d90 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.148+0000 7f52a7fff640 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c78db0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f52a0011fd0 tx=0x7f52a0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800224731947b117 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.148+0000 7f528d7fa640 1 -- 172.21.15.33:0/1051766850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0022070 con 0x556b73c61d90 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.148+0000 7f528d7fa640 1 -- 172.21.15.33:0/1051766850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a0003b70 con 0x556b73c61d90 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.148+0000 7f528d7fa640 1 -- 172.21.15.33:0/1051766850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a002b040 con 0x556b73c61d90 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 -- 172.21.15.33:0/1051766850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 msgr2=0x556b73c78db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c78db0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f52a0011fd0 tx=0x7f52a0009f00 comp rx=0 tx=0).stop 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 -- 172.21.15.33:0/1051766850 shutdown_connections 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c78db0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/1051766850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 -- 172.21.15.33:0/1051766850 >> 172.21.15.33:0/1051766850 conn(0x556b73c578a0 msgr2=0x556b73c5ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 -- 172.21.15.33:0/1051766850 shutdown_connections 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 -- 172.21.15.33:0/1051766850 wait complete. 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.149+0000 7f52b0d6e740 1 -- start start 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 Processor -- start 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f529400e220 con 0x556b73c61d90 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f529400f690 con 0x556b73c5cc20 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f529400e8f0 con 0x556b73c5c230 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52b0d6e740 10 client.0 _ll_register_callbacks cb 0x7f5298005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c6c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38060/0 (socket says 172.21.15.33:38060) 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 -- 172.21.15.33:0/2698628121 learned_addr learned my addr 172.21.15.33:0/2698628121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a6ffd640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a77fe640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 -- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 msgr2=0x556b73c6ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c6ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 -- 172.21.15.33:0/2698628121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 msgr2=0x556b73c6b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a7fff640 1 -- 172.21.15.33:0/2698628121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b73c6c7d0 con 0x556b73c61d90 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.150+0000 7f52a77fe640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c6ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f52a7fff640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c6c1e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f52a0029040 tx=0x7f52a00054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187b873649e6d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f5286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f5286ffd640 1 -- 172.21.15.33:0/2698628121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a002b070 con 0x556b73c61d90 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f5286ffd640 1 -- 172.21.15.33:0/2698628121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f52a0005950 con 0x556b73c61d90 2024-04-24T23:26:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a0003b70 con 0x556b73c61d90 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f5286ffd640 1 -- 172.21.15.33:0/2698628121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52a0046050 con 0x556b73c61d90 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f52b0d6e740 10 client.18516 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.151+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b73c6dad0 con 0x556b73c61d90 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f5286ffd640 1 -- 172.21.15.33:0/2698628121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a0003b70 con 0x556b73c61d90 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f5286ffd640 1 -- 172.21.15.33:0/2698628121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f52a002e070 con 0x556b73c61d90 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52b0d6e740 10 client.18516 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52b0d6e740 10 client.18516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52b0d6e740 20 client.18516 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73cdacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52b0d6e740 4 client.18516 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b73bc7f00 con 0x556b73c72030 2024-04-24T23:26:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52a77fe640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73cdacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f5286ffd640 10 client.18516 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.152+0000 7f52a77fe640 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73cdacb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50f94fd47bec2a95 server_cookie=2e4e63c1ffb5cee0 in_seq=0 out_seq=0 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f5286ffd640 1 -- 172.21.15.33:0/2698628121 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b73bc7f00 con 0x556b73c72030 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f5286ffd640 10 client.18516 handle_command_reply: tid=0 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 client.18516 shutdown 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 msgr2=0x556b73c6c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c6c1e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f52a0029040 tx=0x7f52a00054a0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 shutdown_connections 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556b73c72030 0x556b73cdacb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b73c61d90 0x556b73c6c1e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b73c5cc20 0x556b73c6b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f5286ffd640 0 client.18516 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:26:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 --2- 172.21.15.33:0/2698628121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b73c5c230 0x556b73c6ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 >> 172.21.15.33:0/2698628121 conn(0x556b73c578a0 msgr2=0x556b73c5d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 shutdown_connections 2024-04-24T23:26:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 1 -- 172.21.15.33:0/2698628121 wait complete. 2024-04-24T23:26:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.153+0000 7f52b0d6e740 20 client.18516 trim_cache size 0 max 16384 2024-04-24T23:26:44.159 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:26:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52900a2b90 msgr2=0x7f52900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52900a2b90 0x7f52900a5050 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52a00099e0 tx=0x7f52a001e000 comp rx=0 tx=0).stop 2024-04-24T23:26:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 msgr2=0x7f52a8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 0x7f52a8198050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f529c00d770 tx=0x7f529c00dc30 comp rx=0 tx=0).stop 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 shutdown_connections 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f52900a2b90 0x7f52900a5050 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52a8103360 0x7f52a819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52a81029a0 0x7f52a8198050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 --2- 172.21.15.33:0/1441815456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a81017b0 0x7f52a8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.154+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 >> 172.21.15.33:0/1441815456 conn(0x7f52a80fcd20 msgr2=0x7f52a8105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.155+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 shutdown_connections 2024-04-24T23:26:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.155+0000 7f52ae05d640 1 -- 172.21.15.33:0/1441815456 wait complete. 2024-04-24T23:26:44.170 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/3907948121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4100680 msgr2=0x7fe5b4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/3907948121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4100680 0x7fe5b4100aa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009900 tx=0x7fe5b002a410 comp rx=0 tx=0).stop 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/3907948121 shutdown_connections 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/3907948121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 0x7fe5b4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/3907948121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4101870 0x7fe5b4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/3907948121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4100680 0x7fe5b4100aa0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.296+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/3907948121 >> 172.21.15.33:0/3907948121 conn(0x7fe5b40fbbf0 msgr2=0x7fe5b40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/3907948121 shutdown_connections 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/3907948121 wait complete. 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 Processor -- start 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 -- start start 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4100680 0x7fe5b4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 0x7fe5b4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 0x7fe5b419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b410c350 con 0x7fe5b4101870 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b410c1d0 con 0x7fe5b4100680 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5bb9ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b410c4d0 con 0x7fe5b4102230 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.297+0000 7fe5b8f89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 0x7fe5b4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 0x7fe5b4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38082/0 (socket says 172.21.15.33:38082) 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4100680 0x7fe5b4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 -- 172.21.15.33:0/1718937058 learned_addr learned my addr 172.21.15.33:0/1718937058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 0x7fe5b419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 -- 172.21.15.33:0/1718937058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 msgr2=0x7fe5b419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 0x7fe5b419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 -- 172.21.15.33:0/1718937058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4100680 msgr2=0x7fe5b4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4100680 0x7fe5b4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 -- 172.21.15.33:0/1718937058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b419f730 con 0x7fe5b4101870 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b978a640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4100680 0x7fe5b4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 0x7fe5b419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 0x7fe5b4198050 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400e9f0 tx=0x7fe5a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5aa7fc640 1 -- 172.21.15.33:0/1718937058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400cd80 con 0x7fe5b4101870 2024-04-24T23:26:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b419fa20 con 0x7fe5b4101870 2024-04-24T23:26:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5aa7fc640 1 -- 172.21.15.33:0/1718937058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a4010910 con 0x7fe5b4101870 2024-04-24T23:26:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5aa7fc640 1 -- 172.21.15.33:0/1718937058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40188f0 con 0x7fe5b4101870 2024-04-24T23:26:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.298+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b419ff70 con 0x7fe5b4101870 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.300+0000 7fe5aa7fc640 1 -- 172.21.15.33:0/1718937058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe5a4010ac0 con 0x7fe5b4101870 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 Processor -- start 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 -- start start 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3ab030 0x56553d3afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56553d30a670 con 0x56553d3aa640 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56553d2b2970 con 0x56553d3b01a0 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5be7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56553d306630 con 0x56553d3ab030 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5b9f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5b9f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3aaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38094/0 (socket says 172.21.15.33:38094) 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5b9f8b640 1 -- 172.21.15.33:0/322420463 learned_addr learned my addr 172.21.15.33:0/322420463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5aa7fc640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5a00a2b90 0x7fe5a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.301+0000 7fe5b978a640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3ab030 0x56553d3afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5a00a2b90 0x7fe5a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 -- 172.21.15.33:0/322420463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3ab030 msgr2=0x56553d3afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3ab030 0x56553d3afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 -- 172.21.15.33:0/322420463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 msgr2=0x56553d3b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 -- 172.21.15.33:0/322420463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553d2d6400 con 0x56553d3aa640 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b978a640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3ab030 0x56553d3afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5aa7fc640 1 -- 172.21.15.33:0/1718937058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5a40cf7e0 con 0x7fe5b4101870 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3aaa60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009fd0 tx=0x7fe5ac013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb23b75f5f2cb3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5a00a2b90 0x7fe5a00a5050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac013d30 tx=0x7fe5ac01f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe59f7fe640 1 -- 172.21.15.33:0/322420463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00f3c0 con 0x56553d3aa640 2024-04-24T23:26:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe59f7fe640 1 -- 172.21.15.33:0/322420463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ac022020 con 0x56553d3aa640 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.302+0000 7fe59f7fe640 1 -- 172.21.15.33:0/322420463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac010a90 con 0x56553d3aa640 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 -- 172.21.15.33:0/322420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 msgr2=0x56553d3aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3aaa60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009fd0 tx=0x7fe5ac013830 comp rx=0 tx=0).stop 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 -- 172.21.15.33:0/322420463 shutdown_connections 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3ab030 0x56553d3afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 --2- 172.21.15.33:0/322420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3aaa60 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 -- 172.21.15.33:0/322420463 >> 172.21.15.33:0/322420463 conn(0x56553d3a18a0 msgr2=0x56553d3a1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 -- 172.21.15.33:0/322420463 shutdown_connections 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.303+0000 7fe5be7da740 1 -- 172.21.15.33:0/322420463 wait complete. 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 -- start start 2024-04-24T23:26:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 Processor -- start 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3b01a0 0x56553d3bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56553d3b30f0 con 0x56553d3b01a0 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56553d2c6050 con 0x56553d3ab030 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56553d3b3270 con 0x56553d3aa640 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47194/0 (socket says 172.21.15.33:47194) 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5b978a640 1 -- 172.21.15.33:0/106000182 learned_addr learned my addr 172.21.15.33:0/106000182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.304+0000 7fe5be7da740 10 client.0 _ll_register_callbacks cb 0x56553d01dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3b01a0 0x56553d3bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b978a640 1 -- 172.21.15.33:0/106000182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 msgr2=0x56553d3b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b978a640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b978a640 1 -- 172.21.15.33:0/106000182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3b01a0 msgr2=0x56553d3bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b978a640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3b01a0 0x56553d3bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b978a640 1 -- 172.21.15.33:0/106000182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553d297dc0 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3b01a0 0x56553d3bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5b978a640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b4730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0031440 tx=0x7fe5b0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d62c25127fc5044 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe59cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe59cff9640 1 -- 172.21.15.33:0/106000182 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0035040 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe59cff9640 1 -- 172.21.15.33:0/106000182 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5b0004950 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.305+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ac022020 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe59cff9640 1 -- 172.21.15.33:0/106000182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5b0004b50 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe5be7da740 10 client.27979 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56553d01e3f0 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe59cff9640 1 -- 172.21.15.33:0/106000182 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ac022020 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe59cff9640 1 -- 172.21.15.33:0/106000182 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fe5b0044020 con 0x56553d3aa640 2024-04-24T23:26:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe5be7da740 10 client.27979 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe5be7da740 10 client.27979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.306+0000 7fe5be7da740 20 client.27979 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.307+0000 7fe5be7da740 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d3c05a0 0x56553d3c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.307+0000 7fe5be7da740 4 client.27979 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.307+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56553d317bb0 con 0x56553d3c05a0 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.307+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d3c05a0 0x56553d3c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.307+0000 7fe59cff9640 10 client.27979 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.308+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d3c05a0 0x56553d3c2a40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc19a222bd643960 server_cookie=24745c6294e9c4aa in_seq=0 out_seq=0 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.308+0000 7fe59cff9640 1 -- 172.21.15.33:0/106000182 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56553d317bb0 con 0x56553d3c05a0 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.308+0000 7fe59cff9640 10 client.27979 handle_command_reply: tid=0 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.309+0000 7fe5be7da740 1 client.27979 shutdown 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.309+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 msgr2=0x56553d3b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.309+0000 7fe5be7da740 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b4730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0031440 tx=0x7fe5b0005270 comp rx=0 tx=0).stop 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.309+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 shutdown_connections 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.309+0000 7fe5be7da740 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d3c05a0 0x56553d3c2a40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3b01a0 0x56553d3bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 1 --2- 172.21.15.33:0/106000182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b4730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 >> 172.21.15.33:0/106000182 conn(0x56553d3a18a0 msgr2=0x56553d3a0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe59cff9640 0 client.27979 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 shutdown_connections 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 1 -- 172.21.15.33:0/106000182 wait complete. 2024-04-24T23:26:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.310+0000 7fe5be7da740 20 client.27979 trim_cache size 0 max 16384 2024-04-24T23:26:44.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 Processor -- start 2024-04-24T23:26:44.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 -- start start 2024-04-24T23:26:44.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3b01a0 0x56553d3b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41aaad0 con 0x56553d3aa640 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41a8340 con 0x56553d3ab030 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.326+0000 7fe5be7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41a6190 con 0x56553d3b01a0 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38114/0 (socket says 172.21.15.33:38114) 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b8f89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 -- 172.21.15.33:0/298951691 learned_addr learned my addr 172.21.15.33:0/298951691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3b01a0 0x56553d3b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 -- 172.21.15.33:0/298951691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3b01a0 msgr2=0x56553d3b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3b01a0 0x56553d3b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 -- 172.21.15.33:0/298951691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 msgr2=0x56553d3bbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3bbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 -- 172.21.15.33:0/298951691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553d0e87d0 con 0x56553d3aa640 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.327+0000 7fe5b978a640 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3a02b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0006010 tx=0x7fe5b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efb40224673fbf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.329+0000 7fe59effd640 1 -- 172.21.15.33:0/298951691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0035030 con 0x56553d3aa640 2024-04-24T23:26:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.329+0000 7fe59effd640 1 -- 172.21.15.33:0/298951691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0004cf0 con 0x56553d3aa640 2024-04-24T23:26:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.329+0000 7fe59effd640 1 -- 172.21.15.33:0/298951691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0000e60 con 0x56553d3aa640 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 -- 172.21.15.33:0/298951691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 msgr2=0x56553d3a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3a02b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0006010 tx=0x7fe5b002a410 comp rx=0 tx=0).stop 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 -- 172.21.15.33:0/298951691 shutdown_connections 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3b01a0 0x56553d3b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3ab030 0x56553d3bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 --2- 172.21.15.33:0/298951691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3aa640 0x56553d3a02b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 -- 172.21.15.33:0/298951691 >> 172.21.15.33:0/298951691 conn(0x56553d3a18a0 msgr2=0x56553d3a4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 -- 172.21.15.33:0/298951691 shutdown_connections 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 -- 172.21.15.33:0/298951691 wait complete. 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 -- start start 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.330+0000 7fe5be7da740 1 Processor -- start 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3ab030 0x56553d3b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41a6800 con 0x56553d3ab030 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41a7c70 con 0x56553d3b01a0 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41a6ed0 con 0x56553d3aa640 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5be7da740 10 client.0 _ll_register_callbacks cb 0x7fe5b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47200/0 (socket says 172.21.15.33:47200) 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 -- 172.21.15.33:0/2765087869 learned_addr learned my addr 172.21.15.33:0/2765087869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3ab030 0x56553d3b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b9f8b640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 -- 172.21.15.33:0/2765087869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 msgr2=0x56553d3ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 -- 172.21.15.33:0/2765087869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3ab030 msgr2=0x56553d3b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3ab030 0x56553d3b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 -- 172.21.15.33:0/2765087869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553d3baa90 con 0x56553d3aa640 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe5b978a640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b8fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00036f0 tx=0x7fe5b0004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d7f83428736672 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.331+0000 7fe59cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.333+0000 7fe59cff9640 1 -- 172.21.15.33:0/2765087869 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0035030 con 0x56553d3aa640 2024-04-24T23:26:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.333+0000 7fe59cff9640 1 -- 172.21.15.33:0/2765087869 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5b00318c0 con 0x56553d3aa640 2024-04-24T23:26:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0004cf0 con 0x56553d3aa640 2024-04-24T23:26:44.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe59cff9640 1 -- 172.21.15.33:0/2765087869 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5b0004ea0 con 0x56553d3aa640 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe5be7da740 10 client.27985 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56553d3bbd90 con 0x56553d3aa640 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe59cff9640 1 -- 172.21.15.33:0/2765087869 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0004cf0 con 0x56553d3aa640 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe59cff9640 1 -- 172.21.15.33:0/2765087869 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fe5b0044040 con 0x56553d3aa640 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.335+0000 7fe5be7da740 10 client.27985 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5be7da740 10 client.27985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5be7da740 20 client.27985 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5be7da740 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d426ec0 0x56553d4292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5be7da740 4 client.27985 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56553d311f00 con 0x56553d426ec0 2024-04-24T23:26:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d426ec0 0x56553d4292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe5b8f89640 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d426ec0 0x56553d4292b0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9358b29afb0b547d server_cookie=cb613f320d763b8f in_seq=0 out_seq=0 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.336+0000 7fe59cff9640 10 client.27985 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe59cff9640 1 -- 172.21.15.33:0/2765087869 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56553d311f00 con 0x56553d426ec0 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe59cff9640 10 client.27985 handle_command_reply: tid=0 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe5be7da740 1 client.27985 shutdown 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 msgr2=0x56553d3b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe5be7da740 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b8fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00036f0 tx=0x7fe5b0004970 comp rx=0 tx=0).stop 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 shutdown_connections 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe5be7da740 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56553d426ec0 0x56553d4292b0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.337+0000 7fe5be7da740 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56553d3b01a0 0x56553d3ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe59cff9640 0 client.27985 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe5be7da740 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56553d3ab030 0x56553d3b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe5be7da740 1 --2- 172.21.15.33:0/2765087869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56553d3aa640 0x56553d3b8fc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 >> 172.21.15.33:0/2765087869 conn(0x56553d3a18a0 msgr2=0x56553d3ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 shutdown_connections 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe5be7da740 1 -- 172.21.15.33:0/2765087869 wait complete. 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.338+0000 7fe5be7da740 20 client.27985 trim_cache size 0 max 16384 2024-04-24T23:26:44.340 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5a00a2b90 msgr2=0x7fe5a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5a00a2b90 0x7fe5a00a5050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac013d30 tx=0x7fe5ac01f000 comp rx=0 tx=0).stop 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 msgr2=0x7fe5b4198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 0x7fe5b4198050 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400e9f0 tx=0x7fe5a400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 shutdown_connections 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5a00a2b90 0x7fe5a00a5050 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b4102230 0x7fe5b419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5b4101870 0x7fe5b4198050 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 --2- 172.21.15.33:0/1718937058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4100680 0x7fe5b4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 >> 172.21.15.33:0/1718937058 conn(0x7fe5b40fbbf0 msgr2=0x7fe5b40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 shutdown_connections 2024-04-24T23:26:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:44.339+0000 7fe5bb9ee640 1 -- 172.21.15.33:0/1718937058 wait complete. 2024-04-24T23:26:44.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:44 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4275300854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:44.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:44 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4275300854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:44.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:44 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4275300854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:44.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 23039 28.78 MB/sec warmup 49 sec latency 543.992 ms 2024-04-24T23:26:45.353 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:26:45.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:45.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:26:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 -- 172.21.15.33:0/3558472868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf01006a0 msgr2=0x7f6cf0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/3558472868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf01006a0 0x7f6cf0100ac0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009900 tx=0x7f6cec02a410 comp rx=0 tx=0).stop 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 -- 172.21.15.33:0/3558472868 shutdown_connections 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/3558472868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf0102250 0x7f6cf0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/3558472868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0101890 0x7f6cf0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/3558472868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf01006a0 0x7f6cf0100ac0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 -- 172.21.15.33:0/3558472868 >> 172.21.15.33:0/3558472868 conn(0x7f6cf00fbc30 msgr2=0x7f6cf00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 -- 172.21.15.33:0/3558472868 shutdown_connections 2024-04-24T23:26:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.490+0000 7f6cf7f16640 1 -- 172.21.15.33:0/3558472868 wait complete. 2024-04-24T23:26:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 Processor -- start 2024-04-24T23:26:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 -- start start 2024-04-24T23:26:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01006a0 0x7f6cf01047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 0x7f6cf0102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0102250 0x7f6cf0103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010c320 con 0x7f6cf01006a0 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010c1a0 con 0x7f6cf0102250 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf7f16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010c4a0 con 0x7f6cf0101890 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.491+0000 7f6cf54b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 0x7f6cf0102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf64b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0102250 0x7f6cf0103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01006a0 0x7f6cf01047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 0x7f6cf0102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:47216/0 (socket says 172.21.15.33:47216) 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1317146454 learned_addr learned my addr 172.21.15.33:0/1317146454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf64b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0102250 0x7f6cf0103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40354/0 (socket says 172.21.15.33:40354) 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1317146454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0102250 msgr2=0x7f6cf0103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0102250 0x7f6cf0103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01006a0 msgr2=0x7f6cf01047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01006a0 0x7f6cf01047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1317146454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf01038b0 con 0x7f6cf0101890 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01006a0 0x7f6cf01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.492+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 0x7f6cf0102e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000e9f0 tx=0x7f6ce000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.493+0000 7f6ce6ffd640 1 -- 172.21.15.33:0/1317146454 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000cd80 con 0x7f6cf0101890 2024-04-24T23:26:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.493+0000 7f6ce6ffd640 1 -- 172.21.15.33:0/1317146454 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce0005500 con 0x7f6cf0101890 2024-04-24T23:26:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.493+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf0103ba0 con 0x7f6cf0101890 2024-04-24T23:26:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.493+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf01a65e0 con 0x7f6cf0101890 2024-04-24T23:26:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.493+0000 7f6ce6ffd640 1 -- 172.21.15.33:0/1317146454 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0010690 con 0x7f6cf0101890 2024-04-24T23:26:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.496+0000 7f6ce6ffd640 1 -- 172.21.15.33:0/1317146454 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6ce00046c0 con 0x7f6cf0101890 2024-04-24T23:26:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 Processor -- start 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 -- start start 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52efd90 0x5607b52f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52f5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f58f0 0x5607b52f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607b52532c0 con 0x5607b52f0780 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607b51fb910 con 0x5607b52f58f0 2024-04-24T23:26:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.497+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607b524f280 con 0x5607b52efd90 2024-04-24T23:26:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf64b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52efd90 0x5607b52f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38160/0 (socket says 172.21.15.33:38160) 2024-04-24T23:26:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/1346161054 learned_addr learned my addr 172.21.15.33:0/1346161054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f58f0 0x5607b52f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6ce6ffd640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2c10 0x7f6cdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2c10 0x7f6cdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6ce6ffd640 1 -- 172.21.15.33:0/1317146454 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ce00cf800 con 0x7f6cf0101890 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1346161054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52efd90 msgr2=0x5607b52f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52efd90 0x5607b52f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1346161054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 msgr2=0x5607b52f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.498+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cf54b1640 1 -- 172.21.15.33:0/1346161054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b520cfb0 con 0x5607b52f58f0 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52efd90 0x5607b52f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f58f0 0x5607b52f7d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00ccfc0 tx=0x7f6ce009bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ae803964104021 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2c10 0x7f6cdc0a50d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5607b52f0fe0 tx=0x7f6ce8004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cdbfff640 1 -- 172.21.15.33:0/1346161054 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce009c3d0 con 0x5607b52f58f0 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cdbfff640 1 -- 172.21.15.33:0/1346161054 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce009bcd0 con 0x5607b52f58f0 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.499+0000 7f6cdbfff640 1 -- 172.21.15.33:0/1346161054 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0098340 con 0x5607b52f58f0 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1346161054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f58f0 msgr2=0x5607b52f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f58f0 0x5607b52f7d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00ccfc0 tx=0x7f6ce009bbd0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1346161054 shutdown_connections 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f58f0 0x5607b52f7d80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1346161054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52efd90 0x5607b52f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1346161054 >> 172.21.15.33:0/1346161054 conn(0x5607b52b7950 msgr2=0x5607b52eb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1346161054 shutdown_connections 2024-04-24T23:26:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.500+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1346161054 wait complete. 2024-04-24T23:26:45.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.501+0000 7f6cfac1a740 1 -- start start 2024-04-24T23:26:45.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.501+0000 7f6cfac1a740 1 Processor -- start 2024-04-24T23:26:45.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.501+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52efd90 0x5607b52fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b53016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607b52f87e0 con 0x5607b52f0780 2024-04-24T23:26:45.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607b520ee90 con 0x5607b52efd90 2024-04-24T23:26:45.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607b52f8960 con 0x5607b52f58f0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cfac1a740 10 client.0 _ll_register_callbacks cb 0x5607b524a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52efd90 0x5607b52fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52efd90 0x5607b52fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40366/0 (socket says 172.21.15.33:40366) 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/1336630297 learned_addr learned my addr 172.21.15.33:0/1336630297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b53016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf64b3640 1 -- 172.21.15.33:0/1336630297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52efd90 msgr2=0x5607b52fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52efd90 0x5607b52fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf64b3640 1 -- 172.21.15.33:0/1336630297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 msgr2=0x5607b52fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.502+0000 7f6cf64b3640 1 -- 172.21.15.33:0/1336630297 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b502f7e0 con 0x5607b52f58f0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.503+0000 7f6cf64b3640 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b53016b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8005fd0 tx=0x7f6ce8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9b4b207c561a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.503+0000 7f6cd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.503+0000 7f6cd97fa640 1 -- 172.21.15.33:0/1336630297 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8025070 con 0x5607b52f58f0 2024-04-24T23:26:45.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.503+0000 7f6cd97fa640 1 -- 172.21.15.33:0/1336630297 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ce80047d0 con 0x5607b52f58f0 2024-04-24T23:26:45.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.503+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ce009bcd0 con 0x5607b52f58f0 2024-04-24T23:26:45.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.504+0000 7f6cd97fa640 1 -- 172.21.15.33:0/1336630297 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ce802f050 con 0x5607b52f58f0 2024-04-24T23:26:45.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.504+0000 7f6cfac1a740 10 client.27997 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:45.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.504+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607b508ff70 con 0x5607b52f58f0 2024-04-24T23:26:45.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.504+0000 7f6cd97fa640 1 -- 172.21.15.33:0/1336630297 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce8021950 con 0x5607b52f58f0 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cd97fa640 1 -- 172.21.15.33:0/1336630297 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6ce801f040 con 0x5607b52f58f0 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cfac1a740 10 client.27997 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cfac1a740 10 client.27997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cfac1a740 20 client.27997 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b5305d00 0x5607b5308170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cfac1a740 4 client.27997 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:45.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ce009bcd0 con 0x5607b5305d00 2024-04-24T23:26:45.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.505+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b5305d00 0x5607b5308170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.506+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b5305d00 0x5607b5308170 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dba2acf5bb9f538 server_cookie=a5e550b04c961a3b in_seq=0 out_seq=0 2024-04-24T23:26:45.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.506+0000 7f6cd97fa640 10 client.27997 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.508+0000 7f6cd97fa640 1 -- 172.21.15.33:0/1336630297 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6ce009bcd0 con 0x5607b5305d00 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.508+0000 7f6cd97fa640 10 client.27997 handle_command_reply: tid=0 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.508+0000 7f6cfac1a740 1 client.27997 shutdown 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 msgr2=0x5607b53016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b53016b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8005fd0 tx=0x7f6ce8004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 shutdown_connections 2024-04-24T23:26:45.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b5305d00 0x5607b5308170 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b53016b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52f0780 0x5607b52fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cd97fa640 0 client.27997 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1336630297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52efd90 0x5607b52fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 >> 172.21.15.33:0/1336630297 conn(0x5607b52b7950 msgr2=0x5607b52ea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 shutdown_connections 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1336630297 wait complete. 2024-04-24T23:26:45.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.509+0000 7f6cfac1a740 20 client.27997 trim_cache size 0 max 16384 2024-04-24T23:26:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 Processor -- start 2024-04-24T23:26:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 -- start start 2024-04-24T23:26:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b52ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b52ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b5301670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b34e0 con 0x5607b52efd90 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.532+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b0d50 con 0x5607b52f0780 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01aeba0 con 0x5607b52f58f0 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b52ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b52ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38172/0 (socket says 172.21.15.33:38172) 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/1378369776 learned_addr learned my addr 172.21.15.33:0/1378369776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf64b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b5301670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b52ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/1378369776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 msgr2=0x5607b5301670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b5301670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/1378369776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 msgr2=0x5607b52ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b52ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/1378369776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b4f27170 con 0x5607b52efd90 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b52ea7d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6cec02a930 tx=0x7f6cec0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b840247e953a1240 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cdb7fe640 1 -- 172.21.15.33:0/1378369776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec04a070 con 0x5607b52efd90 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cdb7fe640 1 -- 172.21.15.33:0/1378369776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cec045dc0 con 0x5607b52efd90 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.533+0000 7f6cdb7fe640 1 -- 172.21.15.33:0/1378369776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec035040 con 0x5607b52efd90 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1378369776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 msgr2=0x5607b52ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b52ea7d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6cec02a930 tx=0x7f6cec0058b0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1378369776 shutdown_connections 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b5301670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b52ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/1378369776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b52ea7d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1378369776 >> 172.21.15.33:0/1378369776 conn(0x5607b52b7950 msgr2=0x5607b52ea0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1378369776 shutdown_connections 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.534+0000 7f6cfac1a740 1 -- 172.21.15.33:0/1378369776 wait complete. 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 -- start start 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 Processor -- start 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b5300f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b5301470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b52ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01af210 con 0x5607b52efd90 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b0680 con 0x5607b52f0780 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01af8e0 con 0x5607b52f58f0 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cfac1a740 10 client.0 _ll_register_callbacks cb 0x7f6ce80136b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b5300f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cf54b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b5301470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cf54b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b5301470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:40380/0 (socket says 172.21.15.33:40380) 2024-04-24T23:26:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cf5cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b5300f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38186/0 (socket says 172.21.15.33:38186) 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cf54b1640 1 -- 172.21.15.33:0/3492701849 learned_addr learned my addr 172.21.15.33:0/3492701849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.535+0000 7f6cf64b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b52ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 msgr2=0x5607b52ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b52ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/3492701849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 msgr2=0x5607b5301470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b5301470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cf5cb2640 1 -- 172.21.15.33:0/3492701849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b52ffb80 con 0x5607b52efd90 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cf5cb2640 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b5300f30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009230 tx=0x7f6cec032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbe19a4c013ed64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cd97fa640 1 -- 172.21.15.33:0/3492701849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec035050 con 0x5607b52efd90 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cec045dc0 con 0x5607b52efd90 2024-04-24T23:26:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.536+0000 7f6cd97fa640 1 -- 172.21.15.33:0/3492701849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6cec0453d0 con 0x5607b52efd90 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cd97fa640 1 -- 172.21.15.33:0/3492701849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6cec04a070 con 0x5607b52efd90 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 10 client.18549 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607b536ccc0 con 0x5607b52efd90 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cd97fa640 1 -- 172.21.15.33:0/3492701849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cec045dc0 con 0x5607b52efd90 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cd97fa640 1 -- 172.21.15.33:0/3492701849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6cec055030 con 0x5607b52efd90 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 10 client.18549 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 20 client.18549 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b536d6a0 0x5607b536fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 4 client.18549 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5607b52275a0 con 0x5607b536d6a0 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.537+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b536d6a0 0x5607b536fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.538+0000 7f6cd97fa640 10 client.18549 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.538+0000 7f6cf54b1640 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b536d6a0 0x5607b536fac0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dede433c28ba7eb server_cookie=dfa062f3b702ec68 in_seq=0 out_seq=0 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cd97fa640 1 -- 172.21.15.33:0/3492701849 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5607b52275a0 con 0x5607b536d6a0 2024-04-24T23:26:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cd97fa640 10 client.18549 handle_command_reply: tid=0 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 client.18549 shutdown 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 msgr2=0x5607b5300f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b5300f30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009230 tx=0x7f6cec032ef0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 shutdown_connections 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5607b536d6a0 0x5607b536fac0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cd97fa640 0 client.18549 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607b52f58f0 0x5607b52ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607b52f0780 0x5607b5301470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 --2- 172.21.15.33:0/3492701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607b52efd90 0x5607b5300f30 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.539+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 >> 172.21.15.33:0/3492701849 conn(0x5607b52b7950 msgr2=0x5607b52ea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.540+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 shutdown_connections 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.540+0000 7f6cfac1a740 1 -- 172.21.15.33:0/3492701849 wait complete. 2024-04-24T23:26:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.540+0000 7f6cfac1a740 20 client.18549 trim_cache size 0 max 16384 2024-04-24T23:26:45.551 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:26:45.551 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:26:45.551 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0b45d440-053b-4633-b5b7-e3346cf71099", 2024-04-24T23:26:45.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:26:45.551 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2c10 msgr2=0x7f6cdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2c10 0x7f6cdc0a50d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5607b52f0fe0 tx=0x7f6ce8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 msgr2=0x7f6cf0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 0x7f6cf0102e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000e9f0 tx=0x7f6ce000edf0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 shutdown_connections 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2c10 0x7f6cdc0a50d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0102250 0x7f6cf0103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0101890 0x7f6cf0102e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 --2- 172.21.15.33:0/1317146454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01006a0 0x7f6cf01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 >> 172.21.15.33:0/1317146454 conn(0x7f6cf00fbc30 msgr2=0x7f6cf0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 shutdown_connections 2024-04-24T23:26:45.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.541+0000 7f6cf7f16640 1 -- 172.21.15.33:0/1317146454 wait complete. 2024-04-24T23:26:45.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:26:45.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:26:45.582 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:45 smithi033 ceph-mon[29201]: pgmap v793: 129 pgs: 129 active+clean; 482 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 388 op/s 2024-04-24T23:26:45.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:45 smithi038 ceph-mon[32450]: pgmap v793: 129 pgs: 129 active+clean; 482 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 388 op/s 2024-04-24T23:26:45.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:45 smithi117 ceph-mon[32529]: pgmap v793: 129 pgs: 129 active+clean; 482 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 388 op/s 2024-04-24T23:26:45.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 -- 172.21.15.33:0/465711398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330100680 msgr2=0x7f3330100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 --2- 172.21.15.33:0/465711398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330100680 0x7f3330100aa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f332002a410 comp rx=0 tx=0).stop 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 -- 172.21.15.33:0/465711398 shutdown_connections 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 --2- 172.21.15.33:0/465711398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330102230 0x7f3330108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 --2- 172.21.15.33:0/465711398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330101870 0x7f3330101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 --2- 172.21.15.33:0/465711398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330100680 0x7f3330100aa0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 -- 172.21.15.33:0/465711398 >> 172.21.15.33:0/465711398 conn(0x7f33300fbbf0 msgr2=0x7f33300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 -- 172.21.15.33:0/465711398 shutdown_connections 2024-04-24T23:26:45.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.683+0000 7f333489a640 1 -- 172.21.15.33:0/465711398 wait complete. 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 Processor -- start 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 -- start start 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 0x7f3330199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330101870 0x7f333019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330102230 0x7f33301a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f333010b2d0 con 0x7f3330102230 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f333010b150 con 0x7f3330100680 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.684+0000 7f333489a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f333010b450 con 0x7f3330101870 2024-04-24T23:26:45.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f3327fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330101870 0x7f333019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 0x7f3330199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330102230 0x7f33301a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 0x7f3330199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37508/0 (socket says 172.21.15.33:37508) 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f3327fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330101870 0x7f333019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34404/0 (socket says 172.21.15.33:34404) 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332e59c640 1 -- 172.21.15.33:0/338578170 learned_addr learned my addr 172.21.15.33:0/338578170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 -- 172.21.15.33:0/338578170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330101870 msgr2=0x7f333019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330101870 0x7f333019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 -- 172.21.15.33:0/338578170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 msgr2=0x7f3330199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 0x7f3330199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 -- 172.21.15.33:0/338578170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33301a1980 con 0x7f3330102230 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332e59c640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 0x7f3330199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f332ed9d640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330102230 0x7f33301a1310 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f332800be20 tx=0x7f332800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f33277fe640 1 -- 172.21.15.33:0/338578170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328019070 con 0x7f3330102230 2024-04-24T23:26:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301a1c70 con 0x7f3330102230 2024-04-24T23:26:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f33277fe640 1 -- 172.21.15.33:0/338578170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3328014920 con 0x7f3330102230 2024-04-24T23:26:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f33277fe640 1 -- 172.21.15.33:0/338578170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328012920 con 0x7f3330102230 2024-04-24T23:26:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.685+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301a21c0 con 0x7f3330102230 2024-04-24T23:26:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.688+0000 7f33277fe640 1 -- 172.21.15.33:0/338578170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f332801c070 con 0x7f3330102230 2024-04-24T23:26:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.689+0000 7f33375a1740 1 Processor -- start 2024-04-24T23:26:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.689+0000 7f33375a1740 1 -- start start 2024-04-24T23:26:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.689+0000 7f33277fe640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f331c0a2c60 0x7f331c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f3327fff640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f331c0a2c60 0x7f331c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33375a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbb8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33277fe640 1 -- 172.21.15.33:0/338578170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33280ceca0 con 0x7f3330102230 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33375a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbbd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33375a1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33375a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1abb1c6b0 con 0x55e1abbb8c30 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33375a1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1abac4880 con 0x55e1abbbdda0 2024-04-24T23:26:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbb8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f3327fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbbd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbb8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34406/0 (socket says 172.21.15.33:34406) 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 -- 172.21.15.33:0/3578308069 learned_addr learned my addr 172.21.15.33:0/3578308069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f33375a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e1abb18670 con 0x55e1abbb8240 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f3327fff640 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbbd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38728/0 (socket says 172.21.15.33:38728) 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332ed9d640 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 -- 172.21.15.33:0/3578308069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 msgr2=0x55e1abbc0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 -- 172.21.15.33:0/3578308069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 msgr2=0x55e1abbbd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbbd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f332e59c640 1 -- 172.21.15.33:0/3578308069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ab8fa7d0 con 0x55e1abbb8240 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.690+0000 7f3327fff640 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbbd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.691+0000 7f3327fff640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f331c0a2c60 0x7f331c0a5120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f333019b270 tx=0x7f3318013460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.691+0000 7f332e59c640 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbb8660 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f332002aa00 tx=0x7f3320004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8840f1891e6143e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.691+0000 7f3324ff9640 1 -- 172.21.15.33:0/3578308069 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320035040 con 0x55e1abbb8240 2024-04-24T23:26:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.691+0000 7f3324ff9640 1 -- 172.21.15.33:0/3578308069 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33200053f0 con 0x55e1abbb8240 2024-04-24T23:26:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.691+0000 7f3324ff9640 1 -- 172.21.15.33:0/3578308069 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320031c10 con 0x55e1abbb8240 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 -- 172.21.15.33:0/3578308069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 msgr2=0x55e1abbb8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbb8660 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f332002aa00 tx=0x7f3320004610 comp rx=0 tx=0).stop 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 -- 172.21.15.33:0/3578308069 shutdown_connections 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbbd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- 172.21.15.33:0/3578308069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbb8660 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 -- 172.21.15.33:0/3578308069 >> 172.21.15.33:0/3578308069 conn(0x55e1abbb38b0 msgr2=0x55e1abbb3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 -- 172.21.15.33:0/3578308069 shutdown_connections 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 -- 172.21.15.33:0/3578308069 wait complete. 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 -- start start 2024-04-24T23:26:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 Processor -- start 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbc2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbc2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.692+0000 7f33375a1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f33375a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1abbc0cf0 con 0x55e1abbb8c30 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f33375a1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e1abad7f50 con 0x55e1abbbdda0 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f33375a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e1abbc0e70 con 0x55e1abbb8240 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f332e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbc2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbc2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f332e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbc2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34420/0 (socket says 172.21.15.33:34420) 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f332e59c640 1 -- 172.21.15.33:0/576955639 learned_addr learned my addr 172.21.15.33:0/576955639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f33375a1740 10 client.0 _ll_register_callbacks cb 0x55e1abb13000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 -- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 msgr2=0x55e1abbc2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbc2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 -- 172.21.15.33:0/576955639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 msgr2=0x55e1abbc9be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc9be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 -- 172.21.15.33:0/576955639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ab8303f0 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33200053f0 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3327fff640 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbc2ba0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x55e1abbb96a0 tx=0x7f3318004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a9dee2fe294bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3315ffb640 10 client.18561 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.693+0000 7f3315ffb640 1 -- 172.21.15.33:0/576955639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318005020 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f3315ffb640 1 -- 172.21.15.33:0/576955639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33180051c0 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f3315ffb640 1 -- 172.21.15.33:0/576955639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3318005400 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 10 client.18561 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e1ab7f00f0 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f3315ffb640 1 -- 172.21.15.33:0/576955639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3318020050 con 0x55e1abbb8c30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 10 client.18561 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:45.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 10 client.18561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 20 client.18561 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abbd0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 4 client.18561 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1abb29bf0 con 0x55e1abbce300 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.694+0000 7f332e59c640 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abbd0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.695+0000 7f3315ffb640 10 client.18561 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.695+0000 7f332e59c640 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abbd0780 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1b8b046e19da42 server_cookie=1a6b60949576452d in_seq=0 out_seq=0 2024-04-24T23:26:45.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.696+0000 7f3315ffb640 1 -- 172.21.15.33:0/576955639 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1abb29bf0 con 0x55e1abbce300 2024-04-24T23:26:45.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.696+0000 7f3315ffb640 10 client.18561 handle_command_reply: tid=0 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.696+0000 7f33375a1740 1 client.18561 shutdown 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 msgr2=0x55e1abbc2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbc2ba0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x55e1abbb96a0 tx=0x7f3318004730 comp rx=0 tx=0).stop 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 shutdown_connections 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abbd0780 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbbdda0 0x55e1abbc9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f3315ffb640 0 client.18561 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8c30 0x55e1abbc2ba0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 --2- 172.21.15.33:0/576955639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8240 0x55e1abbc2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 >> 172.21.15.33:0/576955639 conn(0x55e1abbb38b0 msgr2=0x55e1abbb2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 shutdown_connections 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 1 -- 172.21.15.33:0/576955639 wait complete. 2024-04-24T23:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.697+0000 7f33375a1740 20 client.18561 trim_cache size 0 max 16384 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.718+0000 7f33375a1740 1 Processor -- start 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 -- start start 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 0x55e1abbc1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8c30 0x55e1abbc9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbbdda0 0x55e1abbb6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33301aefa0 con 0x55e1abbb8240 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33301ac810 con 0x55e1abbb8c30 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f33375a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33301aa660 con 0x55e1abbbdda0 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 0x55e1abbc1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 0x55e1abbc1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38750/0 (socket says 172.21.15.33:38750) 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f3327fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8c30 0x55e1abbc9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 -- 172.21.15.33:0/1499777628 learned_addr learned my addr 172.21.15.33:0/1499777628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332ed9d640 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbbdda0 0x55e1abbb6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 -- 172.21.15.33:0/1499777628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbbdda0 msgr2=0x55e1abbb6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbbdda0 0x55e1abbb6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 -- 172.21.15.33:0/1499777628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8c30 msgr2=0x55e1abbc9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8c30 0x55e1abbc9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332e59c640 1 -- 172.21.15.33:0/1499777628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1abba5090 con 0x55e1abbb8240 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f3327fff640 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8c30 0x55e1abbc9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.719+0000 7f332ed9d640 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbbdda0 0x55e1abbb6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.720+0000 7f332e59c640 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 0x55e1abbc1c80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3320006010 tx=0x7f33200316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f457e9a1a1b46b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.720+0000 7f3317fff640 1 -- 172.21.15.33:0/1499777628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320035040 con 0x55e1abbb8240 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.720+0000 7f3317fff640 1 -- 172.21.15.33:0/1499777628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3320031e00 con 0x55e1abbb8240 2024-04-24T23:26:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.720+0000 7f3317fff640 1 -- 172.21.15.33:0/1499777628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200035f0 con 0x55e1abbb8240 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.720+0000 7f33375a1740 1 -- 172.21.15.33:0/1499777628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 msgr2=0x55e1abbc1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.720+0000 7f33375a1740 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 0x55e1abbc1c80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3320006010 tx=0x7f33200316b0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 -- 172.21.15.33:0/1499777628 shutdown_connections 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbbdda0 0x55e1abbb6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8c30 0x55e1abbc9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 --2- 172.21.15.33:0/1499777628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbb8240 0x55e1abbc1c80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 -- 172.21.15.33:0/1499777628 >> 172.21.15.33:0/1499777628 conn(0x55e1abbb38b0 msgr2=0x55e1abbba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 -- 172.21.15.33:0/1499777628 shutdown_connections 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 -- 172.21.15.33:0/1499777628 wait complete. 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 -- start start 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.721+0000 7f33375a1740 1 Processor -- start 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 0x55e1abbc7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8c30 0x55e1abbc78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbbdda0 0x55e1abbc88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33301aacd0 con 0x55e1abbbdda0 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33301ac140 con 0x55e1abbb8240 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33301ab3a0 con 0x55e1abbb8c30 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f33375a1740 10 client.0 _ll_register_callbacks cb 0x7f3318009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 0x55e1abbc7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 0x55e1abbc7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:37554/0 (socket says 172.21.15.33:37554) 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332e59c640 1 -- 172.21.15.33:0/3509127113 learned_addr learned my addr 172.21.15.33:0/3509127113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f3327fff640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8c30 0x55e1abbc78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332ed9d640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbbdda0 0x55e1abbc88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332e59c640 1 -- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8c30 msgr2=0x55e1abbc78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332e59c640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8c30 0x55e1abbc78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.722+0000 7f332e59c640 1 -- 172.21.15.33:0/3509127113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbbdda0 msgr2=0x55e1abbc88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f332e59c640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbbdda0 0x55e1abbc88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f332e59c640 1 -- 172.21.15.33:0/3509127113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1abbc8ec0 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f332ed9d640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbbdda0 0x55e1abbc88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f332e59c640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 0x55e1abbc7380 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3320009850 tx=0x7f332002ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f949eb88e2e90272 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f3315ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f3315ffb640 1 -- 172.21.15.33:0/3509127113 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332005d070 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f3315ffb640 1 -- 172.21.15.33:0/3509127113 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3320004900 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3320031e00 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.723+0000 7f3315ffb640 1 -- 172.21.15.33:0/3509127113 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3320035040 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 10 client.28073 fetch_fsmap learned FSMap version 30 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e1abc34980 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f3315ffb640 1 -- 172.21.15.33:0/3509127113 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3320031e00 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f3315ffb640 1 -- 172.21.15.33:0/3509127113 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3320003d20 con 0x55e1abbb8240 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 10 client.28073 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:26:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 10 client.28073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 20 client.28073 populate_metadata read hostname 'smithi033' 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abc36e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 4 client.28073 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1abbc2450 con 0x55e1abbce300 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.724+0000 7f3327fff640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abc36e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.725+0000 7f3327fff640 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abc36e90 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf71ae0a82eec1bf server_cookie=8626bedc9ae0567 in_seq=0 out_seq=0 2024-04-24T23:26:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.725+0000 7f3315ffb640 10 client.28073 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f3315ffb640 1 -- 172.21.15.33:0/3509127113 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e1abbc2450 con 0x55e1abbce300 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f3315ffb640 10 client.28073 handle_command_reply: tid=0 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 client.28073 shutdown 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 msgr2=0x55e1abbc7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 0x55e1abbc7380 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3320009850 tx=0x7f332002ab90 comp rx=0 tx=0).stop 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 shutdown_connections 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e1abbce300 0x55e1abc36e90 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1abbbdda0 0x55e1abbc88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f3315ffb640 0 client.28073 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1abbb8c30 0x55e1abbc78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 --2- 172.21.15.33:0/3509127113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1abbb8240 0x55e1abbc7380 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 >> 172.21.15.33:0/3509127113 conn(0x55e1abbb38b0 msgr2=0x55e1abbb2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 shutdown_connections 2024-04-24T23:26:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 1 -- 172.21.15.33:0/3509127113 wait complete. 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.726+0000 7f33375a1740 20 client.28073 trim_cache size 0 max 16384 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "0b45d440-053b-4633-b5b7-e3346cf71099": { 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0b45d440-053b-4633-b5b7-e3346cf71099", 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:26:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:26:45.732 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:26:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f331c0a2c60 msgr2=0x7f331c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f331c0a2c60 0x7f331c0a5120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f333019b270 tx=0x7f3318013460 comp rx=0 tx=0).stop 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330102230 msgr2=0x7f33301a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330102230 0x7f33301a1310 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f332800be20 tx=0x7f332800bf10 comp rx=0 tx=0).stop 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 shutdown_connections 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f331c0a2c60 0x7f331c0a5120 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3330102230 0x7f33301a1310 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3330101870 0x7f333019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 --2- 172.21.15.33:0/338578170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3330100680 0x7f3330199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 >> 172.21.15.33:0/338578170 conn(0x7f33300fbbf0 msgr2=0x7f33300fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 shutdown_connections 2024-04-24T23:26:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:26:45.727+0000 7f333489a640 1 -- 172.21.15.33:0/338578170 wait complete. 2024-04-24T23:26:45.741 INFO:tasks.cephfs.filesystem:scrub status for tag:0b45d440-053b-4633-b5b7-e3346cf71099 - {'path': '/', 'tag': '0b45d440-053b-4633-b5b7-e3346cf71099', 'options': 'recursive,force'} 2024-04-24T23:26:45.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 23652 28.80 MB/sec warmup 50 sec latency 84.925 ms 2024-04-24T23:26:46.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:46 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:46.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:46 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:46.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:46 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:46.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 24311 28.93 MB/sec warmup 51 sec latency 51.904 ms 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: pgmap v794: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.6 MiB/s wr, 520 op/s 2024-04-24T23:26:47.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:47 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:26:47.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:47.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:26:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: pgmap v794: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.6 MiB/s wr, 520 op/s 2024-04-24T23:26:47.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:47 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:26:47.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:47.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:26:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: pgmap v794: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.6 MiB/s wr, 520 op/s 2024-04-24T23:26:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:47 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:26:47.878 INFO:tasks.workunit.client.0.smithi033.stdout: 10 24844 29.21 MB/sec warmup 52 sec latency 104.205 ms 2024-04-24T23:26:48.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 25165 29.11 MB/sec warmup 53 sec latency 222.368 ms 2024-04-24T23:26:49.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:49 smithi038 ceph-mon[32450]: pgmap v795: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 464 op/s 2024-04-24T23:26:49.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:49 smithi117 ceph-mon[32529]: pgmap v795: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 464 op/s 2024-04-24T23:26:49.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:49 smithi033 ceph-mon[29201]: pgmap v795: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 464 op/s 2024-04-24T23:26:49.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 25412 28.75 MB/sec warmup 54 sec latency 225.658 ms 2024-04-24T23:26:50.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 25869 28.51 MB/sec warmup 55 sec latency 87.784 ms 2024-04-24T23:26:51.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:51 smithi038 ceph-mon[32450]: pgmap v796: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.0 MiB/s wr, 398 op/s 2024-04-24T23:26:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:51 smithi117 ceph-mon[32529]: pgmap v796: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.0 MiB/s wr, 398 op/s 2024-04-24T23:26:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:51 smithi033 ceph-mon[29201]: pgmap v796: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.0 MiB/s wr, 398 op/s 2024-04-24T23:26:51.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 26374 28.54 MB/sec warmup 56 sec latency 141.406 ms 2024-04-24T23:26:52.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 26805 28.44 MB/sec warmup 57 sec latency 96.894 ms 2024-04-24T23:26:53.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:53 smithi038 ceph-mon[32450]: pgmap v797: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.9 MiB/s wr, 466 op/s 2024-04-24T23:26:53.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:53 smithi117 ceph-mon[32529]: pgmap v797: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.9 MiB/s wr, 466 op/s 2024-04-24T23:26:53.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:53 smithi033 ceph-mon[29201]: pgmap v797: 129 pgs: 129 active+clean; 492 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.9 MiB/s wr, 466 op/s 2024-04-24T23:26:53.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 27344 28.35 MB/sec warmup 58 sec latency 79.790 ms 2024-04-24T23:26:54.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 27673 28.06 MB/sec warmup 59 sec latency 353.481 ms 2024-04-24T23:26:55.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:55 smithi038 ceph-mon[32450]: pgmap v798: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 459 op/s 2024-04-24T23:26:55.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:55 smithi117 ceph-mon[32529]: pgmap v798: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 459 op/s 2024-04-24T23:26:55.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:55 smithi033 ceph-mon[29201]: pgmap v798: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 459 op/s 2024-04-24T23:26:55.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 28240 28.15 MB/sec warmup 60 sec latency 126.201 ms 2024-04-24T23:26:56.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 28822 28.35 MB/sec warmup 61 sec latency 118.566 ms 2024-04-24T23:26:57.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:57 smithi038 ceph-mon[32450]: pgmap v799: 129 pgs: 129 active+clean; 504 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.0 MiB/s wr, 528 op/s 2024-04-24T23:26:57.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:57 smithi117 ceph-mon[32529]: pgmap v799: 129 pgs: 129 active+clean; 504 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.0 MiB/s wr, 528 op/s 2024-04-24T23:26:57.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:57 smithi033 ceph-mon[29201]: pgmap v799: 129 pgs: 129 active+clean; 504 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.0 MiB/s wr, 528 op/s 2024-04-24T23:26:57.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 29409 28.53 MB/sec warmup 62 sec latency 115.307 ms 2024-04-24T23:26:58.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:58 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:26:58.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:58 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:26:58.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:58 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:26:58.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:58 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:26:58.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:58 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:26:58.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:58 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:26:58.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:58 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:26:58.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:58 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:26:58.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:58 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:26:58.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 29881 28.49 MB/sec warmup 63 sec latency 90.674 ms 2024-04-24T23:26:59.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:26:59 smithi038 ceph-mon[32450]: pgmap v800: 129 pgs: 129 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-04-24T23:26:59.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:26:59 smithi117 ceph-mon[32529]: pgmap v800: 129 pgs: 129 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-04-24T23:26:59.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:26:59 smithi033 ceph-mon[29201]: pgmap v800: 129 pgs: 129 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-04-24T23:26:59.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 30017 28.13 MB/sec warmup 64 sec latency 271.851 ms 2024-04-24T23:27:00.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 30198 27.76 MB/sec warmup 65 sec latency 246.861 ms 2024-04-24T23:27:01.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:01 smithi038 ceph-mon[32450]: pgmap v801: 129 pgs: 129 active+clean; 503 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 381 op/s 2024-04-24T23:27:01.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:01 smithi117 ceph-mon[32529]: pgmap v801: 129 pgs: 129 active+clean; 503 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 381 op/s 2024-04-24T23:27:01.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:01 smithi033 ceph-mon[29201]: pgmap v801: 129 pgs: 129 active+clean; 503 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 381 op/s 2024-04-24T23:27:01.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 30498 27.51 MB/sec warmup 66 sec latency 93.743 ms 2024-04-24T23:27:02.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 31029 27.48 MB/sec warmup 67 sec latency 92.617 ms 2024-04-24T23:27:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:03 smithi038 ceph-mon[32450]: pgmap v802: 129 pgs: 129 active+clean; 490 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 507 op/s 2024-04-24T23:27:03.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:03 smithi117 ceph-mon[32529]: pgmap v802: 129 pgs: 129 active+clean; 490 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 507 op/s 2024-04-24T23:27:03.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:03 smithi033 ceph-mon[29201]: pgmap v802: 129 pgs: 129 active+clean; 490 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 507 op/s 2024-04-24T23:27:03.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 31877 27.70 MB/sec warmup 68 sec latency 68.102 ms 2024-04-24T23:27:04.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 32316 27.73 MB/sec warmup 69 sec latency 226.481 ms 2024-04-24T23:27:05.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:05 smithi038 ceph-mon[32450]: pgmap v803: 129 pgs: 129 active+clean; 491 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.1 MiB/s wr, 457 op/s 2024-04-24T23:27:05.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:05 smithi117 ceph-mon[32529]: pgmap v803: 129 pgs: 129 active+clean; 491 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.1 MiB/s wr, 457 op/s 2024-04-24T23:27:05.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:05 smithi033 ceph-mon[29201]: pgmap v803: 129 pgs: 129 active+clean; 491 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.1 MiB/s wr, 457 op/s 2024-04-24T23:27:05.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 32800 27.79 MB/sec warmup 70 sec latency 91.640 ms 2024-04-24T23:27:06.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 33225 27.78 MB/sec warmup 71 sec latency 113.839 ms 2024-04-24T23:27:07.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:07 smithi038 ceph-mon[32450]: pgmap v804: 129 pgs: 129 active+clean; 490 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 425 op/s 2024-04-24T23:27:07.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:07 smithi117 ceph-mon[32529]: pgmap v804: 129 pgs: 129 active+clean; 490 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 425 op/s 2024-04-24T23:27:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:07 smithi033 ceph-mon[29201]: pgmap v804: 129 pgs: 129 active+clean; 490 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 425 op/s 2024-04-24T23:27:07.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 33654 27.76 MB/sec warmup 72 sec latency 80.686 ms 2024-04-24T23:27:08.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 33928 27.53 MB/sec warmup 73 sec latency 83.270 ms 2024-04-24T23:27:09.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:09 smithi038 ceph-mon[32450]: pgmap v805: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 419 op/s 2024-04-24T23:27:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:09 smithi117 ceph-mon[32529]: pgmap v805: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 419 op/s 2024-04-24T23:27:09.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:09 smithi033 ceph-mon[29201]: pgmap v805: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 419 op/s 2024-04-24T23:27:09.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 34155 27.28 MB/sec warmup 74 sec latency 444.454 ms 2024-04-24T23:27:10.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 34629 27.19 MB/sec warmup 75 sec latency 136.185 ms 2024-04-24T23:27:11.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:11 smithi038 ceph-mon[32450]: pgmap v806: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2024-04-24T23:27:11.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:11 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:11 smithi117 ceph-mon[32529]: pgmap v806: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2024-04-24T23:27:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:11 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:11.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:11 smithi033 ceph-mon[29201]: pgmap v806: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2024-04-24T23:27:11.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:11 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:11.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 35064 27.14 MB/sec warmup 76 sec latency 150.358 ms 2024-04-24T23:27:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:12.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:27:12.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:12.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:12.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:27:12.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:12.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:12.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:27:12.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 35536 27.06 MB/sec warmup 77 sec latency 77.511 ms 2024-04-24T23:27:13.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:13 smithi038 ceph-mon[32450]: pgmap v807: 129 pgs: 129 active+clean; 514 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.2 MiB/s wr, 509 op/s 2024-04-24T23:27:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:13 smithi117 ceph-mon[32529]: pgmap v807: 129 pgs: 129 active+clean; 514 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.2 MiB/s wr, 509 op/s 2024-04-24T23:27:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:13 smithi033 ceph-mon[29201]: pgmap v807: 129 pgs: 129 active+clean; 514 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.2 MiB/s wr, 509 op/s 2024-04-24T23:27:13.879 INFO:tasks.workunit.client.0.smithi033.stdout: 10 36085 27.02 MB/sec warmup 78 sec latency 57.876 ms 2024-04-24T23:27:14.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 36544 27.04 MB/sec warmup 79 sec latency 163.579 ms 2024-04-24T23:27:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:15 smithi038 ceph-mon[32450]: pgmap v808: 129 pgs: 129 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 406 op/s 2024-04-24T23:27:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:15 smithi117 ceph-mon[32529]: pgmap v808: 129 pgs: 129 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 406 op/s 2024-04-24T23:27:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:15 smithi033 ceph-mon[29201]: pgmap v808: 129 pgs: 129 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 406 op/s 2024-04-24T23:27:15.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:27:15.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:27:15.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 37184 27.27 MB/sec warmup 80 sec latency 86.295 ms 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 -- 172.21.15.33:0/792961522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade41022c0 msgr2=0x7fade41046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 --2- 172.21.15.33:0/792961522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade41022c0 0x7fade41046b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009900 tx=0x7fadd402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 -- 172.21.15.33:0/792961522 shutdown_connections 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 --2- 172.21.15.33:0/792961522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 0x7fade4109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 --2- 172.21.15.33:0/792961522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade4104bf0 0x7fade4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 --2- 172.21.15.33:0/792961522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade41022c0 0x7fade41046b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 -- 172.21.15.33:0/792961522 >> 172.21.15.33:0/792961522 conn(0x7fade40fbc30 msgr2=0x7fade40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 -- 172.21.15.33:0/792961522 shutdown_connections 2024-04-24T23:27:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.891+0000 7fadea97e640 1 -- 172.21.15.33:0/792961522 wait complete. 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 Processor -- start 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 -- start start 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade41022c0 0x7fade4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade4104bf0 0x7fade41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 0x7fade419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fade410d410 con 0x7fade4104bf0 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fade410d290 con 0x7fade41022c0 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fadea97e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fade410d590 con 0x7fade4107520 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fade3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade41022c0 0x7fade4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fade37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade4104bf0 0x7fade41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fade3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade41022c0 0x7fade4197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57972/0 (socket says 172.21.15.33:57972) 2024-04-24T23:27:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fade3fff640 1 -- 172.21.15.33:0/1590425150 learned_addr learned my addr 172.21.15.33:0/1590425150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.892+0000 7fade8f1b640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 0x7fade419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade8f1b640 1 -- 172.21.15.33:0/1590425150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade41022c0 msgr2=0x7fade4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade8f1b640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade41022c0 0x7fade4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade8f1b640 1 -- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade4104bf0 msgr2=0x7fade41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade8f1b640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade4104bf0 0x7fade41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade8f1b640 1 -- 172.21.15.33:0/1590425150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade419f780 con 0x7fade4107520 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade8f1b640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 0x7fade419f110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faddc00da10 tx=0x7faddc00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade17fa640 1 -- 172.21.15.33:0/1590425150 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc00bd70 con 0x7fade4107520 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade419fa70 con 0x7fade4107520 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade419ffc0 con 0x7fade4107520 2024-04-24T23:27:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade17fa640 1 -- 172.21.15.33:0/1590425150 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faddc003e50 con 0x7fade4107520 2024-04-24T23:27:15.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.893+0000 7fade17fa640 1 -- 172.21.15.33:0/1590425150 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc0106c0 con 0x7fade4107520 2024-04-24T23:27:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 Processor -- start 2024-04-24T23:27:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7fade17fa640 1 -- 172.21.15.33:0/1590425150 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7faddc0042a0 con 0x7fade4107520 2024-04-24T23:27:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 -- start start 2024-04-24T23:27:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f67a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f67f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f682230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b00f5de6b0 con 0x55b00f67a240 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b00f586880 con 0x55b00f67ac30 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.896+0000 7faded76d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b00f5da670 con 0x55b00f67fda0 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f67a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f67a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45152/0 (socket says 172.21.15.33:45152) 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 -- 172.21.15.33:0/20525172 learned_addr learned my addr 172.21.15.33:0/20525172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade3fff640 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f67f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f682230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade17fa640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fadcc0a2b90 0x7fadcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 -- 172.21.15.33:0/20525172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 msgr2=0x55b00f682230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f682230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 -- 172.21.15.33:0/20525172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 msgr2=0x55b00f67f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f67f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 -- 172.21.15.33:0/20525172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b00f3bc7d0 con 0x55b00f67a240 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade3fff640 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f67f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade8f1b640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fadcc0a2b90 0x7fadcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade17fa640 1 -- 172.21.15.33:0/1590425150 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faddc0cf480 con 0x7fade4107520 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fade37fe640 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f67a660 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fadd80125f0 tx=0x7fadd800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62355df43282b183 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.897+0000 7fadca7fc640 1 -- 172.21.15.33:0/20525172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8004020 con 0x55b00f67a240 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7fadca7fc640 1 -- 172.21.15.33:0/20525172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd8004c80 con 0x55b00f67a240 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7fadca7fc640 1 -- 172.21.15.33:0/20525172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8005380 con 0x55b00f67a240 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 -- 172.21.15.33:0/20525172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 msgr2=0x55b00f67a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7fade8f1b640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fadcc0a2b90 0x7fadcc0a5050 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faddc09b650 tx=0x7faddc004640 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f67a660 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fadd80125f0 tx=0x7fadd800ba40 comp rx=0 tx=0).stop 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 -- 172.21.15.33:0/20525172 shutdown_connections 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f682230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f67f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 --2- 172.21.15.33:0/20525172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f67a660 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 -- 172.21.15.33:0/20525172 >> 172.21.15.33:0/20525172 conn(0x55b00f6758b0 msgr2=0x55b00f675cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 -- 172.21.15.33:0/20525172 shutdown_connections 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.898+0000 7faded76d740 1 -- 172.21.15.33:0/20525172 wait complete. 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 -- start start 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 Processor -- start 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f6845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f684b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f68bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b00f682d50 con 0x55b00f67a240 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b00f599f50 con 0x55b00f67ac30 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b00f682ed0 con 0x55b00f67fda0 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7faded76d740 10 client.0 _ll_register_callbacks cb 0x55b00f5d5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7fade37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f6845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7fade37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f6845e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45160/0 (socket says 172.21.15.33:45160) 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7fade37fe640 1 -- 172.21.15.33:0/814539028 learned_addr learned my addr 172.21.15.33:0/814539028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.899+0000 7fade3fff640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f684b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade37fe640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f68bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade3fff640 1 -- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 msgr2=0x55b00f68bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade3fff640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f68bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade3fff640 1 -- 172.21.15.33:0/814539028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 msgr2=0x55b00f6845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade3fff640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f6845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade3fff640 1 -- 172.21.15.33:0/814539028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b00f2f23f0 con 0x55b00f67ac30 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade37fe640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f6845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fade3fff640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f684b20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005940 tx=0x7fadd4031a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5b81b019d12f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fadc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fadc37fe640 1 -- 172.21.15.33:0/814539028 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4035040 con 0x55b00f67ac30 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd8004c80 con 0x55b00f67ac30 2024-04-24T23:27:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fadc37fe640 1 -- 172.21.15.33:0/814539028 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fadd4004380 con 0x55b00f67ac30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7fadc37fe640 1 -- 172.21.15.33:0/814539028 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadd404f050 con 0x55b00f67ac30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7faded76d740 10 client.28085 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.900+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b00f2b20f0 con 0x55b00f67ac30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7fadc37fe640 1 -- 172.21.15.33:0/814539028 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd8004c80 con 0x55b00f67ac30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7fadc37fe640 1 -- 172.21.15.33:0/814539028 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fadd4004c30 con 0x55b00f67ac30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7faded76d740 10 client.28085 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7faded76d740 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7faded76d740 20 client.28085 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7faded76d740 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6903c0 0x55b00f692840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7faded76d740 4 client.28085 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b00f5ebbf0 con 0x55b00f6903c0 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7fade37fe640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6903c0 0x55b00f692840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7fade37fe640 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6903c0 0x55b00f692840 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e9c9b279cdfd85 server_cookie=29693db85301acf6 in_seq=0 out_seq=0 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.901+0000 7fadc37fe640 10 client.28085 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7fadc37fe640 1 -- 172.21.15.33:0/814539028 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b00f5ebbf0 con 0x55b00f6903c0 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7fadc37fe640 10 client.28085 handle_command_reply: tid=0 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 client.28085 shutdown 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 msgr2=0x55b00f684b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f684b20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005940 tx=0x7fadd4031a00 comp rx=0 tx=0).stop 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 shutdown_connections 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6903c0 0x55b00f692840 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67fda0 0x55b00f68bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67ac30 0x55b00f684b20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 --2- 172.21.15.33:0/814539028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67a240 0x55b00f6845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 >> 172.21.15.33:0/814539028 conn(0x55b00f6758b0 msgr2=0x55b00f674af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.902+0000 7fadc37fe640 0 client.28085 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.903+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 shutdown_connections 2024-04-24T23:27:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.903+0000 7faded76d740 1 -- 172.21.15.33:0/814539028 wait complete. 2024-04-24T23:27:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.903+0000 7faded76d740 20 client.28085 trim_cache size 0 max 16384 2024-04-24T23:27:15.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 Processor -- start 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 -- start start 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f67c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f68bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f67c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fade41acd20 con 0x55b00f67ac30 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aa590 con 0x55b00f67fda0 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7faded76d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a83e0 con 0x55b00f67a240 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f68bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f68bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45162/0 (socket says 172.21.15.33:45162) 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 -- 172.21.15.33:0/244586973 learned_addr learned my addr 172.21.15.33:0/244586973 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade37fe640 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f67c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade37fe640 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f67c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 -- 172.21.15.33:0/244586973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 msgr2=0x55b00f67c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f67c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 -- 172.21.15.33:0/244586973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 msgr2=0x55b00f67c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f67c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 -- 172.21.15.33:0/244586973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b00f667090 con 0x55b00f67ac30 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.922+0000 7fade3fff640 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f68bb20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009a20 tx=0x7fadd40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644ab3d2b79e6177 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7fadc9ffb640 1 -- 172.21.15.33:0/244586973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4035070 con 0x55b00f67ac30 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7fadc9ffb640 1 -- 172.21.15.33:0/244586973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4057020 con 0x55b00f67ac30 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7fadc9ffb640 1 -- 172.21.15.33:0/244586973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4003e20 con 0x55b00f67ac30 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 -- 172.21.15.33:0/244586973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 msgr2=0x55b00f68bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f68bb20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009a20 tx=0x7fadd40316f0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 -- 172.21.15.33:0/244586973 shutdown_connections 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f67c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f68bb20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 --2- 172.21.15.33:0/244586973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f67c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 -- 172.21.15.33:0/244586973 >> 172.21.15.33:0/244586973 conn(0x55b00f6758b0 msgr2=0x55b00f6845e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 -- 172.21.15.33:0/244586973 shutdown_connections 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.923+0000 7faded76d740 1 -- 172.21.15.33:0/244586973 wait complete. 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 -- start start 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 Processor -- start 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f688b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f6890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f6f4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a8a50 con 0x55b00f67ac30 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a9ec0 con 0x55b00f67fda0 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a9120 con 0x55b00f67a240 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7faded76d740 10 client.0 _ll_register_callbacks cb 0x7fadd4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7fade3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f6890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7fade37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f688b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7fade3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f6890d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45164/0 (socket says 172.21.15.33:45164) 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7fade3fff640 1 -- 172.21.15.33:0/2399404283 learned_addr learned my addr 172.21.15.33:0/2399404283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.924+0000 7fade37fe640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f6f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade37fe640 1 -- 172.21.15.33:0/2399404283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 msgr2=0x55b00f6f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade37fe640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f6f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade37fe640 1 -- 172.21.15.33:0/2399404283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 msgr2=0x55b00f6890d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade37fe640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f6890d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade37fe640 1 -- 172.21.15.33:0/2399404283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b00f6f4f60 con 0x55b00f67a240 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade3fff640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f6890d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fade37fe640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f688b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fadd8004f50 tx=0x7fadd4005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b250a332ced509d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fadc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fadc37fe640 1 -- 172.21.15.33:0/2399404283 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8028040 con 0x55b00f67a240 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fadc37fe640 1 -- 172.21.15.33:0/2399404283 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fadd8004f80 con 0x55b00f67a240 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd4057020 con 0x55b00f67a240 2024-04-24T23:27:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fadc37fe640 1 -- 172.21.15.33:0/2399404283 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadd8004cc0 con 0x55b00f67a240 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7faded76d740 10 client.28036 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b00f6f6270 con 0x55b00f67a240 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.925+0000 7fadc37fe640 1 -- 172.21.15.33:0/2399404283 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4057020 con 0x55b00f67a240 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7fadc37fe640 1 -- 172.21.15.33:0/2399404283 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fadd8032340 con 0x55b00f67a240 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7faded76d740 10 client.28036 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7faded76d740 10 client.28036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7faded76d740 20 client.28036 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7faded76d740 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6f6b60 0x55b00f6f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7faded76d740 4 client.28036 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:15.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b00f5e5f40 con 0x55b00f6f6b60 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7fade3fff640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6f6b60 0x55b00f6f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7fadc37fe640 10 client.28036 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.926+0000 7fade3fff640 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6f6b60 0x55b00f6f8f80 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107396aa67831f50 server_cookie=2e36f87c7d0c3c31 in_seq=0 out_seq=0 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7fadc37fe640 1 -- 172.21.15.33:0/2399404283 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b00f5e5f40 con 0x55b00f6f6b60 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7fadc37fe640 10 client.28036 handle_command_reply: tid=0 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 client.28036 shutdown 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 msgr2=0x55b00f688b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f688b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fadd8004f50 tx=0x7fadd4005e40 comp rx=0 tx=0).stop 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 shutdown_connections 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b00f6f6b60 0x55b00f6f8f80 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b00f67fda0 0x55b00f6f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b00f67ac30 0x55b00f6890d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 --2- 172.21.15.33:0/2399404283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b00f67a240 0x55b00f688b90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 >> 172.21.15.33:0/2399404283 conn(0x55b00f6758b0 msgr2=0x55b00f6845c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7fadc37fe640 0 client.28036 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 shutdown_connections 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 1 -- 172.21.15.33:0/2399404283 wait complete. 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.927+0000 7faded76d740 20 client.28036 trim_cache size 0 max 16384 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:27:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:27:15.938 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:27:15.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fadcc0a2b90 msgr2=0x7fadcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fadcc0a2b90 0x7fadcc0a5050 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faddc09b650 tx=0x7faddc004640 comp rx=0 tx=0).stop 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 msgr2=0x7fade419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 0x7fade419f110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faddc00da10 tx=0x7faddc00ded0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 shutdown_connections 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fadcc0a2b90 0x7fadcc0a5050 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4107520 0x7fade419f110 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade4104bf0 0x7fade41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 --2- 172.21.15.33:0/1590425150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade41022c0 0x7fade4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 >> 172.21.15.33:0/1590425150 conn(0x7fade40fbc30 msgr2=0x7fade4102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 shutdown_connections 2024-04-24T23:27:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:15.928+0000 7fadea97e640 1 -- 172.21.15.33:0/1590425150 wait complete. 2024-04-24T23:27:15.940 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:15.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 -- 172.21.15.33:0/4111378014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220101870 msgr2=0x7f8220101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 --2- 172.21.15.33:0/4111378014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220101870 0x7f8220101cf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8208009900 tx=0x7f820802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 -- 172.21.15.33:0/4111378014 shutdown_connections 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 --2- 172.21.15.33:0/4111378014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 0x7f8220108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 --2- 172.21.15.33:0/4111378014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220101870 0x7f8220101cf0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 --2- 172.21.15.33:0/4111378014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220100680 0x7f8220100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 -- 172.21.15.33:0/4111378014 >> 172.21.15.33:0/4111378014 conn(0x7f82200fbbf0 msgr2=0x7f82200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 -- 172.21.15.33:0/4111378014 shutdown_connections 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.051+0000 7f8224f29640 1 -- 172.21.15.33:0/4111378014 wait complete. 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 Processor -- start 2024-04-24T23:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 -- start start 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220100680 0x7f8220104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220101870 0x7f8220102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 0x7f8220103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f822010c300 con 0x7f8220100680 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f822010c180 con 0x7f8220101870 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8224f29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f822010c480 con 0x7f8220102230 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f821e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220100680 0x7f8220104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f8217fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220101870 0x7f8220102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f821e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220100680 0x7f8220104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45190/0 (socket says 172.21.15.33:45190) 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f821e59c640 1 -- 172.21.15.33:0/4057033101 learned_addr learned my addr 172.21.15.33:0/4057033101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.052+0000 7f821ed9d640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 0x7f8220103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8217fff640 1 -- 172.21.15.33:0/4057033101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 msgr2=0x7f8220103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8217fff640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 0x7f8220103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8217fff640 1 -- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220100680 msgr2=0x7f8220104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8217fff640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220100680 0x7f8220104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8217fff640 1 -- 172.21.15.33:0/4057033101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220103a20 con 0x7f8220101870 2024-04-24T23:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f821ed9d640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 0x7f8220103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8217fff640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220101870 0x7f8220102c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8208031440 tx=0x7f8208004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820803f070 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82201a1b60 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8208003cd0 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208035070 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.053+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201a20d0 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.054+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82201a2300 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.055+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f820803b4f0 con 0x7f8220101870 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.056+0000 7f82177fe640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f820c0a2b90 0x7f820c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.056+0000 7f821e59c640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f820c0a2b90 0x7f820c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.056+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f82080ef780 con 0x7f8220101870 2024-04-24T23:27:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.057+0000 7f821e59c640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f820c0a2b90 0x7f820c0a5050 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f82100058c0 tx=0x7f8210005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.062+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f820803be20 con 0x7f8220101870 2024-04-24T23:27:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.310+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82201a24d0 con 0x7f8220101870 2024-04-24T23:27:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.311+0000 7f82177fe640 1 -- 172.21.15.33:0/4057033101 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11863 (secure 0 0 0) 0x7f82080bbca0 con 0x7f8220101870 2024-04-24T23:27:16.312 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:27:16.312 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:26:01.133927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:27:16.312 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f820c0a2b90 msgr2=0x7f820c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f820c0a2b90 0x7f820c0a5050 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f82100058c0 tx=0x7f8210005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220101870 msgr2=0x7f8220102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220101870 0x7f8220102c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8208031440 tx=0x7f8208004270 comp rx=0 tx=0).stop 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 shutdown_connections 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f820c0a2b90 0x7f820c0a5050 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102230 0x7f8220103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8220101870 0x7f8220102c50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 --2- 172.21.15.33:0/4057033101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8220100680 0x7f8220104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 >> 172.21.15.33:0/4057033101 conn(0x7f82200fbbf0 msgr2=0x7f82200fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 shutdown_connections 2024-04-24T23:27:16.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.313+0000 7f8224f29640 1 -- 172.21.15.33:0/4057033101 wait complete. 2024-04-24T23:27:16.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:27:16.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 -- 172.21.15.33:0/3577982098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44101870 msgr2=0x7fed44101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 --2- 172.21.15.33:0/3577982098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44101870 0x7fed44101cf0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fed38009900 tx=0x7fed3802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 -- 172.21.15.33:0/3577982098 shutdown_connections 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 --2- 172.21.15.33:0/3577982098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed44102230 0x7fed44108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 --2- 172.21.15.33:0/3577982098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44101870 0x7fed44101cf0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 --2- 172.21.15.33:0/3577982098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44100680 0x7fed44100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 -- 172.21.15.33:0/3577982098 >> 172.21.15.33:0/3577982098 conn(0x7fed440fbbf0 msgr2=0x7fed440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 -- 172.21.15.33:0/3577982098 shutdown_connections 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.445+0000 7fed49f09640 1 -- 172.21.15.33:0/3577982098 wait complete. 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 Processor -- start 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 -- start start 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44100680 0x7fed441045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed44101870 0x7fed44102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 0x7fed44103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410c300 con 0x7fed44102230 2024-04-24T23:27:16.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410c180 con 0x7fed44101870 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed49f09640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410c480 con 0x7fed44100680 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed43fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 0x7fed44103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed43fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 0x7fed44103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45210/0 (socket says 172.21.15.33:45210) 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed43fff640 1 -- 172.21.15.33:0/1456238087 learned_addr learned my addr 172.21.15.33:0/1456238087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed42ffd640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed44101870 0x7fed44102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.446+0000 7fed437fe640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44100680 0x7fed441045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed43fff640 1 -- 172.21.15.33:0/1456238087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44100680 msgr2=0x7fed441045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed43fff640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44100680 0x7fed441045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed43fff640 1 -- 172.21.15.33:0/1456238087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed44101870 msgr2=0x7fed44102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed43fff640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed44101870 0x7fed44102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed43fff640 1 -- 172.21.15.33:0/1456238087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed44103a50 con 0x7fed44102230 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed43fff640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 0x7fed44103190 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00eb10 tx=0x7fed3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed40ff9640 1 -- 172.21.15.33:0/1456238087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c004b20 con 0x7fed44102230 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a1c20 con 0x7fed44102230 2024-04-24T23:27:16.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed40ff9640 1 -- 172.21.15.33:0/1456238087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c004cc0 con 0x7fed44102230 2024-04-24T23:27:16.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed40ff9640 1 -- 172.21.15.33:0/1456238087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c010760 con 0x7fed44102230 2024-04-24T23:27:16.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.447+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a20f0 con 0x7fed44102230 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed40ff9640 1 -- 172.21.15.33:0/1456238087 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fed3c005020 con 0x7fed44102230 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 Processor -- start 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 -- start start 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0479650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe047e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe0481220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560fe03dd670 con 0x560fe047ed90 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560fe0385970 con 0x560fe0479230 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed4cc12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fe03d9630 con 0x560fe0479c20 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe047e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0479650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe047e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56604/0 (socket says 172.21.15.33:56604) 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 -- 172.21.15.33:0/4190562981 learned_addr learned my addr 172.21.15.33:0/4190562981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed42ffd640 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe0481220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed40ff9640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed2c0a2b90 0x7fed2c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed43fff640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed2c0a2b90 0x7fed2c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 -- 172.21.15.33:0/4190562981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 msgr2=0x560fe0479650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0479650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 -- 172.21.15.33:0/4190562981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 msgr2=0x560fe0481220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.450+0000 7fed437fe640 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe0481220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed437fe640 1 -- 172.21.15.33:0/4190562981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe03a9400 con 0x560fe0479c20 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed40ff9640 1 -- 172.21.15.33:0/1456238087 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed3c0cf980 con 0x7fed44102230 2024-04-24T23:27:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed437fe640 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe047e720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed34009ca0 tx=0x7fed34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8686f69b733730fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed29ffb640 1 -- 172.21.15.33:0/4190562981 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34004220 con 0x560fe0479c20 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed43fff640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed2c0a2b90 0x7fed2c0a5050 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0cd460 tx=0x7fed3c0057d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed29ffb640 1 -- 172.21.15.33:0/4190562981 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed340043c0 con 0x560fe0479c20 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.451+0000 7fed29ffb640 1 -- 172.21.15.33:0/4190562981 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34019890 con 0x560fe0479c20 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 -- 172.21.15.33:0/4190562981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 msgr2=0x560fe047e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe047e720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed34009ca0 tx=0x7fed34011810 comp rx=0 tx=0).stop 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 -- 172.21.15.33:0/4190562981 shutdown_connections 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe0481220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe047e720 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 --2- 172.21.15.33:0/4190562981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0479650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 -- 172.21.15.33:0/4190562981 >> 172.21.15.33:0/4190562981 conn(0x560fe04748a0 msgr2=0x560fe0474ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 -- 172.21.15.33:0/4190562981 shutdown_connections 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 -- 172.21.15.33:0/4190562981 wait complete. 2024-04-24T23:27:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 -- start start 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.452+0000 7fed4cc12740 1 Processor -- start 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe04832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0483810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe048a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560fe0481d40 con 0x560fe047ed90 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560fe0399050 con 0x560fe0479230 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fe0481ec0 con 0x560fe0479c20 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed4cc12740 10 client.0 _ll_register_callbacks cb 0x560fe00f0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe048a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0483810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe048a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45216/0 (socket says 172.21.15.33:45216) 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed42ffd640 1 -- 172.21.15.33:0/890481187 learned_addr learned my addr 172.21.15.33:0/890481187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed42ffd640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe04832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 -- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 msgr2=0x560fe04832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe04832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 -- 172.21.15.33:0/890481187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 msgr2=0x560fe048a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe048a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 -- 172.21.15.33:0/890481187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe036adc0 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed42ffd640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe048a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed437fe640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0483810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x560fe0475770 tx=0x7fed34005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a2cc6e1dabb483 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.453+0000 7fed22ffd640 1 -- 172.21.15.33:0/890481187 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34028070 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed22ffd640 1 -- 172.21.15.33:0/890481187 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed340035f0 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed340043c0 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed22ffd640 1 -- 172.21.15.33:0/890481187 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed34034050 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 10 client.28054 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fe00f13f0 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed22ffd640 1 -- 172.21.15.33:0/890481187 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed340043c0 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed22ffd640 1 -- 172.21.15.33:0/890481187 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fed340049a0 con 0x560fe0479c20 2024-04-24T23:27:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 10 client.28054 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 10 client.28054 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 20 client.28054 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe048f4e0 0x560fe0491920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 4 client.28054 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe03eabb0 con 0x560fe048f4e0 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.454+0000 7fed42ffd640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe048f4e0 0x560fe0491920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.455+0000 7fed42ffd640 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe048f4e0 0x560fe0491920 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a34ca868ee58dc66 server_cookie=a4b9cf1432afcdf0 in_seq=0 out_seq=0 2024-04-24T23:27:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.455+0000 7fed22ffd640 10 client.28054 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.458+0000 7fed22ffd640 1 -- 172.21.15.33:0/890481187 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fe03eabb0 con 0x560fe048f4e0 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.458+0000 7fed22ffd640 10 client.28054 handle_command_reply: tid=0 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 client.28054 shutdown 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 msgr2=0x560fe0483810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0483810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x560fe0475770 tx=0x7fed34005270 comp rx=0 tx=0).stop 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 shutdown_connections 2024-04-24T23:27:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe048f4e0 0x560fe0491920 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe048a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0483810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed22ffd640 0 client.28054 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 --2- 172.21.15.33:0/890481187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe04832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 >> 172.21.15.33:0/890481187 conn(0x560fe04748a0 msgr2=0x560fe0473b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 shutdown_connections 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 1 -- 172.21.15.33:0/890481187 wait complete. 2024-04-24T23:27:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.459+0000 7fed4cc12740 20 client.28054 trim_cache size 0 max 16384 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 Processor -- start 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 -- start start 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe0479230 0x560fe047e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479c20 0x560fe048a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe047ed90 0x560fe04739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed441aef00 con 0x560fe0479230 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ac770 con 0x560fe0479c20 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed4cc12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed441aa5c0 con 0x560fe047ed90 2024-04-24T23:27:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe0479230 0x560fe047e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479c20 0x560fe048a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe0479230 0x560fe047e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45222/0 (socket says 172.21.15.33:45222) 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed42ffd640 1 -- 172.21.15.33:0/1504398186 learned_addr learned my addr 172.21.15.33:0/1504398186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed42ffd640 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe047ed90 0x560fe04739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed437fe640 1 -- 172.21.15.33:0/1504398186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe047ed90 msgr2=0x560fe04739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed437fe640 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe047ed90 0x560fe04739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed437fe640 1 -- 172.21.15.33:0/1504398186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe0479230 msgr2=0x560fe047e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed437fe640 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe0479230 0x560fe047e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.475+0000 7fed437fe640 1 -- 172.21.15.33:0/1504398186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe01bb7d0 con 0x560fe0479c20 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.476+0000 7fed437fe640 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479c20 0x560fe048a810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x560fe0475740 tx=0x7fed34005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4581501ac360050d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.476+0000 7fed297fa640 1 -- 172.21.15.33:0/1504398186 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34028040 con 0x560fe0479c20 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.476+0000 7fed297fa640 1 -- 172.21.15.33:0/1504398186 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34009af0 con 0x560fe0479c20 2024-04-24T23:27:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.476+0000 7fed297fa640 1 -- 172.21.15.33:0/1504398186 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34003850 con 0x560fe0479c20 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 -- 172.21.15.33:0/1504398186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479c20 msgr2=0x560fe048a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479c20 0x560fe048a810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x560fe0475740 tx=0x7fed34005fd0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 -- 172.21.15.33:0/1504398186 shutdown_connections 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe047ed90 0x560fe04739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479c20 0x560fe048a810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 --2- 172.21.15.33:0/1504398186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe0479230 0x560fe047e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 -- 172.21.15.33:0/1504398186 >> 172.21.15.33:0/1504398186 conn(0x560fe04748a0 msgr2=0x560fe04732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 -- 172.21.15.33:0/1504398186 shutdown_connections 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 -- 172.21.15.33:0/1504398186 wait complete. 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 -- start start 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.477+0000 7fed4cc12740 1 Processor -- start 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0487660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0487ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe04f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed441aac30 con 0x560fe047ed90 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ac0a0 con 0x560fe0479230 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ab300 con 0x560fe0479c20 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed4cc12740 10 client.0 _ll_register_callbacks cb 0x7fed38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0487ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe04f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0487ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56634/0 (socket says 172.21.15.33:56634) 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 -- 172.21.15.33:0/3668060336 learned_addr learned my addr 172.21.15.33:0/3668060336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed42ffd640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0487660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 -- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 msgr2=0x560fe0487660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0487660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 -- 172.21.15.33:0/3668060336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 msgr2=0x560fe04f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe04f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 -- 172.21.15.33:0/3668060336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe048ab30 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed42ffd640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe04f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed437fe640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0487ba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed340092e0 tx=0x7fed34009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3921dda55836803e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.478+0000 7fed22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.479+0000 7fed22ffd640 1 -- 172.21.15.33:0/3668060336 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3403c070 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.479+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34009af0 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.479+0000 7fed22ffd640 1 -- 172.21.15.33:0/3668060336 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed340045e0 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.479+0000 7fed22ffd640 1 -- 172.21.15.33:0/3668060336 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed34048050 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.479+0000 7fed4cc12740 10 client.28060 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.479+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fe04f4ff0 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed22ffd640 1 -- 172.21.15.33:0/3668060336 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34009af0 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed22ffd640 1 -- 172.21.15.33:0/3668060336 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fed3402d440 con 0x560fe0479c20 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed4cc12740 10 client.28060 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed4cc12740 10 client.28060 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed4cc12740 20 client.28060 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed4cc12740 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe04f5970 0x560fe04f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed4cc12740 4 client.28060 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:27:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fe03e4f00 con 0x560fe04f5970 2024-04-24T23:27:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed42ffd640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe04f5970 0x560fe04f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed22ffd640 10 client.28060 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.480+0000 7fed42ffd640 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe04f5970 0x560fe04f7d90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=432cc7bdd52900c7 server_cookie=bf4466bc4a5af95f in_seq=0 out_seq=0 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed22ffd640 1 -- 172.21.15.33:0/3668060336 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fe03e4f00 con 0x560fe04f5970 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed22ffd640 10 client.28060 handle_command_reply: tid=0 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 client.28060 shutdown 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 msgr2=0x560fe0487ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0487ba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed340092e0 tx=0x7fed34009310 comp rx=0 tx=0).stop 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 shutdown_connections 2024-04-24T23:27:16.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x560fe04f5970 0x560fe04f7d90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fe047ed90 0x560fe04f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe0479c20 0x560fe0487ba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 --2- 172.21.15.33:0/3668060336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560fe0479230 0x560fe0487660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed22ffd640 0 client.28060 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 >> 172.21.15.33:0/3668060336 conn(0x560fe04748a0 msgr2=0x560fe04732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 shutdown_connections 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 1 -- 172.21.15.33:0/3668060336 wait complete. 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.483+0000 7fed4cc12740 20 client.28060 trim_cache size 0 max 16384 2024-04-24T23:27:16.487 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed2c0a2b90 msgr2=0x7fed2c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed2c0a2b90 0x7fed2c0a5050 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0cd460 tx=0x7fed3c0057d0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 msgr2=0x7fed44103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 0x7fed44103190 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00eb10 tx=0x7fed3c00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 shutdown_connections 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed2c0a2b90 0x7fed2c0a5050 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed44102230 0x7fed44103190 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed44101870 0x7fed44102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 --2- 172.21.15.33:0/1456238087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44100680 0x7fed441045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 >> 172.21.15.33:0/1456238087 conn(0x7fed440fbbf0 msgr2=0x7fed440fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 shutdown_connections 2024-04-24T23:27:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.484+0000 7fed49f09640 1 -- 172.21.15.33:0/1456238087 wait complete. 2024-04-24T23:27:16.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:27:16.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 -- 172.21.15.33:0/29339278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 msgr2=0x7f2a5c100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 --2- 172.21.15.33:0/29339278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c100980 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009ca0 tx=0x7f2a4c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 -- 172.21.15.33:0/29339278 shutdown_connections 2024-04-24T23:27:16.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 --2- 172.21.15.33:0/29339278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c102110 0x7f2a5c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 --2- 172.21.15.33:0/29339278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c101750 0x7f2a5c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 --2- 172.21.15.33:0/29339278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c100980 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 -- 172.21.15.33:0/29339278 >> 172.21.15.33:0/29339278 conn(0x7f2a5c0fbaf0 msgr2=0x7f2a5c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 -- 172.21.15.33:0/29339278 shutdown_connections 2024-04-24T23:27:16.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.614+0000 7f2a6081a640 1 -- 172.21.15.33:0/29339278 wait complete. 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.615+0000 7f2a6081a640 1 Processor -- start 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 -- start start 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c101750 0x7f2a5c1a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c102110 0x7f2a5c1a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45248/0 (socket says 172.21.15.33:45248) 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c101750 0x7f2a5c1a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c101750 0x7f2a5c1a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56664/0 (socket says 172.21.15.33:56664) 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10c1f0 con 0x7f2a5c100560 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/2206937033 learned_addr learned my addr 172.21.15.33:0/2206937033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10c070 con 0x7f2a5c102110 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10c370 con 0x7f2a5c101750 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.616+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c102110 0x7f2a5c1a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5b59e640 1 -- 172.21.15.33:0/2206937033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c101750 msgr2=0x7f2a5c1a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c101750 0x7f2a5c1a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5b59e640 1 -- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 msgr2=0x7f2a5c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5b59e640 1 -- 172.21.15.33:0/2206937033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c1a5bd0 con 0x7f2a5c102110 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c102110 0x7f2a5c1a5310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e970 tx=0x7f2a5400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a43fff640 1 -- 172.21.15.33:0/2206937033 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400ccc0 con 0x7f2a5c102110 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a43fff640 1 -- 172.21.15.33:0/2206937033 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a54005020 con 0x7f2a5c102110 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a43fff640 1 -- 172.21.15.33:0/2206937033 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54004a00 con 0x7f2a5c102110 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c1a5ec0 con 0x7f2a5c102110 2024-04-24T23:27:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.617+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1a6330 con 0x7f2a5c102110 2024-04-24T23:27:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.620+0000 7f2a43fff640 1 -- 172.21.15.33:0/2206937033 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2a540051d0 con 0x7f2a5c102110 2024-04-24T23:27:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.620+0000 7f2a63dfd740 1 Processor -- start 2024-04-24T23:27:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.620+0000 7f2a63dfd740 1 -- start start 2024-04-24T23:27:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.620+0000 7f2a43fff640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2a440a2c10 0x7f2a440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.620+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 0x55e37760a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.620+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760adb0 0x55e37760f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e3776123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e37756e670 con 0x55e37760a3c0 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e377516970 con 0x55e37760adb0 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e37756a630 con 0x55e37760ff20 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a43fff640 1 -- 172.21.15.33:0/2206937033 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2a540cf240 con 0x7f2a5c102110 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2a440a2c10 0x7f2a440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 0x55e37760a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 0x55e37760a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45258/0 (socket says 172.21.15.33:45258) 2024-04-24T23:27:16.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/3384307210 learned_addr learned my addr 172.21.15.33:0/3384307210 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e3776123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760adb0 0x55e37760f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/3384307210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 msgr2=0x55e3776123b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e3776123b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/3384307210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760adb0 msgr2=0x55e37760f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760adb0 0x55e37760f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/3384307210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37753a400 con 0x55e37760a3c0 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e3776123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 0x55e37760a7e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009390 tx=0x7f2a4c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d25362997ade67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2a440a2c10 0x7f2a440a50d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009fd0 tx=0x7f2a50013460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a417fa640 1 -- 172.21.15.33:0/3384307210 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c009800 con 0x55e37760a3c0 2024-04-24T23:27:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a417fa640 1 -- 172.21.15.33:0/3384307210 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c036040 con 0x55e37760a3c0 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.621+0000 7f2a417fa640 1 -- 172.21.15.33:0/3384307210 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004550 con 0x55e37760a3c0 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 -- 172.21.15.33:0/3384307210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 msgr2=0x55e37760a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 0x55e37760a7e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009390 tx=0x7f2a4c0093c0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 -- 172.21.15.33:0/3384307210 shutdown_connections 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e3776123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760adb0 0x55e37760f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/3384307210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760a3c0 0x55e37760a7e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 -- 172.21.15.33:0/3384307210 >> 172.21.15.33:0/3384307210 conn(0x55e377605a30 msgr2=0x55e377605e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 -- 172.21.15.33:0/3384307210 shutdown_connections 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.622+0000 7f2a63dfd740 1 -- 172.21.15.33:0/3384307210 wait complete. 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 -- start start 2024-04-24T23:27:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 Processor -- start 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e377614340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760adb0 0x55e377614880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 0x55e37761b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e377612cf0 con 0x55e37760ff20 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e37752a050 con 0x55e37760a3c0 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e377612e70 con 0x55e37760adb0 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a63dfd740 10 client.0 _ll_register_callbacks cb 0x55e377281be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 0x55e37761b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e377614340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 0x55e37761b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45262/0 (socket says 172.21.15.33:45262) 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e377614340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58118/0 (socket says 172.21.15.33:58118) 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5b59e640 1 -- 172.21.15.33:0/564725406 learned_addr learned my addr 172.21.15.33:0/564725406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760adb0 0x55e377614880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760adb0 msgr2=0x55e377614880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760adb0 0x55e377614880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/564725406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 msgr2=0x55e37761b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.623+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 0x55e37761b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a5ad9d640 1 -- 172.21.15.33:0/564725406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3774fbdc0 con 0x55e37760a3c0 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 0x55e37761b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760adb0 0x55e377614880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e377614340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009360 tx=0x7f2a4c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e14cf3c480ffbd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a3eff5640 1 -- 172.21.15.33:0/564725406 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c054060 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a3eff5640 1 -- 172.21.15.33:0/564725406 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2a4c004460 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a4c036040 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a3eff5640 1 -- 172.21.15.33:0/564725406 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a4c05b050 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a63dfd740 10 client.28127 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3772823f0 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.624+0000 7f2a3eff5640 1 -- 172.21.15.33:0/564725406 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c036040 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a3eff5640 1 -- 172.21.15.33:0/564725406 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2a4c02fb80 con 0x55e37760a3c0 2024-04-24T23:27:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a63dfd740 10 client.28127 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a63dfd740 10 client.28127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a63dfd740 20 client.28127 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377620140 0x55e377622590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a63dfd740 4 client.28127 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e37757bbb0 con 0x55e377620140 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.625+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377620140 0x55e377622590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.626+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377620140 0x55e377622590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=277b349ff58bc20c server_cookie=4a1daf689d588054 in_seq=0 out_seq=0 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.626+0000 7f2a3eff5640 10 client.28127 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.626+0000 7f2a3eff5640 1 -- 172.21.15.33:0/564725406 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e37757bbb0 con 0x55e377620140 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.626+0000 7f2a3eff5640 10 client.28127 handle_command_reply: tid=0 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 client.28127 shutdown 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 msgr2=0x55e377614340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e377614340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009360 tx=0x7f2a4c004a40 comp rx=0 tx=0).stop 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 shutdown_connections 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377620140 0x55e377622590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760ff20 0x55e37761b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760adb0 0x55e377614880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a3eff5640 0 client.28127 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/564725406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e377614340 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 >> 172.21.15.33:0/564725406 conn(0x55e377605a30 msgr2=0x55e377604b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 shutdown_connections 2024-04-24T23:27:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 1 -- 172.21.15.33:0/564725406 wait complete. 2024-04-24T23:27:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.627+0000 7f2a63dfd740 20 client.28127 trim_cache size 0 max 16384 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.653+0000 7f2a63dfd740 1 Processor -- start 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.653+0000 7f2a63dfd740 1 -- start start 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.653+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e3776042b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.653+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 0x55e37761b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377613960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b3110 con 0x55e37760adb0 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 0x55e37761b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 0x55e37761b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45278/0 (socket says 172.21.15.33:45278) 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 -- 172.21.15.33:0/1710036363 learned_addr learned my addr 172.21.15.33:0/1710036363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b0980 con 0x55e37760a3c0 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1ae7d0 con 0x55e37760ff20 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377613960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 -- 172.21.15.33:0/1710036363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 msgr2=0x55e377613960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377613960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 -- 172.21.15.33:0/1710036363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 msgr2=0x55e3776042b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:27:16.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e3776042b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.654+0000 7f2a5a59c640 1 -- 172.21.15.33:0/1710036363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37734c7d0 con 0x55e37760adb0 2024-04-24T23:27:16.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.655+0000 7f2a5a59c640 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 0x55e37761b7e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x55e37760b820 tx=0x7f2a50021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237d488c5e57442b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.655+0000 7f2a40ff9640 1 -- 172.21.15.33:0/1710036363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50033070 con 0x55e37760adb0 2024-04-24T23:27:16.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.655+0000 7f2a40ff9640 1 -- 172.21.15.33:0/1710036363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5000fe20 con 0x55e37760adb0 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.655+0000 7f2a40ff9640 1 -- 172.21.15.33:0/1710036363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5002d480 con 0x55e37760adb0 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.656+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 msgr2=0x55e37761b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.656+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 0x55e37761b7e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x55e37760b820 tx=0x7f2a50021000 comp rx=0 tx=0).stop 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.657+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 shutdown_connections 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.657+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377613960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.657+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e37760adb0 0x55e37761b7e0 secure :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x55e37760b820 tx=0x7f2a50021000 comp rx=0 tx=0).stop 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.657+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/1710036363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e3776042b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.657+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 >> 172.21.15.33:0/1710036363 conn(0x55e377605a30 msgr2=0x55e37760c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.661+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 shutdown_connections 2024-04-24T23:27:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.662+0000 7f2a63dfd740 1 -- 172.21.15.33:0/1710036363 wait complete. 2024-04-24T23:27:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.663+0000 7f2a63dfd740 1 -- start start 2024-04-24T23:27:16.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.663+0000 7f2a63dfd740 1 Processor -- start 2024-04-24T23:27:16.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e3776186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377618c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a63dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3776826f0 0x55e377684ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a63dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1aee40 con 0x55e3776826f0 2024-04-24T23:27:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377618c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377618c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56698/0 (socket says 172.21.15.33:56698) 2024-04-24T23:27:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a5a59c640 1 -- 172.21.15.33:0/2284684883 learned_addr learned my addr 172.21.15.33:0/2284684883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3776826f0 0x55e377684ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b02b0 con 0x55e37760a3c0 2024-04-24T23:27:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.664+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1af510 con 0x55e37760ff20 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.665+0000 7f2a63dfd740 10 client.0 _ll_register_callbacks cb 0x7f2a50009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.665+0000 7f2a5b59e640 1 -- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 msgr2=0x55e377618c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.665+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377618c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.665+0000 7f2a5b59e640 1 -- 172.21.15.33:0/2284684883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 msgr2=0x55e3776186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.665+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e3776186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.665+0000 7f2a5b59e640 1 -- 172.21.15.33:0/2284684883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e377685020 con 0x55e3776826f0 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.666+0000 7f2a5b59e640 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3776826f0 0x55e377684ae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e940 tx=0x7f2a5409b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36c38c39e5d911 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.666+0000 7f2a3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.666+0000 7f2a3eff5640 1 -- 172.21.15.33:0/2284684883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5401c040 con 0x55e3776826f0 2024-04-24T23:27:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.666+0000 7f2a3eff5640 1 -- 172.21.15.33:0/2284684883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2a540ccac0 con 0x55e3776826f0 2024-04-24T23:27:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.667+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5000fe20 con 0x55e3776826f0 2024-04-24T23:27:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.668+0000 7f2a3eff5640 1 -- 172.21.15.33:0/2284684883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a54056170 con 0x55e3776826f0 2024-04-24T23:27:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.668+0000 7f2a63dfd740 10 client.18627 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.668+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3776862f0 con 0x55e3776826f0 2024-04-24T23:27:16.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:16 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4057033101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:16.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.669+0000 7f2a3eff5640 1 -- 172.21.15.33:0/2284684883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a540a4380 con 0x55e3776826f0 2024-04-24T23:27:16.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.669+0000 7f2a3eff5640 1 -- 172.21.15.33:0/2284684883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2a5404fce0 con 0x55e3776826f0 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.670+0000 7f2a63dfd740 10 client.18627 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.670+0000 7f2a63dfd740 10 client.18627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.670+0000 7f2a63dfd740 20 client.18627 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.670+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377686ca0 0x55e3776890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.670+0000 7f2a63dfd740 4 client.18627 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.670+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2a5000fe20 con 0x55e377686ca0 2024-04-24T23:27:16.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.671+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377686ca0 0x55e3776890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.672+0000 7f2a5ad9d640 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377686ca0 0x55e3776890c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5da96151bfc47e60 server_cookie=52275d5a6427a687 in_seq=0 out_seq=0 2024-04-24T23:27:16.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.672+0000 7f2a3eff5640 10 client.18627 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:16.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a3eff5640 1 -- 172.21.15.33:0/2284684883 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2a5000fe20 con 0x55e377686ca0 2024-04-24T23:27:16.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a3eff5640 10 client.18627 handle_command_reply: tid=0 2024-04-24T23:27:16.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 client.18627 shutdown 2024-04-24T23:27:16.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3776826f0 msgr2=0x55e377684ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3776826f0 0x55e377684ae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e940 tx=0x7f2a5409b710 comp rx=0 tx=0).stop 2024-04-24T23:27:16.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 shutdown_connections 2024-04-24T23:27:16.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e377686ca0 0x55e3776890c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3776826f0 0x55e377684ae0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a3eff5640 0 client.18627 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37760ff20 0x55e377618c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 --2- 172.21.15.33:0/2284684883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e37760a3c0 0x55e3776186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.673+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 >> 172.21.15.33:0/2284684883 conn(0x55e377605a30 msgr2=0x55e377608910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 shutdown_connections 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a63dfd740 1 -- 172.21.15.33:0/2284684883 wait complete. 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a63dfd740 20 client.18627 trim_cache size 0 max 16384 2024-04-24T23:27:16.676 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:27:16.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2a440a2c10 msgr2=0x7f2a440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a6081a640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2a440a2c10 0x7f2a440a50d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009fd0 tx=0x7f2a50013460 comp rx=0 tx=0).stop 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c102110 msgr2=0x7f2a5c1a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.674+0000 7f2a6081a640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c102110 0x7f2a5c1a5310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e970 tx=0x7f2a5400ee30 comp rx=0 tx=0).stop 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 shutdown_connections 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2a440a2c10 0x7f2a440a50d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a5c102110 0x7f2a5c1a5310 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5c101750 0x7f2a5c1a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 --2- 172.21.15.33:0/2206937033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c100560 0x7f2a5c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 >> 172.21.15.33:0/2206937033 conn(0x7f2a5c0fbaf0 msgr2=0x7f2a5c069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 shutdown_connections 2024-04-24T23:27:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.675+0000 7f2a6081a640 1 -- 172.21.15.33:0/2206937033 wait complete. 2024-04-24T23:27:16.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:16 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4057033101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:16.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:16 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4057033101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:16.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1176250296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 msgr2=0x7f7ac8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1176250296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8102e00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009900 tx=0x7f7abc02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1176250296 shutdown_connections 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1176250296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8103340 0x7f7ac8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1176250296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8102e00 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1176250296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ac8101790 0x7f7ac8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1176250296 >> 172.21.15.33:0/1176250296 conn(0x7f7ac80fcd20 msgr2=0x7f7ac80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1176250296 shutdown_connections 2024-04-24T23:27:16.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1176250296 wait complete. 2024-04-24T23:27:16.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 Processor -- start 2024-04-24T23:27:16.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.840+0000 7f7acd3fb640 1 -- start start 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8101790 0x7f7ac8107510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ac8103340 0x7f7ac8104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac810d3f0 con 0x7f7ac8102980 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac810d270 con 0x7f7ac8101790 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac810d570 con 0x7f7ac8103340 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8101790 0x7f7ac8107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45310/0 (socket says 172.21.15.33:45310) 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 -- 172.21.15.33:0/1340603696 learned_addr learned my addr 172.21.15.33:0/1340603696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ac8103340 0x7f7ac8104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 -- 172.21.15.33:0/1340603696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ac8103340 msgr2=0x7f7ac8104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ac8103340 0x7f7ac8104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 -- 172.21.15.33:0/1340603696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8101790 msgr2=0x7f7ac8107510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8101790 0x7f7ac8107510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 -- 172.21.15.33:0/1340603696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac8104940 con 0x7f7ac8102980 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8101790 0x7f7ac8107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7ac67fc640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8107a50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009230 tx=0x7f7abc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7aaffff640 1 -- 172.21.15.33:0/1340603696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc03f030 con 0x7f7ac8102980 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7aaffff640 1 -- 172.21.15.33:0/1340603696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7abc003ce0 con 0x7f7ac8102980 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.841+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac81a6160 con 0x7f7ac8102980 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.842+0000 7f7aaffff640 1 -- 172.21.15.33:0/1340603696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc035070 con 0x7f7ac8102980 2024-04-24T23:27:16.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.842+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81a6510 con 0x7f7ac8102980 2024-04-24T23:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7aaffff640 1 -- 172.21.15.33:0/1340603696 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7abc042030 con 0x7f7ac8102980 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7ad01df740 1 Processor -- start 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7aaffff640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ab00a2b90 0x7f7ab00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7ad01df740 1 -- start start 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ab00a2b90 0x7f7ab00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7aaffff640 1 -- 172.21.15.33:0/1340603696 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7abc0ef8a0 con 0x7f7ac8102980 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.844+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e453650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453c20 0x55944e458720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e45b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ad01df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55944e3b7670 con 0x55944e453c20 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ad01df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55944e35f970 con 0x55944e453230 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ad01df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55944e3b3630 con 0x55944e458d90 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e453650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453c20 0x55944e458720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e453650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58184/0 (socket says 172.21.15.33:58184) 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/2231095422 learned_addr learned my addr 172.21.15.33:0/2231095422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e45b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/2231095422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 msgr2=0x55944e45b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e45b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/2231095422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453c20 msgr2=0x55944e458720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453c20 0x55944e458720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/2231095422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55944e383400 con 0x55944e453230 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ab00a2b90 0x7f7ab00a5050 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005fd0 tx=0x7f7ac0013b60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e453650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80114c0 tx=0x7f7ab8011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229ad18f39fd6721 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7aad7fa640 1 -- 172.21.15.33:0/2231095422 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8004020 con 0x55944e453230 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7aad7fa640 1 -- 172.21.15.33:0/2231095422 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8005500 con 0x55944e453230 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.845+0000 7f7aad7fa640 1 -- 172.21.15.33:0/2231095422 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8004de0 con 0x55944e453230 2024-04-24T23:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 -- 172.21.15.33:0/2231095422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 msgr2=0x55944e453650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e453650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80114c0 tx=0x7f7ab8011980 comp rx=0 tx=0).stop 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 -- 172.21.15.33:0/2231095422 shutdown_connections 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e45b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453c20 0x55944e458720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2231095422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e453650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 -- 172.21.15.33:0/2231095422 >> 172.21.15.33:0/2231095422 conn(0x55944e44e8a0 msgr2=0x55944e44eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 -- 172.21.15.33:0/2231095422 shutdown_connections 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 -- 172.21.15.33:0/2231095422 wait complete. 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.846+0000 7f7ad01df740 1 -- start start 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 Processor -- start 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e45d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e45d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e458d90 0x55944e464850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55944e45bd40 con 0x55944e458d90 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55944e373050 con 0x55944e453230 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55944e45bec0 con 0x55944e453c20 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e45d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e458d90 0x55944e464850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e45d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58186/0 (socket says 172.21.15.33:58186) 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/3553101236 learned_addr learned my addr 172.21.15.33:0/3553101236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac67fc640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e45d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ad01df740 10 client.0 _ll_register_callbacks cb 0x55944e0cabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 msgr2=0x55944e45d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e45d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/3553101236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e458d90 msgr2=0x55944e464850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e458d90 0x55944e464850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/3553101236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55944e344dc0 con 0x55944e453230 2024-04-24T23:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e458d90 0x55944e464850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e45d2d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8011e80 tx=0x7f7ab80051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05c10b8572ea50c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.847+0000 7f7aaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7aaaff5640 1 -- 172.21.15.33:0/3553101236 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8028030 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab8005500 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7aaaff5640 1 -- 172.21.15.33:0/3553101236 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7ab8016040 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7aaaff5640 1 -- 172.21.15.33:0/3553101236 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ab8032050 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 10 client.28139 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55944e0cb3f0 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7aaaff5640 1 -- 172.21.15.33:0/3553101236 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8005500 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7aaaff5640 1 -- 172.21.15.33:0/3553101236 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7ab8023470 con 0x55944e453230 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 10 client.28139 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 10 client.28139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 20 client.28139 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e46b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 4 client.28139 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55944e3c4bb0 con 0x55944e468e40 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.848+0000 7f7ac67fc640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e46b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.849+0000 7f7ac67fc640 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e46b300 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be354e6242a4574 server_cookie=702effbbfa74dba9 in_seq=0 out_seq=0 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.849+0000 7f7aaaff5640 10 client.28139 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.849+0000 7f7aaaff5640 1 -- 172.21.15.33:0/3553101236 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55944e3c4bb0 con 0x55944e468e40 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.849+0000 7f7aaaff5640 10 client.28139 handle_command_reply: tid=0 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 client.28139 shutdown 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 msgr2=0x55944e45d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e45d2d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8011e80 tx=0x7f7ab80051f0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 shutdown_connections 2024-04-24T23:27:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e46b300 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e458d90 0x55944e464850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e45d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7aaaff5640 0 client.28139 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 --2- 172.21.15.33:0/3553101236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453230 0x55944e45d2d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 >> 172.21.15.33:0/3553101236 conn(0x55944e44e8a0 msgr2=0x55944e44db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 shutdown_connections 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 1 -- 172.21.15.33:0/3553101236 wait complete. 2024-04-24T23:27:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.850+0000 7f7ad01df740 20 client.28139 trim_cache size 0 max 16384 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 Processor -- start 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 -- start start 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e44dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e464810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e458d90 0x55944e451480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b3380 con 0x55944e453230 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b0bf0 con 0x55944e458d90 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ad01df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81aea40 con 0x55944e453c20 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e44dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e458d90 0x55944e451480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e44dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45340/0 (socket says 172.21.15.33:45340) 2024-04-24T23:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e464810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/1949254274 learned_addr learned my addr 172.21.15.33:0/1949254274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/1949254274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 msgr2=0x55944e464810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e464810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/1949254274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e458d90 msgr2=0x55944e451480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e458d90 0x55944e451480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/1949254274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55944e1957d0 con 0x55944e453230 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.865+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e458d90 0x55944e451480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.866+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e44dd80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80059f0 tx=0x7f7ab8004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e575b7f394ef31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.866+0000 7f7aacff9640 1 -- 172.21.15.33:0/1949254274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8028040 con 0x55944e453230 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.866+0000 7f7aacff9640 1 -- 172.21.15.33:0/1949254274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8023670 con 0x55944e453230 2024-04-24T23:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.866+0000 7f7aacff9640 1 -- 172.21.15.33:0/1949254274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801d040 con 0x55944e453230 2024-04-24T23:27:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.866+0000 7f7ad01df740 1 -- 172.21.15.33:0/1949254274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 msgr2=0x55944e44dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.866+0000 7f7ad01df740 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e44dd80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80059f0 tx=0x7f7ab8004e60 comp rx=0 tx=0).stop 2024-04-24T23:27:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- 172.21.15.33:0/1949254274 shutdown_connections 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e458d90 0x55944e451480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e453c20 0x55944e464810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 --2- 172.21.15.33:0/1949254274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e44dd80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- 172.21.15.33:0/1949254274 >> 172.21.15.33:0/1949254274 conn(0x55944e44e8a0 msgr2=0x55944e454f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- 172.21.15.33:0/1949254274 shutdown_connections 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- 172.21.15.33:0/1949254274 wait complete. 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- start start 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 Processor -- start 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e4645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453c20 0x55944e461050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e461590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81af0b0 con 0x55944e453230 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b0520 con 0x55944e453c20 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81af780 con 0x55944e458d90 2024-04-24T23:27:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ad01df740 10 client.0 _ll_register_callbacks cb 0x7f7abc003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e4645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ac67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453c20 0x55944e461050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ac6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e4645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45350/0 (socket says 172.21.15.33:45350) 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ac6ffd640 1 -- 172.21.15.33:0/2392516334 learned_addr learned my addr 172.21.15.33:0/2392516334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.867+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e461590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac77fe640 1 -- 172.21.15.33:0/2392516334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453c20 msgr2=0x55944e461050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453c20 0x55944e461050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac77fe640 1 -- 172.21.15.33:0/2392516334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 msgr2=0x55944e4645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e4645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac77fe640 1 -- 172.21.15.33:0/2392516334 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55944e4cd430 con 0x55944e458d90 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e4645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ac77fe640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e461590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55944e4543d0 tx=0x7f7ac0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0005d7eeff679f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7aaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:27:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7aaaff5640 1 -- 172.21.15.33:0/2392516334 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00046a0 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab8023670 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7aaaff5640 1 -- 172.21.15.33:0/2392516334 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7ac0004b20 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7aaaff5640 1 -- 172.21.15.33:0/2392516334 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0004840 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ad01df740 10 client.28096 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.868+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55944e4ce5e0 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7aaaff5640 1 -- 172.21.15.33:0/2392516334 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8023670 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7aaaff5640 1 -- 172.21.15.33:0/2392516334 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7ac0029070 con 0x55944e458d90 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ad01df740 10 client.28096 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ad01df740 10 client.28096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ad01df740 20 client.28096 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e4d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ad01df740 4 client.28096 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55944e3bef00 con 0x55944e468e40 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.869+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e4d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7aaaff5640 10 client.28096 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ac6ffd640 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e4d10b0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80bef7e87011b0aa server_cookie=19c80f88d3cb4a3 in_seq=0 out_seq=0 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7aaaff5640 1 -- 172.21.15.33:0/2392516334 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55944e3bef00 con 0x55944e468e40 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7aaaff5640 10 client.28096 handle_command_reply: tid=0 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 client.28096 shutdown 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 msgr2=0x55944e461590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e461590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55944e4543d0 tx=0x7f7ac0004ab0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 shutdown_connections 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55944e468e40 0x55944e4d10b0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7aaaff5640 0 client.28096 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55944e458d90 0x55944e461590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55944e453c20 0x55944e461050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 --2- 172.21.15.33:0/2392516334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55944e453230 0x55944e4645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.870+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 >> 172.21.15.33:0/2392516334 conn(0x55944e44e8a0 msgr2=0x55944e451480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.871+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 shutdown_connections 2024-04-24T23:27:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.871+0000 7f7ad01df740 1 -- 172.21.15.33:0/2392516334 wait complete. 2024-04-24T23:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.871+0000 7f7ad01df740 20 client.28096 trim_cache size 0 max 16384 2024-04-24T23:27:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ab00a2b90 msgr2=0x7f7ab00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ab00a2b90 0x7f7ab00a5050 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005fd0 tx=0x7f7ac0013b60 comp rx=0 tx=0).stop 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 msgr2=0x7f7ac8107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8107a50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009230 tx=0x7f7abc0042f0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 shutdown_connections 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7ab00a2b90 0x7f7ab00a5050 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ac8103340 0x7f7ac8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ac8102980 0x7f7ac8107a50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 --2- 172.21.15.33:0/1340603696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac8101790 0x7f7ac8107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 >> 172.21.15.33:0/1340603696 conn(0x7f7ac80fcd20 msgr2=0x7f7ac8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 shutdown_connections 2024-04-24T23:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:16.873+0000 7f7acd3fb640 1 -- 172.21.15.33:0/1340603696 wait complete. 2024-04-24T23:27:16.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 37681 27.36 MB/sec warmup 81 sec latency 160.995 ms 2024-04-24T23:27:17.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:17 smithi038 ceph-mon[32450]: pgmap v809: 129 pgs: 129 active+clean; 507 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 448 op/s 2024-04-24T23:27:17.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:17 smithi117 ceph-mon[32529]: pgmap v809: 129 pgs: 129 active+clean; 507 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 448 op/s 2024-04-24T23:27:17.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:17 smithi033 ceph-mon[29201]: pgmap v809: 129 pgs: 129 active+clean; 507 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 448 op/s 2024-04-24T23:27:17.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 38022 27.22 MB/sec warmup 82 sec latency 61.002 ms 2024-04-24T23:27:17.891 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:27:17.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:17.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:27:18.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 -- 172.21.15.33:0/2405681446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8100680 msgr2=0x7f15b8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 --2- 172.21.15.33:0/2405681446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8100680 0x7f15b8100aa0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009900 tx=0x7f15ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 -- 172.21.15.33:0/2405681446 shutdown_connections 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 --2- 172.21.15.33:0/2405681446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b8102230 0x7f15b8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 --2- 172.21.15.33:0/2405681446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15b8101870 0x7f15b8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 --2- 172.21.15.33:0/2405681446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8100680 0x7f15b8100aa0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 -- 172.21.15.33:0/2405681446 >> 172.21.15.33:0/2405681446 conn(0x7f15b80fbbf0 msgr2=0x7f15b80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 -- 172.21.15.33:0/2405681446 shutdown_connections 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.008+0000 7f15bedee640 1 -- 172.21.15.33:0/2405681446 wait complete. 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 Processor -- start 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 -- start start 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15b8100680 0x7f15b8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b8101870 0x7f15b8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 0x7f15b819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15b810c2d0 con 0x7f15b8102230 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15b810c150 con 0x7f15b8101870 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bedee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15b810c450 con 0x7f15b8100680 2024-04-24T23:27:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bd38b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 0x7f15b819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bd38b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 0x7f15b819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45376/0 (socket says 172.21.15.33:45376) 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bd38b640 1 -- 172.21.15.33:0/1700345099 learned_addr learned my addr 172.21.15.33:0/1700345099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15b8100680 0x7f15b8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.009+0000 7f15b7fff640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b8101870 0x7f15b8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bd38b640 1 -- 172.21.15.33:0/1700345099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15b8100680 msgr2=0x7f15b8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bd38b640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15b8100680 0x7f15b8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bd38b640 1 -- 172.21.15.33:0/1700345099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b8101870 msgr2=0x7f15b8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bd38b640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b8101870 0x7f15b8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bd38b640 1 -- 172.21.15.33:0/1700345099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b819f740 con 0x7f15b8102230 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bd38b640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 0x7f15b819f0d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f15b000eb10 tx=0x7f15b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15b5ffb640 1 -- 172.21.15.33:0/1700345099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0004b20 con 0x7f15b8102230 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15b5ffb640 1 -- 172.21.15.33:0/1700345099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b0004cc0 con 0x7f15b8102230 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b819fa30 con 0x7f15b8102230 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15b5ffb640 1 -- 172.21.15.33:0/1700345099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0010760 con 0x7f15b8102230 2024-04-24T23:27:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.010+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b819ff80 con 0x7f15b8102230 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b5ffb640 1 -- 172.21.15.33:0/1700345099 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f15b0005020 con 0x7f15b8102230 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 Processor -- start 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 -- start start 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2ff01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b2ff5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff58f0 0x5580b2ff7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580b2f532c0 con 0x5580b2fefd90 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5580b2efb910 con 0x5580b2ff0780 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15c1bde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5580b2f4f280 con 0x5580b2ff58f0 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b5ffb640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15a80a2c10 0x7f15a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2ff01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bcb8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b2ff5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2ff01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45380/0 (socket says 172.21.15.33:45380) 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b7fff640 1 -- 172.21.15.33:0/1192328028 learned_addr learned my addr 172.21.15.33:0/1192328028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b7fff640 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff58f0 0x5580b2ff7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bd38b640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15a80a2c10 0x7f15a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15b5ffb640 1 -- 172.21.15.33:0/1700345099 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15b00cf900 con 0x7f15b8102230 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1192328028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff58f0 msgr2=0x5580b2ff7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff58f0 0x5580b2ff7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1192328028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 msgr2=0x5580b2ff01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2ff01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.013+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1192328028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b2f0cfb0 con 0x5580b2ff0780 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.014+0000 7f15b7fff640 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff58f0 0x5580b2ff7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.014+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b2ff5280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009230 tx=0x7f15ac005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fc2ce37de02805 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.014+0000 7f15bd38b640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15a80a2c10 0x7f15a80a50d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f15b0098230 tx=0x7f15b0066a50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.014+0000 7f15a2ffd640 1 -- 172.21.15.33:0/1192328028 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac040070 con 0x5580b2ff0780 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.014+0000 7f15a2ffd640 1 -- 172.21.15.33:0/1192328028 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac004030 con 0x5580b2ff0780 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.014+0000 7f15a2ffd640 1 -- 172.21.15.33:0/1192328028 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac035040 con 0x5580b2ff0780 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 -- 172.21.15.33:0/1192328028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 msgr2=0x5580b2ff5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b2ff5280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009230 tx=0x7f15ac005e20 comp rx=0 tx=0).stop 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 -- 172.21.15.33:0/1192328028 shutdown_connections 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff58f0 0x5580b2ff7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b2ff5280 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1192328028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2ff01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 -- 172.21.15.33:0/1192328028 >> 172.21.15.33:0/1192328028 conn(0x5580b2fb7950 msgr2=0x5580b2feb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 -- 172.21.15.33:0/1192328028 shutdown_connections 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 -- 172.21.15.33:0/1192328028 wait complete. 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 -- start start 2024-04-24T23:27:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.015+0000 7f15c1bde740 1 Processor -- start 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2ffa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 0x5580b2ffa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3001700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580b2ff8840 con 0x5580b2ff0780 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5580b2f0ee90 con 0x5580b2ff58f0 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5580b2ff89c0 con 0x5580b2fefd90 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 10 client.0 _ll_register_callbacks cb 0x5580b2f4a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 0x5580b2ffa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 0x5580b2ffa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45392/0 (socket says 172.21.15.33:45392) 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2ffa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 -- 172.21.15.33:0/2010558835 learned_addr learned my addr 172.21.15.33:0/2010558835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15b7fff640 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3001700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 -- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 msgr2=0x5580b2ffa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2ffa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 -- 172.21.15.33:0/2010558835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 msgr2=0x5580b3001700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3001700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 -- 172.21.15.33:0/2010558835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b2d2f7e0 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 0x5580b2ffa6c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005ab0 tx=0x7f15ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a5e9b4008f0af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f1597fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f1597fff640 1 -- 172.21.15.33:0/2010558835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac04c070 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f1597fff640 1 -- 172.21.15.33:0/2010558835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15ac0043e0 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.016+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ac004030 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f1597fff640 1 -- 172.21.15.33:0/2010558835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15ac058050 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 10 client.18657 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580b2d8ff70 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f1597fff640 1 -- 172.21.15.33:0/2010558835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac004030 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f1597fff640 1 -- 172.21.15.33:0/2010558835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f15ac004780 con 0x5580b2ff0780 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 10 client.18657 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 10 client.18657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 20 client.18657 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b3005860 0x5580b3007c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 4 client.18657 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b2f275a0 con 0x5580b3005860 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.017+0000 7f15b7fff640 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b3005860 0x5580b3007c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15b7fff640 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b3005860 0x5580b3007c50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9272c0530d9937c server_cookie=97874fb48ae8cea6 in_seq=0 out_seq=0 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f1597fff640 10 client.18657 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f1597fff640 1 -- 172.21.15.33:0/2010558835 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580b2f275a0 con 0x5580b3005860 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f1597fff640 10 client.18657 handle_command_reply: tid=0 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 client.18657 shutdown 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 msgr2=0x5580b2ffa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 0x5580b2ffa6c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005ab0 tx=0x7f15ac009260 comp rx=0 tx=0).stop 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 shutdown_connections 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b3005860 0x5580b3007c50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3001700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff0780 0x5580b2ffa6c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f1597fff640 0 client.18657 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 --2- 172.21.15.33:0/2010558835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2ffa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.018+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 >> 172.21.15.33:0/2010558835 conn(0x5580b2fb7950 msgr2=0x5580b2fea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.019+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 shutdown_connections 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.019+0000 7f15c1bde740 1 -- 172.21.15.33:0/2010558835 wait complete. 2024-04-24T23:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.019+0000 7f15c1bde740 20 client.18657 trim_cache size 0 max 16384 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 Processor -- start 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 -- start start 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2fea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b30016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff58f0 0x5580b300c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81acd10 con 0x5580b2ff58f0 2024-04-24T23:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81aa580 con 0x5580b2ff0780 2024-04-24T23:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15c1bde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81a83d0 con 0x5580b2fefd90 2024-04-24T23:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15bcb8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b30016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2fea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15bcb8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b30016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58232/0 (socket says 172.21.15.33:58232) 2024-04-24T23:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2fea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56784/0 (socket says 172.21.15.33:56784) 2024-04-24T23:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1260801 learned_addr learned my addr 172.21.15.33:0/1260801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.039+0000 7f15b7fff640 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff58f0 0x5580b300c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1260801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 msgr2=0x5580b2fea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2fea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1260801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff58f0 msgr2=0x5580b300c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff58f0 0x5580b300c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15bcb8a640 1 -- 172.21.15.33:0/1260801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b2c27170 con 0x5580b2ff0780 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b30016c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15ac02a930 tx=0x7f15ac004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed5f231a4c962cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15a27fc640 1 -- 172.21.15.33:0/1260801 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac04c070 con 0x5580b2ff0780 2024-04-24T23:27:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15a27fc640 1 -- 172.21.15.33:0/1260801 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac009520 con 0x5580b2ff0780 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.040+0000 7f15a27fc640 1 -- 172.21.15.33:0/1260801 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac040050 con 0x5580b2ff0780 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 -- 172.21.15.33:0/1260801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 msgr2=0x5580b30016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b30016c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15ac02a930 tx=0x7f15ac004a10 comp rx=0 tx=0).stop 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 -- 172.21.15.33:0/1260801 shutdown_connections 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2ff58f0 0x5580b300c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff0780 0x5580b30016c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 --2- 172.21.15.33:0/1260801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2fefd90 0x5580b2fea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 -- 172.21.15.33:0/1260801 >> 172.21.15.33:0/1260801 conn(0x5580b2fb7950 msgr2=0x5580b2fea0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 -- 172.21.15.33:0/1260801 shutdown_connections 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 -- 172.21.15.33:0/1260801 wait complete. 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 -- start start 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.041+0000 7f15c1bde740 1 Processor -- start 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2fff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff0780 0x5580b2fffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3000bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81a8a40 con 0x5580b2fefd90 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81a9eb0 con 0x5580b2ff58f0 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81a9110 con 0x5580b2ff0780 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15c1bde740 10 client.0 _ll_register_callbacks cb 0x7f15a4009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2fff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2fff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45410/0 (socket says 172.21.15.33:45410) 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff0780 0x5580b2fffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15b7fff640 1 -- 172.21.15.33:0/360352509 learned_addr learned my addr 172.21.15.33:0/360352509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15b7fff640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3000bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 -- 172.21.15.33:0/360352509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 msgr2=0x5580b3000bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3000bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 -- 172.21.15.33:0/360352509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 msgr2=0x5580b2fff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2fff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 -- 172.21.15.33:0/360352509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b30011a0 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15b7fff640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2fff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f15bcb8a640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff0780 0x5580b2fffba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5580b3002710 tx=0x7f15ac009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6685cb201cf1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f1597fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.042+0000 7f1597fff640 1 -- 172.21.15.33:0/360352509 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac040070 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f1597fff640 1 -- 172.21.15.33:0/360352509 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15ac0316b0 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ac009520 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f1597fff640 1 -- 172.21.15.33:0/360352509 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15ac06b050 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 10 client.28114 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580b306c520 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f1597fff640 1 -- 172.21.15.33:0/360352509 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac009520 con 0x5580b2ff0780 2024-04-24T23:27:18.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f1597fff640 1 -- 172.21.15.33:0/360352509 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f15ac051c20 con 0x5580b2ff0780 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 10 client.28114 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 10 client.28114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 20 client.28114 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b306cea0 0x5580b306f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 4 client.28114 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5580b2f60800 con 0x5580b306cea0 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.043+0000 7f15b7fff640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b306cea0 0x5580b306f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f1597fff640 10 client.28114 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f15b7fff640 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b306cea0 0x5580b306f2c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f2cc489ebb8618 server_cookie=a491ed4e3ea34c6 in_seq=0 out_seq=0 2024-04-24T23:27:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f1597fff640 1 -- 172.21.15.33:0/360352509 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5580b2f60800 con 0x5580b306cea0 2024-04-24T23:27:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f1597fff640 10 client.28114 handle_command_reply: tid=0 2024-04-24T23:27:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f15c1bde740 1 client.28114 shutdown 2024-04-24T23:27:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff0780 msgr2=0x5580b2fffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.044+0000 7f15c1bde740 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff0780 0x5580b2fffba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5580b3002710 tx=0x7f15ac009230 comp rx=0 tx=0).stop 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 shutdown_connections 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5580b306cea0 0x5580b306f2c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5580b2ff58f0 0x5580b3000bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f1597fff640 0 client.28114 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580b2ff0780 0x5580b2fffba0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 --2- 172.21.15.33:0/360352509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580b2fefd90 0x5580b2fff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 >> 172.21.15.33:0/360352509 conn(0x5580b2fb7950 msgr2=0x5580b2fea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 shutdown_connections 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 1 -- 172.21.15.33:0/360352509 wait complete. 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15c1bde740 20 client.28114 trim_cache size 0 max 16384 2024-04-24T23:27:18.053 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:27:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:27:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "58880ea5-3a11-4338-b766-c862429cec4c", 2024-04-24T23:27:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:27:18.054 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15a80a2c10 msgr2=0x7f15a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15a80a2c10 0x7f15a80a50d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f15b0098230 tx=0x7f15b0066a50 comp rx=0 tx=0).stop 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 msgr2=0x7f15b819f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 0x7f15b819f0d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f15b000eb10 tx=0x7f15b000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 shutdown_connections 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15a80a2c10 0x7f15a80a50d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b8102230 0x7f15b819f0d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b8101870 0x7f15b8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 --2- 172.21.15.33:0/1700345099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15b8100680 0x7f15b8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.045+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 >> 172.21.15.33:0/1700345099 conn(0x7f15b80fbbf0 msgr2=0x7f15b80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.046+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 shutdown_connections 2024-04-24T23:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.046+0000 7f15bedee640 1 -- 172.21.15.33:0/1700345099 wait complete. 2024-04-24T23:27:18.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:27:18.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 -- 172.21.15.33:0/2897324578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c101870 msgr2=0x7f8d3c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 --2- 172.21.15.33:0/2897324578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c101870 0x7f8d3c101cf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009900 tx=0x7f8d2c02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 -- 172.21.15.33:0/2897324578 shutdown_connections 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 --2- 172.21.15.33:0/2897324578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c102230 0x7f8d3c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 --2- 172.21.15.33:0/2897324578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c101870 0x7f8d3c101cf0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 --2- 172.21.15.33:0/2897324578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 -- 172.21.15.33:0/2897324578 >> 172.21.15.33:0/2897324578 conn(0x7f8d3c0fbbf0 msgr2=0x7f8d3c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 -- 172.21.15.33:0/2897324578 shutdown_connections 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.173+0000 7f8d43c64640 1 -- 172.21.15.33:0/2897324578 wait complete. 2024-04-24T23:27:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 Processor -- start 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 -- start start 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c101870 0x7f8d3c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 0x7f8d3c1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c10c300 con 0x7f8d3c102230 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c10c180 con 0x7f8d3c100680 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d43c64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c10c480 con 0x7f8d3c101870 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d42201640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 0x7f8d3c1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58266/0 (socket says 172.21.15.33:58266) 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d41a00640 1 -- 172.21.15.33:0/1522860732 learned_addr learned my addr 172.21.15.33:0/1522860732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d42201640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 0x7f8d3c1a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45428/0 (socket says 172.21.15.33:45428) 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.174+0000 7f8d411ff640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c101870 0x7f8d3c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d41a00640 1 -- 172.21.15.33:0/1522860732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c101870 msgr2=0x7f8d3c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d41a00640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c101870 0x7f8d3c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d41a00640 1 -- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 msgr2=0x7f8d3c1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d41a00640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 0x7f8d3c1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d41a00640 1 -- 172.21.15.33:0/1522860732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3c1a1970 con 0x7f8d3c100680 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d411ff640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c101870 0x7f8d3c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d42201640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 0x7f8d3c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d41a00640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c069440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9c0 tx=0x7f8d3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d32ffd640 1 -- 172.21.15.33:0/1522860732 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800cd80 con 0x7f8d3c100680 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3c1a1c00 con 0x7f8d3c100680 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d32ffd640 1 -- 172.21.15.33:0/1522860732 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d380035f0 con 0x7f8d3c100680 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d32ffd640 1 -- 172.21.15.33:0/1522860732 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38010690 con 0x7f8d3c100680 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.175+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3c1a2120 con 0x7f8d3c100680 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.177+0000 7f8d32ffd640 1 -- 172.21.15.33:0/1522860732 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d380037a0 con 0x7f8d3c100680 2024-04-24T23:27:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.177+0000 7f8d46975740 1 Processor -- start 2024-04-24T23:27:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 -- start start 2024-04-24T23:27:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc976660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976c30 0x55fdcc97b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc97e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdcc8da6b0 con 0x55fdcc976c30 2024-04-24T23:27:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fdcc882880 con 0x55fdcc976240 2024-04-24T23:27:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d46975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fdcc8d6670 con 0x55fdcc97bda0 2024-04-24T23:27:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d42201640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc976660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976c30 0x55fdcc97b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d42201640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc976660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:58282/0 (socket says 172.21.15.33:58282) 2024-04-24T23:27:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d42201640 1 -- 172.21.15.33:0/3794344728 learned_addr learned my addr 172.21.15.33:0/3794344728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976c30 0x55fdcc97b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45436/0 (socket says 172.21.15.33:45436) 2024-04-24T23:27:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d411ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc97e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d32ffd640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d280a2bc0 0x7f8d280a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.178+0000 7f8d42201640 1 -- 172.21.15.33:0/3794344728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 msgr2=0x55fdcc97e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d42201640 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc97e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d42201640 1 -- 172.21.15.33:0/3794344728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976c30 msgr2=0x55fdcc97b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d42201640 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976c30 0x55fdcc97b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d42201640 1 -- 172.21.15.33:0/3794344728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdcc6b87d0 con 0x55fdcc976240 2024-04-24T23:27:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d32ffd640 1 -- 172.21.15.33:0/1522860732 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8d380cf820 con 0x7f8d3c100680 2024-04-24T23:27:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d42201640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d280a2bc0 0x7f8d280a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d42201640 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc976660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d340108d0 tx=0x7f8d34010d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4775d3bf736b83c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d27fff640 1 -- 172.21.15.33:0/3794344728 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400ee50 con 0x55fdcc976240 2024-04-24T23:27:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d27fff640 1 -- 172.21.15.33:0/3794344728 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3401ddb0 con 0x55fdcc976240 2024-04-24T23:27:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.179+0000 7f8d27fff640 1 -- 172.21.15.33:0/3794344728 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34011c60 con 0x55fdcc976240 2024-04-24T23:27:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 -- 172.21.15.33:0/3794344728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 msgr2=0x55fdcc976660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d42201640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d280a2bc0 0x7f8d280a5080 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d34021000 tx=0x7f8d3401e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc976660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d340108d0 tx=0x7f8d34010d90 comp rx=0 tx=0).stop 2024-04-24T23:27:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 -- 172.21.15.33:0/3794344728 shutdown_connections 2024-04-24T23:27:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc97e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976c30 0x55fdcc97b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 --2- 172.21.15.33:0/3794344728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc976660 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 -- 172.21.15.33:0/3794344728 >> 172.21.15.33:0/3794344728 conn(0x55fdcc9718b0 msgr2=0x55fdcc971cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 -- 172.21.15.33:0/3794344728 shutdown_connections 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 -- 172.21.15.33:0/3794344728 wait complete. 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 -- start start 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.180+0000 7f8d46975740 1 Processor -- start 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc980630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc980b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc97bda0 0x55fdcc987bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdcc97ed50 con 0x55fdcc97bda0 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fdcc895f50 con 0x55fdcc976240 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fdcc97eed0 con 0x55fdcc976c30 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d46975740 10 client.0 _ll_register_callbacks cb 0x55fdcc8d1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d411ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc980b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc97bda0 0x55fdcc987bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc980630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d411ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc980b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56832/0 (socket says 172.21.15.33:56832) 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d411ff640 1 -- 172.21.15.33:0/1400592811 learned_addr learned my addr 172.21.15.33:0/1400592811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 -- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 msgr2=0x55fdcc980b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc980b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 -- 172.21.15.33:0/1400592811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 msgr2=0x55fdcc980630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc980630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 -- 172.21.15.33:0/1400592811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdcc5ee3f0 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d41a00640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc980630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d411ff640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc980b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d42201640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc97bda0 0x55fdcc987bb0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e6c0 tx=0x7f8d34011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ed025c1cd0079 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.181+0000 7f8d257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d257fa640 1 -- 172.21.15.33:0/1400592811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400ee50 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3401ddb0 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d257fa640 1 -- 172.21.15.33:0/1400592811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8d3401d850 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d257fa640 1 -- 172.21.15.33:0/1400592811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d34010400 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 10 client.18675 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdcc5ae0f0 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d257fa640 1 -- 172.21.15.33:0/1400592811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3401ddb0 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d257fa640 1 -- 172.21.15.33:0/1400592811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8d3402f070 con 0x55fdcc97bda0 2024-04-24T23:27:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 10 client.18675 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 20 client.18675 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc98e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 4 client.18675 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.182+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdcc8e7bf0 con 0x55fdcc98c1d0 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.183+0000 7f8d41a00640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc98e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.183+0000 7f8d257fa640 10 client.18675 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.183+0000 7f8d41a00640 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc98e620 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ca27cc28ce96a6 server_cookie=bb16ae81afddc4ad in_seq=0 out_seq=0 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d257fa640 1 -- 172.21.15.33:0/1400592811 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdcc8e7bf0 con 0x55fdcc98c1d0 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d257fa640 10 client.18675 handle_command_reply: tid=0 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 client.18675 shutdown 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc97bda0 msgr2=0x55fdcc987bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc97bda0 0x55fdcc987bb0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e6c0 tx=0x7f8d34011dc0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 shutdown_connections 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc98e620 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc97bda0 0x55fdcc987bb0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc980b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d257fa640 0 client.18675 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 --2- 172.21.15.33:0/1400592811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976240 0x55fdcc980630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 >> 172.21.15.33:0/1400592811 conn(0x55fdcc9718b0 msgr2=0x55fdcc970af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 shutdown_connections 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 1 -- 172.21.15.33:0/1400592811 wait complete. 2024-04-24T23:27:18.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.184+0000 7f8d46975740 20 client.18675 trim_cache size 0 max 16384 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.204+0000 7f8d46975740 1 Processor -- start 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.204+0000 7f8d46975740 1 -- start start 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d46975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc970260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d46975740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976c30 0x55fdcc987b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d46975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc9707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d46975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1aef80 con 0x55fdcc976240 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d46975740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1ac7f0 con 0x55fdcc976c30 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d46975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1aa640 con 0x55fdcc97bda0 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc970260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc970260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45462/0 (socket says 172.21.15.33:45462) 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 -- 172.21.15.33:0/3572659150 learned_addr learned my addr 172.21.15.33:0/3572659150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d411ff640 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976c30 0x55fdcc987b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d42201640 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc9707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 -- 172.21.15.33:0/3572659150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 msgr2=0x55fdcc9707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc9707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 -- 172.21.15.33:0/3572659150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976c30 msgr2=0x55fdcc987b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976c30 0x55fdcc987b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 -- 172.21.15.33:0/3572659150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdcc96d2a0 con 0x55fdcc976240 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d41a00640 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc970260 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8d380ccf30 tx=0x7f8d38003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2dac2fc25272a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.205+0000 7f8d277fe640 1 -- 172.21.15.33:0/3572659150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d380d5020 con 0x55fdcc976240 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d277fe640 1 -- 172.21.15.33:0/3572659150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3809c390 con 0x55fdcc976240 2024-04-24T23:27:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d277fe640 1 -- 172.21.15.33:0/3572659150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3809b0e0 con 0x55fdcc976240 2024-04-24T23:27:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 -- 172.21.15.33:0/3572659150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 msgr2=0x55fdcc970260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc970260 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8d380ccf30 tx=0x7f8d38003a90 comp rx=0 tx=0).stop 2024-04-24T23:27:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 -- 172.21.15.33:0/3572659150 shutdown_connections 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc97bda0 0x55fdcc9707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc976c30 0x55fdcc987b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 --2- 172.21.15.33:0/3572659150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc970260 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 -- 172.21.15.33:0/3572659150 >> 172.21.15.33:0/3572659150 conn(0x55fdcc9718b0 msgr2=0x55fdcc977ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 -- 172.21.15.33:0/3572659150 shutdown_connections 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.206+0000 7f8d46975740 1 -- 172.21.15.33:0/3572659150 wait complete. 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 -- start start 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 Processor -- start 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc9847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc984d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc97bda0 0x55fdcc9f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1aacb0 con 0x55fdcc976240 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1ac120 con 0x55fdcc97bda0 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1ab380 con 0x55fdcc976c30 2024-04-24T23:27:18.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d46975740 10 client.0 _ll_register_callbacks cb 0x7f8d34020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc9847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc9847e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45466/0 (socket says 172.21.15.33:45466) 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d41a00640 1 -- 172.21.15.33:0/180261853 learned_addr learned my addr 172.21.15.33:0/180261853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d411ff640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc984d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.207+0000 7f8d42201640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc97bda0 0x55fdcc9f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d41a00640 1 -- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 msgr2=0x55fdcc984d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d41a00640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc984d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d41a00640 1 -- 172.21.15.33:0/180261853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc97bda0 msgr2=0x55fdcc9f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d41a00640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc97bda0 0x55fdcc9f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d41a00640 1 -- 172.21.15.33:0/180261853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdcc987df0 con 0x55fdcc976240 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d42201640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc97bda0 0x55fdcc9f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d41a00640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc9847e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8d38098660 tx=0x7f8d3809bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdd9e59801b67e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d257fa640 1 -- 172.21.15.33:0/180261853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d380d5050 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d257fa640 1 -- 172.21.15.33:0/180261853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8d3809b0e0 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3809c390 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d257fa640 1 -- 172.21.15.33:0/180261853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d3809bc90 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d46975740 10 client.18681 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdcc9f1d90 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d257fa640 1 -- 172.21.15.33:0/180261853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3809c390 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d257fa640 1 -- 172.21.15.33:0/180261853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8d3809ab50 con 0x55fdcc976240 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d46975740 10 client.18681 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.208+0000 7f8d46975740 10 client.18681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d46975740 20 client.18681 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d46975740 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc9f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d46975740 4 client.18681 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fdcc8e1f40 con 0x55fdcc98c1d0 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d411ff640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc9f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d411ff640 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc9f4760 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e46518b9c10524e server_cookie=a2a37187a659dbaf in_seq=0 out_seq=0 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.209+0000 7f8d257fa640 10 client.18681 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.210+0000 7f8d257fa640 1 -- 172.21.15.33:0/180261853 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fdcc8e1f40 con 0x55fdcc98c1d0 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.210+0000 7f8d257fa640 10 client.18681 handle_command_reply: tid=0 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.210+0000 7f8d46975740 1 client.18681 shutdown 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 msgr2=0x55fdcc9847e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc9847e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8d38098660 tx=0x7f8d3809bb70 comp rx=0 tx=0).stop 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 shutdown_connections 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55fdcc98c1d0 0x55fdcc9f4760 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdcc97bda0 0x55fdcc9f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcc976c30 0x55fdcc984d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d257fa640 0 client.18681 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 --2- 172.21.15.33:0/180261853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdcc976240 0x55fdcc9847e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 >> 172.21.15.33:0/180261853 conn(0x55fdcc9718b0 msgr2=0x55fdcc970260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 shutdown_connections 2024-04-24T23:27:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 1 -- 172.21.15.33:0/180261853 wait complete. 2024-04-24T23:27:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d46975740 20 client.18681 trim_cache size 0 max 16384 2024-04-24T23:27:18.217 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:27:18.217 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:27:18.217 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:27:18.218 INFO:teuthology.orchestra.run.smithi033.stdout: "58880ea5-3a11-4338-b766-c862429cec4c": { 2024-04-24T23:27:18.218 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:27:18.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "58880ea5-3a11-4338-b766-c862429cec4c", 2024-04-24T23:27:18.218 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:27:18.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:27:18.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:27:18.219 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:27:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d280a2bc0 msgr2=0x7f8d280a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d43c64640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d280a2bc0 0x7f8d280a5080 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d34021000 tx=0x7f8d3401e000 comp rx=0 tx=0).stop 2024-04-24T23:27:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 msgr2=0x7f8d3c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.211+0000 7f8d43c64640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c069440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9c0 tx=0x7f8d3800ee80 comp rx=0 tx=0).stop 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 shutdown_connections 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8d280a2bc0 0x7f8d280a5080 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c102230 0x7f8d3c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c101870 0x7f8d3c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 --2- 172.21.15.33:0/1522860732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d3c100680 0x7f8d3c069440 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 >> 172.21.15.33:0/1522860732 conn(0x7f8d3c0fbbf0 msgr2=0x7f8d3c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 shutdown_connections 2024-04-24T23:27:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:18.212+0000 7f8d43c64640 1 -- 172.21.15.33:0/1522860732 wait complete. 2024-04-24T23:27:18.226 INFO:tasks.cephfs.filesystem:scrub status for tag:58880ea5-3a11-4338-b766-c862429cec4c - {'path': '/', 'tag': '58880ea5-3a11-4338-b766-c862429cec4c', 'options': 'recursive,force'} 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:18.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:18.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:18 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:27:18.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:18.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:18.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:18.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:18 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:27:18.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:18.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:18.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:18 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:18.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 38325 27.04 MB/sec warmup 83 sec latency 158.390 ms 2024-04-24T23:27:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:19 smithi038 ceph-mon[32450]: pgmap v810: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 434 op/s 2024-04-24T23:27:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:19 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:19.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:19 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:19.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:19 smithi117 ceph-mon[32529]: pgmap v810: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 434 op/s 2024-04-24T23:27:19.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:19 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:19.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:19 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:19.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:19 smithi033 ceph-mon[29201]: pgmap v810: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 434 op/s 2024-04-24T23:27:19.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:19 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:19.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:19 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:19.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 38562 26.87 MB/sec warmup 84 sec latency 381.544 ms 2024-04-24T23:27:20.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:20 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:20.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:20 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:20.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:20 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:20.880 INFO:tasks.workunit.client.0.smithi033.stdout: 10 38824 26.66 MB/sec warmup 85 sec latency 191.106 ms 2024-04-24T23:27:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:21 smithi038 ceph-mon[32450]: pgmap v811: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 381 op/s 2024-04-24T23:27:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:21 smithi117 ceph-mon[32529]: pgmap v811: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 381 op/s 2024-04-24T23:27:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:21 smithi033 ceph-mon[29201]: pgmap v811: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 381 op/s 2024-04-24T23:27:21.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 39263 26.59 MB/sec warmup 86 sec latency 160.719 ms 2024-04-24T23:27:22.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 40079 26.72 MB/sec warmup 87 sec latency 63.042 ms 2024-04-24T23:27:23.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:23 smithi038 ceph-mon[32450]: pgmap v812: 129 pgs: 129 active+clean; 506 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 8.1 MiB/s wr, 485 op/s 2024-04-24T23:27:23.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:23 smithi117 ceph-mon[32529]: pgmap v812: 129 pgs: 129 active+clean; 506 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 8.1 MiB/s wr, 485 op/s 2024-04-24T23:27:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:23 smithi033 ceph-mon[29201]: pgmap v812: 129 pgs: 129 active+clean; 506 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 8.1 MiB/s wr, 485 op/s 2024-04-24T23:27:23.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 40879 26.98 MB/sec warmup 88 sec latency 42.083 ms 2024-04-24T23:27:24.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 41258 27.00 MB/sec warmup 89 sec latency 231.503 ms 2024-04-24T23:27:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:25 smithi038 ceph-mon[32450]: pgmap v813: 129 pgs: 129 active+clean; 504 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 380 op/s 2024-04-24T23:27:25.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:25 smithi117 ceph-mon[32529]: pgmap v813: 129 pgs: 129 active+clean; 504 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 380 op/s 2024-04-24T23:27:25.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:25 smithi033 ceph-mon[29201]: pgmap v813: 129 pgs: 129 active+clean; 504 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 380 op/s 2024-04-24T23:27:25.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 41610 26.93 MB/sec warmup 90 sec latency 232.545 ms 2024-04-24T23:27:26.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 41859 26.83 MB/sec warmup 91 sec latency 160.179 ms 2024-04-24T23:27:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:27 smithi038 ceph-mon[32450]: pgmap v814: 129 pgs: 129 active+clean; 506 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 447 op/s 2024-04-24T23:27:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:27 smithi117 ceph-mon[32529]: pgmap v814: 129 pgs: 129 active+clean; 506 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 447 op/s 2024-04-24T23:27:27.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:27 smithi033 ceph-mon[29201]: pgmap v814: 129 pgs: 129 active+clean; 506 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 447 op/s 2024-04-24T23:27:27.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 42274 26.71 MB/sec warmup 92 sec latency 117.872 ms 2024-04-24T23:27:28.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 42677 26.65 MB/sec warmup 93 sec latency 121.332 ms 2024-04-24T23:27:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:29 smithi038 ceph-mon[32450]: pgmap v815: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 436 op/s 2024-04-24T23:27:29.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:29 smithi117 ceph-mon[32529]: pgmap v815: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 436 op/s 2024-04-24T23:27:29.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:29 smithi033 ceph-mon[29201]: pgmap v815: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 436 op/s 2024-04-24T23:27:29.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 42970 26.49 MB/sec warmup 94 sec latency 248.333 ms 2024-04-24T23:27:30.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 43220 26.35 MB/sec warmup 95 sec latency 320.468 ms 2024-04-24T23:27:31.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:31 smithi038 ceph-mon[32450]: pgmap v816: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-04-24T23:27:31.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:31 smithi117 ceph-mon[32529]: pgmap v816: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-04-24T23:27:31.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:31 smithi033 ceph-mon[29201]: pgmap v816: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-04-24T23:27:31.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 43753 26.30 MB/sec warmup 96 sec latency 115.758 ms 2024-04-24T23:27:32.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:32 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:27:32.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:32 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:27:32.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:32 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:27:32.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 44363 26.37 MB/sec warmup 97 sec latency 58.268 ms 2024-04-24T23:27:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:33 smithi038 ceph-mon[32450]: pgmap v817: 129 pgs: 129 active+clean; 522 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.6 MiB/s wr, 490 op/s 2024-04-24T23:27:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:27:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:27:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:33 smithi117 ceph-mon[32529]: pgmap v817: 129 pgs: 129 active+clean; 522 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.6 MiB/s wr, 490 op/s 2024-04-24T23:27:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:27:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:27:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:33 smithi033 ceph-mon[29201]: pgmap v817: 129 pgs: 129 active+clean; 522 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.6 MiB/s wr, 490 op/s 2024-04-24T23:27:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:27:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:27:33.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 44938 26.47 MB/sec warmup 98 sec latency 55.417 ms 2024-04-24T23:27:34.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 45492 26.59 MB/sec warmup 99 sec latency 138.283 ms 2024-04-24T23:27:35.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:35 smithi038 ceph-mon[32450]: pgmap v818: 129 pgs: 129 active+clean; 522 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-04-24T23:27:35.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:35 smithi117 ceph-mon[32529]: pgmap v818: 129 pgs: 129 active+clean; 522 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-04-24T23:27:35.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:35 smithi033 ceph-mon[29201]: pgmap v818: 129 pgs: 129 active+clean; 522 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-04-24T23:27:35.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 46124 26.69 MB/sec warmup 100 sec latency 284.688 ms 2024-04-24T23:27:36.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 46358 26.59 MB/sec warmup 101 sec latency 101.758 ms 2024-04-24T23:27:37.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:37 smithi038 ceph-mon[32450]: pgmap v819: 129 pgs: 129 active+clean; 509 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 447 op/s 2024-04-24T23:27:37.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:37 smithi117 ceph-mon[32529]: pgmap v819: 129 pgs: 129 active+clean; 509 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 447 op/s 2024-04-24T23:27:37.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:37 smithi033 ceph-mon[29201]: pgmap v819: 129 pgs: 129 active+clean; 509 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 447 op/s 2024-04-24T23:27:37.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 46606 26.41 MB/sec warmup 102 sec latency 86.722 ms 2024-04-24T23:27:38.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 46898 26.26 MB/sec warmup 103 sec latency 178.109 ms 2024-04-24T23:27:39.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:39 smithi038 ceph-mon[32450]: pgmap v820: 129 pgs: 129 active+clean; 506 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 422 op/s 2024-04-24T23:27:39.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:39 smithi117 ceph-mon[32529]: pgmap v820: 129 pgs: 129 active+clean; 506 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 422 op/s 2024-04-24T23:27:39.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:39 smithi033 ceph-mon[29201]: pgmap v820: 129 pgs: 129 active+clean; 506 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 422 op/s 2024-04-24T23:27:39.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 47346 26.19 MB/sec warmup 104 sec latency 91.770 ms 2024-04-24T23:27:40.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 47783 26.16 MB/sec warmup 105 sec latency 200.570 ms 2024-04-24T23:27:41.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:41 smithi117 ceph-mon[32529]: pgmap v821: 129 pgs: 129 active+clean; 497 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 344 op/s 2024-04-24T23:27:41.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:41 smithi033 ceph-mon[29201]: pgmap v821: 129 pgs: 129 active+clean; 497 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 344 op/s 2024-04-24T23:27:41.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 48402 26.13 MB/sec warmup 106 sec latency 113.412 ms 2024-04-24T23:27:41.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:41 smithi038 ceph-mon[32450]: pgmap v821: 129 pgs: 129 active+clean; 497 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 344 op/s 2024-04-24T23:27:42.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 49047 26.28 MB/sec warmup 107 sec latency 43.694 ms 2024-04-24T23:27:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:43 smithi117 ceph-mon[32529]: pgmap v822: 129 pgs: 129 active+clean; 507 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.4 MiB/s wr, 485 op/s 2024-04-24T23:27:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:43 smithi033 ceph-mon[29201]: pgmap v822: 129 pgs: 129 active+clean; 507 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.4 MiB/s wr, 485 op/s 2024-04-24T23:27:43.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 49679 26.40 MB/sec warmup 108 sec latency 126.282 ms 2024-04-24T23:27:43.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:43 smithi038 ceph-mon[32450]: pgmap v822: 129 pgs: 129 active+clean; 507 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.4 MiB/s wr, 485 op/s 2024-04-24T23:27:44.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 50054 26.41 MB/sec warmup 109 sec latency 78.210 ms 2024-04-24T23:27:45.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:45 smithi117 ceph-mon[32529]: pgmap v823: 129 pgs: 129 active+clean; 508 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 360 op/s 2024-04-24T23:27:45.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:45 smithi033 ceph-mon[29201]: pgmap v823: 129 pgs: 129 active+clean; 508 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 360 op/s 2024-04-24T23:27:45.881 INFO:tasks.workunit.client.0.smithi033.stdout: 10 50230 26.28 MB/sec warmup 110 sec latency 333.551 ms 2024-04-24T23:27:45.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:45 smithi038 ceph-mon[32450]: pgmap v823: 129 pgs: 129 active+clean; 508 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 360 op/s 2024-04-24T23:27:46.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 50484 26.16 MB/sec warmup 111 sec latency 97.044 ms 2024-04-24T23:27:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:47 smithi033 ceph-mon[29201]: pgmap v824: 129 pgs: 129 active+clean; 515 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-04-24T23:27:47.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 51036 26.13 MB/sec warmup 112 sec latency 109.077 ms 2024-04-24T23:27:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:47 smithi038 ceph-mon[32450]: pgmap v824: 129 pgs: 129 active+clean; 515 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-04-24T23:27:47.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:47 smithi117 ceph-mon[32529]: pgmap v824: 129 pgs: 129 active+clean; 515 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-04-24T23:27:48.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:27:48.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.353+0000 7efec1476640 1 -- 172.21.15.33:0/737221509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 msgr2=0x7efebc101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.353+0000 7efec1476640 1 --2- 172.21.15.33:0/737221509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc101bb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7efeac009900 tx=0x7efeac02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 -- 172.21.15.33:0/737221509 shutdown_connections 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 --2- 172.21.15.33:0/737221509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc103340 0x7efebc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 --2- 172.21.15.33:0/737221509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efebc102980 0x7efebc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 --2- 172.21.15.33:0/737221509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc101bb0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 -- 172.21.15.33:0/737221509 >> 172.21.15.33:0/737221509 conn(0x7efebc0fcd20 msgr2=0x7efebc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 -- 172.21.15.33:0/737221509 shutdown_connections 2024-04-24T23:27:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 -- 172.21.15.33:0/737221509 wait complete. 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 Processor -- start 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.354+0000 7efec1476640 1 -- start start 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efec1476640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efec1476640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102980 0x7efebc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efec1476640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efebc103340 0x7efebc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efec1476640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efebc10d430 con 0x7efebc101790 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efec1476640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efebc10d2b0 con 0x7efebc102980 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efec1476640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efebc10d5b0 con 0x7efebc103340 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36072/0 (socket says 172.21.15.33:36072) 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 -- 172.21.15.33:0/651593056 learned_addr learned my addr 172.21.15.33:0/651593056 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efeb27fc640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102980 0x7efebc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebb7fe640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efebc103340 0x7efebc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 -- 172.21.15.33:0/651593056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efebc103340 msgr2=0x7efebc19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efebc103340 0x7efebc19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 -- 172.21.15.33:0/651593056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102980 msgr2=0x7efebc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102980 0x7efebc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 -- 172.21.15.33:0/651593056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc19f710 con 0x7efebc101790 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efeb27fc640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102980 0x7efebc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efebaffd640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc197af0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7efeac031440 tx=0x7efeac032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.355+0000 7efeb8ff9640 1 -- 172.21.15.33:0/651593056 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac03f030 con 0x7efebc101790 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.356+0000 7efeb8ff9640 1 -- 172.21.15.33:0/651593056 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac003ce0 con 0x7efebc101790 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.356+0000 7efeb8ff9640 1 -- 172.21.15.33:0/651593056 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac035070 con 0x7efebc101790 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.356+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc19f9a0 con 0x7efebc101790 2024-04-24T23:27:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.356+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc19fe40 con 0x7efebc101790 2024-04-24T23:27:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.358+0000 7efeb8ff9640 1 -- 172.21.15.33:0/651593056 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efeac042030 con 0x7efebc101790 2024-04-24T23:27:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.358+0000 7efec4189740 1 Processor -- start 2024-04-24T23:27:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.358+0000 7efec4189740 1 -- start start 2024-04-24T23:27:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efec4189740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d11fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efec4189740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 0x56499d124b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efec4189740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d1251b0 0x56499d127640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efec4189740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56499d07f6b0 con 0x56499d1251b0 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efec4189740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56499d027880 con 0x56499d11f650 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efec4189740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56499d07b670 con 0x56499d120040 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efeb8ff9640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efea80a2c10 0x7efea80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 0x56499d124b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d11fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 0x56499d124b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52378/0 (socket says 172.21.15.33:52378) 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebaffd640 1 -- 172.21.15.33:0/3607755556 learned_addr learned my addr 172.21.15.33:0/3607755556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efeb27fc640 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d1251b0 0x56499d127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efeb8ff9640 1 -- 172.21.15.33:0/651593056 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efeac0ef780 con 0x7efebc101790 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efea80a2c10 0x7efea80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 -- 172.21.15.33:0/3607755556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 msgr2=0x56499d124b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 0x56499d124b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 -- 172.21.15.33:0/3607755556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d1251b0 msgr2=0x56499d127640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d1251b0 0x56499d127640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 -- 172.21.15.33:0/3607755556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499ce5d7d0 con 0x56499d11f650 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efeb27fc640 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d1251b0 0x56499d127640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebaffd640 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 0x56499d124b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.359+0000 7efebb7fe640 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d11fa70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb4011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b6aab7402cab40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efeb17fa640 1 -- 172.21.15.33:0/3607755556 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb400f820 con 0x56499d11f650 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efeb17fa640 1 -- 172.21.15.33:0/3607755556 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb4025430 con 0x56499d11f650 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efeb17fa640 1 -- 172.21.15.33:0/3607755556 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb400e5c0 con 0x56499d11f650 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efebb7fe640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efea80a2c10 0x7efea80a50d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efeb4022000 tx=0x7efeb401c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efec4189740 1 -- 172.21.15.33:0/3607755556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 msgr2=0x56499d11fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efec4189740 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d11fa70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb4011aa0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efec4189740 1 -- 172.21.15.33:0/3607755556 shutdown_connections 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.360+0000 7efec4189740 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d1251b0 0x56499d127640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d120040 0x56499d124b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 --2- 172.21.15.33:0/3607755556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d11fa70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- 172.21.15.33:0/3607755556 >> 172.21.15.33:0/3607755556 conn(0x56499d1168b0 msgr2=0x56499d116cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- 172.21.15.33:0/3607755556 shutdown_connections 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- 172.21.15.33:0/3607755556 wait complete. 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- start start 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 Processor -- start 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d129ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56499d128160 con 0x56499d120040 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56499d03af50 con 0x56499d1251b0 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56499d1282e0 con 0x56499d11f650 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efec4189740 10 client.0 _ll_register_callbacks cb 0x56499d076000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efeb27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efebaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d129ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efeb27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36082/0 (socket says 172.21.15.33:36082) 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.361+0000 7efeb27fc640 1 -- 172.21.15.33:0/212931393 learned_addr learned my addr 172.21.15.33:0/212931393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebb7fe640 1 -- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 msgr2=0x56499d129ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebb7fe640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d129ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebb7fe640 1 -- 172.21.15.33:0/212931393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 msgr2=0x56499d12a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebb7fe640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebb7fe640 1 -- 172.21.15.33:0/212931393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499cd933f0 con 0x56499d1251b0 2024-04-24T23:27:48.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efeb27fc640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebaffd640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d129ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efebb7fe640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56499d1208a0 tx=0x7efeb400f940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11504f7619e0ad85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efea27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efea27fc640 1 -- 172.21.15.33:0/212931393 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4029030 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb4025430 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efea27fc640 1 -- 172.21.15.33:0/212931393 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efeb4003b70 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efea27fc640 1 -- 172.21.15.33:0/212931393 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efeb4003d70 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.362+0000 7efec4189740 10 client.28193 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.363+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56499cd530f0 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.363+0000 7efea27fc640 1 -- 172.21.15.33:0/212931393 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb4025430 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.363+0000 7efea27fc640 1 -- 172.21.15.33:0/212931393 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efeb401a030 con 0x56499d1251b0 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.363+0000 7efec4189740 10 client.28193 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efec4189740 10 client.28193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efec4189740 20 client.28193 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efec4189740 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d1356e0 0x56499d137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efec4189740 4 client.28193 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499d08cbf0 con 0x56499d1356e0 2024-04-24T23:27:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efebaffd640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d1356e0 0x56499d137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efea27fc640 10 client.28193 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.364+0000 7efebaffd640 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d1356e0 0x56499d137b10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eec8c7dbc46e0dc server_cookie=a2716faf80f3cb0c in_seq=0 out_seq=0 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efea27fc640 1 -- 172.21.15.33:0/212931393 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56499d08cbf0 con 0x56499d1356e0 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efea27fc640 10 client.28193 handle_command_reply: tid=0 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 client.28193 shutdown 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 msgr2=0x56499d131050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56499d1208a0 tx=0x7efeb400f940 comp rx=0 tx=0).stop 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 shutdown_connections 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d1356e0 0x56499d137b10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131050 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efec4189740 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.365+0000 7efea27fc640 0 client.28193 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.366+0000 7efec4189740 1 --2- 172.21.15.33:0/212931393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d129ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.366+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 >> 172.21.15.33:0/212931393 conn(0x56499d1168b0 msgr2=0x56499d115af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.366+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 shutdown_connections 2024-04-24T23:27:48.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.366+0000 7efec4189740 1 -- 172.21.15.33:0/212931393 wait complete. 2024-04-24T23:27:48.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.366+0000 7efec4189740 20 client.28193 trim_cache size 0 max 16384 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 Processor -- start 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 -- start start 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d121500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d121a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1aab50 con 0x56499d120040 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1a83c0 con 0x56499d1251b0 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.387+0000 7efec4189740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1a6210 con 0x56499d11f650 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d121a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d121a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36098/0 (socket says 172.21.15.33:36098) 2024-04-24T23:27:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 -- 172.21.15.33:0/2021642045 learned_addr learned my addr 172.21.15.33:0/2021642045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efebaffd640 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d121500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 -- 172.21.15.33:0/2021642045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 msgr2=0x56499d121500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d121500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 -- 172.21.15.33:0/2021642045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 msgr2=0x56499d131010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 -- 172.21.15.33:0/2021642045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499d117620 con 0x56499d120040 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efebb7fe640 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb27fc640 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d121a40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7efea4012870 tx=0x7efea4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62244dccf24fcf86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb0ff9640 1 -- 172.21.15.33:0/2021642045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4022070 con 0x56499d120040 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb0ff9640 1 -- 172.21.15.33:0/2021642045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efea400f920 con 0x56499d120040 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.388+0000 7efeb0ff9640 1 -- 172.21.15.33:0/2021642045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4003810 con 0x56499d120040 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 -- 172.21.15.33:0/2021642045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 msgr2=0x56499d121a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d121a40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7efea4012870 tx=0x7efea4005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 -- 172.21.15.33:0/2021642045 shutdown_connections 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d1251b0 0x56499d131010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d121a40 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 --2- 172.21.15.33:0/2021642045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d11f650 0x56499d121500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 -- 172.21.15.33:0/2021642045 >> 172.21.15.33:0/2021642045 conn(0x56499d1168b0 msgr2=0x56499d129ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 -- 172.21.15.33:0/2021642045 shutdown_connections 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 -- 172.21.15.33:0/2021642045 wait complete. 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.389+0000 7efec4189740 1 -- start start 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 Processor -- start 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d12df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d1251b0 0x56499d199c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1a6880 con 0x56499d120040 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1a7cf0 con 0x56499d11f650 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1a6f50 con 0x56499d1251b0 2024-04-24T23:27:48.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efec4189740 10 client.0 _ll_register_callbacks cb 0x7efeac032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:48.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efeb27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d12df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efeb27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36108/0 (socket says 172.21.15.33:36108) 2024-04-24T23:27:48.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efeb27fc640 1 -- 172.21.15.33:0/3143040575 learned_addr learned my addr 172.21.15.33:0/3143040575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebb7fe640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d1251b0 0x56499d199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 -- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d1251b0 msgr2=0x56499d199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d1251b0 0x56499d199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 -- 172.21.15.33:0/3143040575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 msgr2=0x56499d12e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 -- 172.21.15.33:0/3143040575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499d19a1c0 con 0x56499d11f650 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efeb27fc640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.390+0000 7efebaffd640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d12df50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efeac006010 tx=0x7efeac08bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e5ef721401e769 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efea27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efea27fc640 1 -- 172.21.15.33:0/3143040575 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0d1020 con 0x56499d11f650 2024-04-24T23:27:48.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea400f920 con 0x56499d11f650 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efea27fc640 1 -- 172.21.15.33:0/3143040575 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efeac003800 con 0x56499d11f650 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efea27fc640 1 -- 172.21.15.33:0/3143040575 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efeac049040 con 0x56499d11f650 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 10 client.28199 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56499d19b4b0 con 0x56499d11f650 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efea27fc640 1 -- 172.21.15.33:0/3143040575 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efea400f920 con 0x56499d11f650 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efea27fc640 1 -- 172.21.15.33:0/3143040575 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efeac03bca0 con 0x56499d11f650 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 10 client.28199 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 10 client.28199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 20 client.28199 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d19be90 0x56499d19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 4 client.28199 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.391+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56499d086f40 con 0x56499d19be90 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efeb27fc640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d19be90 0x56499d19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efeb27fc640 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d19be90 0x56499d19e2b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f6422b1f0dd5c7 server_cookie=4d42f676197af5f5 in_seq=0 out_seq=0 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efea27fc640 10 client.28199 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efea27fc640 1 -- 172.21.15.33:0/3143040575 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56499d086f40 con 0x56499d19be90 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efea27fc640 10 client.28199 handle_command_reply: tid=0 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 client.28199 shutdown 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 msgr2=0x56499d12df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d12df50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efeac006010 tx=0x7efeac08bbb0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 shutdown_connections 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56499d19be90 0x56499d19e2b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56499d1251b0 0x56499d199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56499d120040 0x56499d12e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efea27fc640 0 client.28199 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 --2- 172.21.15.33:0/3143040575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56499d11f650 0x56499d12df50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 >> 172.21.15.33:0/3143040575 conn(0x56499d1168b0 msgr2=0x56499d129a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.392+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 shutdown_connections 2024-04-24T23:27:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec4189740 1 -- 172.21.15.33:0/3143040575 wait complete. 2024-04-24T23:27:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec4189740 20 client.28199 trim_cache size 0 max 16384 2024-04-24T23:27:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:27:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:27:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:27:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efea80a2c10 msgr2=0x7efea80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efea80a2c10 0x7efea80a50d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efeb4022000 tx=0x7efeb401c000 comp rx=0 tx=0).stop 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 msgr2=0x7efebc197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc197af0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7efeac031440 tx=0x7efeac032ce0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 shutdown_connections 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efea80a2c10 0x7efea80a50d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efebc103340 0x7efebc19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102980 0x7efebc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 --2- 172.21.15.33:0/651593056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efebc101790 0x7efebc197af0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 >> 172.21.15.33:0/651593056 conn(0x7efebc0fcd20 msgr2=0x7efebc105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 shutdown_connections 2024-04-24T23:27:48.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.393+0000 7efec1476640 1 -- 172.21.15.33:0/651593056 wait complete. 2024-04-24T23:27:48.406 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:48.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 -- 172.21.15.33:0/1617337540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8800ff7d0 msgr2=0x7fb8800ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1617337540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8800ff7d0 0x7fb8800ffc50 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009900 tx=0x7fb86c02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 -- 172.21.15.33:0/1617337540 shutdown_connections 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1617337540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880100190 0x7fb880109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1617337540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8800ff7d0 0x7fb8800ffc50 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1617337540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb880102d60 0x7fb880103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 -- 172.21.15.33:0/1617337540 >> 172.21.15.33:0/1617337540 conn(0x7fb8800fb370 msgr2=0x7fb8800fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 -- 172.21.15.33:0/1617337540 shutdown_connections 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.520+0000 7fb886d9e640 1 -- 172.21.15.33:0/1617337540 wait complete. 2024-04-24T23:27:48.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 Processor -- start 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 -- start start 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8800ff7d0 0x7fb880197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 0x7fb880197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880102d60 0x7fb88019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010d410 con 0x7fb880100190 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010d290 con 0x7fb8800ff7d0 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb886d9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010d590 con 0x7fb880102d60 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb87ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 0x7fb880197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb884b3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8800ff7d0 0x7fb880197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb87ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 0x7fb880197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36124/0 (socket says 172.21.15.33:36124) 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.521+0000 7fb87ffff640 1 -- 172.21.15.33:0/1547444901 learned_addr learned my addr 172.21.15.33:0/1547444901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb88533b640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880102d60 0x7fb88019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87ffff640 1 -- 172.21.15.33:0/1547444901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880102d60 msgr2=0x7fb88019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87ffff640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880102d60 0x7fb88019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87ffff640 1 -- 172.21.15.33:0/1547444901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8800ff7d0 msgr2=0x7fb880197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87ffff640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8800ff7d0 0x7fb880197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87ffff640 1 -- 172.21.15.33:0/1547444901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88019f6d0 con 0x7fb880100190 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb884b3a640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8800ff7d0 0x7fb880197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb88533b640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880102d60 0x7fb88019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87ffff640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 0x7fb880197ff0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009230 tx=0x7fb86c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c03f030 con 0x7fb880100190 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.522+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88019f960 con 0x7fb880100190 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.523+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88019fde0 con 0x7fb880100190 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.523+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c003ce0 con 0x7fb880100190 2024-04-24T23:27:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.523+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c049060 con 0x7fb880100190 2024-04-24T23:27:48.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.524+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb864005290 con 0x7fb880100190 2024-04-24T23:27:48.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.526+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb86c042030 con 0x7fb880100190 2024-04-24T23:27:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.526+0000 7fb87dffb640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8700a2c10 0x7fb8700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.527+0000 7fb884b3a640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8700a2c10 0x7fb8700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.527+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb86c0ef730 con 0x7fb880100190 2024-04-24T23:27:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.527+0000 7fb884b3a640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8700a2c10 0x7fb8700a50d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb8800685f0 tx=0x7fb874005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.532+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb86c0bc300 con 0x7fb880100190 2024-04-24T23:27:48.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.810+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb864005580 con 0x7fb880100190 2024-04-24T23:27:48.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.811+0000 7fb87dffb640 1 -- 172.21.15.33:0/1547444901 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11863 (secure 0 0 0) 0x7fb86c0bbc00 con 0x7fb880100190 2024-04-24T23:27:48.813 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:27:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:26:01.133927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:27:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8700a2c10 msgr2=0x7fb8700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8700a2c10 0x7fb8700a50d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb8800685f0 tx=0x7fb874005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 msgr2=0x7fb880197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 0x7fb880197ff0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009230 tx=0x7fb86c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 shutdown_connections 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8700a2c10 0x7fb8700a50d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880102d60 0x7fb88019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb880100190 0x7fb880197ff0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 --2- 172.21.15.33:0/1547444901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8800ff7d0 0x7fb880197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 >> 172.21.15.33:0/1547444901 conn(0x7fb8800fb370 msgr2=0x7fb880105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 shutdown_connections 2024-04-24T23:27:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.814+0000 7fb886d9e640 1 -- 172.21.15.33:0/1547444901 wait complete. 2024-04-24T23:27:48.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:27:48.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 51419 26.08 MB/sec warmup 113 sec latency 126.180 ms 2024-04-24T23:27:48.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.946+0000 7f7c06231640 1 -- 172.21.15.33:0/3732963205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00104bf0 msgr2=0x7f7c00106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.946+0000 7f7c06231640 1 --2- 172.21.15.33:0/3732963205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00104bf0 0x7f7c00106fe0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009900 tx=0x7f7be802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- 172.21.15.33:0/3732963205 shutdown_connections 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 --2- 172.21.15.33:0/3732963205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c00107520 0x7f7c00109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 --2- 172.21.15.33:0/3732963205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00104bf0 0x7f7c00106fe0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 --2- 172.21.15.33:0/3732963205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c001022c0 0x7f7c001046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- 172.21.15.33:0/3732963205 >> 172.21.15.33:0/3732963205 conn(0x7f7c000fbc10 msgr2=0x7f7c000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- 172.21.15.33:0/3732963205 shutdown_connections 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- 172.21.15.33:0/3732963205 wait complete. 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 Processor -- start 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- start start 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c001022c0 0x7f7c00197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c00104bf0 0x7f7c00198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 0x7f7c0019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0010d3f0 con 0x7f7c00107520 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0010d270 con 0x7f7c00104bf0 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7c06231640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0010d570 con 0x7f7c001022c0 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 0x7f7c0019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 0x7f7c0019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36152/0 (socket says 172.21.15.33:36152) 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.947+0000 7f7bfffff640 1 -- 172.21.15.33:0/502097969 learned_addr learned my addr 172.21.15.33:0/502097969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c00104bf0 0x7f7c00198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c001022c0 0x7f7c00197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfffff640 1 -- 172.21.15.33:0/502097969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c001022c0 msgr2=0x7f7c00197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfffff640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c001022c0 0x7f7c00197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfffff640 1 -- 172.21.15.33:0/502097969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c00104bf0 msgr2=0x7f7c00198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfffff640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c00104bf0 0x7f7c00198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfffff640 1 -- 172.21.15.33:0/502097969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0019f6e0 con 0x7f7c00107520 2024-04-24T23:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfffff640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 0x7f7c0019f070 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800eb10 tx=0x7f7bf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfd7fa640 1 -- 172.21.15.33:0/502097969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8004b20 con 0x7f7c00107520 2024-04-24T23:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfd7fa640 1 -- 172.21.15.33:0/502097969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf8004cc0 con 0x7f7c00107520 2024-04-24T23:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0019f9d0 con 0x7f7c00107520 2024-04-24T23:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7bfd7fa640 1 -- 172.21.15.33:0/502097969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8010760 con 0x7f7c00107520 2024-04-24T23:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.948+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0019fef0 con 0x7f7c00107520 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.951+0000 7f7c0901a740 1 Processor -- start 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.951+0000 7f7bfd7fa640 1 -- 172.21.15.33:0/502097969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7bf8005020 con 0x7f7c00107520 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.951+0000 7f7c0901a740 1 -- start start 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdca030 0x55b1ffdceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7c0901a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b1ffd29670 con 0x55b1ffdc9640 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7c0901a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b1ffcd1970 con 0x55b1ffdca030 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7c0901a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b1ffd25630 con 0x55b1ffdcf1a0 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdca030 0x55b1ffdceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36154/0 (socket says 172.21.15.33:36154) 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bf7fff640 1 -- 172.21.15.33:0/364021268 learned_addr learned my addr 172.21.15.33:0/364021268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 -- 172.21.15.33:0/364021268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 msgr2=0x55b1ffdd1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bfd7fa640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7bec0a2c10 0x7f7bec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 -- 172.21.15.33:0/364021268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 msgr2=0x55b1ffdc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 -- 172.21.15.33:0/364021268 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1ffcf5400 con 0x55b1ffdca030 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bfffff640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7bec0a2c10 0x7f7bec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdca030 0x55b1ffdceb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000bbf0 tx=0x7f7bf00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f5721c405b5b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bfd7fa640 1 -- 172.21.15.33:0/502097969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7bf80cf900 con 0x7f7c00107520 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.952+0000 7f7bf5ffb640 1 -- 172.21.15.33:0/364021268 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004030 con 0x55b1ffdca030 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7bf5ffb640 1 -- 172.21.15.33:0/364021268 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf00041d0 con 0x55b1ffdca030 2024-04-24T23:27:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7bf5ffb640 1 -- 172.21.15.33:0/364021268 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00116a0 con 0x55b1ffdca030 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 -- 172.21.15.33:0/364021268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdca030 msgr2=0x55b1ffdceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7bfffff640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7bec0a2c10 0x7f7bec0a50d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8098230 tx=0x7f7bf806bc80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdca030 0x55b1ffdceb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000bbf0 tx=0x7f7bf00107a0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 -- 172.21.15.33:0/364021268 shutdown_connections 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdca030 0x55b1ffdceb30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 --2- 172.21.15.33:0/364021268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 -- 172.21.15.33:0/364021268 >> 172.21.15.33:0/364021268 conn(0x55b1ffdc08a0 msgr2=0x55b1ffdc0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 -- 172.21.15.33:0/364021268 shutdown_connections 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.953+0000 7f7c0901a740 1 -- 172.21.15.33:0/364021268 wait complete. 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 -- start start 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 Processor -- start 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffddad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b1ffdd2150 con 0x55b1ffdc9640 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b1ffce5050 con 0x55b1ffdcf1a0 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b1ffdd22d0 con 0x55b1ffdca030 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7c0901a740 10 client.0 _ll_register_callbacks cb 0x55b1ffa3cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.954+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36156/0 (socket says 172.21.15.33:36156) 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2060194042 learned_addr learned my addr 172.21.15.33:0/2060194042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffddad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2060194042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 msgr2=0x55b1ffdd3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdcf1a0 msgr2=0x55b1ffddad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffddad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2060194042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1ffcb6dc0 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd37c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7be8032fd0 tx=0x7f7be8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae903d284502f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2060194042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8035040 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf00041d0 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2060194042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7be8004480 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2060194042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7be8053050 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7c0901a740 10 client.18717 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.955+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1ffa3d3f0 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2060194042 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf00041d0 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2060194042 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7be804a070 con 0x55b1ffdc9640 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7c0901a740 10 client.18717 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7c0901a740 10 client.18717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7c0901a740 20 client.18717 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffde1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7c0901a740 4 client.18717 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1ffd36bb0 con 0x55b1ffddf4e0 2024-04-24T23:27:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffde1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffde1980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc80b6dc9acb658e server_cookie=4a25b24eb92508d8 in_seq=0 out_seq=0 2024-04-24T23:27:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.956+0000 7f7bdeffd640 10 client.18717 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2060194042 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1ffd36bb0 con 0x55b1ffddf4e0 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7bdeffd640 10 client.18717 handle_command_reply: tid=0 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 client.18717 shutdown 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 msgr2=0x55b1ffdd37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd37c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7be8032fd0 tx=0x7f7be8005fd0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 shutdown_connections 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffde1980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7bdeffd640 0 client.18717 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffddad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2060194042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd37c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 >> 172.21.15.33:0/2060194042 conn(0x55b1ffdc08a0 msgr2=0x55b1ffdbfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 shutdown_connections 2024-04-24T23:27:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 1 -- 172.21.15.33:0/2060194042 wait complete. 2024-04-24T23:27:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.959+0000 7f7c0901a740 20 client.18717 trim_cache size 0 max 16384 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.974+0000 7f7c0901a740 1 Processor -- start 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.974+0000 7f7c0901a740 1 -- start start 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.974+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.974+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 0x55b1ffddad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.974+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdc3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.974+0000 7f7c0901a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001aab00 con 0x55b1ffdca030 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7c0901a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a8370 con 0x55b1ffdc9640 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7c0901a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a61c0 con 0x55b1ffdcf1a0 2024-04-24T23:27:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 0x55b1ffddad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 0x55b1ffddad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36170/0 (socket says 172.21.15.33:36170) 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 -- 172.21.15.33:0/1208635480 learned_addr learned my addr 172.21.15.33:0/1208635480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdc3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 -- 172.21.15.33:0/1208635480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 msgr2=0x55b1ffdc3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdc3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 -- 172.21.15.33:0/1208635480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 msgr2=0x55b1ffdc37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 -- 172.21.15.33:0/1208635480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1ffb077d0 con 0x55b1ffdca030 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdc3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 0x55b1ffddad00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0000c00 tx=0x7f7bf000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fe7095c32bc539 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bf57fa640 1 -- 172.21.15.33:0/1208635480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00037f0 con 0x55b1ffdca030 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bf57fa640 1 -- 172.21.15.33:0/1208635480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0029910 con 0x55b1ffdca030 2024-04-24T23:27:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.975+0000 7f7bf57fa640 1 -- 172.21.15.33:0/1208635480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00276a0 con 0x55b1ffdca030 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 -- 172.21.15.33:0/1208635480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 msgr2=0x55b1ffddad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 0x55b1ffddad00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0000c00 tx=0x7f7bf000b390 comp rx=0 tx=0).stop 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 -- 172.21.15.33:0/1208635480 shutdown_connections 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdc3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdca030 0x55b1ffddad00 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 --2- 172.21.15.33:0/1208635480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdc37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 -- 172.21.15.33:0/1208635480 >> 172.21.15.33:0/1208635480 conn(0x55b1ffdc08a0 msgr2=0x55b1ffdbfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 -- 172.21.15.33:0/1208635480 shutdown_connections 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 -- 172.21.15.33:0/1208635480 wait complete. 2024-04-24T23:27:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.976+0000 7f7c0901a740 1 -- start start 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 Processor -- start 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a6830 con 0x55b1ffdcf1a0 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a7ca0 con 0x55b1ffdc9640 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a6f00 con 0x55b1ffdca030 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7c0901a740 10 client.0 _ll_register_callbacks cb 0x7f7be8005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60954/0 (socket says 172.21.15.33:60954) 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2083252623 learned_addr learned my addr 172.21.15.33:0/2083252623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2083252623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 msgr2=0x55b1ffdd8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2083252623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdcf1a0 msgr2=0x55b1ffdd9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.977+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2083252623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1ffdda5d0 con 0x55b1ffdc9640 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd8a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7be80059d0 tx=0x7f7be8032dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a71fd6ff9a4f5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2083252623 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8035050 con 0x55b1ffdc9640 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf0029910 con 0x55b1ffdc9640 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2083252623 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7be8004980 con 0x55b1ffdc9640 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2083252623 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7be8067050 con 0x55b1ffdc9640 2024-04-24T23:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 10 client.28223 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1ffe46090 con 0x55b1ffdc9640 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2083252623 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0029910 con 0x55b1ffdc9640 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2083252623 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7be804a040 con 0x55b1ffdc9640 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 10 client.28223 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 10 client.28223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 20 client.28223 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffe48a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 4 client.28223 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.978+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1ffd30f00 con 0x55b1ffddf4e0 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.979+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffe48a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.979+0000 7f7bff7fe640 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffe48a80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6fef92c3f384334 server_cookie=84856bd314c07778 in_seq=0 out_seq=0 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.979+0000 7f7bdeffd640 10 client.28223 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7bdeffd640 1 -- 172.21.15.33:0/2083252623 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1ffd30f00 con 0x55b1ffddf4e0 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7bdeffd640 10 client.28223 handle_command_reply: tid=0 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 client.28223 shutdown 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 msgr2=0x55b1ffdd8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd8a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7be80059d0 tx=0x7f7be8032dd0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 shutdown_connections 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55b1ffddf4e0 0x55b1ffe48a80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1ffdcf1a0 0x55b1ffdd9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1ffdca030 0x55b1ffdd8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7bdeffd640 0 client.28223 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 --2- 172.21.15.33:0/2083252623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1ffdc9640 0x55b1ffdd8a90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 >> 172.21.15.33:0/2083252623 conn(0x55b1ffdc08a0 msgr2=0x55b1ffdcaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 shutdown_connections 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 1 -- 172.21.15.33:0/2083252623 wait complete. 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.981+0000 7f7c0901a740 20 client.28223 trim_cache size 0 max 16384 2024-04-24T23:27:48.986 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:27:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7bec0a2c10 msgr2=0x7f7bec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7bec0a2c10 0x7f7bec0a50d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8098230 tx=0x7f7bf806bc80 comp rx=0 tx=0).stop 2024-04-24T23:27:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 msgr2=0x7f7c0019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 0x7f7c0019f070 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800eb10 tx=0x7f7bf800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 shutdown_connections 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7bec0a2c10 0x7f7bec0a50d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c00107520 0x7f7c0019f070 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c00104bf0 0x7f7c00198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 --2- 172.21.15.33:0/502097969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c001022c0 0x7f7c00197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 >> 172.21.15.33:0/502097969 conn(0x7f7c000fbc10 msgr2=0x7f7c00102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 shutdown_connections 2024-04-24T23:27:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:48.982+0000 7f7c06231640 1 -- 172.21.15.33:0/502097969 wait complete. 2024-04-24T23:27:48.994 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:27:49.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/1531482484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 msgr2=0x7f6cf0104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/1531482484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0104690 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009900 tx=0x7f6ce002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/1531482484 shutdown_connections 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/1531482484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0107500 0x7f6cf01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/1531482484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0104bd0 0x7f6cf0106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/1531482484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0104690 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/1531482484 >> 172.21.15.33:0/1531482484 conn(0x7f6cf00fbbf0 msgr2=0x7f6cf00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/1531482484 shutdown_connections 2024-04-24T23:27:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.103+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/1531482484 wait complete. 2024-04-24T23:27:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 Processor -- start 2024-04-24T23:27:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 -- start start 2024-04-24T23:27:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0104bd0 0x7f6cf0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0107500 0x7f6cf019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010d3f0 con 0x7f6cf01022a0 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010d270 con 0x7f6cf0107500 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.104+0000 7f6cf4a5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010d570 con 0x7f6cf0104bd0 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36192/0 (socket says 172.21.15.33:36192) 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 -- 172.21.15.33:0/4157529707 learned_addr learned my addr 172.21.15.33:0/4157529707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0107500 0x7f6cf019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0104bd0 0x7f6cf0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 -- 172.21.15.33:0/4157529707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0104bd0 msgr2=0x7f6cf0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0104bd0 0x7f6cf0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 -- 172.21.15.33:0/4157529707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0107500 msgr2=0x7f6cf019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0107500 0x7f6cf019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 -- 172.21.15.33:0/4157529707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf019f710 con 0x7f6cf01022a0 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0107500 0x7f6cf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cee59c640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0197af0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009230 tx=0x7f6ce00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6ce77fe640 1 -- 172.21.15.33:0/4157529707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce003f030 con 0x7f6cf01022a0 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6ce77fe640 1 -- 172.21.15.33:0/4157529707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce0003ce0 con 0x7f6cf01022a0 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf019f9a0 con 0x7f6cf01022a0 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6ce77fe640 1 -- 172.21.15.33:0/4157529707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0035070 con 0x7f6cf01022a0 2024-04-24T23:27:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.105+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf019fe90 con 0x7f6cf01022a0 2024-04-24T23:27:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 Processor -- start 2024-04-24T23:27:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6ce77fe640 1 -- 172.21.15.33:0/4157529707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6ce0042030 con 0x7f6cf01022a0 2024-04-24T23:27:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 -- start start 2024-04-24T23:27:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fc0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fc5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fc8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dfe5f20670 con 0x55dfe5fc61a0 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dfe5ec8970 con 0x55dfe5fc1030 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cf776e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dfe5f1c630 con 0x55dfe5fc0640 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fc8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.108+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fc5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fc0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fc8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36202/0 (socket says 172.21.15.33:36202) 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fc5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60994/0 (socket says 172.21.15.33:60994) 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fc0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52498/0 (socket says 172.21.15.33:52498) 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 -- 172.21.15.33:0/141362562 learned_addr learned my addr 172.21.15.33:0/141362562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 -- 172.21.15.33:0/141362562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 msgr2=0x55dfe5fc0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce77fe640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2b90 0x7f6cdc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fc0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 -- 172.21.15.33:0/141362562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 msgr2=0x55dfe5fc5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fc5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 -- 172.21.15.33:0/141362562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe5eec400 con 0x55dfe5fc61a0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fc0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6cee59c640 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fc5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2b90 0x7f6cdc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce77fe640 1 -- 172.21.15.33:0/4157529707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ce00ef850 con 0x7f6cf01022a0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fc8630 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009c60 tx=0x7f6cd8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2c065a37fc9fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce4ff9640 1 -- 172.21.15.33:0/141362562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004230 con 0x55dfe5fc61a0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce4ff9640 1 -- 172.21.15.33:0/141362562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd80043d0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.109+0000 7f6ce4ff9640 1 -- 172.21.15.33:0/141362562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004de0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 -- 172.21.15.33:0/141362562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 msgr2=0x55dfe5fc8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2b90 0x7f6cdc0a5050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55dfe5fc1890 tx=0x7f6ce8004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fc8630 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009c60 tx=0x7f6cd8011810 comp rx=0 tx=0).stop 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 -- 172.21.15.33:0/141362562 shutdown_connections 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fc8630 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fc5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 --2- 172.21.15.33:0/141362562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fc0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 -- 172.21.15.33:0/141362562 >> 172.21.15.33:0/141362562 conn(0x55dfe5fbbcb0 msgr2=0x55dfe5fbc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 -- 172.21.15.33:0/141362562 shutdown_connections 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.110+0000 7f6cf776e740 1 -- 172.21.15.33:0/141362562 wait complete. 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 -- start start 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 Processor -- start 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dfe5fc9150 con 0x55dfe5fc61a0 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dfe5edc050 con 0x55dfe5fc1030 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dfe5fc92d0 con 0x55dfe5fc0640 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cf776e740 10 client.0 _ll_register_callbacks cb 0x55dfe5c33be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fb7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36206/0 (socket says 172.21.15.33:36206) 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 -- 172.21.15.33:0/3640231141 learned_addr learned my addr 172.21.15.33:0/3640231141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 -- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 msgr2=0x55dfe5fbaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 -- 172.21.15.33:0/3640231141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 msgr2=0x55dfe5fbb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.111+0000 7f6ceed9d640 1 -- 172.21.15.33:0/3640231141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe5eaddc0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fb7560 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8011f00 tx=0x7f6ce8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e55277717d05248 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/3640231141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce802d060 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/3640231141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ce8028600 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd80043d0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/3640231141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ce8034050 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 10 client.18735 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfe5c343f0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/3640231141 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd80043d0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/3640231141 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6ce800f930 con 0x55dfe5fc61a0 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 10 client.18735 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 10 client.18735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 20 client.18735 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.112+0000 7f6cf776e740 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe5fdaa50 0x55dfe5fdceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.113+0000 7f6cf776e740 4 client.18735 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.113+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfe5f2dbb0 con 0x55dfe5fdaa50 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.113+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe5fdaa50 0x55dfe5fdceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.113+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe5fdaa50 0x55dfe5fdceb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20cbc1ecbf22b8cd server_cookie=814b32a1daa79759 in_seq=0 out_seq=0 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.113+0000 7f6cd5ffb640 10 client.18735 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/3640231141 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfe5f2dbb0 con 0x55dfe5fdaa50 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cd5ffb640 10 client.18735 handle_command_reply: tid=0 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 client.18735 shutdown 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 msgr2=0x55dfe5fb7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fb7560 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8011f00 tx=0x7f6ce8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 shutdown_connections 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe5fdaa50 0x55dfe5fdceb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fb7560 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cd5ffb640 0 client.18735 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 --2- 172.21.15.33:0/3640231141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 >> 172.21.15.33:0/3640231141 conn(0x55dfe5fbbcb0 msgr2=0x55dfe5fbccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 shutdown_connections 2024-04-24T23:27:49.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 1 -- 172.21.15.33:0/3640231141 wait complete. 2024-04-24T23:27:49.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.114+0000 7f6cf776e740 20 client.18735 trim_cache size 0 max 16384 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 Processor -- start 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 -- start start 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fe1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01aab20 con 0x55dfe5fc1030 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01a8390 con 0x55dfe5fc0640 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cf776e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01a61e0 con 0x55dfe5fc61a0 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36210/0 (socket says 172.21.15.33:36210) 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 -- 172.21.15.33:0/246651065 learned_addr learned my addr 172.21.15.33:0/246651065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cee59c640 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fe1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 -- 172.21.15.33:0/246651065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 msgr2=0x55dfe5fe1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fe1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 -- 172.21.15.33:0/246651065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 msgr2=0x55dfe5fbf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 -- 172.21.15.33:0/246651065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe5fb78d0 con 0x55dfe5fc1030 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6cee59c640 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.128+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbf810 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x55dfe5fbcb80 tx=0x7f6cd8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39427cae930613f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cd7fff640 1 -- 172.21.15.33:0/246651065 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8009820 con 0x55dfe5fc1030 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cd7fff640 1 -- 172.21.15.33:0/246651065 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd801f040 con 0x55dfe5fc1030 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cd7fff640 1 -- 172.21.15.33:0/246651065 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8024450 con 0x55dfe5fc1030 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 -- 172.21.15.33:0/246651065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 msgr2=0x55dfe5fbf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbf810 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x55dfe5fbcb80 tx=0x7f6cd8005850 comp rx=0 tx=0).stop 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 -- 172.21.15.33:0/246651065 shutdown_connections 2024-04-24T23:27:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fe1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fbf810 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 --2- 172.21.15.33:0/246651065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fbf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 -- 172.21.15.33:0/246651065 >> 172.21.15.33:0/246651065 conn(0x55dfe5fbbcb0 msgr2=0x55dfe5fbebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 -- 172.21.15.33:0/246651065 shutdown_connections 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.129+0000 7f6cf776e740 1 -- 172.21.15.33:0/246651065 wait complete. 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 -- start start 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 Processor -- start 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fedbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01a6850 con 0x55dfe5fc1030 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01a7cc0 con 0x55dfe5fc0640 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01a6f20 con 0x55dfe5fc61a0 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cf776e740 10 client.0 _ll_register_callbacks cb 0x7f6ce8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fedbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fedbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32802/0 (socket says 172.21.15.33:32802) 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6cee59c640 1 -- 172.21.15.33:0/1992569416 learned_addr learned my addr 172.21.15.33:0/1992569416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.130+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cee59c640 1 -- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 msgr2=0x55dfe5fec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cee59c640 1 -- 172.21.15.33:0/1992569416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 msgr2=0x55dfe5fee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cee59c640 1 -- 172.21.15.33:0/1992569416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe5cfe7d0 con 0x55dfe5fc0640 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fedbe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00ed330 tx=0x7f6ce002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a610e0d4a4ba44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:27:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/1992569416 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0049040 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/1992569416 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ce00ba0d0 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd801f040 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/1992569416 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ce00bb730 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cf776e740 10 client.28241 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfe6041400 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.131+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/1992569416 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd801f040 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/1992569416 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6ce003bc10 con 0x55dfe5fc0640 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 10 client.28241 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 10 client.28241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 20 client.28241 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe6041d80 0x55dfe60441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 4 client.28241 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfe5f27f00 con 0x55dfe6041d80 2024-04-24T23:27:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe6041d80 0x55dfe60441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cd5ffb640 10 client.28241 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6ce7fff640 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe6041d80 0x55dfe60441a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3682d222b393db server_cookie=f3dad5d0f8af8daf in_seq=0 out_seq=0 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cd5ffb640 1 -- 172.21.15.33:0/1992569416 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfe5f27f00 con 0x55dfe6041d80 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cd5ffb640 10 client.28241 handle_command_reply: tid=0 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 client.28241 shutdown 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 msgr2=0x55dfe5fedbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fedbe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00ed330 tx=0x7f6ce002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 shutdown_connections 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55dfe6041d80 0x55dfe60441a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfe5fc61a0 0x55dfe5fec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfe5fc1030 0x55dfe5fee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 --2- 172.21.15.33:0/1992569416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dfe5fc0640 0x55dfe5fedbe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cd5ffb640 0 client.28241 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 >> 172.21.15.33:0/1992569416 conn(0x55dfe5fbbcb0 msgr2=0x55dfe5fbf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.132+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 shutdown_connections 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf776e740 1 -- 172.21.15.33:0/1992569416 wait complete. 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf776e740 20 client.28241 trim_cache size 0 max 16384 2024-04-24T23:27:49.141 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:27:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2b90 msgr2=0x7f6cdc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2b90 0x7f6cdc0a5050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55dfe5fc1890 tx=0x7f6ce8004a40 comp rx=0 tx=0).stop 2024-04-24T23:27:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 msgr2=0x7f6cf0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0197af0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009230 tx=0x7f6ce00042f0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 shutdown_connections 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6cdc0a2b90 0x7f6cdc0a5050 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cf0107500 0x7f6cf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cf0104bd0 0x7f6cf0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 --2- 172.21.15.33:0/4157529707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf01022a0 0x7f6cf0197af0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 >> 172.21.15.33:0/4157529707 conn(0x7f6cf00fbbf0 msgr2=0x7f6cf0102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 shutdown_connections 2024-04-24T23:27:49.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.133+0000 7f6cf4a5d640 1 -- 172.21.15.33:0/4157529707 wait complete. 2024-04-24T23:27:49.146 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.263+0000 7efdb2e15640 1 -- 172.21.15.33:0/478929230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac102210 msgr2=0x7efdac104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.263+0000 7efdb2e15640 1 --2- 172.21.15.33:0/478929230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac102210 0x7efdac104600 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efda0009900 tx=0x7efda002a3f0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 -- 172.21.15.33:0/478929230 shutdown_connections 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 --2- 172.21.15.33:0/478929230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdac107470 0x7efdac1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 --2- 172.21.15.33:0/478929230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac104b40 0x7efdac106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 --2- 172.21.15.33:0/478929230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac102210 0x7efdac104600 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 -- 172.21.15.33:0/478929230 >> 172.21.15.33:0/478929230 conn(0x7efdac0fbc00 msgr2=0x7efdac0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 -- 172.21.15.33:0/478929230 shutdown_connections 2024-04-24T23:27:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 -- 172.21.15.33:0/478929230 wait complete. 2024-04-24T23:27:49.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 Processor -- start 2024-04-24T23:27:49.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.264+0000 7efdb2e15640 1 -- start start 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb2e15640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac102210 0x7efdac197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb2e15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 0x7efdac198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb2e15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdac107470 0x7efdac19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb2e15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d410 con 0x7efdac104b40 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb2e15640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d290 con 0x7efdac102210 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb2e15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d590 con 0x7efdac107470 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 0x7efdac198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac102210 0x7efdac197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 0x7efdac198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36236/0 (socket says 172.21.15.33:36236) 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdabfff640 1 -- 172.21.15.33:0/3407446566 learned_addr learned my addr 172.21.15.33:0/3407446566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb13b2640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdac107470 0x7efdac19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3407446566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdac107470 msgr2=0x7efdac19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdac107470 0x7efdac19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 msgr2=0x7efdac198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 0x7efdac198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3407446566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac19f700 con 0x7efdac102210 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdabfff640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 0x7efdac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.265+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac102210 0x7efdac197ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efda0031440 tx=0x7efda0031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.266+0000 7efda9ffb640 1 -- 172.21.15.33:0/3407446566 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda003f070 con 0x7efdac102210 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.266+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac19f990 con 0x7efdac102210 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.266+0000 7efda9ffb640 1 -- 172.21.15.33:0/3407446566 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda0003cd0 con 0x7efdac102210 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.266+0000 7efda9ffb640 1 -- 172.21.15.33:0/3407446566 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0035070 con 0x7efdac102210 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.266+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac19fed0 con 0x7efdac102210 2024-04-24T23:27:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.269+0000 7efda9ffb640 1 -- 172.21.15.33:0/3407446566 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efda003b560 con 0x7efdac102210 2024-04-24T23:27:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.269+0000 7efdb5c03740 1 Processor -- start 2024-04-24T23:27:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.269+0000 7efdb5c03740 1 -- start start 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efda9ffb640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd9c0a2b90 0x7efd9c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e1dc20 0x561130e22720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e22d90 0x561130e25220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb5c03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561130d81670 con 0x561130e22d90 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb5c03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561130d29970 con 0x561130e1d230 2024-04-24T23:27:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb5c03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561130d7d630 con 0x561130e1dc20 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb13b2640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd9c0a2b90 0x7efd9c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e1d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32830/0 (socket says 172.21.15.33:32830) 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3917524379 learned_addr learned my addr 172.21.15.33:0/3917524379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efda9ffb640 1 -- 172.21.15.33:0/3407446566 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efda00ef810 con 0x7efdac102210 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb13b2640 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e22d90 0x561130e25220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdabfff640 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e1dc20 0x561130e22720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3917524379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e1dc20 msgr2=0x561130e22720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e1dc20 0x561130e22720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3917524379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e22d90 msgr2=0x561130e25220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e22d90 0x561130e25220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.270+0000 7efdb0bb1640 1 -- 172.21.15.33:0/3917524379 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561130d4d400 con 0x561130e1d230 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.271+0000 7efdb13b2640 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e22d90 0x561130e25220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.271+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e1d650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efda00314e0 tx=0x7efda0040860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51f2ec14ff45601 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.271+0000 7efdb13b2640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd9c0a2b90 0x7efd9c0a5050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efda40098b0 tx=0x7efda4009840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.271+0000 7efd96ffd640 1 -- 172.21.15.33:0/3917524379 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0048070 con 0x561130e1d230 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.271+0000 7efd96ffd640 1 -- 172.21.15.33:0/3917524379 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda00b4ff0 con 0x561130e1d230 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.271+0000 7efd96ffd640 1 -- 172.21.15.33:0/3917524379 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00b8400 con 0x561130e1d230 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 -- 172.21.15.33:0/3917524379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 msgr2=0x561130e1d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e1d650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efda00314e0 tx=0x7efda0040860 comp rx=0 tx=0).stop 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 -- 172.21.15.33:0/3917524379 shutdown_connections 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e22d90 0x561130e25220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e1dc20 0x561130e22720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 --2- 172.21.15.33:0/3917524379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e1d650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 -- 172.21.15.33:0/3917524379 >> 172.21.15.33:0/3917524379 conn(0x561130e188a0 msgr2=0x561130e18ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 -- 172.21.15.33:0/3917524379 shutdown_connections 2024-04-24T23:27:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 -- 172.21.15.33:0/3917524379 wait complete. 2024-04-24T23:27:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 -- start start 2024-04-24T23:27:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.272+0000 7efdb5c03740 1 Processor -- start 2024-04-24T23:27:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e22d90 0x561130e2e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561130e25b00 con 0x561130e1d230 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561130d3d050 con 0x561130e1dc20 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36242/0 (socket says 172.21.15.33:36242) 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb0bb1640 1 -- 172.21.15.33:0/2817402355 learned_addr learned my addr 172.21.15.33:0/2817402355 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdabfff640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb13b2640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e22d90 0x561130e2e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561130e25c80 con 0x561130e22d90 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb5c03740 10 client.0 _ll_register_callbacks cb 0x561130a94be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb0bb1640 1 -- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e22d90 msgr2=0x561130e2e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e22d90 0x561130e2e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.273+0000 7efdb0bb1640 1 -- 172.21.15.33:0/2817402355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 msgr2=0x561130e27600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.274+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e27600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.274+0000 7efdb0bb1640 1 -- 172.21.15.33:0/2817402355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561130d0edc0 con 0x561130e1d230 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.274+0000 7efdabfff640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e27600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.274+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e270c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7efda00bbd20 tx=0x7efda00bb6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b543893df53924 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efd8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efd8bfff640 1 -- 172.21.15.33:0/2817402355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0048050 con 0x561130e1d230 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efd8bfff640 1 -- 172.21.15.33:0/2817402355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efda00bc1f0 con 0x561130e1d230 2024-04-24T23:27:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda00b4ff0 con 0x561130e1d230 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efd8bfff640 1 -- 172.21.15.33:0/2817402355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efda00c0090 con 0x561130e1d230 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efdb5c03740 10 client.18753 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.276+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561130a953f0 con 0x561130e1d230 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efd8bfff640 1 -- 172.21.15.33:0/2817402355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda00b4ff0 con 0x561130e1d230 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efd8bfff640 1 -- 172.21.15.33:0/2817402355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efda004b060 con 0x561130e1d230 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdb5c03740 10 client.18753 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdb5c03740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdb5c03740 20 client.18753 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdb5c03740 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e32b40 0x561130e34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdb5c03740 4 client.18753 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561130d8ebb0 con 0x561130e32b40 2024-04-24T23:27:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.277+0000 7efdabfff640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e32b40 0x561130e34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.278+0000 7efd8bfff640 10 client.18753 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.278+0000 7efdabfff640 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e32b40 0x561130e34fb0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d06d2fd6213308 server_cookie=d052863ff290f4ad in_seq=0 out_seq=0 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.279+0000 7efd8bfff640 1 -- 172.21.15.33:0/2817402355 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561130d8ebb0 con 0x561130e32b40 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.279+0000 7efd8bfff640 10 client.18753 handle_command_reply: tid=0 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.279+0000 7efdb5c03740 1 client.18753 shutdown 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.279+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 msgr2=0x561130e270c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.279+0000 7efdb5c03740 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e270c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7efda00bbd20 tx=0x7efda00bb6f0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 shutdown_connections 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e32b40 0x561130e34fb0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e22d90 0x561130e2e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e27600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efd8bfff640 0 client.18753 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 --2- 172.21.15.33:0/2817402355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e270c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 >> 172.21.15.33:0/2817402355 conn(0x561130e188a0 msgr2=0x561130e17b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 shutdown_connections 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 1 -- 172.21.15.33:0/2817402355 wait complete. 2024-04-24T23:27:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.280+0000 7efdb5c03740 20 client.18753 trim_cache size 0 max 16384 2024-04-24T23:27:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 Processor -- start 2024-04-24T23:27:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 -- start start 2024-04-24T23:27:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e2e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e1e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1aac00 con 0x561130e1d230 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a8470 con 0x561130e1dc20 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb5c03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a62c0 con 0x561130e188a0 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e2e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e223e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52562/0 (socket says 172.21.15.33:52562) 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb0bb1640 1 -- 172.21.15.33:0/1961904835 learned_addr learned my addr 172.21.15.33:0/1961904835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.297+0000 7efdb13b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e1e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdb0bb1640 1 -- 172.21.15.33:0/1961904835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 msgr2=0x561130e1e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e1e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdb0bb1640 1 -- 172.21.15.33:0/1961904835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 msgr2=0x561130e2e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e2e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdb0bb1640 1 -- 172.21.15.33:0/1961904835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561130b5f7d0 con 0x561130e188a0 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdabfff640 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e2e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e223e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efda00bba90 tx=0x7efda0009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fb9b431f2e577b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efd967fc640 1 -- 172.21.15.33:0/1961904835 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0048070 con 0x561130e188a0 2024-04-24T23:27:49.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efd967fc640 1 -- 172.21.15.33:0/1961904835 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda00da6e0 con 0x561130e188a0 2024-04-24T23:27:49.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.298+0000 7efd967fc640 1 -- 172.21.15.33:0/1961904835 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00bbff0 con 0x561130e188a0 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 -- 172.21.15.33:0/1961904835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 msgr2=0x561130e223e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e223e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efda00bba90 tx=0x7efda0009880 comp rx=0 tx=0).stop 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 -- 172.21.15.33:0/1961904835 shutdown_connections 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1dc20 0x561130e1e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1d230 0x561130e2e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 --2- 172.21.15.33:0/1961904835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e223e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 -- 172.21.15.33:0/1961904835 >> 172.21.15.33:0/1961904835 conn(0x561130e27760 msgr2=0x561130e17b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 -- 172.21.15.33:0/1961904835 shutdown_connections 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 -- 172.21.15.33:0/1961904835 wait complete. 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 -- start start 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.299+0000 7efdb5c03740 1 Processor -- start 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e2b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e2be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1dc20 0x561130e2ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a6930 con 0x561130e1dc20 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a7da0 con 0x561130e1d230 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a7000 con 0x561130e188a0 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 10 client.0 _ll_register_callbacks cb 0x7efd98009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb13b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1dc20 0x561130e2ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e2be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e2b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb13b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1dc20 0x561130e2ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36260/0 (socket says 172.21.15.33:36260) 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb13b2640 1 -- 172.21.15.33:0/599485137 learned_addr learned my addr 172.21.15.33:0/599485137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 -- 172.21.15.33:0/599485137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 msgr2=0x561130e2be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e2be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 -- 172.21.15.33:0/599485137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1dc20 msgr2=0x561130e2ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1dc20 0x561130e2ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 -- 172.21.15.33:0/599485137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561130e2d430 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb0bb1640 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e2b8f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efda002a910 tx=0x7efda00bb3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbbf4701a6c054a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efd8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efd8bfff640 1 -- 172.21.15.33:0/599485137 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0048070 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561130d88f00 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.300+0000 7efd8bfff640 1 -- 172.21.15.33:0/599485137 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efda00039f0 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efd8bfff640 1 -- 172.21.15.33:0/599485137 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efda00bb6f0 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 10 client.28204 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561130e2d430 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efd8bfff640 1 -- 172.21.15.33:0/599485137 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561130d88f00 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efd8bfff640 1 -- 172.21.15.33:0/599485137 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efda00bac50 con 0x561130e188a0 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 10 client.28204 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 10 client.28204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 20 client.28204 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:49.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e22d90 0x561130e9b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 4 client.28204 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561130d06820 con 0x561130e22d90 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.301+0000 7efdabfff640 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e22d90 0x561130e9b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efd8bfff640 10 client.28204 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efdabfff640 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e22d90 0x561130e9b270 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6235236995ddc3 server_cookie=93417a331277ce55 in_seq=0 out_seq=0 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efd8bfff640 1 -- 172.21.15.33:0/599485137 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561130d06820 con 0x561130e22d90 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efd8bfff640 10 client.28204 handle_command_reply: tid=0 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efdb5c03740 1 client.28204 shutdown 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 msgr2=0x561130e2b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.302+0000 7efdb5c03740 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e2b8f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efda002a910 tx=0x7efda00bb3f0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 shutdown_connections 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561130e22d90 0x561130e9b270 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561130e1dc20 0x561130e2ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561130e1d230 0x561130e2be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efd8bfff640 0 client.28204 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 --2- 172.21.15.33:0/599485137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130e188a0 0x561130e2b8f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 >> 172.21.15.33:0/599485137 conn(0x561130e27760 msgr2=0x561130e1c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 shutdown_connections 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 1 -- 172.21.15.33:0/599485137 wait complete. 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb5c03740 20 client.28204 trim_cache size 0 max 16384 2024-04-24T23:27:49.308 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd9c0a2b90 msgr2=0x7efd9c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd9c0a2b90 0x7efd9c0a5050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efda40098b0 tx=0x7efda4009840 comp rx=0 tx=0).stop 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac102210 msgr2=0x7efdac197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac102210 0x7efdac197ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efda0031440 tx=0x7efda0031b60 comp rx=0 tx=0).stop 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 shutdown_connections 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efd9c0a2b90 0x7efd9c0a5050 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdac107470 0x7efdac19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdac104b40 0x7efdac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 --2- 172.21.15.33:0/3407446566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdac102210 0x7efdac197ae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.303+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 >> 172.21.15.33:0/3407446566 conn(0x7efdac0fbc00 msgr2=0x7efdac102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.304+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 shutdown_connections 2024-04-24T23:27:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:49.304+0000 7efdb2e15640 1 -- 172.21.15.33:0/3407446566 wait complete. 2024-04-24T23:27:49.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:49 smithi033 ceph-mon[29201]: pgmap v825: 129 pgs: 129 active+clean; 521 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 429 op/s 2024-04-24T23:27:49.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:49 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1547444901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:49 smithi117 ceph-mon[32529]: pgmap v825: 129 pgs: 129 active+clean; 521 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 429 op/s 2024-04-24T23:27:49.727 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:49 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1547444901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 51809 26.03 MB/sec warmup 114 sec latency 136.600 ms 2024-04-24T23:27:49.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:49 smithi038 ceph-mon[32450]: pgmap v825: 129 pgs: 129 active+clean; 521 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 429 op/s 2024-04-24T23:27:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:49 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1547444901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:50.317 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:27:50.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:50.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:27:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 -- 172.21.15.33:0/4082909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150103340 msgr2=0x7fd150109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 --2- 172.21.15.33:0/4082909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150103340 0x7fd150109bd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd148009ca0 tx=0x7fd14802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 -- 172.21.15.33:0/4082909851 shutdown_connections 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 --2- 172.21.15.33:0/4082909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150103340 0x7fd150109bd0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 --2- 172.21.15.33:0/4082909851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150102980 0x7fd150102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 --2- 172.21.15.33:0/4082909851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd150101790 0x7fd150101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 -- 172.21.15.33:0/4082909851 >> 172.21.15.33:0/4082909851 conn(0x7fd1500fcd20 msgr2=0x7fd1500ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 -- 172.21.15.33:0/4082909851 shutdown_connections 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.444+0000 7fd156a6b640 1 -- 172.21.15.33:0/4082909851 wait complete. 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 Processor -- start 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 -- start start 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd150101790 0x7fd1500754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 0x7fd150073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150103340 0x7fd150074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd15010d410 con 0x7fd150102980 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd15010d290 con 0x7fd150103340 2024-04-24T23:27:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd156a6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd15010d590 con 0x7fd150101790 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 0x7fd150073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 0x7fd150073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36272/0 (socket says 172.21.15.33:36272) 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 -- 172.21.15.33:0/2558327649 learned_addr learned my addr 172.21.15.33:0/2558327649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd155008640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150103340 0x7fd150074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd154807640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd150101790 0x7fd1500754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 -- 172.21.15.33:0/2558327649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd150101790 msgr2=0x7fd1500754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd150101790 0x7fd1500754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 -- 172.21.15.33:0/2558327649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150103340 msgr2=0x7fd150074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.445+0000 7fd147fff640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150103340 0x7fd150074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd147fff640 1 -- 172.21.15.33:0/2558327649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd150074900 con 0x7fd150102980 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd155008640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150103340 0x7fd150074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd147fff640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 0x7fd150073b30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd14000e9f0 tx=0x7fd14000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd145ffb640 1 -- 172.21.15.33:0/2558327649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14000cd80 con 0x7fd150102980 2024-04-24T23:27:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1501a1b50 con 0x7fd150102980 2024-04-24T23:27:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd145ffb640 1 -- 172.21.15.33:0/2558327649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd140010910 con 0x7fd150102980 2024-04-24T23:27:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd145ffb640 1 -- 172.21.15.33:0/2558327649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140018950 con 0x7fd150102980 2024-04-24T23:27:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.446+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1501a20f0 con 0x7fd150102980 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd145ffb640 1 -- 172.21.15.33:0/2558327649 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd140010ac0 con 0x7fd150102980 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 Processor -- start 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 -- start start 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e2421191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e24211e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e242120d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e24207c2c0 con 0x55e242118d90 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e242024910 con 0x55e242119780 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd15977b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e242078280 con 0x55e24211e8f0 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd155008640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e2421191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd145ffb640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1380a2b90 0x7fd1380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd155008640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e2421191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36280/0 (socket says 172.21.15.33:36280) 2024-04-24T23:27:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd154807640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e24211e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd155008640 1 -- 172.21.15.33:0/622054750 learned_addr learned my addr 172.21.15.33:0/622054750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.449+0000 7fd147fff640 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e242120d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd154807640 1 -- 172.21.15.33:0/622054750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 msgr2=0x55e242120d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd154807640 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e242120d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd145ffb640 1 -- 172.21.15.33:0/2558327649 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd1400cf780 con 0x7fd150102980 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd154807640 1 -- 172.21.15.33:0/622054750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 msgr2=0x55e2421191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd154807640 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e2421191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd154807640 1 -- 172.21.15.33:0/622054750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e242035fb0 con 0x55e242119780 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd155008640 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e2421191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd155008640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1380a2b90 0x7fd1380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd154807640 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e24211e280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009bd0 tx=0x7fd14c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b423b01fd5740f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd136ffd640 1 -- 172.21.15.33:0/622054750 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c004230 con 0x55e242119780 2024-04-24T23:27:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd155008640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1380a2b90 0x7fd1380a5050 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55e242119fe0 tx=0x7fd148040040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd136ffd640 1 -- 172.21.15.33:0/622054750 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c009e40 con 0x55e242119780 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.450+0000 7fd136ffd640 1 -- 172.21.15.33:0/622054750 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c019820 con 0x55e242119780 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 -- 172.21.15.33:0/622054750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 msgr2=0x55e24211e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e24211e280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009bd0 tx=0x7fd14c011810 comp rx=0 tx=0).stop 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 -- 172.21.15.33:0/622054750 shutdown_connections 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e242120d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e24211e280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 --2- 172.21.15.33:0/622054750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e2421191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 -- 172.21.15.33:0/622054750 >> 172.21.15.33:0/622054750 conn(0x55e2420e0950 msgr2=0x55e242114810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 -- 172.21.15.33:0/622054750 shutdown_connections 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.451+0000 7fd15977b740 1 -- 172.21.15.33:0/622054750 wait complete. 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 -- start start 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 Processor -- start 2024-04-24T23:27:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e242123220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e242123760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e24212a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2421218a0 con 0x55e242118d90 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e242037e90 con 0x55e242119780 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e242121a20 con 0x55e24211e8f0 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd154807640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e242123220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd154807640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e242123220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36286/0 (socket says 172.21.15.33:36286) 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 -- 172.21.15.33:0/1041620420 learned_addr learned my addr 172.21.15.33:0/1041620420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.452+0000 7fd15977b740 10 client.0 _ll_register_callbacks cb 0x55e242073970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd147fff640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e242123760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd155008640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e24212a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 -- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 msgr2=0x55e24212a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e24212a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 -- 172.21.15.33:0/1041620420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 msgr2=0x55e242123760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e242123760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 -- 172.21.15.33:0/1041620420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e241e587e0 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd154807640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e242123220 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd14c01b040 tx=0x7fd14c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9829b835d555493c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd12ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd12ffff640 1 -- 172.21.15.33:0/1041620420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c027070 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd12ffff640 1 -- 172.21.15.33:0/1041620420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd14c0054d0 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.453+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2420505a0 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd12ffff640 1 -- 172.21.15.33:0/1041620420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd14c033050 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 10 client.18771 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e241eb8f70 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd12ffff640 1 -- 172.21.15.33:0/1041620420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e2420505a0 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd12ffff640 1 -- 172.21.15.33:0/1041620420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd14c02c560 con 0x55e242118d90 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 10 client.18771 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 10 client.18771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 20 client.18771 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242130f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 4 client.18771 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e242089800 con 0x55e24212eae0 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.454+0000 7fd147fff640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.455+0000 7fd147fff640 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242130f80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94742775d9c72dee server_cookie=8cd7204e4bb99c8e in_seq=0 out_seq=0 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.455+0000 7fd12ffff640 10 client.18771 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.455+0000 7fd12ffff640 1 -- 172.21.15.33:0/1041620420 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e242089800 con 0x55e24212eae0 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.455+0000 7fd12ffff640 10 client.18771 handle_command_reply: tid=0 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.455+0000 7fd15977b740 1 client.18771 shutdown 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 msgr2=0x55e242123220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e242123220 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd14c01b040 tx=0x7fd14c005d90 comp rx=0 tx=0).stop 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 shutdown_connections 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242130f80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd12ffff640 0 client.18771 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e24211e8f0 0x55e24212a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242119780 0x55e242123760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 --2- 172.21.15.33:0/1041620420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242118d90 0x55e242123220 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 >> 172.21.15.33:0/1041620420 conn(0x55e2420e0950 msgr2=0x55e242113950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 shutdown_connections 2024-04-24T23:27:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 1 -- 172.21.15.33:0/1041620420 wait complete. 2024-04-24T23:27:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.456+0000 7fd15977b740 20 client.18771 trim_cache size 0 max 16384 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 Processor -- start 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 -- start start 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242118d90 0x55e242113730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 0x55e24212a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e24211e8f0 0x55e242136850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501aeef0 con 0x55e242119780 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501ac760 con 0x55e24211e8f0 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd15977b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501aa5b0 con 0x55e242118d90 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 0x55e24212a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 0x55e24212a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36300/0 (socket says 172.21.15.33:36300) 2024-04-24T23:27:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd154807640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242118d90 0x55e242113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd147fff640 1 -- 172.21.15.33:0/575162793 learned_addr learned my addr 172.21.15.33:0/575162793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.480+0000 7fd155008640 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e24211e8f0 0x55e242136850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd147fff640 1 -- 172.21.15.33:0/575162793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242118d90 msgr2=0x55e242113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd147fff640 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242118d90 0x55e242113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd147fff640 1 -- 172.21.15.33:0/575162793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e24211e8f0 msgr2=0x55e242136850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd147fff640 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e24211e8f0 0x55e242136850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd147fff640 1 -- 172.21.15.33:0/575162793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e241d50170 con 0x55e242119780 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd147fff640 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 0x55e24212a760 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c6a0 tx=0x7fd14009df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66199df59f3dfa70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd1367fc640 1 -- 172.21.15.33:0/575162793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400b1030 con 0x55e242119780 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd1367fc640 1 -- 172.21.15.33:0/575162793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1400b4e50 con 0x55e242119780 2024-04-24T23:27:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.481+0000 7fd1367fc640 1 -- 172.21.15.33:0/575162793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140026040 con 0x55e242119780 2024-04-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 -- 172.21.15.33:0/575162793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 msgr2=0x55e24212a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 0x55e24212a760 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c6a0 tx=0x7fd14009df90 comp rx=0 tx=0).stop 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 -- 172.21.15.33:0/575162793 shutdown_connections 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e24211e8f0 0x55e242136850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e242119780 0x55e24212a760 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 --2- 172.21.15.33:0/575162793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242118d90 0x55e242113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 -- 172.21.15.33:0/575162793 >> 172.21.15.33:0/575162793 conn(0x55e2420e0950 msgr2=0x55e242118070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 -- 172.21.15.33:0/575162793 shutdown_connections 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 -- 172.21.15.33:0/575162793 wait complete. 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 -- start start 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.482+0000 7fd15977b740 1 Processor -- start 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 0x55e2421281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242119780 0x55e2421286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e24211e8f0 0x55e242129680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501aac20 con 0x55e24211e8f0 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501ac090 con 0x55e242118d90 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501ab2f0 con 0x55e242119780 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd15977b740 10 client.0 _ll_register_callbacks cb 0x7fd14c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd154807640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 0x55e2421281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd147fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242119780 0x55e2421286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e24211e8f0 0x55e242129680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd154807640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 0x55e2421281a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32902/0 (socket says 172.21.15.33:32902) 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd147fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242119780 0x55e2421286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52630/0 (socket says 172.21.15.33:52630) 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd154807640 1 -- 172.21.15.33:0/1543852414 learned_addr learned my addr 172.21.15.33:0/1543852414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 -- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242119780 msgr2=0x55e2421286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242119780 0x55e2421286e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 -- 172.21.15.33:0/1543852414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 msgr2=0x55e2421281a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 0x55e2421281a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 -- 172.21.15.33:0/1543852414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e242129c70 con 0x55e24211e8f0 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd154807640 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 0x55e2421281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd155008640 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e24211e8f0 0x55e242129680 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd148006010 tx=0x7fd1480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b1f0e451b6018a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.483+0000 7fd12ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd12ffff640 1 -- 172.21.15.33:0/1543852414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148059060 con 0x55e24211e8f0 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd12ffff640 1 -- 172.21.15.33:0/1543852414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd148031cb0 con 0x55e24211e8f0 2024-04-24T23:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1400b4e50 con 0x55e24211e8f0 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd12ffff640 1 -- 172.21.15.33:0/1543852414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd148054440 con 0x55e24211e8f0 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 10 client.18777 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2421953f0 con 0x55e24211e8f0 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd12ffff640 1 -- 172.21.15.33:0/1543852414 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1400b4e50 con 0x55e24211e8f0 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd12ffff640 1 -- 172.21.15.33:0/1543852414 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd1480477a0 con 0x55e24211e8f0 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 10 client.18777 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 20 client.18777 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242197e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 4 client.18777 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e242083b50 con 0x55e24212eae0 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.484+0000 7fd154807640 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.485+0000 7fd12ffff640 10 client.18777 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.485+0000 7fd154807640 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242197e60 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0197b57d17f25cc server_cookie=bf191301123ae9 in_seq=0 out_seq=0 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd12ffff640 1 -- 172.21.15.33:0/1543852414 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e242083b50 con 0x55e24212eae0 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd12ffff640 10 client.18777 handle_command_reply: tid=0 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 client.18777 shutdown 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e24211e8f0 msgr2=0x55e242129680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e24211e8f0 0x55e242129680 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd148006010 tx=0x7fd1480095c0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 shutdown_connections 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e24212eae0 0x55e242197e60 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd12ffff640 0 client.18777 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e24211e8f0 0x55e242129680 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e242119780 0x55e2421286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 --2- 172.21.15.33:0/1543852414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e242118d90 0x55e2421281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 >> 172.21.15.33:0/1543852414 conn(0x55e2420e0950 msgr2=0x55e242135840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 shutdown_connections 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 1 -- 172.21.15.33:0/1543852414 wait complete. 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.486+0000 7fd15977b740 20 client.18777 trim_cache size 0 max 16384 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:27:50.492 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "432b328b-603c-4ef9-bc40-0ca41cc710fb", 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1380a2b90 msgr2=0x7fd1380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1380a2b90 0x7fd1380a5050 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55e242119fe0 tx=0x7fd148040040 comp rx=0 tx=0).stop 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 msgr2=0x7fd150073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 0x7fd150073b30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd14000e9f0 tx=0x7fd14000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 shutdown_connections 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd1380a2b90 0x7fd1380a5050 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd150103340 0x7fd150074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd150102980 0x7fd150073b30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 --2- 172.21.15.33:0/2558327649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd150101790 0x7fd1500754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 >> 172.21.15.33:0/2558327649 conn(0x7fd1500fcd20 msgr2=0x7fd150105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 shutdown_connections 2024-04-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.487+0000 7fd156a6b640 1 -- 172.21.15.33:0/2558327649 wait complete. 2024-04-24T23:27:50.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:27:50.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:27:50.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 -- 172.21.15.33:0/94762598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 msgr2=0x7fb8fc100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 --2- 172.21.15.33:0/94762598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc100aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009860 tx=0x7fb8ec02a340 comp rx=0 tx=0).stop 2024-04-24T23:27:50.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 -- 172.21.15.33:0/94762598 shutdown_connections 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 --2- 172.21.15.33:0/94762598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8fc102230 0x7fb8fc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 --2- 172.21.15.33:0/94762598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 0x7fb8fc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 --2- 172.21.15.33:0/94762598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc100aa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 -- 172.21.15.33:0/94762598 >> 172.21.15.33:0/94762598 conn(0x7fb8fc0fbbf0 msgr2=0x7fb8fc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 -- 172.21.15.33:0/94762598 shutdown_connections 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.625+0000 7fb900841640 1 -- 172.21.15.33:0/94762598 wait complete. 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 Processor -- start 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 -- start start 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 0x7fb8fc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8fc102230 0x7fb8fc19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10c300 con 0x7fb8fc101870 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10c180 con 0x7fb8fc102230 2024-04-24T23:27:50.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb900841640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10c480 con 0x7fb8fc100680 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 0x7fb8fc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52648/0 (socket says 172.21.15.33:52648) 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 -- 172.21.15.33:0/224607812 learned_addr learned my addr 172.21.15.33:0/224607812 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8fc102230 0x7fb8fc19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 -- 172.21.15.33:0/224607812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8fc102230 msgr2=0x7fb8fc19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8fc102230 0x7fb8fc19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 -- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 msgr2=0x7fb8fc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.626+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 0x7fb8fc198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb8fa59c640 1 -- 172.21.15.33:0/224607812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc19f720 con 0x7fb8fc100680 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 0x7fb8fc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc197b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0305b0 tx=0x7fb8ec033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb8eb7fe640 1 -- 172.21.15.33:0/224607812 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec03f020 con 0x7fb8fc100680 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb8eb7fe640 1 -- 172.21.15.33:0/224607812 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8ec030850 con 0x7fb8fc100680 2024-04-24T23:27:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb8eb7fe640 1 -- 172.21.15.33:0/224607812 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec037040 con 0x7fb8fc100680 2024-04-24T23:27:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc19f9b0 con 0x7fb8fc100680 2024-04-24T23:27:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.627+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc19fea0 con 0x7fb8fc100680 2024-04-24T23:27:50.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.629+0000 7fb8eb7fe640 1 -- 172.21.15.33:0/224607812 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb8ec042020 con 0x7fb8fc100680 2024-04-24T23:27:50.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.629+0000 7fb90361f740 1 Processor -- start 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 -- start start 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5ed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56084e54c6b0 con 0x56084e5e8c30 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56084e4f4880 con 0x56084e5edda0 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb90361f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56084e548670 con 0x56084e5e8240 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8eb7fe640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8e40a2b90 0x7fb8e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5ed730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36324/0 (socket says 172.21.15.33:36324) 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8fa59c640 1 -- 172.21.15.33:0/3054565418 learned_addr learned my addr 172.21.15.33:0/3054565418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8fad9d640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8e40a2b90 0x7fb8e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8eb7fe640 1 -- 172.21.15.33:0/224607812 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8ec0ef6f0 con 0x7fb8fc100680 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/3054565418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 msgr2=0x56084e5e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/3054565418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 msgr2=0x56084e5ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.630+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/3054565418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084e32a7d0 con 0x56084e5edda0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.631+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f0230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009530 tx=0x7fb8f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a4ef67a030d6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.631+0000 7fb8e8ff9640 1 -- 172.21.15.33:0/3054565418 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0004020 con 0x56084e5edda0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.631+0000 7fb8fad9d640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8e40a2b90 0x7fb8e40a5050 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x56084e5e9490 tx=0x7fb8f4009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.631+0000 7fb8e8ff9640 1 -- 172.21.15.33:0/3054565418 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f00041c0 con 0x56084e5edda0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.631+0000 7fb8e8ff9640 1 -- 172.21.15.33:0/3054565418 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f000f920 con 0x56084e5edda0 2024-04-24T23:27:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- 172.21.15.33:0/3054565418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 msgr2=0x56084e5f0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f0230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009530 tx=0x7fb8f00118d0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- 172.21.15.33:0/3054565418 shutdown_connections 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f0230 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- 172.21.15.33:0/3054565418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- 172.21.15.33:0/3054565418 >> 172.21.15.33:0/3054565418 conn(0x56084e5e38b0 msgr2=0x56084e5e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- 172.21.15.33:0/3054565418 shutdown_connections 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- 172.21.15.33:0/3054565418 wait complete. 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- start start 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 Processor -- start 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56084e5f0d50 con 0x56084e5e8c30 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56084e507f50 con 0x56084e5edda0 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56084e5f0ed0 con 0x56084e5e8240 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb90361f740 10 client.0 _ll_register_callbacks cb 0x56084e543000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb8f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:52656/0 (socket says 172.21.15.33:52656) 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.632+0000 7fb8fa59c640 1 -- 172.21.15.33:0/3316700650 learned_addr learned my addr 172.21.15.33:0/3316700650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 msgr2=0x56084e5f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/3316700650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 msgr2=0x56084e5f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/3316700650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084e2603f0 con 0x56084e5e8c30 2024-04-24T23:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb8f001b040 tx=0x7fb8f0005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6989dc980df14f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/3316700650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f001d050 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/3316700650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8f00035f0 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f00041c0 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/3316700650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8f0032050 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb90361f740 10 client.18789 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56084e2200f0 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.633+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/3316700650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f00041c0 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/3316700650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb8f0004cc0 con 0x56084e5e8c30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb90361f740 10 client.18789 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb90361f740 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb90361f740 20 client.18789 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb90361f740 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e5fe290 0x56084e600680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb90361f740 4 client.18789 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084e559bf0 con 0x56084e5fe290 2024-04-24T23:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e5fe290 0x56084e600680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e5fe290 0x56084e600680 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e60748123a4bce server_cookie=c4564087e3f7578c in_seq=0 out_seq=0 2024-04-24T23:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.634+0000 7fb8e1ffb640 10 client.18789 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.635+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/3316700650 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56084e559bf0 con 0x56084e5fe290 2024-04-24T23:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.635+0000 7fb8e1ffb640 10 client.18789 handle_command_reply: tid=0 2024-04-24T23:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.635+0000 7fb90361f740 1 client.18789 shutdown 2024-04-24T23:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.635+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 msgr2=0x56084e5f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.635+0000 7fb90361f740 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb8f001b040 tx=0x7fb8f0005510 comp rx=0 tx=0).stop 2024-04-24T23:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.635+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 shutdown_connections 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e5fe290 0x56084e600680 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 --2- 172.21.15.33:0/3316700650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb8e1ffb640 0 client.18789 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 >> 172.21.15.33:0/3316700650 conn(0x56084e5e38b0 msgr2=0x56084e5e2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 shutdown_connections 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 1 -- 172.21.15.33:0/3316700650 wait complete. 2024-04-24T23:27:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.636+0000 7fb90361f740 20 client.18789 trim_cache size 0 max 16384 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 Processor -- start 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 -- start start 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1acd80 con 0x56084e5e8c30 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1aa5f0 con 0x56084e5edda0 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb90361f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1a8440 con 0x56084e5e8240 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb8f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb8f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36340/0 (socket says 172.21.15.33:36340) 2024-04-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/2101288863 learned_addr learned my addr 172.21.15.33:0/2101288863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.658+0000 7fb8fad9d640 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/2101288863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 msgr2=0x56084e5f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/2101288863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 msgr2=0x56084e5f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8f9d9b640 1 -- 172.21.15.33:0/2101288863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084e5e3450 con 0x56084e5e8c30 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0011910 tx=0x7fb8f0009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc90e031e4d5be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8e3fff640 1 -- 172.21.15.33:0/2101288863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f003b070 con 0x56084e5e8c30 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8e3fff640 1 -- 172.21.15.33:0/2101288863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f0029070 con 0x56084e5e8c30 2024-04-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.659+0000 7fb8e3fff640 1 -- 172.21.15.33:0/2101288863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f001d040 con 0x56084e5e8c30 2024-04-24T23:27:50.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 -- 172.21.15.33:0/2101288863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 msgr2=0x56084e5f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0011910 tx=0x7fb8f0009cc0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 -- 172.21.15.33:0/2101288863 shutdown_connections 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5edda0 0x56084e5f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8c30 0x56084e5f2bb0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 --2- 172.21.15.33:0/2101288863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5e8240 0x56084e5f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 -- 172.21.15.33:0/2101288863 >> 172.21.15.33:0/2101288863 conn(0x56084e5e38b0 msgr2=0x56084e600150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 -- 172.21.15.33:0/2101288863 shutdown_connections 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.660+0000 7fb90361f740 1 -- 172.21.15.33:0/2101288863 wait complete. 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 -- start start 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 Processor -- start 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 0x56084e5f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5e8c30 0x56084e5f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5edda0 0x56084e5f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1a8ab0 con 0x56084e5e8240 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1a9f20 con 0x56084e5e8c30 2024-04-24T23:27:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1a9180 con 0x56084e5edda0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb90361f740 10 client.0 _ll_register_callbacks cb 0x7fb8ec032810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 0x56084e5f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb8f9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5e8c30 0x56084e5f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 0x56084e5f95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36356/0 (socket says 172.21.15.33:36356) 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb8fa59c640 1 -- 172.21.15.33:0/2330307679 learned_addr learned my addr 172.21.15.33:0/2330307679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.661+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5edda0 0x56084e5f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8fa59c640 1 -- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5edda0 msgr2=0x56084e5f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5edda0 0x56084e5f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8fa59c640 1 -- 172.21.15.33:0/2330307679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5e8c30 msgr2=0x56084e5f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5e8c30 0x56084e5f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8fa59c640 1 -- 172.21.15.33:0/2330307679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084e5e4320 con 0x56084e5e8240 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5e8c30 0x56084e5f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8fa59c640 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 0x56084e5f95e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0b8590 tx=0x7fb8ec003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e1cfbec8989caa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:27:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/2330307679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0bbbc0 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/2330307679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8ec003d10 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f0029070 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/2330307679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec0bff70 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb90361f740 10 client.18795 fetch_fsmap learned FSMap version 30 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.662+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56084e5f1010 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/2330307679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f0029070 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/2330307679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb8ec03a030 con 0x56084e5e8240 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb90361f740 10 client.18795 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb90361f740 10 client.18795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb90361f740 20 client.18795 populate_metadata read hostname 'smithi033' 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb90361f740 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e663b40 0x56084e665f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb90361f740 4 client.18795 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56084e553f40 con 0x56084e663b40 2024-04-24T23:27:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.663+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e663b40 0x56084e665f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.664+0000 7fb8e1ffb640 10 client.18795 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.664+0000 7fb8f9d9b640 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e663b40 0x56084e665f60 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f149ea899265f1e2 server_cookie=716fcba07f85ba7f in_seq=0 out_seq=0 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.665+0000 7fb8e1ffb640 1 -- 172.21.15.33:0/2330307679 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56084e553f40 con 0x56084e663b40 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.665+0000 7fb8e1ffb640 10 client.18795 handle_command_reply: tid=0 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.665+0000 7fb90361f740 1 client.18795 shutdown 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.665+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 msgr2=0x56084e5f95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.665+0000 7fb90361f740 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 0x56084e5f95e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0b8590 tx=0x7fb8ec003760 comp rx=0 tx=0).stop 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 shutdown_connections 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56084e663b40 0x56084e665f60 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56084e5edda0 0x56084e5f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb8e1ffb640 0 client.18795 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56084e5e8c30 0x56084e5f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 --2- 172.21.15.33:0/2330307679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56084e5e8240 0x56084e5f95e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 >> 172.21.15.33:0/2330307679 conn(0x56084e5e38b0 msgr2=0x56084e5f2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 shutdown_connections 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 1 -- 172.21.15.33:0/2330307679 wait complete. 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb90361f740 20 client.18795 trim_cache size 0 max 16384 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout: "432b328b-603c-4ef9-bc40-0ca41cc710fb": { 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "432b328b-603c-4ef9-bc40-0ca41cc710fb", 2024-04-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8e40a2b90 msgr2=0x7fb8e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb900841640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8e40a2b90 0x7fb8e40a5050 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x56084e5e9490 tx=0x7fb8f4009500 comp rx=0 tx=0).stop 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 msgr2=0x7fb8fc197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.666+0000 7fb900841640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc197b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0305b0 tx=0x7fb8ec033d90 comp rx=0 tx=0).stop 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 shutdown_connections 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8e40a2b90 0x7fb8e40a5050 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8fc102230 0x7fb8fc19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc101870 0x7fb8fc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 --2- 172.21.15.33:0/224607812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8fc100680 0x7fb8fc197b00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 >> 172.21.15.33:0/224607812 conn(0x7fb8fc0fbbf0 msgr2=0x7fb8fc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 shutdown_connections 2024-04-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:27:50.667+0000 7fb900841640 1 -- 172.21.15.33:0/224607812 wait complete. 2024-04-24T23:27:50.679 INFO:tasks.cephfs.filesystem:scrub status for tag:432b328b-603c-4ef9-bc40-0ca41cc710fb - {'path': '/', 'tag': '432b328b-603c-4ef9-bc40-0ca41cc710fb', 'options': 'recursive,force'} 2024-04-24T23:27:50.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 52176 25.90 MB/sec warmup 115 sec latency 228.080 ms 2024-04-24T23:27:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: pgmap v826: 129 pgs: 129 active+clean; 520 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 362 op/s 2024-04-24T23:27:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:27:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:51.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:51 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: pgmap v826: 129 pgs: 129 active+clean; 520 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 362 op/s 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:51.722 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:51 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 52751 25.94 MB/sec warmup 116 sec latency 164.564 ms 2024-04-24T23:27:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: pgmap v826: 129 pgs: 129 active+clean; 520 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 362 op/s 2024-04-24T23:27:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:27:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:51.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:51 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:52 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:27:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:52 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:52 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:52.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 53366 26.07 MB/sec warmup 117 sec latency 48.626 ms 2024-04-24T23:27:52.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:52 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:27:52.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:52 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:52.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:52 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:52.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:52 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:27:52.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:52 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:52.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:52 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:27:53.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:53 smithi033 ceph-mon[29201]: pgmap v827: 129 pgs: 129 active+clean; 523 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.8 MiB/s wr, 466 op/s 2024-04-24T23:27:53.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 53979 26.16 MB/sec warmup 118 sec latency 64.202 ms 2024-04-24T23:27:53.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:53 smithi038 ceph-mon[32450]: pgmap v827: 129 pgs: 129 active+clean; 523 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.8 MiB/s wr, 466 op/s 2024-04-24T23:27:53.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:53 smithi117 ceph-mon[32529]: pgmap v827: 129 pgs: 129 active+clean; 523 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.8 MiB/s wr, 466 op/s 2024-04-24T23:27:54.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 54474 26.22 MB/sec warmup 119 sec latency 148.247 ms 2024-04-24T23:27:55.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:55 smithi033 ceph-mon[29201]: pgmap v828: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 380 op/s 2024-04-24T23:27:55.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:55 smithi038 ceph-mon[32450]: pgmap v828: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 380 op/s 2024-04-24T23:27:55.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:55 smithi117 ceph-mon[32529]: pgmap v828: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 380 op/s 2024-04-24T23:27:56.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 54947 12.49 MB/sec execute 1 sec latency 421.499 ms 2024-04-24T23:27:57.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:57 smithi033 ceph-mon[29201]: pgmap v829: 129 pgs: 129 active+clean; 527 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.3 MiB/s wr, 442 op/s 2024-04-24T23:27:57.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 55159 11.31 MB/sec execute 2 sec latency 179.655 ms 2024-04-24T23:27:57.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:57 smithi038 ceph-mon[32450]: pgmap v829: 129 pgs: 129 active+clean; 527 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.3 MiB/s wr, 442 op/s 2024-04-24T23:27:57.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:57 smithi117 ceph-mon[32529]: pgmap v829: 129 pgs: 129 active+clean; 527 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.3 MiB/s wr, 442 op/s 2024-04-24T23:27:58.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 55462 11.25 MB/sec execute 3 sec latency 79.264 ms 2024-04-24T23:27:59.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:27:59 smithi033 ceph-mon[29201]: pgmap v830: 129 pgs: 129 active+clean; 535 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.9 MiB/s wr, 429 op/s 2024-04-24T23:27:59.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 55928 13.15 MB/sec execute 4 sec latency 109.944 ms 2024-04-24T23:27:59.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:27:59 smithi038 ceph-mon[32450]: pgmap v830: 129 pgs: 129 active+clean; 535 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.9 MiB/s wr, 429 op/s 2024-04-24T23:27:59.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:27:59 smithi117 ceph-mon[32529]: pgmap v830: 129 pgs: 129 active+clean; 535 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.9 MiB/s wr, 429 op/s 2024-04-24T23:28:00.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 56469 15.60 MB/sec execute 5 sec latency 176.378 ms 2024-04-24T23:28:01.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:01 smithi033 ceph-mon[29201]: pgmap v831: 129 pgs: 129 active+clean; 538 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 362 op/s 2024-04-24T23:28:01.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 56974 18.63 MB/sec execute 6 sec latency 103.269 ms 2024-04-24T23:28:01.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:01 smithi038 ceph-mon[32450]: pgmap v831: 129 pgs: 129 active+clean; 538 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 362 op/s 2024-04-24T23:28:01.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:01 smithi117 ceph-mon[32529]: pgmap v831: 129 pgs: 129 active+clean; 538 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 362 op/s 2024-04-24T23:28:02.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 57509 20.39 MB/sec execute 7 sec latency 81.742 ms 2024-04-24T23:28:02.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:02 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:28:02.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:02 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:28:02.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:02 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:28:03.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:03 smithi033 ceph-mon[29201]: pgmap v832: 129 pgs: 129 active+clean; 511 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-04-24T23:28:03.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:28:03.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:28:03.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 57990 21.01 MB/sec execute 8 sec latency 69.836 ms 2024-04-24T23:28:03.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:03 smithi038 ceph-mon[32450]: pgmap v832: 129 pgs: 129 active+clean; 511 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-04-24T23:28:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:28:03.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:28:03.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:03 smithi117 ceph-mon[32529]: pgmap v832: 129 pgs: 129 active+clean; 511 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-04-24T23:28:03.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:28:03.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:28:04.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 58487 22.30 MB/sec execute 9 sec latency 81.755 ms 2024-04-24T23:28:05.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 58726 21.82 MB/sec execute 10 sec latency 368.266 ms 2024-04-24T23:28:05.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:05 smithi038 ceph-mon[32450]: pgmap v833: 129 pgs: 129 active+clean; 508 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 400 op/s 2024-04-24T23:28:05.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:05 smithi117 ceph-mon[32529]: pgmap v833: 129 pgs: 129 active+clean; 508 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 400 op/s 2024-04-24T23:28:05.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:05 smithi033 ceph-mon[29201]: pgmap v833: 129 pgs: 129 active+clean; 508 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 400 op/s 2024-04-24T23:28:06.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 58971 20.73 MB/sec execute 11 sec latency 239.559 ms 2024-04-24T23:28:07.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 59279 20.10 MB/sec execute 12 sec latency 146.675 ms 2024-04-24T23:28:07.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:07 smithi038 ceph-mon[32450]: pgmap v834: 129 pgs: 129 active+clean; 509 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 420 op/s 2024-04-24T23:28:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:07 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:07 smithi117 ceph-mon[32529]: pgmap v834: 129 pgs: 129 active+clean; 509 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 420 op/s 2024-04-24T23:28:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:07 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:07.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:07 smithi033 ceph-mon[29201]: pgmap v834: 129 pgs: 129 active+clean; 509 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 420 op/s 2024-04-24T23:28:07.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:07 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:08.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 59927 21.37 MB/sec execute 13 sec latency 58.292 ms 2024-04-24T23:28:09.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 60428 21.18 MB/sec execute 14 sec latency 76.059 ms 2024-04-24T23:28:09.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:09 smithi038 ceph-mon[32450]: pgmap v835: 129 pgs: 129 active+clean; 520 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-04-24T23:28:09.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:09 smithi117 ceph-mon[32529]: pgmap v835: 129 pgs: 129 active+clean; 520 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-04-24T23:28:09.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:09 smithi033 ceph-mon[29201]: pgmap v835: 129 pgs: 129 active+clean; 520 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-04-24T23:28:10.882 INFO:tasks.workunit.client.0.smithi033.stdout: 10 60781 20.97 MB/sec execute 15 sec latency 156.049 ms 2024-04-24T23:28:11.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 61262 21.45 MB/sec execute 16 sec latency 93.319 ms 2024-04-24T23:28:11.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:11 smithi038 ceph-mon[32450]: pgmap v836: 129 pgs: 129 active+clean; 520 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-04-24T23:28:11.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:11 smithi117 ceph-mon[32529]: pgmap v836: 129 pgs: 129 active+clean; 520 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-04-24T23:28:11.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:11 smithi033 ceph-mon[29201]: pgmap v836: 129 pgs: 129 active+clean; 520 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-04-24T23:28:12.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 61853 22.29 MB/sec execute 17 sec latency 84.102 ms 2024-04-24T23:28:12.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:12 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:12.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:12 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:12.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:12 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:13.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 62544 23.14 MB/sec execute 18 sec latency 118.292 ms 2024-04-24T23:28:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:13 smithi038 ceph-mon[32450]: pgmap v837: 129 pgs: 129 active+clean; 517 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 444 op/s 2024-04-24T23:28:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:13.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:13 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:28:13.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:13 smithi117 ceph-mon[32529]: pgmap v837: 129 pgs: 129 active+clean; 517 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 444 op/s 2024-04-24T23:28:13.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:13.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:13.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:13 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:28:13.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:13 smithi033 ceph-mon[29201]: pgmap v837: 129 pgs: 129 active+clean; 517 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 444 op/s 2024-04-24T23:28:13.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:13.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:13.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:13 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:28:14.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 62939 23.33 MB/sec execute 19 sec latency 75.747 ms 2024-04-24T23:28:15.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 63232 22.85 MB/sec execute 20 sec latency 211.106 ms 2024-04-24T23:28:15.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:15 smithi038 ceph-mon[32450]: pgmap v838: 129 pgs: 129 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 378 op/s 2024-04-24T23:28:15.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:15 smithi117 ceph-mon[32529]: pgmap v838: 129 pgs: 129 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 378 op/s 2024-04-24T23:28:15.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:15 smithi033 ceph-mon[29201]: pgmap v838: 129 pgs: 129 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 378 op/s 2024-04-24T23:28:16.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 63563 22.53 MB/sec execute 21 sec latency 528.388 ms 2024-04-24T23:28:17.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 63830 22.30 MB/sec execute 22 sec latency 113.999 ms 2024-04-24T23:28:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:17 smithi038 ceph-mon[32450]: pgmap v839: 129 pgs: 129 active+clean; 524 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.1 MiB/s wr, 460 op/s 2024-04-24T23:28:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:17 smithi117 ceph-mon[32529]: pgmap v839: 129 pgs: 129 active+clean; 524 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.1 MiB/s wr, 460 op/s 2024-04-24T23:28:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:17 smithi033 ceph-mon[29201]: pgmap v839: 129 pgs: 129 active+clean; 524 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.1 MiB/s wr, 460 op/s 2024-04-24T23:28:18.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 64174 21.91 MB/sec execute 23 sec latency 98.835 ms 2024-04-24T23:28:19.883 INFO:tasks.workunit.client.0.smithi033.stdout: 10 64838 21.90 MB/sec execute 24 sec latency 68.447 ms 2024-04-24T23:28:19.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:19 smithi038 ceph-mon[32450]: pgmap v840: 129 pgs: 129 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 436 op/s 2024-04-24T23:28:19.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:19 smithi117 ceph-mon[32529]: pgmap v840: 129 pgs: 129 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 436 op/s 2024-04-24T23:28:19.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:19 smithi033 ceph-mon[29201]: pgmap v840: 129 pgs: 129 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 436 op/s 2024-04-24T23:28:20.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:28:20.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 -- 172.21.15.33:0/1062991974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc102250 msgr2=0x7f21dc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 --2- 172.21.15.33:0/1062991974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc102250 0x7f21dc108ae0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009ca0 tx=0x7f21d402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 -- 172.21.15.33:0/1062991974 shutdown_connections 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 --2- 172.21.15.33:0/1062991974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc102250 0x7f21dc108ae0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 --2- 172.21.15.33:0/1062991974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 0x7f21dc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 --2- 172.21.15.33:0/1062991974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21dc1006a0 0x7f21dc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 -- 172.21.15.33:0/1062991974 >> 172.21.15.33:0/1062991974 conn(0x7f21dc0fbc30 msgr2=0x7f21dc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 -- 172.21.15.33:0/1062991974 shutdown_connections 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.803+0000 7f21e10d4640 1 -- 172.21.15.33:0/1062991974 wait complete. 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 Processor -- start 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 -- start start 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 0x7f21dc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 0x7f21dc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21dc102250 0x7f21dc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10d440 con 0x7f21dc1006a0 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10d2c0 con 0x7f21dc102250 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21e10d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10d5c0 con 0x7f21dc101890 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 0x7f21dc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21da59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 0x7f21dc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21db59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21dc102250 0x7f21dc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 0x7f21dc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41476/0 (socket says 172.21.15.33:41476) 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.804+0000 7f21dad9d640 1 -- 172.21.15.33:0/973485114 learned_addr learned my addr 172.21.15.33:0/973485114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21da59c640 1 -- 172.21.15.33:0/973485114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21dc102250 msgr2=0x7f21dc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21da59c640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21dc102250 0x7f21dc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21da59c640 1 -- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 msgr2=0x7f21dc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21da59c640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 0x7f21dc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21da59c640 1 -- 172.21.15.33:0/973485114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc19f660 con 0x7f21dc101890 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21dad9d640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 0x7f21dc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21da59c640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 0x7f21dc197f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21c400e9f0 tx=0x7f21c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21d3fff640 1 -- 172.21.15.33:0/973485114 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c400ccc0 con 0x7f21dc101890 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21d3fff640 1 -- 172.21.15.33:0/973485114 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c4005020 con 0x7f21dc101890 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc19f950 con 0x7f21dc101890 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21d3fff640 1 -- 172.21.15.33:0/973485114 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40049e0 con 0x7f21dc101890 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.805+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc19fe50 con 0x7f21dc101890 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.807+0000 7f21d3fff640 1 -- 172.21.15.33:0/973485114 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f21c4005560 con 0x7f21dc101890 2024-04-24T23:28:20.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.808+0000 7f21d3fff640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f21c80a2b90 0x7f21c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.808+0000 7f21e3de3740 1 Processor -- start 2024-04-24T23:28:20.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.808+0000 7f21e3de3740 1 -- start start 2024-04-24T23:28:20.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.808+0000 7f21dad9d640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f21c80a2b90 0x7f21c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.808+0000 7f21d3fff640 1 -- 172.21.15.33:0/973485114 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f21c40cf320 con 0x7f21dc101890 2024-04-24T23:28:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a09b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21e3de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557769fff6b0 con 0x55776a09b240 2024-04-24T23:28:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21e3de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557769fa7880 con 0x55776a09bc30 2024-04-24T23:28:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21e3de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557769ffb670 con 0x55776a0a0da0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21dad9d640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f21c80a2b90 0x7f21c80a5050 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc00b4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21db59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21db59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0a3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38580/0 (socket says 172.21.15.33:38580) 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21db59e640 1 -- 172.21.15.33:0/2750811718 learned_addr learned my addr 172.21.15.33:0/2750811718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a09b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.809+0000 7f21da59c640 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21db59e640 1 -- 172.21.15.33:0/2750811718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 msgr2=0x55776a0a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21db59e640 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21db59e640 1 -- 172.21.15.33:0/2750811718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 msgr2=0x55776a09b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21db59e640 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a09b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21db59e640 1 -- 172.21.15.33:0/2750811718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557769ddd7d0 con 0x55776a0a0da0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21db59e640 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0a3230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009ca0 tx=0x7f21d4009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b6c3f7c31e4da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21d17fa640 1 -- 172.21.15.33:0/2750811718 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40098c0 con 0x55776a0a0da0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21d17fa640 1 -- 172.21.15.33:0/2750811718 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d4036040 con 0x55776a0a0da0 2024-04-24T23:28:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.810+0000 7f21d17fa640 1 -- 172.21.15.33:0/2750811718 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4004290 con 0x55776a0a0da0 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 -- 172.21.15.33:0/2750811718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 msgr2=0x55776a0a3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0a3230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009ca0 tx=0x7f21d4009360 comp rx=0 tx=0).stop 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 -- 172.21.15.33:0/2750811718 shutdown_connections 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0a3230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 --2- 172.21.15.33:0/2750811718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a09b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 -- 172.21.15.33:0/2750811718 >> 172.21.15.33:0/2750811718 conn(0x55776a0968b0 msgr2=0x55776a096cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 -- 172.21.15.33:0/2750811718 shutdown_connections 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 -- 172.21.15.33:0/2750811718 wait complete. 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 -- start start 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.812+0000 7f21e3de3740 1 Processor -- start 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 0x55776a0a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09bc30 0x55776a0a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55776a0a7f20 con 0x55776a09bc30 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557769fbaf50 con 0x55776a09b240 2024-04-24T23:28:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55776a0a80a0 con 0x55776a0a0da0 2024-04-24T23:28:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 0x55776a0a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21da59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09bc30 0x55776a0a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 0x55776a0a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49024/0 (socket says 172.21.15.33:49024) 2024-04-24T23:28:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 -- 172.21.15.33:0/1955310547 learned_addr learned my addr 172.21.15.33:0/1955310547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21e3de3740 10 client.0 _ll_register_callbacks cb 0x557769ff6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 -- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 msgr2=0x55776a0b0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0b0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 -- 172.21.15.33:0/1955310547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09bc30 msgr2=0x55776a0a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09bc30 0x55776a0a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.813+0000 7f21dad9d640 1 -- 172.21.15.33:0/1955310547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557769d133f0 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21dad9d640 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 0x55776a0a97d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21cc011bb0 tx=0x7f21cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba659867de915c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21c27fc640 1 -- 172.21.15.33:0/1955310547 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc028440 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21c27fc640 1 -- 172.21.15.33:0/1955310547 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f21cc00e430 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d4036040 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21c27fc640 1 -- 172.21.15.33:0/1955310547 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f21cc00e630 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21e3de3740 10 client.28307 fetch_fsmap learned FSMap version 31 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.814+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557769cd30f0 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21c27fc640 1 -- 172.21.15.33:0/1955310547 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d4036040 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21c27fc640 1 -- 172.21.15.33:0/1955310547 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f21cc026070 con 0x55776a09b240 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21e3de3740 10 client.28307 fetch_fsmap finished waiting for FSMap version 31 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21e3de3740 10 client.28307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21e3de3740 20 client.28307 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21e3de3740 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a0b5bd0 0x55776a0b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21e3de3740 4 client.28307 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55776a00cbf0 con 0x55776a0b5bd0 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.815+0000 7f21da59c640 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a0b5bd0 0x55776a0b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.816+0000 7f21da59c640 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a0b5bd0 0x55776a0b8090 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a080297d7a49a6f4 server_cookie=4bd0c0e73d0f4c23 in_seq=0 out_seq=0 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.816+0000 7f21c27fc640 10 client.28307 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21c27fc640 1 -- 172.21.15.33:0/1955310547 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55776a00cbf0 con 0x55776a0b5bd0 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21c27fc640 10 client.28307 handle_command_reply: tid=0 2024-04-24T23:28:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 client.28307 shutdown 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 msgr2=0x55776a0a97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 0x55776a0a97d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21cc011bb0 tx=0x7f21cc004730 comp rx=0 tx=0).stop 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 shutdown_connections 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a0b5bd0 0x55776a0b8090 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a0a0da0 0x55776a0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09bc30 0x55776a0a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 --2- 172.21.15.33:0/1955310547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09b240 0x55776a0a97d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 >> 172.21.15.33:0/1955310547 conn(0x55776a0968b0 msgr2=0x55776a095af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21c27fc640 0 client.28307 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 shutdown_connections 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 1 -- 172.21.15.33:0/1955310547 wait complete. 2024-04-24T23:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.817+0000 7f21e3de3740 20 client.28307 trim_cache size 0 max 16384 2024-04-24T23:28:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.850+0000 7f21e3de3740 1 Processor -- start 2024-04-24T23:28:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.852+0000 7f21e3de3740 1 -- start start 2024-04-24T23:28:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a0a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09bc30 0x55776a0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 0x55776a099860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21e3de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1aa990 con 0x55776a09b240 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21e3de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1a8230 con 0x55776a0a0da0 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21e3de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1a6080 con 0x55776a09bc30 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.853+0000 7f21db59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 0x55776a099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.854+0000 7f21db59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 0x55776a099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49034/0 (socket says 172.21.15.33:49034) 2024-04-24T23:28:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.854+0000 7f21db59e640 1 -- 172.21.15.33:0/743181861 learned_addr learned my addr 172.21.15.33:0/743181861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.854+0000 7f21da59c640 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09bc30 0x55776a0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.857+0000 7f21dad9d640 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a0a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.859+0000 7f21db59e640 1 -- 172.21.15.33:0/743181861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09bc30 msgr2=0x55776a0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.859+0000 7f21db59e640 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09bc30 0x55776a0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.859+0000 7f21db59e640 1 -- 172.21.15.33:0/743181861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 msgr2=0x55776a0a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.859+0000 7f21db59e640 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a0a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.859+0000 7f21db59e640 1 -- 172.21.15.33:0/743181861 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776a088090 con 0x55776a0a0da0 2024-04-24T23:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.862+0000 7f21db59e640 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 0x55776a099860 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f21d402ade0 tx=0x7f21d4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8a64cb2f971065 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.865+0000 7f21d0ff9640 1 -- 172.21.15.33:0/743181861 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4031450 con 0x55776a0a0da0 2024-04-24T23:28:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.865+0000 7f21d0ff9640 1 -- 172.21.15.33:0/743181861 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d4036040 con 0x55776a0a0da0 2024-04-24T23:28:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.865+0000 7f21d0ff9640 1 -- 172.21.15.33:0/743181861 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4003de0 con 0x55776a0a0da0 2024-04-24T23:28:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.866+0000 7f21e3de3740 1 -- 172.21.15.33:0/743181861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 msgr2=0x55776a099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.866+0000 7f21e3de3740 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 0x55776a099860 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f21d402ade0 tx=0x7f21d4005bd0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.871+0000 7f21e3de3740 1 -- 172.21.15.33:0/743181861 shutdown_connections 2024-04-24T23:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.871+0000 7f21e3de3740 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a0a0da0 0x55776a099860 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.871+0000 7f21e3de3740 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09bc30 0x55776a0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.871+0000 7f21e3de3740 1 --2- 172.21.15.33:0/743181861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a09b240 0x55776a0a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.871+0000 7f21e3de3740 1 -- 172.21.15.33:0/743181861 >> 172.21.15.33:0/743181861 conn(0x55776a0968b0 msgr2=0x55776a0a93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.879+0000 7f21e3de3740 1 -- 172.21.15.33:0/743181861 shutdown_connections 2024-04-24T23:28:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.881+0000 7f21e3de3740 1 -- 172.21.15.33:0/743181861 wait complete. 2024-04-24T23:28:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 -- start start 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 Processor -- start 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09b240 0x55776a0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a0a0da0 0x55776a0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1a66f0 con 0x55776a0a0da0 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1a7b60 con 0x55776a09bc30 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1a6dc0 con 0x55776a09b240 2024-04-24T23:28:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.882+0000 7f21e3de3740 10 client.0 _ll_register_callbacks cb 0x7f21cc0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09b240 0x55776a0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09b240 0x55776a0a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38610/0 (socket says 172.21.15.33:38610) 2024-04-24T23:28:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21dad9d640 1 -- 172.21.15.33:0/4191748255 learned_addr learned my addr 172.21.15.33:0/4191748255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:20.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 65567 23.13 MB/sec execute 25 sec latency 82.474 ms 2024-04-24T23:28:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21db59e640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a0a0da0 0x55776a0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21da59c640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21db59e640 1 -- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09b240 msgr2=0x55776a0a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21db59e640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09b240 0x55776a0a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21db59e640 1 -- 172.21.15.33:0/4191748255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 msgr2=0x55776a0a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21db59e640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.884+0000 7f21db59e640 1 -- 172.21.15.33:0/4191748255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776a0a63e0 con 0x55776a0a0da0 2024-04-24T23:28:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.885+0000 7f21db59e640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a0a0da0 0x55776a0b0c70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f21d4006010 tx=0x7f21d4050c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb675a508f5284e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.885+0000 7f21c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.885+0000 7f21c27fc640 1 -- 172.21.15.33:0/4191748255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40318c0 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.885+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d4036040 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.885+0000 7f21c27fc640 1 -- 172.21.15.33:0/4191748255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f21d402f610 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.885+0000 7f21c27fc640 1 -- 172.21.15.33:0/4191748255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f21d4069050 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 10 client.18813 fetch_fsmap learned FSMap version 31 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55776a0a7650 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21c27fc640 1 -- 172.21.15.33:0/4191748255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d4036040 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21c27fc640 1 -- 172.21.15.33:0/4191748255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f21d4030a80 con 0x55776a0a0da0 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 10 client.18813 fetch_fsmap finished waiting for FSMap version 31 2024-04-24T23:28:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 10 client.18813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 20 client.18813 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a129360 0x55776a12b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 4 client.18813 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.886+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55776a006f40 con 0x55776a129360 2024-04-24T23:28:20.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.891+0000 7f21dad9d640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a129360 0x55776a12b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.899+0000 7f21dad9d640 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a129360 0x55776a12b750 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c51f3bb7f7d325 server_cookie=5686d047ff91800 in_seq=0 out_seq=0 2024-04-24T23:28:20.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.899+0000 7f21c27fc640 10 client.18813 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.940+0000 7f21c27fc640 1 -- 172.21.15.33:0/4191748255 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55776a006f40 con 0x55776a129360 2024-04-24T23:28:20.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.940+0000 7f21c27fc640 10 client.18813 handle_command_reply: tid=0 2024-04-24T23:28:20.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.941+0000 7f21e3de3740 1 client.18813 shutdown 2024-04-24T23:28:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.947+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a0a0da0 msgr2=0x55776a0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.947+0000 7f21e3de3740 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a0a0da0 0x55776a0b0c70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f21d4006010 tx=0x7f21d4050c00 comp rx=0 tx=0).stop 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 shutdown_connections 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21e3de3740 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55776a129360 0x55776a12b750 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21e3de3740 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55776a0a0da0 0x55776a0b0c70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21e3de3740 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55776a09bc30 0x55776a0a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21e3de3740 1 --2- 172.21.15.33:0/4191748255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55776a09b240 0x55776a0a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 >> 172.21.15.33:0/4191748255 conn(0x55776a0968b0 msgr2=0x55776a095260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.948+0000 7f21c27fc640 0 client.18813 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:20.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.951+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 shutdown_connections 2024-04-24T23:28:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.953+0000 7f21e3de3740 1 -- 172.21.15.33:0/4191748255 wait complete. 2024-04-24T23:28:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.953+0000 7f21e3de3740 20 client.18813 trim_cache size 0 max 16384 2024-04-24T23:28:20.956 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:28:20.956 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:28:20.956 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:28:20.956 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f21c80a2b90 msgr2=0x7f21c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f21c80a2b90 0x7f21c80a5050 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc00b4a0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 msgr2=0x7f21dc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 0x7f21dc197f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21c400e9f0 tx=0x7f21c400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 shutdown_connections 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f21c80a2b90 0x7f21c80a5050 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21dc102250 0x7f21dc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21dc101890 0x7f21dc197f80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 --2- 172.21.15.33:0/973485114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc1006a0 0x7f21dc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.958+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 >> 172.21.15.33:0/973485114 conn(0x7f21dc0fbc30 msgr2=0x7f21dc104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.961+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 shutdown_connections 2024-04-24T23:28:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:20.962+0000 7f21e10d4640 1 -- 172.21.15.33:0/973485114 wait complete. 2024-04-24T23:28:20.982 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:20.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.163+0000 7f1778e88640 1 -- 172.21.15.33:0/4047239295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774102230 msgr2=0x7f1774108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.163+0000 7f1778e88640 1 --2- 172.21.15.33:0/4047239295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774102230 0x7f1774108ac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1764009860 tx=0x7f176402a360 comp rx=0 tx=0).stop 2024-04-24T23:28:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.165+0000 7f1778e88640 1 -- 172.21.15.33:0/4047239295 shutdown_connections 2024-04-24T23:28:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.165+0000 7f1778e88640 1 --2- 172.21.15.33:0/4047239295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774102230 0x7f1774108ac0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.165+0000 7f1778e88640 1 --2- 172.21.15.33:0/4047239295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774101870 0x7f1774101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.165+0000 7f1778e88640 1 --2- 172.21.15.33:0/4047239295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774100680 0x7f1774100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.165+0000 7f1778e88640 1 -- 172.21.15.33:0/4047239295 >> 172.21.15.33:0/4047239295 conn(0x7f17740fbbf0 msgr2=0x7f17740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.167+0000 7f1778e88640 1 -- 172.21.15.33:0/4047239295 shutdown_connections 2024-04-24T23:28:21.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.167+0000 7f1778e88640 1 -- 172.21.15.33:0/4047239295 wait complete. 2024-04-24T23:28:21.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 Processor -- start 2024-04-24T23:28:21.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 -- start start 2024-04-24T23:28:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774100680 0x7f177419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774101870 0x7f177419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774102230 0x7f17741a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f177410a8b0 con 0x7f1774100680 2024-04-24T23:28:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f177410a730 con 0x7f1774101870 2024-04-24T23:28:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.168+0000 7f1778e88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f177410aa30 con 0x7f1774102230 2024-04-24T23:28:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.169+0000 7f1772d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774102230 0x7f17741a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.169+0000 7f1772d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774102230 0x7f17741a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38632/0 (socket says 172.21.15.33:38632) 2024-04-24T23:28:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.169+0000 7f1772d9d640 1 -- 172.21.15.33:0/3122508990 learned_addr learned my addr 172.21.15.33:0/3122508990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.169+0000 7f1771d9b640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774101870 0x7f177419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.169+0000 7f177259c640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774100680 0x7f177419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.170+0000 7f177259c640 1 -- 172.21.15.33:0/3122508990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774102230 msgr2=0x7f17741a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.170+0000 7f177259c640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774102230 0x7f17741a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.170+0000 7f177259c640 1 -- 172.21.15.33:0/3122508990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774101870 msgr2=0x7f177419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.170+0000 7f177259c640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774101870 0x7f177419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.170+0000 7f177259c640 1 -- 172.21.15.33:0/3122508990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17741a5c50 con 0x7f1774100680 2024-04-24T23:28:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.170+0000 7f177259c640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774100680 0x7f177419e030 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f176800c940 tx=0x7f176800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.171+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768004220 con 0x7f1774100680 2024-04-24T23:28:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.171+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17680043c0 con 0x7f1774100680 2024-04-24T23:28:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.172+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800f450 con 0x7f1774100680 2024-04-24T23:28:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.172+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17741a5f40 con 0x7f1774100680 2024-04-24T23:28:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.172+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17741a62f0 con 0x7f1774100680 2024-04-24T23:28:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.175+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1774068560 con 0x7f1774100680 2024-04-24T23:28:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.176+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f17680079b0 con 0x7f1774100680 2024-04-24T23:28:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.177+0000 7f17637fe640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f175c0a2ba0 0x7f175c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.177+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f17680cec50 con 0x7f1774100680 2024-04-24T23:28:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.177+0000 7f1771d9b640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f175c0a2ba0 0x7f175c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.178+0000 7f1771d9b640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f175c0a2ba0 0x7f175c0a5060 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f177419f540 tx=0x7f176c009740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.183+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f176801d070 con 0x7f1774100680 2024-04-24T23:28:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.426+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17741051b0 con 0x7f1774100680 2024-04-24T23:28:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.427+0000 7f17637fe640 1 -- 172.21.15.33:0/3122508990 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11861 (secure 0 0 0) 0x7f1768014030 con 0x7f1774100680 2024-04-24T23:28:21.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:28:21.428 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:06.161810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:28:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 31 2024-04-24T23:28:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f175c0a2ba0 msgr2=0x7f175c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f175c0a2ba0 0x7f175c0a5060 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f177419f540 tx=0x7f176c009740 comp rx=0 tx=0).stop 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774100680 msgr2=0x7f177419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774100680 0x7f177419e030 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f176800c940 tx=0x7f176800ce00 comp rx=0 tx=0).stop 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 shutdown_connections 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f175c0a2ba0 0x7f175c0a5060 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1774102230 0x7f17741a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1774101870 0x7f177419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 --2- 172.21.15.33:0/3122508990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1774100680 0x7f177419e030 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 >> 172.21.15.33:0/3122508990 conn(0x7f17740fbbf0 msgr2=0x7f177410d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 shutdown_connections 2024-04-24T23:28:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.429+0000 7f1778e88640 1 -- 172.21.15.33:0/3122508990 wait complete. 2024-04-24T23:28:21.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:28:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 -- 172.21.15.33:0/501572737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f80ffa40 msgr2=0x7f59f80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/501572737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f80ffa40 0x7f59f80ffe60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009930 tx=0x7f59e802a420 comp rx=0 tx=0).stop 2024-04-24T23:28:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 -- 172.21.15.33:0/501572737 shutdown_connections 2024-04-24T23:28:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/501572737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f81033d0 0x7f59f8109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/501572737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f8102a10 0x7f59f8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/501572737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f80ffa40 0x7f59f80ffe60 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 -- 172.21.15.33:0/501572737 >> 172.21.15.33:0/501572737 conn(0x7f59f80fb370 msgr2=0x7f59f80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 -- 172.21.15.33:0/501572737 shutdown_connections 2024-04-24T23:28:21.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.555+0000 7f59fd6f5640 1 -- 172.21.15.33:0/501572737 wait complete. 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 Processor -- start 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 -- start start 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f80ffa40 0x7f59f8102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 0x7f59f81009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f81033d0 0x7f59f8100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59f810d3f0 con 0x7f59f8102a10 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59f810d270 con 0x7f59f81033d0 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59fd6f5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59f810d570 con 0x7f59f80ffa40 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 0x7f59f81009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 0x7f59f81009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41550/0 (socket says 172.21.15.33:41550) 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59f6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f80ffa40 0x7f59f8102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59f67fc640 1 -- 172.21.15.33:0/4271509142 learned_addr learned my addr 172.21.15.33:0/4271509142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.556+0000 7f59f77fe640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f81033d0 0x7f59f8100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f6ffd640 1 -- 172.21.15.33:0/4271509142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f81033d0 msgr2=0x7f59f8100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f6ffd640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f81033d0 0x7f59f8100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f6ffd640 1 -- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 msgr2=0x7f59f81009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f6ffd640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 0x7f59f81009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f6ffd640 1 -- 172.21.15.33:0/4271509142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f8101790 con 0x7f59f80ffa40 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f77fe640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f81033d0 0x7f59f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f67fc640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 0x7f59f81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59f6ffd640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f80ffa40 0x7f59f8102380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59e802a940 tx=0x7f59e8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59dffff640 1 -- 172.21.15.33:0/4271509142 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e803f070 con 0x7f59f80ffa40 2024-04-24T23:28:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f81a1bb0 con 0x7f59f80ffa40 2024-04-24T23:28:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59dffff640 1 -- 172.21.15.33:0/4271509142 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e8003cd0 con 0x7f59f80ffa40 2024-04-24T23:28:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59dffff640 1 -- 172.21.15.33:0/4271509142 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8035070 con 0x7f59f80ffa40 2024-04-24T23:28:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.557+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f81a2120 con 0x7f59f80ffa40 2024-04-24T23:28:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f59dffff640 1 -- 172.21.15.33:0/4271509142 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f59e8043400 con 0x7f59f80ffa40 2024-04-24T23:28:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 Processor -- start 2024-04-24T23:28:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 -- start start 2024-04-24T23:28:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 0x557c3b1b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f59dffff640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59e00a2b90 0x7f59e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557c3b115670 con 0x557c3b1b1230 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557c3b0bd970 con 0x557c3b1b1c20 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.560+0000 7f5a00404740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557c3b111630 con 0x557c3b1b6d90 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 0x557c3b1b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59dffff640 1 -- 172.21.15.33:0/4271509142 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59e80ef460 con 0x7f59f80ffa40 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f6ffd640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59e00a2b90 0x7f59e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 0x557c3b1b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49072/0 (socket says 172.21.15.33:49072) 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 -- 172.21.15.33:0/3464295037 learned_addr learned my addr 172.21.15.33:0/3464295037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 -- 172.21.15.33:0/3464295037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 msgr2=0x557c3b1b9220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1b9220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 -- 172.21.15.33:0/3464295037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 msgr2=0x557c3b1b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 -- 172.21.15.33:0/3464295037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3b0e1400 con 0x557c3b1b1c20 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 0x557c3b1b6720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0115f0 tx=0x7f59ec011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3a69410abf6388 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59f6ffd640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59e00a2b90 0x7f59e00a5050 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59e80bb970 tx=0x7f59e803b460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59dd7fa640 1 -- 172.21.15.33:0/3464295037 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec005020 con 0x557c3b1b1c20 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59dd7fa640 1 -- 172.21.15.33:0/3464295037 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ec0047a0 con 0x557c3b1b1c20 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.561+0000 7f59dd7fa640 1 -- 172.21.15.33:0/3464295037 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec00fae0 con 0x557c3b1b1c20 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 -- 172.21.15.33:0/3464295037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 msgr2=0x557c3b1b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 0x557c3b1b6720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0115f0 tx=0x7f59ec011ab0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 -- 172.21.15.33:0/3464295037 shutdown_connections 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1c20 0x557c3b1b6720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 --2- 172.21.15.33:0/3464295037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 -- 172.21.15.33:0/3464295037 >> 172.21.15.33:0/3464295037 conn(0x557c3b1ac8a0 msgr2=0x557c3b1acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 -- 172.21.15.33:0/3464295037 shutdown_connections 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.562+0000 7f5a00404740 1 -- 172.21.15.33:0/3464295037 wait complete. 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 -- start start 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 Processor -- start 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 0x557c3b1c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557c3b1b9d40 con 0x557c3b1b6d90 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557c3b0d1050 con 0x557c3b1b1230 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557c3b1b9ec0 con 0x557c3b1b1c20 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f5a00404740 10 client.0 _ll_register_callbacks cb 0x557c3ae28be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 0x557c3b1c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 0x557c3b1c28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41582/0 (socket says 172.21.15.33:41582) 2024-04-24T23:28:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 -- 172.21.15.33:0/3411531751 learned_addr learned my addr 172.21.15.33:0/3411531751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 -- 172.21.15.33:0/3411531751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 msgr2=0x557c3b1bb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1bb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 -- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 msgr2=0x557c3b1bb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1bb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.563+0000 7f59f77fe640 1 -- 172.21.15.33:0/3411531751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3b0a2dc0 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 0x557c3b1c28a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f59f0010db0 tx=0x7f59f0010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed4b6a8e30ac0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59daff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59daff5640 1 -- 172.21.15.33:0/3411531751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0019040 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59daff5640 1 -- 172.21.15.33:0/3411531751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59f000ec40 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ec0047a0 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59daff5640 1 -- 172.21.15.33:0/3411531751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f59f0029050 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f5a00404740 10 client.18831 fetch_fsmap learned FSMap version 32 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c3ae293f0 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59daff5640 1 -- 172.21.15.33:0/3411531751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ec0047a0 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.564+0000 7f59daff5640 1 -- 172.21.15.33:0/3411531751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f59f001e330 con 0x557c3b1b6d90 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f5a00404740 10 client.18831 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f5a00404740 10 client.18831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:28:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f5a00404740 20 client.18831 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f5a00404740 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b1c7160 0x557c3b1c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f5a00404740 4 client.18831 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c3b122bb0 con 0x557c3b1c7160 2024-04-24T23:28:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b1c7160 0x557c3b1c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b1c7160 0x557c3b1c95d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d27be0bc3bd7307 server_cookie=a404416316b3450c in_seq=0 out_seq=0 2024-04-24T23:28:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.565+0000 7f59daff5640 10 client.18831 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f59daff5640 1 -- 172.21.15.33:0/3411531751 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c3b122bb0 con 0x557c3b1c7160 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f59daff5640 10 client.18831 handle_command_reply: tid=0 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 client.18831 shutdown 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 msgr2=0x557c3b1c28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 0x557c3b1c28a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f59f0010db0 tx=0x7f59f0010430 comp rx=0 tx=0).stop 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 shutdown_connections 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b1c7160 0x557c3b1c95d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b6d90 0x557c3b1c28a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f59daff5640 0 client.18831 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1bb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 --2- 172.21.15.33:0/3411531751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1bb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 >> 172.21.15.33:0/3411531751 conn(0x557c3b1ac8a0 msgr2=0x557c3b1abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 shutdown_connections 2024-04-24T23:28:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 1 -- 172.21.15.33:0/3411531751 wait complete. 2024-04-24T23:28:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.569+0000 7f5a00404740 20 client.18831 trim_cache size 0 max 16384 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 Processor -- start 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 -- start start 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b6d90 0x557c3b1af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81aef10 con 0x557c3b1b1230 2024-04-24T23:28:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81ac7e0 con 0x557c3b1b6d90 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f5a00404740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81aa630 con 0x557c3b1b1c20 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41590/0 (socket says 172.21.15.33:41590) 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 -- 172.21.15.33:0/1344593004 learned_addr learned my addr 172.21.15.33:0/1344593004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b6d90 0x557c3b1af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 -- 172.21.15.33:0/1344593004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 msgr2=0x557c3b1c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 -- 172.21.15.33:0/1344593004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b6d90 msgr2=0x557c3b1af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b6d90 0x557c3b1af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 -- 172.21.15.33:0/1344593004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3aef37d0 con 0x557c3b1b1230 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.583+0000 7f59f77fe640 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b6340 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f59f000f7e0 tx=0x7f59f00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d292024b956fbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f59dcff9640 1 -- 172.21.15.33:0/1344593004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f000ec10 con 0x557c3b1b1230 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f59dcff9640 1 -- 172.21.15.33:0/1344593004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f0035070 con 0x557c3b1b1230 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f59dcff9640 1 -- 172.21.15.33:0/1344593004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f001ebd0 con 0x557c3b1b1230 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 -- 172.21.15.33:0/1344593004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 msgr2=0x557c3b1b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b6340 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f59f000f7e0 tx=0x7f59f00021d0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 -- 172.21.15.33:0/1344593004 shutdown_connections 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b6d90 0x557c3b1af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b1c20 0x557c3b1c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 --2- 172.21.15.33:0/1344593004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1230 0x557c3b1b6340 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 -- 172.21.15.33:0/1344593004 >> 172.21.15.33:0/1344593004 conn(0x557c3b1ac8a0 msgr2=0x557c3b1abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 -- 172.21.15.33:0/1344593004 shutdown_connections 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.584+0000 7f5a00404740 1 -- 172.21.15.33:0/1344593004 wait complete. 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 -- start start 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 Processor -- start 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 0x557c3b1c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81aaca0 con 0x557c3b1b1c20 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81ac110 con 0x557c3b1b1230 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81ab370 con 0x557c3b1b6d90 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f5a00404740 10 client.0 _ll_register_callbacks cb 0x7f59f0010740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 0x557c3b1c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 0x557c3b1c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41606/0 (socket says 172.21.15.33:41606) 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 -- 172.21.15.33:0/3596021002 learned_addr learned my addr 172.21.15.33:0/3596021002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 -- 172.21.15.33:0/3596021002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 msgr2=0x557c3b1c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 -- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 msgr2=0x557c3b1c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1c0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 -- 172.21.15.33:0/3596021002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3b1c2350 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59f67fc640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 0x557c3b1c0da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f59ec01b040 tx=0x7f59ec0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db165ff14793c187 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.585+0000 7f59daff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f59daff5640 1 -- 172.21.15.33:0/3596021002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec032060 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f0035070 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f59daff5640 1 -- 172.21.15.33:0/3596021002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59ec016040 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f59daff5640 1 -- 172.21.15.33:0/3596021002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f59ec039050 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 10 client.18837 fetch_fsmap learned FSMap version 32 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c3b22ded0 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f59daff5640 1 -- 172.21.15.33:0/3596021002 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f0035070 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f59daff5640 1 -- 172.21.15.33:0/3596021002 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f59ec004040 con 0x557c3b1b1c20 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 10 client.18837 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T23:28:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 10 client.18837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 20 client.18837 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b22e720 0x557c3b230b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 4 client.18837 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.586+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c3b11cf00 con 0x557c3b22e720 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.587+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b22e720 0x557c3b230b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.587+0000 7f59daff5640 10 client.18837 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.587+0000 7f59f77fe640 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b22e720 0x557c3b230b40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43c9bf2d55269623 server_cookie=401c30ec75303093 in_seq=0 out_seq=0 2024-04-24T23:28:21.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f59daff5640 1 -- 172.21.15.33:0/3596021002 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c3b11cf00 con 0x557c3b22e720 2024-04-24T23:28:21.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f59daff5640 10 client.18837 handle_command_reply: tid=0 2024-04-24T23:28:21.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 client.18837 shutdown 2024-04-24T23:28:21.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 msgr2=0x557c3b1c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 0x557c3b1c0da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f59ec01b040 tx=0x7f59ec0092f0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 shutdown_connections 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x557c3b22e720 0x557c3b230b40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c3b1b6d90 0x557c3b1c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c3b1b1c20 0x557c3b1c0da0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f59daff5640 0 client.18837 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 --2- 172.21.15.33:0/3596021002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c3b1b1230 0x557c3b1c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.590+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 >> 172.21.15.33:0/3596021002 conn(0x557c3b1ac8a0 msgr2=0x557c3b1b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 shutdown_connections 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f5a00404740 1 -- 172.21.15.33:0/3596021002 wait complete. 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f5a00404740 20 client.18837 trim_cache size 0 max 16384 2024-04-24T23:28:21.596 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:28:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59e00a2b90 msgr2=0x7f59e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59e00a2b90 0x7f59e00a5050 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f59e80bb970 tx=0x7f59e803b460 comp rx=0 tx=0).stop 2024-04-24T23:28:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f80ffa40 msgr2=0x7f59f8102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f80ffa40 0x7f59f8102380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59e802a940 tx=0x7f59e8004270 comp rx=0 tx=0).stop 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 shutdown_connections 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f59e00a2b90 0x7f59e00a5050 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59f81033d0 0x7f59f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8102a10 0x7f59f81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 --2- 172.21.15.33:0/4271509142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f80ffa40 0x7f59f8102380 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 >> 172.21.15.33:0/4271509142 conn(0x7f59f80fb370 msgr2=0x7f59f8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 shutdown_connections 2024-04-24T23:28:21.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.591+0000 7f59fd6f5640 1 -- 172.21.15.33:0/4271509142 wait complete. 2024-04-24T23:28:21.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:28:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:21 smithi033 ceph-mon[29201]: pgmap v841: 129 pgs: 129 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-04-24T23:28:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:21 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3122508990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.716+0000 7fbcd4596640 1 -- 172.21.15.33:0/1768734026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc101870 msgr2=0x7fbccc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.716+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1768734026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc101870 0x7fbccc101cf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009900 tx=0x7fbcbc02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 -- 172.21.15.33:0/1768734026 shutdown_connections 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1768734026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc102230 0x7fbccc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1768734026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc101870 0x7fbccc101cf0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1768734026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc100680 0x7fbccc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 -- 172.21.15.33:0/1768734026 >> 172.21.15.33:0/1768734026 conn(0x7fbccc0fbbf0 msgr2=0x7fbccc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 -- 172.21.15.33:0/1768734026 shutdown_connections 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 -- 172.21.15.33:0/1768734026 wait complete. 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 Processor -- start 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.717+0000 7fbcd4596640 1 -- start start 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd4596640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 0x7fbccc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd4596640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc101870 0x7fbccc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd4596640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc102230 0x7fbccc19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd4596640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10d420 con 0x7fbccc100680 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd4596640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10d2a0 con 0x7fbccc101870 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd4596640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10d5a0 con 0x7fbccc102230 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 0x7fbccc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 0x7fbccc197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41626/0 (socket says 172.21.15.33:41626) 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 -- 172.21.15.33:0/1908005458 learned_addr learned my addr 172.21.15.33:0/1908005458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc101870 0x7fbccc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc102230 0x7fbccc19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 -- 172.21.15.33:0/1908005458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc102230 msgr2=0x7fbccc19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc102230 0x7fbccc19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 -- 172.21.15.33:0/1908005458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc101870 msgr2=0x7fbccc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc101870 0x7fbccc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 -- 172.21.15.33:0/1908005458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc19f700 con 0x7fbccc100680 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.718+0000 7fbcd2332640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 0x7fbccc197ae0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800e9c0 tx=0x7fbcc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.719+0000 7fbcc37fe640 1 -- 172.21.15.33:0/1908005458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800cd80 con 0x7fbccc100680 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.719+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc19f9f0 con 0x7fbccc100680 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.719+0000 7fbcc37fe640 1 -- 172.21.15.33:0/1908005458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc80035f0 con 0x7fbccc100680 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.719+0000 7fbcc37fe640 1 -- 172.21.15.33:0/1908005458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8010690 con 0x7fbccc100680 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.719+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc19ff10 con 0x7fbccc100680 2024-04-24T23:28:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.721+0000 7fbcc37fe640 1 -- 172.21.15.33:0/1908005458 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbcc80037a0 con 0x7fbccc100680 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.721+0000 7fbcd7376740 1 Processor -- start 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 -- start start 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd5f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd62220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e04bcbe670 con 0x55e04bd5fd90 2024-04-24T23:28:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e04bc66970 con 0x55e04bd5a230 2024-04-24T23:28:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd7376740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e04bcba630 con 0x55e04bd5ac20 2024-04-24T23:28:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcc37fe640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbcb80a2b90 0x7fbcb80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd2b33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd1b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd62220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd2b33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd5a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49128/0 (socket says 172.21.15.33:49128) 2024-04-24T23:28:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd2b33640 1 -- 172.21.15.33:0/4101854878 learned_addr learned my addr 172.21.15.33:0/4101854878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd2332640 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd5f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbcb80a2b90 0x7fbcb80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcc37fe640 1 -- 172.21.15.33:0/1908005458 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbcc80cf7b0 con 0x7fbccc100680 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.722+0000 7fbcd2b33640 1 -- 172.21.15.33:0/4101854878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 msgr2=0x55e04bd5f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd5f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd2b33640 1 -- 172.21.15.33:0/4101854878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 msgr2=0x55e04bd62220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd62220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd2b33640 1 -- 172.21.15.33:0/4101854878 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04bc8a400 con 0x55e04bd5a230 2024-04-24T23:28:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd62220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd5a650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4011e00 tx=0x7fbcc400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933cd3da30002d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbcb80a2b90 0x7fbcb80a5050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400fc90 tx=0x7fbcc4018000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcc0bfe640 1 -- 172.21.15.33:0/4101854878 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4022070 con 0x55e04bd5a230 2024-04-24T23:28:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcc0bfe640 1 -- 172.21.15.33:0/4101854878 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc400ec50 con 0x55e04bd5a230 2024-04-24T23:28:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.723+0000 7fbcc0bfe640 1 -- 172.21.15.33:0/4101854878 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4005810 con 0x55e04bd5a230 2024-04-24T23:28:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 -- 172.21.15.33:0/4101854878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 msgr2=0x55e04bd5a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd5a650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4011e00 tx=0x7fbcc400f790 comp rx=0 tx=0).stop 2024-04-24T23:28:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 -- 172.21.15.33:0/4101854878 shutdown_connections 2024-04-24T23:28:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd62220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd5f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 --2- 172.21.15.33:0/4101854878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd5a650 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 -- 172.21.15.33:0/4101854878 >> 172.21.15.33:0/4101854878 conn(0x55e04bd558a0 msgr2=0x55e04bd55ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 -- 172.21.15.33:0/4101854878 shutdown_connections 2024-04-24T23:28:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 -- 172.21.15.33:0/4101854878 wait complete. 2024-04-24T23:28:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 -- start start 2024-04-24T23:28:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.724+0000 7fbcd7376740 1 Processor -- start 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 0x55e04bd64340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e04bd62ce0 con 0x55e04bd5a230 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e04bc7a050 con 0x55e04bd5fd90 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e04bd62e60 con 0x55e04bd5ac20 2024-04-24T23:28:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd7376740 10 client.0 _ll_register_callbacks cb 0x55e04b9d1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd2332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 0x55e04bd64340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd2b33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd2332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 0x55e04bd64340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41644/0 (socket says 172.21.15.33:41644) 2024-04-24T23:28:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd2332640 1 -- 172.21.15.33:0/3577788121 learned_addr learned my addr 172.21.15.33:0/3577788121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 -- 172.21.15.33:0/3577788121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5fd90 msgr2=0x55e04bd6b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 -- 172.21.15.33:0/3577788121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 msgr2=0x55e04bd64340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 0x55e04bd64340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 -- 172.21.15.33:0/3577788121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e04bc4bdc0 con 0x55e04bd5ac20 2024-04-24T23:28:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd2332640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 0x55e04bd64340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd64880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031440 tx=0x7fbcbc032ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e3f57426810ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.725+0000 7fbcb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:28:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/3577788121 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc035040 con 0x55e04bd5ac20 2024-04-24T23:28:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc400ec50 con 0x55e04bd5ac20 2024-04-24T23:28:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/3577788121 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbcbc004040 con 0x55e04bd5ac20 2024-04-24T23:28:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/3577788121 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc04f050 con 0x55e04bd5ac20 2024-04-24T23:28:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 10 client.28282 fetch_fsmap learned FSMap version 32 2024-04-24T23:28:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e04b9d23f0 con 0x55e04bd5ac20 2024-04-24T23:28:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/3577788121 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc400ec50 con 0x55e04bd5ac20 2024-04-24T23:28:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/3577788121 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fbcbc004780 con 0x55e04bd5ac20 2024-04-24T23:28:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 10 client.28282 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T23:28:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 10 client.28282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:28:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 20 client.28282 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bd74500 0x55e04bd76980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 4 client.28282 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e04bccbbb0 con 0x55e04bd74500 2024-04-24T23:28:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.726+0000 7fbcd2332640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bd74500 0x55e04bd76980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.727+0000 7fbcd2332640 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bd74500 0x55e04bd76980 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=444d02a0269beee8 server_cookie=c0c2ed57f7cfb801 in_seq=0 out_seq=0 2024-04-24T23:28:21.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.727+0000 7fbcb5ffb640 10 client.28282 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:21.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.727+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/3577788121 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e04bccbbb0 con 0x55e04bd74500 2024-04-24T23:28:21.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.727+0000 7fbcb5ffb640 10 client.28282 handle_command_reply: tid=0 2024-04-24T23:28:21.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.727+0000 7fbcd7376740 1 client.28282 shutdown 2024-04-24T23:28:21.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 msgr2=0x55e04bd64880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd64880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031440 tx=0x7fbcbc032ed0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 shutdown_connections 2024-04-24T23:28:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bd74500 0x55e04bd76980 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5ac20 0x55e04bd64880 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3577788121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5a230 0x55e04bd64340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcb5ffb640 0 client.28282 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 >> 172.21.15.33:0/3577788121 conn(0x55e04bd558a0 msgr2=0x55e04bd54b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 shutdown_connections 2024-04-24T23:28:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 1 -- 172.21.15.33:0/3577788121 wait complete. 2024-04-24T23:28:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.728+0000 7fbcd7376740 20 client.28282 trim_cache size 0 max 16384 2024-04-24T23:28:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 Processor -- start 2024-04-24T23:28:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 -- start start 2024-04-24T23:28:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5a230 0x55e04bd58c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5ac20 0x55e04bd591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1aaac0 con 0x55e04bd5fd90 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a8390 con 0x55e04bd5ac20 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.742+0000 7fbcd7376740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a61e0 con 0x55e04bd5a230 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2332640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5a230 0x55e04bd58c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41654/0 (socket says 172.21.15.33:41654) 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 -- 172.21.15.33:0/3536801556 learned_addr learned my addr 172.21.15.33:0/3536801556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5ac20 0x55e04bd591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 -- 172.21.15.33:0/3536801556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5a230 msgr2=0x55e04bd58c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5a230 0x55e04bd58c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 -- 172.21.15.33:0/3536801556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5ac20 msgr2=0x55e04bd591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5ac20 0x55e04bd591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 -- 172.21.15.33:0/3536801556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04ba9c7d0 con 0x55e04bd5fd90 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b880 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4005f40 tx=0x7fbcc4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1637a8fce7f050 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcb7fff640 1 -- 172.21.15.33:0/3536801556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4022040 con 0x55e04bd5fd90 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.743+0000 7fbcb7fff640 1 -- 172.21.15.33:0/3536801556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc4003e50 con 0x55e04bd5fd90 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcb7fff640 1 -- 172.21.15.33:0/3536801556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4026400 con 0x55e04bd5fd90 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 -- 172.21.15.33:0/3536801556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 msgr2=0x55e04bd6b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b880 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4005f40 tx=0x7fbcc4009500 comp rx=0 tx=0).stop 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 -- 172.21.15.33:0/3536801556 shutdown_connections 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5fd90 0x55e04bd6b880 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5ac20 0x55e04bd591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 --2- 172.21.15.33:0/3536801556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5a230 0x55e04bd58c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 -- 172.21.15.33:0/3536801556 >> 172.21.15.33:0/3536801556 conn(0x55e04bd558a0 msgr2=0x55e04bd58580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 -- 172.21.15.33:0/3536801556 shutdown_connections 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.744+0000 7fbcd7376740 1 -- 172.21.15.33:0/3536801556 wait complete. 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 -- start start 2024-04-24T23:28:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 Processor -- start 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 0x55e04bd692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5fd90 0x55e04bd6a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a6850 con 0x55e04bd5ac20 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a7cc0 con 0x55e04bd5a230 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 0x55e04bd692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a6f20 con 0x55e04bd5fd90 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 0x55e04bd692c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41670/0 (socket says 172.21.15.33:41670) 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 -- 172.21.15.33:0/2515969721 learned_addr learned my addr 172.21.15.33:0/2515969721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd2332640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd7376740 10 client.0 _ll_register_callbacks cb 0x7fbcbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd2b33640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5fd90 0x55e04bd6a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 -- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5fd90 msgr2=0x55e04bd6a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5fd90 0x55e04bd6a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 -- 172.21.15.33:0/2515969721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 msgr2=0x55e04bd68d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.745+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd68d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd1b31640 1 -- 172.21.15.33:0/2515969721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e04bd6a8c0 con 0x55e04bd5ac20 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd1b31640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 0x55e04bd692c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031610 tx=0x7fbcbc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeed5f7aa8ae1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/2515969721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc058060 con 0x55e04bd5ac20 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/2515969721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbcbc031a00 con 0x55e04bd5ac20 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc4003e50 con 0x55e04bd5ac20 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/2515969721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc035070 con 0x55e04bd5ac20 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd7376740 10 client.18855 fetch_fsmap learned FSMap version 32 2024-04-24T23:28:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e04bd6bc40 con 0x55e04bd5ac20 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/2515969721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc4003e50 con 0x55e04bd5ac20 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/2515969721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fbcbc0052a0 con 0x55e04bd5ac20 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd7376740 10 client.18855 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.746+0000 7fbcd7376740 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 20 client.18855 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bddb280 0x55e04bddd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 4 client.18855 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e04bcc5f00 con 0x55e04bddb280 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd2332640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bddb280 0x55e04bddd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd2332640 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bddb280 0x55e04bddd6a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddceef8f826b21fd server_cookie=cb7ba66497e6e7c3 in_seq=0 out_seq=0 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcb5ffb640 10 client.18855 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcb5ffb640 1 -- 172.21.15.33:0/2515969721 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e04bcc5f00 con 0x55e04bddb280 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcb5ffb640 10 client.18855 handle_command_reply: tid=0 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 1 client.18855 shutdown 2024-04-24T23:28:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 msgr2=0x55e04bd692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.747+0000 7fbcd7376740 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 0x55e04bd692c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031610 tx=0x7fbcbc005d50 comp rx=0 tx=0).stop 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 shutdown_connections 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55e04bddb280 0x55e04bddd6a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e04bd5fd90 0x55e04bd6a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e04bd5ac20 0x55e04bd692c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcb5ffb640 0 client.18855 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 --2- 172.21.15.33:0/2515969721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e04bd5a230 0x55e04bd68d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 >> 172.21.15.33:0/2515969721 conn(0x55e04bd558a0 msgr2=0x55e04bd58e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 shutdown_connections 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 1 -- 172.21.15.33:0/2515969721 wait complete. 2024-04-24T23:28:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd7376740 20 client.18855 trim_cache size 0 max 16384 2024-04-24T23:28:21.763 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbcb80a2b90 msgr2=0x7fbcb80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbcb80a2b90 0x7fbcb80a5050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400fc90 tx=0x7fbcc4018000 comp rx=0 tx=0).stop 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 msgr2=0x7fbccc197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 0x7fbccc197ae0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800e9c0 tx=0x7fbcc800ee80 comp rx=0 tx=0).stop 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 shutdown_connections 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbcb80a2b90 0x7fbcb80a5050 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbccc102230 0x7fbccc19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbccc101870 0x7fbccc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 --2- 172.21.15.33:0/1908005458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc100680 0x7fbccc197ae0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 >> 172.21.15.33:0/1908005458 conn(0x7fbccc0fbbf0 msgr2=0x7fbccc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 shutdown_connections 2024-04-24T23:28:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.748+0000 7fbcd4596640 1 -- 172.21.15.33:0/1908005458 wait complete. 2024-04-24T23:28:21.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:28:21.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 65911 22.93 MB/sec execute 26 sec latency 266.811 ms 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 -- 172.21.15.33:0/4039978409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 msgr2=0x7fa8b8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 --2- 172.21.15.33:0/4039978409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 0x7fa8b8100aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009900 tx=0x7fa8a802a410 comp rx=0 tx=0).stop 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 -- 172.21.15.33:0/4039978409 shutdown_connections 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 --2- 172.21.15.33:0/4039978409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8102230 0x7fa8b8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 --2- 172.21.15.33:0/4039978409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b8101870 0x7fa8b8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 --2- 172.21.15.33:0/4039978409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 0x7fa8b8100aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 -- 172.21.15.33:0/4039978409 >> 172.21.15.33:0/4039978409 conn(0x7fa8b80fbbf0 msgr2=0x7fa8b80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 -- 172.21.15.33:0/4039978409 shutdown_connections 2024-04-24T23:28:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.897+0000 7fa8bde38640 1 -- 172.21.15.33:0/4039978409 wait complete. 2024-04-24T23:28:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.898+0000 7fa8bde38640 1 Processor -- start 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.898+0000 7fa8bde38640 1 -- start start 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.898+0000 7fa8bde38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 0x7fa8b8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8bde38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 0x7fa8b8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8bde38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b8102230 0x7fa8b819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8bde38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b810b1c0 con 0x7fa8b8101870 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8bde38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b810b040 con 0x7fa8b8100680 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8bde38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b810b340 con 0x7fa8b8102230 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 0x7fa8b8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 0x7fa8b8195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41690/0 (socket says 172.21.15.33:41690) 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 -- 172.21.15.33:0/1944767187 learned_addr learned my addr 172.21.15.33:0/1944767187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 0x7fa8b8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b8102230 0x7fa8b819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 -- 172.21.15.33:0/1944767187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b8102230 msgr2=0x7fa8b819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b8102230 0x7fa8b819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 -- 172.21.15.33:0/1944767187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 msgr2=0x7fa8b8195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 0x7fa8b8195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 -- 172.21.15.33:0/1944767187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b819d520 con 0x7fa8b8101870 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.899+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 0x7fa8b8195e40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac010ad0 tx=0x7fa8ac010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.900+0000 7fa8b4ff9640 1 -- 172.21.15.33:0/1944767187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac01a020 con 0x7fa8b8101870 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.900+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b819d810 con 0x7fa8b8101870 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.900+0000 7fa8b4ff9640 1 -- 172.21.15.33:0/1944767187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ac00eb70 con 0x7fa8b8101870 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.900+0000 7fa8b4ff9640 1 -- 172.21.15.33:0/1944767187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac015640 con 0x7fa8b8101870 2024-04-24T23:28:21.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.900+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b819dd60 con 0x7fa8b8101870 2024-04-24T23:28:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.903+0000 7fa8b4ff9640 1 -- 172.21.15.33:0/1944767187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa8ac01d020 con 0x7fa8b8101870 2024-04-24T23:28:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.903+0000 7fa8c0c24740 1 Processor -- start 2024-04-24T23:28:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.903+0000 7fa8c0c24740 1 -- start start 2024-04-24T23:28:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.903+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c6ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 0x564272c70b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564272bcb670 con 0x564272c6c030 2024-04-24T23:28:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564272b73970 con 0x564272c711a0 2024-04-24T23:28:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564272bc7630 con 0x564272c6b640 2024-04-24T23:28:21.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 0x564272c70b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 0x564272c70b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41698/0 (socket says 172.21.15.33:41698) 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 -- 172.21.15.33:0/544116983 learned_addr learned my addr 172.21.15.33:0/544116983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c6ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 -- 172.21.15.33:0/544116983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 msgr2=0x564272c6ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c6ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 -- 172.21.15.33:0/544116983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 msgr2=0x564272c73630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c73630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b77fe640 1 -- 172.21.15.33:0/544116983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564272b97400 con 0x564272c6c030 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.904+0000 7fa8b4ff9640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8a00a2b90 0x7fa8a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.905+0000 7fa8b4ff9640 1 -- 172.21.15.33:0/1944767187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8ac0d1880 con 0x7fa8b8101870 2024-04-24T23:28:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.905+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8a00a2b90 0x7fa8a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.905+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 0x564272c70b30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8031610 tx=0x7fa8a8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b493fb95ec9bc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.905+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.906+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8a00a2b90 0x7fa8a00a5050 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x564272c6c890 tx=0x7fa8b0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.906+0000 7fa89dffb640 1 -- 172.21.15.33:0/544116983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8040070 con 0x564272c6c030 2024-04-24T23:28:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.906+0000 7fa89dffb640 1 -- 172.21.15.33:0/544116983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8004030 con 0x564272c6c030 2024-04-24T23:28:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.906+0000 7fa89dffb640 1 -- 172.21.15.33:0/544116983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8035040 con 0x564272c6c030 2024-04-24T23:28:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.906+0000 7fa8c0c24740 1 -- 172.21.15.33:0/544116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 msgr2=0x564272c70b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.906+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 0x564272c70b30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8031610 tx=0x7fa8a8005de0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 -- 172.21.15.33:0/544116983 shutdown_connections 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6c030 0x564272c70b30 secure :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8031610 tx=0x7fa8a8005de0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/544116983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c6ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 -- 172.21.15.33:0/544116983 >> 172.21.15.33:0/544116983 conn(0x564272c628a0 msgr2=0x564272c62ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 -- 172.21.15.33:0/544116983 shutdown_connections 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 -- 172.21.15.33:0/544116983 wait complete. 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.907+0000 7fa8c0c24740 1 -- start start 2024-04-24T23:28:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 Processor -- start 2024-04-24T23:28:21.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c711a0 0x564272c75f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c7af30 0x564272c7d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564272c74140 con 0x564272c711a0 2024-04-24T23:28:21.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564272b87050 con 0x564272c7af30 2024-04-24T23:28:21.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564272c742c0 con 0x564272c6b640 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8c0c24740 10 client.0 _ll_register_callbacks cb 0x5642728debe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c711a0 0x564272c75f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c711a0 0x564272c75f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41714/0 (socket says 172.21.15.33:41714) 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8b6ffd640 1 -- 172.21.15.33:0/865893165 learned_addr learned my addr 172.21.15.33:0/865893165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.908+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c7af30 0x564272c7d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8b77fe640 1 -- 172.21.15.33:0/865893165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c7af30 msgr2=0x564272c7d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c7af30 0x564272c7d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8b77fe640 1 -- 172.21.15.33:0/865893165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c711a0 msgr2=0x564272c75f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c711a0 0x564272c75f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8b77fe640 1 -- 172.21.15.33:0/865893165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564272b58dc0 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c759c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x564272c6c4f0 tx=0x7fa8a80056c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e15edd06b3de59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa896ffd640 1 -- 172.21.15.33:0/865893165 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8040070 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.909+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a8004030 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.910+0000 7fa896ffd640 1 -- 172.21.15.33:0/865893165 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8a80043f0 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.910+0000 7fa896ffd640 1 -- 172.21.15.33:0/865893165 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8a8054050 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.910+0000 7fa8c0c24740 10 client.28300 fetch_fsmap learned FSMap version 32 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.910+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642728df3f0 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.910+0000 7fa896ffd640 1 -- 172.21.15.33:0/865893165 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8004030 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa896ffd640 1 -- 172.21.15.33:0/865893165 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fa8a8004780 con 0x564272c6b640 2024-04-24T23:28:21.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa8c0c24740 10 client.28300 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa8c0c24740 10 client.28300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa8c0c24740 20 client.28300 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272c81810 0x564272c83c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa8c0c24740 4 client.28300 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.911+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564272bd8bb0 con 0x564272c81810 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.912+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272c81810 0x564272c83c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.912+0000 7fa896ffd640 10 client.28300 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.912+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272c81810 0x564272c83c00 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c8901c41b8fa42 server_cookie=d16cd9933cd0c672 in_seq=0 out_seq=0 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa896ffd640 1 -- 172.21.15.33:0/865893165 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564272bd8bb0 con 0x564272c81810 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa896ffd640 10 client.28300 handle_command_reply: tid=0 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 client.28300 shutdown 2024-04-24T23:28:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 msgr2=0x564272c759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c759c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x564272c6c4f0 tx=0x7fa8a80056c0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 shutdown_connections 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272c81810 0x564272c83c00 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c7af30 0x564272c7d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c711a0 0x564272c75f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/865893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c759c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.913+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 >> 172.21.15.33:0/865893165 conn(0x564272c628a0 msgr2=0x564272c61b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.914+0000 7fa896ffd640 0 client.28300 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.914+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 shutdown_connections 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.914+0000 7fa8c0c24740 1 -- 172.21.15.33:0/865893165 wait complete. 2024-04-24T23:28:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.914+0000 7fa8c0c24740 20 client.28300 trim_cache size 0 max 16384 2024-04-24T23:28:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:21 smithi038 ceph-mon[32450]: pgmap v841: 129 pgs: 129 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-04-24T23:28:21.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:21 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3122508990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:21.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:21 smithi117 ceph-mon[32529]: pgmap v841: 129 pgs: 129 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-04-24T23:28:21.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:21 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3122508990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 Processor -- start 2024-04-24T23:28:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 -- start start 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 0x564272c6adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81aaae0 con 0x564272c7af30 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81a83b0 con 0x564272c711a0 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81a6200 con 0x564272c6b640 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 0x564272c6adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c882a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38774/0 (socket says 172.21.15.33:38774) 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 0x564272c6adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41722/0 (socket says 172.21.15.33:41722) 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 -- 172.21.15.33:0/4278078020 learned_addr learned my addr 172.21.15.33:0/4278078020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c887e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49194/0 (socket says 172.21.15.33:49194) 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 -- 172.21.15.33:0/4278078020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 msgr2=0x564272c887e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c887e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 -- 172.21.15.33:0/4278078020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 msgr2=0x564272c6adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.931+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 0x564272c6adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.932+0000 7fa8b77fe640 1 -- 172.21.15.33:0/4278078020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642729a97d0 con 0x564272c6b640 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.932+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 0x564272c6adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.932+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c882a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8a802a930 tx=0x7fa8a8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7e7a14530b3d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.932+0000 7fa89d7fa640 1 -- 172.21.15.33:0/4278078020 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a805d070 con 0x564272c6b640 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.932+0000 7fa89d7fa640 1 -- 172.21.15.33:0/4278078020 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8032d30 con 0x564272c6b640 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.932+0000 7fa89d7fa640 1 -- 172.21.15.33:0/4278078020 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8040040 con 0x564272c6b640 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 -- 172.21.15.33:0/4278078020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 msgr2=0x564272c882a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c882a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8a802a930 tx=0x7fa8a8005fd0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 -- 172.21.15.33:0/4278078020 shutdown_connections 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c7af30 0x564272c6adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c887e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/4278078020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c6b640 0x564272c882a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 -- 172.21.15.33:0/4278078020 >> 172.21.15.33:0/4278078020 conn(0x564272c628a0 msgr2=0x564272c61b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 -- 172.21.15.33:0/4278078020 shutdown_connections 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 -- 172.21.15.33:0/4278078020 wait complete. 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 -- start start 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 Processor -- start 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 0x564272c69880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.933+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c69dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8c0c24740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c7af30 0x564272c6a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81a6870 con 0x564272c6b640 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81a7ce0 con 0x564272c711a0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8c0c24740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81a6f40 con 0x564272c7af30 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8c0c24740 10 client.0 _ll_register_callbacks cb 0x7fa8a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 0x564272c69880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 0x564272c69880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41724/0 (socket says 172.21.15.33:41724) 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 -- 172.21.15.33:0/3458148618 learned_addr learned my addr 172.21.15.33:0/3458148618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c69dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b7fff640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c7af30 0x564272c6a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 -- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c7af30 msgr2=0x564272c6a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c7af30 0x564272c6a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 -- 172.21.15.33:0/3458148618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 msgr2=0x564272c69dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c69dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 -- 172.21.15.33:0/3458148618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564272c93b20 con 0x564272c6b640 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c69dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8b77fe640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 0x564272c69880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8004650 tx=0x7fa8a8031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2651bd8ee7a1a2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa896ffd640 1 -- 172.21.15.33:0/3458148618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a805d040 con 0x564272c6b640 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a8032d30 con 0x564272c6b640 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.934+0000 7fa896ffd640 1 -- 172.21.15.33:0/3458148618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8a8004cc0 con 0x564272c6b640 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa896ffd640 1 -- 172.21.15.33:0/3458148618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8a806c050 con 0x564272c6b640 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 10 client.18873 fetch_fsmap learned FSMap version 32 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564272c94dc0 con 0x564272c6b640 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa896ffd640 1 -- 172.21.15.33:0/3458148618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8032d30 con 0x564272c6b640 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa896ffd640 1 -- 172.21.15.33:0/3458148618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fa8a803c710 con 0x564272c6b640 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 10 client.18873 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 20 client.18873 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272cec280 0x564272cee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 4 client.18873 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564272bd2f00 con 0x564272cec280 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.935+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272cec280 0x564272cee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa896ffd640 10 client.18873 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8b6ffd640 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272cec280 0x564272cee6a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d214b5a1d2f1cecf server_cookie=81b88fe070fb875a in_seq=0 out_seq=0 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa896ffd640 1 -- 172.21.15.33:0/3458148618 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564272bd2f00 con 0x564272cec280 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa896ffd640 10 client.18873 handle_command_reply: tid=0 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 client.18873 shutdown 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 msgr2=0x564272c69880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 0x564272c69880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8004650 tx=0x7fa8a8031960 comp rx=0 tx=0).stop 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 shutdown_connections 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564272cec280 0x564272cee6a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564272c7af30 0x564272c6a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564272c711a0 0x564272c69dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 --2- 172.21.15.33:0/3458148618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564272c6b640 0x564272c69880 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa896ffd640 0 client.18873 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 >> 172.21.15.33:0/3458148618 conn(0x564272c628a0 msgr2=0x564272c6cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 shutdown_connections 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 1 -- 172.21.15.33:0/3458148618 wait complete. 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.936+0000 7fa8c0c24740 20 client.18873 trim_cache size 0 max 16384 2024-04-24T23:28:21.947 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8a00a2b90 msgr2=0x7fa8a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8a00a2b90 0x7fa8a00a5050 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x564272c6c890 tx=0x7fa8b0009500 comp rx=0 tx=0).stop 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 msgr2=0x7fa8b8195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 0x7fa8b8195e40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac010ad0 tx=0x7fa8ac010ed0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 shutdown_connections 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa8a00a2b90 0x7fa8a00a5050 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8b8102230 0x7fa8b819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8b8101870 0x7fa8b8195e40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 --2- 172.21.15.33:0/1944767187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8b8100680 0x7fa8b8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 >> 172.21.15.33:0/1944767187 conn(0x7fa8b80fbbf0 msgr2=0x7fa8b8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 shutdown_connections 2024-04-24T23:28:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:21.937+0000 7fa8bde38640 1 -- 172.21.15.33:0/1944767187 wait complete. 2024-04-24T23:28:22.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 66151 22.80 MB/sec execute 27 sec latency 302.589 ms 2024-04-24T23:28:22.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:22 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:22.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:22 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:22.950 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:28:22.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:22.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:28:22.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:22 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 -- 172.21.15.33:0/3324818597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c1039a0 msgr2=0x7f204c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 --2- 172.21.15.33:0/3324818597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c1039a0 0x7f204c103dc0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f203c009900 tx=0x7f203c02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 -- 172.21.15.33:0/3324818597 shutdown_connections 2024-04-24T23:28:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 --2- 172.21.15.33:0/3324818597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c105550 0x7f204c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 --2- 172.21.15.33:0/3324818597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 0x7f204c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 --2- 172.21.15.33:0/3324818597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c1039a0 0x7f204c103dc0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 -- 172.21.15.33:0/3324818597 >> 172.21.15.33:0/3324818597 conn(0x7f204c0fef30 msgr2=0x7f204c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 -- 172.21.15.33:0/3324818597 shutdown_connections 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 -- 172.21.15.33:0/3324818597 wait complete. 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.065+0000 7f2051c31640 1 Processor -- start 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 -- start start 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c1039a0 0x7f204c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 0x7f204c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 0x7f204c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10f5f0 con 0x7f204c105550 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10f470 con 0x7f204c104b90 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f2051c31640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10f770 con 0x7f204c1039a0 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 0x7f204c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c1039a0 0x7f204c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 0x7f204c1a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41738/0 (socket says 172.21.15.33:41738) 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204bfff640 1 -- 172.21.15.33:0/3428207499 learned_addr learned my addr 172.21.15.33:0/3428207499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204affd640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 0x7f204c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204b7fe640 1 -- 172.21.15.33:0/3428207499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 msgr2=0x7f204c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204b7fe640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 0x7f204c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204b7fe640 1 -- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 msgr2=0x7f204c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204b7fe640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 0x7f204c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204b7fe640 1 -- 172.21.15.33:0/3428207499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c1a1900 con 0x7f204c1039a0 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204bfff640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 0x7f204c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.066+0000 7f204affd640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 0x7f204c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.067+0000 7f204b7fe640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c1039a0 0x7f204c199cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f203c009230 tx=0x7f203c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.067+0000 7f2048ff9640 1 -- 172.21.15.33:0/3428207499 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c03f070 con 0x7f204c1039a0 2024-04-24T23:28:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.067+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1a1b90 con 0x7f204c1039a0 2024-04-24T23:28:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.067+0000 7f2048ff9640 1 -- 172.21.15.33:0/3428207499 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f203c003cd0 con 0x7f204c1039a0 2024-04-24T23:28:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.067+0000 7f2048ff9640 1 -- 172.21.15.33:0/3428207499 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c035070 con 0x7f204c1039a0 2024-04-24T23:28:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.067+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1a20a0 con 0x7f204c1039a0 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 Processor -- start 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 -- start start 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c7780 0x5628526cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526cc8f0 0x5628526ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56285262a2c0 con 0x5628526c7780 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628525d2910 con 0x5628526cc8f0 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2054a1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562852626280 con 0x5628526c6d90 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f2048ff9640 1 -- 172.21.15.33:0/3428207499 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f203c03b460 con 0x7f204c1039a0 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f204bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f204affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526cc8f0 0x5628526ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f204bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38820/0 (socket says 172.21.15.33:38820) 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f204affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526cc8f0 0x5628526ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49228/0 (socket says 172.21.15.33:49228) 2024-04-24T23:28:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f204bfff640 1 -- 172.21.15.33:0/4274948144 learned_addr learned my addr 172.21.15.33:0/4274948144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.070+0000 7f204b7fe640 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c7780 0x5628526cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204bfff640 1 -- 172.21.15.33:0/4274948144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526cc8f0 msgr2=0x5628526ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204bfff640 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526cc8f0 0x5628526ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204bfff640 1 -- 172.21.15.33:0/4274948144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c7780 msgr2=0x5628526cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204bfff640 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c7780 0x5628526cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204bfff640 1 -- 172.21.15.33:0/4274948144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628525e3fb0 con 0x5628526c6d90 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204bfff640 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526c71b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20440118b0 tx=0x7f2044011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36ee6c91e2b5cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f2048ff9640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20340a2b90 0x7f20340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f2031ffb640 1 -- 172.21.15.33:0/4274948144 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204401e070 con 0x5628526c6d90 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f2031ffb640 1 -- 172.21.15.33:0/4274948144 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2044011310 con 0x5628526c6d90 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f2031ffb640 1 -- 172.21.15.33:0/4274948144 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044010430 con 0x5628526c6d90 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f204b7fe640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20340a2b90 0x7f20340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.071+0000 7f2048ff9640 1 -- 172.21.15.33:0/3428207499 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f203c0ef850 con 0x7f204c1039a0 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 -- 172.21.15.33:0/4274948144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 msgr2=0x5628526c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526c71b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20440118b0 tx=0x7f2044011d70 comp rx=0 tx=0).stop 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f204b7fe640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20340a2b90 0x7f20340a5050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f203c0ed330 tx=0x7f203c02a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 -- 172.21.15.33:0/4274948144 shutdown_connections 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526cc8f0 0x5628526ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c7780 0x5628526cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 --2- 172.21.15.33:0/4274948144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526c71b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 -- 172.21.15.33:0/4274948144 >> 172.21.15.33:0/4274948144 conn(0x56285268e950 msgr2=0x5628526c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 -- 172.21.15.33:0/4274948144 shutdown_connections 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 -- 172.21.15.33:0/4274948144 wait complete. 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.072+0000 7f2054a1b740 1 -- start start 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 Processor -- start 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 0x5628526d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526cc8f0 0x5628526d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5628526cf8a0 con 0x5628526c6d90 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628525e5e90 con 0x5628526c7780 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5628526cfa20 con 0x5628526cc8f0 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f2054a1b740 10 client.0 _ll_register_callbacks cb 0x562852621970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 0x5628526d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 0x5628526d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41748/0 (socket says 172.21.15.33:41748) 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 -- 172.21.15.33:0/1929987718 learned_addr learned my addr 172.21.15.33:0/1929987718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526cc8f0 0x5628526d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 -- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526cc8f0 msgr2=0x5628526d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526cc8f0 0x5628526d8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 -- 172.21.15.33:0/1929987718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 msgr2=0x5628526d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 -- 172.21.15.33:0/1929987718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628524067e0 con 0x5628526c6d90 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204affd640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f204bfff640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 0x5628526d11c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2044000f80 tx=0x7f2044004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc6d304af7811cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.073+0000 7f202affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f202affd640 1 -- 172.21.15.33:0/1929987718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204402e060 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f202affd640 1 -- 172.21.15.33:0/1929987718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f204400fd70 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2044011310 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f202affd640 1 -- 172.21.15.33:0/1929987718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f204401e070 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 10 client.18885 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562852466f70 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f202affd640 1 -- 172.21.15.33:0/1929987718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2044011310 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f202affd640 1 -- 172.21.15.33:0/1929987718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f20440104c0 con 0x5628526c6d90 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 10 client.18885 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 10 client.18885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 20 client.18885 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628526dca50 0x5628526dee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 4 client.18885 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628525fe5a0 con 0x5628526dca50 2024-04-24T23:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.074+0000 7f204affd640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628526dca50 0x5628526dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.075+0000 7f202affd640 10 client.18885 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.075+0000 7f204affd640 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628526dca50 0x5628526dee40 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20e566ace607fc3 server_cookie=3c3e2cc53ae387b in_seq=0 out_seq=0 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f202affd640 1 -- 172.21.15.33:0/1929987718 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628525fe5a0 con 0x5628526dca50 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f202affd640 10 client.18885 handle_command_reply: tid=0 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 client.18885 shutdown 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 msgr2=0x5628526d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 0x5628526d11c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2044000f80 tx=0x7f2044004c70 comp rx=0 tx=0).stop 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 shutdown_connections 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5628526dca50 0x5628526dee40 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526cc8f0 0x5628526d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1929987718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526c6d90 0x5628526d11c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f202affd640 0 client.18885 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.076+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 >> 172.21.15.33:0/1929987718 conn(0x56285268e950 msgr2=0x5628526c1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.077+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 shutdown_connections 2024-04-24T23:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.077+0000 7f2054a1b740 1 -- 172.21.15.33:0/1929987718 wait complete. 2024-04-24T23:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.077+0000 7f2054a1b740 20 client.18885 trim_cache size 0 max 16384 2024-04-24T23:28:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 Processor -- start 2024-04-24T23:28:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 -- start start 2024-04-24T23:28:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1aedf0 con 0x5628526cc8f0 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1ac6c0 con 0x5628526c7780 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.095+0000 7f2054a1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1aa510 con 0x5628526c6d90 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38836/0 (socket says 172.21.15.33:38836) 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49254/0 (socket says 172.21.15.33:49254) 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204bfff640 1 -- 172.21.15.33:0/924715748 learned_addr learned my addr 172.21.15.33:0/924715748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 -- 172.21.15.33:0/924715748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 msgr2=0x5628526d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 -- 172.21.15.33:0/924715748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 msgr2=0x5628526c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 -- 172.21.15.33:0/924715748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628522fe170 con 0x5628526c7780 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f204affd640 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d8700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5628526df690 tx=0x7f2040005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6264b89ffb98eb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f20317fa640 1 -- 172.21.15.33:0/924715748 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204000fc20 con 0x5628526c7780 2024-04-24T23:28:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f20317fa640 1 -- 172.21.15.33:0/924715748 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f204000fdc0 con 0x5628526c7780 2024-04-24T23:28:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.096+0000 7f20317fa640 1 -- 172.21.15.33:0/924715748 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040004d50 con 0x5628526c7780 2024-04-24T23:28:23.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 -- 172.21.15.33:0/924715748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 msgr2=0x5628526d8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d8700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5628526df690 tx=0x7f2040005f90 comp rx=0 tx=0).stop 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 -- 172.21.15.33:0/924715748 shutdown_connections 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d8700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 --2- 172.21.15.33:0/924715748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 -- 172.21.15.33:0/924715748 >> 172.21.15.33:0/924715748 conn(0x56285268e950 msgr2=0x5628526c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 -- 172.21.15.33:0/924715748 shutdown_connections 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 -- 172.21.15.33:0/924715748 wait complete. 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.097+0000 7f2054a1b740 1 -- start start 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 Processor -- start 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526d7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1aab80 con 0x5628526cc8f0 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1abff0 con 0x5628526c7780 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1ab250 con 0x5628526c6d90 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f2054a1b740 10 client.0 _ll_register_callbacks cb 0x7f20440197d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526d7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f204affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f204bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526d7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41774/0 (socket says 172.21.15.33:41774) 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f204bfff640 1 -- 172.21.15.33:0/1977232026 learned_addr learned my addr 172.21.15.33:0/1977232026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.098+0000 7f204bfff640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204affd640 1 -- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 msgr2=0x5628526d6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204affd640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204affd640 1 -- 172.21.15.33:0/1977232026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 msgr2=0x5628526d7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204affd640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526d7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204affd640 1 -- 172.21.15.33:0/1977232026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628526d7e80 con 0x5628526c7780 2024-04-24T23:28:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204bfff640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526d7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f204affd640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d6880 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5628526df690 tx=0x7f2040012760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9523a5fe5bf8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f202affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f202affd640 1 -- 172.21.15.33:0/1977232026 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204000fc20 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f202affd640 1 -- 172.21.15.33:0/1977232026 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2040022170 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f204000fdc0 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f202affd640 1 -- 172.21.15.33:0/1977232026 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2040039050 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f2054a1b740 10 client.28391 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.099+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562852743540 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f202affd640 1 -- 172.21.15.33:0/1977232026 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f204000fdc0 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f202affd640 1 -- 172.21.15.33:0/1977232026 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f204002c870 con 0x5628526c7780 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f2054a1b740 10 client.28391 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f2054a1b740 10 client.28391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f2054a1b740 20 client.28391 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562852743ea0 0x5628527462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f2054a1b740 4 client.28391 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:28:23.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562852637800 con 0x562852743ea0 2024-04-24T23:28:23.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f204bfff640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562852743ea0 0x5628527462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f202affd640 10 client.28391 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.101+0000 7f204bfff640 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562852743ea0 0x5628527462c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd8694a0a7114ab server_cookie=20747694fa58e903 in_seq=0 out_seq=0 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f202affd640 1 -- 172.21.15.33:0/1977232026 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562852637800 con 0x562852743ea0 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f202affd640 10 client.28391 handle_command_reply: tid=0 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 client.28391 shutdown 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 msgr2=0x5628526d6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d6880 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5628526df690 tx=0x7f2040012760 comp rx=0 tx=0).stop 2024-04-24T23:28:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 shutdown_connections 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562852743ea0 0x5628527462c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628526cc8f0 0x5628526d7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628526c7780 0x5628526d6880 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f202affd640 0 client.28391 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 --2- 172.21.15.33:0/1977232026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628526c6d90 0x5628526d6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 >> 172.21.15.33:0/1977232026 conn(0x56285268e950 msgr2=0x5628526d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 shutdown_connections 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 1 -- 172.21.15.33:0/1977232026 wait complete. 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2054a1b740 20 client.28391 trim_cache size 0 max 16384 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:28:23.107 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:28:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "afabc35e-3bde-43e6-a872-d75955ee2fb0", 2024-04-24T23:28:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:28:23.108 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20340a2b90 msgr2=0x7f20340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2051c31640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20340a2b90 0x7f20340a5050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f203c0ed330 tx=0x7f203c02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c1039a0 msgr2=0x7f204c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.103+0000 7f2051c31640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c1039a0 0x7f204c199cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f203c009230 tx=0x7f203c004270 comp rx=0 tx=0).stop 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 shutdown_connections 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20340a2b90 0x7f20340a5050 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204c105550 0x7f204c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f204c104b90 0x7f204c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 --2- 172.21.15.33:0/3428207499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c1039a0 0x7f204c199cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 >> 172.21.15.33:0/3428207499 conn(0x7f204c0fef30 msgr2=0x7f204c107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 shutdown_connections 2024-04-24T23:28:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.104+0000 7f2051c31640 1 -- 172.21.15.33:0/3428207499 wait complete. 2024-04-24T23:28:23.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:28:23.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 -- 172.21.15.33:0/2538340868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690101870 msgr2=0x7f9690101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 --2- 172.21.15.33:0/2538340868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690101870 0x7f9690101cf0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9684009900 tx=0x7f968402a410 comp rx=0 tx=0).stop 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 -- 172.21.15.33:0/2538340868 shutdown_connections 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 --2- 172.21.15.33:0/2538340868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690102230 0x7f9690108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 --2- 172.21.15.33:0/2538340868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690101870 0x7f9690101cf0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 --2- 172.21.15.33:0/2538340868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100680 0x7f9690100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 -- 172.21.15.33:0/2538340868 >> 172.21.15.33:0/2538340868 conn(0x7f96900fbbf0 msgr2=0x7f96900fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 -- 172.21.15.33:0/2538340868 shutdown_connections 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.269+0000 7f9698415640 1 -- 172.21.15.33:0/2538340868 wait complete. 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 Processor -- start 2024-04-24T23:28:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 -- start start 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 0x7f9690069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690101870 0x7f969019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690102230 0x7f969019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f969010c300 con 0x7f9690100680 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f969010c180 con 0x7f9690101870 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.270+0000 7f9698415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f969010c480 con 0x7f9690102230 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 0x7f9690069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 0x7f9690069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41786/0 (socket says 172.21.15.33:41786) 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 -- 172.21.15.33:0/1873310929 learned_addr learned my addr 172.21.15.33:0/1873310929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96959b0640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690101870 0x7f969019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96969b2640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690102230 0x7f969019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 -- 172.21.15.33:0/1873310929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690102230 msgr2=0x7f969019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690102230 0x7f969019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 -- 172.21.15.33:0/1873310929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690101870 msgr2=0x7f969019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690101870 0x7f969019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 -- 172.21.15.33:0/1873310929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969019f760 con 0x7f9690100680 2024-04-24T23:28:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f96961b1640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 0x7f9690069440 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f968800e9c0 tx=0x7f968800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f968f7fe640 1 -- 172.21.15.33:0/1873310929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968800cd80 con 0x7f9690100680 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f968f7fe640 1 -- 172.21.15.33:0/1873310929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96880035f0 con 0x7f9690100680 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969019f9f0 con 0x7f9690100680 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f968f7fe640 1 -- 172.21.15.33:0/1873310929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688010690 con 0x7f9690100680 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.271+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969019ff10 con 0x7f9690100680 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f968f7fe640 1 -- 172.21.15.33:0/1873310929 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f96880037a0 con 0x7f9690100680 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f969b204740 1 Processor -- start 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f969b204740 1 -- start start 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f969b204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2820ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f969b204740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820d040 0x55bd28211b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f969b204740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282121b0 0x55bd28214640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.274+0000 7f969b204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd2816c6b0 con 0x55bd2820c650 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f969b204740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bd28114880 con 0x55bd282121b0 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f969b204740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bd28168670 con 0x55bd2820d040 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96961b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820d040 0x55bd28211b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96969b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2820ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282121b0 0x55bd28214640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96961b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820d040 0x55bd28211b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38880/0 (socket says 172.21.15.33:38880) 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96961b1640 1 -- 172.21.15.33:0/3560034505 learned_addr learned my addr 172.21.15.33:0/3560034505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f968f7fe640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96780a2b90 0x7f96780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 -- 172.21.15.33:0/3560034505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820d040 msgr2=0x55bd28211b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96969b2640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96780a2b90 0x7f96780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820d040 0x55bd28211b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 -- 172.21.15.33:0/3560034505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 msgr2=0x55bd2820ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2820ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 -- 172.21.15.33:0/3560034505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd27f4a7d0 con 0x55bd282121b0 2024-04-24T23:28:23.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f968f7fe640 1 -- 172.21.15.33:0/1873310929 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96880cf7b0 con 0x7f9690100680 2024-04-24T23:28:23.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.275+0000 7f96959b0640 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282121b0 0x55bd28214640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9684031440 tx=0x7f9684005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cab3f0c6ac3fe2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f96969b2640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96780a2b90 0x7f96780a5050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55bd2820d8a0 tx=0x7f9680004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f968cbfe640 1 -- 172.21.15.33:0/3560034505 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684040070 con 0x55bd282121b0 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f968cbfe640 1 -- 172.21.15.33:0/3560034505 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684004030 con 0x55bd282121b0 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f968cbfe640 1 -- 172.21.15.33:0/3560034505 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035040 con 0x55bd282121b0 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 -- 172.21.15.33:0/3560034505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282121b0 msgr2=0x55bd28214640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282121b0 0x55bd28214640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9684031440 tx=0x7f9684005e00 comp rx=0 tx=0).stop 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 -- 172.21.15.33:0/3560034505 shutdown_connections 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282121b0 0x55bd28214640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820d040 0x55bd28211b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 --2- 172.21.15.33:0/3560034505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2820ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.276+0000 7f969b204740 1 -- 172.21.15.33:0/3560034505 >> 172.21.15.33:0/3560034505 conn(0x55bd282038b0 msgr2=0x55bd28203cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 -- 172.21.15.33:0/3560034505 shutdown_connections 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 -- 172.21.15.33:0/3560034505 wait complete. 2024-04-24T23:28:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 -- start start 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 Processor -- start 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd28216a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd28216fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 0x55bd2821df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd28215100 con 0x55bd282121b0 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bd28127f50 con 0x55bd2820d040 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bd28215280 con 0x55bd2820c650 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f969b204740 10 client.0 _ll_register_callbacks cb 0x55bd28163000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f96969b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 0x55bd2821df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f96969b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 0x55bd2821df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41804/0 (socket says 172.21.15.33:41804) 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f96959b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd28216fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.277+0000 7f96969b2640 1 -- 172.21.15.33:0/4003558242 learned_addr learned my addr 172.21.15.33:0/4003558242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96961b1640 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd28216a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96969b2640 1 -- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 msgr2=0x55bd28216a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96969b2640 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd28216a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96969b2640 1 -- 172.21.15.33:0/4003558242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 msgr2=0x55bd28216fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96969b2640 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd28216fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96969b2640 1 -- 172.21.15.33:0/4003558242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd27e803f0 con 0x55bd282121b0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f96969b2640 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 0x55bd2821df90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9680003dc0 tx=0x7f9680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff9e4b49e472af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f9675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f9675ffb640 1 -- 172.21.15.33:0/4003558242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680025030 con 0x55bd282121b0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f9675ffb640 1 -- 172.21.15.33:0/4003558242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9680004550 con 0x55bd282121b0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9684004030 con 0x55bd282121b0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f9675ffb640 1 -- 172.21.15.33:0/4003558242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9680031050 con 0x55bd282121b0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f969b204740 10 client.18903 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bd27e400f0 con 0x55bd282121b0 2024-04-24T23:28:23.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.278+0000 7f9675ffb640 1 -- 172.21.15.33:0/4003558242 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684004030 con 0x55bd282121b0 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f9675ffb640 1 -- 172.21.15.33:0/4003558242 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f968001f040 con 0x55bd282121b0 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f969b204740 10 client.18903 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f969b204740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f969b204740 20 client.18903 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f969b204740 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd28222850 0x55bd28224cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f969b204740 4 client.18903 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd28179bf0 con 0x55bd28222850 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f96961b1640 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd28222850 0x55bd28224cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f96961b1640 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd28222850 0x55bd28224cb0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be9c0986f217cee server_cookie=efc961f75d5bc024 in_seq=0 out_seq=0 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.279+0000 7f9675ffb640 10 client.18903 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f9675ffb640 1 -- 172.21.15.33:0/4003558242 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd28179bf0 con 0x55bd28222850 2024-04-24T23:28:23.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f9675ffb640 10 client.18903 handle_command_reply: tid=0 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 client.18903 shutdown 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 msgr2=0x55bd2821df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 0x55bd2821df90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9680003dc0 tx=0x7f9680004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 shutdown_connections 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd28222850 0x55bd28224cb0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd282121b0 0x55bd2821df90 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f9675ffb640 0 client.18903 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd28216fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 --2- 172.21.15.33:0/4003558242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd28216a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 >> 172.21.15.33:0/4003558242 conn(0x55bd282038b0 msgr2=0x55bd28202af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 shutdown_connections 2024-04-24T23:28:23.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 1 -- 172.21.15.33:0/4003558242 wait complete. 2024-04-24T23:28:23.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.280+0000 7f969b204740 20 client.18903 trim_cache size 0 max 16384 2024-04-24T23:28:23.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 Processor -- start 2024-04-24T23:28:23.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 -- start start 2024-04-24T23:28:23.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd282038b0 0x55bd28202970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2821df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd282295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96901aaa60 con 0x55bd2820c650 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96901a8330 con 0x55bd2820d040 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.300+0000 7f969b204740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96901a6180 con 0x55bd282038b0 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2821df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96969b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd282295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2821df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41806/0 (socket says 172.21.15.33:41806) 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 -- 172.21.15.33:0/1741091971 learned_addr learned my addr 172.21.15.33:0/1741091971 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96961b1640 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd282038b0 0x55bd28202970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 -- 172.21.15.33:0/1741091971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd282038b0 msgr2=0x55bd28202970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd282038b0 0x55bd28202970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 -- 172.21.15.33:0/1741091971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 msgr2=0x55bd282295b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd282295b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 -- 172.21.15.33:0/1741091971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2820d420 con 0x55bd2820c650 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f96959b0640 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2821df50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9684031440 tx=0x7f9684000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c35696fbc669207 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f9677fff640 1 -- 172.21.15.33:0/1741091971 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684040070 con 0x55bd2820c650 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f9677fff640 1 -- 172.21.15.33:0/1741091971 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684004030 con 0x55bd2820c650 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.301+0000 7f9677fff640 1 -- 172.21.15.33:0/1741091971 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035040 con 0x55bd2820c650 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 -- 172.21.15.33:0/1741091971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 msgr2=0x55bd2821df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2821df50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9684031440 tx=0x7f9684000bc0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 -- 172.21.15.33:0/1741091971 shutdown_connections 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2820d040 0x55bd282295b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820c650 0x55bd2821df50 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 --2- 172.21.15.33:0/1741091971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd282038b0 0x55bd28202970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 -- 172.21.15.33:0/1741091971 >> 172.21.15.33:0/1741091971 conn(0x55bd28216640 msgr2=0x55bd28202260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 -- 172.21.15.33:0/1741091971 shutdown_connections 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.302+0000 7f969b204740 1 -- 172.21.15.33:0/1741091971 wait complete. 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 -- start start 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 Processor -- start 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282038b0 0x55bd2821d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd2821de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 0x55bd2821bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96901a67f0 con 0x55bd2820d040 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96901a7c60 con 0x55bd282038b0 2024-04-24T23:28:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96901a6ec0 con 0x55bd2820c650 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f969b204740 10 client.0 _ll_register_callbacks cb 0x7f9680003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 0x55bd2821bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 0x55bd2821bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41816/0 (socket says 172.21.15.33:41816) 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 -- 172.21.15.33:0/825837646 learned_addr learned my addr 172.21.15.33:0/825837646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96959b0640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd2821de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96961b1640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282038b0 0x55bd2821d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 -- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 msgr2=0x55bd2821de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd2821de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 -- 172.21.15.33:0/825837646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282038b0 msgr2=0x55bd2821d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282038b0 0x55bd2821d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 -- 172.21.15.33:0/825837646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd2821c6f0 con 0x55bd2820d040 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96961b1640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282038b0 0x55bd2821d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f96969b2640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 0x55bd2821bf60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9680005c40 tx=0x7f9680005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e240c0728b9a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.303+0000 7f9675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f9675ffb640 1 -- 172.21.15.33:0/825837646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680039030 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9684004030 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f9675ffb640 1 -- 172.21.15.33:0/825837646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9680004630 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f9675ffb640 1 -- 172.21.15.33:0/825837646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9680043050 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 10 client.18909 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bd282898c0 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f9675ffb640 1 -- 172.21.15.33:0/825837646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684004030 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f9675ffb640 1 -- 172.21.15.33:0/825837646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9680021470 con 0x55bd2820d040 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 10 client.18909 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 20 client.18909 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd282121b0 0x55bd2828c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 4 client.18909 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd28173f40 con 0x55bd282121b0 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.304+0000 7f96961b1640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd282121b0 0x55bd2828c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f96961b1640 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd282121b0 0x55bd2828c210 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f663bf7843eb68e server_cookie=6e7cbc634476d731 in_seq=0 out_seq=0 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f9675ffb640 10 client.18909 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f9675ffb640 1 -- 172.21.15.33:0/825837646 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bd28173f40 con 0x55bd282121b0 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f9675ffb640 10 client.18909 handle_command_reply: tid=0 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 client.18909 shutdown 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 msgr2=0x55bd2821bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 0x55bd2821bf60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9680005c40 tx=0x7f9680005b20 comp rx=0 tx=0).stop 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 shutdown_connections 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bd282121b0 0x55bd2828c210 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd2820d040 0x55bd2821bf60 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd2820c650 0x55bd2821de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f9675ffb640 0 client.18909 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:23.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 --2- 172.21.15.33:0/825837646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd282038b0 0x55bd2821d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 >> 172.21.15.33:0/825837646 conn(0x55bd28216640 msgr2=0x55bd28202af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 shutdown_connections 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 1 -- 172.21.15.33:0/825837646 wait complete. 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.305+0000 7f969b204740 20 client.18909 trim_cache size 0 max 16384 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stdout: "afabc35e-3bde-43e6-a872-d75955ee2fb0": { 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:28:23.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "afabc35e-3bde-43e6-a872-d75955ee2fb0", 2024-04-24T23:28:23.316 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:28:23.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:28:23.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:28:23.316 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:28:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96780a2b90 msgr2=0x7f96780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96780a2b90 0x7f96780a5050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55bd2820d8a0 tx=0x7f9680004ab0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 msgr2=0x7f9690069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 0x7f9690069440 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f968800e9c0 tx=0x7f968800ee80 comp rx=0 tx=0).stop 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 shutdown_connections 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96780a2b90 0x7f96780a5050 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9690102230 0x7f969019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690101870 0x7f969019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 --2- 172.21.15.33:0/1873310929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9690100680 0x7f9690069440 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 >> 172.21.15.33:0/1873310929 conn(0x7f96900fbbf0 msgr2=0x7f9690104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 shutdown_connections 2024-04-24T23:28:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:23.306+0000 7f9698415640 1 -- 172.21.15.33:0/1873310929 wait complete. 2024-04-24T23:28:23.319 INFO:tasks.cephfs.filesystem:scrub status for tag:afabc35e-3bde-43e6-a872-d75955ee2fb0 - {'path': '/', 'tag': 'afabc35e-3bde-43e6-a872-d75955ee2fb0', 'options': 'recursive,force'} 2024-04-24T23:28:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: pgmap v842: 129 pgs: 129 active+clean; 532 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 487 op/s 2024-04-24T23:28:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:28:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:23.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:23 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 66532 22.59 MB/sec execute 28 sec latency 66.107 ms 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: pgmap v842: 129 pgs: 129 active+clean; 532 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 487 op/s 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:23.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:23 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: pgmap v842: 129 pgs: 129 active+clean; 532 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 487 op/s 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:23.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:23 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:24.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 67027 22.67 MB/sec execute 29 sec latency 83.761 ms 2024-04-24T23:28:24.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:24 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:24.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:24 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:24.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:24 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:24.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:24 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:24.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:24 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:24.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:24 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:25.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 67458 22.79 MB/sec execute 30 sec latency 96.659 ms 2024-04-24T23:28:25.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:25 smithi038 ceph-mon[32450]: pgmap v843: 129 pgs: 129 active+clean; 530 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 424 op/s 2024-04-24T23:28:25.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:25 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:25.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:25 smithi117 ceph-mon[32529]: pgmap v843: 129 pgs: 129 active+clean; 530 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 424 op/s 2024-04-24T23:28:25.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:25 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:25.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:25 smithi033 ceph-mon[29201]: pgmap v843: 129 pgs: 129 active+clean; 530 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 424 op/s 2024-04-24T23:28:25.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:25 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:26.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 67743 22.49 MB/sec execute 31 sec latency 450.263 ms 2024-04-24T23:28:27.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 68083 22.47 MB/sec execute 32 sec latency 123.940 ms 2024-04-24T23:28:27.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:27 smithi038 ceph-mon[32450]: pgmap v844: 129 pgs: 129 active+clean; 520 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 420 op/s 2024-04-24T23:28:27.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:27 smithi117 ceph-mon[32529]: pgmap v844: 129 pgs: 129 active+clean; 520 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 420 op/s 2024-04-24T23:28:27.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:27 smithi033 ceph-mon[29201]: pgmap v844: 129 pgs: 129 active+clean; 520 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 420 op/s 2024-04-24T23:28:28.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 68466 22.29 MB/sec execute 33 sec latency 70.828 ms 2024-04-24T23:28:29.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 69050 22.27 MB/sec execute 34 sec latency 72.059 ms 2024-04-24T23:28:29.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:29 smithi038 ceph-mon[32450]: pgmap v845: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.3 MiB/s wr, 403 op/s 2024-04-24T23:28:29.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:29 smithi117 ceph-mon[32529]: pgmap v845: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.3 MiB/s wr, 403 op/s 2024-04-24T23:28:29.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:29 smithi033 ceph-mon[29201]: pgmap v845: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.3 MiB/s wr, 403 op/s 2024-04-24T23:28:30.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 69425 22.50 MB/sec execute 35 sec latency 125.484 ms 2024-04-24T23:28:31.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 69723 22.23 MB/sec execute 36 sec latency 379.988 ms 2024-04-24T23:28:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:31 smithi038 ceph-mon[32450]: pgmap v846: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 354 op/s 2024-04-24T23:28:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:31 smithi117 ceph-mon[32529]: pgmap v846: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 354 op/s 2024-04-24T23:28:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:31 smithi033 ceph-mon[29201]: pgmap v846: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 354 op/s 2024-04-24T23:28:32.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 70212 22.42 MB/sec execute 37 sec latency 94.412 ms 2024-04-24T23:28:33.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 70911 22.82 MB/sec execute 38 sec latency 51.428 ms 2024-04-24T23:28:33.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:33 smithi038 ceph-mon[32450]: pgmap v847: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 473 op/s 2024-04-24T23:28:33.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:33 smithi117 ceph-mon[32529]: pgmap v847: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 473 op/s 2024-04-24T23:28:33.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:33 smithi033 ceph-mon[29201]: pgmap v847: 129 pgs: 129 active+clean; 519 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 473 op/s 2024-04-24T23:28:34.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 71338 22.94 MB/sec execute 39 sec latency 65.111 ms 2024-04-24T23:28:35.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 71605 22.73 MB/sec execute 40 sec latency 127.852 ms 2024-04-24T23:28:35.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:35 smithi038 ceph-mon[32450]: pgmap v848: 129 pgs: 129 active+clean; 521 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 381 op/s 2024-04-24T23:28:35.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:35 smithi117 ceph-mon[32529]: pgmap v848: 129 pgs: 129 active+clean; 521 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 381 op/s 2024-04-24T23:28:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:35 smithi033 ceph-mon[29201]: pgmap v848: 129 pgs: 129 active+clean; 521 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 381 op/s 2024-04-24T23:28:36.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 71786 22.31 MB/sec execute 41 sec latency 442.580 ms 2024-04-24T23:28:37.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 72052 22.15 MB/sec execute 42 sec latency 121.826 ms 2024-04-24T23:28:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:37 smithi038 ceph-mon[32450]: pgmap v849: 129 pgs: 129 active+clean; 521 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2024-04-24T23:28:37.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:37 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:28:37.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:37 smithi117 ceph-mon[32529]: pgmap v849: 129 pgs: 129 active+clean; 521 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2024-04-24T23:28:37.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:37 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:28:37.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:37 smithi033 ceph-mon[29201]: pgmap v849: 129 pgs: 129 active+clean; 521 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2024-04-24T23:28:37.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:37 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:28:38.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 72600 22.26 MB/sec execute 43 sec latency 89.965 ms 2024-04-24T23:28:38.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:38 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:28:38.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:38 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:28:38.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:38 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:28:38.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:38 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:28:38.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:38 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:28:38.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:38 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:28:39.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 73184 22.29 MB/sec execute 44 sec latency 63.458 ms 2024-04-24T23:28:39.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:39 smithi038 ceph-mon[32450]: pgmap v850: 129 pgs: 129 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 430 op/s 2024-04-24T23:28:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:39 smithi117 ceph-mon[32529]: pgmap v850: 129 pgs: 129 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 430 op/s 2024-04-24T23:28:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:39 smithi033 ceph-mon[29201]: pgmap v850: 129 pgs: 129 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 430 op/s 2024-04-24T23:28:40.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 73738 22.67 MB/sec execute 45 sec latency 78.403 ms 2024-04-24T23:28:41.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 74099 22.55 MB/sec execute 46 sec latency 237.198 ms 2024-04-24T23:28:41.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:41 smithi038 ceph-mon[32450]: pgmap v851: 129 pgs: 129 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-24T23:28:41.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:41 smithi117 ceph-mon[32529]: pgmap v851: 129 pgs: 129 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-24T23:28:41.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:41 smithi033 ceph-mon[29201]: pgmap v851: 129 pgs: 129 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-24T23:28:42.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 74393 22.45 MB/sec execute 47 sec latency 247.441 ms 2024-04-24T23:28:43.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 74912 22.64 MB/sec execute 48 sec latency 73.574 ms 2024-04-24T23:28:43.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:43 smithi038 ceph-mon[32450]: pgmap v852: 129 pgs: 129 active+clean; 513 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.2 MiB/s wr, 477 op/s 2024-04-24T23:28:43.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:43 smithi117 ceph-mon[32529]: pgmap v852: 129 pgs: 129 active+clean; 513 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.2 MiB/s wr, 477 op/s 2024-04-24T23:28:43.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:43 smithi033 ceph-mon[29201]: pgmap v852: 129 pgs: 129 active+clean; 513 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.2 MiB/s wr, 477 op/s 2024-04-24T23:28:44.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 75273 22.54 MB/sec execute 49 sec latency 101.945 ms 2024-04-24T23:28:45.885 INFO:tasks.workunit.client.0.smithi033.stdout: 10 75635 22.42 MB/sec execute 50 sec latency 118.580 ms 2024-04-24T23:28:45.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:45 smithi038 ceph-mon[32450]: pgmap v853: 129 pgs: 129 active+clean; 512 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-04-24T23:28:45.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:45 smithi117 ceph-mon[32529]: pgmap v853: 129 pgs: 129 active+clean; 512 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-04-24T23:28:45.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:45 smithi033 ceph-mon[29201]: pgmap v853: 129 pgs: 129 active+clean; 512 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-04-24T23:28:46.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 76001 22.28 MB/sec execute 51 sec latency 306.592 ms 2024-04-24T23:28:47.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 76488 22.45 MB/sec execute 52 sec latency 84.910 ms 2024-04-24T23:28:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:47 smithi038 ceph-mon[32450]: pgmap v854: 129 pgs: 129 active+clean; 513 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 417 op/s 2024-04-24T23:28:47.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:47 smithi117 ceph-mon[32529]: pgmap v854: 129 pgs: 129 active+clean; 513 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 417 op/s 2024-04-24T23:28:47.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:47 smithi033 ceph-mon[29201]: pgmap v854: 129 pgs: 129 active+clean; 513 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 417 op/s 2024-04-24T23:28:48.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 76988 22.52 MB/sec execute 53 sec latency 84.129 ms 2024-04-24T23:28:49.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 77424 22.50 MB/sec execute 54 sec latency 65.351 ms 2024-04-24T23:28:49.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:49 smithi038 ceph-mon[32450]: pgmap v855: 129 pgs: 129 active+clean; 516 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 400 op/s 2024-04-24T23:28:49.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:49 smithi117 ceph-mon[32529]: pgmap v855: 129 pgs: 129 active+clean; 516 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 400 op/s 2024-04-24T23:28:49.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:49 smithi033 ceph-mon[29201]: pgmap v855: 129 pgs: 129 active+clean; 516 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 400 op/s 2024-04-24T23:28:50.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 77841 22.54 MB/sec execute 55 sec latency 129.653 ms 2024-04-24T23:28:51.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:51 smithi038 ceph-mon[32450]: pgmap v856: 129 pgs: 129 active+clean; 520 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 354 op/s 2024-04-24T23:28:51.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:51 smithi117 ceph-mon[32529]: pgmap v856: 129 pgs: 129 active+clean; 520 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 354 op/s 2024-04-24T23:28:51.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:51 smithi033 ceph-mon[29201]: pgmap v856: 129 pgs: 129 active+clean; 520 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 354 op/s 2024-04-24T23:28:51.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 78266 22.49 MB/sec execute 56 sec latency 274.961 ms 2024-04-24T23:28:52.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 78604 22.46 MB/sec execute 57 sec latency 296.176 ms 2024-04-24T23:28:53.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:28:53.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:28:53.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:53 smithi038 ceph-mon[32450]: pgmap v857: 129 pgs: 129 active+clean; 527 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.5 MiB/s wr, 499 op/s 2024-04-24T23:28:53.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:53 smithi117 ceph-mon[32529]: pgmap v857: 129 pgs: 129 active+clean; 527 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.5 MiB/s wr, 499 op/s 2024-04-24T23:28:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 -- 172.21.15.33:0/4011175730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828109660 msgr2=0x7f582810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 --2- 172.21.15.33:0/4011175730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828109660 0x7f582810baf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5818009860 tx=0x7f581802a340 comp rx=0 tx=0).stop 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 -- 172.21.15.33:0/4011175730 shutdown_connections 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 --2- 172.21.15.33:0/4011175730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828109660 0x7f582810baf0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 --2- 172.21.15.33:0/4011175730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58280741a0 0x7f5828074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 --2- 172.21.15.33:0/4011175730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5828075780 0x7f5828073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 -- 172.21.15.33:0/4011175730 >> 172.21.15.33:0/4011175730 conn(0x7f58280fccf0 msgr2=0x7f58280ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.439+0000 7f582f3f9640 1 -- 172.21.15.33:0/4011175730 shutdown_connections 2024-04-24T23:28:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.440+0000 7f582f3f9640 1 -- 172.21.15.33:0/4011175730 wait complete. 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.440+0000 7f582f3f9640 1 Processor -- start 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.440+0000 7f582f3f9640 1 -- start start 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582f3f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280741a0 0x7f5828110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582f3f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 0x7f5828111050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582f3f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5828109660 0x7f58281ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582f3f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5828102990 con 0x7f58280741a0 2024-04-24T23:28:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582f3f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5828102810 con 0x7f5828075780 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582f3f9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5828102b10 con 0x7f5828109660 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582d195640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280741a0 0x7f5828110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 0x7f5828111050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 0x7f5828111050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50958/0 (socket says 172.21.15.33:50958) 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582d195640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280741a0 0x7f5828110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47148/0 (socket says 172.21.15.33:47148) 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 -- 172.21.15.33:0/3823130233 learned_addr learned my addr 172.21.15.33:0/3823130233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582d996640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5828109660 0x7f58281ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 -- 172.21.15.33:0/3823130233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5828109660 msgr2=0x7f58281ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5828109660 0x7f58281ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 -- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280741a0 msgr2=0x7f5828110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280741a0 0x7f5828110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 -- 172.21.15.33:0/3823130233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58281161e0 con 0x7f5828075780 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.441+0000 7f582c994640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 0x7f5828111050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f582000ebc0 tx=0x7f582000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.442+0000 7f581e7fc640 1 -- 172.21.15.33:0/3823130233 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820010630 con 0x7f5828075780 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.442+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58281ae820 con 0x7f5828075780 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.442+0000 7f581e7fc640 1 -- 172.21.15.33:0/3823130233 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5820010cb0 con 0x7f5828075780 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.442+0000 7f581e7fc640 1 -- 172.21.15.33:0/3823130233 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820018bd0 con 0x7f5828075780 2024-04-24T23:28:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.442+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58281aece0 con 0x7f5828075780 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f581e7fc640 1 -- 172.21.15.33:0/3823130233 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58200107d0 con 0x7f5828075780 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 Processor -- start 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 -- start start 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d66155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 0x555d6615a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 0x555d6615d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555d660b96b0 con 0x555d66155c30 2024-04-24T23:28:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555d66061880 con 0x555d66155240 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f58321e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555d660b5670 con 0x555d6615ada0 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582c994640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 0x555d6615d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d195640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 0x555d6615a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d66155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582c994640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 0x555d6615d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34334/0 (socket says 172.21.15.33:34334) 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d66155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50966/0 (socket says 172.21.15.33:50966) 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582c994640 1 -- 172.21.15.33:0/1313163727 learned_addr learned my addr 172.21.15.33:0/1313163727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d195640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 0x555d6615a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47152/0 (socket says 172.21.15.33:47152) 2024-04-24T23:28:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 -- 172.21.15.33:0/1313163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 msgr2=0x555d6615d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 0x555d6615d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 -- 172.21.15.33:0/1313163727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 msgr2=0x555d6615a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 0x555d6615a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d996640 1 -- 172.21.15.33:0/1313163727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d65e977d0 con 0x555d66155240 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f581e7fc640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58140a2b90 0x7f58140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582d195640 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 0x555d6615a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.445+0000 7f582c994640 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 0x555d6615d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.446+0000 7f582c994640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58140a2b90 0x7f58140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.446+0000 7f582d996640 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d66155660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f581802a930 tx=0x7f5818009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b45cd127a41b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.446+0000 7f581e7fc640 1 -- 172.21.15.33:0/3823130233 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f58200cf9f0 con 0x7f5828075780 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.446+0000 7f58137fe640 1 -- 172.21.15.33:0/1313163727 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818041070 con 0x555d66155240 2024-04-24T23:28:53.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.446+0000 7f58137fe640 1 -- 172.21.15.33:0/1313163727 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5818004030 con 0x555d66155240 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.446+0000 7f58137fe640 1 -- 172.21.15.33:0/1313163727 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818037040 con 0x555d66155240 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 -- 172.21.15.33:0/1313163727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 msgr2=0x555d66155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f582c994640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58140a2b90 0x7f58140a5050 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58200a4600 tx=0x7f58200d5530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d66155660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f581802a930 tx=0x7f5818009510 comp rx=0 tx=0).stop 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 -- 172.21.15.33:0/1313163727 shutdown_connections 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d6615ada0 0x555d6615d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155c30 0x555d6615a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 --2- 172.21.15.33:0/1313163727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d66155660 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 -- 172.21.15.33:0/1313163727 >> 172.21.15.33:0/1313163727 conn(0x555d661508b0 msgr2=0x555d66150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 -- 172.21.15.33:0/1313163727 shutdown_connections 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 -- 172.21.15.33:0/1313163727 wait complete. 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 -- start start 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.447+0000 7f58321e8740 1 Processor -- start 2024-04-24T23:28:53.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d6615f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d6615fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66166bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555d6615dcf0 con 0x555d6615ada0 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555d66074f50 con 0x555d66155240 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555d6615de70 con 0x555d66155c30 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f58321e8740 10 client.0 _ll_register_callbacks cb 0x555d660b0000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d6615fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d6615fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34348/0 (socket says 172.21.15.33:34348) 2024-04-24T23:28:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 -- 172.21.15.33:0/3271594729 learned_addr learned my addr 172.21.15.33:0/3271594729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d996640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d6615f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d996640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66166bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 -- 172.21.15.33:0/3271594729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 msgr2=0x555d6615f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d6615f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 -- 172.21.15.33:0/3271594729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 msgr2=0x555d66166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66166bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 -- 172.21.15.33:0/3271594729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d65dcd3f0 con 0x555d66155c30 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f582d195640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d6615fb70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f582400b980 tx=0x7f582400be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5222c0da86bce982 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.448+0000 7f5810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f5810ff9640 1 -- 172.21.15.33:0/3271594729 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824003dc0 con 0x555d66155c30 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f5810ff9640 1 -- 172.21.15.33:0/3271594729 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5824011440 con 0x555d66155c30 2024-04-24T23:28:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5818004030 con 0x555d66155c30 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f5810ff9640 1 -- 172.21.15.33:0/3271594729 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5824011640 con 0x555d66155c30 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 10 client.28354 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555d65d8d0f0 con 0x555d66155c30 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f5810ff9640 1 -- 172.21.15.33:0/3271594729 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5818004030 con 0x555d66155c30 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f5810ff9640 1 -- 172.21.15.33:0/3271594729 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f582401b8f0 con 0x555d66155c30 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 10 client.28354 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 10 client.28354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 20 client.28354 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d6616b5c0 0x555d6616da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 4 client.28354 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d660c6bf0 con 0x555d6616b5c0 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.449+0000 7f582d996640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d6616b5c0 0x555d6616da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.451+0000 7f5810ff9640 10 client.28354 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.451+0000 7f582d996640 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d6616b5c0 0x555d6616da20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a21057cf3836e1 server_cookie=c65a7cd8240dfa63 in_seq=0 out_seq=0 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f5810ff9640 1 -- 172.21.15.33:0/3271594729 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d660c6bf0 con 0x555d6616b5c0 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f5810ff9640 10 client.28354 handle_command_reply: tid=0 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 client.28354 shutdown 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 msgr2=0x555d6615fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d6615fb70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f582400b980 tx=0x7f582400be40 comp rx=0 tx=0).stop 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 shutdown_connections 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d6616b5c0 0x555d6616da20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66166bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d6615fb70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 --2- 172.21.15.33:0/3271594729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155240 0x555d6615f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 >> 172.21.15.33:0/3271594729 conn(0x555d661508b0 msgr2=0x555d6614faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f5810ff9640 0 client.28354 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 shutdown_connections 2024-04-24T23:28:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 1 -- 172.21.15.33:0/3271594729 wait complete. 2024-04-24T23:28:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.452+0000 7f58321e8740 20 client.28354 trim_cache size 0 max 16384 2024-04-24T23:28:53.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:53 smithi033 ceph-mon[29201]: pgmap v857: 129 pgs: 129 active+clean; 527 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.5 MiB/s wr, 499 op/s 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 Processor -- start 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 -- start start 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155240 0x555d66150c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d66166b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d6615ada0 0x555d66157050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58281bba80 con 0x555d66155240 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58281b9350 con 0x555d6615ada0 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f58321e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58281b71a0 con 0x555d66155c30 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d996640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155240 0x555d66150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d66166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d996640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155240 0x555d66150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47172/0 (socket says 172.21.15.33:47172) 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d996640 1 -- 172.21.15.33:0/2950367773 learned_addr learned my addr 172.21.15.33:0/2950367773 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d996640 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d6615ada0 0x555d66157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 -- 172.21.15.33:0/2950367773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d6615ada0 msgr2=0x555d66157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d6615ada0 0x555d66157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 -- 172.21.15.33:0/2950367773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155240 msgr2=0x555d66150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155240 0x555d66150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 -- 172.21.15.33:0/2950367773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d6615a340 con 0x555d66155c30 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f582d195640 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d66166b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x555d6614c380 tx=0x7f58240115d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743ed71dc240069 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f5812ffd640 1 -- 172.21.15.33:0/2950367773 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582402d020 con 0x555d66155c30 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f5812ffd640 1 -- 172.21.15.33:0/2950367773 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582400b480 con 0x555d66155c30 2024-04-24T23:28:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.473+0000 7f5812ffd640 1 -- 172.21.15.33:0/2950367773 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824011d80 con 0x555d66155c30 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- 172.21.15.33:0/2950367773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 msgr2=0x555d66166b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d66166b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x555d6614c380 tx=0x7f58240115d0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- 172.21.15.33:0/2950367773 shutdown_connections 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d6615ada0 0x555d66157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155c30 0x555d66166b70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- 172.21.15.33:0/2950367773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d66155240 0x555d66150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- 172.21.15.33:0/2950367773 >> 172.21.15.33:0/2950367773 conn(0x555d661508b0 msgr2=0x555d6615f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- 172.21.15.33:0/2950367773 shutdown_connections 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- 172.21.15.33:0/2950367773 wait complete. 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- start start 2024-04-24T23:28:53.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 Processor -- start 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155240 0x555d66163cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155c30 0x555d66164200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66165210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58281b7810 con 0x555d6615ada0 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58281b8c80 con 0x555d66155c30 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58281b7ee0 con 0x555d66155240 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 10 client.0 _ll_register_callbacks cb 0x7f58240045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66165210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47182/0 (socket says 172.21.15.33:47182) 2024-04-24T23:28:53.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 -- 172.21.15.33:0/1335353650 learned_addr learned my addr 172.21.15.33:0/1335353650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155240 0x555d66163cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d195640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155c30 0x555d66164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 -- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155240 msgr2=0x555d66163cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155240 0x555d66163cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 -- 172.21.15.33:0/1335353650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155c30 msgr2=0x555d66164200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155c30 0x555d66164200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 -- 172.21.15.33:0/1335353650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d66165800 con 0x555d6615ada0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f582d996640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66165210 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f581802a930 tx=0x7f581802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b863f01675176 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f5810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f5810ff9640 1 -- 172.21.15.33:0/1335353650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818037040 con 0x555d6615ada0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f5810ff9640 1 -- 172.21.15.33:0/1335353650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5818030900 con 0x555d6615ada0 2024-04-24T23:28:53.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582400b480 con 0x555d6615ada0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.475+0000 7f5810ff9640 1 -- 172.21.15.33:0/1335353650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5818041040 con 0x555d6615ada0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 10 client.18927 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555d66166b80 con 0x555d6615ada0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f5810ff9640 1 -- 172.21.15.33:0/1335353650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582400b480 con 0x555d6615ada0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f5810ff9640 1 -- 172.21.15.33:0/1335353650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5818004920 con 0x555d6615ada0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 10 client.18927 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 20 client.18927 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d661d1bb0 0x555d661d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 4 client.18927 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d660c0f40 con 0x555d661d1bb0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.477+0000 7f582d195640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d661d1bb0 0x555d661d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f582d195640 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d661d1bb0 0x555d661d3fd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a73cb7e46c151c server_cookie=2673d233620b867b in_seq=0 out_seq=0 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f5810ff9640 10 client.18927 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f5810ff9640 1 -- 172.21.15.33:0/1335353650 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d660c0f40 con 0x555d661d1bb0 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f5810ff9640 10 client.18927 handle_command_reply: tid=0 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 client.18927 shutdown 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 msgr2=0x555d66165210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66165210 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f581802a930 tx=0x7f581802a340 comp rx=0 tx=0).stop 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 shutdown_connections 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555d661d1bb0 0x555d661d3fd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d6615ada0 0x555d66165210 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d66155c30 0x555d66164200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 --2- 172.21.15.33:0/1335353650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d66155240 0x555d66163cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f5810ff9640 0 client.18927 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:53.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 >> 172.21.15.33:0/1335353650 conn(0x555d661508b0 msgr2=0x555d66157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 shutdown_connections 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 1 -- 172.21.15.33:0/1335353650 wait complete. 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.478+0000 7f58321e8740 20 client.18927 trim_cache size 0 max 16384 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:28:53.485 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:28:53.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58140a2b90 msgr2=0x7f58140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58140a2b90 0x7f58140a5050 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58200a4600 tx=0x7f58200d5530 comp rx=0 tx=0).stop 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 msgr2=0x7f5828111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 0x7f5828111050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f582000ebc0 tx=0x7f582000c6a0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 shutdown_connections 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f58140a2b90 0x7f58140a5050 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5828109660 0x7f58281ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5828075780 0x7f5828111050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 --2- 172.21.15.33:0/3823130233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280741a0 0x7f5828110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 >> 172.21.15.33:0/3823130233 conn(0x7f58280fccf0 msgr2=0x7f5828103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 shutdown_connections 2024-04-24T23:28:53.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.480+0000 7f582f3f9640 1 -- 172.21.15.33:0/3823130233 wait complete. 2024-04-24T23:28:53.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:53.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 -- 172.21.15.33:0/2779447014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 msgr2=0x7f0550109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 --2- 172.21.15.33:0/2779447014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f0550109be0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0548009ca0 tx=0x7f054802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 -- 172.21.15.33:0/2779447014 shutdown_connections 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 --2- 172.21.15.33:0/2779447014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f0550109be0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 --2- 172.21.15.33:0/2779447014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0550102990 0x7f0550102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 --2- 172.21.15.33:0/2779447014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 0x7f0550101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 -- 172.21.15.33:0/2779447014 >> 172.21.15.33:0/2779447014 conn(0x7f05500fccf0 msgr2=0x7f05500ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 -- 172.21.15.33:0/2779447014 shutdown_connections 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 -- 172.21.15.33:0/2779447014 wait complete. 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 Processor -- start 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.607+0000 7f05581b7640 1 -- start start 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 0x7f0550197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0550102990 0x7f0550197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f055019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f055010d450 con 0x7f0550103350 2024-04-24T23:28:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f055010d2d0 con 0x7f0550102990 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f055010d5d0 con 0x7f05501017a0 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f055019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0555f53640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 0x7f0550197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f055019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47208/0 (socket says 172.21.15.33:47208) 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 -- 172.21.15.33:0/4264214049 learned_addr learned my addr 172.21.15.33:0/4264214049 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0555752640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0550102990 0x7f0550197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 -- 172.21.15.33:0/4264214049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 msgr2=0x7f0550197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 0x7f0550197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 -- 172.21.15.33:0/4264214049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0550102990 msgr2=0x7f0550197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0550102990 0x7f0550197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 -- 172.21.15.33:0/4264214049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055019f650 con 0x7f0550103350 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0555f53640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 0x7f0550197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0556754640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f055019efe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f054802f540 tx=0x7f0548031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548003b70 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0548003d10 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055019f8e0 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.608+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054802fb50 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.609+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055019fe80 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.610+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0550108a60 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.611+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f054802fcf0 con 0x7f0550103350 2024-04-24T23:28:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.612+0000 7f0546ffd640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f053c0a2c10 0x7f053c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.612+0000 7f0555f53640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f053c0a2c10 0x7f053c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.612+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f05480f0730 con 0x7f0550103350 2024-04-24T23:28:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.613+0000 7f0555f53640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f053c0a2c10 0x7f053c0a50d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05501027f0 tx=0x7f054c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.619+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05480bd300 con 0x7f0550103350 2024-04-24T23:28:53.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.885+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0550062340 con 0x7f0550103350 2024-04-24T23:28:53.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.885+0000 7f0546ffd640 1 -- 172.21.15.33:0/4264214049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f05480bcc00 con 0x7f0550103350 2024-04-24T23:28:53.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:28:53.885 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:28:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:28:53.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 79217 22.71 MB/sec execute 58 sec latency 48.490 ms 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f053c0a2c10 msgr2=0x7f053c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f053c0a2c10 0x7f053c0a50d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05501027f0 tx=0x7f054c005f90 comp rx=0 tx=0).stop 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 msgr2=0x7f055019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f055019efe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f054802f540 tx=0x7f0548031750 comp rx=0 tx=0).stop 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 shutdown_connections 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f053c0a2c10 0x7f053c0a50d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550103350 0x7f055019efe0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0550102990 0x7f0550197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 --2- 172.21.15.33:0/4264214049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05501017a0 0x7f0550197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 >> 172.21.15.33:0/4264214049 conn(0x7f05500fccf0 msgr2=0x7f0550105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 shutdown_connections 2024-04-24T23:28:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:53.888+0000 7f05581b7640 1 -- 172.21.15.33:0/4264214049 wait complete. 2024-04-24T23:28:53.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:28:54.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 -- 172.21.15.33:0/1227438993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648101790 msgr2=0x7f7648101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 --2- 172.21.15.33:0/1227438993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648101790 0x7f7648101bb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f763802a410 comp rx=0 tx=0).stop 2024-04-24T23:28:54.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 -- 172.21.15.33:0/1227438993 shutdown_connections 2024-04-24T23:28:54.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 --2- 172.21.15.33:0/1227438993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648103340 0x7f7648109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 --2- 172.21.15.33:0/1227438993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7648102980 0x7f7648102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 --2- 172.21.15.33:0/1227438993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648101790 0x7f7648101bb0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 -- 172.21.15.33:0/1227438993 >> 172.21.15.33:0/1227438993 conn(0x7f76480fcd20 msgr2=0x7f76480ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 -- 172.21.15.33:0/1227438993 shutdown_connections 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.017+0000 7f764e458640 1 -- 172.21.15.33:0/1227438993 wait complete. 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 Processor -- start 2024-04-24T23:28:54.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 -- start start 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7648101790 0x7f7648197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648102980 0x7f7648198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 0x7f764819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d400 con 0x7f7648103340 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d280 con 0x7f7648102980 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764e458640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d580 con 0x7f7648101790 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 0x7f764819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 0x7f764819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47234/0 (socket says 172.21.15.33:47234) 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 -- 172.21.15.33:0/530171199 learned_addr learned my addr 172.21.15.33:0/530171199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f76477fe640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648102980 0x7f7648198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f7647fff640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7648101790 0x7f7648197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 -- 172.21.15.33:0/530171199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7648101790 msgr2=0x7f7648197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7648101790 0x7f7648197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 -- 172.21.15.33:0/530171199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648102980 msgr2=0x7f7648198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648102980 0x7f7648198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.018+0000 7f764c9f5640 1 -- 172.21.15.33:0/530171199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764819f710 con 0x7f7648103340 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.019+0000 7f764c9f5640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 0x7f764819f0a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f764000eb10 tx=0x7f764000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.019+0000 7f76457fa640 1 -- 172.21.15.33:0/530171199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640004b20 con 0x7f7648103340 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.019+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764819fa00 con 0x7f7648103340 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.019+0000 7f76457fa640 1 -- 172.21.15.33:0/530171199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7640004cc0 con 0x7f7648103340 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.019+0000 7f76457fa640 1 -- 172.21.15.33:0/530171199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640010760 con 0x7f7648103340 2024-04-24T23:28:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.019+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764819ff50 con 0x7f7648103340 2024-04-24T23:28:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f76457fa640 1 -- 172.21.15.33:0/530171199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7640005020 con 0x7f7648103340 2024-04-24T23:28:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 Processor -- start 2024-04-24T23:28:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 -- start start 2024-04-24T23:28:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d652fe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 0x558d65303720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d65303d90 0x558d65306220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558d65262670 con 0x558d65303d90 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558d6520a970 con 0x558d652fe230 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.022+0000 7f7651167740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d6525e630 con 0x558d652fec20 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 0x558d65303720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d652fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 0x558d65303720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34420/0 (socket says 172.21.15.33:34420) 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 -- 172.21.15.33:0/423570508 learned_addr learned my addr 172.21.15.33:0/423570508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f76457fa640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76300a2b90 0x7f76300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f76477fe640 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d65303d90 0x558d65306220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f764c9f5640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76300a2b90 0x7f76300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 -- 172.21.15.33:0/423570508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 msgr2=0x558d652fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d652fe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 -- 172.21.15.33:0/423570508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d65303d90 msgr2=0x558d65306220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d65303d90 0x558d65306220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 -- 172.21.15.33:0/423570508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6522e400 con 0x558d652fec20 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f76477fe640 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d65303d90 0x558d65306220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f76457fa640 1 -- 172.21.15.33:0/530171199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f76400cf9c0 con 0x7f7648103340 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f7647fff640 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 0x558d65303720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7638031610 tx=0x7f7638005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac903fbcb252a02f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f762e7fc640 1 -- 172.21.15.33:0/423570508 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638040070 con 0x558d652fec20 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f764c9f5640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76300a2b90 0x7f76300a5050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f76400049d0 tx=0x7f76400bc040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f762e7fc640 1 -- 172.21.15.33:0/423570508 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7638004030 con 0x558d652fec20 2024-04-24T23:28:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.023+0000 7f762e7fc640 1 -- 172.21.15.33:0/423570508 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638035040 con 0x558d652fec20 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 -- 172.21.15.33:0/423570508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 msgr2=0x558d65303720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 0x558d65303720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7638031610 tx=0x7f7638005de0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 -- 172.21.15.33:0/423570508 shutdown_connections 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d65303d90 0x558d65306220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fec20 0x558d65303720 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 --2- 172.21.15.33:0/423570508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d652fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 -- 172.21.15.33:0/423570508 >> 172.21.15.33:0/423570508 conn(0x558d652f98a0 msgr2=0x558d652f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 -- 172.21.15.33:0/423570508 shutdown_connections 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.024+0000 7f7651167740 1 -- 172.21.15.33:0/423570508 wait complete. 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 -- start start 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 Processor -- start 2024-04-24T23:28:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d65308330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d65308870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558d65306ce0 con 0x558d652fec20 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558d6521e050 con 0x558d652fe230 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d65306e60 con 0x558d65303d90 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7651167740 10 client.0 _ll_register_callbacks cb 0x558d64f75be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d65308870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d65308330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d65308870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47252/0 (socket says 172.21.15.33:47252) 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f7647fff640 1 -- 172.21.15.33:0/3561479785 learned_addr learned my addr 172.21.15.33:0/3561479785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.025+0000 7f76477fe640 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7647fff640 1 -- 172.21.15.33:0/3561479785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 msgr2=0x558d6530f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7647fff640 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7647fff640 1 -- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 msgr2=0x558d65308330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7647fff640 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d65308330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7647fff640 1 -- 172.21.15.33:0/3561479785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d651efdc0 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7647fff640 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d65308870 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f7638032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9478705e8c88919 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f76277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f76277fe640 1 -- 172.21.15.33:0/3561479785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638035070 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f76277fe640 1 -- 172.21.15.33:0/3561479785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f76380316b0 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7638004030 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f76277fe640 1 -- 172.21.15.33:0/3561479785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7638058050 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7651167740 10 client.18945 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.026+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558d64f763f0 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f76277fe640 1 -- 172.21.15.33:0/3561479785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7638004030 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f76277fe640 1 -- 172.21.15.33:0/3561479785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7638003cd0 con 0x558d652fec20 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f7651167740 10 client.18945 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f7651167740 10 client.18945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f7651167740 20 client.18945 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f7651167740 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d65316190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f7651167740 4 client.18945 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d6526fbb0 con 0x558d65313da0 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.027+0000 7f76477fe640 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d65316190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.028+0000 7f76277fe640 10 client.18945 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.028+0000 7f76477fe640 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d65316190 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb958c299ff807a server_cookie=7358854e23cf311b in_seq=0 out_seq=0 2024-04-24T23:28:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f76277fe640 1 -- 172.21.15.33:0/3561479785 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d6526fbb0 con 0x558d65313da0 2024-04-24T23:28:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f76277fe640 10 client.18945 handle_command_reply: tid=0 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 client.18945 shutdown 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 msgr2=0x558d65308870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d65308870 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f7638032f60 comp rx=0 tx=0).stop 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 shutdown_connections 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d65316190 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f76277fe640 0 client.18945 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d65308870 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 --2- 172.21.15.33:0/3561479785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d65308330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.031+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 >> 172.21.15.33:0/3561479785 conn(0x558d652f98a0 msgr2=0x558d652f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.032+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 shutdown_connections 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.032+0000 7f7651167740 1 -- 172.21.15.33:0/3561479785 wait complete. 2024-04-24T23:28:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.032+0000 7f7651167740 20 client.18945 trim_cache size 0 max 16384 2024-04-24T23:28:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 Processor -- start 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 -- start start 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 0x558d65307950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d65303d90 0x558d652fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76481aaaa0 con 0x558d652fec20 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a8370 con 0x558d65303d90 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7651167740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a61c0 con 0x558d652fe230 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f76477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 0x558d65307950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.049+0000 7f7647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47266/0 (socket says 172.21.15.33:47266) 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 -- 172.21.15.33:0/743754146 learned_addr learned my addr 172.21.15.33:0/743754146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f76477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 0x558d65307950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34434/0 (socket says 172.21.15.33:34434) 2024-04-24T23:28:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f76477fe640 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d65303d90 0x558d652fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 -- 172.21.15.33:0/743754146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 msgr2=0x558d65307950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 0x558d65307950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 -- 172.21.15.33:0/743754146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d65303d90 msgr2=0x558d652fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d65303d90 0x558d652fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 -- 172.21.15.33:0/743754146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d650407d0 con 0x558d652fec20 2024-04-24T23:28:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f76477fe640 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 0x558d65307950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f7647fff640 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530f870 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f763802a930 tx=0x7f7638005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b626d7068b0e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.050+0000 7f762dffb640 1 -- 172.21.15.33:0/743754146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638035070 con 0x558d652fec20 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.051+0000 7f762dffb640 1 -- 172.21.15.33:0/743754146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763803c440 con 0x558d652fec20 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.051+0000 7f762dffb640 1 -- 172.21.15.33:0/743754146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638040040 con 0x558d652fec20 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 -- 172.21.15.33:0/743754146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 msgr2=0x558d6530f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530f870 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f763802a930 tx=0x7f7638005fd0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 -- 172.21.15.33:0/743754146 shutdown_connections 2024-04-24T23:28:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d65303d90 0x558d652fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530f870 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 --2- 172.21.15.33:0/743754146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d652fe230 0x558d65307950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 -- 172.21.15.33:0/743754146 >> 172.21.15.33:0/743754146 conn(0x558d652f98a0 msgr2=0x558d652f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 -- 172.21.15.33:0/743754146 shutdown_connections 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 -- 172.21.15.33:0/743754146 wait complete. 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 -- start start 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.052+0000 7f7651167740 1 Processor -- start 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d6530d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a6830 con 0x558d652fec20 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a7ca0 con 0x558d652fe230 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a6f00 con 0x558d65303d90 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7651167740 10 client.0 _ll_register_callbacks cb 0x7f763c0099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47276/0 (socket says 172.21.15.33:47276) 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f7647fff640 1 -- 172.21.15.33:0/30990299 learned_addr learned my addr 172.21.15.33:0/30990299 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d6530d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 -- 172.21.15.33:0/30990299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 msgr2=0x558d6530e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 -- 172.21.15.33:0/30990299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 msgr2=0x558d6530d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.053+0000 7f76477fe640 1 -- 172.21.15.33:0/30990299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6530edc0 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f76477fe640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d6530d280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f763c004700 tx=0x7f763802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11042ff80fc218cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f76277fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f76277fe640 1 -- 172.21.15.33:0/30990299 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c003c90 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f76277fe640 1 -- 172.21.15.33:0/30990299 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f763c01a040 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763803c440 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f76277fe640 1 -- 172.21.15.33:0/30990299 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f763c003e30 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f7651167740 10 client.28451 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.054+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558d6537a560 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.055+0000 7f76277fe640 1 -- 172.21.15.33:0/30990299 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763803c440 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.055+0000 7f76277fe640 1 -- 172.21.15.33:0/30990299 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f763c020040 con 0x558d652fe230 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.055+0000 7f7651167740 10 client.28451 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.055+0000 7f7651167740 10 client.28451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.055+0000 7f7651167740 20 client.28451 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.055+0000 7f7651167740 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d6537cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.056+0000 7f7651167740 4 client.28451 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:28:54.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.056+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d65308120 con 0x558d65313da0 2024-04-24T23:28:54.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.056+0000 7f7647fff640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d6537cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.056+0000 7f76277fe640 10 client.28451 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:54.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.056+0000 7f7647fff640 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d6537cf70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18962ffd75f8f2a6 server_cookie=d34e7f83bff4d165 in_seq=0 out_seq=0 2024-04-24T23:28:54.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f76277fe640 1 -- 172.21.15.33:0/30990299 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d65308120 con 0x558d65313da0 2024-04-24T23:28:54.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f76277fe640 10 client.28451 handle_command_reply: tid=0 2024-04-24T23:28:54.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 client.28451 shutdown 2024-04-24T23:28:54.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 msgr2=0x558d6530d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d6530d280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f763c004700 tx=0x7f763802a410 comp rx=0 tx=0).stop 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 shutdown_connections 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x558d65313da0 0x558d6537cf70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d65303d90 0x558d6530e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d652fec20 0x558d6530d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f76277fe640 0 client.28451 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 --2- 172.21.15.33:0/30990299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d652fe230 0x558d6530d280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 >> 172.21.15.33:0/30990299 conn(0x558d652f98a0 msgr2=0x558d652f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 shutdown_connections 2024-04-24T23:28:54.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 1 -- 172.21.15.33:0/30990299 wait complete. 2024-04-24T23:28:54.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.062+0000 7f7651167740 20 client.28451 trim_cache size 0 max 16384 2024-04-24T23:28:54.065 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:28:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76300a2b90 msgr2=0x7f76300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76300a2b90 0x7f76300a5050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f76400049d0 tx=0x7f76400bc040 comp rx=0 tx=0).stop 2024-04-24T23:28:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 msgr2=0x7f764819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 0x7f764819f0a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f764000eb10 tx=0x7f764000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 shutdown_connections 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76300a2b90 0x7f76300a5050 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7648103340 0x7f764819f0a0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648102980 0x7f7648198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 --2- 172.21.15.33:0/530171199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7648101790 0x7f7648197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 >> 172.21.15.33:0/530171199 conn(0x7f76480fcd20 msgr2=0x7f7648105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 shutdown_connections 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.063+0000 7f764e458640 1 -- 172.21.15.33:0/530171199 wait complete. 2024-04-24T23:28:54.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.227+0000 7f5eb2161640 1 -- 172.21.15.33:0/2411567713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 msgr2=0x7f5eac101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.227+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2411567713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac101bb0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009900 tx=0x7f5e9c02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 -- 172.21.15.33:0/2411567713 shutdown_connections 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2411567713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eac103340 0x7f5eac109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2411567713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 0x7f5eac102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2411567713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac101bb0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 -- 172.21.15.33:0/2411567713 >> 172.21.15.33:0/2411567713 conn(0x7f5eac0fcd20 msgr2=0x7f5eac0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 -- 172.21.15.33:0/2411567713 shutdown_connections 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 -- 172.21.15.33:0/2411567713 wait complete. 2024-04-24T23:28:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 Processor -- start 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 -- start start 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.228+0000 7f5eb2161640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 0x7f5eac198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eac103340 0x7f5eac19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac10d3e0 con 0x7f5eac101790 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac10d260 con 0x7f5eac103340 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac10d560 con 0x7f5eac102980 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47286/0 (socket says 172.21.15.33:47286) 2024-04-24T23:28:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2080472775 learned_addr learned my addr 172.21.15.33:0/2080472775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 0x7f5eac198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eabfff640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eac103340 0x7f5eac19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2080472775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 msgr2=0x7f5eac198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 0x7f5eac198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2080472775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eac103340 msgr2=0x7f5eac19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eac103340 0x7f5eac19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2080472775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac19f700 con 0x7f5eac101790 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 0x7f5eac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac197ae0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c031610 tx=0x7f5e9c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5ea8ff9640 1 -- 172.21.15.33:0/2080472775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c03f030 con 0x7f5eac101790 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac19f990 con 0x7f5eac101790 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5ea8ff9640 1 -- 172.21.15.33:0/2080472775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e9c003ad0 con 0x7f5eac101790 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5ea8ff9640 1 -- 172.21.15.33:0/2080472775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c035070 con 0x7f5eac101790 2024-04-24T23:28:54.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.229+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac19fe80 con 0x7f5eac101790 2024-04-24T23:28:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 Processor -- start 2024-04-24T23:28:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 -- start start 2024-04-24T23:28:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5ea8ff9640 1 -- 172.21.15.33:0/2080472775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5e9c042030 con 0x7f5eac101790 2024-04-24T23:28:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558506f2d670 con 0x558506fced90 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558506ed5970 con 0x558506fc9c20 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558506f29630 con 0x558506fc9230 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47296/0 (socket says 172.21.15.33:47296) 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eaaffd640 1 -- 172.21.15.33:0/956671386 learned_addr learned my addr 172.21.15.33:0/956671386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.232+0000 7f5eabfff640 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eaaffd640 1 -- 172.21.15.33:0/956671386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 msgr2=0x558506fc9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eaaffd640 1 -- 172.21.15.33:0/956671386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 msgr2=0x558506fce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eaaffd640 1 -- 172.21.15.33:0/956671386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558506ef9400 con 0x558506fced90 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd1220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00114f0 tx=0x7f5ea00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76510fc7ae83a96c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5ea8ff9640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5e940a2b90 0x7f5e940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5e91ffb640 1 -- 172.21.15.33:0/956671386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea001e070 con 0x558506fced90 2024-04-24T23:28:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5e91ffb640 1 -- 172.21.15.33:0/956671386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea00035f0 con 0x558506fced90 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5eabfff640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5e940a2b90 0x7f5e940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.233+0000 7f5e91ffb640 1 -- 172.21.15.33:0/956671386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000f820 con 0x558506fced90 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5ea8ff9640 1 -- 172.21.15.33:0/2080472775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5e9c0ef960 con 0x7f5eac101790 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/956671386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 msgr2=0x558506fd1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eabfff640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5e940a2b90 0x7f5e940a5050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x558506fca480 tx=0x7f5ea4009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd1220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00114f0 tx=0x7f5ea00119b0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/956671386 shutdown_connections 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd1220 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/956671386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/956671386 >> 172.21.15.33:0/956671386 conn(0x558506fc48a0 msgr2=0x558506fc4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/956671386 shutdown_connections 2024-04-24T23:28:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.234+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/956671386 wait complete. 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 -- start start 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 Processor -- start 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fda8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558506fd1d40 con 0x558506fced90 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558506ee9050 con 0x558506fc9c20 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558506fd1ec0 con 0x558506fc9230 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fda8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fda8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47312/0 (socket says 172.21.15.33:47312) 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eabfff640 1 -- 172.21.15.33:0/2737664758 learned_addr learned my addr 172.21.15.33:0/2737664758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.235+0000 7f5eb4e6d740 10 client.0 _ll_register_callbacks cb 0x558506c40be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2737664758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 msgr2=0x558506fd3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2737664758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 msgr2=0x558506fda8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fda8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eab7fe640 1 -- 172.21.15.33:0/2737664758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558506ebadc0 con 0x558506fc9230 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eabfff640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fda8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd3340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0ed120 tx=0x7f5e9c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a000aefc19c03a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5e8affd640 1 -- 172.21.15.33:0/2737664758 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c049070 con 0x558506fc9230 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5e8affd640 1 -- 172.21.15.33:0/2737664758 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5e9c03b460 con 0x558506fc9230 2024-04-24T23:28:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea00035f0 con 0x558506fc9230 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5e8affd640 1 -- 172.21.15.33:0/2737664758 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c0bba20 con 0x558506fc9230 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eb4e6d740 10 client.28396 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558506c413f0 con 0x558506fc9230 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5e8affd640 1 -- 172.21.15.33:0/2737664758 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea00035f0 con 0x558506fc9230 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.236+0000 7f5e8affd640 1 -- 172.21.15.33:0/2737664758 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5e9c0b8550 con 0x558506fc9230 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eb4e6d740 10 client.28396 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eb4e6d740 10 client.28396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eb4e6d740 20 client.28396 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558506fdf1c0 0x558506fe1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eb4e6d740 4 client.28396 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558506f3abb0 con 0x558506fdf1c0 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558506fdf1c0 0x558506fe1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558506fdf1c0 0x558506fe1680 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=556dd8ad67a2c8e1 server_cookie=aeca2bc77ada7ea6 in_seq=0 out_seq=0 2024-04-24T23:28:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.237+0000 7f5e8affd640 10 client.28396 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5e8affd640 1 -- 172.21.15.33:0/2737664758 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558506f3abb0 con 0x558506fdf1c0 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5e8affd640 10 client.28396 handle_command_reply: tid=0 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 client.28396 shutdown 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 msgr2=0x558506fd3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd3340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0ed120 tx=0x7f5e9c009560 comp rx=0 tx=0).stop 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 shutdown_connections 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x558506fdf1c0 0x558506fe1680 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fda8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5e8affd640 0 client.28396 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2737664758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd3340 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 >> 172.21.15.33:0/2737664758 conn(0x558506fc48a0 msgr2=0x558506fc3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 shutdown_connections 2024-04-24T23:28:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2737664758 wait complete. 2024-04-24T23:28:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.238+0000 7f5eb4e6d740 20 client.28396 trim_cache size 0 max 16384 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.252+0000 7f5eb4e6d740 1 Processor -- start 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.252+0000 7f5eb4e6d740 1 -- start start 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 0x558506fda880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fced90 0x558506fc7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1acc50 con 0x558506fc9c20 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1aa520 con 0x558506fced90 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1a8370 con 0x558506fc9230 2024-04-24T23:28:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 0x558506fda880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 0x558506fda880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47320/0 (socket says 172.21.15.33:47320) 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 -- 172.21.15.33:0/2569459538 learned_addr learned my addr 172.21.15.33:0/2569459538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eabfff640 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fced90 0x558506fc7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 -- 172.21.15.33:0/2569459538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 msgr2=0x558506fc77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 -- 172.21.15.33:0/2569459538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fced90 msgr2=0x558506fc7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fced90 0x558506fc7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 -- 172.21.15.33:0/2569459538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558506d0b7d0 con 0x558506fc9c20 2024-04-24T23:28:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 0x558506fda880 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009e20 tx=0x7f5ea0003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a397546339efc29c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.253+0000 7f5e917fa640 1 -- 172.21.15.33:0/2569459538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea001e040 con 0x558506fc9c20 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5e917fa640 1 -- 172.21.15.33:0/2569459538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea0029020 con 0x558506fc9c20 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5e917fa640 1 -- 172.21.15.33:0/2569459538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea002a190 con 0x558506fc9c20 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2569459538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 msgr2=0x558506fda880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 0x558506fda880 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009e20 tx=0x7f5ea0003c50 comp rx=0 tx=0).stop 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2569459538 shutdown_connections 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fced90 0x558506fc7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fc9c20 0x558506fda880 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/2569459538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fc77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.254+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2569459538 >> 172.21.15.33:0/2569459538 conn(0x558506fc48a0 msgr2=0x558506fc3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.255+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2569459538 shutdown_connections 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.255+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/2569459538 wait complete. 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.255+0000 7f5eb4e6d740 1 -- start start 2024-04-24T23:28:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.255+0000 7f5eb4e6d740 1 Processor -- start 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1a89e0 con 0x558506fced90 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1a9e50 con 0x558506fc9c20 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1a90b0 con 0x558506fc9230 2024-04-24T23:28:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eb4e6d740 10 client.0 _ll_register_callbacks cb 0x7f5e9c0f5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47330/0 (socket says 172.21.15.33:47330) 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eabfff640 1 -- 172.21.15.33:0/1216293034 learned_addr learned my addr 172.21.15.33:0/1216293034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34484/0 (socket says 172.21.15.33:34484) 2024-04-24T23:28:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eaaffd640 1 -- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 msgr2=0x558506fd78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eaaffd640 1 -- 172.21.15.33:0/1216293034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 msgr2=0x558506fd8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eaaffd640 1 -- 172.21.15.33:0/1216293034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558506fd9400 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.256+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eaaffd640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd7e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558506fd4010 tx=0x7f5ea0003ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e55839f67150826 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5e8affd640 1 -- 172.21.15.33:0/1216293034 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea001e070 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea0029020 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5e8affd640 1 -- 172.21.15.33:0/1216293034 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ea0005030 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5e8affd640 1 -- 172.21.15.33:0/1216293034 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5ea0039050 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 10 client.28469 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558506fda700 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5e8affd640 1 -- 172.21.15.33:0/1216293034 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea0029020 con 0x558506fc9c20 2024-04-24T23:28:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5e8affd640 1 -- 172.21.15.33:0/1216293034 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5ea002a7a0 con 0x558506fc9c20 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 10 client.28469 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 10 client.28469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 20 client.28469 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55850704e100 0x5585070504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 4 client.28469 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.257+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558506f34f00 con 0x55850704e100 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55850704e100 0x5585070504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eab7fe640 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55850704e100 0x5585070504f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fac69d80201db3b server_cookie=2156cb6a8d2ef1c8 in_seq=0 out_seq=0 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5e8affd640 10 client.28469 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5e8affd640 1 -- 172.21.15.33:0/1216293034 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558506f34f00 con 0x55850704e100 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5e8affd640 10 client.28469 handle_command_reply: tid=0 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 client.28469 shutdown 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 msgr2=0x558506fd7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd7e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558506fd4010 tx=0x7f5ea0003ac0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 shutdown_connections 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55850704e100 0x5585070504f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558506fced90 0x558506fd8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558506fc9c20 0x558506fd7e00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5e8affd640 0 client.28469 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 --2- 172.21.15.33:0/1216293034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558506fc9230 0x558506fd78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.258+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 >> 172.21.15.33:0/1216293034 conn(0x558506fc48a0 msgr2=0x558506fc77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 shutdown_connections 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb4e6d740 1 -- 172.21.15.33:0/1216293034 wait complete. 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb4e6d740 20 client.28469 trim_cache size 0 max 16384 2024-04-24T23:28:54.265 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:28:54.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5e940a2b90 msgr2=0x7f5e940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5e940a2b90 0x7f5e940a5050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x558506fca480 tx=0x7f5ea4009500 comp rx=0 tx=0).stop 2024-04-24T23:28:54.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 msgr2=0x7f5eac197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac197ae0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c031610 tx=0x7f5e9c004270 comp rx=0 tx=0).stop 2024-04-24T23:28:54.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 shutdown_connections 2024-04-24T23:28:54.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5e940a2b90 0x7f5e940a5050 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5eac103340 0x7f5eac19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eac102980 0x7f5eac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 --2- 172.21.15.33:0/2080472775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac101790 0x7f5eac197ae0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 >> 172.21.15.33:0/2080472775 conn(0x7f5eac0fcd20 msgr2=0x7f5eac105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 shutdown_connections 2024-04-24T23:28:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.259+0000 7f5eb2161640 1 -- 172.21.15.33:0/2080472775 wait complete. 2024-04-24T23:28:54.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:28:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 -- 172.21.15.33:0/1460924608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c101870 msgr2=0x7f7f6c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 --2- 172.21.15.33:0/1460924608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c101870 0x7f7f6c101cf0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009900 tx=0x7f7f5c02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 -- 172.21.15.33:0/1460924608 shutdown_connections 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 --2- 172.21.15.33:0/1460924608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c102230 0x7f7f6c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 --2- 172.21.15.33:0/1460924608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c101870 0x7f7f6c101cf0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 --2- 172.21.15.33:0/1460924608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c100680 0x7f7f6c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 -- 172.21.15.33:0/1460924608 >> 172.21.15.33:0/1460924608 conn(0x7f7f6c0fbbf0 msgr2=0x7f7f6c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 -- 172.21.15.33:0/1460924608 shutdown_connections 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.385+0000 7f7f733ce640 1 -- 172.21.15.33:0/1460924608 wait complete. 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 Processor -- start 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 -- start start 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 0x7f7f6c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 0x7f7f6c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10c350 con 0x7f7f6c102230 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10c1d0 con 0x7f7f6c101870 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f733ce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10c4d0 con 0x7f7f6c100680 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 0x7f7f6c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7116a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 0x7f7f6c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 0x7f7f6c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47338/0 (socket says 172.21.15.33:47338) 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 -- 172.21.15.33:0/309117139 learned_addr learned my addr 172.21.15.33:0/309117139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f70969640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 -- 172.21.15.33:0/309117139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 msgr2=0x7f7f6c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 0x7f7f6c0694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 -- 172.21.15.33:0/309117139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 msgr2=0x7f7f6c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7196b640 1 -- 172.21.15.33:0/309117139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c19f6b0 con 0x7f7f6c102230 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.386+0000 7f7f7116a640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 0x7f7f6c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.387+0000 7f7f7196b640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 0x7f7f6c19edf0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400eb10 tx=0x7f7f6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.387+0000 7f7f627fc640 1 -- 172.21.15.33:0/309117139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64004b20 con 0x7f7f6c102230 2024-04-24T23:28:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.387+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c19f9a0 con 0x7f7f6c102230 2024-04-24T23:28:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.387+0000 7f7f627fc640 1 -- 172.21.15.33:0/309117139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f64004cc0 con 0x7f7f6c102230 2024-04-24T23:28:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.387+0000 7f7f627fc640 1 -- 172.21.15.33:0/309117139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64010760 con 0x7f7f6c102230 2024-04-24T23:28:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.387+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c19ff10 con 0x7f7f6c102230 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.389+0000 7f7f627fc640 1 -- 172.21.15.33:0/309117139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7f64005020 con 0x7f7f6c102230 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f627fc640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f580a2b90 0x7f7f580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 Processor -- start 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 -- start start 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f7116a640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f580a2b90 0x7f7f580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f627fc640 1 -- 172.21.15.33:0/309117139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f640cfad0 con 0x7f7f6c102230 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5236ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a52373b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a52376630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a522ce670 con 0x559a5236e640 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a52276970 con 0x559a5236f030 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f761bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a522ca630 con 0x559a523741a0 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f7116a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5236ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f7116a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5236ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47348/0 (socket says 172.21.15.33:47348) 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f7116a640 1 -- 172.21.15.33:0/282173655 learned_addr learned my addr 172.21.15.33:0/282173655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f70969640 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a52373b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.390+0000 7f7f7196b640 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a52376630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f70969640 1 -- 172.21.15.33:0/282173655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 msgr2=0x559a52376630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f70969640 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a52376630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f70969640 1 -- 172.21.15.33:0/282173655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 msgr2=0x559a5236ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f70969640 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5236ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f70969640 1 -- 172.21.15.33:0/282173655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5229a400 con 0x559a5236f030 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f7116a640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f580a2b90 0x7f7f580a5050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f68013db0 tx=0x7f7f68009f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f70969640 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a52373b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02a930 tx=0x7f7f5c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be58ead485ec764 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f577fe640 1 -- 172.21.15.33:0/282173655 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c035040 con 0x559a5236f030 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f577fe640 1 -- 172.21.15.33:0/282173655 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c004400 con 0x559a5236f030 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.391+0000 7f7f577fe640 1 -- 172.21.15.33:0/282173655 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0316b0 con 0x559a5236f030 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 -- 172.21.15.33:0/282173655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 msgr2=0x559a52373b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a52373b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02a930 tx=0x7f7f5c005c00 comp rx=0 tx=0).stop 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 -- 172.21.15.33:0/282173655 shutdown_connections 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a52376630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a52373b30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 --2- 172.21.15.33:0/282173655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5236ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 -- 172.21.15.33:0/282173655 >> 172.21.15.33:0/282173655 conn(0x559a523658a0 msgr2=0x559a52365ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 -- 172.21.15.33:0/282173655 shutdown_connections 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 -- 172.21.15.33:0/282173655 wait complete. 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 -- start start 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.392+0000 7f7f761bc740 1 Processor -- start 2024-04-24T23:28:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236e640 0x559a52378740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236f030 0x559a52378c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 0x559a5237fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a523770f0 con 0x559a523741a0 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a5228a050 con 0x559a5236e640 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a52377270 con 0x559a5236f030 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f761bc740 10 client.0 _ll_register_callbacks cb 0x559a51fe1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f7196b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 0x559a5237fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f7116a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236e640 0x559a52378740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236f030 0x559a52378c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f7196b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 0x559a5237fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47364/0 (socket says 172.21.15.33:47364) 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f7196b640 1 -- 172.21.15.33:0/2602724191 learned_addr learned my addr 172.21.15.33:0/2602724191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 -- 172.21.15.33:0/2602724191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236e640 msgr2=0x559a52378740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236e640 0x559a52378740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 -- 172.21.15.33:0/2602724191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 msgr2=0x559a5237fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 0x559a5237fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 -- 172.21.15.33:0/2602724191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a5225bdc0 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f7196b640 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 0x559a5237fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f70969640 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236f030 0x559a52378c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x559a52366770 tx=0x7f7f5c0049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420c7daafbec1ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.393+0000 7f7f54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f54ff9640 1 -- 172.21.15.33:0/2602724191 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c052060 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c004400 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f54ff9640 1 -- 172.21.15.33:0/2602724191 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f5c004200 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f54ff9640 1 -- 172.21.15.33:0/2602724191 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c035050 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 10 client.28414 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a51fe23f0 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f54ff9640 1 -- 172.21.15.33:0/2602724191 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c004400 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f54ff9640 1 -- 172.21.15.33:0/2602724191 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7f5c003bf0 con 0x559a5236f030 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 10 client.28414 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 10 client.28414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 20 client.28414 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523841b0 0x559a523865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 4 client.28414 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a522dbbb0 con 0x559a523841b0 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.394+0000 7f7f7116a640 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523841b0 0x559a523865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.395+0000 7f7f7116a640 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523841b0 0x559a523865a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dcbcd51a7157c3 server_cookie=c9a2293843faf180 in_seq=0 out_seq=0 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.395+0000 7f7f54ff9640 10 client.28414 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f54ff9640 1 -- 172.21.15.33:0/2602724191 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a522dbbb0 con 0x559a523841b0 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f54ff9640 10 client.28414 handle_command_reply: tid=0 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 client.28414 shutdown 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236f030 msgr2=0x559a52378c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236f030 0x559a52378c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x559a52366770 tx=0x7f7f5c0049a0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 shutdown_connections 2024-04-24T23:28:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523841b0 0x559a523865a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a523741a0 0x559a5237fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236f030 0x559a52378c80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 --2- 172.21.15.33:0/2602724191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236e640 0x559a52378740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f54ff9640 0 client.28414 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 >> 172.21.15.33:0/2602724191 conn(0x559a523658a0 msgr2=0x559a52364b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 shutdown_connections 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 1 -- 172.21.15.33:0/2602724191 wait complete. 2024-04-24T23:28:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.397+0000 7f7f761bc740 20 client.28414 trim_cache size 0 max 16384 2024-04-24T23:28:54.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 Processor -- start 2024-04-24T23:28:54.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 -- start start 2024-04-24T23:28:54.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236e640 0x559a523701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 0x559a523706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a523741a0 0x559a5237fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1aa980 con 0x559a5236f030 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1a8250 con 0x559a523741a0 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f761bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1a60a0 con 0x559a5236e640 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f70969640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 0x559a523706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f70969640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 0x559a523706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47378/0 (socket says 172.21.15.33:47378) 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f70969640 1 -- 172.21.15.33:0/793085006 learned_addr learned my addr 172.21.15.33:0/793085006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f7196b640 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a523741a0 0x559a5237fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.412+0000 7f7f7116a640 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236e640 0x559a523701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f70969640 1 -- 172.21.15.33:0/793085006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236e640 msgr2=0x559a523701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f70969640 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236e640 0x559a523701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f70969640 1 -- 172.21.15.33:0/793085006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a523741a0 msgr2=0x559a5237fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f70969640 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a523741a0 0x559a5237fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f70969640 1 -- 172.21.15.33:0/793085006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a520ac7d0 con 0x559a5236f030 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f7196b640 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a523741a0 0x559a5237fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f70969640 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 0x559a523706f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c031440 tx=0x7f7f5c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94bc61b3649cfa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f56ffd640 1 -- 172.21.15.33:0/793085006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c052040 con 0x559a5236f030 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f56ffd640 1 -- 172.21.15.33:0/793085006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c005850 con 0x559a5236f030 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.413+0000 7f7f56ffd640 1 -- 172.21.15.33:0/793085006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0318c0 con 0x559a5236f030 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 -- 172.21.15.33:0/793085006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 msgr2=0x559a523706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 0x559a523706f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c031440 tx=0x7f7f5c02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:54.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 -- 172.21.15.33:0/793085006 shutdown_connections 2024-04-24T23:28:54.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a523741a0 0x559a5237fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236f030 0x559a523706f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- 172.21.15.33:0/793085006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a5236e640 0x559a523701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 -- 172.21.15.33:0/793085006 >> 172.21.15.33:0/793085006 conn(0x559a523658a0 msgr2=0x559a5236faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 -- 172.21.15.33:0/793085006 shutdown_connections 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 -- 172.21.15.33:0/793085006 wait complete. 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 -- start start 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 Processor -- start 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5237d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a5237de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.414+0000 7f7f761bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a5237ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f761bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1a6710 con 0x559a5236e640 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f761bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1a7b80 con 0x559a5236f030 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f761bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1a6de0 con 0x559a523741a0 2024-04-24T23:28:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f761bc740 10 client.0 _ll_register_callbacks cb 0x7f7f68013900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5237d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5237d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47380/0 (socket says 172.21.15.33:47380) 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 -- 172.21.15.33:0/4097839798 learned_addr learned my addr 172.21.15.33:0/4097839798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7196b640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a5237ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f70969640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a5237de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 -- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 msgr2=0x559a5237ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a5237ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 -- 172.21.15.33:0/4097839798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 msgr2=0x559a5237de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a5237de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 -- 172.21.15.33:0/4097839798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a5237f3d0 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f7116a640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5237d900 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009460 tx=0x7f7f6801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cb3a74edaa2058 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f54ff9640 1 -- 172.21.15.33:0/4097839798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68021db0 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f54ff9640 1 -- 172.21.15.33:0/4097839798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f6801c040 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.415+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c005850 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f54ff9640 1 -- 172.21.15.33:0/4097839798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f68036050 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 10 client.18987 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a523eabf0 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f54ff9640 1 -- 172.21.15.33:0/4097839798 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c005850 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f54ff9640 1 -- 172.21.15.33:0/4097839798 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7f6802a070 con 0x559a5236e640 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 10 client.18987 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 10 client.18987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 20 client.18987 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523eb620 0x559a523eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 4 client.18987 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:28:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a52378530 con 0x559a523eb620 2024-04-24T23:28:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.416+0000 7f7f70969640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523eb620 0x559a523eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f54ff9640 10 client.18987 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f70969640 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523eb620 0x559a523eda40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b4d27421f93cce server_cookie=d11fe5bbc5cb0358 in_seq=0 out_seq=0 2024-04-24T23:28:54.424 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:54 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4264214049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:54.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f54ff9640 1 -- 172.21.15.33:0/4097839798 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a52378530 con 0x559a523eb620 2024-04-24T23:28:54.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f54ff9640 10 client.18987 handle_command_reply: tid=0 2024-04-24T23:28:54.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 client.18987 shutdown 2024-04-24T23:28:54.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 msgr2=0x559a5237d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5237d900 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009460 tx=0x7f7f6801d000 comp rx=0 tx=0).stop 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 shutdown_connections 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559a523eb620 0x559a523eda40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a523741a0 0x559a5237ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a5236f030 0x559a5237de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f54ff9640 0 client.18987 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 --2- 172.21.15.33:0/4097839798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a5236e640 0x559a5237d900 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 >> 172.21.15.33:0/4097839798 conn(0x559a523658a0 msgr2=0x559a52369040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 shutdown_connections 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.417+0000 7f7f761bc740 1 -- 172.21.15.33:0/4097839798 wait complete. 2024-04-24T23:28:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f761bc740 20 client.18987 trim_cache size 0 max 16384 2024-04-24T23:28:54.427 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:28:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f580a2b90 msgr2=0x7f7f580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f580a2b90 0x7f7f580a5050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f68013db0 tx=0x7f7f68009f50 comp rx=0 tx=0).stop 2024-04-24T23:28:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 msgr2=0x7f7f6c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 0x7f7f6c19edf0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400eb10 tx=0x7f7f6400c2d0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 shutdown_connections 2024-04-24T23:28:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7f580a2b90 0x7f7f580a5050 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f6c102230 0x7f7f6c19edf0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c101870 0x7f7f6c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 --2- 172.21.15.33:0/309117139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f6c100680 0x7f7f6c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 >> 172.21.15.33:0/309117139 conn(0x7f7f6c0fbbf0 msgr2=0x7f7f6c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 shutdown_connections 2024-04-24T23:28:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:54.418+0000 7f7f733ce640 1 -- 172.21.15.33:0/309117139 wait complete. 2024-04-24T23:28:54.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:54 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4264214049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:54.432 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:54 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4264214049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:54.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 79616 22.67 MB/sec execute 59 sec latency 102.354 ms 2024-04-24T23:28:55.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:55 smithi038 ceph-mon[32450]: pgmap v858: 129 pgs: 129 active+clean; 527 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 383 op/s 2024-04-24T23:28:55.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:55 smithi117 ceph-mon[32529]: pgmap v858: 129 pgs: 129 active+clean; 527 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 383 op/s 2024-04-24T23:28:55.433 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:28:55.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:55.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:28:55.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:55 smithi033 ceph-mon[29201]: pgmap v858: 129 pgs: 129 active+clean; 527 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 383 op/s 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 -- 172.21.15.33:0/2227978453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25600ffe20 msgr2=0x7f2560100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 --2- 172.21.15.33:0/2227978453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25600ffe20 0x7f2560100240 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f255c009900 tx=0x7f255c02a410 comp rx=0 tx=0).stop 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 -- 172.21.15.33:0/2227978453 shutdown_connections 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 --2- 172.21.15.33:0/2227978453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 0x7f2560108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 --2- 172.21.15.33:0/2227978453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2560100f70 0x7f25601013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 --2- 172.21.15.33:0/2227978453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25600ffe20 0x7f2560100240 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.550+0000 7f2567b16640 1 -- 172.21.15.33:0/2227978453 >> 172.21.15.33:0/2227978453 conn(0x7f25600fb370 msgr2=0x7f25600fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 -- 172.21.15.33:0/2227978453 shutdown_connections 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 -- 172.21.15.33:0/2227978453 wait complete. 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 Processor -- start 2024-04-24T23:28:55.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 -- start start 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600ffe20 0x7f256010b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 0x7f2560109a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 0x7f2560109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f256010d460 con 0x7f2560100f70 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f256010d2e0 con 0x7f25600ffe20 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f2567b16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f256010d5e0 con 0x7f2560101930 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f25650b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 0x7f2560109a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f25658b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600ffe20 0x7f256010b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f25650b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 0x7f2560109a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47412/0 (socket says 172.21.15.33:47412) 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.551+0000 7f25650b1640 1 -- 172.21.15.33:0/1049210829 learned_addr learned my addr 172.21.15.33:0/1049210829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25660b3640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 0x7f2560109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25650b1640 1 -- 172.21.15.33:0/1049210829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 msgr2=0x7f2560109fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25650b1640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 0x7f2560109fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25650b1640 1 -- 172.21.15.33:0/1049210829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600ffe20 msgr2=0x7f256010b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25650b1640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600ffe20 0x7f256010b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25650b1640 1 -- 172.21.15.33:0/1049210829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256010a830 con 0x7f2560100f70 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25660b3640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 0x7f2560109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25658b2640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600ffe20 0x7f256010b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f25650b1640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 0x7f2560109a60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f255000e9c0 tx=0x7f255000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f2556ffd640 1 -- 172.21.15.33:0/1049210829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255000cd80 con 0x7f2560100f70 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f2556ffd640 1 -- 172.21.15.33:0/1049210829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25500035f0 con 0x7f2560100f70 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f2556ffd640 1 -- 172.21.15.33:0/1049210829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550010690 con 0x7f2560100f70 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25601a1bd0 con 0x7f2560100f70 2024-04-24T23:28:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.552+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25601a2170 con 0x7f2560100f70 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.554+0000 7f2556ffd640 1 -- 172.21.15.33:0/1049210829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f25500037a0 con 0x7f2560100f70 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 Processor -- start 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 -- start start 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa91e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa923280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa925d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a9aa8812c0 con 0x55a9aa91dd90 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a9aa829910 con 0x55a9aa9238f0 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f256a81b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a9aa87d280 con 0x55a9aa91e780 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f2556ffd640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f254c0a2c10 0x7f254c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f25658b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa923280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f25658b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa923280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:34584/0 (socket says 172.21.15.33:34584) 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f25658b2640 1 -- 172.21.15.33:0/1418316554 learned_addr learned my addr 172.21.15.33:0/1418316554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f25660b3640 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa91e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.555+0000 7f25650b1640 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa925d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25660b3640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f254c0a2c10 0x7f254c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f2556ffd640 1 -- 172.21.15.33:0/1049210829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f25500cf880 con 0x7f2560100f70 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25658b2640 1 -- 172.21.15.33:0/1418316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 msgr2=0x55a9aa925d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25658b2640 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa925d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25658b2640 1 -- 172.21.15.33:0/1418316554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 msgr2=0x55a9aa91e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25658b2640 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa91e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25658b2640 1 -- 172.21.15.33:0/1418316554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9aa83afb0 con 0x55a9aa91e780 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25658b2640 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa923280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f255c02a930 tx=0x7f255c031760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a9aad9e179fb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f25660b3640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f254c0a2c10 0x7f254c0a50d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2558013c10 tx=0x7f255800a920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f254bfff640 1 -- 172.21.15.33:0/1418316554 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c040070 con 0x55a9aa91e780 2024-04-24T23:28:55.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f254bfff640 1 -- 172.21.15.33:0/1418316554 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c005040 con 0x55a9aa91e780 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.556+0000 7f254bfff640 1 -- 172.21.15.33:0/1418316554 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c035040 con 0x55a9aa91e780 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 -- 172.21.15.33:0/1418316554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 msgr2=0x55a9aa923280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa923280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f255c02a930 tx=0x7f255c031760 comp rx=0 tx=0).stop 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 -- 172.21.15.33:0/1418316554 shutdown_connections 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa925d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa923280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 --2- 172.21.15.33:0/1418316554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa91e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 -- 172.21.15.33:0/1418316554 >> 172.21.15.33:0/1418316554 conn(0x55a9aa8e5950 msgr2=0x55a9aa919810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 -- 172.21.15.33:0/1418316554 shutdown_connections 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 -- 172.21.15.33:0/1418316554 wait complete. 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 -- start start 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.557+0000 7f256a81b740 1 Processor -- start 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa928180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa91e780 0x55a9aa9286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 0x55a9aa92f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a9aa926840 con 0x55a9aa9238f0 2024-04-24T23:28:55.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a9aa83ce90 con 0x55a9aa91e780 2024-04-24T23:28:55.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a9aa9269c0 con 0x55a9aa91dd90 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f256a81b740 10 client.0 _ll_register_callbacks cb 0x55a9aa878970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 0x55a9aa92f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 0x55a9aa92f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47430/0 (socket says 172.21.15.33:47430) 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 -- 172.21.15.33:0/115801563 learned_addr learned my addr 172.21.15.33:0/115801563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25650b1640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa91e780 0x55a9aa9286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25658b2640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa928180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 -- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 msgr2=0x55a9aa928180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa928180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 -- 172.21.15.33:0/115801563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa91e780 msgr2=0x55a9aa9286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa91e780 0x55a9aa9286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 -- 172.21.15.33:0/115801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9aa65d7e0 con 0x55a9aa9238f0 2024-04-24T23:28:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25650b1640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa91e780 0x55a9aa9286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25660b3640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 0x55a9aa92f700 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f255800e670 tx=0x7f255800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5eb7506b30f696 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.558+0000 7f25497fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f25497fa640 1 -- 172.21.15.33:0/115801563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558004d40 con 0x55a9aa9238f0 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f255c005040 con 0x55a9aa9238f0 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f25497fa640 1 -- 172.21.15.33:0/115801563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2558003970 con 0x55a9aa9238f0 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f25497fa640 1 -- 172.21.15.33:0/115801563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2558031050 con 0x55a9aa9238f0 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 10 client.18999 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:55.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a9aa6bdf70 con 0x55a9aa9238f0 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f25497fa640 1 -- 172.21.15.33:0/115801563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c005040 con 0x55a9aa9238f0 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f25497fa640 1 -- 172.21.15.33:0/115801563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f255801f600 con 0x55a9aa9238f0 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 10 client.18999 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 10 client.18999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 20 client.18999 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa933c10 0x55a9aa936090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 4 client.18999 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9aa8555a0 con 0x55a9aa933c10 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.559+0000 7f25658b2640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa933c10 0x55a9aa936090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.560+0000 7f25497fa640 10 client.18999 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.560+0000 7f25658b2640 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa933c10 0x55a9aa936090 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2664e51ebaa647b server_cookie=6d626ee77d363ddd in_seq=0 out_seq=0 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.560+0000 7f25497fa640 1 -- 172.21.15.33:0/115801563 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9aa8555a0 con 0x55a9aa933c10 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.560+0000 7f25497fa640 10 client.18999 handle_command_reply: tid=0 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.560+0000 7f256a81b740 1 client.18999 shutdown 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 msgr2=0x55a9aa92f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 0x55a9aa92f700 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f255800e670 tx=0x7f255800a620 comp rx=0 tx=0).stop 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 shutdown_connections 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa933c10 0x55a9aa936090 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa9238f0 0x55a9aa92f700 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa91e780 0x55a9aa9286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 --2- 172.21.15.33:0/115801563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa928180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f25497fa640 0 client.18999 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 >> 172.21.15.33:0/115801563 conn(0x55a9aa8e5950 msgr2=0x55a9aa918950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 shutdown_connections 2024-04-24T23:28:55.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 1 -- 172.21.15.33:0/115801563 wait complete. 2024-04-24T23:28:55.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.561+0000 7f256a81b740 20 client.18999 trim_cache size 0 max 16384 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 Processor -- start 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 -- start start 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa9180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa92f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa918600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f25601aef50 con 0x55a9aa91dd90 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ac820 con 0x55a9aa9238f0 2024-04-24T23:28:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f256a81b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25601aa670 con 0x55a9aa91e780 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa9180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa9180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47440/0 (socket says 172.21.15.33:47440) 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 -- 172.21.15.33:0/1691511683 learned_addr learned my addr 172.21.15.33:0/1691511683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25660b3640 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa918600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 -- 172.21.15.33:0/1691511683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 msgr2=0x55a9aa92f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25650b1640 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa92f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa92f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 -- 172.21.15.33:0/1691511683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 msgr2=0x55a9aa918600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa918600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 -- 172.21.15.33:0/1691511683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9aa555170 con 0x55a9aa91dd90 2024-04-24T23:28:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25660b3640 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa918600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.579+0000 7f25658b2640 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa9180c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f255c005ab0 tx=0x7f255c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811ea65d82003f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f254b7fe640 1 -- 172.21.15.33:0/1691511683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c040050 con 0x55a9aa91dd90 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f254b7fe640 1 -- 172.21.15.33:0/1691511683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c005560 con 0x55a9aa91dd90 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f254b7fe640 1 -- 172.21.15.33:0/1691511683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c035040 con 0x55a9aa91dd90 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 -- 172.21.15.33:0/1691511683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 msgr2=0x55a9aa9180c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa9180c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f255c005ab0 tx=0x7f255c031400 comp rx=0 tx=0).stop 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 -- 172.21.15.33:0/1691511683 shutdown_connections 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa918600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91e780 0x55a9aa92f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 --2- 172.21.15.33:0/1691511683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91dd90 0x55a9aa9180c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 -- 172.21.15.33:0/1691511683 >> 172.21.15.33:0/1691511683 conn(0x55a9aa8e5950 msgr2=0x55a9aa91fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 -- 172.21.15.33:0/1691511683 shutdown_connections 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.580+0000 7f256a81b740 1 -- 172.21.15.33:0/1691511683 wait complete. 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 -- start start 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 Processor -- start 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa92d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 0x55a9aa92d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa92e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f25601aace0 con 0x55a9aa91e780 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ac150 con 0x55a9aa9238f0 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ab3b0 con 0x55a9aa91dd90 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f256a81b740 10 client.0 _ll_register_callbacks cb 0x7f25580134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f25650b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 0x55a9aa92d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f25650b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 0x55a9aa92d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47444/0 (socket says 172.21.15.33:47444) 2024-04-24T23:28:55.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f25650b1640 1 -- 172.21.15.33:0/1495176856 learned_addr learned my addr 172.21.15.33:0/1495176856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f25660b3640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa92e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.581+0000 7f25658b2640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa92d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25650b1640 1 -- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 msgr2=0x55a9aa92d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25650b1640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa92d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25650b1640 1 -- 172.21.15.33:0/1495176856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 msgr2=0x55a9aa92e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25650b1640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa92e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25650b1640 1 -- 172.21.15.33:0/1495176856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9aa92ee60 con 0x55a9aa91e780 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25660b3640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa92e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25650b1640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 0x55a9aa92d860 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2550098150 tx=0x7f255001da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6fb8bd01d6ef30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25497fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25497fa640 1 -- 172.21.15.33:0/1495176856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500a00a0 con 0x55a9aa91e780 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25497fa640 1 -- 172.21.15.33:0/1495176856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2550003b30 con 0x55a9aa91e780 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f255c005560 con 0x55a9aa91e780 2024-04-24T23:28:55.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25497fa640 1 -- 172.21.15.33:0/1495176856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f25500a4960 con 0x55a9aa91e780 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 10 client.19005 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a9aa99a780 con 0x55a9aa91e780 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25497fa640 1 -- 172.21.15.33:0/1495176856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c005560 con 0x55a9aa91e780 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f25497fa640 1 -- 172.21.15.33:0/1495176856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f255009abb0 con 0x55a9aa91e780 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 10 client.19005 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 20 client.19005 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa99b160 0x55a9aa99d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 4 client.19005 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.582+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a9aa88e800 con 0x55a9aa99b160 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.583+0000 7f25658b2640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa99b160 0x55a9aa99d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.583+0000 7f25658b2640 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa99b160 0x55a9aa99d550 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a581767d17cb1b server_cookie=f6e205f30664e09e in_seq=0 out_seq=0 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.583+0000 7f25497fa640 10 client.19005 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f25497fa640 1 -- 172.21.15.33:0/1495176856 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a9aa88e800 con 0x55a9aa99b160 2024-04-24T23:28:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f25497fa640 10 client.19005 handle_command_reply: tid=0 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 client.19005 shutdown 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 msgr2=0x55a9aa92d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 0x55a9aa92d860 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2550098150 tx=0x7f255001da50 comp rx=0 tx=0).stop 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 shutdown_connections 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a9aa99b160 0x55a9aa99d550 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9aa9238f0 0x55a9aa92e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f25497fa640 0 client.19005 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9aa91e780 0x55a9aa92d860 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 --2- 172.21.15.33:0/1495176856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9aa91dd90 0x55a9aa92d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 >> 172.21.15.33:0/1495176856 conn(0x55a9aa8e5950 msgr2=0x55a9aa9180c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 shutdown_connections 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.584+0000 7f256a81b740 1 -- 172.21.15.33:0/1495176856 wait complete. 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f256a81b740 20 client.19005 trim_cache size 0 max 16384 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3c18bac3-299a-4587-a895-d288f9c79731", 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:28:55.592 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:28:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f254c0a2c10 msgr2=0x7f254c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f254c0a2c10 0x7f254c0a50d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2558013c10 tx=0x7f255800a920 comp rx=0 tx=0).stop 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 msgr2=0x7f2560109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 0x7f2560109a60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f255000e9c0 tx=0x7f255000ee80 comp rx=0 tx=0).stop 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 shutdown_connections 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f254c0a2c10 0x7f254c0a50d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560101930 0x7f2560109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2560100f70 0x7f2560109a60 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 --2- 172.21.15.33:0/1049210829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600ffe20 0x7f256010b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 >> 172.21.15.33:0/1049210829 conn(0x7f25600fb370 msgr2=0x7f2560104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 shutdown_connections 2024-04-24T23:28:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.585+0000 7f2567b16640 1 -- 172.21.15.33:0/1049210829 wait complete. 2024-04-24T23:28:55.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:28:55.597 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:28:55.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 -- 172.21.15.33:0/836326615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654101870 msgr2=0x7f4654101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 --2- 172.21.15.33:0/836326615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654101870 0x7f4654101cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4648009900 tx=0x7f464802a410 comp rx=0 tx=0).stop 2024-04-24T23:28:55.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 -- 172.21.15.33:0/836326615 shutdown_connections 2024-04-24T23:28:55.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 --2- 172.21.15.33:0/836326615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654102230 0x7f4654108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 --2- 172.21.15.33:0/836326615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654101870 0x7f4654101cf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 --2- 172.21.15.33:0/836326615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 0x7f4654100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 -- 172.21.15.33:0/836326615 >> 172.21.15.33:0/836326615 conn(0x7f46540fbbf0 msgr2=0x7f46540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 -- 172.21.15.33:0/836326615 shutdown_connections 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 -- 172.21.15.33:0/836326615 wait complete. 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.756+0000 7f4659d09640 1 Processor -- start 2024-04-24T23:28:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 -- start start 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 0x7f4654069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 0x7f465419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654102230 0x7f465419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f465410c300 con 0x7f4654101870 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f465410c180 con 0x7f4654100680 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4659d09640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f465410c480 con 0x7f4654102230 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4652ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 0x7f465419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4652ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 0x7f465419c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56150/0 (socket says 172.21.15.33:56150) 2024-04-24T23:28:55.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4652ffd640 1 -- 172.21.15.33:0/1785210228 learned_addr learned my addr 172.21.15.33:0/1785210228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f46537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 0x7f4654069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4653fff640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654102230 0x7f465419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f46537fe640 1 -- 172.21.15.33:0/1785210228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654102230 msgr2=0x7f465419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f46537fe640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654102230 0x7f465419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f46537fe640 1 -- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 msgr2=0x7f465419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f46537fe640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 0x7f465419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f46537fe640 1 -- 172.21.15.33:0/1785210228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465419f7b0 con 0x7f4654100680 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.757+0000 7f4652ffd640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 0x7f465419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.758+0000 7f46537fe640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 0x7f4654069440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f464400e9c0 tx=0x7f464400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.758+0000 7f4650ff9640 1 -- 172.21.15.33:0/1785210228 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464400ccc0 con 0x7f4654100680 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.758+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465419fa40 con 0x7f4654100680 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.758+0000 7f4650ff9640 1 -- 172.21.15.33:0/1785210228 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46440035f0 con 0x7f4654100680 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.758+0000 7f4650ff9640 1 -- 172.21.15.33:0/1785210228 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464400ce60 con 0x7f4654100680 2024-04-24T23:28:55.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.758+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465419ff60 con 0x7f4654100680 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.760+0000 7f4650ff9640 1 -- 172.21.15.33:0/1785210228 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f46440037a0 con 0x7f4654100680 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 Processor -- start 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 -- start start 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e36889c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688a4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3688006b0 con 0x55e36889c240 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3687a8880 con 0x55e36889cc30 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f465ca0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e3687fc670 con 0x55e3688a1da0 2024-04-24T23:28:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e36889c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e36889c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56152/0 (socket says 172.21.15.33:56152) 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 -- 172.21.15.33:0/1935108377 learned_addr learned my addr 172.21.15.33:0/1935108377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f46537fe640 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4652ffd640 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4650ff9640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f463c0a2b90 0x7f463c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 -- 172.21.15.33:0/1935108377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 msgr2=0x55e3688a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 -- 172.21.15.33:0/1935108377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 msgr2=0x55e3688a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.761+0000 7f4653fff640 1 -- 172.21.15.33:0/1935108377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3685de7d0 con 0x55e36889c240 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4653fff640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f463c0a2b90 0x7f463c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4650ff9640 1 -- 172.21.15.33:0/1785210228 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46440cf6d0 con 0x7f4654100680 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4653fff640 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e36889c660 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f464c011570 tx=0x7f464c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f41b69d89a4382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4639ffb640 1 -- 172.21.15.33:0/1935108377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c017070 con 0x55e36889c240 2024-04-24T23:28:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4639ffb640 1 -- 172.21.15.33:0/1935108377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c00fd20 con 0x55e36889c240 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4653fff640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f463c0a2b90 0x7f463c0a5050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f464c021000 tx=0x7f464c022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.762+0000 7f4639ffb640 1 -- 172.21.15.33:0/1935108377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c026b40 con 0x55e36889c240 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 -- 172.21.15.33:0/1935108377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 msgr2=0x55e36889c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e36889c660 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f464c011570 tx=0x7f464c011a30 comp rx=0 tx=0).stop 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 -- 172.21.15.33:0/1935108377 shutdown_connections 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 --2- 172.21.15.33:0/1935108377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e36889c660 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 -- 172.21.15.33:0/1935108377 >> 172.21.15.33:0/1935108377 conn(0x55e3688978b0 msgr2=0x55e368897cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 -- 172.21.15.33:0/1935108377 shutdown_connections 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 -- 172.21.15.33:0/1935108377 wait complete. 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.763+0000 7f465ca0b740 1 -- start start 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 Processor -- start 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e36889c240 0x55e3688aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3688a1da0 0x55e3688b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3688a4cf0 con 0x55e3688a1da0 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3687bbf50 con 0x55e36889cc30 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e3688a4e70 con 0x55e36889c240 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f465ca0b740 10 client.0 _ll_register_callbacks cb 0x55e3687f7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4652ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3688a1da0 0x55e3688b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4652ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688aadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57086/0 (socket says 172.21.15.33:57086) 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4652ffd640 1 -- 172.21.15.33:0/374053391 learned_addr learned my addr 172.21.15.33:0/374053391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f46537fe640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e36889c240 0x55e3688aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4653fff640 1 -- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e36889c240 msgr2=0x55e3688aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4653fff640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e36889c240 0x55e3688aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4653fff640 1 -- 172.21.15.33:0/374053391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 msgr2=0x55e3688aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4653fff640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4653fff640 1 -- 172.21.15.33:0/374053391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3685143f0 con 0x55e3688a1da0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f4652ffd640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.764+0000 7f46537fe640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e36889c240 0x55e3688aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4653fff640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3688a1da0 0x55e3688b1df0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f464c00ffd0 tx=0x7f464c004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c879686e13f810 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4632ffd640 1 -- 172.21.15.33:0/374053391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c017030 con 0x55e3688a1da0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4632ffd640 1 -- 172.21.15.33:0/374053391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f464c00f640 con 0x55e3688a1da0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464c00fd20 con 0x55e3688a1da0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4632ffd640 1 -- 172.21.15.33:0/374053391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f464c03c050 con 0x55e3688a1da0 2024-04-24T23:28:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 10 client.19017 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e3684d40f0 con 0x55e3688a1da0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4632ffd640 1 -- 172.21.15.33:0/374053391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c00fd20 con 0x55e3688a1da0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f4632ffd640 1 -- 172.21.15.33:0/374053391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f464c037030 con 0x55e3688a1da0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 10 client.19017 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 20 client.19017 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e3688b64c0 0x55e3688b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 4 client.19017 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.765+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e36880dbf0 con 0x55e3688b64c0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.766+0000 7f46537fe640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e3688b64c0 0x55e3688b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.766+0000 7f4632ffd640 10 client.19017 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.766+0000 7f46537fe640 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e3688b64c0 0x55e3688b8910 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148b37932daea037 server_cookie=10c94ea65d704280 in_seq=0 out_seq=0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f4632ffd640 1 -- 172.21.15.33:0/374053391 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e36880dbf0 con 0x55e3688b64c0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f4632ffd640 10 client.19017 handle_command_reply: tid=0 2024-04-24T23:28:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 client.19017 shutdown 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3688a1da0 msgr2=0x55e3688b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3688a1da0 0x55e3688b1df0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f464c00ffd0 tx=0x7f464c004c30 comp rx=0 tx=0).stop 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 shutdown_connections 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e3688b64c0 0x55e3688b8910 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3688a1da0 0x55e3688b1df0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 --2- 172.21.15.33:0/374053391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e36889c240 0x55e3688aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f4632ffd640 0 client.19017 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 >> 172.21.15.33:0/374053391 conn(0x55e3688978b0 msgr2=0x55e368896af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 shutdown_connections 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 1 -- 172.21.15.33:0/374053391 wait complete. 2024-04-24T23:28:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.767+0000 7f465ca0b740 20 client.19017 trim_cache size 0 max 16384 2024-04-24T23:28:55.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 Processor -- start 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 -- start start 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46541aaad0 con 0x55e36889c240 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46541a83a0 con 0x55e36889cc30 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f465ca0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46541a61f0 con 0x55e3688a1da0 2024-04-24T23:28:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688a5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56172/0 (socket says 172.21.15.33:56172) 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 -- 172.21.15.33:0/2677210172 learned_addr learned my addr 172.21.15.33:0/2677210172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f4652ffd640 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f4653fff640 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 -- 172.21.15.33:0/2677210172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 msgr2=0x55e3688b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 -- 172.21.15.33:0/2677210172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 msgr2=0x55e3688a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.786+0000 7f46537fe640 1 -- 172.21.15.33:0/2677210172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e368891f00 con 0x55e36889c240 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.787+0000 7f46537fe640 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688a5890 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4644098570 tx=0x7f4644097f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0b94057c18f0be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.787+0000 7f46397fa640 1 -- 172.21.15.33:0/2677210172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464409bba0 con 0x55e36889c240 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.787+0000 7f46397fa640 1 -- 172.21.15.33:0/2677210172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464409ff50 con 0x55e36889c240 2024-04-24T23:28:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.787+0000 7f46397fa640 1 -- 172.21.15.33:0/2677210172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464409be00 con 0x55e36889c240 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.787+0000 7f465ca0b740 1 -- 172.21.15.33:0/2677210172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 msgr2=0x55e3688a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.787+0000 7f465ca0b740 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688a5890 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4644098570 tx=0x7f4644097f70 comp rx=0 tx=0).stop 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- 172.21.15.33:0/2677210172 shutdown_connections 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 --2- 172.21.15.33:0/2677210172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688a5890 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- 172.21.15.33:0/2677210172 >> 172.21.15.33:0/2677210172 conn(0x55e3688978b0 msgr2=0x55e3688aa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- 172.21.15.33:0/2677210172 shutdown_connections 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- 172.21.15.33:0/2677210172 wait complete. 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- start start 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 Processor -- start 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46541a6860 con 0x55e36889c240 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46541a7cd0 con 0x55e36889cc30 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46541a6f30 con 0x55e3688a1da0 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f465ca0b740 10 client.0 _ll_register_callbacks cb 0x7f4644097ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f46537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f46537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56180/0 (socket says 172.21.15.33:56180) 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f46537fe640 1 -- 172.21.15.33:0/4253147255 learned_addr learned my addr 172.21.15.33:0/4253147255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f4652ffd640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.788+0000 7f4653fff640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f46537fe640 1 -- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 msgr2=0x55e3688aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f46537fe640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f46537fe640 1 -- 172.21.15.33:0/4253147255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 msgr2=0x55e3688ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f46537fe640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f46537fe640 1 -- 172.21.15.33:0/4253147255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3688aef90 con 0x55e36889c240 2024-04-24T23:28:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f46537fe640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688b19f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f464400f010 tx=0x7f46440ccde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05dbf28f3edfef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f4632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f4632ffd640 1 -- 172.21.15.33:0/4253147255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440b8070 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f4632ffd640 1 -- 172.21.15.33:0/4253147255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4644003b50 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464409ff50 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f4632ffd640 1 -- 172.21.15.33:0/4253147255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4644003d50 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f465ca0b740 10 client.19023 fetch_fsmap learned FSMap version 33 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e36891ba40 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.789+0000 7f4632ffd640 1 -- 172.21.15.33:0/4253147255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464409ff50 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f4632ffd640 1 -- 172.21.15.33:0/4253147255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f46440ba2c0 con 0x55e36889c240 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f465ca0b740 10 client.19023 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f465ca0b740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f465ca0b740 20 client.19023 populate_metadata read hostname 'smithi033' 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f465ca0b740 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e36891c2e0 0x55e36891e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f465ca0b740 4 client.19023 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e368807f40 con 0x55e36891c2e0 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f4652ffd640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e36891c2e0 0x55e36891e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f4652ffd640 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e36891c2e0 0x55e36891e700 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3ed7c3a8951f43 server_cookie=a18bf100cec99e8a in_seq=0 out_seq=0 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.790+0000 7f4632ffd640 10 client.19023 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.791+0000 7f4632ffd640 1 -- 172.21.15.33:0/4253147255 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55e368807f40 con 0x55e36891c2e0 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.791+0000 7f4632ffd640 10 client.19023 handle_command_reply: tid=0 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.791+0000 7f465ca0b740 1 client.19023 shutdown 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 msgr2=0x55e3688b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688b19f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f464400f010 tx=0x7f46440ccde0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 shutdown_connections 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e36891c2e0 0x55e36891e700 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3688a1da0 0x55e3688aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e36889cc30 0x55e3688ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 --2- 172.21.15.33:0/4253147255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e36889c240 0x55e3688b19f0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4632ffd640 0 client.19023 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:28:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 >> 172.21.15.33:0/4253147255 conn(0x55e3688978b0 msgr2=0x55e36889a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 shutdown_connections 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 1 -- 172.21.15.33:0/4253147255 wait complete. 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f465ca0b740 20 client.19023 trim_cache size 0 max 16384 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "3c18bac3-299a-4587-a895-d288f9c79731": { 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "3c18bac3-299a-4587-a895-d288f9c79731", 2024-04-24T23:28:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:28:55.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:28:55.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:28:55.800 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:28:55.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f463c0a2b90 msgr2=0x7f463c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f463c0a2b90 0x7f463c0a5050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f464c021000 tx=0x7f464c022000 comp rx=0 tx=0).stop 2024-04-24T23:28:55.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 msgr2=0x7f4654069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 0x7f4654069440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f464400e9c0 tx=0x7f464400edc0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 shutdown_connections 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f463c0a2b90 0x7f463c0a5050 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4654102230 0x7f465419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4654101870 0x7f465419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 --2- 172.21.15.33:0/1785210228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4654100680 0x7f4654069440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.792+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 >> 172.21.15.33:0/1785210228 conn(0x7f46540fbbf0 msgr2=0x7f4654104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.793+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 shutdown_connections 2024-04-24T23:28:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:28:55.793+0000 7f4659d09640 1 -- 172.21.15.33:0/1785210228 wait complete. 2024-04-24T23:28:55.804 INFO:tasks.cephfs.filesystem:scrub status for tag:3c18bac3-299a-4587-a895-d288f9c79731 - {'path': '/', 'tag': '3c18bac3-299a-4587-a895-d288f9c79731', 'options': 'recursive,force'} 2024-04-24T23:28:55.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 79935 22.56 MB/sec execute 60 sec latency 131.736 ms 2024-04-24T23:28:56.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 80365 22.64 MB/sec execute 61 sec latency 220.418 ms 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:57 smithi038 ceph-mon[32450]: pgmap v859: 129 pgs: 129 active+clean; 527 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 396 op/s 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:28:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:57 smithi117 ceph-mon[32529]: pgmap v859: 129 pgs: 129 active+clean; 527 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 396 op/s 2024-04-24T23:28:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:28:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:28:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:57 smithi033 ceph-mon[29201]: pgmap v859: 129 pgs: 129 active+clean; 527 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 396 op/s 2024-04-24T23:28:57.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 80615 22.46 MB/sec execute 62 sec latency 236.614 ms 2024-04-24T23:28:58.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:58 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:58.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:58 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:58.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:58 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:28:58.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 81043 22.34 MB/sec execute 63 sec latency 87.799 ms 2024-04-24T23:28:59.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:28:59 smithi117 ceph-mon[32529]: pgmap v860: 129 pgs: 129 active+clean; 531 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.7 MiB/s wr, 407 op/s 2024-04-24T23:28:59.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:28:59 smithi033 ceph-mon[29201]: pgmap v860: 129 pgs: 129 active+clean; 531 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.7 MiB/s wr, 407 op/s 2024-04-24T23:28:59.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:28:59 smithi038 ceph-mon[32450]: pgmap v860: 129 pgs: 129 active+clean; 531 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.7 MiB/s wr, 407 op/s 2024-04-24T23:28:59.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 81671 22.49 MB/sec execute 64 sec latency 66.220 ms 2024-04-24T23:29:00.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 82453 22.91 MB/sec execute 65 sec latency 54.920 ms 2024-04-24T23:29:01.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:01 smithi033 ceph-mon[29201]: pgmap v861: 129 pgs: 129 active+clean; 533 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.0 MiB/s wr, 386 op/s 2024-04-24T23:29:01.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:01 smithi038 ceph-mon[32450]: pgmap v861: 129 pgs: 129 active+clean; 533 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.0 MiB/s wr, 386 op/s 2024-04-24T23:29:01.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:01 smithi117 ceph-mon[32529]: pgmap v861: 129 pgs: 129 active+clean; 533 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.0 MiB/s wr, 386 op/s 2024-04-24T23:29:01.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 82778 22.89 MB/sec execute 66 sec latency 104.135 ms 2024-04-24T23:29:02.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 82998 22.71 MB/sec execute 67 sec latency 369.761 ms 2024-04-24T23:29:03.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:03 smithi033 ceph-mon[29201]: pgmap v862: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.7 MiB/s wr, 469 op/s 2024-04-24T23:29:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:03 smithi038 ceph-mon[32450]: pgmap v862: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.7 MiB/s wr, 469 op/s 2024-04-24T23:29:03.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:03 smithi117 ceph-mon[32529]: pgmap v862: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.7 MiB/s wr, 469 op/s 2024-04-24T23:29:03.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 83314 22.61 MB/sec execute 68 sec latency 101.151 ms 2024-04-24T23:29:04.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 83751 22.66 MB/sec execute 69 sec latency 114.309 ms 2024-04-24T23:29:05.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:05 smithi033 ceph-mon[29201]: pgmap v863: 129 pgs: 129 active+clean; 514 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-04-24T23:29:05.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:05 smithi038 ceph-mon[32450]: pgmap v863: 129 pgs: 129 active+clean; 514 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-04-24T23:29:05.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:05 smithi117 ceph-mon[32529]: pgmap v863: 129 pgs: 129 active+clean; 514 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-04-24T23:29:05.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 84289 22.68 MB/sec execute 70 sec latency 139.677 ms 2024-04-24T23:29:06.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 84864 22.81 MB/sec execute 71 sec latency 77.983 ms 2024-04-24T23:29:07.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:07 smithi033 ceph-mon[29201]: pgmap v864: 129 pgs: 129 active+clean; 516 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 422 op/s 2024-04-24T23:29:07.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:07 smithi038 ceph-mon[32450]: pgmap v864: 129 pgs: 129 active+clean; 516 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 422 op/s 2024-04-24T23:29:07.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:07 smithi117 ceph-mon[32529]: pgmap v864: 129 pgs: 129 active+clean; 516 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 422 op/s 2024-04-24T23:29:07.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 85156 22.62 MB/sec execute 72 sec latency 442.064 ms 2024-04-24T23:29:08.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:08 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:29:08.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:08 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:29:08.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:08 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:29:08.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:08 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:29:08.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:08 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:29:08.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:08 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:29:08.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 85327 22.41 MB/sec execute 73 sec latency 536.315 ms 2024-04-24T23:29:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:09 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:29:09.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:09 smithi033 ceph-mon[29201]: pgmap v865: 129 pgs: 129 active+clean; 523 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.6 MiB/s wr, 422 op/s 2024-04-24T23:29:09.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:09 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:29:09.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:09 smithi038 ceph-mon[32450]: pgmap v865: 129 pgs: 129 active+clean; 523 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.6 MiB/s wr, 422 op/s 2024-04-24T23:29:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:09 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:29:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:09 smithi117 ceph-mon[32529]: pgmap v865: 129 pgs: 129 active+clean; 523 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.6 MiB/s wr, 422 op/s 2024-04-24T23:29:09.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 85956 22.65 MB/sec execute 74 sec latency 56.615 ms 2024-04-24T23:29:10.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 86393 22.72 MB/sec execute 75 sec latency 120.015 ms 2024-04-24T23:29:11.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:11 smithi038 ceph-mon[32450]: pgmap v866: 129 pgs: 129 active+clean; 525 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 375 op/s 2024-04-24T23:29:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:11 smithi117 ceph-mon[32529]: pgmap v866: 129 pgs: 129 active+clean; 525 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 375 op/s 2024-04-24T23:29:11.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:11 smithi033 ceph-mon[29201]: pgmap v866: 129 pgs: 129 active+clean; 525 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 375 op/s 2024-04-24T23:29:11.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 86834 22.70 MB/sec execute 76 sec latency 105.989 ms 2024-04-24T23:29:12.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 87108 22.58 MB/sec execute 77 sec latency 422.456 ms 2024-04-24T23:29:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:13 smithi038 ceph-mon[32450]: pgmap v867: 129 pgs: 129 active+clean; 526 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.7 MiB/s wr, 513 op/s 2024-04-24T23:29:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:13 smithi117 ceph-mon[32529]: pgmap v867: 129 pgs: 129 active+clean; 526 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.7 MiB/s wr, 513 op/s 2024-04-24T23:29:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:13 smithi033 ceph-mon[29201]: pgmap v867: 129 pgs: 129 active+clean; 526 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.7 MiB/s wr, 513 op/s 2024-04-24T23:29:13.886 INFO:tasks.workunit.client.0.smithi033.stdout: 10 87561 22.56 MB/sec execute 78 sec latency 116.326 ms 2024-04-24T23:29:14.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:14 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:14 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:14 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:14.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 87958 22.64 MB/sec execute 79 sec latency 205.308 ms 2024-04-24T23:29:15.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: pgmap v868: 129 pgs: 129 active+clean; 530 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 527 op/s 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: Unable to set osd_memory_target on smithi038 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:29:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:29:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: pgmap v868: 129 pgs: 129 active+clean; 530 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 527 op/s 2024-04-24T23:29:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-24T23:29:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T23:29:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: Unable to set osd_memory_target on smithi038 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:29:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:29:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: pgmap v868: 129 pgs: 129 active+clean; 530 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 527 op/s 2024-04-24T23:29:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-24T23:29:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T23:29:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:29:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-04-24T23:29:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: Unable to set osd_memory_target on smithi038 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-24T23:29:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:29:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:29:15.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:29:15.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 88247 22.55 MB/sec execute 80 sec latency 83.682 ms 2024-04-24T23:29:16.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 88650 22.53 MB/sec execute 81 sec latency 83.876 ms 2024-04-24T23:29:17.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:17 smithi038 ceph-mon[32450]: pgmap v869: 129 pgs: 129 active+clean; 533 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 568 op/s 2024-04-24T23:29:17.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:17 smithi117 ceph-mon[32529]: pgmap v869: 129 pgs: 129 active+clean; 533 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 568 op/s 2024-04-24T23:29:17.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:17 smithi033 ceph-mon[29201]: pgmap v869: 129 pgs: 129 active+clean; 533 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 568 op/s 2024-04-24T23:29:17.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 88979 22.40 MB/sec execute 82 sec latency 342.596 ms 2024-04-24T23:29:18.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 89428 22.37 MB/sec execute 83 sec latency 127.061 ms 2024-04-24T23:29:19.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:19 smithi038 ceph-mon[32450]: pgmap v870: 129 pgs: 129 active+clean; 533 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 542 op/s 2024-04-24T23:29:19.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:19 smithi117 ceph-mon[32529]: pgmap v870: 129 pgs: 129 active+clean; 533 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 542 op/s 2024-04-24T23:29:19.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:19 smithi033 ceph-mon[29201]: pgmap v870: 129 pgs: 129 active+clean; 533 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 542 op/s 2024-04-24T23:29:19.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 89959 22.53 MB/sec execute 84 sec latency 100.691 ms 2024-04-24T23:29:20.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 90550 22.66 MB/sec execute 85 sec latency 139.415 ms 2024-04-24T23:29:21.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:21 smithi038 ceph-mon[32450]: pgmap v871: 129 pgs: 129 active+clean; 534 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-04-24T23:29:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:21 smithi117 ceph-mon[32529]: pgmap v871: 129 pgs: 129 active+clean; 534 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-04-24T23:29:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:21 smithi033 ceph-mon[29201]: pgmap v871: 129 pgs: 129 active+clean; 534 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-04-24T23:29:21.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 91017 22.70 MB/sec execute 86 sec latency 71.544 ms 2024-04-24T23:29:22.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 91349 22.66 MB/sec execute 87 sec latency 349.079 ms 2024-04-24T23:29:23.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:23 smithi038 ceph-mon[32450]: pgmap v872: 129 pgs: 129 active+clean; 528 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 576 op/s 2024-04-24T23:29:23.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:23 smithi117 ceph-mon[32529]: pgmap v872: 129 pgs: 129 active+clean; 528 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 576 op/s 2024-04-24T23:29:23.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:23 smithi033 ceph-mon[29201]: pgmap v872: 129 pgs: 129 active+clean; 528 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 576 op/s 2024-04-24T23:29:23.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 91624 22.59 MB/sec execute 88 sec latency 124.904 ms 2024-04-24T23:29:24.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 91910 22.48 MB/sec execute 89 sec latency 73.272 ms 2024-04-24T23:29:25.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:25 smithi038 ceph-mon[32450]: pgmap v873: 129 pgs: 129 active+clean; 518 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-04-24T23:29:25.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:25 smithi117 ceph-mon[32529]: pgmap v873: 129 pgs: 129 active+clean; 518 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-04-24T23:29:25.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:25 smithi033 ceph-mon[29201]: pgmap v873: 129 pgs: 129 active+clean; 518 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-04-24T23:29:25.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:29:25.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:29:25.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 92287 22.41 MB/sec execute 90 sec latency 82.568 ms 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 -- 172.21.15.33:0/2885835913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 msgr2=0x7f3654100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2885835913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654100aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3648009860 tx=0x7f364802a340 comp rx=0 tx=0).stop 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 -- 172.21.15.33:0/2885835913 shutdown_connections 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2885835913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3654102230 0x7f3654108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2885835913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f3654101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2885835913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654100aa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 -- 172.21.15.33:0/2885835913 >> 172.21.15.33:0/2885835913 conn(0x7f36540fbbf0 msgr2=0x7f36540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 -- 172.21.15.33:0/2885835913 shutdown_connections 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.932+0000 7f3659cbc640 1 -- 172.21.15.33:0/2885835913 wait complete. 2024-04-24T23:29:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.933+0000 7f3659cbc640 1 Processor -- start 2024-04-24T23:29:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.933+0000 7f3659cbc640 1 -- start start 2024-04-24T23:29:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3659cbc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3659cbc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f365419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3659cbc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3654102230 0x7f36541a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3659cbc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f365410a750 con 0x7f3654101870 2024-04-24T23:29:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3659cbc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f365410a5d0 con 0x7f3654102230 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3659cbc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f365410a8d0 con 0x7f3654100680 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f365419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f365419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45080/0 (socket says 172.21.15.33:45080) 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56822/0 (socket says 172.21.15.33:56822) 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3652ffd640 1 -- 172.21.15.33:0/2814773278 learned_addr learned my addr 172.21.15.33:0/2814773278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3653fff640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3654102230 0x7f36541a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 -- 172.21.15.33:0/2814773278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3654102230 msgr2=0x7f36541a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3654102230 0x7f36541a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 -- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 msgr2=0x7f365419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f365419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 -- 172.21.15.33:0/2814773278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36541a1950 con 0x7f3654100680 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f365419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.934+0000 7f36537fe640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654199d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f364802a860 tx=0x7f3648033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.935+0000 7f3650ff9640 1 -- 172.21.15.33:0/2814773278 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648037040 con 0x7f3654100680 2024-04-24T23:29:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.935+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36541a1be0 con 0x7f3654100680 2024-04-24T23:29:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.935+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541a20f0 con 0x7f3654100680 2024-04-24T23:29:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.935+0000 7f3650ff9640 1 -- 172.21.15.33:0/2814773278 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3648030850 con 0x7f3654100680 2024-04-24T23:29:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.935+0000 7f3650ff9640 1 -- 172.21.15.33:0/2814773278 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36480046c0 con 0x7f3654100680 2024-04-24T23:29:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f3650ff9640 1 -- 172.21.15.33:0/2814773278 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3648004860 con 0x7f3654100680 2024-04-24T23:29:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 Processor -- start 2024-04-24T23:29:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 -- start start 2024-04-24T23:29:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dcfd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd02730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f3650ff9640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f363c0a2b90 0x7f363c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd05230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a59dc616b0 con 0x55a59dcfd240 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a59dc09880 con 0x55a59dcfdc30 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.938+0000 7f365c9c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a59dc5d670 con 0x55a59dd02da0 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dcfd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f36537fe640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f363c0a2b90 0x7f363c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dcfd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45086/0 (socket says 172.21.15.33:45086) 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 -- 172.21.15.33:0/974846342 learned_addr learned my addr 172.21.15.33:0/974846342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3652ffd640 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd02730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3650ff9640 1 -- 172.21.15.33:0/2814773278 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f36480ef1c0 con 0x7f3654100680 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd05230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 -- 172.21.15.33:0/974846342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 msgr2=0x55a59dd05230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd05230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 -- 172.21.15.33:0/974846342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 msgr2=0x55a59dd02730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd02730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 -- 172.21.15.33:0/974846342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59da3f7d0 con 0x55a59dcfd240 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f36537fe640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f363c0a2b90 0x7f363c0a5050 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f36480b7fd0 tx=0x7f36480b7200 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.939+0000 7f3653fff640 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dcfd660 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f364c009b80 tx=0x7f364c009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509759833e8f83c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.940+0000 7f3639ffb640 1 -- 172.21.15.33:0/974846342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c01a070 con 0x55a59dcfd240 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.940+0000 7f3639ffb640 1 -- 172.21.15.33:0/974846342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c003e50 con 0x55a59dcfd240 2024-04-24T23:29:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.940+0000 7f3639ffb640 1 -- 172.21.15.33:0/974846342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0042b0 con 0x55a59dcfd240 2024-04-24T23:29:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 -- 172.21.15.33:0/974846342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 msgr2=0x55a59dcfd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dcfd660 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f364c009b80 tx=0x7f364c009cf0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 -- 172.21.15.33:0/974846342 shutdown_connections 2024-04-24T23:29:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd05230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 --2- 172.21.15.33:0/974846342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dcfd660 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 -- 172.21.15.33:0/974846342 >> 172.21.15.33:0/974846342 conn(0x55a59dcf88b0 msgr2=0x55a59dcf8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 -- 172.21.15.33:0/974846342 shutdown_connections 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 -- 172.21.15.33:0/974846342 wait complete. 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.941+0000 7f365c9c5740 1 -- start start 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 Processor -- start 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a59dd05d50 con 0x55a59dcfd240 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a59dc1cf50 con 0x55a59dd02da0 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a59dd05ed0 con 0x55a59dcfdc30 2024-04-24T23:29:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f3653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f3653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45092/0 (socket says 172.21.15.33:45092) 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f3653fff640 1 -- 172.21.15.33:0/2154956301 learned_addr learned my addr 172.21.15.33:0/2154956301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f365c9c5740 10 client.0 _ll_register_callbacks cb 0x55a59dc58000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.942+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3653fff640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3652ffd640 1 -- 172.21.15.33:0/2154956301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dd02da0 msgr2=0x55a59dd0ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3652ffd640 1 -- 172.21.15.33:0/2154956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 msgr2=0x55a59dd076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3652ffd640 1 -- 172.21.15.33:0/2154956301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59d9753f0 con 0x55a59dcfdc30 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3653fff640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55a59dcfe030 tx=0x7f364400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9562f359d123cacb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:29:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3632ffd640 1 -- 172.21.15.33:0/2154956301 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644004050 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f3632ffd640 1 -- 172.21.15.33:0/2154956301 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f36440041f0 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.943+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c003e50 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f3632ffd640 1 -- 172.21.15.33:0/2154956301 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3644025050 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 10 client.28462 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a59d9350f0 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f3632ffd640 1 -- 172.21.15.33:0/2154956301 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c003e50 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f3632ffd640 1 -- 172.21.15.33:0/2154956301 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f364401a5c0 con 0x55a59dcfdc30 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 10 client.28462 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 10 client.28462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 20 client.28462 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd13a70 0x55a59dd15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 4 client.28462 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a59dc6ebf0 con 0x55a59dd13a70 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.944+0000 7f3653fff640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd13a70 0x55a59dd15ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.945+0000 7f3653fff640 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd13a70 0x55a59dd15ef0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d400bb43a832c85 server_cookie=a9d697c7e7b5f337 in_seq=0 out_seq=0 2024-04-24T23:29:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.945+0000 7f3632ffd640 10 client.28462 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.945+0000 7f3632ffd640 1 -- 172.21.15.33:0/2154956301 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a59dc6ebf0 con 0x55a59dd13a70 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.945+0000 7f3632ffd640 10 client.28462 handle_command_reply: tid=0 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.945+0000 7f365c9c5740 1 client.28462 shutdown 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dcfdc30 msgr2=0x55a59dd07c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55a59dcfe030 tx=0x7f364400f790 comp rx=0 tx=0).stop 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 shutdown_connections 2024-04-24T23:29:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd13a70 0x55a59dd15ef0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2154956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 >> 172.21.15.33:0/2154956301 conn(0x55a59dcf88b0 msgr2=0x55a59dcf7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f3632ffd640 0 client.28462 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 shutdown_connections 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 1 -- 172.21.15.33:0/2154956301 wait complete. 2024-04-24T23:29:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.946+0000 7f365c9c5740 20 client.28462 trim_cache size 0 max 16384 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.969+0000 7f365c9c5740 1 Processor -- start 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.969+0000 7f365c9c5740 1 -- start start 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f365c9c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3640012440 con 0x55a59dcfd240 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f365c9c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f364000fd10 con 0x55a59dcfdc30 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f365c9c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f364000db60 con 0x55a59dd02da0 2024-04-24T23:29:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45098/0 (socket says 172.21.15.33:45098) 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 -- 172.21.15.33:0/3339009576 learned_addr learned my addr 172.21.15.33:0/3339009576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3652ffd640 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 -- 172.21.15.33:0/3339009576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 msgr2=0x55a59dd0ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 -- 172.21.15.33:0/3339009576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 msgr2=0x55a59dd07c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 -- 172.21.15.33:0/3339009576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59dcea090 con 0x55a59dcfd240 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3652ffd640 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:25.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.970+0000 7f3653fff640 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f364c009ae0 tx=0x7f364c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc246145a06773b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.971+0000 7f36397fa640 1 -- 172.21.15.33:0/3339009576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c01a040 con 0x55a59dcfd240 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.971+0000 7f36397fa640 1 -- 172.21.15.33:0/3339009576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c003e50 con 0x55a59dcfd240 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.971+0000 7f36397fa640 1 -- 172.21.15.33:0/3339009576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c02e700 con 0x55a59dcfd240 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 -- 172.21.15.33:0/3339009576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 msgr2=0x55a59dd076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f364c009ae0 tx=0x7f364c009b10 comp rx=0 tx=0).stop 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 -- 172.21.15.33:0/3339009576 shutdown_connections 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd0ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfdc30 0x55a59dd07c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 --2- 172.21.15.33:0/3339009576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfd240 0x55a59dd076f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 -- 172.21.15.33:0/3339009576 >> 172.21.15.33:0/3339009576 conn(0x55a59dcf88b0 msgr2=0x55a59dd06300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 -- 172.21.15.33:0/3339009576 shutdown_connections 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.972+0000 7f365c9c5740 1 -- 172.21.15.33:0/3339009576 wait complete. 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 -- start start 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 Processor -- start 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfd240 0x55a59dd0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 0x55a59dd0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd78060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f364000e1d0 con 0x55a59dcfdc30 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f364000f640 con 0x55a59dcfd240 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f364000e8a0 con 0x55a59dd02da0 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f365c9c5740 10 client.0 _ll_register_callbacks cb 0x7f3644009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.973+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 0x55a59dd0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 0x55a59dd0be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45106/0 (socket says 172.21.15.33:45106) 2024-04-24T23:29:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 -- 172.21.15.33:0/2521409389 learned_addr learned my addr 172.21.15.33:0/2521409389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3653fff640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfd240 0x55a59dd0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3653fff640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd78060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 -- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 msgr2=0x55a59dd78060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd78060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 -- 172.21.15.33:0/2521409389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfd240 msgr2=0x55a59dd0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfd240 0x55a59dd0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 -- 172.21.15.33:0/2521409389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59dd0ee00 con 0x55a59dcfdc30 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3652ffd640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 0x55a59dd0be40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3644009340 tx=0x7f364401a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4772653fb0e212bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3632ffd640 1 -- 172.21.15.33:0/2521409389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364402d070 con 0x55a59dcfdc30 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f3632ffd640 1 -- 172.21.15.33:0/2521409389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3644005070 con 0x55a59dcfdc30 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.974+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c003e50 con 0x55a59dcfdc30 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f3632ffd640 1 -- 172.21.15.33:0/2521409389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3644039050 con 0x55a59dcfdc30 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 10 client.19041 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a59dd79530 con 0x55a59dcfdc30 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f3632ffd640 1 -- 172.21.15.33:0/2521409389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c003e50 con 0x55a59dcfdc30 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f3632ffd640 1 -- 172.21.15.33:0/2521409389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3644003ca0 con 0x55a59dcfdc30 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 10 client.19041 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 20 client.19041 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd79f00 0x55a59dd7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 4 client.19041 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a59dd056b0 con 0x55a59dd79f00 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.975+0000 7f3653fff640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd79f00 0x55a59dd7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.976+0000 7f3632ffd640 10 client.19041 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.976+0000 7f3653fff640 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd79f00 0x55a59dd7c320 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db038c629fe0aec3 server_cookie=3e03e1dee9b52a5f in_seq=0 out_seq=0 2024-04-24T23:29:25.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f3632ffd640 1 -- 172.21.15.33:0/2521409389 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a59dd056b0 con 0x55a59dd79f00 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f3632ffd640 10 client.19041 handle_command_reply: tid=0 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 client.19041 shutdown 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 msgr2=0x55a59dd0be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 0x55a59dd0be40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3644009340 tx=0x7f364401a470 comp rx=0 tx=0).stop 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 shutdown_connections 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a59dd79f00 0x55a59dd7c320 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a59dd02da0 0x55a59dd78060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a59dcfdc30 0x55a59dd0be40 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f3632ffd640 0 client.19041 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 --2- 172.21.15.33:0/2521409389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a59dcfd240 0x55a59dd0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 >> 172.21.15.33:0/2521409389 conn(0x55a59dcf88b0 msgr2=0x55a59dd076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.978+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 shutdown_connections 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f365c9c5740 1 -- 172.21.15.33:0/2521409389 wait complete. 2024-04-24T23:29:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f365c9c5740 20 client.19041 trim_cache size 0 max 16384 2024-04-24T23:29:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:29:25.984 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:29:25.984 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:29:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f363c0a2b90 msgr2=0x7f363c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f363c0a2b90 0x7f363c0a5050 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f36480b7fd0 tx=0x7f36480b7200 comp rx=0 tx=0).stop 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 msgr2=0x7f3654199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654199d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f364802a860 tx=0x7f3648033d90 comp rx=0 tx=0).stop 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 shutdown_connections 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f363c0a2b90 0x7f363c0a5050 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3654102230 0x7f36541a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654101870 0x7f365419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 --2- 172.21.15.33:0/2814773278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3654100680 0x7f3654199d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 >> 172.21.15.33:0/2814773278 conn(0x7f36540fbbf0 msgr2=0x7f36540fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 shutdown_connections 2024-04-24T23:29:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:25.979+0000 7f3659cbc640 1 -- 172.21.15.33:0/2814773278 wait complete. 2024-04-24T23:29:25.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:29:25.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 -- 172.21.15.33:0/39095510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c102200 msgr2=0x7fb26c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 --2- 172.21.15.33:0/39095510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c102200 0x7fb26c1045f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb268009900 tx=0x7fb26802a400 comp rx=0 tx=0).stop 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 -- 172.21.15.33:0/39095510 shutdown_connections 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 --2- 172.21.15.33:0/39095510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb26c107460 0x7fb26c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 --2- 172.21.15.33:0/39095510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb26c104b30 0x7fb26c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 --2- 172.21.15.33:0/39095510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c102200 0x7fb26c1045f0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.117+0000 7fb272ab3640 1 -- 172.21.15.33:0/39095510 >> 172.21.15.33:0/39095510 conn(0x7fb26c0fbbf0 msgr2=0x7fb26c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.118+0000 7fb272ab3640 1 -- 172.21.15.33:0/39095510 shutdown_connections 2024-04-24T23:29:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.118+0000 7fb272ab3640 1 -- 172.21.15.33:0/39095510 wait complete. 2024-04-24T23:29:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.119+0000 7fb272ab3640 1 Processor -- start 2024-04-24T23:29:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.120+0000 7fb272ab3640 1 -- start start 2024-04-24T23:29:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb272ab3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb26c102200 0x7fb26c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb272ab3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb26c104b30 0x7fb26c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb272ab3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 0x7fb26c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb272ab3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10d450 con 0x7fb26c107460 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb272ab3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10d2d0 con 0x7fb26c102200 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb272ab3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10d5d0 con 0x7fb26c104b30 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb271050640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 0x7fb26c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb271050640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 0x7fb26c1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45130/0 (socket says 172.21.15.33:45130) 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb271050640 1 -- 172.21.15.33:0/1448099385 learned_addr learned my addr 172.21.15.33:0/1448099385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb263fff640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb26c104b30 0x7fb26c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.121+0000 7fb27084f640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb26c102200 0x7fb26c101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb271050640 1 -- 172.21.15.33:0/1448099385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb26c104b30 msgr2=0x7fb26c1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb271050640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb26c104b30 0x7fb26c1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb271050640 1 -- 172.21.15.33:0/1448099385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb26c102200 msgr2=0x7fb26c101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb271050640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb26c102200 0x7fb26c101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb271050640 1 -- 172.21.15.33:0/1448099385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c100c30 con 0x7fb26c107460 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb271050640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 0x7fb26c1006f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb26400eaa0 tx=0x7fb26400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26400cdb0 con 0x7fb26c107460 2024-04-24T23:29:26.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.122+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c100f20 con 0x7fb26c107460 2024-04-24T23:29:26.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.123+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb264005020 con 0x7fb26c107460 2024-04-24T23:29:26.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.123+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2640116b0 con 0x7fb26c107460 2024-04-24T23:29:26.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.123+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1a6580 con 0x7fb26c107460 2024-04-24T23:29:26.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.124+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c0fd6e0 con 0x7fb26c107460 2024-04-24T23:29:26.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.126+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb264005560 con 0x7fb26c107460 2024-04-24T23:29:26.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.127+0000 7fb261ffb640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2540a2ce0 0x7fb2540a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.127+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2640cf580 con 0x7fb26c107460 2024-04-24T23:29:26.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.128+0000 7fb27084f640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2540a2ce0 0x7fb2540a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.129+0000 7fb27084f640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2540a2ce0 0x7fb2540a51a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb268031440 tx=0x7fb268000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.137+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb264016030 con 0x7fb26c107460 2024-04-24T23:29:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.483+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb26c062340 con 0x7fb26c107460 2024-04-24T23:29:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.484+0000 7fb261ffb640 1 -- 172.21.15.33:0/1448099385 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7fb26409baa0 con 0x7fb26c107460 2024-04-24T23:29:26.485 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:29:26.485 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:29:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2540a2ce0 msgr2=0x7fb2540a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2540a2ce0 0x7fb2540a51a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb268031440 tx=0x7fb268000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 msgr2=0x7fb26c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 0x7fb26c1006f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb26400eaa0 tx=0x7fb26400ef60 comp rx=0 tx=0).stop 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 shutdown_connections 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb2540a2ce0 0x7fb2540a51a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c107460 0x7fb26c1006f0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb26c104b30 0x7fb26c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 --2- 172.21.15.33:0/1448099385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb26c102200 0x7fb26c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 >> 172.21.15.33:0/1448099385 conn(0x7fb26c0fbbf0 msgr2=0x7fb26c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 shutdown_connections 2024-04-24T23:29:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.486+0000 7fb272ab3640 1 -- 172.21.15.33:0/1448099385 wait complete. 2024-04-24T23:29:26.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:29:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 -- 172.21.15.33:0/3006635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4102980 msgr2=0x7f54d4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/3006635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4102980 0x7f54d4102e00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009900 tx=0x7f54c402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 -- 172.21.15.33:0/3006635961 shutdown_connections 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/3006635961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4103340 0x7f54d4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/3006635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4102980 0x7f54d4102e00 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/3006635961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 0x7f54d4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 -- 172.21.15.33:0/3006635961 >> 172.21.15.33:0/3006635961 conn(0x7f54d40fcd20 msgr2=0x7f54d40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 -- 172.21.15.33:0/3006635961 shutdown_connections 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 -- 172.21.15.33:0/3006635961 wait complete. 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 Processor -- start 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.617+0000 7f54dc7ce640 1 -- start start 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dc7ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 0x7f54d4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dc7ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4102980 0x7f54d419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dc7ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 0x7f54d41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dc7ce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410d3f0 con 0x7f54d4103340 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dc7ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410d270 con 0x7f54d4102980 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dc7ce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410d570 con 0x7f54d4101790 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 0x7f54d41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4102980 0x7f54d419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54da56a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 0x7f54d4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 0x7f54d41a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45154/0 (socket says 172.21.15.33:45154) 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 -- 172.21.15.33:0/1600607840 learned_addr learned my addr 172.21.15.33:0/1600607840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 -- 172.21.15.33:0/1600607840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 msgr2=0x7f54d4199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 0x7f54d4199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 -- 172.21.15.33:0/1600607840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4102980 msgr2=0x7f54d419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4102980 0x7f54d419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 -- 172.21.15.33:0/1600607840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d41a1900 con 0x7f54d4103340 2024-04-24T23:29:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54da56a640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 0x7f54d4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54d9d69640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4102980 0x7f54d419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54dad6b640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 0x7f54d41a1260 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00eb10 tx=0x7f54cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.618+0000 7f54cb7fe640 1 -- 172.21.15.33:0/1600607840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc004b20 con 0x7f54d4103340 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.619+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d41a1bf0 con 0x7f54d4103340 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.619+0000 7f54cb7fe640 1 -- 172.21.15.33:0/1600607840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc004cc0 con 0x7f54d4103340 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.619+0000 7f54cb7fe640 1 -- 172.21.15.33:0/1600607840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc010760 con 0x7f54d4103340 2024-04-24T23:29:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.619+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d41a2110 con 0x7f54d4103340 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.621+0000 7f54df5b0740 1 Processor -- start 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.621+0000 7f54cb7fe640 1 -- 172.21.15.33:0/1600607840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f54cc005020 con 0x7f54d4103340 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.621+0000 7f54df5b0740 1 -- start start 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde90650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fde95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fde98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54df5b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5630fddf4670 con 0x5630fde95d90 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54df5b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5630fdd9c970 con 0x5630fde90230 2024-04-24T23:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54df5b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630fddf0630 con 0x5630fde90c20 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde90650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fde95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fde95720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56916/0 (socket says 172.21.15.33:56916) 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde90650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60790/0 (socket says 172.21.15.33:60790) 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 -- 172.21.15.33:0/834703520 learned_addr learned my addr 172.21.15.33:0/834703520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54d9d69640 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fde98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54cb7fe640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54c00a2b90 0x7f54c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 -- 172.21.15.33:0/834703520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 msgr2=0x5630fde90650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde90650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 -- 172.21.15.33:0/834703520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 msgr2=0x5630fde98220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fde98220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 -- 172.21.15.33:0/834703520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fddc0400 con 0x5630fde90c20 2024-04-24T23:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54dad6b640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54c00a2b90 0x7f54c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54d9d69640 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fde98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54cb7fe640 1 -- 172.21.15.33:0/1600607840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54cc0cf980 con 0x7f54d4103340 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.622+0000 7f54da56a640 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fde95720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f54d00114f0 tx=0x7f54d00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d92ff79853aceac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54c8ff9640 1 -- 172.21.15.33:0/834703520 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0005020 con 0x5630fde90c20 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54c8ff9640 1 -- 172.21.15.33:0/834703520 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d00047a0 con 0x5630fde90c20 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54c8ff9640 1 -- 172.21.15.33:0/834703520 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d000f930 con 0x5630fde90c20 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 -- 172.21.15.33:0/834703520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 msgr2=0x5630fde95720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fde95720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f54d00114f0 tx=0x7f54d00119b0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54dad6b640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54c00a2b90 0x7f54c00a5050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0982b0 tx=0x7f54cc01bb30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 -- 172.21.15.33:0/834703520 shutdown_connections 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fde98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fde95720 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 --2- 172.21.15.33:0/834703520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde90650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 -- 172.21.15.33:0/834703520 >> 172.21.15.33:0/834703520 conn(0x5630fde8b8a0 msgr2=0x5630fde8bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.623+0000 7f54df5b0740 1 -- 172.21.15.33:0/834703520 shutdown_connections 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 -- 172.21.15.33:0/834703520 wait complete. 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 -- start start 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 Processor -- start 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90230 0x5630fde9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 0x5630fde9a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fdea18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5630fde98d40 con 0x5630fde95d90 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5630fddb0050 con 0x5630fde90c20 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630fde98ec0 con 0x5630fde90230 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54df5b0740 10 client.0 _ll_register_callbacks cb 0x5630fdb07be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90230 0x5630fde9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54da56a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 0x5630fde9a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fdea18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54da56a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 0x5630fde9a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60794/0 (socket says 172.21.15.33:60794) 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.624+0000 7f54da56a640 1 -- 172.21.15.33:0/3459328035 learned_addr learned my addr 172.21.15.33:0/3459328035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54d9d69640 1 -- 172.21.15.33:0/3459328035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90230 msgr2=0x5630fde9a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54d9d69640 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90230 0x5630fde9a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54d9d69640 1 -- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 msgr2=0x5630fde9a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54d9d69640 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 0x5630fde9a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54d9d69640 1 -- 172.21.15.33:0/3459328035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fdd81dc0 con 0x5630fde95d90 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54da56a640 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 0x5630fde9a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54d9d69640 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fdea18e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f54c4031440 tx=0x7f54c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3760a7347a11ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54bdffb640 1 -- 172.21.15.33:0/3459328035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4035070 con 0x5630fde95d90 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54bdffb640 1 -- 172.21.15.33:0/3459328035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54c4004960 con 0x5630fde95d90 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d00047a0 con 0x5630fde95d90 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54bdffb640 1 -- 172.21.15.33:0/3459328035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54c4053050 con 0x5630fde95d90 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54df5b0740 10 client.19059 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5630fdb083f0 con 0x5630fde95d90 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54bdffb640 1 -- 172.21.15.33:0/3459328035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d00047a0 con 0x5630fde95d90 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.625+0000 7f54bdffb640 1 -- 172.21.15.33:0/3459328035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f54c403da30 con 0x5630fde95d90 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54df5b0740 10 client.19059 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54df5b0740 10 client.19059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54df5b0740 20 client.19059 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54df5b0740 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdea61e0 0x5630fdea8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54df5b0740 4 client.19059 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630fde01bb0 con 0x5630fdea61e0 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54da56a640 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdea61e0 0x5630fdea8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54da56a640 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdea61e0 0x5630fdea8610 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0ce12428eed17dd server_cookie=e8d4794a077fae5a in_seq=0 out_seq=0 2024-04-24T23:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.626+0000 7f54bdffb640 10 client.19059 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.627+0000 7f54bdffb640 1 -- 172.21.15.33:0/3459328035 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630fde01bb0 con 0x5630fdea61e0 2024-04-24T23:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.627+0000 7f54bdffb640 10 client.19059 handle_command_reply: tid=0 2024-04-24T23:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 client.19059 shutdown 2024-04-24T23:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 msgr2=0x5630fdea18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fdea18e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f54c4031440 tx=0x7f54c4005fd0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 shutdown_connections 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdea61e0 0x5630fdea8610 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde95d90 0x5630fdea18e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90c20 0x5630fde9a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54bdffb640 0 client.19059 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 --2- 172.21.15.33:0/3459328035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90230 0x5630fde9a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 >> 172.21.15.33:0/3459328035 conn(0x5630fde8b8a0 msgr2=0x5630fde8ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 shutdown_connections 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 1 -- 172.21.15.33:0/3459328035 wait complete. 2024-04-24T23:29:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.628+0000 7f54df5b0740 20 client.19059 trim_cache size 0 max 16384 2024-04-24T23:29:26.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 Processor -- start 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 -- start start 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 0x5630fde95370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fdea18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde95d90 0x5630fde8e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41aee60 con 0x5630fde90230 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ac730 con 0x5630fde95d90 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54df5b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41aa580 con 0x5630fde90c20 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 0x5630fde95370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 0x5630fde95370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45186/0 (socket says 172.21.15.33:45186) 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54d9d69640 1 -- 172.21.15.33:0/2952362723 learned_addr learned my addr 172.21.15.33:0/2952362723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54da56a640 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fdea18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.645+0000 7f54d9d69640 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde95d90 0x5630fde8e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54d9d69640 1 -- 172.21.15.33:0/2952362723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 msgr2=0x5630fdea18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54d9d69640 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fdea18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54d9d69640 1 -- 172.21.15.33:0/2952362723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde95d90 msgr2=0x5630fde8e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54d9d69640 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde95d90 0x5630fde8e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54d9d69640 1 -- 172.21.15.33:0/2952362723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fdbd27d0 con 0x5630fde90230 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54da56a640 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fdea18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54d9d69640 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 0x5630fde95370 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f54c40059d0 tx=0x7f54c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f87183cfea67c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54bffff640 1 -- 172.21.15.33:0/2952362723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4035040 con 0x5630fde90230 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54bffff640 1 -- 172.21.15.33:0/2952362723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c4031bd0 con 0x5630fde90230 2024-04-24T23:29:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.646+0000 7f54bffff640 1 -- 172.21.15.33:0/2952362723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c403cd00 con 0x5630fde90230 2024-04-24T23:29:26.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 -- 172.21.15.33:0/2952362723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 msgr2=0x5630fde95370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 0x5630fde95370 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f54c40059d0 tx=0x7f54c4005e80 comp rx=0 tx=0).stop 2024-04-24T23:29:26.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 -- 172.21.15.33:0/2952362723 shutdown_connections 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde95d90 0x5630fde8e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde90c20 0x5630fdea18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 --2- 172.21.15.33:0/2952362723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90230 0x5630fde95370 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 -- 172.21.15.33:0/2952362723 >> 172.21.15.33:0/2952362723 conn(0x5630fde8b8a0 msgr2=0x5630fde8ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 -- 172.21.15.33:0/2952362723 shutdown_connections 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 -- 172.21.15.33:0/2952362723 wait complete. 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 -- start start 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.647+0000 7f54df5b0740 1 Processor -- start 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde9f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90c20 0x5630fde9f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde95d90 0x5630fdea0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41aabf0 con 0x5630fde90c20 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ac060 con 0x5630fde90230 2024-04-24T23:29:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ab2c0 con 0x5630fde95d90 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54df5b0740 10 client.0 _ll_register_callbacks cb 0x7f54c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54da56a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90c20 0x5630fde9f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde9f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54da56a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90c20 0x5630fde9f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45194/0 (socket says 172.21.15.33:45194) 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54da56a640 1 -- 172.21.15.33:0/606029922 learned_addr learned my addr 172.21.15.33:0/606029922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde95d90 0x5630fdea0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 -- 172.21.15.33:0/606029922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde95d90 msgr2=0x5630fdea0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde95d90 0x5630fdea0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 -- 172.21.15.33:0/606029922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90c20 msgr2=0x5630fde9f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90c20 0x5630fde9f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.648+0000 7f54d9d69640 1 -- 172.21.15.33:0/606029922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fdea0b80 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54d9d69640 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde9f040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54c40314e0 tx=0x7f54c405d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7b50a2a9eb88b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54bdffb640 1 -- 172.21.15.33:0/606029922 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4035040 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c4031bd0 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54bdffb640 1 -- 172.21.15.33:0/606029922 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54c403ccb0 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54bdffb640 1 -- 172.21.15.33:0/606029922 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54c403d450 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54df5b0740 10 client.28562 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5630fdf0c7c0 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.649+0000 7f54bdffb640 1 -- 172.21.15.33:0/606029922 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c4031bd0 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54bdffb640 1 -- 172.21.15.33:0/606029922 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f54c4004bf0 con 0x5630fde90230 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54df5b0740 10 client.28562 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54df5b0740 10 client.28562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54df5b0740 20 client.28562 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54df5b0740 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdf0d0c0 0x5630fdf0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54df5b0740 4 client.28562 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630fddfbf00 con 0x5630fdf0d0c0 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54da56a640 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdf0d0c0 0x5630fdf0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54da56a640 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdf0d0c0 0x5630fdf0f4e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea3ccf1bd392b2da server_cookie=555e338b619ca6fd in_seq=0 out_seq=0 2024-04-24T23:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.650+0000 7f54bdffb640 10 client.28562 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.652+0000 7f54bdffb640 1 -- 172.21.15.33:0/606029922 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630fddfbf00 con 0x5630fdf0d0c0 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.652+0000 7f54bdffb640 10 client.28562 handle_command_reply: tid=0 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.652+0000 7f54df5b0740 1 client.28562 shutdown 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.652+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 msgr2=0x5630fde9f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.652+0000 7f54df5b0740 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde9f040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54c40314e0 tx=0x7f54c405d040 comp rx=0 tx=0).stop 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.652+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 shutdown_connections 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630fdf0d0c0 0x5630fdf0f4e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630fde95d90 0x5630fdea0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54bdffb640 0 client.28562 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630fde90c20 0x5630fde9f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 --2- 172.21.15.33:0/606029922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630fde90230 0x5630fde9f040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 >> 172.21.15.33:0/606029922 conn(0x5630fde8b8a0 msgr2=0x5630fde8ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 shutdown_connections 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 1 -- 172.21.15.33:0/606029922 wait complete. 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.653+0000 7f54df5b0740 20 client.28562 trim_cache size 0 max 16384 2024-04-24T23:29:26.658 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54c00a2b90 msgr2=0x7f54c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54c00a2b90 0x7f54c00a5050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0982b0 tx=0x7f54cc01bb30 comp rx=0 tx=0).stop 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 msgr2=0x7f54d41a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 0x7f54d41a1260 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00eb10 tx=0x7f54cc00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 shutdown_connections 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f54c00a2b90 0x7f54c00a5050 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d4103340 0x7f54d41a1260 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54d4102980 0x7f54d419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 --2- 172.21.15.33:0/1600607840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54d4101790 0x7f54d4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.654+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 >> 172.21.15.33:0/1600607840 conn(0x7f54d40fcd20 msgr2=0x7f54d4073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.655+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 shutdown_connections 2024-04-24T23:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.655+0000 7f54dc7ce640 1 -- 172.21.15.33:0/1600607840 wait complete. 2024-04-24T23:29:26.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:29:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 -- 172.21.15.33:0/4212476484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24104aa0 msgr2=0x7f8c24106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/4212476484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24104aa0 0x7f8c24106e90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009900 tx=0x7f8c1802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 -- 172.21.15.33:0/4212476484 shutdown_connections 2024-04-24T23:29:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/4212476484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c241073d0 0x7f8c241097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/4212476484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24104aa0 0x7f8c24106e90 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/4212476484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 0x7f8c24104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 -- 172.21.15.33:0/4212476484 >> 172.21.15.33:0/4212476484 conn(0x7f8c240fbae0 msgr2=0x7f8c240fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 -- 172.21.15.33:0/4212476484 shutdown_connections 2024-04-24T23:29:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.788+0000 7f8c28b9e640 1 -- 172.21.15.33:0/4212476484 wait complete. 2024-04-24T23:29:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 Processor -- start 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 -- start start 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 0x7f8c24101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c24104aa0 0x7f8c24100120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 0x7f8c24100660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410d310 con 0x7f8c241073d0 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410d190 con 0x7f8c24104aa0 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c28b9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410d490 con 0x7f8c24102170 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 0x7f8c24100660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c24104aa0 0x7f8c24100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 0x7f8c24101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c24104aa0 0x7f8c24100120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60822/0 (socket says 172.21.15.33:60822) 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 0x7f8c24100660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45212/0 (socket says 172.21.15.33:45212) 2024-04-24T23:29:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.789+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3527421156 learned_addr learned my addr 172.21.15.33:0/3527421156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c237fe640 1 -- 172.21.15.33:0/3527421156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c24104aa0 msgr2=0x7f8c24100120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c24104aa0 0x7f8c24100120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c237fe640 1 -- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 msgr2=0x7f8c24100660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 0x7f8c24100660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c237fe640 1 -- 172.21.15.33:0/3527421156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c24100ba0 con 0x7f8c24102170 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c23fff640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 0x7f8c24100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 0x7f8c24101a40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400e9f0 tx=0x7f8c1400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c20ff9640 1 -- 172.21.15.33:0/3527421156 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400ccc0 con 0x7f8c24102170 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c24100e90 con 0x7f8c24102170 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c20ff9640 1 -- 172.21.15.33:0/3527421156 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14005020 con 0x7f8c24102170 2024-04-24T23:29:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c20ff9640 1 -- 172.21.15.33:0/3527421156 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004a00 con 0x7f8c24102170 2024-04-24T23:29:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.790+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c241a6450 con 0x7f8c24102170 2024-04-24T23:29:26.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.792+0000 7f8c20ff9640 1 -- 172.21.15.33:0/3527421156 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8c140051d0 con 0x7f8c24102170 2024-04-24T23:29:26.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 Processor -- start 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 -- start start 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c20ff9640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c0c0a2b90 0x7f8c0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506be17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506be68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be6f20 0x564506be93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564506b45670 con 0x564506be6f20 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564506aed970 con 0x564506be13c0 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564506b41630 con 0x564506be1db0 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c23fff640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c0c0a2b90 0x7f8c0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c20ff9640 1 -- 172.21.15.33:0/3527421156 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8c140cf300 con 0x7f8c24102170 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be6f20 0x564506be93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506be17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506be17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60826/0 (socket says 172.21.15.33:60826) 2024-04-24T23:29:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be6f20 0x564506be93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45224/0 (socket says 172.21.15.33:45224) 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c237fe640 1 -- 172.21.15.33:0/559849182 learned_addr learned my addr 172.21.15.33:0/559849182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.793+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506be68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c237fe640 1 -- 172.21.15.33:0/559849182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 msgr2=0x564506be68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c237fe640 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506be68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c237fe640 1 -- 172.21.15.33:0/559849182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be6f20 msgr2=0x564506be93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c237fe640 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be6f20 0x564506be93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c237fe640 1 -- 172.21.15.33:0/559849182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564506b11400 con 0x564506be13c0 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c23fff640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c0c0a2b90 0x7f8c0c0a5050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009b80 tx=0x7f8c1c011710 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c237fe640 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506be17e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c140a3f10 tx=0x7f8c1409bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f534d35ce97a774 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c09ffb640 1 -- 172.21.15.33:0/559849182 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1401c040 con 0x564506be13c0 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c09ffb640 1 -- 172.21.15.33:0/559849182 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1409fb80 con 0x564506be13c0 2024-04-24T23:29:26.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.794+0000 7f8c09ffb640 1 -- 172.21.15.33:0/559849182 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140860d0 con 0x564506be13c0 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 -- 172.21.15.33:0/559849182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 msgr2=0x564506be17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506be17e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c140a3f10 tx=0x7f8c1409bed0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 -- 172.21.15.33:0/559849182 shutdown_connections 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be6f20 0x564506be93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506be68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/559849182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506be17e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 -- 172.21.15.33:0/559849182 >> 172.21.15.33:0/559849182 conn(0x564506bdca30 msgr2=0x564506bdce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 -- 172.21.15.33:0/559849182 shutdown_connections 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 -- 172.21.15.33:0/559849182 wait complete. 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 -- start start 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.795+0000 7f8c2cbef740 1 Processor -- start 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506beb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506beb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be6f20 0x564506bf29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564506be9e70 con 0x564506be1db0 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564506b01050 con 0x564506be13c0 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564506be9ff0 con 0x564506be6f20 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506beb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506beb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506beb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45230/0 (socket says 172.21.15.33:45230) 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c22ffd640 1 -- 172.21.15.33:0/1648642928 learned_addr learned my addr 172.21.15.33:0/1648642928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c23fff640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be6f20 0x564506bf29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.796+0000 7f8c2cbef740 10 client.0 _ll_register_callbacks cb 0x564506858be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c23fff640 1 -- 172.21.15.33:0/1648642928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 msgr2=0x564506beb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c23fff640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506beb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c23fff640 1 -- 172.21.15.33:0/1648642928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 msgr2=0x564506beb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c23fff640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506beb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c23fff640 1 -- 172.21.15.33:0/1648642928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564506ad2dc0 con 0x564506be6f20 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506beb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506beb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c23fff640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be6f20 0x564506bf29b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099a0 tx=0x7f8c1c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=865cff5aed8fb6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1648642928 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c024070 con 0x564506be6f20 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1648642928 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8c1c01e610 con 0x564506be6f20 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c1409fb80 con 0x564506be6f20 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1648642928 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c1c033050 con 0x564506be6f20 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c2cbef740 10 client.28501 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.797+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5645068593f0 con 0x564506be6f20 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1648642928 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1409fb80 con 0x564506be6f20 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1648642928 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8c1c028910 con 0x564506be6f20 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c2cbef740 10 client.28501 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c2cbef740 10 client.28501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c2cbef740 20 client.28501 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506bf72d0 0x564506bf9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c2cbef740 4 client.28501 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564506b52bb0 con 0x564506bf72d0 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.798+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506bf72d0 0x564506bf9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.799+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506bf72d0 0x564506bf9730 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=467beaa548e59197 server_cookie=664817c8f5046c43 in_seq=0 out_seq=0 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.799+0000 7f8c02ffd640 10 client.28501 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.800+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1648642928 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564506b52bb0 con 0x564506bf72d0 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.800+0000 7f8c02ffd640 10 client.28501 handle_command_reply: tid=0 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.800+0000 7f8c2cbef740 1 client.28501 shutdown 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.800+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be6f20 msgr2=0x564506bf29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.800+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be6f20 0x564506bf29b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099a0 tx=0x7f8c1c01d000 comp rx=0 tx=0).stop 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 shutdown_connections 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506bf72d0 0x564506bf9730 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be6f20 0x564506bf29b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506beb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c02ffd640 0 client.28501 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/1648642928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be13c0 0x564506beb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 >> 172.21.15.33:0/1648642928 conn(0x564506bdca30 msgr2=0x564506bdbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 shutdown_connections 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.801+0000 7f8c2cbef740 1 -- 172.21.15.33:0/1648642928 wait complete. 2024-04-24T23:29:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.802+0000 7f8c2cbef740 20 client.28501 trim_cache size 0 max 16384 2024-04-24T23:29:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.822+0000 7f8c2cbef740 1 Processor -- start 2024-04-24T23:29:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.822+0000 7f8c2cbef740 1 -- start start 2024-04-24T23:29:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.822+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be13c0 0x564506bd79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.822+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506bf2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bdf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241b3130 con 0x564506be1db0 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241b0a00 con 0x564506be6f20 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241ae850 con 0x564506be13c0 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506bf2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506bf2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45240/0 (socket says 172.21.15.33:45240) 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3307595088 learned_addr learned my addr 172.21.15.33:0/3307595088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be13c0 0x564506bd79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c23fff640 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bdf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3307595088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be13c0 msgr2=0x564506bd79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be13c0 0x564506bd79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3307595088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 msgr2=0x564506bdf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bdf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3307595088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645069237d0 con 0x564506be1db0 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506bf2970 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006010 tx=0x7f8c18004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7210cc41939b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c097fa640 1 -- 172.21.15.33:0/3307595088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18035040 con 0x564506be1db0 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c097fa640 1 -- 172.21.15.33:0/3307595088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c180044a0 con 0x564506be1db0 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.823+0000 7f8c097fa640 1 -- 172.21.15.33:0/3307595088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1803d450 con 0x564506be1db0 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 -- 172.21.15.33:0/3307595088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 msgr2=0x564506bf2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506bf2970 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006010 tx=0x7f8c18004330 comp rx=0 tx=0).stop 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 -- 172.21.15.33:0/3307595088 shutdown_connections 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bdf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be1db0 0x564506bf2970 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/3307595088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be13c0 0x564506bd79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 -- 172.21.15.33:0/3307595088 >> 172.21.15.33:0/3307595088 conn(0x564506bdca30 msgr2=0x564506bdbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 -- 172.21.15.33:0/3307595088 shutdown_connections 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.824+0000 7f8c2cbef740 1 -- 172.21.15.33:0/3307595088 wait complete. 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 -- start start 2024-04-24T23:29:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 Processor -- start 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 0x564506bf0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506bf06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bf16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241aeec0 con 0x564506be13c0 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241b0330 con 0x564506be6f20 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241af590 con 0x564506be1db0 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c2cbef740 10 client.0 _ll_register_callbacks cb 0x7f8c1c003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 0x564506bf0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bf16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 0x564506bf0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45248/0 (socket says 172.21.15.33:45248) 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c237fe640 1 -- 172.21.15.33:0/293191639 learned_addr learned my addr 172.21.15.33:0/293191639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.825+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506bf06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c23fff640 1 -- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 msgr2=0x564506bf06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c23fff640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506bf06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c23fff640 1 -- 172.21.15.33:0/293191639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 msgr2=0x564506bf0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c23fff640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 0x564506bf0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c23fff640 1 -- 172.21.15.33:0/293191639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564506bf1cb0 con 0x564506be6f20 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c237fe640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 0x564506bf0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c23fff640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bf16c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c011950 tx=0x7f8c1c0286b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafe0e030e9e62dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c02ffd640 1 -- 172.21.15.33:0/293191639 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c03d070 con 0x564506be6f20 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c180044a0 con 0x564506be6f20 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c02ffd640 1 -- 172.21.15.33:0/293191639 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8c1c028ab0 con 0x564506be6f20 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c02ffd640 1 -- 172.21.15.33:0/293191639 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c1c049050 con 0x564506be6f20 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c2cbef740 10 client.28580 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:26.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564506c5d940 con 0x564506be6f20 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.826+0000 7f8c02ffd640 1 -- 172.21.15.33:0/293191639 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c180044a0 con 0x564506be6f20 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c02ffd640 1 -- 172.21.15.33:0/293191639 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8c1c01ebf0 con 0x564506be6f20 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 10 client.28580 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 10 client.28580 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 20 client.28580 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506c5e230 0x564506c60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 4 client.28580 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564506b4cf00 con 0x564506c5e230 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c237fe640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506c5e230 0x564506c60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c02ffd640 10 client.28580 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c237fe640 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506c5e230 0x564506c60650 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69485c4bf6274635 server_cookie=2da3c88800f9d89 in_seq=0 out_seq=0 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c02ffd640 1 -- 172.21.15.33:0/293191639 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564506b4cf00 con 0x564506c5e230 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c02ffd640 10 client.28580 handle_command_reply: tid=0 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.827+0000 7f8c2cbef740 1 client.28580 shutdown 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 msgr2=0x564506bf16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bf16c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c011950 tx=0x7f8c1c0286b0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 shutdown_connections 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564506c5e230 0x564506c60650 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564506be6f20 0x564506bf16c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564506be1db0 0x564506bf06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c02ffd640 0 client.28580 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 --2- 172.21.15.33:0/293191639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564506be13c0 0x564506bf0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 >> 172.21.15.33:0/293191639 conn(0x564506bdca30 msgr2=0x564506be26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 shutdown_connections 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 1 -- 172.21.15.33:0/293191639 wait complete. 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c2cbef740 20 client.28580 trim_cache size 0 max 16384 2024-04-24T23:29:26.836 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:29:26.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c0c0a2b90 msgr2=0x7f8c0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c0c0a2b90 0x7f8c0c0a5050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009b80 tx=0x7f8c1c011710 comp rx=0 tx=0).stop 2024-04-24T23:29:26.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 msgr2=0x7f8c24101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 0x7f8c24101a40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400e9f0 tx=0x7f8c1400edf0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 shutdown_connections 2024-04-24T23:29:26.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8c0c0a2b90 0x7f8c0c0a5050 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c241073d0 0x7f8c24100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c24104aa0 0x7f8c24100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 --2- 172.21.15.33:0/3527421156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c24102170 0x7f8c24101a40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.828+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 >> 172.21.15.33:0/3527421156 conn(0x7f8c240fbae0 msgr2=0x7f8c24102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.829+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 shutdown_connections 2024-04-24T23:29:26.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.829+0000 7f8c28b9e640 1 -- 172.21.15.33:0/3527421156 wait complete. 2024-04-24T23:29:26.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:29:26.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 92957 22.52 MB/sec execute 91 sec latency 49.896 ms 2024-04-24T23:29:26.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 -- 172.21.15.33:0/3461739561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 msgr2=0x7f9718101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 --2- 172.21.15.33:0/3461739561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718101bb0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f970c009900 tx=0x7f970c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 -- 172.21.15.33:0/3461739561 shutdown_connections 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 --2- 172.21.15.33:0/3461739561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 0x7f9718109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 --2- 172.21.15.33:0/3461739561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9718102980 0x7f9718102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 --2- 172.21.15.33:0/3461739561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718101bb0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 -- 172.21.15.33:0/3461739561 >> 172.21.15.33:0/3461739561 conn(0x7f97180fcd20 msgr2=0x7f97180ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 -- 172.21.15.33:0/3461739561 shutdown_connections 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.960+0000 7f971ed5a640 1 -- 172.21.15.33:0/3461739561 wait complete. 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.961+0000 7f971ed5a640 1 Processor -- start 2024-04-24T23:29:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.961+0000 7f971ed5a640 1 -- start start 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.961+0000 7f971ed5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971ed5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9718102980 0x7f971819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971ed5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 0x7f97181a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971ed5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f971810d410 con 0x7f9718101790 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971ed5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f971810d290 con 0x7f9718102980 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971ed5a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f971810d590 con 0x7f9718103340 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971caf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f9717fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9718102980 0x7f971819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971caf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45266/0 (socket says 172.21.15.33:45266) 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971caf6640 1 -- 172.21.15.33:0/167250141 learned_addr learned my addr 172.21.15.33:0/167250141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971d2f7640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 0x7f97181a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971d2f7640 1 -- 172.21.15.33:0/167250141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9718102980 msgr2=0x7f971819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971d2f7640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9718102980 0x7f971819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971d2f7640 1 -- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 msgr2=0x7f9718199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971d2f7640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.962+0000 7f971d2f7640 1 -- 172.21.15.33:0/167250141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97181a1960 con 0x7f9718103340 2024-04-24T23:29:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.963+0000 7f971d2f7640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 0x7f97181a12f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9710009fd0 tx=0x7f971000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.963+0000 7f9715ffb640 1 -- 172.21.15.33:0/167250141 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000ce20 con 0x7f9718103340 2024-04-24T23:29:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.963+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97181a1c50 con 0x7f9718103340 2024-04-24T23:29:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.963+0000 7f9715ffb640 1 -- 172.21.15.33:0/167250141 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9710005020 con 0x7f9718103340 2024-04-24T23:29:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.963+0000 7f9715ffb640 1 -- 172.21.15.33:0/167250141 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710010720 con 0x7f9718103340 2024-04-24T23:29:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.963+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a21a0 con 0x7f9718103340 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.966+0000 7f9715ffb640 1 -- 172.21.15.33:0/167250141 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f97100108c0 con 0x7f9718103340 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 Processor -- start 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 -- start start 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 0x5640deb11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb19220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5640dea75670 con 0x5640deb11230 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5640dea1d970 con 0x5640deb11c20 2024-04-24T23:29:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9721a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640dea71630 con 0x5640deb16d90 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f971caf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 0x5640deb11650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 0x5640deb11650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45276/0 (socket says 172.21.15.33:45276) 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9717fff640 1 -- 172.21.15.33:0/2524679313 learned_addr learned my addr 172.21.15.33:0/2524679313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f971caf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb16720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60888/0 (socket says 172.21.15.33:60888) 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9717fff640 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb19220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f9715ffb640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f97000a2c10 0x7f97000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f971d2f7640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f97000a2c10 0x7f97000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f971caf6640 1 -- 172.21.15.33:0/2524679313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 msgr2=0x5640deb19220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.967+0000 7f971caf6640 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb19220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f971caf6640 1 -- 172.21.15.33:0/2524679313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 msgr2=0x5640deb11650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f971caf6640 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 0x5640deb11650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f971caf6640 1 -- 172.21.15.33:0/2524679313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640dea41400 con 0x5640deb11c20 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f9717fff640 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 0x5640deb11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f9715ffb640 1 -- 172.21.15.33:0/167250141 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f97100cf9b0 con 0x7f9718103340 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f971caf6640 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb16720 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f970c005ab0 tx=0x7f970c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93731e8e882073d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f971d2f7640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f97000a2c10 0x7f97000a50d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f971009bb80 tx=0x7f9710007dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f96feffd640 1 -- 172.21.15.33:0/2524679313 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c035040 con 0x5640deb11c20 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f96feffd640 1 -- 172.21.15.33:0/2524679313 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c0318b0 con 0x5640deb11c20 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.968+0000 7f96feffd640 1 -- 172.21.15.33:0/2524679313 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c0049e0 con 0x5640deb11c20 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 -- 172.21.15.33:0/2524679313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 msgr2=0x5640deb16720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb16720 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f970c005ab0 tx=0x7f970c005de0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 -- 172.21.15.33:0/2524679313 shutdown_connections 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb19220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb16720 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 --2- 172.21.15.33:0/2524679313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11230 0x5640deb11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 -- 172.21.15.33:0/2524679313 >> 172.21.15.33:0/2524679313 conn(0x5640deb0c8a0 msgr2=0x5640deb0cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 -- 172.21.15.33:0/2524679313 shutdown_connections 2024-04-24T23:29:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.969+0000 7f9721a64740 1 -- 172.21.15.33:0/2524679313 wait complete. 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.970+0000 7f9721a64740 1 -- start start 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.970+0000 7f9721a64740 1 Processor -- start 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.970+0000 7f9721a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.970+0000 7f9721a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb1b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9721a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb16d90 0x5640deb228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9721a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5640deb19ce0 con 0x5640deb16d90 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9721a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5640dea31050 con 0x5640deb11c20 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9721a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640deb19e60 con 0x5640deb11230 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9721a64740 10 client.0 _ll_register_callbacks cb 0x5640de788be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f971caf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb1b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f971caf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb1b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60902/0 (socket says 172.21.15.33:60902) 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f971caf6640 1 -- 172.21.15.33:0/2635584132 learned_addr learned my addr 172.21.15.33:0/2635584132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb1b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57014/0 (socket says 172.21.15.33:57014) 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb16d90 0x5640deb228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 -- 172.21.15.33:0/2635584132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 msgr2=0x5640deb1b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb1b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 -- 172.21.15.33:0/2635584132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb16d90 msgr2=0x5640deb228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb16d90 0x5640deb228a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 -- 172.21.15.33:0/2635584132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640dea02dc0 con 0x5640deb11230 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f971caf6640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb1b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f9717fff640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb1b320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fd0 tx=0x7f9708004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3084896c58a2f3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.971+0000 7f96f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:29:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f96f7fff640 1 -- 172.21.15.33:0/2635584132 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970801a040 con 0x5640deb11230 2024-04-24T23:29:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f96f7fff640 1 -- 172.21.15.33:0/2635584132 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9708004250 con 0x5640deb11230 2024-04-24T23:29:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c0318b0 con 0x5640deb11230 2024-04-24T23:29:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f96f7fff640 1 -- 172.21.15.33:0/2635584132 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f970802b050 con 0x5640deb11230 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 10 client.28519 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5640de7893f0 con 0x5640deb11230 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f96f7fff640 1 -- 172.21.15.33:0/2635584132 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c0318b0 con 0x5640deb11230 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f96f7fff640 1 -- 172.21.15.33:0/2635584132 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f97080203f0 con 0x5640deb11230 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 10 client.28519 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 10 client.28519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 20 client.28519 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb275e0 0x5640deb29a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 4 client.28519 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.972+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640dea82bb0 con 0x5640deb275e0 2024-04-24T23:29:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.973+0000 7f971caf6640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb275e0 0x5640deb29a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.973+0000 7f96f7fff640 10 client.28519 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.973+0000 7f971caf6640 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb275e0 0x5640deb29a80 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5346875ca9a8d241 server_cookie=9308dd8f09f29ab3 in_seq=0 out_seq=0 2024-04-24T23:29:26.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f96f7fff640 1 -- 172.21.15.33:0/2635584132 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640dea82bb0 con 0x5640deb275e0 2024-04-24T23:29:26.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f96f7fff640 10 client.28519 handle_command_reply: tid=0 2024-04-24T23:29:26.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 client.28519 shutdown 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 msgr2=0x5640deb1b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb1b320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fd0 tx=0x7f9708004730 comp rx=0 tx=0).stop 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 shutdown_connections 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb275e0 0x5640deb29a80 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb16d90 0x5640deb228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11c20 0x5640deb1b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 --2- 172.21.15.33:0/2635584132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb1b320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f96f7fff640 0 client.28519 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:26.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.974+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 >> 172.21.15.33:0/2635584132 conn(0x5640deb0c8a0 msgr2=0x5640deb0bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.975+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 shutdown_connections 2024-04-24T23:29:26.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.975+0000 7f9721a64740 1 -- 172.21.15.33:0/2635584132 wait complete. 2024-04-24T23:29:26.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.975+0000 7f9721a64740 20 client.28519 trim_cache size 0 max 16384 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 Processor -- start 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 -- start start 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb0f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb22860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb16d90 0x5640deb0fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97181aaad0 con 0x5640deb11c20 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f97181a83a0 con 0x5640deb16d90 2024-04-24T23:29:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9721a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97181a6230 con 0x5640deb11230 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb22860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb22860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45294/0 (socket says 172.21.15.33:45294) 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 -- 172.21.15.33:0/1451535236 learned_addr learned my addr 172.21.15.33:0/1451535236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9717fff640 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb0f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f9717fff640 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb16d90 0x5640deb0fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 -- 172.21.15.33:0/1451535236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 msgr2=0x5640deb0f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb0f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 -- 172.21.15.33:0/1451535236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb16d90 msgr2=0x5640deb0fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb16d90 0x5640deb0fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.990+0000 7f971caf6640 1 -- 172.21.15.33:0/1451535236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640de8537d0 con 0x5640deb11c20 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.991+0000 7f971caf6640 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb22860 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f970c009230 tx=0x7f970c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82c49ba868b1d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.991+0000 7f96fe7fc640 1 -- 172.21.15.33:0/1451535236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c04f040 con 0x5640deb11c20 2024-04-24T23:29:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.991+0000 7f96fe7fc640 1 -- 172.21.15.33:0/1451535236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c0318b0 con 0x5640deb11c20 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.991+0000 7f96fe7fc640 1 -- 172.21.15.33:0/1451535236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c035040 con 0x5640deb11c20 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- 172.21.15.33:0/1451535236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 msgr2=0x5640deb22860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb22860 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f970c009230 tx=0x7f970c009260 comp rx=0 tx=0).stop 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- 172.21.15.33:0/1451535236 shutdown_connections 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb16d90 0x5640deb0fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb22860 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- 172.21.15.33:0/1451535236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb11230 0x5640deb0f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- 172.21.15.33:0/1451535236 >> 172.21.15.33:0/1451535236 conn(0x5640deb0c8a0 msgr2=0x5640deb0bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- 172.21.15.33:0/1451535236 shutdown_connections 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- 172.21.15.33:0/1451535236 wait complete. 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- start start 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 Processor -- start 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11230 0x5640deb1f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb1f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb8bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97181a6880 con 0x5640deb11c20 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f97181a7cd0 con 0x5640deb11230 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.992+0000 7f9721a64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97181a6f30 con 0x5640deb16d90 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f9721a64740 10 client.0 _ll_register_callbacks cb 0x7f97080126b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb1f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb1f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45304/0 (socket says 172.21.15.33:45304) 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 -- 172.21.15.33:0/1830417149 learned_addr learned my addr 172.21.15.33:0/1830417149 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f9717fff640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11230 0x5640deb1f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f9717fff640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb8bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 -- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 msgr2=0x5640deb8bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb8bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 -- 172.21.15.33:0/1830417149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11230 msgr2=0x5640deb1f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11230 0x5640deb1f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 -- 172.21.15.33:0/1830417149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640deb22a20 con 0x5640deb11c20 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f971caf6640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb1f950 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f970c006010 tx=0x7f970c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffba57e346daaab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f96f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f96f7fff640 1 -- 172.21.15.33:0/1830417149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c035030 con 0x5640deb11c20 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f96f7fff640 1 -- 172.21.15.33:0/1830417149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f970c0316b0 con 0x5640deb11c20 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c0318b0 con 0x5640deb11c20 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f96f7fff640 1 -- 172.21.15.33:0/1830417149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f970c070050 con 0x5640deb11c20 2024-04-24T23:29:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f9721a64740 10 client.19101 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.993+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5640deb8d0b0 con 0x5640deb11c20 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f96f7fff640 1 -- 172.21.15.33:0/1830417149 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c0318b0 con 0x5640deb11c20 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f96f7fff640 1 -- 172.21.15.33:0/1830417149 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f970c049070 con 0x5640deb11c20 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9721a64740 10 client.19101 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9721a64740 10 client.19101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9721a64740 20 client.19101 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9721a64740 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb8d590 0x5640deb8f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9721a64740 4 client.19101 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640dea7cf00 con 0x5640deb8d590 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9717fff640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb8d590 0x5640deb8f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f96f7fff640 10 client.19101 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.994+0000 7f9717fff640 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb8d590 0x5640deb8f9b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9abbe846e62201 server_cookie=9a35629264e1df92 in_seq=0 out_seq=0 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f96f7fff640 1 -- 172.21.15.33:0/1830417149 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640dea7cf00 con 0x5640deb8d590 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f96f7fff640 10 client.19101 handle_command_reply: tid=0 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 client.19101 shutdown 2024-04-24T23:29:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 msgr2=0x5640deb1f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb1f950 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f970c006010 tx=0x7f970c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 shutdown_connections 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5640deb8d590 0x5640deb8f9b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640deb16d90 0x5640deb8bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640deb11c20 0x5640deb1f950 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f96f7fff640 0 client.19101 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 --2- 172.21.15.33:0/1830417149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640deb11230 0x5640deb1f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 >> 172.21.15.33:0/1830417149 conn(0x5640deb0c8a0 msgr2=0x5640deb12620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 shutdown_connections 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 1 -- 172.21.15.33:0/1830417149 wait complete. 2024-04-24T23:29:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.995+0000 7f9721a64740 20 client.19101 trim_cache size 0 max 16384 2024-04-24T23:29:27.003 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:29:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f97000a2c10 msgr2=0x7f97000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f97000a2c10 0x7f97000a50d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f971009bb80 tx=0x7f9710007dd0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 msgr2=0x7f97181a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 0x7f97181a12f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9710009fd0 tx=0x7f971000ef10 comp rx=0 tx=0).stop 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 shutdown_connections 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f97000a2c10 0x7f97000a50d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718103340 0x7f97181a12f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9718102980 0x7f971819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 --2- 172.21.15.33:0/167250141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9718101790 0x7f9718199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 >> 172.21.15.33:0/167250141 conn(0x7f97180fcd20 msgr2=0x7f9718073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 shutdown_connections 2024-04-24T23:29:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:26.996+0000 7f971ed5a640 1 -- 172.21.15.33:0/167250141 wait complete. 2024-04-24T23:29:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:27 smithi038 ceph-mon[32450]: pgmap v874: 129 pgs: 129 active+clean; 521 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 417 op/s 2024-04-24T23:29:27.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1448099385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:27 smithi117 ceph-mon[32529]: pgmap v874: 129 pgs: 129 active+clean; 521 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 417 op/s 2024-04-24T23:29:27.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1448099385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:27.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:27 smithi033 ceph-mon[29201]: pgmap v874: 129 pgs: 129 active+clean; 521 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 417 op/s 2024-04-24T23:29:27.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1448099385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:27.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 93463 22.62 MB/sec execute 92 sec latency 231.255 ms 2024-04-24T23:29:28.012 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:29:28.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:28.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.124+0000 7f157ec12640 1 -- 172.21.15.33:0/114277434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 msgr2=0x7f1578101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.124+0000 7f157ec12640 1 --2- 172.21.15.33:0/114277434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578101cf0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1564009900 tx=0x7f156402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- 172.21.15.33:0/114277434 shutdown_connections 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 --2- 172.21.15.33:0/114277434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1578102230 0x7f1578108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 --2- 172.21.15.33:0/114277434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578101cf0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 --2- 172.21.15.33:0/114277434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 0x7f1578100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- 172.21.15.33:0/114277434 >> 172.21.15.33:0/114277434 conn(0x7f15780fbbf0 msgr2=0x7f15780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- 172.21.15.33:0/114277434 shutdown_connections 2024-04-24T23:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- 172.21.15.33:0/114277434 wait complete. 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 Processor -- start 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- start start 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 0x7f15781045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1578102230 0x7f1578103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f157810c330 con 0x7f1578101870 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f157810c1b0 con 0x7f1578102230 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.125+0000 7f157ec12640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f157810c4b0 con 0x7f1578100680 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45334/0 (socket says 172.21.15.33:45334) 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 -- 172.21.15.33:0/3776816295 learned_addr learned my addr 172.21.15.33:0/3776816295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 0x7f15781045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f157d1af640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1578102230 0x7f1578103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 -- 172.21.15.33:0/3776816295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 msgr2=0x7f15781045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 0x7f15781045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 -- 172.21.15.33:0/3776816295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1578102230 msgr2=0x7f1578103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1578102230 0x7f1578103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 -- 172.21.15.33:0/3776816295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1578103a50 con 0x7f1578101870 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 0x7f15781045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1577fff640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578102c50 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1564031440 tx=0x7f1564032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1575ffb640 1 -- 172.21.15.33:0/3776816295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156403f030 con 0x7f1578101870 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1575ffb640 1 -- 172.21.15.33:0/3776816295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1564003ce0 con 0x7f1578101870 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15781a1bf0 con 0x7f1578101870 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.126+0000 7f1575ffb640 1 -- 172.21.15.33:0/3776816295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564035070 con 0x7f1578101870 2024-04-24T23:29:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.127+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15781a2100 con 0x7f1578101870 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.129+0000 7f1575ffb640 1 -- 172.21.15.33:0/3776816295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1564042030 con 0x7f1578101870 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.129+0000 7f15819fb740 1 Processor -- start 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.129+0000 7f15819fb740 1 -- start start 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f15819fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f15819fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c58280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f15819fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c5ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c58280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157d1af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c58280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45346/0 (socket says 172.21.15.33:45346) 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 -- 172.21.15.33:0/3413152772 learned_addr learned my addr 172.21.15.33:0/3413152772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f15819fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e552bb62c0 con 0x55e552c53780 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e552b5e910 con 0x55e552c52d90 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e552bb2280 con 0x55e552c588f0 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f1575ffb640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15680a2c10 0x7f15680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f1577fff640 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c5ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157d1af640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15680a2c10 0x7f15680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 -- 172.21.15.33:0/3413152772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 msgr2=0x55e552c5ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c5ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 -- 172.21.15.33:0/3413152772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 msgr2=0x55e552c531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157c9ae640 1 -- 172.21.15.33:0/3413152772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e552b6ffb0 con 0x55e552c53780 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f157d1af640 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.130+0000 7f1575ffb640 1 -- 172.21.15.33:0/3776816295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15640ef7b0 con 0x7f1578101870 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.131+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c58280 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f156c0114f0 tx=0x7f156c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15477f033ca9d8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.131+0000 7f157d1af640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15680a2c10 0x7f15680a50d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55e552c53fe0 tx=0x7f1570004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.131+0000 7f1562ffd640 1 -- 172.21.15.33:0/3413152772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c01e070 con 0x55e552c53780 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.131+0000 7f1562ffd640 1 -- 172.21.15.33:0/3413152772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f156c00faa0 con 0x55e552c53780 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.131+0000 7f1562ffd640 1 -- 172.21.15.33:0/3413152772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c019c70 con 0x55e552c53780 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 msgr2=0x55e552c58280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c58280 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f156c0114f0 tx=0x7f156c0119b0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 shutdown_connections 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c5ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c58280 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 --2- 172.21.15.33:0/3413152772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 >> 172.21.15.33:0/3413152772 conn(0x55e552c1a950 msgr2=0x55e552c4e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 shutdown_connections 2024-04-24T23:29:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.132+0000 7f15819fb740 1 -- 172.21.15.33:0/3413152772 wait complete. 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 -- start start 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 Processor -- start 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c52d90 0x55e552c5d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c5d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e552c5b8a0 con 0x55e552c52d90 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e552b71e90 con 0x55e552c53780 2024-04-24T23:29:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e552c5ba20 con 0x55e552c588f0 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f1577fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c5d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f157c9ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c52d90 0x55e552c5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f1577fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c5d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60960/0 (socket says 172.21.15.33:60960) 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f1577fff640 1 -- 172.21.15.33:0/3647916578 learned_addr learned my addr 172.21.15.33:0/3647916578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f157d1af640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.133+0000 7f15819fb740 10 client.0 _ll_register_callbacks cb 0x55e552bad970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1577fff640 1 -- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 msgr2=0x55e552c64800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1577fff640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c64800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1577fff640 1 -- 172.21.15.33:0/3647916578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c52d90 msgr2=0x55e552c5d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1577fff640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c52d90 0x55e552c5d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1577fff640 1 -- 172.21.15.33:0/3647916578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5529927e0 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f157d1af640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1577fff640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c5d7c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f15640bba30 tx=0x7f1564032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a900ed2c53b536d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1557fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.134+0000 7f1557fff640 1 -- 172.21.15.33:0/3647916578 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15640bbd30 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f1557fff640 1 -- 172.21.15.33:0/3647916578 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f156403bcd0 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f156c00faa0 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f1557fff640 1 -- 172.21.15.33:0/3647916578 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1564049030 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 10 client.28610 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e5529f2f70 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f1557fff640 1 -- 172.21.15.33:0/3647916578 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f156c00faa0 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f1557fff640 1 -- 172.21.15.33:0/3647916578 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f15640d9050 con 0x55e552c53780 2024-04-24T23:29:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 10 client.28610 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 10 client.28610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 20 client.28610 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552c68c80 0x55e552c6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 4 client.28610 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.135+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e552b8a5a0 con 0x55e552c68c80 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.136+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552c68c80 0x55e552c6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.136+0000 7f1557fff640 10 client.28610 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.136+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552c68c80 0x55e552c6b0f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c56a9e9ea1a93c server_cookie=833caba2fb605847 in_seq=0 out_seq=0 2024-04-24T23:29:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f1557fff640 1 -- 172.21.15.33:0/3647916578 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e552b8a5a0 con 0x55e552c68c80 2024-04-24T23:29:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f1557fff640 10 client.28610 handle_command_reply: tid=0 2024-04-24T23:29:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 client.28610 shutdown 2024-04-24T23:29:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 msgr2=0x55e552c5d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c5d7c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f15640bba30 tx=0x7f1564032f90 comp rx=0 tx=0).stop 2024-04-24T23:29:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 shutdown_connections 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552c68c80 0x55e552c6b0f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c5d7c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f1557fff640 0 client.28610 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 --2- 172.21.15.33:0/3647916578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c52d90 0x55e552c5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 >> 172.21.15.33:0/3647916578 conn(0x55e552c1a950 msgr2=0x55e552c4d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 shutdown_connections 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 1 -- 172.21.15.33:0/3647916578 wait complete. 2024-04-24T23:29:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.137+0000 7f15819fb740 20 client.28610 trim_cache size 0 max 16384 2024-04-24T23:29:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.166+0000 7f15819fb740 1 Processor -- start 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.166+0000 7f15819fb740 1 -- start start 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.166+0000 7f15819fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c52d90 0x55e552c51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.166+0000 7f15819fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.166+0000 7f15819fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c588f0 0x55e552c6f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f15819fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15781aef00 con 0x55e552c588f0 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f15819fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ac7d0 con 0x55e552c53780 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f15819fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15781aa620 con 0x55e552c52d90 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60962/0 (socket says 172.21.15.33:60962) 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 -- 172.21.15.33:0/2658309386 learned_addr learned my addr 172.21.15.33:0/2658309386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f157c9ae640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c52d90 0x55e552c51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f157d1af640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c588f0 0x55e552c6f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 -- 172.21.15.33:0/2658309386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c52d90 msgr2=0x55e552c51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c52d90 0x55e552c51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 -- 172.21.15.33:0/2658309386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c588f0 msgr2=0x55e552c6f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c588f0 0x55e552c6f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 -- 172.21.15.33:0/2658309386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55288a170 con 0x55e552c53780 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f157d1af640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c588f0 0x55e552c6f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f157c9ae640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c52d90 0x55e552c51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.167+0000 7f1577fff640 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c647c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f15640bbcc0 tx=0x7f15640bbc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147b0c7462f18dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.168+0000 7f15627fc640 1 -- 172.21.15.33:0/2658309386 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15640bb870 con 0x55e552c53780 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.168+0000 7f15627fc640 1 -- 172.21.15.33:0/2658309386 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15640bbd30 con 0x55e552c53780 2024-04-24T23:29:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.168+0000 7f15627fc640 1 -- 172.21.15.33:0/2658309386 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15640bc060 con 0x55e552c53780 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 -- 172.21.15.33:0/2658309386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 msgr2=0x55e552c647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c647c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f15640bbcc0 tx=0x7f15640bbc30 comp rx=0 tx=0).stop 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 -- 172.21.15.33:0/2658309386 shutdown_connections 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c588f0 0x55e552c6f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c53780 0x55e552c647c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 --2- 172.21.15.33:0/2658309386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c52d90 0x55e552c51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 -- 172.21.15.33:0/2658309386 >> 172.21.15.33:0/2658309386 conn(0x55e552c1a950 msgr2=0x55e552c51700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 -- 172.21.15.33:0/2658309386 shutdown_connections 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 -- 172.21.15.33:0/2658309386 wait complete. 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.169+0000 7f15819fb740 1 -- start start 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 Processor -- start 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c62ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15781aac90 con 0x55e552c53780 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ac100 con 0x55e552c52d90 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15781ab360 con 0x55e552c588f0 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f15819fb740 10 client.0 _ll_register_callbacks cb 0x7f1564005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f157d1af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c62ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45376/0 (socket says 172.21.15.33:45376) 2024-04-24T23:29:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.170+0000 7f1577fff640 1 -- 172.21.15.33:0/3942339723 learned_addr learned my addr 172.21.15.33:0/3942339723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1577fff640 1 -- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 msgr2=0x55e552c62ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1577fff640 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c62ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1577fff640 1 -- 172.21.15.33:0/3942339723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 msgr2=0x55e552c643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1577fff640 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1577fff640 1 -- 172.21.15.33:0/3942339723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e552c63430 con 0x55e552c53780 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f157d1af640 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c62ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1577fff640 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c629b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1564005ec0 tx=0x7f15640c4670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f760e27c189e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.171+0000 7f1557fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.172+0000 7f1557fff640 1 -- 172.21.15.33:0/3942339723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15640bb870 con 0x55e552c53780 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.172+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15640bbd30 con 0x55e552c53780 2024-04-24T23:29:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.172+0000 7f1557fff640 1 -- 172.21.15.33:0/3942339723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15640da600 con 0x55e552c53780 2024-04-24T23:29:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.172+0000 7f1557fff640 1 -- 172.21.15.33:0/3942339723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f15640b7fd0 con 0x55e552c53780 2024-04-24T23:29:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.174+0000 7f15819fb740 10 client.19119 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.174+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e552c63430 con 0x55e552c53780 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f1557fff640 1 -- 172.21.15.33:0/3942339723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15640bbd30 con 0x55e552c53780 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f1557fff640 1 -- 172.21.15.33:0/3942339723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f15640d9070 con 0x55e552c53780 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f15819fb740 10 client.19119 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f15819fb740 10 client.19119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f15819fb740 20 client.19119 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f15819fb740 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552cd8e70 0x55e552cdb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f15819fb740 4 client.19119 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e552bc3800 con 0x55e552cd8e70 2024-04-24T23:29:28.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.175+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552cd8e70 0x55e552cdb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.176+0000 7f1557fff640 10 client.19119 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.177+0000 7f157c9ae640 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552cd8e70 0x55e552cdb290 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b7fa11b6256545e server_cookie=8f0b0d540106764 in_seq=0 out_seq=0 2024-04-24T23:29:28.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.178+0000 7f1557fff640 1 -- 172.21.15.33:0/3942339723 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e552bc3800 con 0x55e552cd8e70 2024-04-24T23:29:28.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.178+0000 7f1557fff640 10 client.19119 handle_command_reply: tid=0 2024-04-24T23:29:28.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.178+0000 7f15819fb740 1 client.19119 shutdown 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.178+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 msgr2=0x55e552c629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.178+0000 7f15819fb740 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c629b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1564005ec0 tx=0x7f15640c4670 comp rx=0 tx=0).stop 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 shutdown_connections 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e552cd8e70 0x55e552cdb290 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f1557fff640 0 client.19119 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e552c588f0 0x55e552c62ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e552c53780 0x55e552c629b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 --2- 172.21.15.33:0/3942339723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e552c52d90 0x55e552c643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 >> 172.21.15.33:0/3942339723 conn(0x55e552c1a950 msgr2=0x55e552c4d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 shutdown_connections 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 1 -- 172.21.15.33:0/3942339723 wait complete. 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.179+0000 7f15819fb740 20 client.19119 trim_cache size 0 max 16384 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "c9cb574b-bacc-4332-93e5-75c2d61fd85b", 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:29:28.181 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:29:28.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.181+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15680a2c10 msgr2=0x7f15680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.181+0000 7f157ec12640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15680a2c10 0x7f15680a50d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55e552c53fe0 tx=0x7f1570004ab0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.181+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 msgr2=0x7f1578102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.181+0000 7f157ec12640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578102c50 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1564031440 tx=0x7f1564032ce0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.182+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 shutdown_connections 2024-04-24T23:29:28.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.182+0000 7f157ec12640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f15680a2c10 0x7f15680a50d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.182+0000 7f157ec12640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1578102230 0x7f1578103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.182+0000 7f157ec12640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1578101870 0x7f1578102c50 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.182+0000 7f157ec12640 1 --2- 172.21.15.33:0/3776816295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578100680 0x7f15781045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.182+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 >> 172.21.15.33:0/3776816295 conn(0x7f15780fbbf0 msgr2=0x7f15780fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.183+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 shutdown_connections 2024-04-24T23:29:28.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.184+0000 7f157ec12640 1 -- 172.21.15.33:0/3776816295 wait complete. 2024-04-24T23:29:28.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:29:28.202 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:29:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.375+0000 7fddb6612640 1 -- 172.21.15.33:0/853197926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0100680 msgr2=0x7fddb0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.375+0000 7fddb6612640 1 --2- 172.21.15.33:0/853197926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0100680 0x7fddb0100aa0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fdda001a910 tx=0x7fdda003b450 comp rx=0 tx=0).stop 2024-04-24T23:29:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.377+0000 7fddb6612640 1 -- 172.21.15.33:0/853197926 shutdown_connections 2024-04-24T23:29:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.377+0000 7fddb6612640 1 --2- 172.21.15.33:0/853197926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0102230 0x7fddb0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.377+0000 7fddb6612640 1 --2- 172.21.15.33:0/853197926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddb0101870 0x7fddb0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.377+0000 7fddb6612640 1 --2- 172.21.15.33:0/853197926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0100680 0x7fddb0100aa0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.377+0000 7fddb6612640 1 -- 172.21.15.33:0/853197926 >> 172.21.15.33:0/853197926 conn(0x7fddb00fbbf0 msgr2=0x7fddb00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.380+0000 7fddb6612640 1 -- 172.21.15.33:0/853197926 shutdown_connections 2024-04-24T23:29:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.380+0000 7fddb6612640 1 -- 172.21.15.33:0/853197926 wait complete. 2024-04-24T23:29:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.381+0000 7fddb6612640 1 Processor -- start 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.381+0000 7fddb6612640 1 -- start start 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb6612640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddb0100680 0x7fddb0072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb6612640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0101870 0x7fddb0073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb6612640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0102230 0x7fddb0071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb6612640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddb006bbb0 con 0x7fddb0102230 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb6612640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddb006ba30 con 0x7fddb0100680 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb6612640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb006bd30 con 0x7fddb0101870 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0101870 0x7fddb0073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0101870 0x7fddb0073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57096/0 (socket says 172.21.15.33:57096) 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddaf7fe640 1 -- 172.21.15.33:0/2036877358 learned_addr learned my addr 172.21.15.33:0/2036877358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.382+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0102230 0x7fddb0071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.383+0000 7fddb4baf640 1 -- 172.21.15.33:0/2036877358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0101870 msgr2=0x7fddb0073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.383+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0101870 0x7fddb0073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.383+0000 7fddb4baf640 1 -- 172.21.15.33:0/2036877358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddb0100680 msgr2=0x7fddb0072d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.383+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddb0100680 0x7fddb0072d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.383+0000 7fddb4baf640 1 -- 172.21.15.33:0/2036877358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0071b40 con 0x7fddb0102230 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.383+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0102230 0x7fddb0071340 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fdda800be00 tx=0x7fdda800bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.384+0000 7fddad7fa640 1 -- 172.21.15.33:0/2036877358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8019070 con 0x7fddb0102230 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.384+0000 7fddad7fa640 1 -- 172.21.15.33:0/2036877358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda8014920 con 0x7fddb0102230 2024-04-24T23:29:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.384+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb0071e30 con 0x7fddb0102230 2024-04-24T23:29:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.385+0000 7fddad7fa640 1 -- 172.21.15.33:0/2036877358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8019070 con 0x7fddb0102230 2024-04-24T23:29:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.385+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb01a6790 con 0x7fddb0102230 2024-04-24T23:29:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.388+0000 7fddad7fa640 1 -- 172.21.15.33:0/2036877358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdda801c070 con 0x7fddb0102230 2024-04-24T23:29:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.389+0000 7fddb93fc740 1 Processor -- start 2024-04-24T23:29:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.390+0000 7fddb93fc740 1 -- start start 2024-04-24T23:29:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.390+0000 7fddad7fa640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd980a2c50 0x7fdd980a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.390+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.390+0000 7fddad7fa640 1 -- 172.21.15.33:0/2036877358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdda80ce990 con 0x7fddb0102230 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddaf7fe640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd980a2c50 0x7fdd980a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57098/0 (socket says 172.21.15.33:57098) 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddaf7fe640 1 -- 172.21.15.33:0/2377679659 learned_addr learned my addr 172.21.15.33:0/2377679659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cbc6146b0 con 0x556cbc6b0240 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556cbc5bc880 con 0x556cbc6b5da0 2024-04-24T23:29:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.391+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556cbc610670 con 0x556cbc6b0c30 2024-04-24T23:29:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.392+0000 7fddaffff640 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.392+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.393+0000 7fddb4baf640 1 -- 172.21.15.33:0/2377679659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 msgr2=0x556cbc6b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.393+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.393+0000 7fddb4baf640 1 -- 172.21.15.33:0/2377679659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 msgr2=0x556cbc6b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.393+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.393+0000 7fddb4baf640 1 -- 172.21.15.33:0/2377679659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbc3f27d0 con 0x556cbc6b5da0 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.393+0000 7fddaf7fe640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd980a2c50 0x7fdd980a5110 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x556cbc6b1030 tx=0x7fdda4009900 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.394+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6b8230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdda80cc470 tx=0x7fdda8014d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a379955478ad8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.395+0000 7fdd967fc640 1 -- 172.21.15.33:0/2377679659 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda809ae60 con 0x556cbc6b5da0 2024-04-24T23:29:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.395+0000 7fdd967fc640 1 -- 172.21.15.33:0/2377679659 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda809f210 con 0x556cbc6b5da0 2024-04-24T23:29:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.395+0000 7fdd967fc640 1 -- 172.21.15.33:0/2377679659 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80a38a0 con 0x556cbc6b5da0 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.396+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 msgr2=0x556cbc6b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6b8230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdda80cc470 tx=0x7fdda8014d60 comp rx=0 tx=0).stop 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 shutdown_connections 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6b8230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2377679659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 >> 172.21.15.33:0/2377679659 conn(0x556cbc6ab8b0 msgr2=0x556cbc6abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 shutdown_connections 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.397+0000 7fddb93fc740 1 -- 172.21.15.33:0/2377679659 wait complete. 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 -- start start 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 Processor -- start 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b0240 0x556cbc6ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0c30 0x556cbc6bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 0x556cbc6c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cbc6b8d50 con 0x556cbc6b0c30 2024-04-24T23:29:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556cbc5cff50 con 0x556cbc6b0240 2024-04-24T23:29:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556cbc6b8ed0 con 0x556cbc6b5da0 2024-04-24T23:29:28.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.398+0000 7fddb93fc740 10 client.0 _ll_register_callbacks cb 0x556cbc60b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0c30 0x556cbc6bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 0x556cbc6c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 0x556cbc6c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57112/0 (socket says 172.21.15.33:57112) 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 -- 172.21.15.33:0/1129650458 learned_addr learned my addr 172.21.15.33:0/1129650458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddaffff640 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b0240 0x556cbc6ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 -- 172.21.15.33:0/1129650458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b0240 msgr2=0x556cbc6ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b0240 0x556cbc6ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 -- 172.21.15.33:0/1129650458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0c30 msgr2=0x556cbc6bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0c30 0x556cbc6bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 -- 172.21.15.33:0/1129650458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbc3283f0 con 0x556cbc6b5da0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fdd8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.399+0000 7fddb4baf640 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 0x556cbc6c1c50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdda8097270 tx=0x7fdda8004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d771caf98270d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/1129650458 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda809ae60 con 0x556cbc6b5da0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda809f210 con 0x556cbc6b5da0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/1129650458 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdda80b9020 con 0x556cbc6b5da0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/1129650458 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdda809ae60 con 0x556cbc6b5da0 2024-04-24T23:29:28.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fddb93fc740 10 client.28552 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556cbc2e80f0 con 0x556cbc6b5da0 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.400+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/1129650458 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda809f210 con 0x556cbc6b5da0 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/1129650458 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdda8009020 con 0x556cbc6b5da0 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddb93fc740 10 client.28552 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddb93fc740 10 client.28552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddb93fc740 20 client.28552 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddb93fc740 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc6c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddb93fc740 4 client.28552 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cbc621bf0 con 0x556cbc6c6140 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddaffff640 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc6c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fdd8f7fe640 10 client.28552 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.401+0000 7fddaffff640 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc6c8530 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92db07730822626c server_cookie=8a352fc4aad3b59c in_seq=0 out_seq=0 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.402+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/1129650458 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cbc621bf0 con 0x556cbc6c6140 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.402+0000 7fdd8f7fe640 10 client.28552 handle_command_reply: tid=0 2024-04-24T23:29:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.402+0000 7fddb93fc740 1 client.28552 shutdown 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 msgr2=0x556cbc6c1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 0x556cbc6c1c50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdda8097270 tx=0x7fdda8004a80 comp rx=0 tx=0).stop 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 shutdown_connections 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc6c8530 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b5da0 0x556cbc6c1c50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fdd8f7fe640 0 client.28552 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0c30 0x556cbc6bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 --2- 172.21.15.33:0/1129650458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b0240 0x556cbc6ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 >> 172.21.15.33:0/1129650458 conn(0x556cbc6ab8b0 msgr2=0x556cbc6aaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 shutdown_connections 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 1 -- 172.21.15.33:0/1129650458 wait complete. 2024-04-24T23:29:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.403+0000 7fddb93fc740 20 client.28552 trim_cache size 0 max 16384 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 Processor -- start 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 -- start start 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01b32a0 con 0x556cbc6b0240 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddaffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddaffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45434/0 (socket says 172.21.15.33:45434) 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddaffff640 1 -- 172.21.15.33:0/2041778100 learned_addr learned my addr 172.21.15.33:0/2041778100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddaf7fe640 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01b0b70 con 0x556cbc6b5da0 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01ae9c0 con 0x556cbc6b0c30 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.423+0000 7fddb4baf640 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaffff640 1 -- 172.21.15.33:0/2041778100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 msgr2=0x556cbc6c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaffff640 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaffff640 1 -- 172.21.15.33:0/2041778100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 msgr2=0x556cbc6ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaffff640 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaffff640 1 -- 172.21.15.33:0/2041778100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbc69d090 con 0x556cbc6b0240 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaf7fe640 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fddaffff640 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b9190 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdda001a910 tx=0x7fdda0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82678478aab845a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fdd95ffb640 1 -- 172.21.15.33:0/2041778100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0046040 con 0x556cbc6b0240 2024-04-24T23:29:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fdd95ffb640 1 -- 172.21.15.33:0/2041778100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0056330 con 0x556cbc6b0240 2024-04-24T23:29:28.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.424+0000 7fdd95ffb640 1 -- 172.21.15.33:0/2041778100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0005350 con 0x556cbc6b0240 2024-04-24T23:29:28.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 msgr2=0x556cbc6b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b9190 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdda001a910 tx=0x7fdda0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 shutdown_connections 2024-04-24T23:29:28.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 --2- 172.21.15.33:0/2041778100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6b9190 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 >> 172.21.15.33:0/2041778100 conn(0x556cbc6ab8b0 msgr2=0x556cbc6b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 shutdown_connections 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.425+0000 7fddb93fc740 1 -- 172.21.15.33:0/2041778100 wait complete. 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddb93fc740 1 -- start start 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddb93fc740 1 Processor -- start 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddb93fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddaffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.426+0000 7fddaffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6beea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45448/0 (socket says 172.21.15.33:45448) 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 -- 172.21.15.33:0/3939444528 learned_addr learned my addr 172.21.15.33:0/3939444528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01af030 con 0x556cbc6b0240 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01b04a0 con 0x556cbc6b5da0 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb01af700 con 0x556cbc6b0c30 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb4baf640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb93fc740 10 client.0 _ll_register_callbacks cb 0x7fdda4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 -- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 msgr2=0x556cbc6bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 -- 172.21.15.33:0/3939444528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 msgr2=0x556cbc6c03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6c03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbc6c09e0 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb4baf640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddaffff640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6beea0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005ab0 tx=0x7fdda0056c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26b999322b426b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fdd8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/3939444528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0063070 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/3939444528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdda0004b30 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.427+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda0056330 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/3939444528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdda0070050 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 10 client.19137 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556cbc6c1cf0 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/3939444528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0056330 con 0x556cbc6b0240 2024-04-24T23:29:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/3939444528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdda0040510 con 0x556cbc6b0240 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 10 client.19137 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 10 client.19137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 20 client.19137 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc72e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 4 client.19137 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556cbc61bf40 con 0x556cbc6c6140 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.428+0000 7fddaf7fe640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc72e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fdd8f7fe640 10 client.19137 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddaf7fe640 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc72e9f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0d6a489744d814 server_cookie=f3961703372d3d82 in_seq=0 out_seq=0 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fdd8f7fe640 1 -- 172.21.15.33:0/3939444528 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556cbc61bf40 con 0x556cbc6c6140 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fdd8f7fe640 10 client.19137 handle_command_reply: tid=0 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 client.19137 shutdown 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 msgr2=0x556cbc6beea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6beea0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005ab0 tx=0x7fdda0056c20 comp rx=0 tx=0).stop 2024-04-24T23:29:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 shutdown_connections 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556cbc6c6140 0x556cbc72e9f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556cbc6b5da0 0x556cbc6c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556cbc6b0c30 0x556cbc6bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 --2- 172.21.15.33:0/3939444528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cbc6b0240 0x556cbc6beea0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fdd8f7fe640 0 client.19137 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 >> 172.21.15.33:0/3939444528 conn(0x556cbc6ab8b0 msgr2=0x556cbc6aa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 shutdown_connections 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 1 -- 172.21.15.33:0/3939444528 wait complete. 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.429+0000 7fddb93fc740 20 client.19137 trim_cache size 0 max 16384 2024-04-24T23:29:28.437 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "c9cb574b-bacc-4332-93e5-75c2d61fd85b": { 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "c9cb574b-bacc-4332-93e5-75c2d61fd85b", 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:29:28.438 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:29:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd980a2c50 msgr2=0x7fdd980a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd980a2c50 0x7fdd980a5110 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x556cbc6b1030 tx=0x7fdda4009900 comp rx=0 tx=0).stop 2024-04-24T23:29:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0102230 msgr2=0x7fddb0071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0102230 0x7fddb0071340 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fdda800be00 tx=0x7fdda800bef0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 shutdown_connections 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd980a2c50 0x7fdd980a5110 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddb0102230 0x7fddb0071340 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0101870 0x7fddb0073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 --2- 172.21.15.33:0/2036877358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddb0100680 0x7fddb0072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 >> 172.21.15.33:0/2036877358 conn(0x7fddb00fbbf0 msgr2=0x7fddb0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 shutdown_connections 2024-04-24T23:29:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:28.430+0000 7fddb6612640 1 -- 172.21.15.33:0/2036877358 wait complete. 2024-04-24T23:29:28.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:28 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:29:28.443 INFO:tasks.cephfs.filesystem:scrub status for tag:c9cb574b-bacc-4332-93e5-75c2d61fd85b - {'path': '/', 'tag': 'c9cb574b-bacc-4332-93e5-75c2d61fd85b', 'options': 'recursive,force'} 2024-04-24T23:29:28.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:28 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:29:28.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:28 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:29:28.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 93735 22.48 MB/sec execute 93 sec latency 130.256 ms 2024-04-24T23:29:29.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:29 smithi038 ceph-mon[32450]: pgmap v875: 129 pgs: 129 active+clean; 524 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 385 op/s 2024-04-24T23:29:29.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:29 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:29:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:29 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:29.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:29 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:29:29.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:29 smithi117 ceph-mon[32529]: pgmap v875: 129 pgs: 129 active+clean; 524 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 385 op/s 2024-04-24T23:29:29.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:29 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:29:29.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:29 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:29.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:29 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:29:29.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:29 smithi033 ceph-mon[29201]: pgmap v875: 129 pgs: 129 active+clean; 524 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 385 op/s 2024-04-24T23:29:29.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:29 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:29:29.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:29 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:29.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:29 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:29:29.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 94215 22.57 MB/sec execute 94 sec latency 80.056 ms 2024-04-24T23:29:30.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:29:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:29:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:29:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:29:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:29:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:29:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:30.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 94777 22.68 MB/sec execute 95 sec latency 68.013 ms 2024-04-24T23:29:31.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:31 smithi038 ceph-mon[32450]: pgmap v876: 129 pgs: 129 active+clean; 523 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-04-24T23:29:31.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:31 smithi117 ceph-mon[32529]: pgmap v876: 129 pgs: 129 active+clean; 523 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-04-24T23:29:31.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:31 smithi033 ceph-mon[29201]: pgmap v876: 129 pgs: 129 active+clean; 523 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-04-24T23:29:31.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 95303 22.75 MB/sec execute 96 sec latency 83.812 ms 2024-04-24T23:29:32.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 95654 22.69 MB/sec execute 97 sec latency 280.576 ms 2024-04-24T23:29:33.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:33 smithi038 ceph-mon[32450]: pgmap v877: 129 pgs: 129 active+clean; 529 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.9 MiB/s wr, 492 op/s 2024-04-24T23:29:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:33 smithi117 ceph-mon[32529]: pgmap v877: 129 pgs: 129 active+clean; 529 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.9 MiB/s wr, 492 op/s 2024-04-24T23:29:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:33 smithi033 ceph-mon[29201]: pgmap v877: 129 pgs: 129 active+clean; 529 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.9 MiB/s wr, 492 op/s 2024-04-24T23:29:33.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 96010 22.67 MB/sec execute 98 sec latency 217.795 ms 2024-04-24T23:29:34.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 96328 22.59 MB/sec execute 99 sec latency 170.718 ms 2024-04-24T23:29:35.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:35 smithi038 ceph-mon[32450]: pgmap v878: 129 pgs: 129 active+clean; 531 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 441 op/s 2024-04-24T23:29:35.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:35 smithi117 ceph-mon[32529]: pgmap v878: 129 pgs: 129 active+clean; 531 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 441 op/s 2024-04-24T23:29:35.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:35 smithi033 ceph-mon[29201]: pgmap v878: 129 pgs: 129 active+clean; 531 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 441 op/s 2024-04-24T23:29:35.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 96819 22.63 MB/sec execute 100 sec latency 87.760 ms 2024-04-24T23:29:36.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 97316 22.62 MB/sec execute 101 sec latency 118.545 ms 2024-04-24T23:29:37.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:37 smithi038 ceph-mon[32450]: pgmap v879: 129 pgs: 129 active+clean; 534 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 484 op/s 2024-04-24T23:29:37.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:37 smithi117 ceph-mon[32529]: pgmap v879: 129 pgs: 129 active+clean; 534 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 484 op/s 2024-04-24T23:29:37.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:37 smithi033 ceph-mon[29201]: pgmap v879: 129 pgs: 129 active+clean; 534 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 484 op/s 2024-04-24T23:29:37.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 97609 22.55 MB/sec execute 102 sec latency 279.230 ms 2024-04-24T23:29:38.887 INFO:tasks.workunit.client.0.smithi033.stdout: 10 97976 22.51 MB/sec execute 103 sec latency 341.783 ms 2024-04-24T23:29:39.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:39 smithi038 ceph-mon[32450]: pgmap v880: 129 pgs: 129 active+clean; 535 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 460 op/s 2024-04-24T23:29:39.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:39 smithi117 ceph-mon[32529]: pgmap v880: 129 pgs: 129 active+clean; 535 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 460 op/s 2024-04-24T23:29:39.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:39 smithi033 ceph-mon[29201]: pgmap v880: 129 pgs: 129 active+clean; 535 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 460 op/s 2024-04-24T23:29:39.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 98721 22.71 MB/sec execute 104 sec latency 64.217 ms 2024-04-24T23:29:40.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 99251 22.85 MB/sec execute 105 sec latency 84.461 ms 2024-04-24T23:29:41.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:41 smithi038 ceph-mon[32450]: pgmap v881: 129 pgs: 129 active+clean; 537 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 434 op/s 2024-04-24T23:29:41.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:41 smithi117 ceph-mon[32529]: pgmap v881: 129 pgs: 129 active+clean; 537 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 434 op/s 2024-04-24T23:29:41.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:41 smithi033 ceph-mon[29201]: pgmap v881: 129 pgs: 129 active+clean; 537 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 434 op/s 2024-04-24T23:29:41.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 99594 22.81 MB/sec execute 106 sec latency 84.057 ms 2024-04-24T23:29:42.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 99881 22.72 MB/sec execute 107 sec latency 142.785 ms 2024-04-24T23:29:43.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:43 smithi038 ceph-mon[32450]: pgmap v882: 129 pgs: 129 active+clean; 533 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-04-24T23:29:43.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:29:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:29:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:29:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:43 smithi117 ceph-mon[32529]: pgmap v882: 129 pgs: 129 active+clean; 533 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-04-24T23:29:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:29:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:29:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:29:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:43 smithi033 ceph-mon[29201]: pgmap v882: 129 pgs: 129 active+clean; 533 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-04-24T23:29:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:29:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:29:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:29:43.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 100122 22.58 MB/sec execute 108 sec latency 317.069 ms 2024-04-24T23:29:44.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 100536 22.63 MB/sec execute 109 sec latency 86.188 ms 2024-04-24T23:29:45.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:45 smithi038 ceph-mon[32450]: pgmap v883: 129 pgs: 129 active+clean; 519 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 401 op/s 2024-04-24T23:29:45.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:45 smithi117 ceph-mon[32529]: pgmap v883: 129 pgs: 129 active+clean; 519 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 401 op/s 2024-04-24T23:29:45.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:45 smithi033 ceph-mon[29201]: pgmap v883: 129 pgs: 129 active+clean; 519 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 401 op/s 2024-04-24T23:29:45.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 100988 22.63 MB/sec execute 110 sec latency 67.242 ms 2024-04-24T23:29:46.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 101620 22.67 MB/sec execute 111 sec latency 53.085 ms 2024-04-24T23:29:47.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:47 smithi038 ceph-mon[32450]: pgmap v884: 129 pgs: 129 active+clean; 524 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 440 op/s 2024-04-24T23:29:47.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:47 smithi117 ceph-mon[32529]: pgmap v884: 129 pgs: 129 active+clean; 524 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 440 op/s 2024-04-24T23:29:47.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:47 smithi033 ceph-mon[29201]: pgmap v884: 129 pgs: 129 active+clean; 524 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 440 op/s 2024-04-24T23:29:47.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 102177 22.75 MB/sec execute 112 sec latency 56.082 ms 2024-04-24T23:29:48.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 102489 22.69 MB/sec execute 113 sec latency 366.347 ms 2024-04-24T23:29:49.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:49 smithi038 ceph-mon[32450]: pgmap v885: 129 pgs: 129 active+clean; 525 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 393 op/s 2024-04-24T23:29:49.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:49 smithi117 ceph-mon[32529]: pgmap v885: 129 pgs: 129 active+clean; 525 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 393 op/s 2024-04-24T23:29:49.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:49 smithi033 ceph-mon[29201]: pgmap v885: 129 pgs: 129 active+clean; 525 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 393 op/s 2024-04-24T23:29:49.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 102919 22.76 MB/sec execute 114 sec latency 62.046 ms 2024-04-24T23:29:50.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 103313 22.73 MB/sec execute 115 sec latency 73.472 ms 2024-04-24T23:29:51.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:51 smithi038 ceph-mon[32450]: pgmap v886: 129 pgs: 129 active+clean; 525 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-04-24T23:29:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:51 smithi117 ceph-mon[32529]: pgmap v886: 129 pgs: 129 active+clean; 525 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-04-24T23:29:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:51 smithi033 ceph-mon[29201]: pgmap v886: 129 pgs: 129 active+clean; 525 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-04-24T23:29:51.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 103859 22.77 MB/sec execute 116 sec latency 72.770 ms 2024-04-24T23:29:52.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 104444 22.87 MB/sec execute 117 sec latency 131.232 ms 2024-04-24T23:29:53.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:53 smithi038 ceph-mon[32450]: pgmap v887: 129 pgs: 129 active+clean; 531 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-04-24T23:29:53.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:53 smithi117 ceph-mon[32529]: pgmap v887: 129 pgs: 129 active+clean; 531 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-04-24T23:29:53.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:53 smithi033 ceph-mon[29201]: pgmap v887: 129 pgs: 129 active+clean; 531 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-04-24T23:29:53.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 104614 22.74 MB/sec execute 118 sec latency 398.732 ms 2024-04-24T23:29:54.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 104933 22.73 MB/sec execute 119 sec latency 130.446 ms 2024-04-24T23:29:55.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:55 smithi038 ceph-mon[32450]: pgmap v888: 129 pgs: 129 active+clean; 536 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 398 op/s 2024-04-24T23:29:55.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:55 smithi117 ceph-mon[32529]: pgmap v888: 129 pgs: 129 active+clean; 536 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 398 op/s 2024-04-24T23:29:55.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:55 smithi033 ceph-mon[29201]: pgmap v888: 129 pgs: 129 active+clean; 536 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 398 op/s 2024-04-24T23:29:55.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 105304 22.65 MB/sec execute 120 sec latency 120.319 ms 2024-04-24T23:29:56.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 105874 22.67 MB/sec execute 121 sec latency 83.756 ms 2024-04-24T23:29:57.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:57 smithi038 ceph-mon[32450]: pgmap v889: 129 pgs: 129 active+clean; 537 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 432 op/s 2024-04-24T23:29:57.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:57 smithi117 ceph-mon[32529]: pgmap v889: 129 pgs: 129 active+clean; 537 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 432 op/s 2024-04-24T23:29:57.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:57 smithi033 ceph-mon[29201]: pgmap v889: 129 pgs: 129 active+clean; 537 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 432 op/s 2024-04-24T23:29:57.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 106290 22.69 MB/sec execute 122 sec latency 89.721 ms 2024-04-24T23:29:58.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:29:58.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:29:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.581+0000 7f6ec8790640 1 -- 172.21.15.33:0/2785986656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0102970 msgr2=0x7f6ec0102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.581+0000 7f6ec8790640 1 --2- 172.21.15.33:0/2785986656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0102970 0x7f6ec0102df0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009900 tx=0x7f6eb002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 -- 172.21.15.33:0/2785986656 shutdown_connections 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 --2- 172.21.15.33:0/2785986656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ec0103330 0x7f6ec0109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 --2- 172.21.15.33:0/2785986656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0102970 0x7f6ec0102df0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 --2- 172.21.15.33:0/2785986656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ec0101780 0x7f6ec0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 -- 172.21.15.33:0/2785986656 >> 172.21.15.33:0/2785986656 conn(0x7f6ec00fccf0 msgr2=0x7f6ec00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 -- 172.21.15.33:0/2785986656 shutdown_connections 2024-04-24T23:29:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.582+0000 7f6ec8790640 1 -- 172.21.15.33:0/2785986656 wait complete. 2024-04-24T23:29:58.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 Processor -- start 2024-04-24T23:29:58.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 -- start start 2024-04-24T23:29:58.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 0x7f6ec0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ec0102970 0x7f6ec0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ec0103330 0x7f6ec019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010d4d0 con 0x7f6ec0101780 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010d350 con 0x7f6ec0102970 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 0x7f6ec0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec8790640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010d650 con 0x7f6ec0103330 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 0x7f6ec0197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49544/0 (socket says 172.21.15.33:49544) 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 -- 172.21.15.33:0/1751319978 learned_addr learned my addr 172.21.15.33:0/1751319978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ec0102970 0x7f6ec0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ec0103330 0x7f6ec019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 -- 172.21.15.33:0/1751319978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ec0103330 msgr2=0x7f6ec019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ec0103330 0x7f6ec019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 -- 172.21.15.33:0/1751319978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ec0102970 msgr2=0x7f6ec0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ec0102970 0x7f6ec0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.583+0000 7f6ec652c640 1 -- 172.21.15.33:0/1751319978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec019f580 con 0x7f6ec0101780 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.584+0000 7f6ec652c640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 0x7f6ec0197960 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00d770 tx=0x7f6ebc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.584+0000 7f6eb77fe640 1 -- 172.21.15.33:0/1751319978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc017070 con 0x7f6ec0101780 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.584+0000 7f6eb77fe640 1 -- 172.21.15.33:0/1751319978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc009c30 con 0x7f6ec0101780 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.584+0000 7f6eb77fe640 1 -- 172.21.15.33:0/1751319978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc003dd0 con 0x7f6ec0101780 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.584+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec019f870 con 0x7f6ec0101780 2024-04-24T23:29:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.584+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec019fdf0 con 0x7f6ec0101780 2024-04-24T23:29:58.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.589+0000 7f6eb77fe640 1 -- 172.21.15.33:0/1751319978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6ebc003f70 con 0x7f6ec0101780 2024-04-24T23:29:58.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.591+0000 7f6ecb57c740 1 Processor -- start 2024-04-24T23:29:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.591+0000 7f6ecb57c740 1 -- start start 2024-04-24T23:29:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.591+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.591+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9af40f6b0 con 0x55c9af4abbb0 2024-04-24T23:29:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c9af3b7880 con 0x55c9af4aed10 2024-04-24T23:29:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c9af40b670 con 0x55c9af4ab1c0 2024-04-24T23:29:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ec5d2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ec5d2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4bbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33984/0 (socket says 172.21.15.33:33984) 2024-04-24T23:29:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.592+0000 7f6ec5d2b640 1 -- 172.21.15.33:0/2411921519 learned_addr learned my addr 172.21.15.33:0/2411921519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.593+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.593+0000 7f6eb77fe640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6eac0a2d60 0x7f6eac0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.593+0000 7f6eb77fe640 1 -- 172.21.15.33:0/1751319978 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ebc0cece0 con 0x7f6ec0101780 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.593+0000 7f6ec652c640 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.594+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6eac0a2d60 0x7f6eac0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.594+0000 7f6ec5d2b640 1 -- 172.21.15.33:0/2411921519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 msgr2=0x55c9af4ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.594+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.594+0000 7f6ec5d2b640 1 -- 172.21.15.33:0/2411921519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 msgr2=0x55c9af4ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.594+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.594+0000 7f6ec5d2b640 1 -- 172.21.15.33:0/2411921519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9af1ed7d0 con 0x55c9af4aed10 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.595+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6eac0a2d60 0x7f6eac0a5220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55c9af4ac410 tx=0x7f6eb800e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.595+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4bbb80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009900 tx=0x7f6eb0004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a009a0942ff6d821 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.595+0000 7f6eb4ff9640 1 -- 172.21.15.33:0/2411921519 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0035040 con 0x55c9af4aed10 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.595+0000 7f6eb4ff9640 1 -- 172.21.15.33:0/2411921519 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0004740 con 0x55c9af4aed10 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.596+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2411921519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 msgr2=0x55c9af4bbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.596+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4bbb80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009900 tx=0x7f6eb0004180 comp rx=0 tx=0).stop 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.597+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2411921519 shutdown_connections 2024-04-24T23:29:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.597+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4bbb80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.597+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.597+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2411921519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.597+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2411921519 >> 172.21.15.33:0/2411921519 conn(0x55c9af4a68b0 msgr2=0x55c9af4a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.599+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2411921519 shutdown_connections 2024-04-24T23:29:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.601+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2411921519 wait complete. 2024-04-24T23:29:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.601+0000 7f6ecb57c740 1 -- start start 2024-04-24T23:29:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.601+0000 7f6ecb57c740 1 Processor -- start 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.601+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.601+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.601+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9af4bc570 con 0x55c9af4abbb0 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c9af3caf50 con 0x55c9af4aed10 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c9af4bc6f0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ecb57c740 10 client.0 _ll_register_callbacks cb 0x55c9af406000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4be050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:32874/0 (socket says 172.21.15.33:32874) 2024-04-24T23:29:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ec652c640 1 -- 172.21.15.33:0/298605730 learned_addr learned my addr 172.21.15.33:0/298605730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.602+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.603+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.604+0000 7f6ec652c640 1 -- 172.21.15.33:0/298605730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 msgr2=0x55c9af4c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.604+0000 7f6ec652c640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.604+0000 7f6ec652c640 1 -- 172.21.15.33:0/298605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 msgr2=0x55c9af4be590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.604+0000 7f6ec652c640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4be590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.604+0000 7f6ec652c640 1 -- 172.21.15.33:0/298605730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9af1233f0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.605+0000 7f6ec652c640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4be050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cc760 tx=0x7f6ebc09ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89680fbef825a2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.605+0000 7f6ea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:29:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.605+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/298605730 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc020050 con 0x55c9af4ab1c0 2024-04-24T23:29:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.605+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/298605730 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ebc09f170 con 0x55c9af4ab1c0 2024-04-24T23:29:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.606+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c9af41cbf0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.607+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/298605730 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ebc009de0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.607+0000 7f6ecb57c740 10 client.28570 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.607+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9af0e30f0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.607+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/298605730 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c9af41cbf0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.607+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/298605730 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ebc0977c0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ecb57c740 10 client.28570 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ecb57c740 10 client.28570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ecb57c740 20 client.28570 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af4b37f0 0x55c9af4b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ecb57c740 4 client.28570 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9af416f40 con 0x55c9af4b37f0 2024-04-24T23:29:58.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af4b37f0 0x55c9af4b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.608+0000 7f6ea9ffb640 10 client.28570 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.609+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af4b37f0 0x55c9af4b5c80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a41f23dd63e873 server_cookie=2130ad6b80d0c8eb in_seq=0 out_seq=0 2024-04-24T23:29:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.610+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/298605730 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6eb004fae0 con 0x55c9af4b37f0 2024-04-24T23:29:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.610+0000 7f6ea9ffb640 10 client.28570 handle_command_reply: tid=0 2024-04-24T23:29:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.610+0000 7f6ecb57c740 1 client.28570 shutdown 2024-04-24T23:29:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.610+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 msgr2=0x55c9af4be050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.610+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4be050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cc760 tx=0x7f6ebc09ac90 comp rx=0 tx=0).stop 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 shutdown_connections 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af4b37f0 0x55c9af4b5c80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4abbb0 0x55c9af4be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/298605730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4ab1c0 0x55c9af4be050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 >> 172.21.15.33:0/298605730 conn(0x55c9af4a68b0 msgr2=0x55c9af4ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.618+0000 7f6ea9ffb640 0 client.28570 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.619+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 shutdown_connections 2024-04-24T23:29:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.619+0000 7f6ecb57c740 1 -- 172.21.15.33:0/298605730 wait complete. 2024-04-24T23:29:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.619+0000 7f6ecb57c740 20 client.28570 trim_cache size 0 max 16384 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 Processor -- start 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 -- start start 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4abbb0 0x55c9af4c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4aed10 0x55c9af4bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49576/0 (socket says 172.21.15.33:49576) 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ec652c640 1 -- 172.21.15.33:0/2154063356 learned_addr learned my addr 172.21.15.33:0/2154063356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01aa9e0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a82b0 con 0x55c9af4abbb0 2024-04-24T23:29:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a6100 con 0x55c9af4aed10 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.651+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4abbb0 0x55c9af4c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4aed10 0x55c9af4bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec652c640 1 -- 172.21.15.33:0/2154063356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4aed10 msgr2=0x55c9af4bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec652c640 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4aed10 0x55c9af4bd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec652c640 1 -- 172.21.15.33:0/2154063356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4abbb0 msgr2=0x55c9af4c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec652c640 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4abbb0 0x55c9af4c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec652c640 1 -- 172.21.15.33:0/2154063356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9af4abf90 con 0x55c9af4ab1c0 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.652+0000 7f6ec652c640 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4a5970 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc003820 tx=0x7f6ebc099660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95d101db428ce00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.653+0000 7f6eabfff640 1 -- 172.21.15.33:0/2154063356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc020030 con 0x55c9af4ab1c0 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.653+0000 7f6eabfff640 1 -- 172.21.15.33:0/2154063356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc015070 con 0x55c9af4ab1c0 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.653+0000 7f6eabfff640 1 -- 172.21.15.33:0/2154063356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc004380 con 0x55c9af4ab1c0 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 msgr2=0x55c9af4a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4a5970 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc003820 tx=0x7f6ebc099660 comp rx=0 tx=0).stop 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 shutdown_connections 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4aed10 0x55c9af4bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4abbb0 0x55c9af4c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/2154063356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4a5970 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 >> 172.21.15.33:0/2154063356 conn(0x55c9af4a68b0 msgr2=0x55c9af4a5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 shutdown_connections 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 -- 172.21.15.33:0/2154063356 wait complete. 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 -- start start 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.654+0000 7f6ecb57c740 1 Processor -- start 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4abbb0 0x55c9af4c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a6770 con 0x55c9af4ab1c0 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a7be0 con 0x55c9af4aed10 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a6e40 con 0x55c9af4abbb0 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ecb57c740 10 client.0 _ll_register_callbacks cb 0x7f6ebc003770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4c0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49584/0 (socket says 172.21.15.33:49584) 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 -- 172.21.15.33:0/3923959470 learned_addr learned my addr 172.21.15.33:0/3923959470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec6d2d640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4abbb0 0x55c9af4c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 -- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4abbb0 msgr2=0x55c9af4c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4abbb0 0x55c9af4c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 -- 172.21.15.33:0/3923959470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 msgr2=0x55c9af4c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 -- 172.21.15.33:0/3923959470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9af4c4640 con 0x55c9af4ab1c0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ec652c640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4c0b60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc099400 tx=0x7f6ebc0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d3d7420e6666cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.655+0000 7f6ea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/3923959470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc020070 con 0x55c9af4ab1c0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/3923959470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ebc0977c0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc015070 con 0x55c9af4ab1c0 2024-04-24T23:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/3923959470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ebc09f500 con 0x55c9af4ab1c0 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ecb57c740 10 client.19152 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9af5383b0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.656+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/3923959470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc015070 con 0x55c9af4ab1c0 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/3923959470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ebc09a1a0 con 0x55c9af4ab1c0 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ecb57c740 10 client.19152 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ecb57c740 10 client.19152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ecb57c740 20 client.19152 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af538a50 0x55c9af53ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ecb57c740 4 client.19152 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9af3946f0 con 0x55c9af538a50 2024-04-24T23:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.657+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af538a50 0x55c9af53ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.658+0000 7f6ea9ffb640 10 client.19152 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.658+0000 7f6ec5d2b640 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af538a50 0x55c9af53ae40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e193dd2fe01e08 server_cookie=186b8973166beef1 in_seq=0 out_seq=0 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.658+0000 7f6ea9ffb640 1 -- 172.21.15.33:0/3923959470 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9af3946f0 con 0x55c9af538a50 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.658+0000 7f6ea9ffb640 10 client.19152 handle_command_reply: tid=0 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 client.19152 shutdown 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 msgr2=0x55c9af4c0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4c0b60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc099400 tx=0x7f6ebc0046e0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 shutdown_connections 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c9af538a50 0x55c9af53ae40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9af4aed10 0x55c9af4c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ea9ffb640 0 client.19152 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9af4abbb0 0x55c9af4c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 --2- 172.21.15.33:0/3923959470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9af4ab1c0 0x55c9af4c0b60 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.659+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 >> 172.21.15.33:0/3923959470 conn(0x55c9af4a68b0 msgr2=0x55c9af4a9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.660+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 shutdown_connections 2024-04-24T23:29:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.660+0000 7f6ecb57c740 1 -- 172.21.15.33:0/3923959470 wait complete. 2024-04-24T23:29:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.660+0000 7f6ecb57c740 20 client.19152 trim_cache size 0 max 16384 2024-04-24T23:29:58.663 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:29:58.663 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:29:58.663 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:29:58.663 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.661+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6eac0a2d60 msgr2=0x7f6eac0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.661+0000 7f6ec8790640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6eac0a2d60 0x7f6eac0a5220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55c9af4ac410 tx=0x7f6eb800e040 comp rx=0 tx=0).stop 2024-04-24T23:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.661+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 msgr2=0x7f6ec0197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.661+0000 7f6ec8790640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 0x7f6ec0197960 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00d770 tx=0x7f6ebc00dc30 comp rx=0 tx=0).stop 2024-04-24T23:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 shutdown_connections 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6eac0a2d60 0x7f6eac0a5220 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ec0103330 0x7f6ec019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ec0102970 0x7f6ec0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 --2- 172.21.15.33:0/1751319978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec0101780 0x7f6ec0197960 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 >> 172.21.15.33:0/1751319978 conn(0x7f6ec00fccf0 msgr2=0x7f6ec0105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 shutdown_connections 2024-04-24T23:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.662+0000 7f6ec8790640 1 -- 172.21.15.33:0/1751319978 wait complete. 2024-04-24T23:29:58.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:29:58.679 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 -- 172.21.15.33:0/60936046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00104b30 msgr2=0x7f0c00106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 --2- 172.21.15.33:0/60936046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00104b30 0x7f0c00106f20 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009900 tx=0x7f0bf002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 -- 172.21.15.33:0/60936046 shutdown_connections 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 --2- 172.21.15.33:0/60936046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c00107460 0x7f0c001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 --2- 172.21.15.33:0/60936046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00104b30 0x7f0c00106f20 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 --2- 172.21.15.33:0/60936046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c00102200 0x7f0c001045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 -- 172.21.15.33:0/60936046 >> 172.21.15.33:0/60936046 conn(0x7f0c000fbbf0 msgr2=0x7f0c000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:58.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 -- 172.21.15.33:0/60936046 shutdown_connections 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.811+0000 7f0c072f7640 1 -- 172.21.15.33:0/60936046 wait complete. 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 Processor -- start 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 -- start start 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c00102200 0x7f0c00199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c00104b30 0x7f0c0019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 0x7f0c001a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0010d400 con 0x7f0c00107460 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0010d280 con 0x7f0c00102200 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c072f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0010d580 con 0x7f0c00104b30 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 0x7f0c001a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 0x7f0c001a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49598/0 (socket says 172.21.15.33:49598) 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 -- 172.21.15.33:0/295256321 learned_addr learned my addr 172.21.15.33:0/295256321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05093640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c00102200 0x7f0c00199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c04892640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c00104b30 0x7f0c0019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 -- 172.21.15.33:0/295256321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c00104b30 msgr2=0x7f0c0019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c00104b30 0x7f0c0019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 -- 172.21.15.33:0/295256321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c00102200 msgr2=0x7f0c00199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c00102200 0x7f0c00199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.812+0000 7f0c05894640 1 -- 172.21.15.33:0/295256321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001a1920 con 0x7f0c00107460 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.813+0000 7f0c05894640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 0x7f0c001a1280 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800eb10 tx=0x7f0bf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.813+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004b20 con 0x7f0c00107460 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.813+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf8004cc0 con 0x7f0c00107460 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.813+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001a1c10 con 0x7f0c00107460 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.813+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8010760 con 0x7f0c00107460 2024-04-24T23:29:58.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.813+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a2160 con 0x7f0c00107460 2024-04-24T23:29:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.814+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c00108fb0 con 0x7f0c00107460 2024-04-24T23:29:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.815+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0bf8005000 con 0x7f0c00107460 2024-04-24T23:29:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.816+0000 7f0bf67fc640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0bec0a2b90 0x7f0bec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.816+0000 7f0c05093640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0bec0a2b90 0x7f0bec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.816+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0bf80cfe60 con 0x7f0c00107460 2024-04-24T23:29:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.816+0000 7f0c05093640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0bec0a2b90 0x7f0bec0a5050 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0c0019b1e0 tx=0x7f0bfc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:58.820+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0bf809ca30 con 0x7f0c00107460 2024-04-24T23:29:58.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 106659 22.71 MB/sec execute 123 sec latency 305.175 ms 2024-04-24T23:29:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.064+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c0002d170 con 0x7f0c00107460 2024-04-24T23:29:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.065+0000 7f0bf67fc640 1 -- 172.21.15.33:0/295256321 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f0bf809c330 con 0x7f0c00107460 2024-04-24T23:29:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:29:59.065 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:29:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0bec0a2b90 msgr2=0x7f0bec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0bec0a2b90 0x7f0bec0a5050 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0c0019b1e0 tx=0x7f0bfc005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 msgr2=0x7f0c001a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 0x7f0c001a1280 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800eb10 tx=0x7f0bf800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 shutdown_connections 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0bec0a2b90 0x7f0bec0a5050 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c00107460 0x7f0c001a1280 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c00104b30 0x7f0c0019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 --2- 172.21.15.33:0/295256321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c00102200 0x7f0c00199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 >> 172.21.15.33:0/295256321 conn(0x7f0c000fbbf0 msgr2=0x7f0c00101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 shutdown_connections 2024-04-24T23:29:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.067+0000 7f0c072f7640 1 -- 172.21.15.33:0/295256321 wait complete. 2024-04-24T23:29:59.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:29:59.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/294798889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4101350 msgr2=0x7fa1d41017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/294798889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4101350 0x7fa1d41017d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009ca0 tx=0x7fa1cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/294798889 shutdown_connections 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/294798889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4101350 0x7fa1d41017d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/294798889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1d41009b0 0x7fa1d4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/294798889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1d4100050 0x7fa1d4100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/294798889 >> 172.21.15.33:0/294798889 conn(0x7fa1d40fbbf0 msgr2=0x7fa1d40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/294798889 shutdown_connections 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.191+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/294798889 wait complete. 2024-04-24T23:29:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 Processor -- start 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 -- start start 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 0x7fa1d4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1d41009b0 0x7fa1d419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1d4101350 0x7fa1d41a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410f5f0 con 0x7fa1d4100050 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410f470 con 0x7fa1d4101350 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d9b7d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410f770 con 0x7fa1d41009b0 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 0x7fa1d4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 0x7fa1d4199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49618/0 (socket says 172.21.15.33:49618) 2024-04-24T23:29:59.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.192+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1610564461 learned_addr learned my addr 172.21.15.33:0/1610564461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1d41009b0 0x7fa1d419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1d4101350 0x7fa1d41a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1610564461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1d41009b0 msgr2=0x7fa1d419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1d41009b0 0x7fa1d419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1610564461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1d4101350 msgr2=0x7fa1d41a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1d4101350 0x7fa1d41a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d37fe640 1 -- 172.21.15.33:0/1610564461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41a1890 con 0x7fa1d4100050 2024-04-24T23:29:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 0x7fa1d4199c80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400e9f0 tx=0x7fa1c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1610564461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400cd80 con 0x7fa1d4100050 2024-04-24T23:29:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d41a1b80 con 0x7fa1d4100050 2024-04-24T23:29:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1610564461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c4010910 con 0x7fa1d4100050 2024-04-24T23:29:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1610564461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c40188f0 con 0x7fa1d4100050 2024-04-24T23:29:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.193+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41a20d0 con 0x7fa1d4100050 2024-04-24T23:29:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.196+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1610564461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa1c4010ac0 con 0x7fa1d4100050 2024-04-24T23:29:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.196+0000 7fa1d0ff9640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1bc0a2b90 0x7fa1bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.196+0000 7fa1dc885740 1 Processor -- start 2024-04-24T23:29:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1bc0a2b90 0x7fa1bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d0ff9640 1 -- 172.21.15.33:0/1610564461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1c40cf870 con 0x7fa1d4100050 2024-04-24T23:29:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 -- start start 2024-04-24T23:29:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62052e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f620533720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f620536220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f620492670 con 0x55f620533d90 2024-04-24T23:29:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f62043a970 con 0x55f62052e230 2024-04-24T23:29:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1dc885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f62048e630 con 0x55f62052ec20 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f620536220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62052e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f620536220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49622/0 (socket says 172.21.15.33:49622) 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3650712594 learned_addr learned my addr 172.21.15.33:0/3650712594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f620533720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1bc0a2b90 0x7fa1bc0a5050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009c20 tx=0x7fa1c8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3650712594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 msgr2=0x55f620533720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f620533720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3650712594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 msgr2=0x55f62052e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62052e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3650712594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62045e400 con 0x55f620533d90 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62052e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.197+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f620536220 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009ca0 tx=0x7fa1cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739a4a42de9b7f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1b9ffb640 1 -- 172.21.15.33:0/3650712594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0098c0 con 0x55f620533d90 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1b9ffb640 1 -- 172.21.15.33:0/3650712594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc036040 con 0x55f620533d90 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1b9ffb640 1 -- 172.21.15.33:0/3650712594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004290 con 0x55f620533d90 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 -- 172.21.15.33:0/3650712594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 msgr2=0x55f620536220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f620536220 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009ca0 tx=0x7fa1cc004a40 comp rx=0 tx=0).stop 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 -- 172.21.15.33:0/3650712594 shutdown_connections 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f620536220 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f620533720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3650712594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62052e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 -- 172.21.15.33:0/3650712594 >> 172.21.15.33:0/3650712594 conn(0x55f6205298a0 msgr2=0x55f620529ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.198+0000 7fa1dc885740 1 -- 172.21.15.33:0/3650712594 shutdown_connections 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 -- 172.21.15.33:0/3650712594 wait complete. 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 -- start start 2024-04-24T23:29:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 Processor -- start 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620538090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 0x55f6205385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f620533d90 0x55f62053f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f620536b00 con 0x55f62052ec20 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f62044e050 con 0x55f62052e230 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f620536c80 con 0x55f620533d90 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1dc885740 10 client.0 _ll_register_callbacks cb 0x55f6201a5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 0x55f6205385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 0x55f6205385d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49624/0 (socket says 172.21.15.33:49624) 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3358706440 learned_addr learned my addr 172.21.15.33:0/3358706440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f620533d90 0x55f62053f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.199+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620538090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3358706440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f620533d90 msgr2=0x55f62053f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f620533d90 0x55f62053f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 msgr2=0x55f620538090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620538090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d2ffd640 1 -- 172.21.15.33:0/3358706440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62041fdc0 con 0x55f62052ec20 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f620533d90 0x55f62053f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620538090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 0x55f6205385d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa1c801d980 tx=0x7fa1c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3d000880a919d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/3358706440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c802b060 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/3358706440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1c8004030 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc036040 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/3358706440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1c8032050 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1dc885740 10 client.19170 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.200+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f6201a63f0 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/3358706440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc036040 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/3358706440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa1c8025070 con 0x55f62052ec20 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1dc885740 10 client.19170 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1dc885740 10 client.19170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1dc885740 20 client.19170 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f620543fb0 0x55f620546430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1dc885740 4 client.19170 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62049fbb0 con 0x55f620543fb0 2024-04-24T23:29:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.201+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f620543fb0 0x55f620546430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.202+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f620543fb0 0x55f620546430 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=762cf2f1a951d16b server_cookie=d2246b8a61925d94 in_seq=0 out_seq=0 2024-04-24T23:29:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.202+0000 7fa1b2ffd640 10 client.19170 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:59.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.207+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/3358706440 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f62049fbb0 con 0x55f620543fb0 2024-04-24T23:29:59.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.207+0000 7fa1b2ffd640 10 client.19170 handle_command_reply: tid=0 2024-04-24T23:29:59.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.207+0000 7fa1dc885740 1 client.19170 shutdown 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 msgr2=0x55f6205385d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 0x55f6205385d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa1c801d980 tx=0x7fa1c8004730 comp rx=0 tx=0).stop 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 shutdown_connections 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f620543fb0 0x55f620546430 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f620533d90 0x55f62053f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62052ec20 0x55f6205385d0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1b2ffd640 0 client.19170 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3358706440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620538090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 >> 172.21.15.33:0/3358706440 conn(0x55f6205298a0 msgr2=0x55f620528b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 shutdown_connections 2024-04-24T23:29:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 1 -- 172.21.15.33:0/3358706440 wait complete. 2024-04-24T23:29:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.208+0000 7fa1dc885740 20 client.19170 trim_cache size 0 max 16384 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 Processor -- start 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 -- start start 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620533310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f6205289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41aeda0 con 0x55f620533d90 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41ac670 con 0x55f62052e230 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.228+0000 7fa1dc885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41aa4c0 con 0x55f62052ec20 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f6205289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620533310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620533310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34064/0 (socket says 172.21.15.33:34064) 2024-04-24T23:29:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d37fe640 1 -- 172.21.15.33:0/3098978885 learned_addr learned my addr 172.21.15.33:0/3098978885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d2ffd640 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3098978885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 msgr2=0x55f62053f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3098978885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 msgr2=0x55f620533310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620533310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 -- 172.21.15.33:0/3098978885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6202707d0 con 0x55f620533d90 2024-04-24T23:29:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620533310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.229+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f6205289c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc02ade0 tx=0x7fa1cc005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e8d557267cfc79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.230+0000 7fa1b97fa640 1 -- 172.21.15.33:0/3098978885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc051070 con 0x55f620533d90 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.230+0000 7fa1b97fa640 1 -- 172.21.15.33:0/3098978885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc036040 con 0x55f620533d90 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.230+0000 7fa1b97fa640 1 -- 172.21.15.33:0/3098978885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004360 con 0x55f620533d90 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 -- 172.21.15.33:0/3098978885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 msgr2=0x55f6205289c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f6205289c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc02ade0 tx=0x7fa1cc005f70 comp rx=0 tx=0).stop 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 -- 172.21.15.33:0/3098978885 shutdown_connections 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f6205289c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 --2- 172.21.15.33:0/3098978885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f620533310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 -- 172.21.15.33:0/3098978885 >> 172.21.15.33:0/3098978885 conn(0x55f6205298a0 msgr2=0x55f6205282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 -- 172.21.15.33:0/3098978885 shutdown_connections 2024-04-24T23:29:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 -- 172.21.15.33:0/3098978885 wait complete. 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.231+0000 7fa1dc885740 1 -- start start 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 Processor -- start 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62053ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f62053df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41aab30 con 0x55f620533d90 2024-04-24T23:29:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41abfa0 con 0x55f62052e230 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41ab200 con 0x55f62052ec20 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.232+0000 7fa1dc885740 10 client.0 _ll_register_callbacks cb 0x7fa1c8009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f62053df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f62053df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49648/0 (socket says 172.21.15.33:49648) 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 -- 172.21.15.33:0/2076375337 learned_addr learned my addr 172.21.15.33:0/2076375337 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 -- 172.21.15.33:0/2076375337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 msgr2=0x55f62053cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 -- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 msgr2=0x55f62053ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62053ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 -- 172.21.15.33:0/2076375337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62053e510 con 0x55f620533d90 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1d3fff640 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f62053df20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc031c10 tx=0x7fa1cc009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d7533815d50366 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.233+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/2076375337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc051070 con 0x55f620533d90 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/2076375337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1cc003970 con 0x55f620533d90 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc036040 con 0x55f620533d90 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/2076375337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1cc003b70 con 0x55f620533d90 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1dc885740 10 client.19176 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f62053f810 con 0x55f620533d90 2024-04-24T23:29:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/2076375337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc036040 con 0x55f620533d90 2024-04-24T23:29:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.234+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/2076375337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa1cc02f7d0 con 0x55f620533d90 2024-04-24T23:29:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1dc885740 10 client.19176 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1dc885740 10 client.19176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1dc885740 20 client.19176 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1dc885740 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f6205aa940 0x55f6205acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1dc885740 4 client.19176 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f620499f00 con 0x55f6205aa940 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.235+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f6205aa940 0x55f6205acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.238+0000 7fa1d37fe640 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f6205aa940 0x55f6205acd50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=639d2d3f3997ffa2 server_cookie=6b80f3240fb36aec in_seq=0 out_seq=0 2024-04-24T23:29:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.238+0000 7fa1b2ffd640 10 client.19176 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:59.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.243+0000 7fa1b2ffd640 1 -- 172.21.15.33:0/2076375337 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f620499f00 con 0x55f6205aa940 2024-04-24T23:29:59.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.243+0000 7fa1b2ffd640 10 client.19176 handle_command_reply: tid=0 2024-04-24T23:29:59.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.243+0000 7fa1dc885740 1 client.19176 shutdown 2024-04-24T23:29:59.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.244+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 msgr2=0x55f62053df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.244+0000 7fa1dc885740 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f62053df20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc031c10 tx=0x7fa1cc009940 comp rx=0 tx=0).stop 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.244+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 shutdown_connections 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.244+0000 7fa1dc885740 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55f6205aa940 0x55f6205acd50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f620533d90 0x55f62053df20 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62052ec20 0x55f62053cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 1 --2- 172.21.15.33:0/2076375337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62052e230 0x55f62053ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 >> 172.21.15.33:0/2076375337 conn(0x55f6205298a0 msgr2=0x55f6205282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1b2ffd640 0 client.19176 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 shutdown_connections 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 1 -- 172.21.15.33:0/2076375337 wait complete. 2024-04-24T23:29:59.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.245+0000 7fa1dc885740 20 client.19176 trim_cache size 0 max 16384 2024-04-24T23:29:59.246 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.246+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1bc0a2b90 msgr2=0x7fa1bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.246+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1bc0a2b90 0x7fa1bc0a5050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009c20 tx=0x7fa1c8005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 msgr2=0x7fa1d4199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 0x7fa1d4199c80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400e9f0 tx=0x7fa1c400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 shutdown_connections 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1bc0a2b90 0x7fa1bc0a5050 secure :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009c20 tx=0x7fa1c8005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1d4101350 0x7fa1d41a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1d41009b0 0x7fa1d419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 --2- 172.21.15.33:0/1610564461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4100050 0x7fa1d4199c80 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 >> 172.21.15.33:0/1610564461 conn(0x7fa1d40fbbf0 msgr2=0x7fa1d40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 shutdown_connections 2024-04-24T23:29:59.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.247+0000 7fa1d9b7d640 1 -- 172.21.15.33:0/1610564461 wait complete. 2024-04-24T23:29:59.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 -- 172.21.15.33:0/4277352080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580101870 msgr2=0x7f3580101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 --2- 172.21.15.33:0/4277352080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580101870 0x7f3580101cf0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3570009900 tx=0x7f357002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 -- 172.21.15.33:0/4277352080 shutdown_connections 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 --2- 172.21.15.33:0/4277352080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3580102230 0x7f3580108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 --2- 172.21.15.33:0/4277352080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580101870 0x7f3580101cf0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 --2- 172.21.15.33:0/4277352080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580100680 0x7f3580100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 -- 172.21.15.33:0/4277352080 >> 172.21.15.33:0/4277352080 conn(0x7f35800fbbf0 msgr2=0x7f35800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 -- 172.21.15.33:0/4277352080 shutdown_connections 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 -- 172.21.15.33:0/4277352080 wait complete. 2024-04-24T23:29:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 Processor -- start 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.415+0000 7f35885f9640 1 -- start start 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3580100680 0x7f3580104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580101870 0x7f3580102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 0x7f3580103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f358010c300 con 0x7f3580102230 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f358010c180 con 0x7f3580100680 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f358010c480 con 0x7f3580101870 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 0x7f3580103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 0x7f3580103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49656/0 (socket says 172.21.15.33:49656) 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 -- 172.21.15.33:0/1079097912 learned_addr learned my addr 172.21.15.33:0/1079097912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586395640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3580100680 0x7f3580104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3585b94640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580101870 0x7f3580102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 -- 172.21.15.33:0/1079097912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580101870 msgr2=0x7f3580102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580101870 0x7f3580102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 -- 172.21.15.33:0/1079097912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3580100680 msgr2=0x7f3580104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3580100680 0x7f3580104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 -- 172.21.15.33:0/1079097912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3580103a20 con 0x7f3580102230 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f3586b96640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 0x7f3580103190 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f357800eb10 tx=0x7f357800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35777fe640 1 -- 172.21.15.33:0/1079097912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578004b20 con 0x7f3580102230 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35777fe640 1 -- 172.21.15.33:0/1079097912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3578004cc0 con 0x7f3580102230 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35801a1c10 con 0x7f3580102230 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35777fe640 1 -- 172.21.15.33:0/1079097912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578010760 con 0x7f3580102230 2024-04-24T23:29:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.416+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801a2130 con 0x7f3580102230 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.419+0000 7f35777fe640 1 -- 172.21.15.33:0/1079097912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3578005020 con 0x7f3580102230 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.419+0000 7f35777fe640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f356c0a2c10 0x7f356c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.419+0000 7f35777fe640 1 -- 172.21.15.33:0/1079097912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35780cf940 con 0x7f3580102230 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.419+0000 7f3586395640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f356c0a2c10 0x7f356c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 Processor -- start 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 -- start start 2024-04-24T23:29:59.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e3123650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f3586395640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f356c0a2c10 0x7f356c0a50d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f35801016d0 tx=0x7f357c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e3128720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3128d90 0x5560e312b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5560e3087670 con 0x5560e3123230 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5560e302f970 con 0x5560e3128d90 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f358b3e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5560e3083630 con 0x5560e3123c20 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f3586395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e3123650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f3585b94640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e3128720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f3586395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e3123650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49672/0 (socket says 172.21.15.33:49672) 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f3586395640 1 -- 172.21.15.33:0/1733609565 learned_addr learned my addr 172.21.15.33:0/1733609565 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.420+0000 7f3586b96640 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3128d90 0x5560e312b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3585b94640 1 -- 172.21.15.33:0/1733609565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3128d90 msgr2=0x5560e312b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3585b94640 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3128d90 0x5560e312b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3585b94640 1 -- 172.21.15.33:0/1733609565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 msgr2=0x5560e3123650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3585b94640 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e3123650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3585b94640 1 -- 172.21.15.33:0/1733609565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e3053400 con 0x5560e3123c20 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3586b96640 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3128d90 0x5560e312b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3585b94640 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e3128720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3570031440 tx=0x7f3570005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b02f991d4933e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3574bfe640 1 -- 172.21.15.33:0/1733609565 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570040070 con 0x5560e3123c20 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3574bfe640 1 -- 172.21.15.33:0/1733609565 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3570004030 con 0x5560e3123c20 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.421+0000 7f3574bfe640 1 -- 172.21.15.33:0/1733609565 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570035040 con 0x5560e3123c20 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- 172.21.15.33:0/1733609565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 msgr2=0x5560e3128720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e3128720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3570031440 tx=0x7f3570005e00 comp rx=0 tx=0).stop 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- 172.21.15.33:0/1733609565 shutdown_connections 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3128d90 0x5560e312b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e3128720 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- 172.21.15.33:0/1733609565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e3123650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- 172.21.15.33:0/1733609565 >> 172.21.15.33:0/1733609565 conn(0x5560e311e8a0 msgr2=0x5560e311eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- 172.21.15.33:0/1733609565 shutdown_connections 2024-04-24T23:29:59.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- 172.21.15.33:0/1733609565 wait complete. 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- start start 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 Processor -- start 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 0x5560e312d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e312d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3128d90 0x5560e3134880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5560e312bce0 con 0x5560e3128d90 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5560e3043050 con 0x5560e3123230 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5560e312be60 con 0x5560e3123c20 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.422+0000 7f358b3e3740 10 client.0 _ll_register_callbacks cb 0x5560e2d9abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586395640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 0x5560e312d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3128d90 0x5560e3134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3585b94640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e312d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586395640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 0x5560e312d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34120/0 (socket says 172.21.15.33:34120) 2024-04-24T23:29:59.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586395640 1 -- 172.21.15.33:0/3220707902 learned_addr learned my addr 172.21.15.33:0/3220707902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 -- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 msgr2=0x5560e312d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e312d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 -- 172.21.15.33:0/3220707902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 msgr2=0x5560e312d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 0x5560e312d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 -- 172.21.15.33:0/3220707902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e3014dc0 con 0x5560e3128d90 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586395640 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 0x5560e312d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3586b96640 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3128d90 0x5560e3134880 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f357809bb10 tx=0x7f357809b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d0c310182d5fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3569ffb640 1 -- 172.21.15.33:0/3220707902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35780a01c0 con 0x5560e3128d90 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3570004030 con 0x5560e3128d90 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3569ffb640 1 -- 172.21.15.33:0/3220707902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f357809c070 con 0x5560e3128d90 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f3569ffb640 1 -- 172.21.15.33:0/3220707902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f35780a4a80 con 0x5560e3128d90 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f358b3e3740 10 client.19188 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.423+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5560e2d9b3f0 con 0x5560e3128d90 2024-04-24T23:29:59.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f3569ffb640 1 -- 172.21.15.33:0/3220707902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3570004030 con 0x5560e3128d90 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f3569ffb640 1 -- 172.21.15.33:0/3220707902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3578005410 con 0x5560e3128d90 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f358b3e3740 10 client.19188 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f358b3e3740 10 client.19188 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f358b3e3740 20 client.19188 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f358b3e3740 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e31395a0 0x5560e313ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f358b3e3740 4 client.19188 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e3094bb0 con 0x5560e31395a0 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.424+0000 7f3586395640 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e31395a0 0x5560e313ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.425+0000 7f3569ffb640 10 client.19188 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.425+0000 7f3586395640 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e31395a0 0x5560e313ba60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4aa7cce8725d87e2 server_cookie=33c02162c6a00800 in_seq=0 out_seq=0 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.425+0000 7f3569ffb640 1 -- 172.21.15.33:0/3220707902 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560e3094bb0 con 0x5560e31395a0 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f3569ffb640 10 client.19188 handle_command_reply: tid=0 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 client.19188 shutdown 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3128d90 msgr2=0x5560e3134880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3128d90 0x5560e3134880 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f357809bb10 tx=0x7f357809b770 comp rx=0 tx=0).stop 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 shutdown_connections 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e31395a0 0x5560e313ba60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3128d90 0x5560e3134880 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3123c20 0x5560e312d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 --2- 172.21.15.33:0/3220707902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123230 0x5560e312d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 >> 172.21.15.33:0/3220707902 conn(0x5560e311e8a0 msgr2=0x5560e311db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f3569ffb640 0 client.19188 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 shutdown_connections 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 1 -- 172.21.15.33:0/3220707902 wait complete. 2024-04-24T23:29:59.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.426+0000 7f358b3e3740 20 client.19188 trim_cache size 0 max 16384 2024-04-24T23:29:59.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 Processor -- start 2024-04-24T23:29:59.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 -- start start 2024-04-24T23:29:59.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e311d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3134840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e31246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35801aee50 con 0x5560e3123230 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35801ac720 con 0x5560e3123c20 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f358b3e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35801aa570 con 0x5560e3128d90 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e311d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3585b94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3134840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e311d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49690/0 (socket says 172.21.15.33:49690) 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 -- 172.21.15.33:0/284350608 learned_addr learned my addr 172.21.15.33:0/284350608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586b96640 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e31246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 -- 172.21.15.33:0/284350608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 msgr2=0x5560e31246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e31246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 -- 172.21.15.33:0/284350608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 msgr2=0x5560e3134840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3134840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586395640 1 -- 172.21.15.33:0/284350608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e2e657d0 con 0x5560e3123230 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3586b96640 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e31246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:59.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.440+0000 7f3585b94640 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3134840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.441+0000 7f3586395640 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e311d9c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f357c009560 tx=0x7f357c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6f7c7dbaef36b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.441+0000 7f356bfff640 1 -- 172.21.15.33:0/284350608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c031070 con 0x5560e3123230 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.441+0000 7f356bfff640 1 -- 172.21.15.33:0/284350608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357c01ddf0 con 0x5560e3123230 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.441+0000 7f356bfff640 1 -- 172.21.15.33:0/284350608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c02a660 con 0x5560e3123230 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- 172.21.15.33:0/284350608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 msgr2=0x5560e311d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e311d9c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f357c009560 tx=0x7f357c014000 comp rx=0 tx=0).stop 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- 172.21.15.33:0/284350608 shutdown_connections 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e31246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3134840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- 172.21.15.33:0/284350608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e311d9c0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- 172.21.15.33:0/284350608 >> 172.21.15.33:0/284350608 conn(0x5560e311e8a0 msgr2=0x5560e311d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- 172.21.15.33:0/284350608 shutdown_connections 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- 172.21.15.33:0/284350608 wait complete. 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- start start 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 Processor -- start 2024-04-24T23:29:59.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e31317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3131d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e319dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35801aabe0 con 0x5560e3123230 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35801ac050 con 0x5560e3123c20 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35801ab2b0 con 0x5560e3128d90 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f358b3e3740 10 client.0 _ll_register_callbacks cb 0x7f357809b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f3586395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e31317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f3585b94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f3586b96640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e319dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f3586395640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e31317c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49706/0 (socket says 172.21.15.33:49706) 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f3586395640 1 -- 172.21.15.33:0/2615936540 learned_addr learned my addr 172.21.15.33:0/2615936540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.442+0000 7f3585b94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34134/0 (socket says 172.21.15.33:34134) 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586b96640 1 -- 172.21.15.33:0/2615936540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 msgr2=0x5560e3131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586b96640 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3131d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586b96640 1 -- 172.21.15.33:0/2615936540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 msgr2=0x5560e31317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586b96640 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e31317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586b96640 1 -- 172.21.15.33:0/2615936540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e319e1c0 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586395640 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e31317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3586b96640 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e319dbd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3578005880 tx=0x7f3578005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab547eaf4342e884 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3569ffb640 1 -- 172.21.15.33:0/2615936540 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35780a01c0 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c01ddf0 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3569ffb640 1 -- 172.21.15.33:0/2615936540 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35780a47f0 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3569ffb640 1 -- 172.21.15.33:0/2615936540 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f357809c510 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f358b3e3740 10 client.28612 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5560e319f460 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.443+0000 7f3569ffb640 1 -- 172.21.15.33:0/2615936540 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357c01ddf0 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f3569ffb640 1 -- 172.21.15.33:0/2615936540 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3578081030 con 0x5560e3128d90 2024-04-24T23:29:59.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 10 client.28612 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 10 client.28612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 20 client.28612 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e319fca0 0x5560e31a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 4 client.28612 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560e308ef00 con 0x5560e319fca0 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f3586395640 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e319fca0 0x5560e31a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f3569ffb640 10 client.28612 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f3586395640 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e319fca0 0x5560e31a20c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b1afb40e460aa4f server_cookie=f9e1d9594d8572d3 in_seq=0 out_seq=0 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f3569ffb640 1 -- 172.21.15.33:0/2615936540 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560e308ef00 con 0x5560e319fca0 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f3569ffb640 10 client.28612 handle_command_reply: tid=0 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 1 client.28612 shutdown 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 msgr2=0x5560e319dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.444+0000 7f358b3e3740 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e319dbd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3578005880 tx=0x7f3578005b40 comp rx=0 tx=0).stop 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 shutdown_connections 2024-04-24T23:29:59.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5560e319fca0 0x5560e31a20c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560e3128d90 0x5560e319dbd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f3569ffb640 0 client.28612 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5560e3123c20 0x5560e3131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 --2- 172.21.15.33:0/2615936540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560e3123230 0x5560e31317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 >> 172.21.15.33:0/2615936540 conn(0x5560e311e8a0 msgr2=0x5560e311d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 shutdown_connections 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 1 -- 172.21.15.33:0/2615936540 wait complete. 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f358b3e3740 20 client.28612 trim_cache size 0 max 16384 2024-04-24T23:29:59.451 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:29:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f356c0a2c10 msgr2=0x7f356c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f356c0a2c10 0x7f356c0a50d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f35801016d0 tx=0x7f357c005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 msgr2=0x7f3580103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 0x7f3580103190 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f357800eb10 tx=0x7f357800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 shutdown_connections 2024-04-24T23:29:59.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f356c0a2c10 0x7f356c0a50d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3580102230 0x7f3580103190 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580101870 0x7f3580102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 --2- 172.21.15.33:0/1079097912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3580100680 0x7f3580104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 >> 172.21.15.33:0/1079097912 conn(0x7f35800fbbf0 msgr2=0x7f35800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 shutdown_connections 2024-04-24T23:29:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.445+0000 7f35885f9640 1 -- 172.21.15.33:0/1079097912 wait complete. 2024-04-24T23:29:59.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:59 smithi033 ceph-mon[29201]: pgmap v890: 129 pgs: 129 active+clean; 540 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-04-24T23:29:59.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:29:59 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/295256321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:59.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:29:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.627+0000 7f6de96bc640 1 -- 172.21.15.33:0/2037512076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de40ff810 msgr2=0x7f6de40ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.627+0000 7f6de96bc640 1 --2- 172.21.15.33:0/2037512076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de40ff810 0x7f6de40ffc90 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009900 tx=0x7f6dd802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 -- 172.21.15.33:0/2037512076 shutdown_connections 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 --2- 172.21.15.33:0/2037512076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de41001d0 0x7f6de4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 --2- 172.21.15.33:0/2037512076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de40ff810 0x7f6de40ffc90 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 --2- 172.21.15.33:0/2037512076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de4102da0 0x7f6de41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 -- 172.21.15.33:0/2037512076 >> 172.21.15.33:0/2037512076 conn(0x7f6de40fb370 msgr2=0x7f6de40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 -- 172.21.15.33:0/2037512076 shutdown_connections 2024-04-24T23:29:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 -- 172.21.15.33:0/2037512076 wait complete. 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.628+0000 7f6de96bc640 1 Processor -- start 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 -- start start 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40ff810 0x7f6de4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de41001d0 0x7f6de4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 0x7f6de419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d450 con 0x7f6de4102da0 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d2d0 con 0x7f6de41001d0 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de96bc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d5d0 con 0x7f6de40ff810 2024-04-24T23:29:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 0x7f6de419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 0x7f6de419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49728/0 (socket says 172.21.15.33:49728) 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de41001d0 0x7f6de4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de37fe640 1 -- 172.21.15.33:0/752891639 learned_addr learned my addr 172.21.15.33:0/752891639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.629+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40ff810 0x7f6de4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de27fc640 1 -- 172.21.15.33:0/752891639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40ff810 msgr2=0x7f6de4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de27fc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40ff810 0x7f6de4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de27fc640 1 -- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 msgr2=0x7f6de419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de27fc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 0x7f6de419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de27fc640 1 -- 172.21.15.33:0/752891639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de41009a0 con 0x7f6de41001d0 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de37fe640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 0x7f6de419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.630+0000 7f6de27fc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de41001d0 0x7f6de4198010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009850 tx=0x7f6dd8032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.633+0000 7f6dcbfff640 1 -- 172.21.15.33:0/752891639 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd803f030 con 0x7f6de41001d0 2024-04-24T23:29:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.633+0000 7f6dcbfff640 1 -- 172.21.15.33:0/752891639 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8003ad0 con 0x7f6de41001d0 2024-04-24T23:29:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.633+0000 7f6dcbfff640 1 -- 172.21.15.33:0/752891639 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8035070 con 0x7f6de41001d0 2024-04-24T23:29:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.633+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de4100c30 con 0x7f6de41001d0 2024-04-24T23:29:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.633+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de41010a0 con 0x7f6de41001d0 2024-04-24T23:29:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.636+0000 7f6dec3c5740 1 Processor -- start 2024-04-24T23:29:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.638+0000 7f6dcbfff640 1 -- 172.21.15.33:0/752891639 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6dd8042030 con 0x7f6de41001d0 2024-04-24T23:29:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dcbfff640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6dcc0a2ce0 0x7f6dcc0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dcbfff640 1 -- 172.21.15.33:0/752891639 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6dd80ef790 con 0x7f6de41001d0 2024-04-24T23:29:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 -- start start 2024-04-24T23:29:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61e1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5651b613d670 con 0x5651b61d9230 2024-04-24T23:29:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5651b60e5970 con 0x5651b61d9c20 2024-04-24T23:29:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651b6139630 con 0x5651b61ded90 2024-04-24T23:29:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.640+0000 7f6de37fe640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6dcc0a2ce0 0x7f6dcc0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.641+0000 7f6de27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.641+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.641+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61e1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33066/0 (socket says 172.21.15.33:33066) 2024-04-24T23:29:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.641+0000 7f6de37fe640 1 -- 172.21.15.33:0/752673415 learned_addr learned my addr 172.21.15.33:0/752673415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.641+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6dcc0a2ce0 0x7f6dcc0a51a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009fd0 tx=0x7f6ddc00f3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 -- 172.21.15.33:0/752673415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 msgr2=0x5651b61de720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61de720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 -- 172.21.15.33:0/752673415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 msgr2=0x5651b61d9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61d9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 -- 172.21.15.33:0/752673415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b6109400 con 0x5651b61ded90 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6de37fe640 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61e1220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc013ae0 tx=0x7f6ddc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e692a55bb10b01e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6dc97fa640 1 -- 172.21.15.33:0/752673415 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01da60 con 0x5651b61ded90 2024-04-24T23:29:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.642+0000 7f6dc97fa640 1 -- 172.21.15.33:0/752673415 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc01dc00 con 0x5651b61ded90 2024-04-24T23:29:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.643+0000 7f6dc97fa640 1 -- 172.21.15.33:0/752673415 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00e5b0 con 0x5651b61ded90 2024-04-24T23:29:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.643+0000 7f6dec3c5740 1 -- 172.21.15.33:0/752673415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 msgr2=0x5651b61e1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.643+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61e1220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc013ae0 tx=0x7f6ddc01c000 comp rx=0 tx=0).stop 2024-04-24T23:29:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 -- 172.21.15.33:0/752673415 shutdown_connections 2024-04-24T23:29:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61e1220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/752673415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61d9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 -- 172.21.15.33:0/752673415 >> 172.21.15.33:0/752673415 conn(0x5651b61d48a0 msgr2=0x5651b61d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 -- 172.21.15.33:0/752673415 shutdown_connections 2024-04-24T23:29:59.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 -- 172.21.15.33:0/752673415 wait complete. 2024-04-24T23:29:59.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.644+0000 7f6dec3c5740 1 -- start start 2024-04-24T23:29:59.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 Processor -- start 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5651b61e1d90 con 0x5651b61d9230 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5651b60f9050 con 0x5651b61d9c20 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651b61e1f10 con 0x5651b61ded90 2024-04-24T23:29:59.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6dec3c5740 10 client.0 _ll_register_callbacks cb 0x5651b5e50be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61e3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49742/0 (socket says 172.21.15.33:49742) 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 -- 172.21.15.33:0/3586805467 learned_addr learned my addr 172.21.15.33:0/3586805467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de27fc640 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 -- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 msgr2=0x5651b61ea8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 -- 172.21.15.33:0/3586805467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 msgr2=0x5651b61e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.645+0000 7f6de2ffd640 1 -- 172.21.15.33:0/3586805467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b60cadc0 con 0x5651b61d9230 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61e3340 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40126e0 tx=0x7f6dd4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bec98f8c4ac11af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/3586805467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40121d0 con 0x5651b61d9230 2024-04-24T23:29:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ddc01dc00 con 0x5651b61d9230 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/3586805467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6dd4004050 con 0x5651b61d9230 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/3586805467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6dd4025050 con 0x5651b61d9230 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dec3c5740 10 client.19206 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.646+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651b5e513f0 con 0x5651b61d9230 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/3586805467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc01dc00 con 0x5651b61d9230 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/3586805467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6dd4003890 con 0x5651b61d9230 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dec3c5740 10 client.19206 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dec3c5740 10 client.19206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dec3c5740 20 client.19206 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b6200630 0x5651b6202af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dec3c5740 4 client.19206 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651b614abb0 con 0x5651b6200630 2024-04-24T23:29:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.647+0000 7f6de27fc640 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b6200630 0x5651b6202af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.648+0000 7f6dc6ff5640 10 client.19206 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.649+0000 7f6de27fc640 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b6200630 0x5651b6202af0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a092acf77f7ad server_cookie=cffb7a52ce4d3ad2 in_seq=0 out_seq=0 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.649+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/3586805467 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651b614abb0 con 0x5651b6200630 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.650+0000 7f6dc6ff5640 10 client.19206 handle_command_reply: tid=0 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.650+0000 7f6dec3c5740 1 client.19206 shutdown 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.650+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 msgr2=0x5651b61e3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.650+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61e3340 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40126e0 tx=0x7f6dd4012ba0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 shutdown_connections 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b6200630 0x5651b6202af0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b61ea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/3586805467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9230 0x5651b61e3340 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 >> 172.21.15.33:0/3586805467 conn(0x5651b61d48a0 msgr2=0x5651b61d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dc6ff5640 0 client.19206 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:59.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 shutdown_connections 2024-04-24T23:29:59.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.651+0000 7f6dec3c5740 1 -- 172.21.15.33:0/3586805467 wait complete. 2024-04-24T23:29:59.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.652+0000 7f6dec3c5740 20 client.19206 trim_cache size 0 max 16384 2024-04-24T23:29:59.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:59 smithi038 ceph-mon[32450]: pgmap v890: 129 pgs: 129 active+clean; 540 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-04-24T23:29:59.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:29:59 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/295256321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 Processor -- start 2024-04-24T23:29:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 -- start start 2024-04-24T23:29:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 0x5651b61d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9c20 0x5651b61ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b62073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a6710 con 0x5651b61d9c20 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a3fe0 con 0x5651b61d9230 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.673+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4071d20 con 0x5651b61ded90 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 0x5651b61d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 0x5651b61d3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34186/0 (socket says 172.21.15.33:34186) 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 -- 172.21.15.33:0/2714805230 learned_addr learned my addr 172.21.15.33:0/2714805230 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de37fe640 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b62073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de27fc640 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9c20 0x5651b61ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 -- 172.21.15.33:0/2714805230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 msgr2=0x5651b62073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b62073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 -- 172.21.15.33:0/2714805230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9c20 msgr2=0x5651b61ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9c20 0x5651b61ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 -- 172.21.15.33:0/2714805230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b5f1b7d0 con 0x5651b61d9230 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.674+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 0x5651b61d3d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40126b0 tx=0x7f6dd4012250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74931589f68b97b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.675+0000 7f6dc8ff9640 1 -- 172.21.15.33:0/2714805230 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40104b0 con 0x5651b61d9230 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.675+0000 7f6dc8ff9640 1 -- 172.21.15.33:0/2714805230 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd402e070 con 0x5651b61d9230 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.675+0000 7f6dc8ff9640 1 -- 172.21.15.33:0/2714805230 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400f790 con 0x5651b61d9230 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.676+0000 7f6dec3c5740 1 -- 172.21.15.33:0/2714805230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 msgr2=0x5651b61d3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.676+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 0x5651b61d3d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40126b0 tx=0x7f6dd4012250 comp rx=0 tx=0).stop 2024-04-24T23:29:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:59 smithi117 ceph-mon[32529]: pgmap v890: 129 pgs: 129 active+clean; 540 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-04-24T23:29:59.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:29:59 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/295256321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.678+0000 7f6dec3c5740 1 -- 172.21.15.33:0/2714805230 shutdown_connections 2024-04-24T23:29:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.678+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61ded90 0x5651b62073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.678+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61d9c20 0x5651b61ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.678+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/2714805230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9230 0x5651b61d3d80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.678+0000 7f6dec3c5740 1 -- 172.21.15.33:0/2714805230 >> 172.21.15.33:0/2714805230 conn(0x5651b61d48a0 msgr2=0x5651b61daf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.679+0000 7f6dec3c5740 1 -- 172.21.15.33:0/2714805230 shutdown_connections 2024-04-24T23:29:59.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.679+0000 7f6dec3c5740 1 -- 172.21.15.33:0/2714805230 wait complete. 2024-04-24T23:29:59.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.679+0000 7f6dec3c5740 1 -- start start 2024-04-24T23:29:59.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.680+0000 7f6dec3c5740 1 Processor -- start 2024-04-24T23:29:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.680+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61d9230 0x5651b61e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.680+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.680+0000 7f6dec3c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 0x5651b61e9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6dec3c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4072390 con 0x5651b61ded90 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 0x5651b61e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 0x5651b61e9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49762/0 (socket says 172.21.15.33:49762) 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6de37fe640 1 -- 172.21.15.33:0/4198025735 learned_addr learned my addr 172.21.15.33:0/4198025735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a3a80 con 0x5651b61d9c20 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4072a60 con 0x5651b61d9230 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6dec3c5740 10 client.0 _ll_register_callbacks cb 0x7f6dd4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.681+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61d9230 0x5651b61e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6de37fe640 1 -- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61d9230 msgr2=0x5651b61e7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6de37fe640 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61d9230 0x5651b61e7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6de37fe640 1 -- 172.21.15.33:0/4198025735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 msgr2=0x5651b61e8310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6de37fe640 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6de37fe640 1 -- 172.21.15.33:0/4198025735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b61e9910 con 0x5651b61ded90 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6de37fe640 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 0x5651b61e9320 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc022000 tx=0x7f6ddc00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce673a11fc916369 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.682+0000 7f6dc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.683+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/4198025735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01da60 con 0x5651b61ded90 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.683+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd402e070 con 0x5651b61ded90 2024-04-24T23:29:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.683+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/4198025735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ddc01dc00 con 0x5651b61ded90 2024-04-24T23:29:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.684+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/4198025735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ddc0049d0 con 0x5651b61ded90 2024-04-24T23:29:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.684+0000 7f6dec3c5740 10 client.19212 fetch_fsmap learned FSMap version 33 2024-04-24T23:29:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.684+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651b6266c70 con 0x5651b61ded90 2024-04-24T23:29:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.685+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/4198025735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd402e070 con 0x5651b61ded90 2024-04-24T23:29:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/4198025735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ddc024070 con 0x5651b61ded90 2024-04-24T23:29:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dec3c5740 10 client.19212 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:29:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dec3c5740 10 client.19212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dec3c5740 20 client.19212 populate_metadata read hostname 'smithi033' 2024-04-24T23:29:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b62675a0 0x5651b62699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dec3c5740 4 client.19212 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:29:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.687+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651b6144f00 con 0x5651b62675a0 2024-04-24T23:29:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.688+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b62675a0 0x5651b62699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.695+0000 7f6de2ffd640 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b62675a0 0x5651b62699c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2acc1efcd3147f7 server_cookie=946ce2ba1f811b8e in_seq=0 out_seq=0 2024-04-24T23:29:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.695+0000 7f6dc6ff5640 10 client.19212 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.698+0000 7f6dc6ff5640 1 -- 172.21.15.33:0/4198025735 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651b6144f00 con 0x5651b62675a0 2024-04-24T23:29:59.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.698+0000 7f6dc6ff5640 10 client.19212 handle_command_reply: tid=0 2024-04-24T23:29:59.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.698+0000 7f6dec3c5740 1 client.19212 shutdown 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 msgr2=0x5651b61e9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 0x5651b61e9320 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc022000 tx=0x7f6ddc00ff60 comp rx=0 tx=0).stop 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 shutdown_connections 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5651b62675a0 0x5651b62699c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651b61ded90 0x5651b61e9320 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5651b61d9c20 0x5651b61e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 --2- 172.21.15.33:0/4198025735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b61d9230 0x5651b61e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 >> 172.21.15.33:0/4198025735 conn(0x5651b61d48a0 msgr2=0x5651b61d7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.699+0000 7f6dc6ff5640 0 client.19212 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:29:59.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.700+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 shutdown_connections 2024-04-24T23:29:59.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.700+0000 7f6dec3c5740 1 -- 172.21.15.33:0/4198025735 wait complete. 2024-04-24T23:29:59.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.701+0000 7f6dec3c5740 20 client.19212 trim_cache size 0 max 16384 2024-04-24T23:29:59.702 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:29:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.703+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6dcc0a2ce0 msgr2=0x7f6dcc0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.703+0000 7f6de96bc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6dcc0a2ce0 0x7f6dcc0a51a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009fd0 tx=0x7f6ddc00f3c0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.703+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de41001d0 msgr2=0x7f6de4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.703+0000 7f6de96bc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de41001d0 0x7f6de4198010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009850 tx=0x7f6dd8032c60 comp rx=0 tx=0).stop 2024-04-24T23:29:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.704+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 shutdown_connections 2024-04-24T23:29:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.704+0000 7f6de96bc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6dcc0a2ce0 0x7f6dcc0a51a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.704+0000 7f6de96bc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de4102da0 0x7f6de419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.704+0000 7f6de96bc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6de41001d0 0x7f6de4198010 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.704+0000 7f6de96bc640 1 --2- 172.21.15.33:0/752891639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40ff810 0x7f6de4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.704+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 >> 172.21.15.33:0/752891639 conn(0x7f6de40fb370 msgr2=0x7f6de4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.705+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 shutdown_connections 2024-04-24T23:29:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:29:59.706+0000 7f6de96bc640 1 -- 172.21.15.33:0/752891639 wait complete. 2024-04-24T23:29:59.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 107129 22.74 MB/sec execute 124 sec latency 237.417 ms 2024-04-24T23:30:00.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:00 smithi038 ceph-mon[32450]: overall HEALTH_OK 2024-04-24T23:30:00.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:00 smithi117 ceph-mon[32529]: overall HEALTH_OK 2024-04-24T23:30:00.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:00 smithi033 ceph-mon[29201]: overall HEALTH_OK 2024-04-24T23:30:00.728 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:30:00.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:30:00.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 -- 172.21.15.33:0/1358387525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102250 msgr2=0x7f4664108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 --2- 172.21.15.33:0/1358387525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102250 0x7f4664108ae0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f465c009ca0 tx=0x7f465c02c790 comp rx=0 tx=0).stop 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 -- 172.21.15.33:0/1358387525 shutdown_connections 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 --2- 172.21.15.33:0/1358387525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664102250 0x7f4664108ae0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 --2- 172.21.15.33:0/1358387525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4664101890 0x7f4664101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 --2- 172.21.15.33:0/1358387525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 -- 172.21.15.33:0/1358387525 >> 172.21.15.33:0/1358387525 conn(0x7f46640fbc10 msgr2=0x7f46640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.841+0000 7f4669534640 1 -- 172.21.15.33:0/1358387525 shutdown_connections 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.842+0000 7f4669534640 1 -- 172.21.15.33:0/1358387525 wait complete. 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.842+0000 7f4669534640 1 Processor -- start 2024-04-24T23:30:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.842+0000 7f4669534640 1 -- start start 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4669534640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4669534640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664101890 0x7f466419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4669534640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4664102250 0x7f46641a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4669534640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f466410b270 con 0x7f4664101890 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4669534640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f466410b0f0 con 0x7f4664102250 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4669534640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f466410b3f0 con 0x7f46641006a0 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33106/0 (socket says 172.21.15.33:33106) 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 -- 172.21.15.33:0/3225312105 learned_addr learned my addr 172.21.15.33:0/3225312105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f46637fe640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4664102250 0x7f46641a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f46627fc640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664101890 0x7f466419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 -- 172.21.15.33:0/3225312105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4664102250 msgr2=0x7f46641a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4664102250 0x7f46641a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 -- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664101890 msgr2=0x7f466419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664101890 0x7f466419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 -- 172.21.15.33:0/3225312105 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a18d0 con 0x7f46641006a0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.843+0000 7f4662ffd640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664199c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f465400e9c0 tx=0x7f465400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.844+0000 7f464bfff640 1 -- 172.21.15.33:0/3225312105 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465400cd80 con 0x7f46641006a0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.844+0000 7f464bfff640 1 -- 172.21.15.33:0/3225312105 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46540035f0 con 0x7f46641006a0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.844+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a1bc0 con 0x7f46641006a0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.844+0000 7f464bfff640 1 -- 172.21.15.33:0/3225312105 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654010690 con 0x7f46641006a0 2024-04-24T23:30:00.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.844+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a2110 con 0x7f46641006a0 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f464bfff640 1 -- 172.21.15.33:0/3225312105 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f46540037a0 con 0x7f46641006a0 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 Processor -- start 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 -- start start 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad3c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad3ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea7ac9a2c0 con 0x55ea7ad37780 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea7ac42910 con 0x55ea7ad3c8f0 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f466c244740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea7ac96280 con 0x55ea7ad36d90 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad3c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad371b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33120/0 (socket says 172.21.15.33:33120) 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 -- 172.21.15.33:0/1194837489 learned_addr learned my addr 172.21.15.33:0/1194837489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad3c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49800/0 (socket says 172.21.15.33:49800) 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46627fc640 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad3ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 -- 172.21.15.33:0/1194837489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 msgr2=0x55ea7ad3ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad3ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 -- 172.21.15.33:0/1194837489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 msgr2=0x55ea7ad3c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.847+0000 7f46637fe640 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad3c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f464bfff640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f464c0a2c10 0x7f464c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f46637fe640 1 -- 172.21.15.33:0/1194837489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea7ac53fb0 con 0x55ea7ad36d90 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f4662ffd640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f464c0a2c10 0x7f464c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f464bfff640 1 -- 172.21.15.33:0/3225312105 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46540cf800 con 0x7f46641006a0 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f46637fe640 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad371b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f465c031fd0 tx=0x7f465c009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67eaa0b71a3c1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f46497fa640 1 -- 172.21.15.33:0/1194837489 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c003970 con 0x55ea7ad36d90 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f46497fa640 1 -- 172.21.15.33:0/1194837489 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c042070 con 0x55ea7ad36d90 2024-04-24T23:30:00.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.848+0000 7f46497fa640 1 -- 172.21.15.33:0/1194837489 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c004410 con 0x55ea7ad36d90 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 -- 172.21.15.33:0/1194837489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 msgr2=0x55ea7ad371b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f4662ffd640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f464c0a2c10 0x7f464c0a50d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4654098640 tx=0x7f46540d5530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad371b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f465c031fd0 tx=0x7f465c009380 comp rx=0 tx=0).stop 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 -- 172.21.15.33:0/1194837489 shutdown_connections 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad3ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad3c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 --2- 172.21.15.33:0/1194837489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad371b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 -- 172.21.15.33:0/1194837489 >> 172.21.15.33:0/1194837489 conn(0x55ea7acfe950 msgr2=0x55ea7ad32810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 -- 172.21.15.33:0/1194837489 shutdown_connections 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.849+0000 7f466c244740 1 -- 172.21.15.33:0/1194837489 wait complete. 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 -- start start 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 Processor -- start 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea7ad3f840 con 0x55ea7ad37780 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea7ac55e90 con 0x55ea7ad3c8f0 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea7ad3f9c0 con 0x55ea7ad36d90 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33122/0 (socket says 172.21.15.33:33122) 2024-04-24T23:30:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46637fe640 1 -- 172.21.15.33:0/195301071 learned_addr learned my addr 172.21.15.33:0/195301071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46637fe640 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f466c244740 10 client.0 _ll_register_callbacks cb 0x55ea7ac91970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46627fc640 1 -- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 msgr2=0x55ea7ad411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46627fc640 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46627fc640 1 -- 172.21.15.33:0/195301071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 msgr2=0x55ea7ad48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46627fc640 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.850+0000 7f46627fc640 1 -- 172.21.15.33:0/195301071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea7aa767e0 con 0x55ea7ad37780 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f46627fc640 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad416f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f466419b190 tx=0x7f4658012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9de98f822a333c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f4646ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f4646ff5640 1 -- 172.21.15.33:0/195301071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800f8c0 con 0x55ea7ad37780 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f4646ff5640 1 -- 172.21.15.33:0/195301071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f465800fa60 con 0x55ea7ad37780 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c042070 con 0x55ea7ad37780 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f4646ff5640 1 -- 172.21.15.33:0/195301071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f465800fc60 con 0x55ea7ad37780 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f466c244740 10 client.19224 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ea7aad6f70 con 0x55ea7ad37780 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f4646ff5640 1 -- 172.21.15.33:0/195301071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c042070 con 0x55ea7ad37780 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f4646ff5640 1 -- 172.21.15.33:0/195301071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f465801c420 con 0x55ea7ad37780 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f466c244740 10 client.19224 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.851+0000 7f466c244740 10 client.19224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f466c244740 20 client.19224 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f466c244740 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7ad4cd90 0x55ea7ad4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f466c244740 4 client.19224 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea7ac6e5a0 con 0x55ea7ad4cd90 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f46637fe640 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7ad4cd90 0x55ea7ad4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f46637fe640 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7ad4cd90 0x55ea7ad4f210 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72f46bb04dfe254 server_cookie=8e40aac636170cb1 in_seq=0 out_seq=0 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.852+0000 7f4646ff5640 10 client.19224 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f4646ff5640 1 -- 172.21.15.33:0/195301071 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea7ac6e5a0 con 0x55ea7ad4cd90 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f4646ff5640 10 client.19224 handle_command_reply: tid=0 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 client.19224 shutdown 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 msgr2=0x55ea7ad416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad416f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f466419b190 tx=0x7f4658012b60 comp rx=0 tx=0).stop 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 shutdown_connections 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7ad4cd90 0x55ea7ad4f210 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad416f0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 --2- 172.21.15.33:0/195301071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad36d90 0x55ea7ad411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 >> 172.21.15.33:0/195301071 conn(0x55ea7acfe950 msgr2=0x55ea7ad31950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f4646ff5640 0 client.19224 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 shutdown_connections 2024-04-24T23:30:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 1 -- 172.21.15.33:0/195301071 wait complete. 2024-04-24T23:30:00.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.853+0000 7f466c244740 20 client.19224 trim_cache size 0 max 16384 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 Processor -- start 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 -- start start 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad36d90 0x55ea7ad310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad37780 0x55ea7ad486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad31600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46641aef30 con 0x55ea7ad36d90 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46641ac800 con 0x55ea7ad3c8f0 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f466c244740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46641aa650 con 0x55ea7ad37780 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad36d90 0x55ea7ad310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad36d90 0x55ea7ad310c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49806/0 (socket says 172.21.15.33:49806) 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46637fe640 1 -- 172.21.15.33:0/2505639721 learned_addr learned my addr 172.21.15.33:0/2505639721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46627fc640 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad37780 0x55ea7ad486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46637fe640 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46627fc640 1 -- 172.21.15.33:0/2505639721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 msgr2=0x55ea7ad31600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46627fc640 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad31600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46627fc640 1 -- 172.21.15.33:0/2505639721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad36d90 msgr2=0x55ea7ad310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46627fc640 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad36d90 0x55ea7ad310c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.874+0000 7f46627fc640 1 -- 172.21.15.33:0/2505639721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea7a96e170 con 0x55ea7ad37780 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.875+0000 7f46627fc640 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad37780 0x55ea7ad486f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4658009e00 tx=0x7f4658009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d9e56baf62f05d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.875+0000 7f4648ff9640 1 -- 172.21.15.33:0/2505639721 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465801cb50 con 0x55ea7ad37780 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.875+0000 7f4648ff9640 1 -- 172.21.15.33:0/2505639721 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465801ccf0 con 0x55ea7ad37780 2024-04-24T23:30:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.875+0000 7f4648ff9640 1 -- 172.21.15.33:0/2505639721 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580035f0 con 0x55ea7ad37780 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- 172.21.15.33:0/2505639721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad37780 msgr2=0x55ea7ad486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad37780 0x55ea7ad486f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4658009e00 tx=0x7f4658009ea0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- 172.21.15.33:0/2505639721 shutdown_connections 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad37780 0x55ea7ad486f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- 172.21.15.33:0/2505639721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad36d90 0x55ea7ad310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- 172.21.15.33:0/2505639721 >> 172.21.15.33:0/2505639721 conn(0x55ea7acfe950 msgr2=0x55ea7ad38be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- 172.21.15.33:0/2505639721 shutdown_connections 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- 172.21.15.33:0/2505639721 wait complete. 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- start start 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 Processor -- start 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad36d90 0x55ea7ad46430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad46970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad47910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46641aacc0 con 0x55ea7ad37780 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46641ac130 con 0x55ea7ad36d90 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.876+0000 7f466c244740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46641ab390 con 0x55ea7ad3c8f0 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f466c244740 10 client.0 _ll_register_callbacks cb 0x7f4658009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad46970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad36d90 0x55ea7ad46430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad46970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49814/0 (socket says 172.21.15.33:49814) 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 -- 172.21.15.33:0/4103336649 learned_addr learned my addr 172.21.15.33:0/4103336649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46637fe640 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad47910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 -- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad3c8f0 msgr2=0x55ea7ad47910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad47910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 -- 172.21.15.33:0/4103336649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad36d90 msgr2=0x55ea7ad46430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad36d90 0x55ea7ad46430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 -- 172.21.15.33:0/4103336649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea7ad47f00 con 0x55ea7ad37780 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f4646ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.877+0000 7f46627fc640 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad46970 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f465801ca50 tx=0x7f4658009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb95a09cca749895 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f4646ff5640 1 -- 172.21.15.33:0/4103336649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800f9a0 con 0x55ea7ad37780 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f4646ff5640 1 -- 172.21.15.33:0/4103336649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46580050c0 con 0x55ea7ad37780 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465801ccf0 con 0x55ea7ad37780 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f4646ff5640 1 -- 172.21.15.33:0/4103336649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f46580052e0 con 0x55ea7ad37780 2024-04-24T23:30:00.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 10 client.19230 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ea7adb3960 con 0x55ea7ad37780 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f4646ff5640 1 -- 172.21.15.33:0/4103336649 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465801ccf0 con 0x55ea7ad37780 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f4646ff5640 1 -- 172.21.15.33:0/4103336649 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4658031030 con 0x55ea7ad37780 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 10 client.19230 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 10 client.19230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 20 client.19230 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7adb41a0 0x55ea7adb65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 4 client.19230 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.878+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea7aca7800 con 0x55ea7adb41a0 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.879+0000 7f46637fe640 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7adb41a0 0x55ea7adb65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.879+0000 7f4646ff5640 10 client.19230 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.879+0000 7f46637fe640 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7adb41a0 0x55ea7adb65c0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ad6b7676221110 server_cookie=19174770aa49d50e in_seq=0 out_seq=0 2024-04-24T23:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f4646ff5640 1 -- 172.21.15.33:0/4103336649 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea7aca7800 con 0x55ea7adb41a0 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f4646ff5640 10 client.19230 handle_command_reply: tid=0 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 client.19230 shutdown 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 msgr2=0x55ea7ad46970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad46970 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f465801ca50 tx=0x7f4658009f90 comp rx=0 tx=0).stop 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 shutdown_connections 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ea7adb41a0 0x55ea7adb65c0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea7ad3c8f0 0x55ea7ad47910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f4646ff5640 0 client.19230 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea7ad37780 0x55ea7ad46970 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 --2- 172.21.15.33:0/4103336649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea7ad36d90 0x55ea7ad46430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 >> 172.21.15.33:0/4103336649 conn(0x55ea7acfe950 msgr2=0x55ea7ad357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 shutdown_connections 2024-04-24T23:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.881+0000 7f466c244740 1 -- 172.21.15.33:0/4103336649 wait complete. 2024-04-24T23:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f466c244740 20 client.19230 trim_cache size 0 max 16384 2024-04-24T23:30:00.886 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:30:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:30:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "db9e221b-28d2-45d5-a2a3-1495119f558e", 2024-04-24T23:30:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:30:00.886 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f464c0a2c10 msgr2=0x7f464c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f464c0a2c10 0x7f464c0a50d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4654098640 tx=0x7f46540d5530 comp rx=0 tx=0).stop 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 msgr2=0x7f4664199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664199c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f465400e9c0 tx=0x7f465400ee80 comp rx=0 tx=0).stop 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 shutdown_connections 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f464c0a2c10 0x7f464c0a50d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4664102250 0x7f46641a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4664101890 0x7f466419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 --2- 172.21.15.33:0/3225312105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641006a0 0x7f4664199c90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:00.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 >> 172.21.15.33:0/3225312105 conn(0x7f46640fbc10 msgr2=0x7f4664104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:00.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 shutdown_connections 2024-04-24T23:30:00.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:00.882+0000 7f4669534640 1 -- 172.21.15.33:0/3225312105 wait complete. 2024-04-24T23:30:00.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 107523 22.74 MB/sec execute 125 sec latency 56.728 ms 2024-04-24T23:30:00.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:00.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:01.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 -- 172.21.15.33:0/183936974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e801006a0 msgr2=0x7f1e80100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/183936974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e801006a0 0x7f1e80100ac0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009900 tx=0x7f1e7002a410 comp rx=0 tx=0).stop 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 -- 172.21.15.33:0/183936974 shutdown_connections 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/183936974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e80102250 0x7f1e80108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/183936974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e80101890 0x7f1e80101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/183936974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e801006a0 0x7f1e80100ac0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 -- 172.21.15.33:0/183936974 >> 172.21.15.33:0/183936974 conn(0x7f1e800fbc30 msgr2=0x7f1e800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 -- 172.21.15.33:0/183936974 shutdown_connections 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.012+0000 7f1e84e9a640 1 -- 172.21.15.33:0/183936974 wait complete. 2024-04-24T23:30:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.013+0000 7f1e84e9a640 1 Processor -- start 2024-04-24T23:30:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.013+0000 7f1e84e9a640 1 -- start start 2024-04-24T23:30:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.013+0000 7f1e84e9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e801006a0 0x7f1e80197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.013+0000 7f1e84e9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e80101890 0x7f1e80198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.013+0000 7f1e84e9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 0x7f1e8019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e84e9a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010d4a0 con 0x7f1e80102250 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e84e9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010d320 con 0x7f1e80101890 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e84e9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010d620 con 0x7f1e801006a0 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 0x7f1e8019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 0x7f1e8019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49826/0 (socket says 172.21.15.33:49826) 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 -- 172.21.15.33:0/2033229419 learned_addr learned my addr 172.21.15.33:0/2033229419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e801006a0 0x7f1e80197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e80101890 0x7f1e80198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 -- 172.21.15.33:0/2033229419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e801006a0 msgr2=0x7f1e80197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e801006a0 0x7f1e80197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 -- 172.21.15.33:0/2033229419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e80101890 msgr2=0x7f1e80198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e80101890 0x7f1e80198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 -- 172.21.15.33:0/2033229419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8019f720 con 0x7f1e80102250 2024-04-24T23:30:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e7ed9d640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 0x7f1e8019f0b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800eaa0 tx=0x7f1e7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:01.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e77fff640 1 -- 172.21.15.33:0/2033229419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800cdb0 con 0x7f1e80102250 2024-04-24T23:30:01.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8019fa10 con 0x7f1e80102250 2024-04-24T23:30:01.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e77fff640 1 -- 172.21.15.33:0/2033229419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e78005020 con 0x7f1e80102250 2024-04-24T23:30:01.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e77fff640 1 -- 172.21.15.33:0/2033229419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e780116b0 con 0x7f1e80102250 2024-04-24T23:30:01.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.014+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8019ff60 con 0x7f1e80102250 2024-04-24T23:30:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e77fff640 1 -- 172.21.15.33:0/2033229419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1e78005560 con 0x7f1e80102250 2024-04-24T23:30:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 Processor -- start 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 -- start start 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599cb84f6b0 con 0x5599cb8eb240 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5599cb7f7880 con 0x5599cb8ebc30 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5599cb84b670 con 0x5599cb8f0da0 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e7e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8eb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49836/0 (socket says 172.21.15.33:49836) 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e7e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:34248/0 (socket says 172.21.15.33:34248) 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e75d9b640 1 -- 172.21.15.33:0/3977617857 learned_addr learned my addr 172.21.15.33:0/3977617857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.017+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e77fff640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e6c0a2b90 0x7f1e6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7e59c640 1 -- 172.21.15.33:0/3977617857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 msgr2=0x5599cb8f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7e59c640 1 -- 172.21.15.33:0/3977617857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 msgr2=0x5599cb8eb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8eb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7e59c640 1 -- 172.21.15.33:0/3977617857 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cb62d7d0 con 0x5599cb8ebc30 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7ed9d640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e6c0a2b90 0x7f1e6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e77fff640 1 -- 172.21.15.33:0/2033229419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1e780cf570 con 0x7f1e80102250 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f0730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009230 tx=0x7f1e70005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92d41254f6e228a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e7ed9d640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e6c0a2b90 0x7f1e6c0a5050 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1e780cd050 tx=0x7f1e78066770 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e74d99640 1 -- 172.21.15.33:0/3977617857 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70035040 con 0x5599cb8ebc30 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e74d99640 1 -- 172.21.15.33:0/3977617857 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70004490 con 0x5599cb8ebc30 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.018+0000 7f1e74d99640 1 -- 172.21.15.33:0/3977617857 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700316b0 con 0x5599cb8ebc30 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3977617857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 msgr2=0x5599cb8f0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f0730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009230 tx=0x7f1e70005c00 comp rx=0 tx=0).stop 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3977617857 shutdown_connections 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f0730 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3977617857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3977617857 >> 172.21.15.33:0/3977617857 conn(0x5599cb8e68b0 msgr2=0x5599cb8e6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3977617857 shutdown_connections 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.019+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3977617857 wait complete. 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 -- start start 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 Processor -- start 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599cb8f3cf0 con 0x5599cb8eb240 2024-04-24T23:30:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5599cb80af50 con 0x5599cb8ebc30 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5599cb8f3e70 con 0x5599cb8f0da0 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e87ba1740 10 client.0 _ll_register_callbacks cb 0x5599cb846000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49852/0 (socket says 172.21.15.33:49852) 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 -- 172.21.15.33:0/2448961660 learned_addr learned my addr 172.21.15.33:0/2448961660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 -- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 msgr2=0x5599cb8fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 -- 172.21.15.33:0/2448961660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 msgr2=0x5599cb8f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e75d9b640 1 -- 172.21.15.33:0/2448961660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cb5633f0 con 0x5599cb8eb240 2024-04-24T23:30:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.020+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5600 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1e68013e90 tx=0x7f1e68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e5832814792acc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e65ffb640 1 -- 172.21.15.33:0/2448961660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004050 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e65ffb640 1 -- 172.21.15.33:0/2448961660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1e680041f0 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70004490 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e65ffb640 1 -- 172.21.15.33:0/2448961660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1e6802a050 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 10 client.19242 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599cb5230f0 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e65ffb640 1 -- 172.21.15.33:0/2448961660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70004490 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e65ffb640 1 -- 172.21.15.33:0/2448961660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1e68025030 con 0x5599cb8eb240 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 10 client.19242 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 10 client.19242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 20 client.19242 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb901440 0x5599cb9038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 4 client.19242 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.021+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599cb85cbf0 con 0x5599cb901440 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.022+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb901440 0x5599cb9038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.022+0000 7f1e65ffb640 10 client.19242 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.022+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb901440 0x5599cb9038e0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe1590847b4e160 server_cookie=24aaeb050fac275b in_seq=0 out_seq=0 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e65ffb640 1 -- 172.21.15.33:0/2448961660 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599cb85cbf0 con 0x5599cb901440 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e65ffb640 10 client.19242 handle_command_reply: tid=0 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 client.19242 shutdown 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 msgr2=0x5599cb8f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5600 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1e68013e90 tx=0x7f1e68005f90 comp rx=0 tx=0).stop 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 shutdown_connections 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb901440 0x5599cb9038e0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e65ffb640 0 client.19242 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/2448961660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5600 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 >> 172.21.15.33:0/2448961660 conn(0x5599cb8e68b0 msgr2=0x5599cb8e5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 shutdown_connections 2024-04-24T23:30:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 1 -- 172.21.15.33:0/2448961660 wait complete. 2024-04-24T23:30:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.023+0000 7f1e87ba1740 20 client.19242 trim_cache size 0 max 16384 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 Processor -- start 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 -- start start 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801accd0 con 0x5599cb8eb240 2024-04-24T23:30:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801aa5a0 con 0x5599cb8ebc30 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801a83f0 con 0x5599cb8f0da0 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49868/0 (socket says 172.21.15.33:49868) 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e75d9b640 1 -- 172.21.15.33:0/1935312445 learned_addr learned my addr 172.21.15.33:0/1935312445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.041+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 -- 172.21.15.33:0/1935312445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 msgr2=0x5599cb8f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 -- 172.21.15.33:0/1935312445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 msgr2=0x5599cb8fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 -- 172.21.15.33:0/1935312445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cb8e7300 con 0x5599cb8eb240 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5210 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f1e680099c0 tx=0x7f1e68009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d355bcddf7efb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e67fff640 1 -- 172.21.15.33:0/1935312445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68005070 con 0x5599cb8eb240 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e67fff640 1 -- 172.21.15.33:0/1935312445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e680035f0 con 0x5599cb8eb240 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.042+0000 7f1e67fff640 1 -- 172.21.15.33:0/1935312445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68026660 con 0x5599cb8eb240 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- 172.21.15.33:0/1935312445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 msgr2=0x5599cb8f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5210 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f1e680099c0 tx=0x7f1e68009a50 comp rx=0 tx=0).stop 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- 172.21.15.33:0/1935312445 shutdown_connections 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb8f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/1935312445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f5210 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- 172.21.15.33:0/1935312445 >> 172.21.15.33:0/1935312445 conn(0x5599cb8e68b0 msgr2=0x5599cb8f48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- 172.21.15.33:0/1935312445 shutdown_connections 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- 172.21.15.33:0/1935312445 wait complete. 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- start start 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 Processor -- start 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb969e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801a8a60 con 0x5599cb8eb240 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801a9ed0 con 0x5599cb8ebc30 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.043+0000 7f1e87ba1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801a9130 con 0x5599cb8f0da0 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e87ba1740 10 client.0 _ll_register_callbacks cb 0x7f1e68013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49880/0 (socket says 172.21.15.33:49880) 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 -- 172.21.15.33:0/3762036213 learned_addr learned my addr 172.21.15.33:0/3762036213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb969e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 -- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 msgr2=0x5599cb969e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb969e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 -- 172.21.15.33:0/3762036213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 msgr2=0x5599cb8fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 -- 172.21.15.33:0/3762036213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cb96a410 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e75d9b640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f9b30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1e68016040 tx=0x7f1e68009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a33dac37bf099b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e65ffb640 1 -- 172.21.15.33:0/3762036213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68010c60 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e65ffb640 1 -- 172.21.15.33:0/3762036213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1e68010e00 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.044+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e680035f0 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e65ffb640 1 -- 172.21.15.33:0/3762036213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1e68043050 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 10 client.19248 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599cb96b6c0 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e65ffb640 1 -- 172.21.15.33:0/3762036213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e680035f0 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e65ffb640 1 -- 172.21.15.33:0/3762036213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1e68025040 con 0x5599cb8eb240 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 10 client.19248 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 20 client.19248 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb96bed0 0x5599cb96e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 4 client.19248 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599cb856f40 con 0x5599cb96bed0 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb96bed0 0x5599cb96e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e65ffb640 10 client.19248 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.045+0000 7f1e7e59c640 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb96bed0 0x5599cb96e2f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada83dc7ca41f322 server_cookie=87c80e361f1c146e in_seq=0 out_seq=0 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e65ffb640 1 -- 172.21.15.33:0/3762036213 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5599cb856f40 con 0x5599cb96bed0 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e65ffb640 10 client.19248 handle_command_reply: tid=0 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 client.19248 shutdown 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 msgr2=0x5599cb8f9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f9b30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1e68016040 tx=0x7f1e68009950 comp rx=0 tx=0).stop 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 shutdown_connections 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5599cb96bed0 0x5599cb96e2f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599cb8f0da0 0x5599cb969e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e65ffb640 0 client.19248 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599cb8ebc30 0x5599cb8fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 --2- 172.21.15.33:0/3762036213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599cb8eb240 0x5599cb8f9b30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 >> 172.21.15.33:0/3762036213 conn(0x5599cb8e68b0 msgr2=0x5599cb8f5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 shutdown_connections 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 1 -- 172.21.15.33:0/3762036213 wait complete. 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e87ba1740 20 client.19248 trim_cache size 0 max 16384 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:30:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "db9e221b-28d2-45d5-a2a3-1495119f558e": { 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "db9e221b-28d2-45d5-a2a3-1495119f558e", 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:30:01.055 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e6c0a2b90 msgr2=0x7f1e6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e6c0a2b90 0x7f1e6c0a5050 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1e780cd050 tx=0x7f1e78066770 comp rx=0 tx=0).stop 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 msgr2=0x7f1e8019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 0x7f1e8019f0b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800eaa0 tx=0x7f1e7800ef60 comp rx=0 tx=0).stop 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 shutdown_connections 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f1e6c0a2b90 0x7f1e6c0a5050 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e80102250 0x7f1e8019f0b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e80101890 0x7f1e80198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 --2- 172.21.15.33:0/2033229419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e801006a0 0x7f1e80197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.046+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 >> 172.21.15.33:0/2033229419 conn(0x7f1e800fbc30 msgr2=0x7f1e80104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.047+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 shutdown_connections 2024-04-24T23:30:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:01.047+0000 7f1e84e9a640 1 -- 172.21.15.33:0/2033229419 wait complete. 2024-04-24T23:30:01.058 INFO:tasks.cephfs.filesystem:scrub status for tag:db9e221b-28d2-45d5-a2a3-1495119f558e - {'path': '/', 'tag': 'db9e221b-28d2-45d5-a2a3-1495119f558e', 'options': 'recursive,force'} 2024-04-24T23:30:01.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:01 smithi038 ceph-mon[32450]: pgmap v891: 129 pgs: 129 active+clean; 541 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 385 op/s 2024-04-24T23:30:01.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:01 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:01.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:01 smithi117 ceph-mon[32529]: pgmap v891: 129 pgs: 129 active+clean; 541 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 385 op/s 2024-04-24T23:30:01.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:01 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:01.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:01 smithi033 ceph-mon[29201]: pgmap v891: 129 pgs: 129 active+clean; 541 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 385 op/s 2024-04-24T23:30:01.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:01 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:01.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 107942 22.74 MB/sec execute 126 sec latency 117.660 ms 2024-04-24T23:30:02.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:30:02.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:02.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:02.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:02.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:30:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:02.888 INFO:tasks.workunit.client.0.smithi033.stdout: 10 108227 22.70 MB/sec execute 127 sec latency 108.454 ms 2024-04-24T23:30:03.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:03 smithi033 ceph-mon[29201]: pgmap v892: 129 pgs: 129 active+clean; 544 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-04-24T23:30:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:03 smithi038 ceph-mon[32450]: pgmap v892: 129 pgs: 129 active+clean; 544 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-04-24T23:30:03.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:03 smithi117 ceph-mon[32529]: pgmap v892: 129 pgs: 129 active+clean; 544 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-04-24T23:30:03.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 108399 22.57 MB/sec execute 128 sec latency 403.293 ms 2024-04-24T23:30:04.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 108692 22.49 MB/sec execute 129 sec latency 125.279 ms 2024-04-24T23:30:05.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:05 smithi038 ceph-mon[32450]: pgmap v893: 129 pgs: 129 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 410 op/s 2024-04-24T23:30:05.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:05 smithi117 ceph-mon[32529]: pgmap v893: 129 pgs: 129 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 410 op/s 2024-04-24T23:30:05.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:05 smithi033 ceph-mon[29201]: pgmap v893: 129 pgs: 129 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 410 op/s 2024-04-24T23:30:05.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 109067 22.45 MB/sec execute 130 sec latency 110.874 ms 2024-04-24T23:30:06.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 109789 22.54 MB/sec execute 131 sec latency 49.114 ms 2024-04-24T23:30:07.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:07 smithi038 ceph-mon[32450]: pgmap v894: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.6 MiB/s wr, 448 op/s 2024-04-24T23:30:07.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:07 smithi117 ceph-mon[32529]: pgmap v894: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.6 MiB/s wr, 448 op/s 2024-04-24T23:30:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:07 smithi033 ceph-mon[29201]: pgmap v894: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.6 MiB/s wr, 448 op/s 2024-04-24T23:30:07.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 110415 22.66 MB/sec execute 132 sec latency 78.236 ms 2024-04-24T23:30:08.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 110649 22.62 MB/sec execute 133 sec latency 289.751 ms 2024-04-24T23:30:09.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:09 smithi038 ceph-mon[32450]: pgmap v895: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 459 op/s 2024-04-24T23:30:09.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:09 smithi117 ceph-mon[32529]: pgmap v895: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 459 op/s 2024-04-24T23:30:09.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:09 smithi033 ceph-mon[29201]: pgmap v895: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 459 op/s 2024-04-24T23:30:09.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 111005 22.62 MB/sec execute 134 sec latency 133.955 ms 2024-04-24T23:30:10.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 111404 22.59 MB/sec execute 135 sec latency 127.960 ms 2024-04-24T23:30:11.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:11 smithi038 ceph-mon[32450]: pgmap v896: 129 pgs: 129 active+clean; 528 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.4 MiB/s wr, 404 op/s 2024-04-24T23:30:11.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:11 smithi117 ceph-mon[32529]: pgmap v896: 129 pgs: 129 active+clean; 528 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.4 MiB/s wr, 404 op/s 2024-04-24T23:30:11.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:11 smithi033 ceph-mon[29201]: pgmap v896: 129 pgs: 129 active+clean; 528 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.4 MiB/s wr, 404 op/s 2024-04-24T23:30:11.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 111894 22.63 MB/sec execute 136 sec latency 128.490 ms 2024-04-24T23:30:12.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 112383 22.64 MB/sec execute 137 sec latency 59.157 ms 2024-04-24T23:30:13.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:13 smithi038 ceph-mon[32450]: pgmap v897: 129 pgs: 129 active+clean; 529 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 442 op/s 2024-04-24T23:30:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:30:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:30:13.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:30:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:13 smithi117 ceph-mon[32529]: pgmap v897: 129 pgs: 129 active+clean; 529 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 442 op/s 2024-04-24T23:30:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:30:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:30:13.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:30:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:13 smithi033 ceph-mon[29201]: pgmap v897: 129 pgs: 129 active+clean; 529 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 442 op/s 2024-04-24T23:30:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:30:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:30:13.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:30:13.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 112754 22.62 MB/sec execute 138 sec latency 287.269 ms 2024-04-24T23:30:14.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 113058 22.58 MB/sec execute 139 sec latency 216.581 ms 2024-04-24T23:30:15.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:15 smithi033 ceph-mon[29201]: pgmap v898: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 441 op/s 2024-04-24T23:30:15.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:15 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:30:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:15 smithi038 ceph-mon[32450]: pgmap v898: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 441 op/s 2024-04-24T23:30:15.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:15 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:30:15.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:15 smithi117 ceph-mon[32529]: pgmap v898: 129 pgs: 129 active+clean; 536 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 441 op/s 2024-04-24T23:30:15.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:15 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:30:15.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 113578 22.61 MB/sec execute 140 sec latency 97.236 ms 2024-04-24T23:30:16.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: Adjusting osd_memory_target on smithi117 to 176.2M 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: Unable to set osd_memory_target on smithi117 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:16.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: Adjusting osd_memory_target on smithi117 to 176.2M 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: Unable to set osd_memory_target on smithi117 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:30:16.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:16.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:16.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: Adjusting osd_memory_target on smithi117 to 176.2M 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: Unable to set osd_memory_target on smithi117 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:16.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:30:16.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 114034 22.62 MB/sec execute 141 sec latency 63.331 ms 2024-04-24T23:30:17.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 114466 22.60 MB/sec execute 142 sec latency 77.277 ms 2024-04-24T23:30:17.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:17 smithi038 ceph-mon[32450]: pgmap v899: 129 pgs: 129 active+clean; 536 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.0 MiB/s wr, 448 op/s 2024-04-24T23:30:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:17 smithi117 ceph-mon[32529]: pgmap v899: 129 pgs: 129 active+clean; 536 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.0 MiB/s wr, 448 op/s 2024-04-24T23:30:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:17 smithi033 ceph-mon[29201]: pgmap v899: 129 pgs: 129 active+clean; 536 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.0 MiB/s wr, 448 op/s 2024-04-24T23:30:18.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 114851 22.60 MB/sec execute 143 sec latency 252.401 ms 2024-04-24T23:30:19.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 115370 22.65 MB/sec execute 144 sec latency 148.533 ms 2024-04-24T23:30:19.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:19 smithi038 ceph-mon[32450]: pgmap v900: 129 pgs: 129 active+clean; 543 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.4 MiB/s wr, 438 op/s 2024-04-24T23:30:19.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:19 smithi117 ceph-mon[32529]: pgmap v900: 129 pgs: 129 active+clean; 543 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.4 MiB/s wr, 438 op/s 2024-04-24T23:30:19.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:19 smithi033 ceph-mon[29201]: pgmap v900: 129 pgs: 129 active+clean; 543 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.4 MiB/s wr, 438 op/s 2024-04-24T23:30:20.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 115812 22.69 MB/sec execute 145 sec latency 117.137 ms 2024-04-24T23:30:21.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 116084 22.64 MB/sec execute 146 sec latency 66.812 ms 2024-04-24T23:30:21.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:21 smithi038 ceph-mon[32450]: pgmap v901: 129 pgs: 129 active+clean; 544 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 360 op/s 2024-04-24T23:30:21.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:21 smithi117 ceph-mon[32529]: pgmap v901: 129 pgs: 129 active+clean; 544 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 360 op/s 2024-04-24T23:30:21.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:21 smithi033 ceph-mon[29201]: pgmap v901: 129 pgs: 129 active+clean; 544 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 360 op/s 2024-04-24T23:30:22.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 116448 22.59 MB/sec execute 147 sec latency 76.990 ms 2024-04-24T23:30:23.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 116829 22.54 MB/sec execute 148 sec latency 160.845 ms 2024-04-24T23:30:23.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:23 smithi038 ceph-mon[32450]: pgmap v902: 129 pgs: 129 active+clean; 545 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 446 op/s 2024-04-24T23:30:23.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:23 smithi117 ceph-mon[32529]: pgmap v902: 129 pgs: 129 active+clean; 545 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 446 op/s 2024-04-24T23:30:23.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:23 smithi033 ceph-mon[29201]: pgmap v902: 129 pgs: 129 active+clean; 545 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 446 op/s 2024-04-24T23:30:24.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 117129 22.53 MB/sec execute 149 sec latency 340.952 ms 2024-04-24T23:30:25.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 117554 22.53 MB/sec execute 150 sec latency 107.390 ms 2024-04-24T23:30:25.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:25 smithi038 ceph-mon[32450]: pgmap v903: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.7 MiB/s wr, 445 op/s 2024-04-24T23:30:25.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:25 smithi117 ceph-mon[32529]: pgmap v903: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.7 MiB/s wr, 445 op/s 2024-04-24T23:30:25.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:25 smithi033 ceph-mon[29201]: pgmap v903: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.7 MiB/s wr, 445 op/s 2024-04-24T23:30:26.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 118221 22.57 MB/sec execute 151 sec latency 60.924 ms 2024-04-24T23:30:27.889 INFO:tasks.workunit.client.0.smithi033.stdout: 10 118691 22.64 MB/sec execute 152 sec latency 71.553 ms 2024-04-24T23:30:27.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:27 smithi038 ceph-mon[32450]: pgmap v904: 129 pgs: 129 active+clean; 550 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.8 MiB/s wr, 416 op/s 2024-04-24T23:30:27.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:27 smithi117 ceph-mon[32529]: pgmap v904: 129 pgs: 129 active+clean; 550 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.8 MiB/s wr, 416 op/s 2024-04-24T23:30:27.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:27 smithi033 ceph-mon[29201]: pgmap v904: 129 pgs: 129 active+clean; 550 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.8 MiB/s wr, 416 op/s 2024-04-24T23:30:28.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 119239 22.70 MB/sec execute 153 sec latency 93.585 ms 2024-04-24T23:30:29.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 119401 22.64 MB/sec execute 154 sec latency 322.908 ms 2024-04-24T23:30:29.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:29 smithi038 ceph-mon[32450]: pgmap v905: 129 pgs: 129 active+clean; 546 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 428 op/s 2024-04-24T23:30:29.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:29 smithi117 ceph-mon[32529]: pgmap v905: 129 pgs: 129 active+clean; 546 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 428 op/s 2024-04-24T23:30:29.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:29 smithi033 ceph-mon[29201]: pgmap v905: 129 pgs: 129 active+clean; 546 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 428 op/s 2024-04-24T23:30:30.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 119680 22.58 MB/sec execute 155 sec latency 122.278 ms 2024-04-24T23:30:31.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:31.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:31.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 -- 172.21.15.33:0/3472691777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 msgr2=0x7ff254106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 --2- 172.21.15.33:0/3472691777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 0x7ff254106f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff248009860 tx=0x7ff24802a360 comp rx=0 tx=0).stop 2024-04-24T23:30:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 -- 172.21.15.33:0/3472691777 shutdown_connections 2024-04-24T23:30:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 --2- 172.21.15.33:0/3472691777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2541074c0 0x7ff2541098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 --2- 172.21.15.33:0/3472691777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 0x7ff254106f80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 --2- 172.21.15.33:0/3472691777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff254102260 0x7ff254104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 -- 172.21.15.33:0/3472691777 >> 172.21.15.33:0/3472691777 conn(0x7ff2540fbbf0 msgr2=0x7ff2540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 -- 172.21.15.33:0/3472691777 shutdown_connections 2024-04-24T23:30:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.174+0000 7ff25975a640 1 -- 172.21.15.33:0/3472691777 wait complete. 2024-04-24T23:30:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 Processor -- start 2024-04-24T23:30:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 -- start start 2024-04-24T23:30:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 0x7ff254101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 0x7ff254100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2541074c0 0x7ff254100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410d3d0 con 0x7ff254102260 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410d250 con 0x7ff2541074c0 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff25975a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410d550 con 0x7ff254104b90 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 0x7ff254101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 0x7ff254101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36690/0 (socket says 172.21.15.33:36690) 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff252ffd640 1 -- 172.21.15.33:0/2056227388 learned_addr learned my addr 172.21.15.33:0/2056227388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff2537fe640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2541074c0 0x7ff254100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.175+0000 7ff2527fc640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 0x7ff254100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff252ffd640 1 -- 172.21.15.33:0/2056227388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 msgr2=0x7ff254100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff252ffd640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 0x7ff254100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff252ffd640 1 -- 172.21.15.33:0/2056227388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2541074c0 msgr2=0x7ff254100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff252ffd640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2541074c0 0x7ff254100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff252ffd640 1 -- 172.21.15.33:0/2056227388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff254100fe0 con 0x7ff254102260 2024-04-24T23:30:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff2537fe640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2541074c0 0x7ff254100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff252ffd640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 0x7ff254101bd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e9f0 tx=0x7ff24400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff23bfff640 1 -- 172.21.15.33:0/2056227388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400cd80 con 0x7ff254102260 2024-04-24T23:30:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2541a1c00 con 0x7ff254102260 2024-04-24T23:30:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2541a2170 con 0x7ff254102260 2024-04-24T23:30:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff23bfff640 1 -- 172.21.15.33:0/2056227388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff244010910 con 0x7ff254102260 2024-04-24T23:30:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.176+0000 7ff23bfff640 1 -- 172.21.15.33:0/2056227388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244018980 con 0x7ff254102260 2024-04-24T23:30:31.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.179+0000 7ff25c464740 1 Processor -- start 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.179+0000 7ff23bfff640 1 -- 172.21.15.33:0/2056227388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff244018b20 con 0x7ff254102260 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.179+0000 7ff25c464740 1 -- start start 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff25c464740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879d240 0x56245879d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff25c464740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879dc30 0x5624587a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff25c464740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587a5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff25c464740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5624587016b0 con 0x5624587a2da0 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff25c464740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5624586a9880 con 0x56245879dc30 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff25c464740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624586fd670 con 0x56245879d240 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879dc30 0x5624587a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587a5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36694/0 (socket says 172.21.15.33:36694) 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 -- 172.21.15.33:0/2889231613 learned_addr learned my addr 172.21.15.33:0/2889231613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2537fe640 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879d240 0x56245879d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 -- 172.21.15.33:0/2889231613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879d240 msgr2=0x56245879d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879d240 0x56245879d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 -- 172.21.15.33:0/2889231613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879dc30 msgr2=0x5624587a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879dc30 0x5624587a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff2527fc640 1 -- 172.21.15.33:0/2889231613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624584df7d0 con 0x5624587a2da0 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff252ffd640 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879dc30 0x5624587a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.180+0000 7ff23bfff640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff23c0a2c10 0x7ff23c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.181+0000 7ff2527fc640 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587a5230 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff24802a950 tx=0x7ff248000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c955fa2a31e12d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.181+0000 7ff23bfff640 1 -- 172.21.15.33:0/2056227388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2440cf780 con 0x7ff254102260 2024-04-24T23:30:31.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.181+0000 7ff2397fa640 1 -- 172.21.15.33:0/2889231613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248037040 con 0x5624587a2da0 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.181+0000 7ff2537fe640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff23c0a2c10 0x7ff23c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.181+0000 7ff2397fa640 1 -- 172.21.15.33:0/2889231613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff248009470 con 0x5624587a2da0 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.181+0000 7ff2397fa640 1 -- 172.21.15.33:0/2889231613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248003a80 con 0x5624587a2da0 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 -- 172.21.15.33:0/2889231613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 msgr2=0x5624587a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587a5230 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff24802a950 tx=0x7ff248000bc0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff2537fe640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff23c0a2c10 0x7ff23c0a50d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x56245879e490 tx=0x7ff24c010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 -- 172.21.15.33:0/2889231613 shutdown_connections 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587a5230 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879dc30 0x5624587a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 --2- 172.21.15.33:0/2889231613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879d240 0x56245879d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 -- 172.21.15.33:0/2889231613 >> 172.21.15.33:0/2889231613 conn(0x5624587988b0 msgr2=0x562458798cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 -- 172.21.15.33:0/2889231613 shutdown_connections 2024-04-24T23:30:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 -- 172.21.15.33:0/2889231613 wait complete. 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.182+0000 7ff25c464740 1 -- start start 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 Processor -- start 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 0x5624587a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624587a2da0 0x5624587aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5624587a5cf0 con 0x56245879dc30 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5624586bcf50 con 0x56245879d240 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624587a5e70 con 0x5624587a2da0 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff25c464740 10 client.0 _ll_register_callbacks cb 0x5624586f8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 0x5624587a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 0x5624587a7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36696/0 (socket says 172.21.15.33:36696) 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 -- 172.21.15.33:0/1224333237 learned_addr learned my addr 172.21.15.33:0/1224333237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2537fe640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624587a2da0 0x5624587aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 -- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624587a2da0 msgr2=0x5624587aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624587a2da0 0x5624587aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 -- 172.21.15.33:0/1224333237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 msgr2=0x5624587a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.183+0000 7ff2527fc640 1 -- 172.21.15.33:0/1224333237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624584153f0 con 0x56245879dc30 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 0x5624587a7b10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff248030480 tx=0x7ff248030770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2e940705520e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff236ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff236ff5640 1 -- 172.21.15.33:0/1224333237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248037040 con 0x56245879dc30 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff236ff5640 1 -- 172.21.15.33:0/1224333237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff248004030 con 0x56245879dc30 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff248009470 con 0x56245879dc30 2024-04-24T23:30:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff236ff5640 1 -- 172.21.15.33:0/1224333237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff248004230 con 0x56245879dc30 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff25c464740 10 client.19260 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5624583d50f0 con 0x56245879dc30 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff236ff5640 1 -- 172.21.15.33:0/1224333237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff248009470 con 0x56245879dc30 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff236ff5640 1 -- 172.21.15.33:0/1224333237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff24803d6e0 con 0x56245879dc30 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff25c464740 10 client.19260 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.184+0000 7ff25c464740 10 client.19260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff25c464740 20 client.19260 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff25c464740 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5624587b3580 0x5624587b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff25c464740 4 client.19260 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56245870ebf0 con 0x5624587b3580 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5624587b3580 0x5624587b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff236ff5640 10 client.19260 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.185+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5624587b3580 0x5624587b5a20 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f23d3d035c41c5 server_cookie=ef32ab7f126b5734 in_seq=0 out_seq=0 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff236ff5640 1 -- 172.21.15.33:0/1224333237 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56245870ebf0 con 0x5624587b3580 2024-04-24T23:30:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff236ff5640 10 client.19260 handle_command_reply: tid=0 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 client.19260 shutdown 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 msgr2=0x5624587a7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 0x5624587a7b10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff248030480 tx=0x7ff248030770 comp rx=0 tx=0).stop 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 shutdown_connections 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5624587b3580 0x5624587b5a20 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624587a2da0 0x5624587aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879dc30 0x5624587a7b10 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 --2- 172.21.15.33:0/1224333237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 >> 172.21.15.33:0/1224333237 conn(0x5624587988b0 msgr2=0x562458797af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff236ff5640 0 client.19260 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 shutdown_connections 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.186+0000 7ff25c464740 1 -- 172.21.15.33:0/1224333237 wait complete. 2024-04-24T23:30:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.187+0000 7ff25c464740 20 client.19260 trim_cache size 0 max 16384 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 Processor -- start 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 -- start start 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 0x5624587a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624587a2da0 0x5624587a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff240012490 con 0x56245879d240 2024-04-24T23:30:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000fd60 con 0x5624587a2da0 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff25c464740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000dbb0 con 0x56245879dc30 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 0x5624587a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 0x5624587a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36710/0 (socket says 172.21.15.33:36710) 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff252ffd640 1 -- 172.21.15.33:0/1277065272 learned_addr learned my addr 172.21.15.33:0/1277065272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.210+0000 7ff2537fe640 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624587a2da0 0x5624587a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff252ffd640 1 -- 172.21.15.33:0/1277065272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 msgr2=0x5624587aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff252ffd640 1 -- 172.21.15.33:0/1277065272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624587a2da0 msgr2=0x5624587a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624587a2da0 0x5624587a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff252ffd640 1 -- 172.21.15.33:0/1277065272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245879ea40 con 0x56245879d240 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 0x5624587a62a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff244000c00 tx=0x7ff244018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36d53930f61cada server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff238ff9640 1 -- 172.21.15.33:0/1277065272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2440b2070 con 0x56245879d240 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff238ff9640 1 -- 172.21.15.33:0/1277065272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2440a4860 con 0x56245879d240 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.211+0000 7ff238ff9640 1 -- 172.21.15.33:0/1277065272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2440d1bc0 con 0x56245879d240 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 -- 172.21.15.33:0/1277065272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 msgr2=0x5624587a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 0x5624587a62a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff244000c00 tx=0x7ff244018420 comp rx=0 tx=0).stop 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 -- 172.21.15.33:0/1277065272 shutdown_connections 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624587a2da0 0x5624587a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 --2- 172.21.15.33:0/1277065272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56245879d240 0x5624587a62a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 -- 172.21.15.33:0/1277065272 >> 172.21.15.33:0/1277065272 conn(0x5624587988b0 msgr2=0x56245879ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 -- 172.21.15.33:0/1277065272 shutdown_connections 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 -- 172.21.15.33:0/1277065272 wait complete. 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 -- start start 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.212+0000 7ff25c464740 1 Processor -- start 2024-04-24T23:30:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000e220 con 0x5624587a2da0 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000f690 con 0x56245879d240 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000e8f0 con 0x56245879dc30 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff25c464740 10 client.0 _ll_register_callbacks cb 0x7ff244005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff2537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff2537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36716/0 (socket says 172.21.15.33:36716) 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54808/0 (socket says 172.21.15.33:54808) 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff2537fe640 1 -- 172.21.15.33:0/1723220559 learned_addr learned my addr 172.21.15.33:0/1723220559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 -- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 msgr2=0x5624587ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 -- 172.21.15.33:0/1723220559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 msgr2=0x5624587ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 -- 172.21.15.33:0/1723220559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624587abe40 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff2537fe640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff252ffd640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587ae7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff24409a1f0 tx=0x7ff244098470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a73fb875f3e5ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.213+0000 7ff236ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff236ff5640 1 -- 172.21.15.33:0/1723220559 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2440b2070 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2440a4860 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff236ff5640 1 -- 172.21.15.33:0/1723220559 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff244010530 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff236ff5640 1 -- 172.21.15.33:0/1723220559 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff244010ac0 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 10 client.28748 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562458818bf0 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff236ff5640 1 -- 172.21.15.33:0/1723220559 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2440a4860 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff236ff5640 1 -- 172.21.15.33:0/1723220559 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff24409b180 con 0x56245879d240 2024-04-24T23:30:31.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 10 client.28748 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 10 client.28748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 20 client.28748 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562458819580 0x56245881b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 4 client.28748 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562458708f40 con 0x562458819580 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.214+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562458819580 0x56245881b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff236ff5640 10 client.28748 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff2527fc640 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562458819580 0x56245881b9a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b6fcb09236cc54 server_cookie=cda5276174e7aa1d in_seq=0 out_seq=0 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff236ff5640 1 -- 172.21.15.33:0/1723220559 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562458708f40 con 0x562458819580 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff236ff5640 10 client.28748 handle_command_reply: tid=0 2024-04-24T23:30:31.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 client.28748 shutdown 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 msgr2=0x5624587ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587ae7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff24409a1f0 tx=0x7ff244098470 comp rx=0 tx=0).stop 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 shutdown_connections 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x562458819580 0x56245881b9a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624587a2da0 0x5624587ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56245879dc30 0x5624587ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 --2- 172.21.15.33:0/1723220559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56245879d240 0x5624587ae7f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff236ff5640 0 client.28748 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 >> 172.21.15.33:0/1723220559 conn(0x5624587988b0 msgr2=0x5624587a62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 shutdown_connections 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 1 -- 172.21.15.33:0/1723220559 wait complete. 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.215+0000 7ff25c464740 20 client.28748 trim_cache size 0 max 16384 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:30:31.224 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:30:31.225 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:30:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff23c0a2c10 msgr2=0x7ff23c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff23c0a2c10 0x7ff23c0a50d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x56245879e490 tx=0x7ff24c010040 comp rx=0 tx=0).stop 2024-04-24T23:30:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 msgr2=0x7ff254101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 0x7ff254101bd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e9f0 tx=0x7ff24400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 shutdown_connections 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff23c0a2c10 0x7ff23c0a50d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2541074c0 0x7ff254100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff254104b90 0x7ff254100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 --2- 172.21.15.33:0/2056227388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254102260 0x7ff254101bd0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 >> 172.21.15.33:0/2056227388 conn(0x7ff2540fbbf0 msgr2=0x7ff254102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 shutdown_connections 2024-04-24T23:30:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.216+0000 7ff25975a640 1 -- 172.21.15.33:0/2056227388 wait complete. 2024-04-24T23:30:31.229 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:30:31.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:30:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 -- 172.21.15.33:0/3093736541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4100680 msgr2=0x7f50a4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 --2- 172.21.15.33:0/3093736541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4100680 0x7f50a4100aa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a002a410 comp rx=0 tx=0).stop 2024-04-24T23:30:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 -- 172.21.15.33:0/3093736541 shutdown_connections 2024-04-24T23:30:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 --2- 172.21.15.33:0/3093736541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50a4102230 0x7f50a4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 --2- 172.21.15.33:0/3093736541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a4101870 0x7f50a4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 --2- 172.21.15.33:0/3093736541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4100680 0x7f50a4100aa0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 -- 172.21.15.33:0/3093736541 >> 172.21.15.33:0/3093736541 conn(0x7f50a40fbbf0 msgr2=0x7f50a40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 -- 172.21.15.33:0/3093736541 shutdown_connections 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.351+0000 7f50ac67d640 1 -- 172.21.15.33:0/3093736541 wait complete. 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 Processor -- start 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 -- start start 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50a4100680 0x7f50a4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 0x7f50a4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a4102230 0x7f50a419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410c350 con 0x7f50a4101870 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410c1d0 con 0x7f50a4102230 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50ac67d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410c4d0 con 0x7f50a4100680 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 0x7f50a4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 0x7f50a4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36734/0 (socket says 172.21.15.33:36734) 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 -- 172.21.15.33:0/2077716845 learned_addr learned my addr 172.21.15.33:0/2077716845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50aa419640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50a4100680 0x7f50a4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50aac1a640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a4102230 0x7f50a419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 -- 172.21.15.33:0/2077716845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50a4100680 msgr2=0x7f50a4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50a4100680 0x7f50a4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 -- 172.21.15.33:0/2077716845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a4102230 msgr2=0x7f50a419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a4102230 0x7f50a419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 -- 172.21.15.33:0/2077716845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a419f6d0 con 0x7f50a4101870 2024-04-24T23:30:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.352+0000 7f50a9c18640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 0x7f50a4197ff0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f509400e9f0 tx=0x7f509400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.353+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509400cd80 con 0x7f50a4101870 2024-04-24T23:30:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.353+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5094010910 con 0x7f50a4101870 2024-04-24T23:30:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.353+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a419f9c0 con 0x7f50a4101870 2024-04-24T23:30:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.353+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094018990 con 0x7f50a4101870 2024-04-24T23:30:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.353+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a419ff60 con 0x7f50a4101870 2024-04-24T23:30:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.354+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a41a0190 con 0x7f50a4101870 2024-04-24T23:30:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.356+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5094010ac0 con 0x7f50a4101870 2024-04-24T23:30:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.356+0000 7f509b7fe640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f50900a2c10 0x7f50900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.357+0000 7f50aa419640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f50900a2c10 0x7f50900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.357+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f50940cfa90 con 0x7f50a4101870 2024-04-24T23:30:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.357+0000 7f50aa419640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f50900a2c10 0x7f50900a50d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f50a002aa00 tx=0x7f50a00315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.360+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f509409bfb0 con 0x7f50a4101870 2024-04-24T23:30:31.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.632+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50a41a0360 con 0x7f50a4101870 2024-04-24T23:30:31.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.633+0000 7f509b7fe640 1 -- 172.21.15.33:0/2077716845 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f50940a02b0 con 0x7f50a4101870 2024-04-24T23:30:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:30:31.634 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:30:31.635 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.635+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f50900a2c10 msgr2=0x7f50900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.635+0000 7f50ac67d640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f50900a2c10 0x7f50900a50d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f50a002aa00 tx=0x7f50a00315b0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 msgr2=0x7f50a4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 0x7f50a4197ff0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f509400e9f0 tx=0x7f509400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 shutdown_connections 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f50900a2c10 0x7f50900a50d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a4102230 0x7f50a419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a4101870 0x7f50a4197ff0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 --2- 172.21.15.33:0/2077716845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50a4100680 0x7f50a4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 >> 172.21.15.33:0/2077716845 conn(0x7f50a40fbbf0 msgr2=0x7f50a40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 shutdown_connections 2024-04-24T23:30:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.636+0000 7f50ac67d640 1 -- 172.21.15.33:0/2077716845 wait complete. 2024-04-24T23:30:31.650 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 -- 172.21.15.33:0/3193800931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a8108740 msgr2=0x7fe5a810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 --2- 172.21.15.33:0/3193800931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a8108740 0x7fe5a810ab70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a410 comp rx=0 tx=0).stop 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 -- 172.21.15.33:0/3193800931 shutdown_connections 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 --2- 172.21.15.33:0/3193800931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a810b530 0x7fe5a81143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 --2- 172.21.15.33:0/3193800931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a810b0b0 0x7fe5a8111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 --2- 172.21.15.33:0/3193800931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a8108740 0x7fe5a810ab70 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 -- 172.21.15.33:0/3193800931 >> 172.21.15.33:0/3193800931 conn(0x7fe5a806e5d0 msgr2=0x7fe5a8070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 -- 172.21.15.33:0/3193800931 shutdown_connections 2024-04-24T23:30:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.794+0000 7fe5ae970640 1 -- 172.21.15.33:0/3193800931 wait complete. 2024-04-24T23:30:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 Processor -- start 2024-04-24T23:30:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 -- start start 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 0x7fe5a819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a810b0b0 0x7fe5a819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a810b530 0x7fe5a81a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8117e80 con 0x7fe5a810b530 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8117d00 con 0x7fe5a810b0b0 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5ae970640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8118000 con 0x7fe5a8108740 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 0x7fe5a819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 0x7fe5a819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51136/0 (socket says 172.21.15.33:51136) 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5a7fff640 1 -- 172.21.15.33:0/918668213 learned_addr learned my addr 172.21.15.33:0/918668213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a810b0b0 0x7fe5a819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.795+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a810b530 0x7fe5a81a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.796+0000 7fe5a7fff640 1 -- 172.21.15.33:0/918668213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a810b0b0 msgr2=0x7fe5a819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.796+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a810b0b0 0x7fe5a819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.796+0000 7fe5a7fff640 1 -- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a810b530 msgr2=0x7fe5a81a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.796+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a810b530 0x7fe5a81a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.796+0000 7fe5a7fff640 1 -- 172.21.15.33:0/918668213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a5540 con 0x7fe5a8108740 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.797+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 0x7fe5a819d920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe598009850 tx=0x7fe598032bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.797+0000 7fe5a5ffb640 1 -- 172.21.15.33:0/918668213 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59803f030 con 0x7fe5a8108740 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.797+0000 7fe5a5ffb640 1 -- 172.21.15.33:0/918668213 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598003ad0 con 0x7fe5a8108740 2024-04-24T23:30:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.797+0000 7fe5a5ffb640 1 -- 172.21.15.33:0/918668213 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598035070 con 0x7fe5a8108740 2024-04-24T23:30:31.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.798+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a57d0 con 0x7fe5a8108740 2024-04-24T23:30:31.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.798+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81b7520 con 0x7fe5a8108740 2024-04-24T23:30:31.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.802+0000 7fe5a5ffb640 1 -- 172.21.15.33:0/918668213 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe598042030 con 0x7fe5a8108740 2024-04-24T23:30:31.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.803+0000 7fe5a5ffb640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5940a2d90 0x7fe5940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.803+0000 7fe5a5ffb640 1 -- 172.21.15.33:0/918668213 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5980ef960 con 0x7fe5a8108740 2024-04-24T23:30:31.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.804+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5940a2d90 0x7fe5940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.806+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5940a2d90 0x7fe5940a5250 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5a819ee30 tx=0x7fe590005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.807+0000 7fe5b1684740 1 Processor -- start 2024-04-24T23:30:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.807+0000 7fe5b1684740 1 -- start start 2024-04-24T23:30:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.807+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.807+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3ebd90 0x563c0f3ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe59f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe59f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3eb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54846/0 (socket says 172.21.15.33:54846) 2024-04-24T23:30:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe59f7fe640 1 -- 172.21.15.33:0/3920009597 learned_addr learned my addr 172.21.15.33:0/3920009597 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c0f34a670 con 0x563c0f3e6230 2024-04-24T23:30:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563c0f2f2970 con 0x563c0f3e6c20 2024-04-24T23:30:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c0f346630 con 0x563c0f3ebd90 2024-04-24T23:30:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.808+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3ebd90 0x563c0f3ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.809+0000 7fe59f7fe640 1 -- 172.21.15.33:0/3920009597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3ebd90 msgr2=0x563c0f3ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.809+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3ebd90 0x563c0f3ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.809+0000 7fe59f7fe640 1 -- 172.21.15.33:0/3920009597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 msgr2=0x563c0f3e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.809+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.809+0000 7fe59f7fe640 1 -- 172.21.15.33:0/3920009597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0f316400 con 0x563c0f3e6c20 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.809+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3eb720 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5900058c0 tx=0x7fe590014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb875b2df8ccb970 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.810+0000 7fe59e7fc640 1 -- 172.21.15.33:0/3920009597 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590026430 con 0x563c0f3e6c20 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.810+0000 7fe59e7fc640 1 -- 172.21.15.33:0/3920009597 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59001c8f0 con 0x563c0f3e6c20 2024-04-24T23:30:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.810+0000 7fe59e7fc640 1 -- 172.21.15.33:0/3920009597 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59001da60 con 0x563c0f3e6c20 2024-04-24T23:30:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.811+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 msgr2=0x563c0f3eb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.811+0000 7fe5b1684740 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3eb720 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5900058c0 tx=0x7fe590014000 comp rx=0 tx=0).stop 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 shutdown_connections 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3ebd90 0x563c0f3ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3eb720 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 --2- 172.21.15.33:0/3920009597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 >> 172.21.15.33:0/3920009597 conn(0x563c0f3e18a0 msgr2=0x563c0f3e1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 shutdown_connections 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 -- 172.21.15.33:0/3920009597 wait complete. 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.812+0000 7fe5b1684740 1 -- start start 2024-04-24T23:30:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.813+0000 7fe5b1684740 1 Processor -- start 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.813+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6230 0x563c0f3f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.813+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.813+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 0x563c0f3f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5b1684740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c0f3eed40 con 0x563c0f3ebd90 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5b1684740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563c0f306050 con 0x563c0f3e6c20 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5b1684740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c0f3eeec0 con 0x563c0f3e6230 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5b1684740 10 client.0 _ll_register_callbacks cb 0x563c0f05dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 0x563c0f3f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 0x563c0f3f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36762/0 (socket says 172.21.15.33:36762) 2024-04-24T23:30:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2200266820 learned_addr learned my addr 172.21.15.33:0/2200266820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6230 0x563c0f3f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2200266820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6230 msgr2=0x563c0f3f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6230 0x563c0f3f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 msgr2=0x563c0f3f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.814+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2200266820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0f2d7dc0 con 0x563c0f3ebd90 2024-04-24T23:30:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.815+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 0x563c0f3f7900 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00129e0 tx=0x7fe5a0012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a7c65a1103030 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.815+0000 7fe5877fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.816+0000 7fe5877fe640 1 -- 172.21.15.33:0/2200266820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003970 con 0x563c0f3ebd90 2024-04-24T23:30:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.816+0000 7fe5877fe640 1 -- 172.21.15.33:0/2200266820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5a0003b10 con 0x563c0f3ebd90 2024-04-24T23:30:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.816+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59001c8f0 con 0x563c0f3ebd90 2024-04-24T23:30:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.816+0000 7fe5877fe640 1 -- 172.21.15.33:0/2200266820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe5a0003d50 con 0x563c0f3ebd90 2024-04-24T23:30:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.816+0000 7fe5b1684740 10 client.19284 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.816+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563c0f05e3f0 con 0x563c0f3ebd90 2024-04-24T23:30:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5877fe640 1 -- 172.21.15.33:0/2200266820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a000fe10 con 0x563c0f3ebd90 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5877fe640 1 -- 172.21.15.33:0/2200266820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe5a0010430 con 0x563c0f3ebd90 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5b1684740 10 client.19284 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5b1684740 10 client.19284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5b1684740 20 client.19284 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3fc330 0x563c0f3fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5b1684740 4 client.19284 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.817+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe59001c8f0 con 0x563c0f3fc330 2024-04-24T23:30:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.819+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3fc330 0x563c0f3fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.820+0000 7fe5877fe640 10 client.19284 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:30:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.820+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3fc330 0x563c0f3fe790 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4676ce3f29e32a16 server_cookie=956c73823077900a in_seq=0 out_seq=0 2024-04-24T23:30:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.828+0000 7fe5877fe640 1 -- 172.21.15.33:0/2200266820 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fe59001c8f0 con 0x563c0f3fc330 2024-04-24T23:30:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.828+0000 7fe5877fe640 10 client.19284 handle_command_reply: tid=0 2024-04-24T23:30:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.828+0000 7fe5b1684740 1 client.19284 shutdown 2024-04-24T23:30:31.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.831+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 msgr2=0x563c0f3f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.831+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 0x563c0f3f7900 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00129e0 tx=0x7fe5a0012ea0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.832+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 shutdown_connections 2024-04-24T23:30:31.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.832+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3fc330 0x563c0f3fe790 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.832+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3ebd90 0x563c0f3f7900 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.832+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.832+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2200266820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6230 0x563c0f3f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.832+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 >> 172.21.15.33:0/2200266820 conn(0x563c0f3e18a0 msgr2=0x563c0f3e0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.833+0000 7fe5877fe640 0 client.19284 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:30:31.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.833+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 shutdown_connections 2024-04-24T23:30:31.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.833+0000 7fe5b1684740 1 -- 172.21.15.33:0/2200266820 wait complete. 2024-04-24T23:30:31.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.834+0000 7fe5b1684740 20 client.19284 trim_cache size 0 max 16384 2024-04-24T23:30:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.861+0000 7fe5b1684740 1 Processor -- start 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.865+0000 7fe5b1684740 1 -- start start 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.865+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e18a0 0x563c0f3e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.865+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.865+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6c20 0x563c0f3f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.865+0000 7fe5b1684740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81c4330 con 0x563c0f3e6230 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.866+0000 7fe5b1684740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81c1c00 con 0x563c0f3e18a0 2024-04-24T23:30:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.866+0000 7fe5b1684740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81bfa50 con 0x563c0f3e6c20 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.866+0000 7fe59f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.866+0000 7fe59f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36778/0 (socket says 172.21.15.33:36778) 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.866+0000 7fe59f7fe640 1 -- 172.21.15.33:0/2473694852 learned_addr learned my addr 172.21.15.33:0/2473694852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.866+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6c20 0x563c0f3f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.867+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2473694852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e18a0 msgr2=0x563c0f3e7dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.867+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e18a0 0x563c0f3e7dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.867+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2473694852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 msgr2=0x563c0f3e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.867+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.867+0000 7fe5acf0d640 1 -- 172.21.15.33:0/2473694852 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0f1287d0 con 0x563c0f3e6c20 2024-04-24T23:30:31.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.868+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6c20 0x563c0f3f78c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000a010 tx=0x7fe5a0010d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df4b21cdfe1fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.868+0000 7fe59dffb640 1 -- 172.21.15.33:0/2473694852 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003970 con 0x563c0f3e6c20 2024-04-24T23:30:31.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.868+0000 7fe59dffb640 1 -- 172.21.15.33:0/2473694852 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a000fe10 con 0x563c0f3e6c20 2024-04-24T23:30:31.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.868+0000 7fe59dffb640 1 -- 172.21.15.33:0/2473694852 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0005340 con 0x563c0f3e6c20 2024-04-24T23:30:31.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 -- 172.21.15.33:0/2473694852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6c20 msgr2=0x563c0f3f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6c20 0x563c0f3f78c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000a010 tx=0x7fe5a0010d20 comp rx=0 tx=0).stop 2024-04-24T23:30:31.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 -- 172.21.15.33:0/2473694852 shutdown_connections 2024-04-24T23:30:31.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e6c20 0x563c0f3f78c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2473694852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e18a0 0x563c0f3e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.869+0000 7fe5b1684740 1 -- 172.21.15.33:0/2473694852 >> 172.21.15.33:0/2473694852 conn(0x563c0f3eff40 msgr2=0x563c0f3e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.870+0000 7fe5b1684740 1 -- 172.21.15.33:0/2473694852 shutdown_connections 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.870+0000 7fe5b1684740 1 -- 172.21.15.33:0/2473694852 wait complete. 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.871+0000 7fe5b1684740 1 -- start start 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.871+0000 7fe5b1684740 1 Processor -- start 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.872+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 0x563c0f3f7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.872+0000 7fe5b1684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.872+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 0x563c0f3f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.872+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 0x563c0f3f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51194/0 (socket says 172.21.15.33:51194) 2024-04-24T23:30:31.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.872+0000 7fe5a7fff640 1 -- 172.21.15.33:0/2237685434 learned_addr learned my addr 172.21.15.33:0/2237685434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:31.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.872+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.873+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81c00c0 con 0x563c0f3e6230 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.873+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81c1530 con 0x563c0f3e6c20 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.873+0000 7fe5acf0d640 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.873+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81c0790 con 0x563c0f3e18a0 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.873+0000 7fe5b1684740 10 client.0 _ll_register_callbacks cb 0x7fe5a0009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5a7fff640 1 -- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 msgr2=0x563c0f3f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5a7fff640 1 -- 172.21.15.33:0/2237685434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 msgr2=0x563c0f3f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5a7fff640 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0f3f5dc0 con 0x563c0f3e18a0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5a7fff640 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 0x563c0f3f7170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe5980b81a0 tx=0x7fe5980bba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9683fe2f0d66bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.874+0000 7fe5877fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.875+0000 7fe5877fe640 1 -- 172.21.15.33:0/2237685434 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980bf040 con 0x563c0f3e18a0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.875+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a000fe10 con 0x563c0f3e18a0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.875+0000 7fe5877fe640 1 -- 172.21.15.33:0/2237685434 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe598003c80 con 0x563c0f3e18a0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.875+0000 7fe5877fe640 1 -- 172.21.15.33:0/2237685434 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe5980c0180 con 0x563c0f3e18a0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.876+0000 7fe5b1684740 10 client.28705 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.876+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563c0f463240 con 0x563c0f3e18a0 2024-04-24T23:30:31.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.876+0000 7fe5877fe640 1 -- 172.21.15.33:0/2237685434 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5980c4a40 con 0x563c0f3e18a0 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.876+0000 7fe5877fe640 1 -- 172.21.15.33:0/2237685434 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe5980d1040 con 0x563c0f3e18a0 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.877+0000 7fe5b1684740 10 client.28705 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.877+0000 7fe5b1684740 10 client.28705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.877+0000 7fe5b1684740 20 client.28705 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.878+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3ebd90 0x563c0f465b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.878+0000 7fe5b1684740 4 client.28705 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.878+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe5a000fe10 con 0x563c0f3ebd90 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.878+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3ebd90 0x563c0f465b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.884+0000 7fe59f7fe640 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3ebd90 0x563c0f465b90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26e1dc3b50fc853c server_cookie=c3db60f1e009b8b in_seq=0 out_seq=0 2024-04-24T23:30:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.885+0000 7fe5877fe640 10 client.28705 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:30:31.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5877fe640 1 -- 172.21.15.33:0/2237685434 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe5a000fe10 con 0x563c0f3ebd90 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5877fe640 10 client.28705 handle_command_reply: tid=0 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 client.28705 shutdown 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 msgr2=0x563c0f3f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 0x563c0f3f7170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe5980b81a0 tx=0x7fe5980bba40 comp rx=0 tx=0).stop 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 shutdown_connections 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x563c0f3ebd90 0x563c0f465b90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c0f3e6c20 0x563c0f3f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c0f3e6230 0x563c0f3f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 --2- 172.21.15.33:0/2237685434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c0f3e18a0 0x563c0f3f7170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 >> 172.21.15.33:0/2237685434 conn(0x563c0f3eff40 msgr2=0x563c0f3e7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5877fe640 0 client.28705 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:30:31.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 shutdown_connections 2024-04-24T23:30:31.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.888+0000 7fe5b1684740 1 -- 172.21.15.33:0/2237685434 wait complete. 2024-04-24T23:30:31.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.889+0000 7fe5b1684740 20 client.28705 trim_cache size 0 max 16384 2024-04-24T23:30:31.894 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:30:31.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 120026 22.53 MB/sec execute 156 sec latency 62.894 ms 2024-04-24T23:30:31.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5940a2d90 msgr2=0x7fe5940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5940a2d90 0x7fe5940a5250 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5a819ee30 tx=0x7fe590005f90 comp rx=0 tx=0).stop 2024-04-24T23:30:31.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 msgr2=0x7fe5a819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:31.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 0x7fe5a819d920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe598009850 tx=0x7fe598032bf0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 shutdown_connections 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe5940a2d90 0x7fe5940a5250 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a810b530 0x7fe5a81a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a810b0b0 0x7fe5a819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 --2- 172.21.15.33:0/918668213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a8108740 0x7fe5a819d920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 >> 172.21.15.33:0/918668213 conn(0x7fe5a806e5d0 msgr2=0x7fe5a8070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 shutdown_connections 2024-04-24T23:30:31.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:31.890+0000 7fe5ae970640 1 -- 172.21.15.33:0/918668213 wait complete. 2024-04-24T23:30:31.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:31 smithi033 ceph-mon[29201]: pgmap v906: 129 pgs: 129 active+clean; 543 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 342 op/s 2024-04-24T23:30:31.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:30:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:31 smithi038 ceph-mon[32450]: pgmap v906: 129 pgs: 129 active+clean; 543 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 342 op/s 2024-04-24T23:30:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:31 smithi117 ceph-mon[32529]: pgmap v906: 129 pgs: 129 active+clean; 543 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 342 op/s 2024-04-24T23:30:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 -- 172.21.15.33:0/3364397669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 msgr2=0x7fdb24101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/3364397669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24101cf0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009900 tx=0x7fdb1802a410 comp rx=0 tx=0).stop 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 -- 172.21.15.33:0/3364397669 shutdown_connections 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/3364397669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb24102230 0x7fdb24108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/3364397669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24101cf0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/3364397669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 0x7fdb24100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 -- 172.21.15.33:0/3364397669 >> 172.21.15.33:0/3364397669 conn(0x7fdb240fbbf0 msgr2=0x7fdb240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 -- 172.21.15.33:0/3364397669 shutdown_connections 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.050+0000 7fdb2ad42640 1 -- 172.21.15.33:0/3364397669 wait complete. 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 Processor -- start 2024-04-24T23:30:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 -- start start 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 0x7fdb2410ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24109340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb24102230 0x7fdb24109880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2410d460 con 0x7fdb24101870 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2410d2e0 con 0x7fdb24100680 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb2ad42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2410d5e0 con 0x7fdb24102230 2024-04-24T23:30:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb292df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb24102230 0x7fdb24109880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36804/0 (socket says 172.21.15.33:36804) 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 0x7fdb2410ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.051+0000 7fdb23fff640 1 -- 172.21.15.33:0/2358987698 learned_addr learned my addr 172.21.15.33:0/2358987698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb28ade640 1 -- 172.21.15.33:0/2358987698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb24102230 msgr2=0x7fdb24109880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb28ade640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb24102230 0x7fdb24109880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb28ade640 1 -- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 msgr2=0x7fdb24109340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb28ade640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24109340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb28ade640 1 -- 172.21.15.33:0/2358987698 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2410a110 con 0x7fdb24100680 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb23fff640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb28ade640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 0x7fdb2410ad00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e9c0 tx=0x7fdb1400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb21ffb640 1 -- 172.21.15.33:0/2358987698 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400ccc0 con 0x7fdb24100680 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241a1ba0 con 0x7fdb24100680 2024-04-24T23:30:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.052+0000 7fdb21ffb640 1 -- 172.21.15.33:0/2358987698 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb140035f0 con 0x7fdb24100680 2024-04-24T23:30:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.053+0000 7fdb21ffb640 1 -- 172.21.15.33:0/2358987698 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400ce60 con 0x7fdb24100680 2024-04-24T23:30:32.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.053+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241a2140 con 0x7fdb24100680 2024-04-24T23:30:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.055+0000 7fdb21ffb640 1 -- 172.21.15.33:0/2358987698 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdb140037a0 con 0x7fdb24100680 2024-04-24T23:30:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb21ffb640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdb0c0a2b90 0x7fdb0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 Processor -- start 2024-04-24T23:30:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 -- start start 2024-04-24T23:30:32.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb23fff640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdb0c0a2b90 0x7fdb0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb21ffb640 1 -- 172.21.15.33:0/2358987698 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb140cf890 con 0x7fdb24100680 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4230 0x55d4445d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4c20 0x55d4445d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d9d90 0x55d4445dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d444538670 con 0x55d4445d4c20 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d4444e0970 con 0x55d4445d4230 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb2da51740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d444534630 con 0x55d4445d9d90 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4c20 0x55d4445d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb292df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d9d90 0x55d4445dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4c20 0x55d4445d9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36808/0 (socket says 172.21.15.33:36808) 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb23fff640 1 -- 172.21.15.33:0/3836115986 learned_addr learned my addr 172.21.15.33:0/3836115986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb28ade640 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4230 0x55d4445d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb292df640 1 -- 172.21.15.33:0/3836115986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4230 msgr2=0x55d4445d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb292df640 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4230 0x55d4445d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb292df640 1 -- 172.21.15.33:0/3836115986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4c20 msgr2=0x55d4445d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb23fff640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdb0c0a2b90 0x7fdb0c0a5050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb180096c0 tx=0x7fdb18031890 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb292df640 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4c20 0x55d4445d9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb292df640 1 -- 172.21.15.33:0/3836115986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d444504400 con 0x55d4445d9d90 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.056+0000 7fdb28ade640 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4230 0x55d4445d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.057+0000 7fdb292df640 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d9d90 0x55d4445dc220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009f00 tx=0x7fdb1c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce0ec4cd9efd6fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.057+0000 7fdb0affd640 1 -- 172.21.15.33:0/3836115986 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c004d20 con 0x55d4445d9d90 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.057+0000 7fdb0affd640 1 -- 172.21.15.33:0/3836115986 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c004050 con 0x55d4445d9d90 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.057+0000 7fdb0affd640 1 -- 172.21.15.33:0/3836115986 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c005280 con 0x55d4445d9d90 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.057+0000 7fdb2da51740 1 -- 172.21.15.33:0/3836115986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d9d90 msgr2=0x55d4445dc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.057+0000 7fdb2da51740 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d9d90 0x55d4445dc220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009f00 tx=0x7fdb1c011810 comp rx=0 tx=0).stop 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- 172.21.15.33:0/3836115986 shutdown_connections 2024-04-24T23:30:32.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d9d90 0x55d4445dc220 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4c20 0x55d4445d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 --2- 172.21.15.33:0/3836115986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4230 0x55d4445d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- 172.21.15.33:0/3836115986 >> 172.21.15.33:0/3836115986 conn(0x55d4445cf8a0 msgr2=0x55d4445cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- 172.21.15.33:0/3836115986 shutdown_connections 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- 172.21.15.33:0/3836115986 wait complete. 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- start start 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 Processor -- start 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4445dcd40 con 0x55d4445d4230 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d4444f4050 con 0x55d4445d9d90 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d4445dcec0 con 0x55d4445d4c20 2024-04-24T23:30:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb2da51740 10 client.0 _ll_register_callbacks cb 0x55d44424bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.058+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445de300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36814/0 (socket says 172.21.15.33:36814) 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 -- 172.21.15.33:0/2721697219 learned_addr learned my addr 172.21.15.33:0/2721697219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb23fff640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb292df640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 -- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 msgr2=0x55d4445de840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445de840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 -- 172.21.15.33:0/2721697219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 msgr2=0x55d4445e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 -- 172.21.15.33:0/2721697219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4444c5dc0 con 0x55d4445d4230 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb28ade640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445de300 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009230 tx=0x7fdb1409be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb26516ab4d103f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb03fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb03fff640 1 -- 172.21.15.33:0/2721697219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140b9060 con 0x55d4445d4230 2024-04-24T23:30:32.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb03fff640 1 -- 172.21.15.33:0/2721697219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb1409b350 con 0x55d4445d4230 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.059+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1c004050 con 0x55d4445d4230 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb03fff640 1 -- 172.21.15.33:0/2721697219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb1409a0c0 con 0x55d4445d4230 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 10 client.19299 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d44424c3f0 con 0x55d4445d4230 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb03fff640 1 -- 172.21.15.33:0/2721697219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c004050 con 0x55d4445d4230 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb03fff640 1 -- 172.21.15.33:0/2721697219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdb1409ad50 con 0x55d4445d4230 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 10 client.19299 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 10 client.19299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 20 client.19299 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4445ea040 0x55d4445ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 4 client.19299 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d444545bb0 con 0x55d4445ea040 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.060+0000 7fdb23fff640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4445ea040 0x55d4445ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.061+0000 7fdb03fff640 10 client.19299 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:30:32.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.061+0000 7fdb23fff640 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4445ea040 0x55d4445ec4d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c63c31201a8cef14 server_cookie=ce360add43319ee4 in_seq=0 out_seq=0 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb03fff640 1 -- 172.21.15.33:0/2721697219 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d444545bb0 con 0x55d4445ea040 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb03fff640 10 client.19299 handle_command_reply: tid=0 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 client.19299 shutdown 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 msgr2=0x55d4445de300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445de300 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009230 tx=0x7fdb1409be40 comp rx=0 tx=0).stop 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 shutdown_connections 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4445ea040 0x55d4445ec4d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445de840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb03fff640 0 client.19299 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:30:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 --2- 172.21.15.33:0/2721697219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445de300 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 >> 172.21.15.33:0/2721697219 conn(0x55d4445cf8a0 msgr2=0x55d4445ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 shutdown_connections 2024-04-24T23:30:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 1 -- 172.21.15.33:0/2721697219 wait complete. 2024-04-24T23:30:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.062+0000 7fdb2da51740 20 client.19299 trim_cache size 0 max 16384 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.078+0000 7fdb2da51740 1 Processor -- start 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 -- start start 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 0x55d4445d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4230 0x55d4445e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4c20 0x55d4445d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241aeee0 con 0x55d4445cf8a0 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241ac7b0 con 0x55d4445d4c20 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb2da51740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241aa600 con 0x55d4445d4230 2024-04-24T23:30:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 0x55d4445d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 0x55d4445d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36818/0 (socket says 172.21.15.33:36818) 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 -- 172.21.15.33:0/1172931898 learned_addr learned my addr 172.21.15.33:0/1172931898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb23fff640 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4230 0x55d4445e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb292df640 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4c20 0x55d4445d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 -- 172.21.15.33:0/1172931898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4230 msgr2=0x55d4445e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4230 0x55d4445e5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 -- 172.21.15.33:0/1172931898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4c20 msgr2=0x55d4445d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4c20 0x55d4445d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 -- 172.21.15.33:0/1172931898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4443167d0 con 0x55d4445cf8a0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.079+0000 7fdb28ade640 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 0x55d4445d2830 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fdb1409bc90 tx=0x7fdb1409bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddbf9c7f26a5bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb0a7fc640 1 -- 172.21.15.33:0/1172931898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140b9070 con 0x55d4445cf8a0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb0a7fc640 1 -- 172.21.15.33:0/1172931898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1407ab70 con 0x55d4445cf8a0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb0a7fc640 1 -- 172.21.15.33:0/1172931898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1409b150 con 0x55d4445cf8a0 2024-04-24T23:30:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 -- 172.21.15.33:0/1172931898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 msgr2=0x55d4445d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 0x55d4445d2830 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fdb1409bc90 tx=0x7fdb1409bc00 comp rx=0 tx=0).stop 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 -- 172.21.15.33:0/1172931898 shutdown_connections 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d4c20 0x55d4445d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4230 0x55d4445e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 --2- 172.21.15.33:0/1172931898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445cf8a0 0x55d4445d2830 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.080+0000 7fdb2da51740 1 -- 172.21.15.33:0/1172931898 >> 172.21.15.33:0/1172931898 conn(0x55d4445d98d0 msgr2=0x55d4445ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 -- 172.21.15.33:0/1172931898 shutdown_connections 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 -- 172.21.15.33:0/1172931898 wait complete. 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 -- start start 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 Processor -- start 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445e3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241aac70 con 0x55d4445d4230 2024-04-24T23:30:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241ac0e0 con 0x55d4445d9d90 2024-04-24T23:30:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241ab340 con 0x55d4445d4c20 2024-04-24T23:30:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb2da51740 10 client.0 _ll_register_callbacks cb 0x7fdb1409b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb28ade640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445e54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36834/0 (socket says 172.21.15.33:36834) 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb28ade640 1 -- 172.21.15.33:0/407831221 learned_addr learned my addr 172.21.15.33:0/407831221 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb23fff640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445e3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.081+0000 7fdb292df640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb28ade640 1 -- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 msgr2=0x55d4445e3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb28ade640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445e3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb28ade640 1 -- 172.21.15.33:0/407831221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 msgr2=0x55d4445e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb28ade640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb28ade640 1 -- 172.21.15.33:0/407831221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4445e46a0 con 0x55d4445d4230 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb28ade640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445e54d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fdb14098160 tx=0x7fdb1409be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8698a969070817a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb03fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb03fff640 1 -- 172.21.15.33:0/407831221 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140a8020 con 0x55d4445d4230 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb03fff640 1 -- 172.21.15.33:0/407831221 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb140a4dc0 con 0x55d4445d4230 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1407ab70 con 0x55d4445d4230 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb03fff640 1 -- 172.21.15.33:0/407831221 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdb140b9070 con 0x55d4445d4230 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb2da51740 10 client.19305 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.082+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d4445e46a0 con 0x55d4445d4230 2024-04-24T23:30:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb03fff640 1 -- 172.21.15.33:0/407831221 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1407ab70 con 0x55d4445d4230 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb03fff640 1 -- 172.21.15.33:0/407831221 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdb14004000 con 0x55d4445d4230 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb2da51740 10 client.19305 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb2da51740 10 client.19305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb2da51740 20 client.19305 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb2da51740 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4446518e0 0x55d444653cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb2da51740 4 client.19305 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d44453ff00 con 0x55d4446518e0 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.083+0000 7fdb23fff640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4446518e0 0x55d444653cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb03fff640 10 client.19305 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb23fff640 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4446518e0 0x55d444653cd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f58a775d551f2787 server_cookie=a457e2da7a80186d in_seq=0 out_seq=0 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb03fff640 1 -- 172.21.15.33:0/407831221 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d44453ff00 con 0x55d4446518e0 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb03fff640 10 client.19305 handle_command_reply: tid=0 2024-04-24T23:30:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 client.19305 shutdown 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 msgr2=0x55d4445e54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445e54d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fdb14098160 tx=0x7fdb1409be40 comp rx=0 tx=0).stop 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 shutdown_connections 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d4446518e0 0x55d444653cd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d4445d9d90 0x55d4445e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4445d4c20 0x55d4445e3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 --2- 172.21.15.33:0/407831221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4445d4230 0x55d4445e54d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 >> 172.21.15.33:0/407831221 conn(0x55d4445d98d0 msgr2=0x55d4445d56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb03fff640 0 client.19305 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 shutdown_connections 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.084+0000 7fdb2da51740 1 -- 172.21.15.33:0/407831221 wait complete. 2024-04-24T23:30:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.085+0000 7fdb2da51740 20 client.19305 trim_cache size 0 max 16384 2024-04-24T23:30:32.092 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdb0c0a2b90 msgr2=0x7fdb0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdb0c0a2b90 0x7fdb0c0a5050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb180096c0 tx=0x7fdb18031890 comp rx=0 tx=0).stop 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 msgr2=0x7fdb2410ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 0x7fdb2410ad00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e9c0 tx=0x7fdb1400ee80 comp rx=0 tx=0).stop 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 shutdown_connections 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdb0c0a2b90 0x7fdb0c0a5050 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb24102230 0x7fdb24109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.086+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb24101870 0x7fdb24109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.087+0000 7fdb2ad42640 1 --2- 172.21.15.33:0/2358987698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb24100680 0x7fdb2410ad00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.087+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 >> 172.21.15.33:0/2358987698 conn(0x7fdb240fbbf0 msgr2=0x7fdb24104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.087+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 shutdown_connections 2024-04-24T23:30:32.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.087+0000 7fdb2ad42640 1 -- 172.21.15.33:0/2358987698 wait complete. 2024-04-24T23:30:32.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 -- 172.21.15.33:0/3468764943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac104b30 msgr2=0x7fb8ac106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/3468764943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac104b30 0x7fb8ac106f20 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009900 tx=0x7fb8a002a410 comp rx=0 tx=0).stop 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 -- 172.21.15.33:0/3468764943 shutdown_connections 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/3468764943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8ac107460 0x7fb8ac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/3468764943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac104b30 0x7fb8ac106f20 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/3468764943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac102200 0x7fb8ac1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 -- 172.21.15.33:0/3468764943 >> 172.21.15.33:0/3468764943 conn(0x7fb8ac0fbbf0 msgr2=0x7fb8ac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 -- 172.21.15.33:0/3468764943 shutdown_connections 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.218+0000 7fb8b1f22640 1 -- 172.21.15.33:0/3468764943 wait complete. 2024-04-24T23:30:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.219+0000 7fb8b1f22640 1 Processor -- start 2024-04-24T23:30:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.219+0000 7fb8b1f22640 1 -- start start 2024-04-24T23:30:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8b1f22640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 0x7fb8ac197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8b1f22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8ac104b30 0x7fb8ac198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8b1f22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac107460 0x7fb8ac19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8b1f22640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d440 con 0x7fb8ac102200 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8b1f22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d2c0 con 0x7fb8ac107460 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8b1f22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d5c0 con 0x7fb8ac104b30 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 0x7fb8ac197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 0x7fb8ac197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36850/0 (socket says 172.21.15.33:36850) 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 -- 172.21.15.33:0/230929883 learned_addr learned my addr 172.21.15.33:0/230929883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8ac104b30 0x7fb8ac198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8abfff640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac107460 0x7fb8ac19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 -- 172.21.15.33:0/230929883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8ac104b30 msgr2=0x7fb8ac198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8ac104b30 0x7fb8ac198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 -- 172.21.15.33:0/230929883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac107460 msgr2=0x7fb8ac19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac107460 0x7fb8ac19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 -- 172.21.15.33:0/230929883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac19f730 con 0x7fb8ac102200 2024-04-24T23:30:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.220+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 0x7fb8ac197b10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00e9f0 tx=0x7fb89c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.221+0000 7fb8a8ff9640 1 -- 172.21.15.33:0/230929883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00cd80 con 0x7fb8ac102200 2024-04-24T23:30:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.221+0000 7fb8a8ff9640 1 -- 172.21.15.33:0/230929883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89c010910 con 0x7fb8ac102200 2024-04-24T23:30:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.221+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac19fa20 con 0x7fb8ac102200 2024-04-24T23:30:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.221+0000 7fb8a8ff9640 1 -- 172.21.15.33:0/230929883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c018990 con 0x7fb8ac102200 2024-04-24T23:30:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.221+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac19ff40 con 0x7fb8ac102200 2024-04-24T23:30:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.223+0000 7fb8a8ff9640 1 -- 172.21.15.33:0/230929883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb89c010ac0 con 0x7fb8ac102200 2024-04-24T23:30:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.223+0000 7fb8b4c32740 1 Processor -- start 2024-04-24T23:30:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 -- start start 2024-04-24T23:30:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x563863513650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x563863518720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x56386351b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563863477670 con 0x563863513230 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56386341f970 con 0x563863513c20 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563863473630 con 0x563863518d90 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x563863513650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x563863513650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36854/0 (socket says 172.21.15.33:36854) 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 -- 172.21.15.33:0/3847224307 learned_addr learned my addr 172.21.15.33:0/3847224307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8a8ff9640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8940a2be0 0x7fb8940a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x563863518720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x56386351b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8940a2be0 0x7fb8940a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 -- 172.21.15.33:0/3847224307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 msgr2=0x56386351b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x56386351b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8a8ff9640 1 -- 172.21.15.33:0/230929883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb89c0cf730 con 0x7fb8ac102200 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 -- 172.21.15.33:0/3847224307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 msgr2=0x563863518720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x563863518720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8abfff640 1 -- 172.21.15.33:0/3847224307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563863443400 con 0x563863513230 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.224+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x563863518720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb8abfff640 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x563863513650 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4005c40 tx=0x7fb8a400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd6d85424cfb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb891ffb640 1 -- 172.21.15.33:0/3847224307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a401c050 con 0x563863513230 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb8abfff640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8940a2be0 0x7fb8940a50a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4026000 tx=0x7fb8a4023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb891ffb640 1 -- 172.21.15.33:0/3847224307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a401d040 con 0x563863513230 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb891ffb640 1 -- 172.21.15.33:0/3847224307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4026030 con 0x563863513230 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb8b4c32740 1 -- 172.21.15.33:0/3847224307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 msgr2=0x563863513650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.225+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x563863513650 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4005c40 tx=0x7fb8a400e780 comp rx=0 tx=0).stop 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 -- 172.21.15.33:0/3847224307 shutdown_connections 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x56386351b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x563863518720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/3847224307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x563863513650 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 -- 172.21.15.33:0/3847224307 >> 172.21.15.33:0/3847224307 conn(0x56386350e8a0 msgr2=0x56386350eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 -- 172.21.15.33:0/3847224307 shutdown_connections 2024-04-24T23:30:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 -- 172.21.15.33:0/3847224307 wait complete. 2024-04-24T23:30:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 -- start start 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 Processor -- start 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x56386351d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.226+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x56386351d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x56386351d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x56386351d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56386351bd40 con 0x563863513c20 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563863433050 con 0x563863513230 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56386351bec0 con 0x563863518d90 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x56386351d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36864/0 (socket says 172.21.15.33:36864) 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x56386351d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54956/0 (socket says 172.21.15.33:54956) 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1531625269 learned_addr learned my addr 172.21.15.33:0/1531625269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 10 client.0 _ll_register_callbacks cb 0x56386318abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8abfff640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 msgr2=0x5638635248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1531625269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 msgr2=0x56386351d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x56386351d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1531625269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563863404dc0 con 0x563863513c20 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x56386351d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8abfff640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x56386351d8a0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0005c40 tx=0x7fb8a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48024383f96282c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb88affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb88affd640 1 -- 172.21.15.33:0/1531625269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0035040 con 0x563863513c20 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a401d040 con 0x563863513c20 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb88affd640 1 -- 172.21.15.33:0/1531625269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8a00316b0 con 0x563863513c20 2024-04-24T23:30:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb88affd640 1 -- 172.21.15.33:0/1531625269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb8a004f050 con 0x563863513c20 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 10 client.19317 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.227+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56386318b3f0 con 0x563863513c20 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb88affd640 1 -- 172.21.15.33:0/1531625269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a401d040 con 0x563863513c20 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb88affd640 1 -- 172.21.15.33:0/1531625269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb8a0004c50 con 0x563863513c20 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8b4c32740 10 client.19317 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8b4c32740 10 client.19317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8b4c32740 20 client.19317 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863529140 0x56386352b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8b4c32740 4 client.19317 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563863484bb0 con 0x563863529140 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863529140 0x56386352b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb88affd640 10 client.19317 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.228+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863529140 0x56386352b5d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a272f487c87171e server_cookie=ef366480fb2ad7cc in_seq=0 out_seq=0 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.229+0000 7fb88affd640 1 -- 172.21.15.33:0/1531625269 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563863484bb0 con 0x563863529140 2024-04-24T23:30:32.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.229+0000 7fb88affd640 10 client.19317 handle_command_reply: tid=0 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.229+0000 7fb8b4c32740 1 client.19317 shutdown 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 msgr2=0x56386351d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x56386351d8a0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0005c40 tx=0x7fb8a0004600 comp rx=0 tx=0).stop 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 shutdown_connections 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863529140 0x56386352b5d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x56386351d8a0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1531625269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x56386351d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 >> 172.21.15.33:0/1531625269 conn(0x56386350e8a0 msgr2=0x56386350db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb88affd640 0 client.19317 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 shutdown_connections 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1531625269 wait complete. 2024-04-24T23:30:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.230+0000 7fb8b4c32740 20 client.19317 trim_cache size 0 max 16384 2024-04-24T23:30:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 Processor -- start 2024-04-24T23:30:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 -- start start 2024-04-24T23:30:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x56386352fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x5638635248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1aaa00 con 0x563863513230 2024-04-24T23:30:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1a82d0 con 0x563863513c20 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.244+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1a6120 con 0x563863518d90 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x5638635248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635303d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51302/0 (socket says 172.21.15.33:51302) 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x5638635248a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54972/0 (socket says 172.21.15.33:54972) 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8abfff640 1 -- 172.21.15.33:0/1284159727 learned_addr learned my addr 172.21.15.33:0/1284159727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1284159727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 msgr2=0x5638635303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1284159727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 msgr2=0x56386352fe90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x56386352fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 -- 172.21.15.33:0/1284159727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638632557d0 con 0x563863513c20 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8abfff640 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.245+0000 7fb8aaffd640 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x5638635248a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0003780 tx=0x7fb8a0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e886b30a26d98a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8917fa640 1 -- 172.21.15.33:0/1284159727 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0059070 con 0x563863513c20 2024-04-24T23:30:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8917fa640 1 -- 172.21.15.33:0/1284159727 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a0041070 con 0x563863513c20 2024-04-24T23:30:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8917fa640 1 -- 172.21.15.33:0/1284159727 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0035030 con 0x563863513c20 2024-04-24T23:30:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1284159727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 msgr2=0x5638635248a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x5638635248a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0003780 tx=0x7fb8a0031400 comp rx=0 tx=0).stop 2024-04-24T23:30:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1284159727 shutdown_connections 2024-04-24T23:30:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x5638635303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513c20 0x5638635248a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/1284159727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513230 0x56386352fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.246+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1284159727 >> 172.21.15.33:0/1284159727 conn(0x56386350e8a0 msgr2=0x5638635120e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1284159727 shutdown_connections 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 -- 172.21.15.33:0/1284159727 wait complete. 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 -- start start 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 Processor -- start 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x563863522980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x563863522ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x563863523e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1a6790 con 0x563863513c20 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1a7c00 con 0x563863513230 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1a6e60 con 0x563863518d90 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8b4c32740 10 client.0 _ll_register_callbacks cb 0x7fb8a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x563863522ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x563863522980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x563863523e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x563863523e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51304/0 (socket says 172.21.15.33:51304) 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 -- 172.21.15.33:0/2153721968 learned_addr learned my addr 172.21.15.33:0/2153721968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x563863522980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54974/0 (socket says 172.21.15.33:54974) 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.247+0000 7fb8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x563863522ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36882/0 (socket says 172.21.15.33:36882) 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 -- 172.21.15.33:0/2153721968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 msgr2=0x563863522980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x563863522980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 -- 172.21.15.33:0/2153721968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 msgr2=0x563863522ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x563863522ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 -- 172.21.15.33:0/2153721968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563863524450 con 0x563863518d90 2024-04-24T23:30:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x563863522980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8abfff640 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x563863523e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4022b20 tx=0x7fb8a4022b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3ca91e1213e003 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb88affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb88affd640 1 -- 172.21.15.33:0/2153721968 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a401c040 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb88affd640 1 -- 172.21.15.33:0/2153721968 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8a4022cf0 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.248+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a0041070 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb88affd640 1 -- 172.21.15.33:0/2153721968 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb8a4013ab0 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 10 client.28741 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563863590040 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb88affd640 1 -- 172.21.15.33:0/2153721968 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a0041070 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb88affd640 1 -- 172.21.15.33:0/2153721968 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb8a40374c0 con 0x563863518d90 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 10 client.28741 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 10 client.28741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 20 client.28741 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863590790 0x563863592bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 4 client.28741 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56386347ef00 con 0x563863590790 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863590790 0x563863592bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.249+0000 7fb88affd640 10 client.28741 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.250+0000 7fb8ab7fe640 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863590790 0x563863592bb0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deddee1a7d01d841 server_cookie=518b223044b80dc3 in_seq=0 out_seq=0 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb88affd640 1 -- 172.21.15.33:0/2153721968 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56386347ef00 con 0x563863590790 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb88affd640 10 client.28741 handle_command_reply: tid=0 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 client.28741 shutdown 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 msgr2=0x563863523e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x563863523e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4022b20 tx=0x7fb8a4022b50 comp rx=0 tx=0).stop 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 shutdown_connections 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563863590790 0x563863592bb0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563863518d90 0x563863523e60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563863513c20 0x563863522ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 --2- 172.21.15.33:0/2153721968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563863513230 0x563863522980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb88affd640 0 client.28741 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 >> 172.21.15.33:0/2153721968 conn(0x56386350e8a0 msgr2=0x5638635146c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 shutdown_connections 2024-04-24T23:30:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 1 -- 172.21.15.33:0/2153721968 wait complete. 2024-04-24T23:30:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.251+0000 7fb8b4c32740 20 client.28741 trim_cache size 0 max 16384 2024-04-24T23:30:32.259 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8940a2be0 msgr2=0x7fb8940a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8940a2be0 0x7fb8940a50a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4026000 tx=0x7fb8a4023000 comp rx=0 tx=0).stop 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 msgr2=0x7fb8ac197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 0x7fb8ac197b10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00e9f0 tx=0x7fb89c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 shutdown_connections 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8940a2be0 0x7fb8940a50a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ac107460 0x7fb8ac19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8ac104b30 0x7fb8ac198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 --2- 172.21.15.33:0/230929883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8ac102200 0x7fb8ac197b10 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 >> 172.21.15.33:0/230929883 conn(0x7fb8ac0fbbf0 msgr2=0x7fb8ac102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 shutdown_connections 2024-04-24T23:30:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:32.252+0000 7fb8b1f22640 1 -- 172.21.15.33:0/230929883 wait complete. 2024-04-24T23:30:32.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:32 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2077716845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:32.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 120615 22.56 MB/sec execute 157 sec latency 80.686 ms 2024-04-24T23:30:32.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:32 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2077716845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:32.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:32 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2077716845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:33.266 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:30:33.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:30:33.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:30:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 -- 172.21.15.33:0/3226587463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c102980 msgr2=0x7f817c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 --2- 172.21.15.33:0/3226587463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c102980 0x7f817c102e00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f8170009900 tx=0x7f817002a410 comp rx=0 tx=0).stop 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 -- 172.21.15.33:0/3226587463 shutdown_connections 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 --2- 172.21.15.33:0/3226587463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c103340 0x7f817c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 --2- 172.21.15.33:0/3226587463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c102980 0x7f817c102e00 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 --2- 172.21.15.33:0/3226587463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f817c101790 0x7f817c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 -- 172.21.15.33:0/3226587463 >> 172.21.15.33:0/3226587463 conn(0x7f817c0fcd20 msgr2=0x7f817c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 -- 172.21.15.33:0/3226587463 shutdown_connections 2024-04-24T23:30:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 -- 172.21.15.33:0/3226587463 wait complete. 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.396+0000 7f8181ef5640 1 Processor -- start 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 -- start start 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 0x7f817c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c102980 0x7f817c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f817c103340 0x7f817c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f817c10d410 con 0x7f817c101790 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f817c10d290 con 0x7f817c103340 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f8181ef5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f817c10d590 con 0x7f817c102980 2024-04-24T23:30:33.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 0x7f817c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 0x7f817c197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36894/0 (socket says 172.21.15.33:36894) 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 -- 172.21.15.33:0/2924479049 learned_addr learned my addr 172.21.15.33:0/2924479049 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817affd640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c102980 0x7f817c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817bfff640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f817c103340 0x7f817c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 -- 172.21.15.33:0/2924479049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c102980 msgr2=0x7f817c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c102980 0x7f817c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 -- 172.21.15.33:0/2924479049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f817c103340 msgr2=0x7f817c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f817c103340 0x7f817c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 -- 172.21.15.33:0/2924479049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c19f6a0 con 0x7f817c101790 2024-04-24T23:30:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.397+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 0x7f817c197a80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9c0 tx=0x7f816c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.398+0000 7f8178ff9640 1 -- 172.21.15.33:0/2924479049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c00cd80 con 0x7f817c101790 2024-04-24T23:30:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.398+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c19f990 con 0x7f817c101790 2024-04-24T23:30:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.398+0000 7f8178ff9640 1 -- 172.21.15.33:0/2924479049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816c0035f0 con 0x7f817c101790 2024-04-24T23:30:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.398+0000 7f8178ff9640 1 -- 172.21.15.33:0/2924479049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c010690 con 0x7f817c101790 2024-04-24T23:30:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.398+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c19ff00 con 0x7f817c101790 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.400+0000 7f8184c04740 1 Processor -- start 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 -- start start 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d7281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d72d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d72fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e1d68b2c0 con 0x557e1d72d8f0 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e1d633910 con 0x557e1d727d90 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8184c04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557e1d687280 con 0x557e1d728780 2024-04-24T23:30:33.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f8178ff9640 1 -- 172.21.15.33:0/2924479049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f816c0037a0 con 0x7f817c101790 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d72fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d7281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d72fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36902/0 (socket says 172.21.15.33:36902) 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d7281b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:54992/0 (socket says 172.21.15.33:54992) 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 -- 172.21.15.33:0/2697746624 learned_addr learned my addr 172.21.15.33:0/2697746624 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d72d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 -- 172.21.15.33:0/2697746624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 msgr2=0x557e1d72d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d72d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 -- 172.21.15.33:0/2697746624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 msgr2=0x557e1d7281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d7281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.401+0000 7f817affd640 1 -- 172.21.15.33:0/2697746624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1d644fb0 con 0x557e1d72d8f0 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f817affd640 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d72fd80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f817002a930 tx=0x7f8170005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f756998dcf9fdce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f8178ff9640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81640a2b90 0x7f81640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f8161ffb640 1 -- 172.21.15.33:0/2697746624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170040070 con 0x557e1d72d8f0 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f8161ffb640 1 -- 172.21.15.33:0/2697746624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8170004030 con 0x557e1d72d8f0 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f8161ffb640 1 -- 172.21.15.33:0/2697746624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170035040 con 0x557e1d72d8f0 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f8178ff9640 1 -- 172.21.15.33:0/2924479049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f816c0cf7a0 con 0x7f817c101790 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.402+0000 7f817bfff640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81640a2b90 0x7f81640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 -- 172.21.15.33:0/2697746624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 msgr2=0x557e1d72fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d72fd80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f817002a930 tx=0x7f8170005de0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f817bfff640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81640a2b90 0x7f81640a5050 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x557e1d728fe0 tx=0x7f8174015040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 -- 172.21.15.33:0/2697746624 shutdown_connections 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d72fd80 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d72d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 --2- 172.21.15.33:0/2697746624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d7281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 -- 172.21.15.33:0/2697746624 >> 172.21.15.33:0/2697746624 conn(0x557e1d6ef950 msgr2=0x557e1d723810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 -- 172.21.15.33:0/2697746624 shutdown_connections 2024-04-24T23:30:33.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 -- 172.21.15.33:0/2697746624 wait complete. 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 -- start start 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.403+0000 7f8184c04740 1 Processor -- start 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 0x557e1d7321b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d7326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d72d8f0 0x557e1d7396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e1d730840 con 0x557e1d727d90 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e1d646e90 con 0x557e1d728780 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557e1d7309c0 con 0x557e1d72d8f0 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f8184c04740 10 client.0 _ll_register_callbacks cb 0x557e1d682970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 0x557e1d7321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 0x557e1d7321b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36908/0 (socket says 172.21.15.33:36908) 2024-04-24T23:30:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d7326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817b7fe640 1 -- 172.21.15.33:0/3161160227 learned_addr learned my addr 172.21.15.33:0/3161160227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817bfff640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d72d8f0 0x557e1d7396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 -- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d72d8f0 msgr2=0x557e1d7396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d72d8f0 0x557e1d7396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 -- 172.21.15.33:0/3161160227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 msgr2=0x557e1d7321b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 0x557e1d7321b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 -- 172.21.15.33:0/3161160227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1d4677e0 con 0x557e1d728780 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817b7fe640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 0x557e1d7321b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f817affd640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d7326f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81700314e0 tx=0x7f8170009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0cc6a2c558498c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f815affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.404+0000 7f815affd640 1 -- 172.21.15.33:0/3161160227 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817004c020 con 0x557e1d728780 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8170004030 con 0x557e1d728780 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f815affd640 1 -- 172.21.15.33:0/3161160227 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81700043e0 con 0x557e1d728780 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f815affd640 1 -- 172.21.15.33:0/3161160227 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8170057050 con 0x557e1d728780 2024-04-24T23:30:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 10 client.28814 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e1d4c7f70 con 0x557e1d728780 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f815affd640 1 -- 172.21.15.33:0/3161160227 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8170004030 con 0x557e1d728780 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f815affd640 1 -- 172.21.15.33:0/3161160227 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8170004780 con 0x557e1d728780 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 10 client.28814 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 10 client.28814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 20 client.28814 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d73d820 0x557e1d73fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 4 client.28814 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.405+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e1d65f5a0 con 0x557e1d73d820 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.406+0000 7f817b7fe640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d73d820 0x557e1d73fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.406+0000 7f815affd640 10 client.28814 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.406+0000 7f817b7fe640 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d73d820 0x557e1d73fc10 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df23a5e7be38ac6 server_cookie=79a0c7796e8e9aa0 in_seq=0 out_seq=0 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f815affd640 1 -- 172.21.15.33:0/3161160227 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e1d65f5a0 con 0x557e1d73d820 2024-04-24T23:30:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f815affd640 10 client.28814 handle_command_reply: tid=0 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 client.28814 shutdown 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 msgr2=0x557e1d7326f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d7326f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81700314e0 tx=0x7f8170009590 comp rx=0 tx=0).stop 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 shutdown_connections 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d73d820 0x557e1d73fc10 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d72d8f0 0x557e1d7396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f815affd640 0 client.28814 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d7326f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 --2- 172.21.15.33:0/3161160227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d727d90 0x557e1d7321b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 >> 172.21.15.33:0/3161160227 conn(0x557e1d6ef950 msgr2=0x557e1d722950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 shutdown_connections 2024-04-24T23:30:33.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 1 -- 172.21.15.33:0/3161160227 wait complete. 2024-04-24T23:30:33.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.407+0000 7f8184c04740 20 client.28814 trim_cache size 0 max 16384 2024-04-24T23:30:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.427+0000 7f8184c04740 1 Processor -- start 2024-04-24T23:30:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.427+0000 7f8184c04740 1 -- start start 2024-04-24T23:30:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f8184c04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d727d90 0x557e1d7227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f8184c04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d722d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f8184c04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d739680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f8184c04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1aaa00 con 0x557e1d72d8f0 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f8184c04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1a82d0 con 0x557e1d728780 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f8184c04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1a6120 con 0x557e1d727d90 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d739680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d739680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36916/0 (socket says 172.21.15.33:36916) 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d722d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 -- 172.21.15.33:0/2478673022 learned_addr learned my addr 172.21.15.33:0/2478673022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d727d90 0x557e1d7227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 -- 172.21.15.33:0/2478673022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d727d90 msgr2=0x557e1d7227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d727d90 0x557e1d7227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 -- 172.21.15.33:0/2478673022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 msgr2=0x557e1d722d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d722d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 -- 172.21.15.33:0/2478673022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1d35f170 con 0x557e1d72d8f0 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f817bfff640 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d739680 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f817400ec20 tx=0x7f817401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcad663e859924e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f81617fa640 1 -- 172.21.15.33:0/2478673022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174031070 con 0x557e1d72d8f0 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f81617fa640 1 -- 172.21.15.33:0/2478673022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8174003970 con 0x557e1d72d8f0 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.428+0000 7f81617fa640 1 -- 172.21.15.33:0/2478673022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174004630 con 0x557e1d72d8f0 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 -- 172.21.15.33:0/2478673022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 msgr2=0x557e1d739680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d739680 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f817400ec20 tx=0x7f817401f000 comp rx=0 tx=0).stop 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 -- 172.21.15.33:0/2478673022 shutdown_connections 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d739680 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d728780 0x557e1d722d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 --2- 172.21.15.33:0/2478673022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d727d90 0x557e1d7227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 -- 172.21.15.33:0/2478673022 >> 172.21.15.33:0/2478673022 conn(0x557e1d6ef950 msgr2=0x557e1d7220c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 -- 172.21.15.33:0/2478673022 shutdown_connections 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 -- 172.21.15.33:0/2478673022 wait complete. 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.429+0000 7f8184c04740 1 -- start start 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 Processor -- start 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d736e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d7373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d738360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1a6790 con 0x557e1d72d8f0 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1a7c00 con 0x557e1d727d90 2024-04-24T23:30:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1a6e60 con 0x557e1d728780 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f8184c04740 10 client.0 _ll_register_callbacks cb 0x7f8174009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f817bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d738360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d736e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f817affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d7373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d736e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55022/0 (socket says 172.21.15.33:55022) 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f817bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d738360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36930/0 (socket says 172.21.15.33:36930) 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.430+0000 7f817b7fe640 1 -- 172.21.15.33:0/1773761742 learned_addr learned my addr 172.21.15.33:0/1773761742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817affd640 1 -- 172.21.15.33:0/1773761742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 msgr2=0x557e1d736e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817affd640 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d736e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817affd640 1 -- 172.21.15.33:0/1773761742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 msgr2=0x557e1d738360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817affd640 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d738360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817affd640 1 -- 172.21.15.33:0/1773761742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1d738950 con 0x557e1d728780 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817bfff640 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d738360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f817affd640 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d7373a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f81700057c0 tx=0x7f8170005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d525c981f4b972 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f815affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f815affd640 1 -- 172.21.15.33:0/1773761742 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817004c040 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8174003970 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f815affd640 1 -- 172.21.15.33:0/1773761742 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8170005220 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f815affd640 1 -- 172.21.15.33:0/1773761742 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8170040070 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f8184c04740 10 client.28759 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e1d7a3da0 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.431+0000 7f815affd640 1 -- 172.21.15.33:0/1773761742 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8174003970 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f815affd640 1 -- 172.21.15.33:0/1773761742 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f817003c560 con 0x557e1d728780 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f8184c04740 10 client.28759 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f8184c04740 10 client.28759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f8184c04740 20 client.28759 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f8184c04740 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d7a4730 0x557e1d7a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f8184c04740 4 client.28759 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557e1d698800 con 0x557e1d7a4730 2024-04-24T23:30:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f817b7fe640 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d7a4730 0x557e1d7a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f817b7fe640 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d7a4730 0x557e1d7a6b50 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993d5b2f316812f6 server_cookie=51c798db46ebdc6f in_seq=0 out_seq=0 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.432+0000 7f815affd640 10 client.28759 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f815affd640 1 -- 172.21.15.33:0/1773761742 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557e1d698800 con 0x557e1d7a4730 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f815affd640 10 client.28759 handle_command_reply: tid=0 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 client.28759 shutdown 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 msgr2=0x557e1d7373a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d7373a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f81700057c0 tx=0x7f8170005fd0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 shutdown_connections 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e1d7a4730 0x557e1d7a6b50 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e1d72d8f0 0x557e1d738360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f815affd640 0 client.28759 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e1d728780 0x557e1d7373a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 --2- 172.21.15.33:0/1773761742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e1d727d90 0x557e1d736e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 >> 172.21.15.33:0/1773761742 conn(0x557e1d6ef950 msgr2=0x557e1d722950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 shutdown_connections 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.433+0000 7f8184c04740 1 -- 172.21.15.33:0/1773761742 wait complete. 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8184c04740 20 client.28759 trim_cache size 0 max 16384 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "4426e723-6f83-4aed-aa9a-566bd33c5966", 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:30:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:30:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81640a2b90 msgr2=0x7f81640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81640a2b90 0x7f81640a5050 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x557e1d728fe0 tx=0x7f8174015040 comp rx=0 tx=0).stop 2024-04-24T23:30:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 msgr2=0x7f817c197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 0x7f817c197a80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9c0 tx=0x7f816c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:30:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 shutdown_connections 2024-04-24T23:30:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f81640a2b90 0x7f81640a5050 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f817c103340 0x7f817c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c102980 0x7f817c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 --2- 172.21.15.33:0/2924479049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c101790 0x7f817c197a80 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 >> 172.21.15.33:0/2924479049 conn(0x7f817c0fcd20 msgr2=0x7f817c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 shutdown_connections 2024-04-24T23:30:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.434+0000 7f8181ef5640 1 -- 172.21.15.33:0/2924479049 wait complete. 2024-04-24T23:30:33.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:33.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:33.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 -- 172.21.15.33:0/2098223384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 msgr2=0x7fb8f0109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 --2- 172.21.15.33:0/2098223384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f0109bd0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009ca0 tx=0x7fb8e802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 -- 172.21.15.33:0/2098223384 shutdown_connections 2024-04-24T23:30:33.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 --2- 172.21.15.33:0/2098223384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f0109bd0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 --2- 172.21.15.33:0/2098223384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8f0102980 0x7fb8f0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 --2- 172.21.15.33:0/2098223384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 0x7fb8f0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 -- 172.21.15.33:0/2098223384 >> 172.21.15.33:0/2098223384 conn(0x7fb8f00fcd20 msgr2=0x7fb8f00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 -- 172.21.15.33:0/2098223384 shutdown_connections 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.561+0000 7fb8f625d640 1 -- 172.21.15.33:0/2098223384 wait complete. 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 Processor -- start 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 -- start start 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 0x7fb8f0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8f0102980 0x7fb8f019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f01a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010d410 con 0x7fb8f0103340 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010d290 con 0x7fb8f0101790 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8f625d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010d590 con 0x7fb8f0102980 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f01a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f01a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36958/0 (socket says 172.21.15.33:36958) 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8effff640 1 -- 172.21.15.33:0/3898188925 learned_addr learned my addr 172.21.15.33:0/3898188925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8f0102980 0x7fb8f019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.562+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 0x7fb8f0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8effff640 1 -- 172.21.15.33:0/3898188925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8f0102980 msgr2=0x7fb8f019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8effff640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8f0102980 0x7fb8f019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8effff640 1 -- 172.21.15.33:0/3898188925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 msgr2=0x7fb8f0199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8effff640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 0x7fb8f0199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8effff640 1 -- 172.21.15.33:0/3898188925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f01a18e0 con 0x7fb8f0103340 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 0x7fb8f0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8effff640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f01a1240 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8031ae0 tx=0x7fb8e8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8ecff9640 1 -- 172.21.15.33:0/3898188925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8003b70 con 0x7fb8f0103340 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8ecff9640 1 -- 172.21.15.33:0/3898188925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e8003d10 con 0x7fb8f0103340 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f01a1b70 con 0x7fb8f0103340 2024-04-24T23:30:33.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8ecff9640 1 -- 172.21.15.33:0/3898188925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e802fc40 con 0x7fb8f0103340 2024-04-24T23:30:33.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.563+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f01a20c0 con 0x7fb8f0103340 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8ecff9640 1 -- 172.21.15.33:0/3898188925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb8e8004050 con 0x7fb8f0103340 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 Processor -- start 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 -- start start 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58617660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf5861c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf5861f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf5857b6b0 con 0x55bf58617240 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf58523880 con 0x55bf58617c30 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf58577670 con 0x55bf5861cda0 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.566+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58617660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ecff9640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8d80a2b90 0x7fb8d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58617660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36964/0 (socket says 172.21.15.33:36964) 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8eeffd640 1 -- 172.21.15.33:0/3858904283 learned_addr learned my addr 172.21.15.33:0/3858904283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf5861c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf5861f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8effff640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8d80a2b90 0x7fb8d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ecff9640 1 -- 172.21.15.33:0/3898188925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8e80f04a0 con 0x7fb8f0103340 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/3858904283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 msgr2=0x55bf5861f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf5861f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/3858904283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 msgr2=0x55bf58617660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58617660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/3858904283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf583597d0 con 0x55bf58617c30 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58617660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8effff640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8d80a2b90 0x7fb8d80a5050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8005e30 tx=0x7fb8e8037c40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf5861c730 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb8f01027e0 tx=0x7fb8e00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402257b49cd0000f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8d5ffb640 1 -- 172.21.15.33:0/3858904283 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0004310 con 0x55bf58617c30 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8d5ffb640 1 -- 172.21.15.33:0/3858904283 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e00044b0 con 0x55bf58617c30 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.567+0000 7fb8d5ffb640 1 -- 172.21.15.33:0/3858904283 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00035f0 con 0x55bf58617c30 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3858904283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 msgr2=0x55bf5861c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf5861c730 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb8f01027e0 tx=0x7fb8e00118d0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3858904283 shutdown_connections 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf5861f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf5861c730 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3858904283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58617660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3858904283 >> 172.21.15.33:0/3858904283 conn(0x55bf586128b0 msgr2=0x55bf58612cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3858904283 shutdown_connections 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.568+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3858904283 wait complete. 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 -- start start 2024-04-24T23:30:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 Processor -- start 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 0x55bf58621630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58621b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58628bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf5861fd50 con 0x55bf5861cda0 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf58536f50 con 0x55bf58617c30 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf5861fed0 con 0x55bf58617240 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8f8f6f740 10 client.0 _ll_register_callbacks cb 0x55bf58572000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58628bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8ef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58621b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 0x55bf58621630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8ef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58621b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55054/0 (socket says 172.21.15.33:55054) 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 0x55bf58621630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51398/0 (socket says 172.21.15.33:51398) 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.569+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/2166263001 learned_addr learned my addr 172.21.15.33:0/2166263001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8eeffd640 1 -- 172.21.15.33:0/2166263001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 msgr2=0x55bf58621b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58621b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8eeffd640 1 -- 172.21.15.33:0/2166263001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 msgr2=0x55bf58628bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58628bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8eeffd640 1 -- 172.21.15.33:0/2166263001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5828f3f0 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 0x55bf58621630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009710 tx=0x7fb8e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeacd87d70c18d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8ceffd640 1 -- 172.21.15.33:0/2166263001 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4004220 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8ceffd640 1 -- 172.21.15.33:0/2166263001 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8e40043c0 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e00044b0 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8ceffd640 1 -- 172.21.15.33:0/2166263001 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb8e4005020 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8f8f6f740 10 client.28771 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf5824f0f0 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8ceffd640 1 -- 172.21.15.33:0/2166263001 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e00044b0 con 0x55bf58617240 2024-04-24T23:30:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8ceffd640 1 -- 172.21.15.33:0/2166263001 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e4004c60 con 0x55bf58617240 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.570+0000 7fb8f8f6f740 10 client.28771 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8f8f6f740 10 client.28771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8f8f6f740 20 client.28771 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf5862d4a0 0x55bf5862f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8f8f6f740 4 client.28771 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf58588bf0 con 0x55bf5862d4a0 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf5862d4a0 0x55bf5862f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf5862d4a0 0x55bf5862f960 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd7e1fda50cbc0c server_cookie=8106f22f59f25e7a in_seq=0 out_seq=0 2024-04-24T23:30:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.571+0000 7fb8ceffd640 10 client.28771 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8ceffd640 1 -- 172.21.15.33:0/2166263001 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf58588bf0 con 0x55bf5862d4a0 2024-04-24T23:30:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8ceffd640 10 client.28771 handle_command_reply: tid=0 2024-04-24T23:30:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 client.28771 shutdown 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 msgr2=0x55bf58621630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 0x55bf58621630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009710 tx=0x7fb8e4009780 comp rx=0 tx=0).stop 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 shutdown_connections 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf5862d4a0 0x55bf5862f960 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58628bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58621b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8ceffd640 0 client.28771 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/2166263001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617240 0x55bf58621630 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 >> 172.21.15.33:0/2166263001 conn(0x55bf586128b0 msgr2=0x55bf58611af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 shutdown_connections 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/2166263001 wait complete. 2024-04-24T23:30:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.572+0000 7fb8f8f6f740 20 client.28771 trim_cache size 0 max 16384 2024-04-24T23:30:33.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 Processor -- start 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 -- start start 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58621950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58628b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf58615630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01aee90 con 0x55bf58617240 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01ac760 con 0x55bf58617c30 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01aa5b0 con 0x55bf5861cda0 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58621950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58621950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36974/0 (socket says 172.21.15.33:36974) 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 -- 172.21.15.33:0/635982988 learned_addr learned my addr 172.21.15.33:0/635982988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58628b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf58615630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 -- 172.21.15.33:0/635982988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 msgr2=0x55bf58615630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf58615630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 -- 172.21.15.33:0/635982988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 msgr2=0x55bf58628b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58628b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 -- 172.21.15.33:0/635982988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf58618010 con 0x55bf58617240 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.592+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58621950 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009d00 tx=0x7fb8e4009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46878e50c103a3b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8d57fa640 1 -- 172.21.15.33:0/635982988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4005480 con 0x55bf58617240 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8d57fa640 1 -- 172.21.15.33:0/635982988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4005620 con 0x55bf58617240 2024-04-24T23:30:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8d57fa640 1 -- 172.21.15.33:0/635982988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4004ce0 con 0x55bf58617240 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/635982988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 msgr2=0x55bf58621950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58621950 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009d00 tx=0x7fb8e4009c90 comp rx=0 tx=0).stop 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/635982988 shutdown_connections 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf5861cda0 0x55bf58615630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617c30 0x55bf58628b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/635982988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58617240 0x55bf58621950 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/635982988 >> 172.21.15.33:0/635982988 conn(0x55bf586128b0 msgr2=0x55bf58621240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.593+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/635982988 shutdown_connections 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/635982988 wait complete. 2024-04-24T23:30:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 -- start start 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 Processor -- start 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617240 0x55bf586262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 0x55bf58626810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58627820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01aac20 con 0x55bf5861cda0 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01ac090 con 0x55bf58617240 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01ab2f0 con 0x55bf58617c30 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8f8f6f740 10 client.0 _ll_register_callbacks cb 0x7fb8e4013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 0x55bf58626810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617240 0x55bf586262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617240 0x55bf586262d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:55068/0 (socket says 172.21.15.33:55068) 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 0x55bf58626810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51424/0 (socket says 172.21.15.33:51424) 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8eeffd640 1 -- 172.21.15.33:0/3774958539 learned_addr learned my addr 172.21.15.33:0/3774958539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:30:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.594+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58627820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/3774958539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617240 msgr2=0x55bf586262d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617240 0x55bf586262d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/3774958539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 msgr2=0x55bf58627820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58627820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ef7fe640 1 -- 172.21.15.33:0/3774958539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf58627e10 con 0x55bf58617c30 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58627820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ef7fe640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 0x55bf58626810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd59b6f42490cf98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ceffd640 1 -- 172.21.15.33:0/3774958539 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0032020 con 0x55bf58617c30 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ceffd640 1 -- 172.21.15.33:0/3774958539 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8e0004260 con 0x55bf58617c30 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e4005620 con 0x55bf58617c30 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8ceffd640 1 -- 172.21.15.33:0/3774958539 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb8e003d050 con 0x55bf58617c30 2024-04-24T23:30:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8f8f6f740 10 client.28777 fetch_fsmap learned FSMap version 33 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.595+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf58693aa0 con 0x55bf58617c30 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8ceffd640 1 -- 172.21.15.33:0/3774958539 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4005620 con 0x55bf58617c30 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8ceffd640 1 -- 172.21.15.33:0/3774958539 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e0003990 con 0x55bf58617c30 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8f8f6f740 10 client.28777 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8f8f6f740 10 client.28777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8f8f6f740 20 client.28777 populate_metadata read hostname 'smithi033' 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf58694300 0x55bf586966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8f8f6f740 4 client.28777 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf58582f40 con 0x55bf58694300 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf58694300 0x55bf586966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8ceffd640 10 client.28777 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.596+0000 7fb8eeffd640 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf58694300 0x55bf586966f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda0d4d31d688f57 server_cookie=c711656d71f1b6d6 in_seq=0 out_seq=0 2024-04-24T23:30:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8ceffd640 1 -- 172.21.15.33:0/3774958539 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bf58582f40 con 0x55bf58694300 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8ceffd640 10 client.28777 handle_command_reply: tid=0 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 client.28777 shutdown 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 msgr2=0x55bf58626810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 0x55bf58626810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e0009310 comp rx=0 tx=0).stop 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 shutdown_connections 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bf58694300 0x55bf586966f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf5861cda0 0x55bf58627820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf58617c30 0x55bf58626810 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8ceffd640 0 client.28777 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 --2- 172.21.15.33:0/3774958539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf58617240 0x55bf586262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 >> 172.21.15.33:0/3774958539 conn(0x55bf586128b0 msgr2=0x55bf58611260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 shutdown_connections 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 1 -- 172.21.15.33:0/3774958539 wait complete. 2024-04-24T23:30:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.597+0000 7fb8f8f6f740 20 client.28777 trim_cache size 0 max 16384 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: "4426e723-6f83-4aed-aa9a-566bd33c5966": { 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "4426e723-6f83-4aed-aa9a-566bd33c5966", 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:30:33.604 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:30:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8d80a2b90 msgr2=0x7fb8d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8d80a2b90 0x7fb8d80a5050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8005e30 tx=0x7fb8e8037c40 comp rx=0 tx=0).stop 2024-04-24T23:30:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 msgr2=0x7fb8f01a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f01a1240 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8031ae0 tx=0x7fb8e8031ee0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 shutdown_connections 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8d80a2b90 0x7fb8d80a5050 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8f0103340 0x7fb8f01a1240 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8f0102980 0x7fb8f019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 --2- 172.21.15.33:0/3898188925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8f0101790 0x7fb8f0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 >> 172.21.15.33:0/3898188925 conn(0x7fb8f00fcd20 msgr2=0x7fb8f0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 shutdown_connections 2024-04-24T23:30:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:30:33.598+0000 7fb8f625d640 1 -- 172.21.15.33:0/3898188925 wait complete. 2024-04-24T23:30:33.606 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:33 smithi033 ceph-mon[29201]: pgmap v907: 129 pgs: 129 active+clean; 538 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2024-04-24T23:30:33.611 INFO:tasks.cephfs.filesystem:scrub status for tag:4426e723-6f83-4aed-aa9a-566bd33c5966 - {'path': '/', 'tag': '4426e723-6f83-4aed-aa9a-566bd33c5966', 'options': 'recursive,force'} 2024-04-24T23:30:33.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 121106 22.62 MB/sec execute 158 sec latency 99.052 ms 2024-04-24T23:30:33.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:33 smithi038 ceph-mon[32450]: pgmap v907: 129 pgs: 129 active+clean; 538 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2024-04-24T23:30:33.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:33 smithi117 ceph-mon[32529]: pgmap v907: 129 pgs: 129 active+clean; 538 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2024-04-24T23:30:34.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 121318 22.54 MB/sec execute 159 sec latency 190.565 ms 2024-04-24T23:30:34.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:30:34.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:34 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:30:34.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:34 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:30:34.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:34 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:35.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 121690 22.51 MB/sec execute 160 sec latency 86.737 ms 2024-04-24T23:30:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:35 smithi038 ceph-mon[32450]: pgmap v908: 129 pgs: 129 active+clean; 539 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 421 op/s 2024-04-24T23:30:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:35 smithi117 ceph-mon[32529]: pgmap v908: 129 pgs: 129 active+clean; 539 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 421 op/s 2024-04-24T23:30:35.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:35 smithi033 ceph-mon[29201]: pgmap v908: 129 pgs: 129 active+clean; 539 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 421 op/s 2024-04-24T23:30:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:36.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 122157 22.51 MB/sec execute 161 sec latency 105.457 ms 2024-04-24T23:30:37.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 122725 22.57 MB/sec execute 162 sec latency 109.861 ms 2024-04-24T23:30:37.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:37 smithi038 ceph-mon[32450]: pgmap v909: 129 pgs: 129 active+clean; 540 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-04-24T23:30:37.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:37 smithi117 ceph-mon[32529]: pgmap v909: 129 pgs: 129 active+clean; 540 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-04-24T23:30:37.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:37 smithi033 ceph-mon[29201]: pgmap v909: 129 pgs: 129 active+clean; 540 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-04-24T23:30:38.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 123224 22.64 MB/sec execute 163 sec latency 80.529 ms 2024-04-24T23:30:39.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 123630 22.63 MB/sec execute 164 sec latency 287.948 ms 2024-04-24T23:30:39.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:39 smithi038 ceph-mon[32450]: pgmap v910: 129 pgs: 129 active+clean; 544 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 435 op/s 2024-04-24T23:30:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:39 smithi117 ceph-mon[32529]: pgmap v910: 129 pgs: 129 active+clean; 544 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 435 op/s 2024-04-24T23:30:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:39 smithi033 ceph-mon[29201]: pgmap v910: 129 pgs: 129 active+clean; 544 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 435 op/s 2024-04-24T23:30:40.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 123877 22.58 MB/sec execute 165 sec latency 165.261 ms 2024-04-24T23:30:41.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 124344 22.58 MB/sec execute 166 sec latency 76.478 ms 2024-04-24T23:30:41.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:41 smithi038 ceph-mon[32450]: pgmap v911: 129 pgs: 129 active+clean; 546 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 402 op/s 2024-04-24T23:30:41.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:41 smithi117 ceph-mon[32529]: pgmap v911: 129 pgs: 129 active+clean; 546 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 402 op/s 2024-04-24T23:30:41.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:41 smithi033 ceph-mon[29201]: pgmap v911: 129 pgs: 129 active+clean; 546 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 402 op/s 2024-04-24T23:30:42.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 124700 22.58 MB/sec execute 167 sec latency 79.677 ms 2024-04-24T23:30:43.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 125060 22.52 MB/sec execute 168 sec latency 144.893 ms 2024-04-24T23:30:43.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:43 smithi038 ceph-mon[32450]: pgmap v912: 129 pgs: 129 active+clean; 553 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 515 op/s 2024-04-24T23:30:43.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:43 smithi117 ceph-mon[32529]: pgmap v912: 129 pgs: 129 active+clean; 553 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 515 op/s 2024-04-24T23:30:43.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:43 smithi033 ceph-mon[29201]: pgmap v912: 129 pgs: 129 active+clean; 553 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 515 op/s 2024-04-24T23:30:44.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 125350 22.45 MB/sec execute 169 sec latency 272.107 ms 2024-04-24T23:30:45.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 125882 22.50 MB/sec execute 170 sec latency 80.966 ms 2024-04-24T23:30:45.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:45 smithi038 ceph-mon[32450]: pgmap v913: 129 pgs: 129 active+clean; 553 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 431 op/s 2024-04-24T23:30:45.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:45 smithi117 ceph-mon[32529]: pgmap v913: 129 pgs: 129 active+clean; 553 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 431 op/s 2024-04-24T23:30:45.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:45 smithi033 ceph-mon[29201]: pgmap v913: 129 pgs: 129 active+clean; 553 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 431 op/s 2024-04-24T23:30:46.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 126622 22.60 MB/sec execute 171 sec latency 99.458 ms 2024-04-24T23:30:47.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 127015 22.63 MB/sec execute 172 sec latency 120.165 ms 2024-04-24T23:30:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:47 smithi038 ceph-mon[32450]: pgmap v914: 129 pgs: 129 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.7 MiB/s wr, 462 op/s 2024-04-24T23:30:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:47 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:30:47.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:47 smithi117 ceph-mon[32529]: pgmap v914: 129 pgs: 129 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.7 MiB/s wr, 462 op/s 2024-04-24T23:30:47.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:47 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:30:47.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:47 smithi033 ceph-mon[29201]: pgmap v914: 129 pgs: 129 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.7 MiB/s wr, 462 op/s 2024-04-24T23:30:47.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:47 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:30:48.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 127435 22.66 MB/sec execute 173 sec latency 68.932 ms 2024-04-24T23:30:48.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:48 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:30:48.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:48 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:30:48.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:48 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:30:48.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:48 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:30:48.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:48 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:30:48.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:48 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:30:49.825 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:49 smithi117 ceph-mon[32529]: pgmap v915: 129 pgs: 129 active+clean; 558 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 437 op/s 2024-04-24T23:30:49.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 127646 22.60 MB/sec execute 174 sec latency 405.411 ms 2024-04-24T23:30:49.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:49 smithi038 ceph-mon[32450]: pgmap v915: 129 pgs: 129 active+clean; 558 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 437 op/s 2024-04-24T23:30:49.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:49 smithi033 ceph-mon[29201]: pgmap v915: 129 pgs: 129 active+clean; 558 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 437 op/s 2024-04-24T23:30:50.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 128082 22.57 MB/sec execute 175 sec latency 103.497 ms 2024-04-24T23:30:51.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 128423 22.53 MB/sec execute 176 sec latency 127.281 ms 2024-04-24T23:30:51.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:51 smithi038 ceph-mon[32450]: pgmap v916: 129 pgs: 129 active+clean; 558 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 365 op/s 2024-04-24T23:30:51.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:51 smithi117 ceph-mon[32529]: pgmap v916: 129 pgs: 129 active+clean; 558 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 365 op/s 2024-04-24T23:30:51.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:51 smithi033 ceph-mon[29201]: pgmap v916: 129 pgs: 129 active+clean; 558 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 365 op/s 2024-04-24T23:30:52.890 INFO:tasks.workunit.client.0.smithi033.stdout: 10 128922 22.57 MB/sec execute 177 sec latency 87.634 ms 2024-04-24T23:30:53.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 129375 22.56 MB/sec execute 178 sec latency 70.863 ms 2024-04-24T23:30:53.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:53 smithi038 ceph-mon[32450]: pgmap v917: 129 pgs: 129 active+clean; 564 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 449 op/s 2024-04-24T23:30:53.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:53 smithi117 ceph-mon[32529]: pgmap v917: 129 pgs: 129 active+clean; 564 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 449 op/s 2024-04-24T23:30:53.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:53 smithi033 ceph-mon[29201]: pgmap v917: 129 pgs: 129 active+clean; 564 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 449 op/s 2024-04-24T23:30:54.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 129721 22.57 MB/sec execute 179 sec latency 265.955 ms 2024-04-24T23:30:55.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 130184 22.55 MB/sec execute 180 sec latency 120.384 ms 2024-04-24T23:30:55.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:55 smithi038 ceph-mon[32450]: pgmap v918: 129 pgs: 129 active+clean; 567 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 378 op/s 2024-04-24T23:30:55.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:55 smithi117 ceph-mon[32529]: pgmap v918: 129 pgs: 129 active+clean; 567 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 378 op/s 2024-04-24T23:30:55.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:55 smithi033 ceph-mon[29201]: pgmap v918: 129 pgs: 129 active+clean; 567 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 378 op/s 2024-04-24T23:30:56.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 130634 22.55 MB/sec execute 181 sec latency 67.811 ms 2024-04-24T23:30:57.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 130967 22.55 MB/sec execute 182 sec latency 156.378 ms 2024-04-24T23:30:57.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:57 smithi038 ceph-mon[32450]: pgmap v919: 129 pgs: 129 active+clean; 550 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 423 op/s 2024-04-24T23:30:57.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:57 smithi117 ceph-mon[32529]: pgmap v919: 129 pgs: 129 active+clean; 550 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 423 op/s 2024-04-24T23:30:57.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:57 smithi033 ceph-mon[29201]: pgmap v919: 129 pgs: 129 active+clean; 550 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 423 op/s 2024-04-24T23:30:58.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 131460 22.59 MB/sec execute 183 sec latency 57.335 ms 2024-04-24T23:30:59.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 131935 22.62 MB/sec execute 184 sec latency 286.944 ms 2024-04-24T23:30:59.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:30:59 smithi038 ceph-mon[32450]: pgmap v920: 129 pgs: 129 active+clean; 552 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.6 MiB/s wr, 382 op/s 2024-04-24T23:30:59.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:30:59 smithi117 ceph-mon[32529]: pgmap v920: 129 pgs: 129 active+clean; 552 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.6 MiB/s wr, 382 op/s 2024-04-24T23:30:59.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:30:59 smithi033 ceph-mon[29201]: pgmap v920: 129 pgs: 129 active+clean; 552 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.6 MiB/s wr, 382 op/s 2024-04-24T23:31:00.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 132380 22.60 MB/sec execute 185 sec latency 352.491 ms 2024-04-24T23:31:01.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 132628 22.57 MB/sec execute 186 sec latency 116.890 ms 2024-04-24T23:31:01.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:01 smithi038 ceph-mon[32450]: pgmap v921: 129 pgs: 129 active+clean; 552 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 335 op/s 2024-04-24T23:31:01.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:01 smithi117 ceph-mon[32529]: pgmap v921: 129 pgs: 129 active+clean; 552 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 335 op/s 2024-04-24T23:31:01.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:01 smithi033 ceph-mon[29201]: pgmap v921: 129 pgs: 129 active+clean; 552 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 335 op/s 2024-04-24T23:31:02.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 133007 22.53 MB/sec execute 187 sec latency 65.898 ms 2024-04-24T23:31:03.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:03.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 -- 172.21.15.33:0/121109850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4107460 msgr2=0x7f87c41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 --2- 172.21.15.33:0/121109850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4107460 0x7f87c41098b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009ca0 tx=0x7f87bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 -- 172.21.15.33:0/121109850 shutdown_connections 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 --2- 172.21.15.33:0/121109850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4107460 0x7f87c41098b0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 --2- 172.21.15.33:0/121109850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c4104b30 0x7f87c4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 --2- 172.21.15.33:0/121109850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 0x7f87c41045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 -- 172.21.15.33:0/121109850 >> 172.21.15.33:0/121109850 conn(0x7f87c40fbbf0 msgr2=0x7f87c40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 -- 172.21.15.33:0/121109850 shutdown_connections 2024-04-24T23:31:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 -- 172.21.15.33:0/121109850 wait complete. 2024-04-24T23:31:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.727+0000 7f87c99d5640 1 Processor -- start 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 -- start start 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 0x7f87c4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 0x7f87c4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c4107460 0x7f87c419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410d420 con 0x7f87c4104b30 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410d2a0 con 0x7f87c4102200 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c99d5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410d5a0 con 0x7f87c4107460 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 0x7f87c4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 0x7f87c4197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60618/0 (socket says 172.21.15.33:60618) 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c27fc640 1 -- 172.21.15.33:0/1544108399 learned_addr learned my addr 172.21.15.33:0/1544108399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 0x7f87c4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.728+0000 7f87c37fe640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c4107460 0x7f87c419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c27fc640 1 -- 172.21.15.33:0/1544108399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c4107460 msgr2=0x7f87c419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c27fc640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c4107460 0x7f87c419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c27fc640 1 -- 172.21.15.33:0/1544108399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 msgr2=0x7f87c4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c27fc640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 0x7f87c4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c27fc640 1 -- 172.21.15.33:0/1544108399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c419f6a0 con 0x7f87c4104b30 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 0x7f87c4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c27fc640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 0x7f87c4197fc0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f87b800e9f0 tx=0x7f87b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c89d3640 1 -- 172.21.15.33:0/1544108399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b800ccc0 con 0x7f87c4104b30 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c89d3640 1 -- 172.21.15.33:0/1544108399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b8005500 con 0x7f87c4104b30 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c419f990 con 0x7f87c4104b30 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.729+0000 7f87c89d3640 1 -- 172.21.15.33:0/1544108399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8010690 con 0x7f87c4104b30 2024-04-24T23:31:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.730+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c40ffff0 con 0x7f87c4104b30 2024-04-24T23:31:03.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.732+0000 7f87c89d3640 1 -- 172.21.15.33:0/1544108399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f87b80046c0 con 0x7f87c4104b30 2024-04-24T23:31:03.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.733+0000 7f87c89d3640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87ac0a6fa0 0x7f87ac0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 Processor -- start 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 -- start start 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87ac0a6fa0 0x7f87ac0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f9f5186b0 con 0x560f9f5c25b0 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560f9f4c0880 con 0x560f9f5bca50 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560f9f514670 con 0x560f9f5bd440 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c89d3640 1 -- 172.21.15.33:0/1544108399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87b80cf7a0 con 0x7f87c4104b30 2024-04-24T23:31:03.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5c1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35328/0 (socket says 172.21.15.33:35328) 2024-04-24T23:31:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c27fc640 1 -- 172.21.15.33:0/3736357018 learned_addr learned my addr 172.21.15.33:0/3736357018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.734+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5bce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c37fe640 1 -- 172.21.15.33:0/3736357018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 msgr2=0x560f9f5c1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c37fe640 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5c1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c37fe640 1 -- 172.21.15.33:0/3736357018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 msgr2=0x560f9f5bce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c37fe640 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5bce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c37fe640 1 -- 172.21.15.33:0/3736357018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9f2f67d0 con 0x560f9f5c25b0 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c27fc640 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87ac0a6fa0 0x7f87ac0a9460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f87b40095f0 tx=0x7f87b4011cb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87c37fe640 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5c4a40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f87bc031ae0 tx=0x7f87bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ed1497a015342c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87a9ffb640 1 -- 172.21.15.33:0/3736357018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc009800 con 0x560f9f5c25b0 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87a9ffb640 1 -- 172.21.15.33:0/3736357018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87bc036040 con 0x560f9f5c25b0 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.735+0000 7f87a9ffb640 1 -- 172.21.15.33:0/3736357018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc004030 con 0x560f9f5c25b0 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 -- 172.21.15.33:0/3736357018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 msgr2=0x560f9f5c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5c4a40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f87bc031ae0 tx=0x7f87bc004a40 comp rx=0 tx=0).stop 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 -- 172.21.15.33:0/3736357018 shutdown_connections 2024-04-24T23:31:03.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5c4a40 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/3736357018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5bce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.736+0000 7f87cc7c3740 1 -- 172.21.15.33:0/3736357018 >> 172.21.15.33:0/3736357018 conn(0x560f9f5af8b0 msgr2=0x560f9f5afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 -- 172.21.15.33:0/3736357018 shutdown_connections 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 -- 172.21.15.33:0/3736357018 wait complete. 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 -- start start 2024-04-24T23:31:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 Processor -- start 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 0x560f9f5b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 0x560f9f5bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f9f5b4130 con 0x560f9f5bca50 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560f9f4d3f50 con 0x560f9f5c25b0 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.737+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560f9f5b42b0 con 0x560f9f5bd440 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87cc7c3740 10 client.0 _ll_register_callbacks cb 0x560f9f50f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 0x560f9f5b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 0x560f9f5bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 0x560f9f5b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60646/0 (socket says 172.21.15.33:60646) 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c2ffd640 1 -- 172.21.15.33:0/187869881 learned_addr learned my addr 172.21.15.33:0/187869881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35334/0 (socket says 172.21.15.33:35334) 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 0x560f9f5bafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57110/0 (socket says 172.21.15.33:57110) 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 -- 172.21.15.33:0/187869881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 msgr2=0x560f9f5bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 0x560f9f5bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 -- 172.21.15.33:0/187869881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 msgr2=0x560f9f5b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 0x560f9f5b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 -- 172.21.15.33:0/187869881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9f22c3f0 con 0x560f9f5bd440 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c37fe640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 0x560f9f5bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 0x560f9f5b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87c27fc640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b5f90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f87b80a43b0 tx=0x7f87b80056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a629093b079f3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.738+0000 7f87a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:31:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87a2ffd640 1 -- 172.21.15.33:0/187869881 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80a0020 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87a2ffd640 1 -- 172.21.15.33:0/187869881 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87b808e180 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87bc036040 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87a2ffd640 1 -- 172.21.15.33:0/187869881 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f87b808e380 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87cc7c3740 10 client.28789 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f9f1ec0f0 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.739+0000 7f87a2ffd640 1 -- 172.21.15.33:0/187869881 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87bc036040 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87a2ffd640 1 -- 172.21.15.33:0/187869881 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f87b809ad30 con 0x560f9f5bd440 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87cc7c3740 10 client.28789 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87cc7c3740 10 client.28789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87cc7c3740 20 client.28789 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f5db160 0x560f9f5dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87cc7c3740 4 client.28789 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9f525bf0 con 0x560f9f5db160 2024-04-24T23:31:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.740+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f5db160 0x560f9f5dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.741+0000 7f87a2ffd640 10 client.28789 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.741+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f5db160 0x560f9f5dd5f0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3265c853a6ef59a1 server_cookie=f92c1fce1b0b6d45 in_seq=0 out_seq=0 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.741+0000 7f87a2ffd640 1 -- 172.21.15.33:0/187869881 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f9f525bf0 con 0x560f9f5db160 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.741+0000 7f87a2ffd640 10 client.28789 handle_command_reply: tid=0 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.741+0000 7f87cc7c3740 1 client.28789 shutdown 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 msgr2=0x560f9f5b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b5f90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f87b80a43b0 tx=0x7f87b80056b0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 shutdown_connections 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f5db160 0x560f9f5dd5f0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5c25b0 0x560f9f5bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b5f90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87a2ffd640 0 client.28789 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/187869881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5bca50 0x560f9f5b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 >> 172.21.15.33:0/187869881 conn(0x560f9f5af8b0 msgr2=0x560f9f5aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 shutdown_connections 2024-04-24T23:31:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 1 -- 172.21.15.33:0/187869881 wait complete. 2024-04-24T23:31:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.742+0000 7f87cc7c3740 20 client.28789 trim_cache size 0 max 16384 2024-04-24T23:31:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 Processor -- start 2024-04-24T23:31:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 -- start start 2024-04-24T23:31:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41aa7f0 con 0x560f9f5c25b0 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41a80c0 con 0x560f9f5bca50 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41a5fc0 con 0x560f9f5bd440 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60650/0 (socket says 172.21.15.33:60650) 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87c37fe640 1 -- 172.21.15.33:0/446982035 learned_addr learned my addr 172.21.15.33:0/446982035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87c2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.762+0000 7f87c27fc640 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c37fe640 1 -- 172.21.15.33:0/446982035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 msgr2=0x560f9f5b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c37fe640 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c37fe640 1 -- 172.21.15.33:0/446982035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 msgr2=0x560f9f5b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c37fe640 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c37fe640 1 -- 172.21.15.33:0/446982035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9f5af450 con 0x560f9f5c25b0 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87c37fe640 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5b32e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f87bc006010 tx=0x7f87bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1054bd71957029b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87a97fa640 1 -- 172.21.15.33:0/446982035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc051070 con 0x560f9f5c25b0 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87a97fa640 1 -- 172.21.15.33:0/446982035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87bc036040 con 0x560f9f5c25b0 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.763+0000 7f87a97fa640 1 -- 172.21.15.33:0/446982035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0046b0 con 0x560f9f5c25b0 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 -- 172.21.15.33:0/446982035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 msgr2=0x560f9f5b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5b32e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f87bc006010 tx=0x7f87bc004ab0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 -- 172.21.15.33:0/446982035 shutdown_connections 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5b32e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/446982035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 -- 172.21.15.33:0/446982035 >> 172.21.15.33:0/446982035 conn(0x560f9f5af8b0 msgr2=0x560f9f5afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 -- 172.21.15.33:0/446982035 shutdown_connections 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 -- 172.21.15.33:0/446982035 wait complete. 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.764+0000 7f87cc7c3740 1 -- start start 2024-04-24T23:31:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 Processor -- start 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41a6580 con 0x560f9f5c25b0 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41a79f0 con 0x560f9f5bca50 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41a6c50 con 0x560f9f5bd440 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87cc7c3740 10 client.0 _ll_register_callbacks cb 0x7f87b809b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5ecaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60664/0 (socket says 172.21.15.33:60664) 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c37fe640 1 -- 172.21.15.33:0/129613469 learned_addr learned my addr 172.21.15.33:0/129613469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c27fc640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c2ffd640 1 -- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 msgr2=0x560f9f5ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c2ffd640 1 -- 172.21.15.33:0/129613469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 msgr2=0x560f9f5ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c2ffd640 1 -- 172.21.15.33:0/129613469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9f5ed2d0 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c37fe640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.765+0000 7f87c27fc640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87c2ffd640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5ee430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x560f9f5b5040 tx=0x7f87b4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1277fa6ebbd585af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87a2ffd640 1 -- 172.21.15.33:0/129613469 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b401a040 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87bc036040 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87a2ffd640 1 -- 172.21.15.33:0/129613469 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f87b4026040 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87a2ffd640 1 -- 172.21.15.33:0/129613469 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f87b403b050 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 10 client.28856 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f9f641e30 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87a2ffd640 1 -- 172.21.15.33:0/129613469 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87bc036040 con 0x560f9f5bca50 2024-04-24T23:31:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87a2ffd640 1 -- 172.21.15.33:0/129613469 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f87b401d030 con 0x560f9f5bca50 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 10 client.28856 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 10 client.28856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 20 client.28856 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f6427b0 0x560f9f644bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 4 client.28856 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.766+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f9f51ff40 con 0x560f9f6427b0 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87c27fc640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f6427b0 0x560f9f644bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87a2ffd640 10 client.28856 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87c27fc640 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f6427b0 0x560f9f644bd0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d175a42fbf21883 server_cookie=284039b580193b8d in_seq=0 out_seq=0 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87a2ffd640 1 -- 172.21.15.33:0/129613469 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f9f51ff40 con 0x560f9f6427b0 2024-04-24T23:31:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87a2ffd640 10 client.28856 handle_command_reply: tid=0 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87cc7c3740 1 client.28856 shutdown 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.767+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 msgr2=0x560f9f5ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5ee430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x560f9f5b5040 tx=0x7f87b4004730 comp rx=0 tx=0).stop 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 shutdown_connections 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560f9f6427b0 0x560f9f644bd0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f9f5c25b0 0x560f9f5ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560f9f5bd440 0x560f9f5ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 --2- 172.21.15.33:0/129613469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560f9f5bca50 0x560f9f5ee430 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87a2ffd640 0 client.28856 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 >> 172.21.15.33:0/129613469 conn(0x560f9f5af8b0 msgr2=0x560f9f5ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 shutdown_connections 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 1 -- 172.21.15.33:0/129613469 wait complete. 2024-04-24T23:31:03.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87cc7c3740 20 client.28856 trim_cache size 0 max 16384 2024-04-24T23:31:03.776 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:31:03.776 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:31:03.776 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:31:03.776 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87ac0a6fa0 msgr2=0x7f87ac0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87ac0a6fa0 0x7f87ac0a9460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f87b40095f0 tx=0x7f87b4011cb0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 msgr2=0x7f87c4197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 0x7f87c4197fc0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f87b800e9f0 tx=0x7f87b800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 shutdown_connections 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f87ac0a6fa0 0x7f87ac0a9460 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c4107460 0x7f87c419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87c4104b30 0x7f87c4197fc0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 --2- 172.21.15.33:0/1544108399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87c4102200 0x7f87c4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.768+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 >> 172.21.15.33:0/1544108399 conn(0x7f87c40fbbf0 msgr2=0x7f87c4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.769+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 shutdown_connections 2024-04-24T23:31:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.769+0000 7f87c99d5640 1 -- 172.21.15.33:0/1544108399 wait complete. 2024-04-24T23:31:03.779 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:31:03.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:31:03.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 133491 22.52 MB/sec execute 188 sec latency 159.676 ms 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 -- 172.21.15.33:0/3204085723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 msgr2=0x7f68ec101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 --2- 172.21.15.33:0/3204085723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec101bd0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009900 tx=0x7f68dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 -- 172.21.15.33:0/3204085723 shutdown_connections 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 --2- 172.21.15.33:0/3204085723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ec103360 0x7f68ec109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 --2- 172.21.15.33:0/3204085723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68ec1029a0 0x7f68ec102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 --2- 172.21.15.33:0/3204085723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec101bd0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 -- 172.21.15.33:0/3204085723 >> 172.21.15.33:0/3204085723 conn(0x7f68ec0fcd20 msgr2=0x7f68ec0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 -- 172.21.15.33:0/3204085723 shutdown_connections 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.915+0000 7f68f13c4640 1 -- 172.21.15.33:0/3204085723 wait complete. 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 Processor -- start 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 -- start start 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ec1029a0 0x7f68ec073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68ec103360 0x7f68ec074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec10d450 con 0x7f68ec1017b0 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec10d2d0 con 0x7f68ec103360 2024-04-24T23:31:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.916+0000 7f68f13c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec10d5d0 con 0x7f68ec1029a0 2024-04-24T23:31:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60682/0 (socket says 172.21.15.33:60682) 2024-04-24T23:31:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 -- 172.21.15.33:0/1624061234 learned_addr learned my addr 172.21.15.33:0/1624061234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68ea7fc640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ec1029a0 0x7f68ec073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eb7fe640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68ec103360 0x7f68ec074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 -- 172.21.15.33:0/1624061234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ec1029a0 msgr2=0x7f68ec073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ec1029a0 0x7f68ec073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 -- 172.21.15.33:0/1624061234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68ec103360 msgr2=0x7f68ec074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68ec103360 0x7f68ec074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.917+0000 7f68eaffd640 1 -- 172.21.15.33:0/1624061234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec19ba90 con 0x7f68ec1017b0 2024-04-24T23:31:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.918+0000 7f68eaffd640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec0754f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f68dc031440 tx=0x7f68dc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:03 smithi038 ceph-mon[32450]: pgmap v922: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 441 op/s 2024-04-24T23:31:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.920+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc050030 con 0x7f68ec1017b0 2024-04-24T23:31:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.920+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68dc003ce0 con 0x7f68ec1017b0 2024-04-24T23:31:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.920+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc035070 con 0x7f68ec1017b0 2024-04-24T23:31:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.920+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec0745b0 con 0x7f68ec1017b0 2024-04-24T23:31:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.920+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec19bd20 con 0x7f68ec1017b0 2024-04-24T23:31:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.922+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec0689b0 con 0x7f68ec1017b0 2024-04-24T23:31:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.924+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f68dc053030 con 0x7f68ec1017b0 2024-04-24T23:31:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.925+0000 7f68d3fff640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68d40a2ce0 0x7f68d40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.925+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68dc100b50 con 0x7f68ec1017b0 2024-04-24T23:31:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.925+0000 7f68ea7fc640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68d40a2ce0 0x7f68d40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:03.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:03 smithi117 ceph-mon[32529]: pgmap v922: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 441 op/s 2024-04-24T23:31:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.925+0000 7f68ea7fc640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68d40a2ce0 0x7f68d40a51a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f68ec075260 tx=0x7f68e000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:03.929+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f68dc05a030 con 0x7f68ec1017b0 2024-04-24T23:31:03.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:03 smithi033 ceph-mon[29201]: pgmap v922: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 441 op/s 2024-04-24T23:31:04.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.183+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68ec108bf0 con 0x7f68ec1017b0 2024-04-24T23:31:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.183+0000 7f68d3fff640 1 -- 172.21.15.33:0/1624061234 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f68dc0cd7a0 con 0x7f68ec1017b0 2024-04-24T23:31:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:31:04.183 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:31:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:31:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.185+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68d40a2ce0 msgr2=0x7f68d40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.185+0000 7f68f13c4640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68d40a2ce0 0x7f68d40a51a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f68ec075260 tx=0x7f68e000b410 comp rx=0 tx=0).stop 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.185+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 msgr2=0x7f68ec0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.185+0000 7f68f13c4640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec0754f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f68dc031440 tx=0x7f68dc005e50 comp rx=0 tx=0).stop 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 shutdown_connections 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f68d40a2ce0 0x7f68d40a51a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68ec103360 0x7f68ec074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68ec1029a0 0x7f68ec073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 --2- 172.21.15.33:0/1624061234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68ec1017b0 0x7f68ec0754f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 >> 172.21.15.33:0/1624061234 conn(0x7f68ec0fcd20 msgr2=0x7f68ec105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 shutdown_connections 2024-04-24T23:31:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.186+0000 7f68f13c4640 1 -- 172.21.15.33:0/1624061234 wait complete. 2024-04-24T23:31:04.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:31:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.311+0000 7f55c5cee640 1 -- 172.21.15.33:0/3881287780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 msgr2=0x7f55c0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.311+0000 7f55c5cee640 1 --2- 172.21.15.33:0/3881287780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0101cf0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b402a410 comp rx=0 tx=0).stop 2024-04-24T23:31:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.311+0000 7f55c5cee640 1 -- 172.21.15.33:0/3881287780 shutdown_connections 2024-04-24T23:31:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.311+0000 7f55c5cee640 1 --2- 172.21.15.33:0/3881287780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 0x7f55c0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.311+0000 7f55c5cee640 1 --2- 172.21.15.33:0/3881287780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0101cf0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.311+0000 7f55c5cee640 1 --2- 172.21.15.33:0/3881287780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 0x7f55c0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.312+0000 7f55c5cee640 1 -- 172.21.15.33:0/3881287780 >> 172.21.15.33:0/3881287780 conn(0x7f55c00fbbf0 msgr2=0x7f55c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.312+0000 7f55c5cee640 1 -- 172.21.15.33:0/3881287780 shutdown_connections 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.312+0000 7f55c5cee640 1 -- 172.21.15.33:0/3881287780 wait complete. 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.312+0000 7f55c5cee640 1 Processor -- start 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.312+0000 7f55c5cee640 1 -- start start 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55c5cee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 0x7f55c0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55c5cee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55c5cee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 0x7f55c019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55c5cee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55c010c300 con 0x7f55c0101870 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55c5cee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f55c010c180 con 0x7f55c0102230 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55c5cee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55c010c480 con 0x7f55c0100680 2024-04-24T23:31:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60708/0 (socket says 172.21.15.33:60708) 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 -- 172.21.15.33:0/2586569782 learned_addr learned my addr 172.21.15.33:0/2586569782 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 0x7f55c0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55bffff640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 0x7f55c019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 -- 172.21.15.33:0/2586569782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 msgr2=0x7f55c0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 0x7f55c0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 -- 172.21.15.33:0/2586569782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 msgr2=0x7f55c019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 0x7f55c019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 -- 172.21.15.33:0/2586569782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c019f6f0 con 0x7f55c0101870 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 0x7f55c0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55bffff640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 0x7f55c019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55beffd640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0198010 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009230 tx=0x7f55b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.313+0000 7f55bcff9640 1 -- 172.21.15.33:0/2586569782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b403f030 con 0x7f55c0101870 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.314+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c019f980 con 0x7f55c0101870 2024-04-24T23:31:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.314+0000 7f55bcff9640 1 -- 172.21.15.33:0/2586569782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b4003ce0 con 0x7f55c0101870 2024-04-24T23:31:04.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.314+0000 7f55bcff9640 1 -- 172.21.15.33:0/2586569782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4035070 con 0x7f55c0101870 2024-04-24T23:31:04.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.314+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c019fec0 con 0x7f55c0101870 2024-04-24T23:31:04.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55bcff9640 1 -- 172.21.15.33:0/2586569782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f55b4042030 con 0x7f55c0101870 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 Processor -- start 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 -- start start 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 0x55eee36d7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eee3632670 con 0x55eee36d3030 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eee35da970 con 0x55eee36d2640 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55c89f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eee362e630 con 0x55eee36d81a0 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 0x55eee36d7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 0x55eee36d7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60716/0 (socket says 172.21.15.33:60716) 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55bf7fe640 1 -- 172.21.15.33:0/2498669833 learned_addr learned my addr 172.21.15.33:0/2498669833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55bcff9640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f55a80a2b90 0x7f55a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.316+0000 7f55bffff640 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bcff9640 1 -- 172.21.15.33:0/2586569782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f55b40ef890 con 0x7f55c0101870 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bffff640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f55a80a2b90 0x7f55a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55beffd640 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bf7fe640 1 -- 172.21.15.33:0/2498669833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 msgr2=0x55eee36da630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36da630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bf7fe640 1 -- 172.21.15.33:0/2498669833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 msgr2=0x55eee36d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bf7fe640 1 -- 172.21.15.33:0/2498669833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee35fe400 con 0x55eee36d3030 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bffff640 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 0x55eee36d7b30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f55b0011820 tx=0x7f55b0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2044a9f2f6d0dbfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55bffff640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f55a80a2b90 0x7f55a80a5050 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55eee36d3890 tx=0x7f55b8004730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55a5ffb640 1 -- 172.21.15.33:0/2498669833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0011220 con 0x55eee36d3030 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55a5ffb640 1 -- 172.21.15.33:0/2498669833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b000f9a0 con 0x55eee36d3030 2024-04-24T23:31:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.317+0000 7f55a5ffb640 1 -- 172.21.15.33:0/2498669833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0004610 con 0x55eee36d3030 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 -- 172.21.15.33:0/2498669833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 msgr2=0x55eee36d7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 0x55eee36d7b30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f55b0011820 tx=0x7f55b0011ce0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 -- 172.21.15.33:0/2498669833 shutdown_connections 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36da630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d3030 0x55eee36d7b30 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2498669833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 -- 172.21.15.33:0/2498669833 >> 172.21.15.33:0/2498669833 conn(0x55eee36c98a0 msgr2=0x55eee36c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 -- 172.21.15.33:0/2498669833 shutdown_connections 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 -- 172.21.15.33:0/2498669833 wait complete. 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.318+0000 7f55c89f7740 1 -- start start 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 Processor -- start 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 0x55eee36dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eee36db150 con 0x55eee36d2640 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55eee35ee050 con 0x55eee36d3030 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eee36db2d0 con 0x55eee36d81a0 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55c89f7740 10 client.0 _ll_register_callbacks cb 0x55eee3345be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 0x55eee36dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 0x55eee36dc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60732/0 (socket says 172.21.15.33:60732) 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 -- 172.21.15.33:0/1528502035 learned_addr learned my addr 172.21.15.33:0/1528502035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55beffd640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bffff640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 -- 172.21.15.33:0/1528502035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 msgr2=0x55eee36e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 -- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 msgr2=0x55eee36dcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36dcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55bf7fe640 1 -- 172.21.15.33:0/1528502035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee35bfdc0 con 0x55eee36d2640 2024-04-24T23:31:04.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.319+0000 7f55beffd640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 0x55eee36dc740 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f55b0004090 tx=0x7f55b0003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d38ea2a52604ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f559effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f559effd640 1 -- 172.21.15.33:0/1528502035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0027020 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f559effd640 1 -- 172.21.15.33:0/1528502035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f55b0005580 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b000f9a0 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f559effd640 1 -- 172.21.15.33:0/1528502035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55b0032050 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 10 client.19392 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eee33463f0 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f559effd640 1 -- 172.21.15.33:0/1528502035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b000f9a0 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f559effd640 1 -- 172.21.15.33:0/1528502035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f55b0018070 con 0x55eee36d2640 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 10 client.19392 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 10 client.19392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 20 client.19392 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee36ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 4 client.19392 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:04.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.320+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee363fbb0 con 0x55eee36e83e0 2024-04-24T23:31:04.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.321+0000 7f55beffd640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee36ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.321+0000 7f559effd640 10 client.19392 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:04.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.321+0000 7f55beffd640 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee36ea880 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=240940daa6dd36ec server_cookie=d4f6166a7c9e6538 in_seq=0 out_seq=0 2024-04-24T23:31:04.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f559effd640 1 -- 172.21.15.33:0/1528502035 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eee363fbb0 con 0x55eee36e83e0 2024-04-24T23:31:04.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f559effd640 10 client.19392 handle_command_reply: tid=0 2024-04-24T23:31:04.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 client.19392 shutdown 2024-04-24T23:31:04.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 msgr2=0x55eee36dc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 0x55eee36dc740 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f55b0004090 tx=0x7f55b0003c60 comp rx=0 tx=0).stop 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 shutdown_connections 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee36ea880 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d81a0 0x55eee36e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f559effd640 0 client.19392 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 --2- 172.21.15.33:0/1528502035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d2640 0x55eee36dc740 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 >> 172.21.15.33:0/1528502035 conn(0x55eee36c98a0 msgr2=0x55eee36c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 shutdown_connections 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 1 -- 172.21.15.33:0/1528502035 wait complete. 2024-04-24T23:31:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.323+0000 7f55c89f7740 20 client.19392 trim_cache size 0 max 16384 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.338+0000 7f55c89f7740 1 Processor -- start 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.338+0000 7f55c89f7740 1 -- start start 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d2640 0x55eee36d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36dbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55c89f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01aa9d0 con 0x55eee36d81a0 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55c89f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01a82a0 con 0x55eee36d3030 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55c89f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01a60f0 con 0x55eee36d2640 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d2640 0x55eee36d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36d4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57200/0 (socket says 172.21.15.33:57200) 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 -- 172.21.15.33:0/2358596378 learned_addr learned my addr 172.21.15.33:0/2358596378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55bffff640 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36dbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 -- 172.21.15.33:0/2358596378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d2640 msgr2=0x55eee36d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d2640 0x55eee36d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 -- 172.21.15.33:0/2358596378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 msgr2=0x55eee36dbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36dbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.339+0000 7f55beffd640 1 -- 172.21.15.33:0/2358596378 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee34107d0 con 0x55eee36d3030 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.340+0000 7f55beffd640 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36d4510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55b40314e0 tx=0x7f55b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3298b76145fc9de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.340+0000 7f55a57fa640 1 -- 172.21.15.33:0/2358596378 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4049040 con 0x55eee36d3030 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.340+0000 7f55a57fa640 1 -- 172.21.15.33:0/2358596378 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b40bbe10 con 0x55eee36d3030 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.340+0000 7f55a57fa640 1 -- 172.21.15.33:0/2358596378 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b403bc10 con 0x55eee36d3030 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 -- 172.21.15.33:0/2358596378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 msgr2=0x55eee36d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36d4510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55b40314e0 tx=0x7f55b4009260 comp rx=0 tx=0).stop 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 -- 172.21.15.33:0/2358596378 shutdown_connections 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36dbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d3030 0x55eee36d4510 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 --2- 172.21.15.33:0/2358596378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d2640 0x55eee36d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 -- 172.21.15.33:0/2358596378 >> 172.21.15.33:0/2358596378 conn(0x55eee36c98a0 msgr2=0x55eee36d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 -- 172.21.15.33:0/2358596378 shutdown_connections 2024-04-24T23:31:04.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 -- 172.21.15.33:0/2358596378 wait complete. 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 -- start start 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.341+0000 7f55c89f7740 1 Processor -- start 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36e0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d3030 0x55eee36e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01a6760 con 0x55eee36d81a0 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01a7bd0 con 0x55eee36d2640 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01a6e30 con 0x55eee36d3030 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55c89f7740 10 client.0 _ll_register_callbacks cb 0x7f55b8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36e0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36e19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60752/0 (socket says 172.21.15.33:60752) 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36e0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57202/0 (socket says 172.21.15.33:57202) 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bffff640 1 -- 172.21.15.33:0/3372999218 learned_addr learned my addr 172.21.15.33:0/3372999218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55beffd640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d3030 0x55eee36e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 -- 172.21.15.33:0/3372999218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d3030 msgr2=0x55eee36e09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d3030 0x55eee36e09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 -- 172.21.15.33:0/3372999218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 msgr2=0x55eee36e19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36e19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 -- 172.21.15.33:0/3372999218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee36e1fa0 con 0x55eee36d2640 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bffff640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36e19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f55bf7fe640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36e0460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55b000a010 tx=0x7f55b0009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c83e61254dc16195 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.342+0000 7f559effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f559effd640 1 -- 172.21.15.33:0/3372999218 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0027070 con 0x55eee36d2640 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b40bbe10 con 0x55eee36d2640 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f559effd640 1 -- 172.21.15.33:0/3372999218 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f55b0011230 con 0x55eee36d2640 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f559effd640 1 -- 172.21.15.33:0/3372999218 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55b001e040 con 0x55eee36d2640 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f55c89f7740 10 client.28880 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eee36e32a0 con 0x55eee36d2640 2024-04-24T23:31:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.343+0000 7f559effd640 1 -- 172.21.15.33:0/3372999218 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b40bbe10 con 0x55eee36d2640 2024-04-24T23:31:04.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f559effd640 1 -- 172.21.15.33:0/3372999218 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f55b0018040 con 0x55eee36d2640 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55c89f7740 10 client.28880 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55c89f7740 10 client.28880 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55c89f7740 20 client.28880 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55c89f7740 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee3750e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55c89f7740 4 client.28880 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eee3639f00 con 0x55eee36e83e0 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55beffd640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee3750e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f559effd640 10 client.28880 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.344+0000 7f55beffd640 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee3750e90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=566e0e99cd4a65c5 server_cookie=564a96bd9cb3ea63 in_seq=0 out_seq=0 2024-04-24T23:31:04.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.348+0000 7f559effd640 1 -- 172.21.15.33:0/3372999218 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eee3639f00 con 0x55eee36e83e0 2024-04-24T23:31:04.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.348+0000 7f559effd640 10 client.28880 handle_command_reply: tid=0 2024-04-24T23:31:04.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.348+0000 7f55c89f7740 1 client.28880 shutdown 2024-04-24T23:31:04.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.348+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 msgr2=0x55eee36e0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.348+0000 7f55c89f7740 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36e0460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55b000a010 tx=0x7f55b0009f00 comp rx=0 tx=0).stop 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 shutdown_connections 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55eee36e83e0 0x55eee3750e90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eee36d81a0 0x55eee36e19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eee36d3030 0x55eee36e09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f559effd640 0 client.28880 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 --2- 172.21.15.33:0/3372999218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eee36d2640 0x55eee36e0460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 >> 172.21.15.33:0/3372999218 conn(0x55eee36c98a0 msgr2=0x55eee36d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 shutdown_connections 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 1 -- 172.21.15.33:0/3372999218 wait complete. 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c89f7740 20 client.28880 trim_cache size 0 max 16384 2024-04-24T23:31:04.349 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:31:04.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f55a80a2b90 msgr2=0x7f55a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c5cee640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f55a80a2b90 0x7f55a80a5050 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55eee36d3890 tx=0x7f55b8004730 comp rx=0 tx=0).stop 2024-04-24T23:31:04.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 msgr2=0x7f55c0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.349+0000 7f55c5cee640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0198010 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009230 tx=0x7f55b40042f0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 shutdown_connections 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f55a80a2b90 0x7f55a80a5050 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55c0102230 0x7f55c019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55c0101870 0x7f55c0198010 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 --2- 172.21.15.33:0/2586569782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c0100680 0x7f55c0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 >> 172.21.15.33:0/2586569782 conn(0x7f55c00fbbf0 msgr2=0x7f55c00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 shutdown_connections 2024-04-24T23:31:04.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.350+0000 7f55c5cee640 1 -- 172.21.15.33:0/2586569782 wait complete. 2024-04-24T23:31:04.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 -- 172.21.15.33:0/3925802311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a280ffa50 msgr2=0x7f3a280ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 --2- 172.21.15.33:0/3925802311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a280ffa50 0x7f3a280ffe70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009930 tx=0x7f3a1802a420 comp rx=0 tx=0).stop 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 -- 172.21.15.33:0/3925802311 shutdown_connections 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 --2- 172.21.15.33:0/3925802311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a281033e0 0x7f3a28109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 --2- 172.21.15.33:0/3925802311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a28102a20 0x7f3a28102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 --2- 172.21.15.33:0/3925802311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a280ffa50 0x7f3a280ffe70 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 -- 172.21.15.33:0/3925802311 >> 172.21.15.33:0/3925802311 conn(0x7f3a280fb380 msgr2=0x7f3a280fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 -- 172.21.15.33:0/3925802311 shutdown_connections 2024-04-24T23:31:04.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.487+0000 7f3a2e064640 1 -- 172.21.15.33:0/3925802311 wait complete. 2024-04-24T23:31:04.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 Processor -- start 2024-04-24T23:31:04.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 -- start start 2024-04-24T23:31:04.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280ffa50 0x7f3a28197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102a20 0x7f3a28198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 0x7f3a2819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2810d420 con 0x7f3a281033e0 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2810d2a0 con 0x7f3a28102a20 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a2e064640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2810d5a0 con 0x7f3a280ffa50 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 0x7f3a2819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102a20 0x7f3a28198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 0x7f3a2819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60774/0 (socket says 172.21.15.33:60774) 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280ffa50 0x7f3a28197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.488+0000 7f3a27fff640 1 -- 172.21.15.33:0/117552713 learned_addr learned my addr 172.21.15.33:0/117552713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a27fff640 1 -- 172.21.15.33:0/117552713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280ffa50 msgr2=0x7f3a28197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a27fff640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280ffa50 0x7f3a28197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a27fff640 1 -- 172.21.15.33:0/117552713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102a20 msgr2=0x7f3a28198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a27fff640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102a20 0x7f3a28198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a27fff640 1 -- 172.21.15.33:0/117552713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2819f760 con 0x7f3a281033e0 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a27fff640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 0x7f3a2819f0f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000eb10 tx=0x7f3a2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a24ff9640 1 -- 172.21.15.33:0/117552713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20004b20 con 0x7f3a281033e0 2024-04-24T23:31:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2819fa50 con 0x7f3a281033e0 2024-04-24T23:31:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a24ff9640 1 -- 172.21.15.33:0/117552713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a20004cc0 con 0x7f3a281033e0 2024-04-24T23:31:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a24ff9640 1 -- 172.21.15.33:0/117552713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20010760 con 0x7f3a281033e0 2024-04-24T23:31:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.489+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2819ffa0 con 0x7f3a281033e0 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.491+0000 7f3a24ff9640 1 -- 172.21.15.33:0/117552713 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a20005020 con 0x7f3a281033e0 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 Processor -- start 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 -- start start 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564e24907670 con 0x564e249a7640 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e248af970 con 0x564e249a8030 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a30d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e24903630 con 0x564e249ad1a0 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a24ff9640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a100a2b90 0x7f3a100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60784/0 (socket says 172.21.15.33:60784) 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3307572427 learned_addr learned my addr 172.21.15.33:0/3307572427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a27fff640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a100a2b90 0x7f3a100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.492+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a24ff9640 1 -- 172.21.15.33:0/117552713 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a200cf980 con 0x7f3a281033e0 2024-04-24T23:31:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3307572427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 msgr2=0x564e249af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3307572427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 msgr2=0x564e249acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3307572427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e248d3400 con 0x564e249a7640 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a7a60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c013630 tx=0x7f3a1c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3489fb464b786f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a0dffb640 1 -- 172.21.15.33:0/3307572427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004050 con 0x564e249a7640 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a27fff640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a100a2b90 0x7f3a100a5050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3a200a4590 tx=0x7f3a200c69e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a0dffb640 1 -- 172.21.15.33:0/3307572427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c0041f0 con 0x564e249a7640 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.493+0000 7f3a0dffb640 1 -- 172.21.15.33:0/3307572427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00f870 con 0x564e249a7640 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 -- 172.21.15.33:0/3307572427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 msgr2=0x564e249a7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a7a60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c013630 tx=0x7f3a1c009780 comp rx=0 tx=0).stop 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 -- 172.21.15.33:0/3307572427 shutdown_connections 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3307572427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a7a60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 -- 172.21.15.33:0/3307572427 >> 172.21.15.33:0/3307572427 conn(0x564e2499e8a0 msgr2=0x564e2499eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 -- 172.21.15.33:0/3307572427 shutdown_connections 2024-04-24T23:31:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 -- 172.21.15.33:0/3307572427 wait complete. 2024-04-24T23:31:04.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.494+0000 7f3a30d71740 1 -- start start 2024-04-24T23:31:04.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 Processor -- start 2024-04-24T23:31:04.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a7640 0x564e249b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 0x564e249b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564e249b0150 con 0x564e249a8030 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e248c3050 con 0x564e249ad1a0 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e249b02d0 con 0x564e249a7640 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a30d71740 10 client.0 _ll_register_callbacks cb 0x564e2461abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 0x564e249b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 0x564e249b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60800/0 (socket says 172.21.15.33:60800) 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a7640 0x564e249b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a277fe640 1 -- 172.21.15.33:0/2405025044 learned_addr learned my addr 172.21.15.33:0/2405025044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a277fe640 1 -- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a7640 msgr2=0x564e249b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.495+0000 7f3a277fe640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a7640 0x564e249b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a277fe640 1 -- 172.21.15.33:0/2405025044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 msgr2=0x564e249b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a277fe640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a277fe640 1 -- 172.21.15.33:0/2405025044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e24894dc0 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a277fe640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 0x564e249b1cd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009900 tx=0x7f3a18032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad36e4f42909c965 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a06ffd640 1 -- 172.21.15.33:0/2405025044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18035040 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a1c0041f0 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a06ffd640 1 -- 172.21.15.33:0/2405025044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a180318b0 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a06ffd640 1 -- 172.21.15.33:0/2405025044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3a18004040 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a30d71740 10 client.19410 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564e2461b3f0 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.496+0000 7f3a06ffd640 1 -- 172.21.15.33:0/2405025044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c0041f0 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a06ffd640 1 -- 172.21.15.33:0/2405025044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3a18004880 con 0x564e249a8030 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a30d71740 10 client.19410 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a30d71740 10 client.19410 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a30d71740 20 client.19410 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a30d71740 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e249bd610 0x564e249bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a30d71740 4 client.19410 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e24914bb0 con 0x564e249bd610 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e249bd610 0x564e249bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e249bd610 0x564e249bfa20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2cdea5dae76efd6 server_cookie=34280a5410dcb114 in_seq=0 out_seq=0 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.497+0000 7f3a06ffd640 10 client.19410 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.498+0000 7f3a06ffd640 1 -- 172.21.15.33:0/2405025044 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e24914bb0 con 0x564e249bd610 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.498+0000 7f3a06ffd640 10 client.19410 handle_command_reply: tid=0 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.498+0000 7f3a30d71740 1 client.19410 shutdown 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 msgr2=0x564e249b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 0x564e249b1cd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009900 tx=0x7f3a18032e20 comp rx=0 tx=0).stop 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 shutdown_connections 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e249bd610 0x564e249bfa20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a06ffd640 0 client.19410 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a8030 0x564e249b1cd0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 --2- 172.21.15.33:0/2405025044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a7640 0x564e249b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 >> 172.21.15.33:0/2405025044 conn(0x564e2499e8a0 msgr2=0x564e2499db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 shutdown_connections 2024-04-24T23:31:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 1 -- 172.21.15.33:0/2405025044 wait complete. 2024-04-24T23:31:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.499+0000 7f3a30d71740 20 client.19410 trim_cache size 0 max 16384 2024-04-24T23:31:04.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.514+0000 7f3a30d71740 1 Processor -- start 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.514+0000 7f3a30d71740 1 -- start start 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.514+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.514+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.514+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.514+0000 7f3a30d71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281aaa60 con 0x564e249a7640 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a30d71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281a8330 con 0x564e249a8030 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a30d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281a6180 con 0x564e249ad1a0 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60802/0 (socket says 172.21.15.33:60802) 2024-04-24T23:31:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 -- 172.21.15.33:0/1460867527 learned_addr learned my addr 172.21.15.33:0/1460867527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a277fe640 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 -- 172.21.15.33:0/1460867527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 msgr2=0x564e249b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 -- 172.21.15.33:0/1460867527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 msgr2=0x564e249a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 -- 172.21.15.33:0/1460867527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e246e57d0 con 0x564e249a7640 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a1c90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005e00 tx=0x7f3a1c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53954a278e1b0240 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.515+0000 7f3a0d7fa640 1 -- 172.21.15.33:0/1460867527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0052a0 con 0x564e249a7640 2024-04-24T23:31:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a0d7fa640 1 -- 172.21.15.33:0/1460867527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c005440 con 0x564e249a7640 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a0d7fa640 1 -- 172.21.15.33:0/1460867527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0314c0 con 0x564e249a7640 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 -- 172.21.15.33:0/1460867527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 msgr2=0x564e249a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a1c90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005e00 tx=0x7f3a1c009780 comp rx=0 tx=0).stop 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 -- 172.21.15.33:0/1460867527 shutdown_connections 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249ad1a0 0x564e249b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249a8030 0x564e249a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 --2- 172.21.15.33:0/1460867527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249a1c90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 -- 172.21.15.33:0/1460867527 >> 172.21.15.33:0/1460867527 conn(0x564e2499e8a0 msgr2=0x564e249a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 -- 172.21.15.33:0/1460867527 shutdown_connections 2024-04-24T23:31:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.516+0000 7f3a30d71740 1 -- 172.21.15.33:0/1460867527 wait complete. 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 -- start start 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 Processor -- start 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a8030 0x564e249b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281a67f0 con 0x564e249a7640 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281a7c60 con 0x564e249ad1a0 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281a6ec0 con 0x564e249a8030 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.517+0000 7f3a30d71740 10 client.0 _ll_register_callbacks cb 0x7f3a18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a8030 0x564e249b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a8030 0x564e249b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35460/0 (socket says 172.21.15.33:35460) 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a277fe640 1 -- 172.21.15.33:0/3580909829 learned_addr learned my addr 172.21.15.33:0/3580909829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60812/0 (socket says 172.21.15.33:60812) 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a8030 msgr2=0x564e249b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a8030 0x564e249b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3580909829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 msgr2=0x564e249b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 -- 172.21.15.33:0/3580909829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e249b7d70 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a26ffd640 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249b6230 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c034040 tx=0x7f3a1c005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470f697eed71cb33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a06ffd640 1 -- 172.21.15.33:0/3580909829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c031450 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a06ffd640 1 -- 172.21.15.33:0/3580909829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a1c0315f0 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a1c005440 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a06ffd640 1 -- 172.21.15.33:0/3580909829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3a1c0317f0 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a30d71740 10 client.19416 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.518+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564e249b9070 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a06ffd640 1 -- 172.21.15.33:0/3580909829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c005440 con 0x564e249a7640 2024-04-24T23:31:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a06ffd640 1 -- 172.21.15.33:0/3580909829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3a1c03b070 con 0x564e249a7640 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a30d71740 10 client.19416 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a30d71740 10 client.19416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a30d71740 20 client.19416 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e24a24120 0x564e24a26540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a30d71740 4 client.19416 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e249b1580 con 0x564e24a24120 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a277fe640 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e24a24120 0x564e24a26540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a277fe640 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e24a24120 0x564e24a26540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5848cad2c4d5fb26 server_cookie=870dbd7591ee3a8a in_seq=0 out_seq=0 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.519+0000 7f3a06ffd640 10 client.19416 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a06ffd640 1 -- 172.21.15.33:0/3580909829 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e249b1580 con 0x564e24a24120 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a06ffd640 10 client.19416 handle_command_reply: tid=0 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 client.19416 shutdown 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 msgr2=0x564e249b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249b6230 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c034040 tx=0x7f3a1c005630 comp rx=0 tx=0).stop 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 shutdown_connections 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x564e24a24120 0x564e24a26540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e249ad1a0 0x564e249b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e249a8030 0x564e249b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a06ffd640 0 client.19416 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:04.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 --2- 172.21.15.33:0/3580909829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e249a7640 0x564e249b6230 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 >> 172.21.15.33:0/3580909829 conn(0x564e2499e8a0 msgr2=0x564e249a1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 shutdown_connections 2024-04-24T23:31:04.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 1 -- 172.21.15.33:0/3580909829 wait complete. 2024-04-24T23:31:04.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a30d71740 20 client.19416 trim_cache size 0 max 16384 2024-04-24T23:31:04.531 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:31:04.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a100a2b90 msgr2=0x7f3a100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a2e064640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a100a2b90 0x7f3a100a5050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3a200a4590 tx=0x7f3a200c69e0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 msgr2=0x7f3a2819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.520+0000 7f3a2e064640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 0x7f3a2819f0f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000eb10 tx=0x7f3a2000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 shutdown_connections 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a100a2b90 0x7f3a100a5050 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a281033e0 0x7f3a2819f0f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102a20 0x7f3a28198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 --2- 172.21.15.33:0/117552713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a280ffa50 0x7f3a28197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 >> 172.21.15.33:0/117552713 conn(0x7f3a280fb380 msgr2=0x7f3a28105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 shutdown_connections 2024-04-24T23:31:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.521+0000 7f3a2e064640 1 -- 172.21.15.33:0/117552713 wait complete. 2024-04-24T23:31:04.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:31:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.650+0000 7f2864b09640 1 -- 172.21.15.33:0/2767532317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860102230 msgr2=0x7f2860108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.650+0000 7f2864b09640 1 --2- 172.21.15.33:0/2767532317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860102230 0x7f2860108ac0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2858009ca0 tx=0x7f285802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 -- 172.21.15.33:0/2767532317 shutdown_connections 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 --2- 172.21.15.33:0/2767532317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860102230 0x7f2860108ac0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 --2- 172.21.15.33:0/2767532317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2860101870 0x7f2860101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 --2- 172.21.15.33:0/2767532317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860100680 0x7f2860100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 -- 172.21.15.33:0/2767532317 >> 172.21.15.33:0/2767532317 conn(0x7f28600fbbf0 msgr2=0x7f28600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 -- 172.21.15.33:0/2767532317 shutdown_connections 2024-04-24T23:31:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.651+0000 7f2864b09640 1 -- 172.21.15.33:0/2767532317 wait complete. 2024-04-24T23:31:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 Processor -- start 2024-04-24T23:31:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 -- start start 2024-04-24T23:31:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 0x7f2860199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2860101870 0x7f286019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860102230 0x7f28601a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f286010c300 con 0x7f2860100680 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f286010c180 con 0x7f2860101870 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f2864b09640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f286010c480 con 0x7f2860102230 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 0x7f2860199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 0x7f2860199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60822/0 (socket says 172.21.15.33:60822) 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 -- 172.21.15.33:0/3828350407 learned_addr learned my addr 172.21.15.33:0/3828350407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285dd9b640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2860101870 0x7f286019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285ed9d640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860102230 0x7f28601a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 -- 172.21.15.33:0/3828350407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860102230 msgr2=0x7f28601a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860102230 0x7f28601a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 -- 172.21.15.33:0/3828350407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2860101870 msgr2=0x7f286019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2860101870 0x7f286019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 -- 172.21.15.33:0/3828350407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a18b0 con 0x7f2860100680 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.652+0000 7f285e59c640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 0x7f2860199c70 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f285000e9c0 tx=0x7f285000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.653+0000 7f284f7fe640 1 -- 172.21.15.33:0/3828350407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285000cd80 con 0x7f2860100680 2024-04-24T23:31:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.653+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a1ba0 con 0x7f2860100680 2024-04-24T23:31:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.653+0000 7f284f7fe640 1 -- 172.21.15.33:0/3828350407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28500035f0 con 0x7f2860100680 2024-04-24T23:31:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.653+0000 7f284f7fe640 1 -- 172.21.15.33:0/3828350407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850010690 con 0x7f2860100680 2024-04-24T23:31:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.653+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a20a0 con 0x7f2860100680 2024-04-24T23:31:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.655+0000 7f284f7fe640 1 -- 172.21.15.33:0/3828350407 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f28500037a0 con 0x7f2860100680 2024-04-24T23:31:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.655+0000 7f286780e740 1 Processor -- start 2024-04-24T23:31:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.655+0000 7f286780e740 1 -- start start 2024-04-24T23:31:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.655+0000 7f286780e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.655+0000 7f286780e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ecc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f286780e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ecf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f286780e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559766e2b670 con 0x559766ec7c20 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f286780e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559766dd3970 con 0x559766ec7230 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f286780e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559766e27630 con 0x559766eccd90 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ecc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ecf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ecc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60824/0 (socket says 172.21.15.33:60824) 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285e59c640 1 -- 172.21.15.33:0/360101159 learned_addr learned my addr 172.21.15.33:0/360101159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285ed9d640 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f284f7fe640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f28480a2b90 0x7f28480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 -- 172.21.15.33:0/360101159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 msgr2=0x559766ec7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 -- 172.21.15.33:0/360101159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 msgr2=0x559766ecc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ecc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 -- 172.21.15.33:0/360101159 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559766df7400 con 0x559766eccd90 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285ed9d640 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285e59c640 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ecc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285ed9d640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f28480a2b90 0x7f28480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f284f7fe640 1 -- 172.21.15.33:0/3828350407 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28500cf720 con 0x7f2860100680 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.656+0000 7f285dd9b640 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ecf220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2854009bf0 tx=0x7f2854011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a9d8d16d0c9b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.657+0000 7f284cff9640 1 -- 172.21.15.33:0/360101159 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854004050 con 0x559766eccd90 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.657+0000 7f285ed9d640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f28480a2b90 0x7f28480a5050 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x559766ec8480 tx=0x7f285803f040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.657+0000 7f284cff9640 1 -- 172.21.15.33:0/360101159 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2854005500 con 0x559766eccd90 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.657+0000 7f284cff9640 1 -- 172.21.15.33:0/360101159 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854004e00 con 0x559766eccd90 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- 172.21.15.33:0/360101159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 msgr2=0x559766ecf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ecf220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2854009bf0 tx=0x7f2854011810 comp rx=0 tx=0).stop 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- 172.21.15.33:0/360101159 shutdown_connections 2024-04-24T23:31:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ecf220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ecc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- 172.21.15.33:0/360101159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- 172.21.15.33:0/360101159 >> 172.21.15.33:0/360101159 conn(0x559766ec28a0 msgr2=0x559766ec2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- 172.21.15.33:0/360101159 shutdown_connections 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- 172.21.15.33:0/360101159 wait complete. 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- start start 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 Processor -- start 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559766ecfd40 con 0x559766ec7230 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559766de7050 con 0x559766eccd90 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559766ecfec0 con 0x559766ec7c20 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.658+0000 7f286780e740 10 client.0 _ll_register_callbacks cb 0x559766b3ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60838/0 (socket says 172.21.15.33:60838) 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 -- 172.21.15.33:0/1531041322 learned_addr learned my addr 172.21.15.33:0/1531041322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285ed9d640 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 -- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 msgr2=0x559766ed1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 -- 172.21.15.33:0/1531041322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 msgr2=0x559766ed8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 -- 172.21.15.33:0/1531041322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559766db8dc0 con 0x559766ec7230 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f285e59c640 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed1300 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f28500ccee0 tx=0x7f28500096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e77bff2b5f12d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f2845ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f2845ffb640 1 -- 172.21.15.33:0/1531041322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285009b9c0 con 0x559766ec7230 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f2845ffb640 1 -- 172.21.15.33:0/1531041322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28500cf400 con 0x559766ec7230 2024-04-24T23:31:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2854005500 con 0x559766ec7230 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f2845ffb640 1 -- 172.21.15.33:0/1531041322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f285009b550 con 0x559766ec7230 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f286780e740 10 client.19428 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.659+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559766b3f3f0 con 0x559766ec7230 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f2845ffb640 1 -- 172.21.15.33:0/1531041322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2854005500 con 0x559766ec7230 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f2845ffb640 1 -- 172.21.15.33:0/1531041322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f28500cfbd0 con 0x559766ec7230 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f286780e740 10 client.19428 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f286780e740 10 client.19428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f286780e740 20 client.19428 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f286780e740 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766edd260 0x559766edf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f286780e740 4 client.19428 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559766e38bb0 con 0x559766edd260 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f285dd9b640 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766edd260 0x559766edf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f285dd9b640 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766edd260 0x559766edf6f0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f603ac89fe5e4c60 server_cookie=9516f723d91d40e1 in_seq=0 out_seq=0 2024-04-24T23:31:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.660+0000 7f2845ffb640 10 client.19428 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:04.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.661+0000 7f2845ffb640 1 -- 172.21.15.33:0/1531041322 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559766e38bb0 con 0x559766edd260 2024-04-24T23:31:04.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.661+0000 7f2845ffb640 10 client.19428 handle_command_reply: tid=0 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.661+0000 7f286780e740 1 client.19428 shutdown 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.661+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 msgr2=0x559766ed1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.661+0000 7f286780e740 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed1300 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f28500ccee0 tx=0x7f28500096e0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 shutdown_connections 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766edd260 0x559766edf6f0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f2845ffb640 0 client.19428 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 --2- 172.21.15.33:0/1531041322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed1300 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 >> 172.21.15.33:0/1531041322 conn(0x559766ec28a0 msgr2=0x559766ec1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 shutdown_connections 2024-04-24T23:31:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 1 -- 172.21.15.33:0/1531041322 wait complete. 2024-04-24T23:31:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.662+0000 7f286780e740 20 client.19428 trim_cache size 0 max 16384 2024-04-24T23:31:04.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.706+0000 7f286780e740 1 Processor -- start 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 -- start start 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ed8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ec89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28601aedd0 con 0x559766ec7c20 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ac6a0 con 0x559766ec7230 2024-04-24T23:31:04.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.710+0000 7f286780e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28601aa4f0 con 0x559766eccd90 2024-04-24T23:31:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.714+0000 7f285ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ec89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.715+0000 7f285ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ec89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35518/0 (socket says 172.21.15.33:35518) 2024-04-24T23:31:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.715+0000 7f285ed9d640 1 -- 172.21.15.33:0/1247366792 learned_addr learned my addr 172.21.15.33:0/1247366792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.715+0000 7f285e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.715+0000 7f285dd9b640 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ed8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.717+0000 7f285ed9d640 1 -- 172.21.15.33:0/1247366792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 msgr2=0x559766ec84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.717+0000 7f285ed9d640 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.718+0000 7f285ed9d640 1 -- 172.21.15.33:0/1247366792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 msgr2=0x559766ed8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.718+0000 7f285ed9d640 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ed8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.718+0000 7f285ed9d640 1 -- 172.21.15.33:0/1247366792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559766c097d0 con 0x559766eccd90 2024-04-24T23:31:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.721+0000 7f285ed9d640 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ec89f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2858009ca0 tx=0x7f2858043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892ea8cd07e00a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.725+0000 7f2847fff640 1 -- 172.21.15.33:0/1247366792 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285802fae0 con 0x559766eccd90 2024-04-24T23:31:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.725+0000 7f2847fff640 1 -- 172.21.15.33:0/1247366792 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2858036040 con 0x559766eccd90 2024-04-24T23:31:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.725+0000 7f2847fff640 1 -- 172.21.15.33:0/1247366792 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858046680 con 0x559766eccd90 2024-04-24T23:31:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.727+0000 7f286780e740 1 -- 172.21.15.33:0/1247366792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 msgr2=0x559766ec89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.727+0000 7f286780e740 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ec89f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2858009ca0 tx=0x7f2858043000 comp rx=0 tx=0).stop 2024-04-24T23:31:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.731+0000 7f286780e740 1 -- 172.21.15.33:0/1247366792 shutdown_connections 2024-04-24T23:31:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.731+0000 7f286780e740 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766eccd90 0x559766ec89f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.731+0000 7f286780e740 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7c20 0x559766ed8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.731+0000 7f286780e740 1 --2- 172.21.15.33:0/1247366792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766ec7230 0x559766ec84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.731+0000 7f286780e740 1 -- 172.21.15.33:0/1247366792 >> 172.21.15.33:0/1247366792 conn(0x559766ec28a0 msgr2=0x559766ec1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.738+0000 7f286780e740 1 -- 172.21.15.33:0/1247366792 shutdown_connections 2024-04-24T23:31:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.741+0000 7f286780e740 1 -- 172.21.15.33:0/1247366792 wait complete. 2024-04-24T23:31:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.742+0000 7f286780e740 1 -- start start 2024-04-24T23:31:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.742+0000 7f286780e740 1 Processor -- start 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f286780e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f286780e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f286780e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f286780e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28601aab60 con 0x559766ec7230 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f286780e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28601abfd0 con 0x559766eccd90 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f286780e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ab230 con 0x559766ec7c20 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f285ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f285ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57314/0 (socket says 172.21.15.33:57314) 2024-04-24T23:31:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.743+0000 7f285ed9d640 1 -- 172.21.15.33:0/2691515033 learned_addr learned my addr 172.21.15.33:0/2691515033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.744+0000 7f286780e740 10 client.0 _ll_register_callbacks cb 0x7f28500d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.744+0000 7f285ed9d640 1 -- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 msgr2=0x559766ed6a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:31:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.744+0000 7f285ed9d640 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed6a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.744+0000 7f285ed9d640 1 -- 172.21.15.33:0/2691515033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 msgr2=0x559766ed64f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:31:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.744+0000 7f285ed9d640 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed64f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.744+0000 7f285ed9d640 1 -- 172.21.15.33:0/2691515033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559766ed8030 con 0x559766eccd90 2024-04-24T23:31:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.747+0000 7f285ed9d640 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed7a40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2858053040 tx=0x7f285802fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ade24f811234d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.747+0000 7f2845ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:31:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.747+0000 7f2845ffb640 1 -- 172.21.15.33:0/2691515033 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858046650 con 0x559766eccd90 2024-04-24T23:31:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.747+0000 7f2845ffb640 1 -- 172.21.15.33:0/2691515033 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28580467f0 con 0x559766eccd90 2024-04-24T23:31:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.750+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2858036040 con 0x559766eccd90 2024-04-24T23:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.754+0000 7f2845ffb640 1 -- 172.21.15.33:0/2691515033 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2858046a70 con 0x559766eccd90 2024-04-24T23:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.755+0000 7f286780e740 10 client.28913 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.755+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559766f43e30 con 0x559766eccd90 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.759+0000 7f2845ffb640 1 -- 172.21.15.33:0/2691515033 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2858036040 con 0x559766eccd90 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.759+0000 7f2845ffb640 1 -- 172.21.15.33:0/2691515033 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2858059a00 con 0x559766eccd90 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.760+0000 7f286780e740 10 client.28913 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.760+0000 7f286780e740 10 client.28913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.760+0000 7f286780e740 20 client.28913 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.760+0000 7f286780e740 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766f446e0 0x559766f46b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.760+0000 7f286780e740 4 client.28913 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.760+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559766e32f00 con 0x559766f446e0 2024-04-24T23:31:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.768+0000 7f285e59c640 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766f446e0 0x559766f46b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:04.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.784+0000 7f285e59c640 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766f446e0 0x559766f46b00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96e1e5cf3622fdf server_cookie=10417f303704a633 in_seq=0 out_seq=0 2024-04-24T23:31:04.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.785+0000 7f2845ffb640 10 client.28913 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.788+0000 7f2845ffb640 1 -- 172.21.15.33:0/2691515033 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559766e32f00 con 0x559766f446e0 2024-04-24T23:31:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.788+0000 7f2845ffb640 10 client.28913 handle_command_reply: tid=0 2024-04-24T23:31:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.789+0000 7f286780e740 1 client.28913 shutdown 2024-04-24T23:31:04.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.793+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 msgr2=0x559766ed7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.793+0000 7f286780e740 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed7a40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2858053040 tx=0x7f285802fd10 comp rx=0 tx=0).stop 2024-04-24T23:31:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.795+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 shutdown_connections 2024-04-24T23:31:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.795+0000 7f286780e740 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559766f446e0 0x559766f46b00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.796+0000 7f286780e740 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559766eccd90 0x559766ed7a40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.796+0000 7f286780e740 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559766ec7c20 0x559766ed6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.796+0000 7f286780e740 1 --2- 172.21.15.33:0/2691515033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559766ec7230 0x559766ed64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.796+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 >> 172.21.15.33:0/2691515033 conn(0x559766ec28a0 msgr2=0x559766ec57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.796+0000 7f2845ffb640 0 client.28913 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:04.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.799+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 shutdown_connections 2024-04-24T23:31:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.803+0000 7f286780e740 1 -- 172.21.15.33:0/2691515033 wait complete. 2024-04-24T23:31:04.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.806+0000 7f286780e740 20 client.28913 trim_cache size 0 max 16384 2024-04-24T23:31:04.810 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:31:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.815+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f28480a2b90 msgr2=0x7f28480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.815+0000 7f2864b09640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f28480a2b90 0x7f28480a5050 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x559766ec8480 tx=0x7f285803f040 comp rx=0 tx=0).stop 2024-04-24T23:31:04.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.815+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 msgr2=0x7f2860199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:04.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.815+0000 7f2864b09640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 0x7f2860199c70 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f285000e9c0 tx=0x7f285000ee80 comp rx=0 tx=0).stop 2024-04-24T23:31:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.817+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 shutdown_connections 2024-04-24T23:31:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.817+0000 7f2864b09640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f28480a2b90 0x7f28480a5050 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.817+0000 7f2864b09640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860102230 0x7f28601a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.817+0000 7f2864b09640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2860101870 0x7f286019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.817+0000 7f2864b09640 1 --2- 172.21.15.33:0/3828350407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860100680 0x7f2860199c70 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.817+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 >> 172.21.15.33:0/3828350407 conn(0x7f28600fbbf0 msgr2=0x7f28600fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.818+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 shutdown_connections 2024-04-24T23:31:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:04.821+0000 7f2864b09640 1 -- 172.21.15.33:0/3828350407 wait complete. 2024-04-24T23:31:04.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:04 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1624061234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:04.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 133950 22.57 MB/sec execute 189 sec latency 226.624 ms 2024-04-24T23:31:04.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:04 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1624061234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:04.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:04 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1624061234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:05.844 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:31:05.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:31:05.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:31:05.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:05 smithi033 ceph-mon[29201]: pgmap v923: 129 pgs: 129 active+clean; 566 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 395 op/s 2024-04-24T23:31:05.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 134508 22.58 MB/sec execute 190 sec latency 265.739 ms 2024-04-24T23:31:05.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:05 smithi038 ceph-mon[32450]: pgmap v923: 129 pgs: 129 active+clean; 566 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 395 op/s 2024-04-24T23:31:05.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:05 smithi117 ceph-mon[32529]: pgmap v923: 129 pgs: 129 active+clean; 566 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 395 op/s 2024-04-24T23:31:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.987+0000 7fedef7fe640 1 -- 172.21.15.33:0/2992904805 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede802f340 con 0x7fedf01097c0 2024-04-24T23:31:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 -- 172.21.15.33:0/2992904805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 msgr2=0x7fedf010e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/2992904805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 0x7fedf010e080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fede8009ca0 tx=0x7fede802a8f0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 -- 172.21.15.33:0/2992904805 shutdown_connections 2024-04-24T23:31:05.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/2992904805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 0x7fedf010e080 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/2992904805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 0x7fedf0109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/2992904805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.988+0000 7fedf5e3c640 1 -- 172.21.15.33:0/2992904805 >> 172.21.15.33:0/2992904805 conn(0x7fedf006e650 msgr2=0x7fedf0070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:05.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.991+0000 7fedf5e3c640 1 -- 172.21.15.33:0/2992904805 shutdown_connections 2024-04-24T23:31:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.991+0000 7fedf5e3c640 1 -- 172.21.15.33:0/2992904805 wait complete. 2024-04-24T23:31:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 Processor -- start 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 -- start start 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0116df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 0x7fedf0117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 0x7fedf01b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedf011c180 con 0x7fedf0108e20 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedf011c000 con 0x7fedf01097c0 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf5e3c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedf011c300 con 0x7fedf0072b80 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf4e3a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0116df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf563b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 0x7fedf01b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf4e3a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0116df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33400/0 (socket says 172.21.15.33:33400) 2024-04-24T23:31:05.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.992+0000 7fedf4e3a640 1 -- 172.21.15.33:0/936830116 learned_addr learned my addr 172.21.15.33:0/936830116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:05.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedeffff640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 0x7fedf0117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf4e3a640 1 -- 172.21.15.33:0/936830116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 msgr2=0x7fedf01b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf4e3a640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 0x7fedf01b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf4e3a640 1 -- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 msgr2=0x7fedf0117330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf4e3a640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 0x7fedf0117330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf4e3a640 1 -- 172.21.15.33:0/936830116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf01179d0 con 0x7fedf0072b80 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedeffff640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 0x7fedf0117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf4e3a640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0116df0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fede400b450 tx=0x7fede400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fededffb640 1 -- 172.21.15.33:0/936830116 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4015020 con 0x7fedf0072b80 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf01b6ef0 con 0x7fedf0072b80 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.993+0000 7fededffb640 1 -- 172.21.15.33:0/936830116 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede4011d70 con 0x7fedf0072b80 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.994+0000 7fededffb640 1 -- 172.21.15.33:0/936830116 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede400f340 con 0x7fedf0072b80 2024-04-24T23:31:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.994+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01b73d0 con 0x7fedf0072b80 2024-04-24T23:31:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.997+0000 7fededffb640 1 -- 172.21.15.33:0/936830116 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fede4018020 con 0x7fedf0072b80 2024-04-24T23:31:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.997+0000 7fedf9daa740 1 Processor -- start 2024-04-24T23:31:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.997+0000 7fedf9daa740 1 -- start start 2024-04-24T23:31:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fededffb640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fedd80a2c60 0x7fedd80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c02470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedf9daa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c1c3b602c0 con 0x55c1c3c02ae0 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedf9daa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c1c3b08910 con 0x55c1c3bfcf80 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedf9daa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c1c3b5c280 con 0x55c1c3bfd970 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fededffb640 1 -- 172.21.15.33:0/936830116 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fede40cedd0 con 0x7fedf0072b80 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedeffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c02470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedeffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c02470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33408/0 (socket says 172.21.15.33:33408) 2024-04-24T23:31:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.998+0000 7fedeffff640 1 -- 172.21.15.33:0/1453203837 learned_addr learned my addr 172.21.15.33:0/1453203837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedf563b640 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedeffff640 1 -- 172.21.15.33:0/1453203837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 msgr2=0x55c1c3bfd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedeffff640 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedeffff640 1 -- 172.21.15.33:0/1453203837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 msgr2=0x55c1c3c04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedeffff640 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c04f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedeffff640 1 -- 172.21.15.33:0/1453203837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1c3b19fb0 con 0x55c1c3bfd970 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedf563b640 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:05.999+0000 7fedeffff640 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c02470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fede0009c00 tx=0x7fede0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89387a4e10eb9cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.000+0000 7fedf4e3a640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fedd80a2c60 0x7fedd80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.000+0000 7fedd6ffd640 1 -- 172.21.15.33:0/1453203837 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0004030 con 0x55c1c3bfd970 2024-04-24T23:31:06.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.000+0000 7fedd6ffd640 1 -- 172.21.15.33:0/1453203837 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede00041d0 con 0x55c1c3bfd970 2024-04-24T23:31:06.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.000+0000 7fedd6ffd640 1 -- 172.21.15.33:0/1453203837 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00126b0 con 0x55c1c3bfd970 2024-04-24T23:31:06.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.002+0000 7fedf9daa740 1 -- 172.21.15.33:0/1453203837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 msgr2=0x55c1c3c02470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.002+0000 7fedf9daa740 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c02470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fede0009c00 tx=0x7fede0010740 comp rx=0 tx=0).stop 2024-04-24T23:31:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.002+0000 7fedf4e3a640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fedd80a2c60 0x7fedd80a5120 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fede40a39e0 tx=0x7fede4004700 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.004+0000 7fedf9daa740 1 -- 172.21.15.33:0/1453203837 shutdown_connections 2024-04-24T23:31:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.004+0000 7fedf9daa740 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.004+0000 7fedf9daa740 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c02470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.004+0000 7fedf9daa740 1 --2- 172.21.15.33:0/1453203837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.004+0000 7fedf9daa740 1 -- 172.21.15.33:0/1453203837 >> 172.21.15.33:0/1453203837 conn(0x55c1c3bc4950 msgr2=0x55c1c3bf8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.005+0000 7fedf9daa740 1 -- 172.21.15.33:0/1453203837 shutdown_connections 2024-04-24T23:31:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.005+0000 7fedf9daa740 1 -- 172.21.15.33:0/1453203837 wait complete. 2024-04-24T23:31:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.005+0000 7fedf9daa740 1 -- start start 2024-04-24T23:31:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.005+0000 7fedf9daa740 1 Processor -- start 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.006+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.007+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfd970 0x55c1c3c079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.007+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.007+0000 7fedf9daa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c1c3c05a90 con 0x55c1c3bfd970 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.007+0000 7fedf9daa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c1c3b1be90 con 0x55c1c3bfcf80 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.007+0000 7fedf9daa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c1c3c05c10 con 0x55c1c3c02ae0 2024-04-24T23:31:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.007+0000 7fedf9daa740 10 client.0 _ll_register_callbacks cb 0x55c1c3b57970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.008+0000 7fedf563b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.008+0000 7fedf563b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c07460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41190/0 (socket says 172.21.15.33:41190) 2024-04-24T23:31:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.008+0000 7fedf563b640 1 -- 172.21.15.33:0/4209568108 learned_addr learned my addr 172.21.15.33:0/4209568108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.009+0000 7fedeffff640 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfd970 0x55c1c3c079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.009+0000 7fedeffff640 1 -- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3c02ae0 msgr2=0x55c1c3c0e9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.009+0000 7fedeffff640 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.009+0000 7fedeffff640 1 -- 172.21.15.33:0/4209568108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 msgr2=0x55c1c3c07460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.009+0000 7fedeffff640 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c07460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.009+0000 7fedeffff640 1 -- 172.21.15.33:0/4209568108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1c393c7e0 con 0x55c1c3bfd970 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.010+0000 7fedeffff640 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfd970 0x55c1c3c079a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x55c1c3bf98b0 tx=0x7fede0003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23daa9714225f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.010+0000 7fedcffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:06.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.013+0000 7fedcffff640 1 -- 172.21.15.33:0/4209568108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0012450 con 0x55c1c3bfd970 2024-04-24T23:31:06.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.013+0000 7fedcffff640 1 -- 172.21.15.33:0/4209568108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fede00125f0 con 0x55c1c3bfd970 2024-04-24T23:31:06.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.013+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede00041d0 con 0x55c1c3bfd970 2024-04-24T23:31:06.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.020+0000 7fedcffff640 1 -- 172.21.15.33:0/4209568108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fede00127f0 con 0x55c1c3bfd970 2024-04-24T23:31:06.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.021+0000 7fedf9daa740 10 client.19437 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:06.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.021+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c1c399cf70 con 0x55c1c3bfd970 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedcffff640 1 -- 172.21.15.33:0/4209568108 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0029070 con 0x55c1c3bfd970 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedcffff640 1 -- 172.21.15.33:0/4209568108 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fede000f040 con 0x55c1c3bfd970 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf9daa740 10 client.19437 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf9daa740 10 client.19437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf9daa740 20 client.19437 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf9daa740 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c12c00 0x55c1c3c14ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf9daa740 4 client.19437 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:06.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fede00041d0 con 0x55c1c3c12c00 2024-04-24T23:31:06.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.022+0000 7fedf563b640 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c12c00 0x55c1c3c14ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.024+0000 7fedf563b640 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c12c00 0x55c1c3c14ff0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732de845492272fb server_cookie=30ed591719333b22 in_seq=0 out_seq=0 2024-04-24T23:31:06.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.024+0000 7fedcffff640 10 client.19437 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.025+0000 7fedcffff640 1 -- 172.21.15.33:0/4209568108 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fede00041d0 con 0x55c1c3c12c00 2024-04-24T23:31:06.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.025+0000 7fedcffff640 10 client.19437 handle_command_reply: tid=0 2024-04-24T23:31:06.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.026+0000 7fedf9daa740 1 client.19437 shutdown 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfd970 msgr2=0x55c1c3c079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfd970 0x55c1c3c079a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x55c1c3bf98b0 tx=0x7fede0003be0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 shutdown_connections 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c12c00 0x55c1c3c14ff0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfd970 0x55c1c3c079a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedcffff640 0 client.19437 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 --2- 172.21.15.33:0/4209568108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c07460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.027+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 >> 172.21.15.33:0/4209568108 conn(0x55c1c3bc4950 msgr2=0x55c1c3bf7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.029+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 shutdown_connections 2024-04-24T23:31:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.032+0000 7fedf9daa740 1 -- 172.21.15.33:0/4209568108 wait complete. 2024-04-24T23:31:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.032+0000 7fedf9daa740 20 client.19437 trim_cache size 0 max 16384 2024-04-24T23:31:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.070+0000 7fedf9daa740 1 Processor -- start 2024-04-24T23:31:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 -- start start 2024-04-24T23:31:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedeffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedeffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33436/0 (socket says 172.21.15.33:33436) 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf563b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf563b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41200/0 (socket says 172.21.15.33:41200) 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedeffff640 1 -- 172.21.15.33:0/602284262 learned_addr learned my addr 172.21.15.33:0/602284262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01c4200 con 0x55c1c3c02ae0 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01c1ad0 con 0x55c1c3bfcf80 2024-04-24T23:31:06.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.071+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01bf920 con 0x55c1c3bfd970 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedf563b640 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedeffff640 1 -- 172.21.15.33:0/602284262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 msgr2=0x55c1c3bfc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedeffff640 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedeffff640 1 -- 172.21.15.33:0/602284262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 msgr2=0x55c1c3c198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedeffff640 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedeffff640 1 -- 172.21.15.33:0/602284262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1c3c15360 con 0x55c1c3bfd970 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.072+0000 7fedeffff640 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e9a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fede0009b20 tx=0x7fede00093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562c05836f8d4180 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.073+0000 7fedd67fc640 1 -- 172.21.15.33:0/602284262 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0036020 con 0x55c1c3bfd970 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.073+0000 7fedd67fc640 1 -- 172.21.15.33:0/602284262 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0029070 con 0x55c1c3bfd970 2024-04-24T23:31:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.073+0000 7fedd67fc640 1 -- 172.21.15.33:0/602284262 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede002e6b0 con 0x55c1c3bfd970 2024-04-24T23:31:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 msgr2=0x55c1c3c0e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e9a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fede0009b20 tx=0x7fede00093d0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 shutdown_connections 2024-04-24T23:31:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e9a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 --2- 172.21.15.33:0/602284262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3bfcf80 0x55c1c3bfc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.074+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 >> 172.21.15.33:0/602284262 conn(0x55c1c3bc4950 msgr2=0x55c1c3bfb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.075+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 shutdown_connections 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.075+0000 7fedf9daa740 1 -- 172.21.15.33:0/602284262 wait complete. 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.075+0000 7fedf9daa740 1 -- start start 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 Processor -- start 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c0e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01bff90 con 0x55c1c3bfcf80 2024-04-24T23:31:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01c1400 con 0x55c1c3c02ae0 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01c0660 con 0x55c1c3bfd970 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf9daa740 10 client.0 _ll_register_callbacks cb 0x7fede0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf563b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c0e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf563b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c0e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45678/0 (socket says 172.21.15.33:45678) 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf563b640 1 -- 172.21.15.33:0/2283808480 learned_addr learned my addr 172.21.15.33:0/2283808480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedeffff640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.076+0000 7fedf563b640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf563b640 1 -- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 msgr2=0x55c1c3c0e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf563b640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf563b640 1 -- 172.21.15.33:0/2283808480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3c02ae0 msgr2=0x55c1c3c0c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf563b640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf563b640 1 -- 172.21.15.33:0/2283808480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1c3834170 con 0x55c1c3bfcf80 2024-04-24T23:31:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf563b640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c0e330 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fede802aee0 tx=0x7fede00039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3eae906775ca4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedcffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedcffff640 1 -- 172.21.15.33:0/2283808480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede804d070 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedcffff640 1 -- 172.21.15.33:0/2283808480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fede8003d80 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.077+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede0029070 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedcffff640 1 -- 172.21.15.33:0/2283808480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fede8047320 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 10 client.19443 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c1c3c79b60 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedcffff640 1 -- 172.21.15.33:0/2283808480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede8036040 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedcffff640 1 -- 172.21.15.33:0/2283808480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fede802fcd0 con 0x55c1c3bfcf80 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 10 client.19443 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 10 client.19443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:06.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 20 client.19443 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c7a4f0 0x55c1c3c7c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 4 client.19443 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:31:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.078+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fede0029070 con 0x55c1c3c7a4f0 2024-04-24T23:31:06.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.079+0000 7fedeffff640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c7a4f0 0x55c1c3c7c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.087+0000 7fedeffff640 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c7a4f0 0x55c1c3c7c910 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd173dd44d5f1033 server_cookie=462cdd8f3e5986ea in_seq=0 out_seq=0 2024-04-24T23:31:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.087+0000 7fedcffff640 10 client.19443 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.092+0000 7fedcffff640 1 -- 172.21.15.33:0/2283808480 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fede0029070 con 0x55c1c3c7a4f0 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.092+0000 7fedcffff640 10 client.19443 handle_command_reply: tid=0 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.092+0000 7fedf9daa740 1 client.19443 shutdown 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.092+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 msgr2=0x55c1c3c0e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.092+0000 7fedf9daa740 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c0e330 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fede802aee0 tx=0x7fede00039b0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 shutdown_connections 2024-04-24T23:31:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c1c3c7a4f0 0x55c1c3c7c910 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1c3c02ae0 0x55c1c3c0c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1c3bfd970 0x55c1c3c0e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 --2- 172.21.15.33:0/2283808480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1c3bfcf80 0x55c1c3c0e330 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 >> 172.21.15.33:0/2283808480 conn(0x55c1c3bc4950 msgr2=0x55c1c3bf72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedcffff640 0 client.19443 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 shutdown_connections 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 1 -- 172.21.15.33:0/2283808480 wait complete. 2024-04-24T23:31:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.093+0000 7fedf9daa740 20 client.19443 trim_cache size 0 max 16384 2024-04-24T23:31:06.095 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:31:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:31:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "9d6dff9c-7859-4f5e-9b9e-97a10d3daa62", 2024-04-24T23:31:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:31:06.095 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fedd80a2c60 msgr2=0x7fedd80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fedd80a2c60 0x7fedd80a5120 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fede40a39e0 tx=0x7fede4004700 comp rx=0 tx=0).stop 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 msgr2=0x7fedf0116df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0116df0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fede400b450 tx=0x7fede400b910 comp rx=0 tx=0).stop 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 shutdown_connections 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fedd80a2c60 0x7fedd80a5120 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fede40a39e0 tx=0x7fede4004700 comp rx=0 tx=0).stop 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedf01097c0 0x7fedf01b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedf0108e20 0x7fedf0117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 --2- 172.21.15.33:0/936830116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf0072b80 0x7fedf0116df0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fede400b450 tx=0x7fede400b910 comp rx=0 tx=0).stop 2024-04-24T23:31:06.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.094+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 >> 172.21.15.33:0/936830116 conn(0x7fedf006e650 msgr2=0x7fedf010c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.095+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 shutdown_connections 2024-04-24T23:31:06.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.095+0000 7fedf5e3c640 1 -- 172.21.15.33:0/936830116 wait complete. 2024-04-24T23:31:06.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:06.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.322+0000 7ff52e895640 1 -- 172.21.15.33:0/3345062848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528101870 msgr2=0x7ff528101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.322+0000 7ff52e895640 1 --2- 172.21.15.33:0/3345062848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528101870 0x7ff528101cf0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009900 tx=0x7ff51c02a410 comp rx=0 tx=0).stop 2024-04-24T23:31:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.323+0000 7ff52e895640 1 -- 172.21.15.33:0/3345062848 shutdown_connections 2024-04-24T23:31:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.323+0000 7ff52e895640 1 --2- 172.21.15.33:0/3345062848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff528108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.323+0000 7ff52e895640 1 --2- 172.21.15.33:0/3345062848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528101870 0x7ff528101cf0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.323+0000 7ff52e895640 1 --2- 172.21.15.33:0/3345062848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528100680 0x7ff528100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.323+0000 7ff52e895640 1 -- 172.21.15.33:0/3345062848 >> 172.21.15.33:0/3345062848 conn(0x7ff5280fbc30 msgr2=0x7ff5280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.326+0000 7ff52e895640 1 -- 172.21.15.33:0/3345062848 shutdown_connections 2024-04-24T23:31:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.326+0000 7ff52e895640 1 -- 172.21.15.33:0/3345062848 wait complete. 2024-04-24T23:31:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.328+0000 7ff52e895640 1 Processor -- start 2024-04-24T23:31:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.328+0000 7ff52e895640 1 -- start start 2024-04-24T23:31:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.328+0000 7ff52e895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528100680 0x7ff528197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52e895640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528101870 0x7ff528197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52e895640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff52819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52e895640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810d480 con 0x7ff528100680 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52e895640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810d300 con 0x7ff528101870 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52ce32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff52819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52ce32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff52819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33462/0 (socket says 172.21.15.33:33462) 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52ce32640 1 -- 172.21.15.33:0/2374764895 learned_addr learned my addr 172.21.15.33:0/2374764895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.329+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810d600 con 0x7ff528102230 2024-04-24T23:31:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.330+0000 7ff52ce32640 1 -- 172.21.15.33:0/2374764895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528101870 msgr2=0x7ff528197fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:31:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.330+0000 7ff52ce32640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528101870 0x7ff528197fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.330+0000 7ff52ce32640 1 -- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528100680 msgr2=0x7ff528197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.330+0000 7ff52ce32640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528100680 0x7ff528197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.330+0000 7ff52ce32640 1 -- 172.21.15.33:0/2374764895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52819f6a0 con 0x7ff528102230 2024-04-24T23:31:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.332+0000 7ff52ce32640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff52819f030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff52000eb10 tx=0x7ff52000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.335+0000 7ff5257fa640 1 -- 172.21.15.33:0/2374764895 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520004b20 con 0x7ff528102230 2024-04-24T23:31:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.335+0000 7ff5257fa640 1 -- 172.21.15.33:0/2374764895 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff520004cc0 con 0x7ff528102230 2024-04-24T23:31:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.335+0000 7ff5257fa640 1 -- 172.21.15.33:0/2374764895 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520005650 con 0x7ff528102230 2024-04-24T23:31:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.335+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5281094e0 con 0x7ff528102230 2024-04-24T23:31:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.335+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281098f0 con 0x7ff528102230 2024-04-24T23:31:06.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.338+0000 7ff53159c740 1 Processor -- start 2024-04-24T23:31:06.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff5257fa640 1 -- 172.21.15.33:0/2374764895 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff5200057f0 con 0x7ff528102230 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 -- start start 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55889b2336b0 con 0x55889b2cfc30 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55889b1db880 con 0x55889b2d4da0 2024-04-24T23:31:06.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.339+0000 7ff53159c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55889b22f670 con 0x55889b2cf240 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff5277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2d4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45714/0 (socket says 172.21.15.33:45714) 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 -- 172.21.15.33:0/1030200579 learned_addr learned my addr 172.21.15.33:0/1030200579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff5277fe640 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 -- 172.21.15.33:0/1030200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 msgr2=0x55889b2cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 -- 172.21.15.33:0/1030200579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 msgr2=0x55889b2d7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2d7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 -- 172.21.15.33:0/1030200579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889b0117d0 con 0x55889b2cfc30 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff5257fa640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5100a2d60 0x7ff5100a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.340+0000 7ff527fff640 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2d4730 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff51800bb20 tx=0x7ff51800d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56e773e78f63adc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.341+0000 7ff5257fa640 1 -- 172.21.15.33:0/2374764895 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff5200cfa20 con 0x7ff528102230 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.341+0000 7ff5277fe640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5100a2d60 0x7ff5100a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.341+0000 7ff50e7fc640 1 -- 172.21.15.33:0/1030200579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518016020 con 0x55889b2cfc30 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.341+0000 7ff50e7fc640 1 -- 172.21.15.33:0/1030200579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff51800dd10 con 0x55889b2cfc30 2024-04-24T23:31:06.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.341+0000 7ff50e7fc640 1 -- 172.21.15.33:0/1030200579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800f880 con 0x55889b2cfc30 2024-04-24T23:31:06.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 -- 172.21.15.33:0/1030200579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 msgr2=0x55889b2d4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2d4730 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff51800bb20 tx=0x7ff51800d2e0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff5277fe640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5100a2d60 0x7ff5100a5220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55889b2d0490 tx=0x7ff51c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 -- 172.21.15.33:0/1030200579 shutdown_connections 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2d7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2d4730 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 --2- 172.21.15.33:0/1030200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 -- 172.21.15.33:0/1030200579 >> 172.21.15.33:0/1030200579 conn(0x55889b2ca8b0 msgr2=0x55889b2cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 -- 172.21.15.33:0/1030200579 shutdown_connections 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.342+0000 7ff53159c740 1 -- 172.21.15.33:0/1030200579 wait complete. 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.343+0000 7ff53159c740 1 -- start start 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.343+0000 7ff53159c740 1 Processor -- start 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.343+0000 7ff53159c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2ea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.343+0000 7ff53159c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff53159c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff53159c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55889b2e8d60 con 0x55889b2cfc30 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff53159c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55889b1eef50 con 0x55889b2d4da0 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff53159c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55889b2e8ee0 con 0x55889b2cf240 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2ea710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:33474/0 (socket says 172.21.15.33:33474) 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 -- 172.21.15.33:0/186240419 learned_addr learned my addr 172.21.15.33:0/186240419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff5277fe640 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff52ce32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff53159c740 10 client.0 _ll_register_callbacks cb 0x55889b22a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 -- 172.21.15.33:0/186240419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 msgr2=0x55889b2f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 -- 172.21.15.33:0/186240419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 msgr2=0x55889b2eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2eac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff527fff640 1 -- 172.21.15.33:0/186240419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889af473f0 con 0x55889b2cf240 2024-04-24T23:31:06.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.344+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51800dd10 con 0x55889b2cf240 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff527fff640 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2ea710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff518004700 tx=0x7ff51800f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267308667a8d8bde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff5077fe640 10 client.28876 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff5077fe640 1 -- 172.21.15.33:0/186240419 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800fb10 con 0x55889b2cf240 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff5077fe640 1 -- 172.21.15.33:0/186240419 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff51800fcb0 con 0x55889b2cf240 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff5077fe640 1 -- 172.21.15.33:0/186240419 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff51800feb0 con 0x55889b2cf240 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff53159c740 10 client.28876 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.345+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55889af070f0 con 0x55889b2cf240 2024-04-24T23:31:06.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff5077fe640 1 -- 172.21.15.33:0/186240419 <== mon.1 v2:172.21.15.117:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff518010460 con 0x55889b2cf240 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff53159c740 10 client.28876 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff53159c740 10 client.28876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff53159c740 20 client.28876 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff53159c740 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b2f60a0 0x55889b2f8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff53159c740 4 client.28876 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.346+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889b240bf0 con 0x55889b2f60a0 2024-04-24T23:31:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.347+0000 7ff5277fe640 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b2f60a0 0x55889b2f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.348+0000 7ff5077fe640 10 client.28876 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.348+0000 7ff5277fe640 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b2f60a0 0x55889b2f8490 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b8a6a16ca631d0 server_cookie=189294fd9c9d24a2 in_seq=0 out_seq=0 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.349+0000 7ff5077fe640 1 -- 172.21.15.33:0/186240419 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55889b240bf0 con 0x55889b2f60a0 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.349+0000 7ff5077fe640 10 client.28876 handle_command_reply: tid=0 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.349+0000 7ff53159c740 1 client.28876 shutdown 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 msgr2=0x55889b2ea710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2ea710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff518004700 tx=0x7ff51800f810 comp rx=0 tx=0).stop 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 shutdown_connections 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b2f60a0 0x55889b2f8490 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff5077fe640 0 client.28876 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2d4da0 0x55889b2f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2cfc30 0x55889b2eac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 --2- 172.21.15.33:0/186240419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cf240 0x55889b2ea710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 >> 172.21.15.33:0/186240419 conn(0x55889b2ca8b0 msgr2=0x55889b2c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 shutdown_connections 2024-04-24T23:31:06.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 1 -- 172.21.15.33:0/186240419 wait complete. 2024-04-24T23:31:06.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.350+0000 7ff53159c740 20 client.28876 trim_cache size 0 max 16384 2024-04-24T23:31:06.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 Processor -- start 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 -- start start 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5281aaa10 con 0x55889b2d4da0 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff5281a82e0 con 0x55889b2cf240 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff53159c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff528071960 con 0x55889b2cfc30 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41248/0 (socket says 172.21.15.33:41248) 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff527fff640 1 -- 172.21.15.33:0/532961991 learned_addr learned my addr 172.21.15.33:0/532961991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff52ce32640 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.374+0000 7ff5277fe640 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff527fff640 1 -- 172.21.15.33:0/532961991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 msgr2=0x55889b2f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff527fff640 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff527fff640 1 -- 172.21.15.33:0/532961991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 msgr2=0x55889b2e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff527fff640 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2e9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff527fff640 1 -- 172.21.15.33:0/532961991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889b2bc090 con 0x55889b2cf240 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff5277fe640 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff527fff640 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2e9280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff51800f5d0 tx=0x7ff51801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbdeecc8819e6d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff50dffb640 1 -- 172.21.15.33:0/532961991 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518016070 con 0x55889b2cf240 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff50dffb640 1 -- 172.21.15.33:0/532961991 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff518037020 con 0x55889b2cf240 2024-04-24T23:31:06.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.375+0000 7ff50dffb640 1 -- 172.21.15.33:0/532961991 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518004c80 con 0x55889b2cf240 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 -- 172.21.15.33:0/532961991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 msgr2=0x55889b2e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2e9280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff51800f5d0 tx=0x7ff51801e000 comp rx=0 tx=0).stop 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 -- 172.21.15.33:0/532961991 shutdown_connections 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 --2- 172.21.15.33:0/532961991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2e9280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 -- 172.21.15.33:0/532961991 >> 172.21.15.33:0/532961991 conn(0x55889b2ca8b0 msgr2=0x55889b2ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 -- 172.21.15.33:0/532961991 shutdown_connections 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 -- 172.21.15.33:0/532961991 wait complete. 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.376+0000 7ff53159c740 1 -- start start 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 Processor -- start 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2ef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff528071fd0 con 0x55889b2d4da0 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41264/0 (socket says 172.21.15.33:41264) 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff527fff640 1 -- 172.21.15.33:0/3275395586 learned_addr learned my addr 172.21.15.33:0/3275395586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff5277fe640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff528073440 con 0x55889b2cf240 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff52ce32640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280726a0 con 0x55889b2cfc30 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.377+0000 7ff53159c740 10 client.0 _ll_register_callbacks cb 0x7ff5200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff52ce32640 1 -- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 msgr2=0x55889b2ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff52ce32640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff52ce32640 1 -- 172.21.15.33:0/3275395586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 msgr2=0x55889b2ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff52ce32640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff52ce32640 1 -- 172.21.15.33:0/3275395586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889b2f0e00 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff527fff640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff52ce32640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2f0810 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff5200982f0 tx=0x7ff5200cd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841e522034cc7050 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff5077fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff5077fe640 1 -- 172.21.15.33:0/3275395586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200a4b00 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff5077fe640 1 -- 172.21.15.33:0/3275395586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff520019340 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff518037020 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff5077fe640 1 -- 172.21.15.33:0/3275395586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5200ad050 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.378+0000 7ff53159c740 10 client.19458 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55889b35c630 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff5077fe640 1 -- 172.21.15.33:0/3275395586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff518037020 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff5077fe640 1 -- 172.21.15.33:0/3275395586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff52009b2e0 con 0x55889b2d4da0 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 10 client.19458 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 10 client.19458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 20 client.19458 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:06.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b35cfd0 0x55889b35f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 4 client.19458 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55889b23af40 con 0x55889b35cfd0 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.379+0000 7ff527fff640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b35cfd0 0x55889b35f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.380+0000 7ff5077fe640 10 client.19458 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.380+0000 7ff527fff640 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b35cfd0 0x55889b35f3c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bc4139b3cd9ad1 server_cookie=247e25cb19ce7669 in_seq=0 out_seq=0 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.380+0000 7ff5077fe640 1 -- 172.21.15.33:0/3275395586 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55889b23af40 con 0x55889b35cfd0 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.380+0000 7ff5077fe640 10 client.19458 handle_command_reply: tid=0 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.380+0000 7ff53159c740 1 client.19458 shutdown 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 msgr2=0x55889b2f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2f0810 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff5200982f0 tx=0x7ff5200cd000 comp rx=0 tx=0).stop 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 shutdown_connections 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55889b35cfd0 0x55889b35f3c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889b2d4da0 0x55889b2f0810 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55889b2cfc30 0x55889b2ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff5077fe640 0 client.19458 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 --2- 172.21.15.33:0/3275395586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55889b2cf240 0x55889b2ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 >> 172.21.15.33:0/3275395586 conn(0x55889b2ca8b0 msgr2=0x55889b2ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 shutdown_connections 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 1 -- 172.21.15.33:0/3275395586 wait complete. 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.381+0000 7ff53159c740 20 client.19458 trim_cache size 0 max 16384 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "9d6dff9c-7859-4f5e-9b9e-97a10d3daa62": { 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:31:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "9d6dff9c-7859-4f5e-9b9e-97a10d3daa62", 2024-04-24T23:31:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:31:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:31:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:31:06.386 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:31:06.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5100a2d60 msgr2=0x7ff5100a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5100a2d60 0x7ff5100a5220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55889b2d0490 tx=0x7ff51c031400 comp rx=0 tx=0).stop 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 msgr2=0x7ff52819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff52819f030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff52000eb10 tx=0x7ff52000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 shutdown_connections 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff5100a2d60 0x7ff5100a5220 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff528102230 0x7ff52819f030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528101870 0x7ff528197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 --2- 172.21.15.33:0/2374764895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff528100680 0x7ff528197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 >> 172.21.15.33:0/2374764895 conn(0x7ff5280fbc30 msgr2=0x7ff528104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 shutdown_connections 2024-04-24T23:31:06.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:06.382+0000 7ff52e895640 1 -- 172.21.15.33:0/2374764895 wait complete. 2024-04-24T23:31:06.393 INFO:tasks.cephfs.filesystem:scrub status for tag:9d6dff9c-7859-4f5e-9b9e-97a10d3daa62 - {'path': '/', 'tag': '9d6dff9c-7859-4f5e-9b9e-97a10d3daa62', 'options': 'recursive,force'} 2024-04-24T23:31:06.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 134927 22.57 MB/sec execute 191 sec latency 81.118 ms 2024-04-24T23:31:06.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:31:06.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:06.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:06.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:06.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:06 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:31:06.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:06.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:06 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:06.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:06 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:07.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 135356 22.60 MB/sec execute 192 sec latency 106.148 ms 2024-04-24T23:31:07.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:07 smithi038 ceph-mon[32450]: pgmap v924: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 398 op/s 2024-04-24T23:31:07.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:07 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:07 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:07 smithi117 ceph-mon[32529]: pgmap v924: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 398 op/s 2024-04-24T23:31:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:07 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:07 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:07.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:07 smithi033 ceph-mon[29201]: pgmap v924: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 398 op/s 2024-04-24T23:31:07.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:07 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:07.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:07 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:08.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 135865 22.68 MB/sec execute 193 sec latency 90.343 ms 2024-04-24T23:31:08.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:08 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:08.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:08 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:08.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:08 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:09.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 136080 22.61 MB/sec execute 194 sec latency 111.263 ms 2024-04-24T23:31:09.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:09 smithi038 ceph-mon[32450]: pgmap v925: 129 pgs: 129 active+clean; 563 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 381 op/s 2024-04-24T23:31:09.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:09 smithi117 ceph-mon[32529]: pgmap v925: 129 pgs: 129 active+clean; 563 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 381 op/s 2024-04-24T23:31:09.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:09 smithi033 ceph-mon[29201]: pgmap v925: 129 pgs: 129 active+clean; 563 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 381 op/s 2024-04-24T23:31:10.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 136286 22.54 MB/sec execute 195 sec latency 410.608 ms 2024-04-24T23:31:11.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 136600 22.47 MB/sec execute 196 sec latency 101.409 ms 2024-04-24T23:31:11.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:11 smithi038 ceph-mon[32450]: pgmap v926: 129 pgs: 129 active+clean; 568 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-04-24T23:31:11.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:11 smithi117 ceph-mon[32529]: pgmap v926: 129 pgs: 129 active+clean; 568 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-04-24T23:31:11.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:11 smithi033 ceph-mon[29201]: pgmap v926: 129 pgs: 129 active+clean; 568 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-04-24T23:31:12.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 137262 22.52 MB/sec execute 197 sec latency 45.235 ms 2024-04-24T23:31:13.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 137681 22.53 MB/sec execute 198 sec latency 265.797 ms 2024-04-24T23:31:13.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:13 smithi038 ceph-mon[32450]: pgmap v927: 129 pgs: 129 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 527 op/s 2024-04-24T23:31:13.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:13 smithi117 ceph-mon[32529]: pgmap v927: 129 pgs: 129 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 527 op/s 2024-04-24T23:31:13.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:13 smithi033 ceph-mon[29201]: pgmap v927: 129 pgs: 129 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 527 op/s 2024-04-24T23:31:14.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 138020 22.53 MB/sec execute 199 sec latency 103.540 ms 2024-04-24T23:31:15.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 138292 22.52 MB/sec execute 200 sec latency 286.817 ms 2024-04-24T23:31:15.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:15 smithi038 ceph-mon[32450]: pgmap v928: 129 pgs: 129 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 445 op/s 2024-04-24T23:31:15.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:15 smithi117 ceph-mon[32529]: pgmap v928: 129 pgs: 129 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 445 op/s 2024-04-24T23:31:15.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:15 smithi033 ceph-mon[29201]: pgmap v928: 129 pgs: 129 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 445 op/s 2024-04-24T23:31:16.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 138622 22.45 MB/sec execute 201 sec latency 115.906 ms 2024-04-24T23:31:16.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:16 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:31:16.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:16 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:31:16.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:16 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:31:17.891 INFO:tasks.workunit.client.0.smithi033.stdout: 10 139133 22.47 MB/sec execute 202 sec latency 98.699 ms 2024-04-24T23:31:17.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:17 smithi038 ceph-mon[32450]: pgmap v929: 129 pgs: 129 active+clean; 572 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 468 op/s 2024-04-24T23:31:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:31:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:31:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:31:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:17 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:31:17.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:17 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:31:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:17 smithi117 ceph-mon[32529]: pgmap v929: 129 pgs: 129 active+clean; 572 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 468 op/s 2024-04-24T23:31:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:31:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:31:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:31:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:17 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:31:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:17 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:31:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:17 smithi033 ceph-mon[29201]: pgmap v929: 129 pgs: 129 active+clean; 572 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 468 op/s 2024-04-24T23:31:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:31:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:31:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:31:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:17 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:31:17.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:17 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:31:18.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 139650 22.54 MB/sec execute 203 sec latency 70.808 ms 2024-04-24T23:31:18.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:18 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:31:18.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:18 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:31:18.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:18 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:31:19.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 140230 22.61 MB/sec execute 204 sec latency 45.500 ms 2024-04-24T23:31:19.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:19 smithi038 ceph-mon[32450]: pgmap v930: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.1 MiB/s wr, 441 op/s 2024-04-24T23:31:19.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:19 smithi117 ceph-mon[32529]: pgmap v930: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.1 MiB/s wr, 441 op/s 2024-04-24T23:31:19.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:19 smithi033 ceph-mon[29201]: pgmap v930: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.1 MiB/s wr, 441 op/s 2024-04-24T23:31:20.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 140358 22.52 MB/sec execute 205 sec latency 312.828 ms 2024-04-24T23:31:21.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 140701 22.47 MB/sec execute 206 sec latency 81.326 ms 2024-04-24T23:31:21.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:21 smithi038 ceph-mon[32450]: pgmap v931: 129 pgs: 129 active+clean; 557 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 390 op/s 2024-04-24T23:31:21.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:21 smithi117 ceph-mon[32529]: pgmap v931: 129 pgs: 129 active+clean; 557 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 390 op/s 2024-04-24T23:31:21.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:21 smithi033 ceph-mon[29201]: pgmap v931: 129 pgs: 129 active+clean; 557 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 390 op/s 2024-04-24T23:31:22.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 141188 22.51 MB/sec execute 207 sec latency 65.354 ms 2024-04-24T23:31:23.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 141549 22.44 MB/sec execute 208 sec latency 159.013 ms 2024-04-24T23:31:23.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:23 smithi038 ceph-mon[32450]: pgmap v932: 129 pgs: 129 active+clean; 568 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 454 op/s 2024-04-24T23:31:23.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:23 smithi117 ceph-mon[32529]: pgmap v932: 129 pgs: 129 active+clean; 568 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 454 op/s 2024-04-24T23:31:23.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:23 smithi033 ceph-mon[29201]: pgmap v932: 129 pgs: 129 active+clean; 568 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 454 op/s 2024-04-24T23:31:24.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 142097 22.49 MB/sec execute 209 sec latency 83.194 ms 2024-04-24T23:31:25.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 142591 22.51 MB/sec execute 210 sec latency 251.112 ms 2024-04-24T23:31:25.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:25 smithi038 ceph-mon[32450]: pgmap v933: 129 pgs: 129 active+clean; 571 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 380 op/s 2024-04-24T23:31:25.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:25 smithi117 ceph-mon[32529]: pgmap v933: 129 pgs: 129 active+clean; 571 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 380 op/s 2024-04-24T23:31:25.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:25 smithi033 ceph-mon[29201]: pgmap v933: 129 pgs: 129 active+clean; 571 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 380 op/s 2024-04-24T23:31:26.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 143227 22.56 MB/sec execute 211 sec latency 110.363 ms 2024-04-24T23:31:27.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 143608 22.61 MB/sec execute 212 sec latency 63.984 ms 2024-04-24T23:31:27.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:27 smithi038 ceph-mon[32450]: pgmap v934: 129 pgs: 129 active+clean; 569 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.6 MiB/s wr, 416 op/s 2024-04-24T23:31:28.000 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:27 smithi033 ceph-mon[29201]: pgmap v934: 129 pgs: 129 active+clean; 569 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.6 MiB/s wr, 416 op/s 2024-04-24T23:31:28.000 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:27 smithi117 ceph-mon[32529]: pgmap v934: 129 pgs: 129 active+clean; 569 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.6 MiB/s wr, 416 op/s 2024-04-24T23:31:28.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 144021 22.61 MB/sec execute 213 sec latency 62.130 ms 2024-04-24T23:31:29.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 144338 22.58 MB/sec execute 214 sec latency 135.176 ms 2024-04-24T23:31:29.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:29 smithi038 ceph-mon[32450]: pgmap v935: 129 pgs: 129 active+clean; 570 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 345 op/s 2024-04-24T23:31:29.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:29 smithi117 ceph-mon[32529]: pgmap v935: 129 pgs: 129 active+clean; 570 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 345 op/s 2024-04-24T23:31:29.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:29 smithi033 ceph-mon[29201]: pgmap v935: 129 pgs: 129 active+clean; 570 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 345 op/s 2024-04-24T23:31:30.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 144699 22.54 MB/sec execute 215 sec latency 310.971 ms 2024-04-24T23:31:31.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 144963 22.49 MB/sec execute 216 sec latency 122.014 ms 2024-04-24T23:31:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:31 smithi038 ceph-mon[32450]: pgmap v936: 129 pgs: 129 active+clean; 574 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-04-24T23:31:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:31 smithi117 ceph-mon[32529]: pgmap v936: 129 pgs: 129 active+clean; 574 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-04-24T23:31:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:31 smithi033 ceph-mon[29201]: pgmap v936: 129 pgs: 129 active+clean; 574 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-04-24T23:31:32.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 145459 22.52 MB/sec execute 217 sec latency 72.783 ms 2024-04-24T23:31:33.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 145957 22.51 MB/sec execute 218 sec latency 64.097 ms 2024-04-24T23:31:33.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:33 smithi038 ceph-mon[32450]: pgmap v937: 129 pgs: 129 active+clean; 584 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 471 op/s 2024-04-24T23:31:33.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:33 smithi117 ceph-mon[32529]: pgmap v937: 129 pgs: 129 active+clean; 584 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 471 op/s 2024-04-24T23:31:33.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:33 smithi033 ceph-mon[29201]: pgmap v937: 129 pgs: 129 active+clean; 584 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 471 op/s 2024-04-24T23:31:34.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 146553 22.55 MB/sec execute 219 sec latency 83.963 ms 2024-04-24T23:31:35.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 146955 22.55 MB/sec execute 220 sec latency 361.645 ms 2024-04-24T23:31:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:35 smithi038 ceph-mon[32450]: pgmap v938: 129 pgs: 129 active+clean; 584 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 377 op/s 2024-04-24T23:31:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:35 smithi117 ceph-mon[32529]: pgmap v938: 129 pgs: 129 active+clean; 584 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 377 op/s 2024-04-24T23:31:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:35 smithi033 ceph-mon[29201]: pgmap v938: 129 pgs: 129 active+clean; 584 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 377 op/s 2024-04-24T23:31:36.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:36.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.544+0000 7fc367590640 1 -- 172.21.15.33:0/1622845602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc360109fe0 msgr2=0x7fc36010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.544+0000 7fc367590640 1 --2- 172.21.15.33:0/1622845602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc360109fe0 0x7fc36010a460 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009860 tx=0x7fc35c02a360 comp rx=0 tx=0).stop 2024-04-24T23:31:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.545+0000 7fc367590640 1 -- 172.21.15.33:0/1622845602 shutdown_connections 2024-04-24T23:31:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.545+0000 7fc367590640 1 --2- 172.21.15.33:0/1622845602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36010a9a0 0x7fc3601112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.545+0000 7fc367590640 1 --2- 172.21.15.33:0/1622845602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc360109fe0 0x7fc36010a460 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.545+0000 7fc367590640 1 --2- 172.21.15.33:0/1622845602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360108e90 0x7fc3601092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.545+0000 7fc367590640 1 -- 172.21.15.33:0/1622845602 >> 172.21.15.33:0/1622845602 conn(0x7fc36006f620 msgr2=0x7fc360071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.547+0000 7fc367590640 1 -- 172.21.15.33:0/1622845602 shutdown_connections 2024-04-24T23:31:36.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.549+0000 7fc367590640 1 -- 172.21.15.33:0/1622845602 wait complete. 2024-04-24T23:31:36.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.549+0000 7fc367590640 1 Processor -- start 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.550+0000 7fc367590640 1 -- start start 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.550+0000 7fc367590640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360108e90 0x7fc3601a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc367590640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc360109fe0 0x7fc3601a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc367590640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36010a9a0 0x7fc3601a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc367590640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc36006a3d0 con 0x7fc36010a9a0 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc367590640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc36006a250 con 0x7fc360108e90 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc367590640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc36006a550 con 0x7fc360109fe0 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc364b2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc360109fe0 0x7fc3601a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc364b2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc360109fe0 0x7fc3601a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36646/0 (socket says 172.21.15.33:36646) 2024-04-24T23:31:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc364b2b640 1 -- 172.21.15.33:0/1369928843 learned_addr learned my addr 172.21.15.33:0/1369928843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.551+0000 7fc365b2d640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36010a9a0 0x7fc3601a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.552+0000 7fc365b2d640 1 -- 172.21.15.33:0/1369928843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc360109fe0 msgr2=0x7fc3601a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.552+0000 7fc365b2d640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc360109fe0 0x7fc3601a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.552+0000 7fc365b2d640 1 -- 172.21.15.33:0/1369928843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360108e90 msgr2=0x7fc3601a0330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.552+0000 7fc365b2d640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360108e90 0x7fc3601a0330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.553+0000 7fc365b2d640 1 -- 172.21.15.33:0/1369928843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3601a7f50 con 0x7fc36010a9a0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.554+0000 7fc365b2d640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36010a9a0 0x7fc3601a78e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc35800efc0 tx=0x7fc35800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.554+0000 7fc3567fc640 1 -- 172.21.15.33:0/1369928843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358003970 con 0x7fc36010a9a0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.554+0000 7fc3567fc640 1 -- 172.21.15.33:0/1369928843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc358004020 con 0x7fc36010a9a0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.554+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc360114600 con 0x7fc36010a9a0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.554+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc360114b50 con 0x7fc36010a9a0 2024-04-24T23:31:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.554+0000 7fc3567fc640 1 -- 172.21.15.33:0/1369928843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35800b9b0 con 0x7fc36010a9a0 2024-04-24T23:31:36.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.560+0000 7fc36a37b740 1 Processor -- start 2024-04-24T23:31:36.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 -- start start 2024-04-24T23:31:36.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 0x563168d97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d985b0 0x563168d9aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563168cee6b0 con 0x563168d985b0 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563168c96880 con 0x563168d93440 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.561+0000 7fc36a37b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563168cea670 con 0x563168d92a50 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.562+0000 7fc36532c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 0x563168d97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 0x563168d97f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48080/0 (socket says 172.21.15.33:48080) 2024-04-24T23:31:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 -- 172.21.15.33:0/3296666733 learned_addr learned my addr 172.21.15.33:0/3296666733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 -- 172.21.15.33:0/3296666733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 msgr2=0x563168d92e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 -- 172.21.15.33:0/3296666733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d985b0 msgr2=0x563168d9aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d985b0 0x563168d9aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.563+0000 7fc36532c640 1 -- 172.21.15.33:0/3296666733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563168acc7d0 con 0x563168d93440 2024-04-24T23:31:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.564+0000 7fc3567fc640 1 -- 172.21.15.33:0/1369928843 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc35801d020 con 0x7fc36010a9a0 2024-04-24T23:31:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.564+0000 7fc36532c640 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 0x563168d97f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc350009770 tx=0x7fc350009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5918447fac12aefd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.566+0000 7fc34b7fe640 1 -- 172.21.15.33:0/3296666733 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350015020 con 0x563168d93440 2024-04-24T23:31:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.566+0000 7fc34b7fe640 1 -- 172.21.15.33:0/3296666733 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc350011420 con 0x563168d93440 2024-04-24T23:31:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.567+0000 7fc36a37b740 1 -- 172.21.15.33:0/3296666733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 msgr2=0x563168d97f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.567+0000 7fc36a37b740 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 0x563168d97f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc350009770 tx=0x7fc350009c30 comp rx=0 tx=0).stop 2024-04-24T23:31:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.568+0000 7fc3567fc640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc34c0a2d60 0x7fc34c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.568+0000 7fc364b2b640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc34c0a2d60 0x7fc34c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.569+0000 7fc3567fc640 1 -- 172.21.15.33:0/1369928843 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc3580cf270 con 0x7fc36010a9a0 2024-04-24T23:31:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.572+0000 7fc364b2b640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc34c0a2d60 0x7fc34c0a5220 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3601a1840 tx=0x7fc35c02aea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.573+0000 7fc36a37b740 1 -- 172.21.15.33:0/3296666733 shutdown_connections 2024-04-24T23:31:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.573+0000 7fc36a37b740 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d985b0 0x563168d9aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.573+0000 7fc36a37b740 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d93440 0x563168d97f40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.573+0000 7fc36a37b740 1 --2- 172.21.15.33:0/3296666733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d92e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.573+0000 7fc36a37b740 1 -- 172.21.15.33:0/3296666733 >> 172.21.15.33:0/3296666733 conn(0x563168d858b0 msgr2=0x563168d85cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.578+0000 7fc36a37b740 1 -- 172.21.15.33:0/3296666733 shutdown_connections 2024-04-24T23:31:36.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.579+0000 7fc36a37b740 1 -- 172.21.15.33:0/3296666733 wait complete. 2024-04-24T23:31:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.579+0000 7fc36a37b740 1 -- start start 2024-04-24T23:31:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 Processor -- start 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d985b0 0x563168d8c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d911a0 0x563168d91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563168d8a390 con 0x563168d911a0 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563168ca9f50 con 0x563168d985b0 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563168d8a510 con 0x563168d92a50 2024-04-24T23:31:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc36a37b740 10 client.0 _ll_register_callbacks cb 0x563168ce5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc365b2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d911a0 0x563168d91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc365b2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d911a0 0x563168d91620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54052/0 (socket says 172.21.15.33:54052) 2024-04-24T23:31:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.580+0000 7fc365b2d640 1 -- 172.21.15.33:0/959714608 learned_addr learned my addr 172.21.15.33:0/959714608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc36532c640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d985b0 0x563168d8c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 -- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 msgr2=0x563168d8bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d8bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 -- 172.21.15.33:0/959714608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d911a0 msgr2=0x563168d91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d911a0 0x563168d91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 -- 172.21.15.33:0/959714608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563168a023f0 con 0x563168d985b0 2024-04-24T23:31:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.581+0000 7fc364b2b640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d985b0 0x563168d8c170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c0304f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73d4552f232dfda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.582+0000 7fc348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:31:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.582+0000 7fc348ff9640 1 -- 172.21.15.33:0/959714608 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c037070 con 0x563168d985b0 2024-04-24T23:31:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.582+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563168cfbbf0 con 0x563168d985b0 2024-04-24T23:31:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.582+0000 7fc348ff9640 1 -- 172.21.15.33:0/959714608 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc35c030be0 con 0x563168d985b0 2024-04-24T23:31:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.582+0000 7fc348ff9640 1 -- 172.21.15.33:0/959714608 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc35c056050 con 0x563168d985b0 2024-04-24T23:31:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.583+0000 7fc36a37b740 10 client.28949 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.583+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5631689c20f0 con 0x563168d985b0 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc348ff9640 1 -- 172.21.15.33:0/959714608 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563168cfbbf0 con 0x563168d985b0 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc348ff9640 1 -- 172.21.15.33:0/959714608 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc35c003820 con 0x563168d985b0 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36a37b740 10 client.28949 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36a37b740 10 client.28949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36a37b740 20 client.28949 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36a37b740 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168db1890 0x563168db3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36a37b740 4 client.28949 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563168cf5f40 con 0x563168db1890 2024-04-24T23:31:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.584+0000 7fc36532c640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168db1890 0x563168db3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.588+0000 7fc36532c640 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168db1890 0x563168db3d10 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd97692dc8ad7ff server_cookie=10ba13e60df8c54b in_seq=0 out_seq=0 2024-04-24T23:31:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.588+0000 7fc348ff9640 10 client.28949 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.591+0000 7fc348ff9640 1 -- 172.21.15.33:0/959714608 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563168cf5f40 con 0x563168db1890 2024-04-24T23:31:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.591+0000 7fc348ff9640 10 client.28949 handle_command_reply: tid=0 2024-04-24T23:31:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.591+0000 7fc36a37b740 1 client.28949 shutdown 2024-04-24T23:31:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d985b0 msgr2=0x563168d8c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d985b0 0x563168d8c170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c0304f0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 shutdown_connections 2024-04-24T23:31:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168db1890 0x563168db3d10 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d911a0 0x563168d91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d985b0 0x563168d8c170 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c0304f0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 --2- 172.21.15.33:0/959714608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d92a50 0x563168d8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.596+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 >> 172.21.15.33:0/959714608 conn(0x563168d858b0 msgr2=0x563168d84af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.599+0000 7fc348ff9640 0 client.28949 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:36.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.604+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 shutdown_connections 2024-04-24T23:31:36.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.607+0000 7fc36a37b740 1 -- 172.21.15.33:0/959714608 wait complete. 2024-04-24T23:31:36.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.607+0000 7fc36a37b740 20 client.28949 trim_cache size 0 max 16384 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 Processor -- start 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 -- start start 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168db38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d92a50 0x563168d8b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d84260 0x563168d846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3440125f0 con 0x563168d84260 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc34400fec0 con 0x563168d92a50 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.646+0000 7fc36a37b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc34400dd10 con 0x563168d911a0 2024-04-24T23:31:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.647+0000 7fc36532c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168db38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.647+0000 7fc36532c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168db38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36680/0 (socket says 172.21.15.33:36680) 2024-04-24T23:31:36.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.647+0000 7fc36532c640 1 -- 172.21.15.33:0/2178934635 learned_addr learned my addr 172.21.15.33:0/2178934635 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:36.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.647+0000 7fc364b2b640 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d92a50 0x563168d8b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.648+0000 7fc36532c640 1 -- 172.21.15.33:0/2178934635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d92a50 msgr2=0x563168d8b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.648+0000 7fc36532c640 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d92a50 0x563168d8b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.648+0000 7fc36532c640 1 -- 172.21.15.33:0/2178934635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d84260 msgr2=0x563168d846e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.648+0000 7fc36532c640 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d84260 0x563168d846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.648+0000 7fc36532c640 1 -- 172.21.15.33:0/2178934635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563168d94de0 con 0x563168d911a0 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.648+0000 7fc36532c640 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168db38a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x563168d93eb0 tx=0x7fc350009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d95efdcc753dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.649+0000 7fc34affd640 1 -- 172.21.15.33:0/2178934635 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350015070 con 0x563168d911a0 2024-04-24T23:31:36.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.650+0000 7fc34affd640 1 -- 172.21.15.33:0/2178934635 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35000be10 con 0x563168d911a0 2024-04-24T23:31:36.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.651+0000 7fc36a37b740 1 -- 172.21.15.33:0/2178934635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 msgr2=0x563168db38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.651+0000 7fc36a37b740 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168db38a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x563168d93eb0 tx=0x7fc350009c30 comp rx=0 tx=0).stop 2024-04-24T23:31:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.654+0000 7fc36a37b740 1 -- 172.21.15.33:0/2178934635 shutdown_connections 2024-04-24T23:31:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.654+0000 7fc36a37b740 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d84260 0x563168d846e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.654+0000 7fc36a37b740 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d92a50 0x563168d8b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.654+0000 7fc36a37b740 1 --2- 172.21.15.33:0/2178934635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168db38a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.655+0000 7fc36a37b740 1 -- 172.21.15.33:0/2178934635 >> 172.21.15.33:0/2178934635 conn(0x563168d858b0 msgr2=0x563168d8a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.655+0000 7fc36a37b740 1 -- 172.21.15.33:0/2178934635 shutdown_connections 2024-04-24T23:31:36.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.655+0000 7fc36a37b740 1 -- 172.21.15.33:0/2178934635 wait complete. 2024-04-24T23:31:36.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.655+0000 7fc36a37b740 1 -- start start 2024-04-24T23:31:36.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.655+0000 7fc36a37b740 1 Processor -- start 2024-04-24T23:31:36.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d84260 0x563168dc2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168dc3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d92a50 0x563168dc4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc34400e380 con 0x563168d92a50 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc34400f7f0 con 0x563168d84260 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc34400ea50 con 0x563168d911a0 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.656+0000 7fc36a37b740 10 client.0 _ll_register_callbacks cb 0x7fc35c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.657+0000 7fc364b2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168dc3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.657+0000 7fc364b2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168dc3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36684/0 (socket says 172.21.15.33:36684) 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.657+0000 7fc364b2b640 1 -- 172.21.15.33:0/1308748657 learned_addr learned my addr 172.21.15.33:0/1308748657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.658+0000 7fc364b2b640 1 -- 172.21.15.33:0/1308748657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d84260 msgr2=0x563168dc2e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:31:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.658+0000 7fc364b2b640 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d84260 0x563168dc2e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.658+0000 7fc364b2b640 1 -- 172.21.15.33:0/1308748657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d92a50 msgr2=0x563168dc4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.658+0000 7fc364b2b640 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d92a50 0x563168dc4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.658+0000 7fc364b2b640 1 -- 172.21.15.33:0/1308748657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563168dc4970 con 0x563168d911a0 2024-04-24T23:31:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.660+0000 7fc364b2b640 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168dc3370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc35c02d020 tx=0x7fc35c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7860cd345530b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.660+0000 7fc348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:31:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.660+0000 7fc348ff9640 1 -- 172.21.15.33:0/1308748657 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c05f070 con 0x563168d911a0 2024-04-24T23:31:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.660+0000 7fc348ff9640 1 -- 172.21.15.33:0/1308748657 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc35c030df0 con 0x563168d911a0 2024-04-24T23:31:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.662+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35000be10 con 0x563168d911a0 2024-04-24T23:31:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.664+0000 7fc348ff9640 1 -- 172.21.15.33:0/1308748657 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc35c037040 con 0x563168d911a0 2024-04-24T23:31:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.664+0000 7fc36a37b740 10 client.28897 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.664+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563168e17f80 con 0x563168d911a0 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.667+0000 7fc348ff9640 1 -- 172.21.15.33:0/1308748657 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35000be10 con 0x563168d911a0 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.667+0000 7fc348ff9640 1 -- 172.21.15.33:0/1308748657 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc35c062020 con 0x563168d911a0 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.667+0000 7fc36a37b740 10 client.28897 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.667+0000 7fc36a37b740 10 client.28897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.667+0000 7fc36a37b740 20 client.28897 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.667+0000 7fc36a37b740 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168e189a0 0x563168e1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.668+0000 7fc36a37b740 4 client.28897 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.668+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563168c736f0 con 0x563168e189a0 2024-04-24T23:31:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.668+0000 7fc36532c640 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168e189a0 0x563168e1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.672+0000 7fc36532c640 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168e189a0 0x563168e1adc0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf438680d78f82b server_cookie=14bfba559bc5f426 in_seq=0 out_seq=0 2024-04-24T23:31:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.672+0000 7fc348ff9640 10 client.28897 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.672+0000 7fc348ff9640 1 -- 172.21.15.33:0/1308748657 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563168c736f0 con 0x563168e189a0 2024-04-24T23:31:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.672+0000 7fc348ff9640 10 client.28897 handle_command_reply: tid=0 2024-04-24T23:31:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.672+0000 7fc36a37b740 1 client.28897 shutdown 2024-04-24T23:31:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.673+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 msgr2=0x563168dc3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.673+0000 7fc36a37b740 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168dc3370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc35c02d020 tx=0x7fc35c0045b0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.674+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 shutdown_connections 2024-04-24T23:31:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.674+0000 7fc36a37b740 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563168e189a0 0x563168e1adc0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.674+0000 7fc36a37b740 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563168d92a50 0x563168dc4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.674+0000 7fc36a37b740 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563168d911a0 0x563168dc3370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.674+0000 7fc36a37b740 1 --2- 172.21.15.33:0/1308748657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563168d84260 0x563168dc2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.674+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 >> 172.21.15.33:0/1308748657 conn(0x563168d858b0 msgr2=0x563168d8a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.675+0000 7fc348ff9640 0 client.28897 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:36.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.678+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 shutdown_connections 2024-04-24T23:31:36.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.679+0000 7fc36a37b740 1 -- 172.21.15.33:0/1308748657 wait complete. 2024-04-24T23:31:36.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.680+0000 7fc36a37b740 20 client.28897 trim_cache size 0 max 16384 2024-04-24T23:31:36.683 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:31:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:31:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:31:36.684 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:31:36.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.688+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc34c0a2d60 msgr2=0x7fc34c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.688+0000 7fc367590640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc34c0a2d60 0x7fc34c0a5220 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3601a1840 tx=0x7fc35c02aea0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.688+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36010a9a0 msgr2=0x7fc3601a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.688+0000 7fc367590640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36010a9a0 0x7fc3601a78e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc35800efc0 tx=0x7fc35800c800 comp rx=0 tx=0).stop 2024-04-24T23:31:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.691+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 shutdown_connections 2024-04-24T23:31:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.691+0000 7fc367590640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc34c0a2d60 0x7fc34c0a5220 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.691+0000 7fc367590640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36010a9a0 0x7fc3601a78e0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.692+0000 7fc367590640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc360109fe0 0x7fc3601a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.692+0000 7fc367590640 1 --2- 172.21.15.33:0/1369928843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360108e90 0x7fc3601a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.692+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 >> 172.21.15.33:0/1369928843 conn(0x7fc36006f620 msgr2=0x7fc36010d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.694+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 shutdown_connections 2024-04-24T23:31:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.694+0000 7fc367590640 1 -- 172.21.15.33:0/1369928843 wait complete. 2024-04-24T23:31:36.715 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:31:36.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 -- 172.21.15.33:0/723244081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860101890 msgr2=0x7f4860101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 --2- 172.21.15.33:0/723244081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860101890 0x7f4860101d10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4850009860 tx=0x7f485002a360 comp rx=0 tx=0).stop 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 -- 172.21.15.33:0/723244081 shutdown_connections 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 --2- 172.21.15.33:0/723244081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860102250 0x7f4860108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 --2- 172.21.15.33:0/723244081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860101890 0x7f4860101d10 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 --2- 172.21.15.33:0/723244081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48601006a0 0x7f4860100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 -- 172.21.15.33:0/723244081 >> 172.21.15.33:0/723244081 conn(0x7f48600fbc30 msgr2=0x7f48600fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 -- 172.21.15.33:0/723244081 shutdown_connections 2024-04-24T23:31:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.875+0000 7f4864c58640 1 -- 172.21.15.33:0/723244081 wait complete. 2024-04-24T23:31:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 Processor -- start 2024-04-24T23:31:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 -- start start 2024-04-24T23:31:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48601006a0 0x7f486019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 0x7f486019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860102250 0x7f48601a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f485e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48601006a0 0x7f486019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f485dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 0x7f486019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f485dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 0x7f486019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48124/0 (socket says 172.21.15.33:48124) 2024-04-24T23:31:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f485dd9b640 1 -- 172.21.15.33:0/2874288905 learned_addr learned my addr 172.21.15.33:0/2874288905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f486010ab00 con 0x7f4860102250 2024-04-24T23:31:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f486010a980 con 0x7f4860101890 2024-04-24T23:31:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.877+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f486010ac80 con 0x7f48601006a0 2024-04-24T23:31:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485ed9d640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860102250 0x7f48601a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485dd9b640 1 -- 172.21.15.33:0/2874288905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48601006a0 msgr2=0x7f486019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485dd9b640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48601006a0 0x7f486019dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485dd9b640 1 -- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860102250 msgr2=0x7f48601a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485dd9b640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860102250 0x7f48601a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485dd9b640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48601a5b00 con 0x7f4860101890 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f485dd9b640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 0x7f486019e420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f485002a950 tx=0x7f4850033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.878+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485003f020 con 0x7f4860101890 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.879+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4850030860 con 0x7f4860101890 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.879+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48601a5d90 con 0x7f4860101890 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.879+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850037040 con 0x7f4860101890 2024-04-24T23:31:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.879+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48601a61d0 con 0x7f4860101890 2024-04-24T23:31:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.880+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48601a6400 con 0x7f4860101890 2024-04-24T23:31:36.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.884+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4850042020 con 0x7f4860101890 2024-04-24T23:31:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.885+0000 7f484f7fe640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48480a2c10 0x7f48480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.885+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f48500efcd0 con 0x7f4860101890 2024-04-24T23:31:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.886+0000 7f485e59c640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48480a2c10 0x7f48480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.886+0000 7f485e59c640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48480a2c10 0x7f48480a50d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f48601016f0 tx=0x7f4854009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:36.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:36.890+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4850049020 con 0x7f4860101890 2024-04-24T23:31:36.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 147279 22.54 MB/sec execute 221 sec latency 111.228 ms 2024-04-24T23:31:37.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.230+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48601a65d0 con 0x7f4860101890 2024-04-24T23:31:37.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.231+0000 7f484f7fe640 1 -- 172.21.15.33:0/2874288905 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f48500bc840 con 0x7f4860101890 2024-04-24T23:31:37.232 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:31:37.233 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:31:37.233 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48480a2c10 msgr2=0x7f48480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48480a2c10 0x7f48480a50d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f48601016f0 tx=0x7f4854009260 comp rx=0 tx=0).stop 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 msgr2=0x7f486019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 0x7f486019e420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f485002a950 tx=0x7f4850033d90 comp rx=0 tx=0).stop 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 shutdown_connections 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f48480a2c10 0x7f48480a50d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860102250 0x7f48601a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4860101890 0x7f486019e420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 --2- 172.21.15.33:0/2874288905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48601006a0 0x7f486019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 >> 172.21.15.33:0/2874288905 conn(0x7f48600fbc30 msgr2=0x7f486010dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 shutdown_connections 2024-04-24T23:31:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.234+0000 7f4864c58640 1 -- 172.21.15.33:0/2874288905 wait complete. 2024-04-24T23:31:37.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:31:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 -- 172.21.15.33:0/507829841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c104ba0 msgr2=0x7f6f8c106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 --2- 172.21.15.33:0/507829841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c104ba0 0x7f6f8c106f90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009900 tx=0x7f6f8002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 -- 172.21.15.33:0/507829841 shutdown_connections 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 --2- 172.21.15.33:0/507829841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 --2- 172.21.15.33:0/507829841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c104ba0 0x7f6f8c106f90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 --2- 172.21.15.33:0/507829841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c102270 0x7f6f8c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 -- 172.21.15.33:0/507829841 >> 172.21.15.33:0/507829841 conn(0x7f6f8c0fbc00 msgr2=0x7f6f8c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 -- 172.21.15.33:0/507829841 shutdown_connections 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.363+0000 7f6f92895640 1 -- 172.21.15.33:0/507829841 wait complete. 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 Processor -- start 2024-04-24T23:31:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 -- start start 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 0x7f6f8c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c104ba0 0x7f6f8c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10d400 con 0x7f6f8c102270 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10d280 con 0x7f6f8c1074d0 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f92895640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10d580 con 0x7f6f8c104ba0 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f8bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 0x7f6f8c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f90e32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f90e32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48150/0 (socket says 172.21.15.33:48150) 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f90e32640 1 -- 172.21.15.33:0/156775495 learned_addr learned my addr 172.21.15.33:0/156775495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.364+0000 7f6f8bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 0x7f6f8c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54116/0 (socket says 172.21.15.33:54116) 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c104ba0 0x7f6f8c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8bfff640 1 -- 172.21.15.33:0/156775495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c104ba0 msgr2=0x7f6f8c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8bfff640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c104ba0 0x7f6f8c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8bfff640 1 -- 172.21.15.33:0/156775495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 msgr2=0x7f6f8c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8bfff640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8bfff640 1 -- 172.21.15.33:0/156775495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c1a1970 con 0x7f6f8c102270 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f90e32640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.365+0000 7f6f8bfff640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 0x7f6f8c199d50 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00e9f0 tx=0x7f6f7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.366+0000 7f6f897fa640 1 -- 172.21.15.33:0/156775495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ccc0 con 0x7f6f8c102270 2024-04-24T23:31:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.366+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c1a1c60 con 0x7f6f8c102270 2024-04-24T23:31:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.366+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c1a2180 con 0x7f6f8c102270 2024-04-24T23:31:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.366+0000 7f6f897fa640 1 -- 172.21.15.33:0/156775495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c005020 con 0x7f6f8c102270 2024-04-24T23:31:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.367+0000 7f6f897fa640 1 -- 172.21.15.33:0/156775495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c01c070 con 0x7f6f8c102270 2024-04-24T23:31:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.369+0000 7f6f9559b740 1 Processor -- start 2024-04-24T23:31:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.369+0000 7f6f9559b740 1 -- start start 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f897fa640 1 -- 172.21.15.33:0/156775495 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6f7c004020 con 0x7f6f8c102270 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d5230 0x55ea2e9d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9da720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9dd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f9559b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea2e939670 con 0x55ea2e9dad90 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f9559b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea2e8e1970 con 0x55ea2e9d5c20 2024-04-24T23:31:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f9559b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea2e935630 con 0x55ea2e9d5230 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9dd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54124/0 (socket says 172.21.15.33:54124) 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3560149378 learned_addr learned my addr 172.21.15.33:0/3560149378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f90e32640 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d5230 0x55ea2e9d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f8bfff640 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9da720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f90e32640 1 -- 172.21.15.33:0/3560149378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5c20 msgr2=0x55ea2e9da720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f90e32640 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9da720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f90e32640 1 -- 172.21.15.33:0/3560149378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 msgr2=0x55ea2e9dd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f90e32640 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9dd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f90e32640 1 -- 172.21.15.33:0/3560149378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2e905400 con 0x55ea2e9d5230 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.370+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9dd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f897fa640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f740a2d90 0x7f6f740a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f90e32640 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d5230 0x55ea2e9d5650 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f840117e0 tx=0x7f6f84011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17e3a59865c0156 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f8bfff640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f740a2d90 0x7f6f740a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f897fa640 1 -- 172.21.15.33:0/156775495 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f7c0cee20 con 0x7f6f8c102270 2024-04-24T23:31:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f727fc640 1 -- 172.21.15.33:0/3560149378 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84011310 con 0x55ea2e9d5230 2024-04-24T23:31:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f727fc640 1 -- 172.21.15.33:0/3560149378 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8400f9a0 con 0x55ea2e9d5230 2024-04-24T23:31:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.371+0000 7f6f727fc640 1 -- 172.21.15.33:0/3560149378 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84005730 con 0x55ea2e9d5230 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 -- 172.21.15.33:0/3560149378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d5230 msgr2=0x55ea2e9d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f8bfff640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f740a2d90 0x7f6f740a5250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0cc900 tx=0x7f6f7c0d2900 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d5230 0x55ea2e9d5650 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f840117e0 tx=0x7f6f84011ca0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 -- 172.21.15.33:0/3560149378 shutdown_connections 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9dd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9da720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3560149378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d5230 0x55ea2e9d5650 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 -- 172.21.15.33:0/3560149378 >> 172.21.15.33:0/3560149378 conn(0x55ea2e9d08a0 msgr2=0x55ea2e9d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 -- 172.21.15.33:0/3560149378 shutdown_connections 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 -- 172.21.15.33:0/3560149378 wait complete. 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.372+0000 7f6f9559b740 1 -- start start 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 Processor -- start 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea2e9ddd40 con 0x55ea2e9d5c20 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea2e8f5050 con 0x55ea2e9d5230 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea2e9ddec0 con 0x55ea2e9dad90 2024-04-24T23:31:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f9559b740 10 client.0 _ll_register_callbacks cb 0x55ea2e64cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9df7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54140/0 (socket says 172.21.15.33:54140) 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/628150158 learned_addr learned my addr 172.21.15.33:0/628150158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f90e32640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f90e32640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/628150158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9dad90 msgr2=0x55ea2e9e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 msgr2=0x55ea2e9df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/628150158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2e8c6dc0 con 0x55ea2e9d5c20 2024-04-24T23:31:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f90e32640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9df7f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009230 tx=0x7f6f80032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1fbcafcabaf753 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.373+0000 7f6f6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/628150158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80035040 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/628150158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f80004040 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8400f9a0 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/628150158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6f8004f050 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 10 client.19485 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ea2e64d3f0 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/628150158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8400f9a0 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/628150158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6f800047f0 con 0x55ea2e9d5c20 2024-04-24T23:31:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 10 client.19485 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 10 client.19485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 20 client.19485 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2e9eb190 0x55ea2e9ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 4 client.19485 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.374+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea2e946bb0 con 0x55ea2e9eb190 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.375+0000 7f6f90e32640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2e9eb190 0x55ea2e9ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.375+0000 7f6f90e32640 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2e9eb190 0x55ea2e9ed5f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ef8cba1241f9724 server_cookie=d69ab686b42344a5 in_seq=0 out_seq=0 2024-04-24T23:31:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.375+0000 7f6f6b7fe640 10 client.19485 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.380+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/628150158 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea2e946bb0 con 0x55ea2e9eb190 2024-04-24T23:31:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.380+0000 7f6f6b7fe640 10 client.19485 handle_command_reply: tid=0 2024-04-24T23:31:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 client.19485 shutdown 2024-04-24T23:31:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 msgr2=0x55ea2e9df7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9df7f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009230 tx=0x7f6f80032ed0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 shutdown_connections 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2e9eb190 0x55ea2e9ed5f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5c20 0x55ea2e9df7f0 secure :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009230 tx=0x7f6f80032ed0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 --2- 172.21.15.33:0/628150158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f6b7fe640 0 client.19485 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.381+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 >> 172.21.15.33:0/628150158 conn(0x55ea2e9d08a0 msgr2=0x55ea2e9cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.382+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 shutdown_connections 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.382+0000 7f6f9559b740 1 -- 172.21.15.33:0/628150158 wait complete. 2024-04-24T23:31:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.382+0000 7f6f9559b740 20 client.19485 trim_cache size 0 max 16384 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.399+0000 7f6f9559b740 1 Processor -- start 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.399+0000 7f6f9559b740 1 -- start start 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f9559b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78012460 con 0x55ea2e9dad90 2024-04-24T23:31:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f9559b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800fd30 con 0x55ea2e9d5230 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f9559b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800db80 con 0x55ea2e9d3890 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54146/0 (socket says 172.21.15.33:54146) 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3649824220 learned_addr learned my addr 172.21.15.33:0/3649824220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f90e32640 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f90e32640 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3649824220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 msgr2=0x55ea2e9d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3649824220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 msgr2=0x55ea2e9da2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9da2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.400+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3649824220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2e7177d0 con 0x55ea2e9dad90 2024-04-24T23:31:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.401+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e67f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6f80004bf0 tx=0x7f6f800058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f550e4d34e83d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.401+0000 7f6f71ffb640 1 -- 172.21.15.33:0/3649824220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80057020 con 0x55ea2e9dad90 2024-04-24T23:31:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.401+0000 7f6f71ffb640 1 -- 172.21.15.33:0/3649824220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f80031cd0 con 0x55ea2e9dad90 2024-04-24T23:31:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.401+0000 7f6f71ffb640 1 -- 172.21.15.33:0/3649824220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80035070 con 0x55ea2e9dad90 2024-04-24T23:31:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.402+0000 7f6f9559b740 1 -- 172.21.15.33:0/3649824220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 msgr2=0x55ea2e9e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.402+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e67f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6f80004bf0 tx=0x7f6f800058d0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 -- 172.21.15.33:0/3649824220 shutdown_connections 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e67f0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 --2- 172.21.15.33:0/3649824220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9d5230 0x55ea2e9da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 -- 172.21.15.33:0/3649824220 >> 172.21.15.33:0/3649824220 conn(0x55ea2e9d08a0 msgr2=0x55ea2e9cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 -- 172.21.15.33:0/3649824220 shutdown_connections 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 -- 172.21.15.33:0/3649824220 wait complete. 2024-04-24T23:31:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.404+0000 7f6f9559b740 1 -- start start 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 Processor -- start 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5230 0x55ea2e9e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f90e32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5230 0x55ea2e9e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f90e32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36760/0 (socket says 172.21.15.33:36760) 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5230 0x55ea2e9e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54162/0 (socket says 172.21.15.33:54162) 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f90e32640 1 -- 172.21.15.33:0/722023725 learned_addr learned my addr 172.21.15.33:0/722023725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800e1f0 con 0x55ea2e9d5230 2024-04-24T23:31:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800f660 con 0x55ea2e9dad90 2024-04-24T23:31:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800e8c0 con 0x55ea2e9d3890 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.405+0000 7f6f90e32640 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f90e32640 1 -- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9dad90 msgr2=0x55ea2e9e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f90e32640 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f90e32640 1 -- 172.21.15.33:0/722023725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5230 msgr2=0x55ea2e9e43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f90e32640 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5230 0x55ea2e9e43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f90e32640 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2e9e59c0 con 0x55ea2e9d3890 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f9559b740 10 client.0 _ll_register_callbacks cb 0x7f6f80005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f80031cd0 con 0x55ea2e9d3890 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f90e32640 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9e3e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005560 tx=0x7f6f84009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edecdf9d7cea0237 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f6b7fe640 10 client.28921 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/722023725 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8401e040 con 0x55ea2e9d3890 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.406+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/722023725 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f84011420 con 0x55ea2e9d3890 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/722023725 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6f8400f8c0 con 0x55ea2e9d3890 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 10 client.28921 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55ea2e9e6a70 con 0x55ea2e9d3890 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/722023725 <== mon.1 v2:172.21.15.117:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6f84024040 con 0x55ea2e9d3890 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 10 client.28921 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 10 client.28921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:31:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 20 client.28921 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2ea51ae0 0x55ea2ea53f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 4 client.28921 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea2e940f00 con 0x55ea2ea51ae0 2024-04-24T23:31:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2ea51ae0 0x55ea2ea53f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f8b7fe640 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2ea51ae0 0x55ea2ea53f80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90d2bcf0339bcd78 server_cookie=3c06c89152992f6e in_seq=0 out_seq=0 2024-04-24T23:31:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.407+0000 7f6f6b7fe640 10 client.28921 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f6b7fe640 1 -- 172.21.15.33:0/722023725 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea2e940f00 con 0x55ea2ea51ae0 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f6b7fe640 10 client.28921 handle_command_reply: tid=0 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 client.28921 shutdown 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 msgr2=0x55ea2e9e3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9e3e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005560 tx=0x7f6f84009500 comp rx=0 tx=0).stop 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 shutdown_connections 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55ea2ea51ae0 0x55ea2ea53f80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea2e9dad90 0x55ea2e9e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f6b7fe640 0 client.28921 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea2e9d5230 0x55ea2e9e43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 --2- 172.21.15.33:0/722023725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea2e9d3890 0x55ea2e9e3e80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 >> 172.21.15.33:0/722023725 conn(0x55ea2e9d08a0 msgr2=0x55ea2e9cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 shutdown_connections 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.411+0000 7f6f9559b740 1 -- 172.21.15.33:0/722023725 wait complete. 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.412+0000 7f6f9559b740 20 client.28921 trim_cache size 0 max 16384 2024-04-24T23:31:37.414 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.412+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f740a2d90 msgr2=0x7f6f740a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.412+0000 7f6f92895640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f740a2d90 0x7f6f740a5250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0cc900 tx=0x7f6f7c0d2900 comp rx=0 tx=0).stop 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.412+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 msgr2=0x7f6f8c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.412+0000 7f6f92895640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 0x7f6f8c199d50 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00e9f0 tx=0x7f6f7c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 shutdown_connections 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6f740a2d90 0x7f6f740a5250 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f8c1074d0 0x7f6f8c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8c104ba0 0x7f6f8c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 --2- 172.21.15.33:0/156775495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f8c102270 0x7f6f8c199d50 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 >> 172.21.15.33:0/156775495 conn(0x7f6f8c0fbc00 msgr2=0x7f6f8c101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 shutdown_connections 2024-04-24T23:31:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.413+0000 7f6f92895640 1 -- 172.21.15.33:0/156775495 wait complete. 2024-04-24T23:31:37.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:31:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 -- 172.21.15.33:0/2400319675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50104b90 msgr2=0x7f6d50105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 --2- 172.21.15.33:0/2400319675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50104b90 0x7f6d50105010 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009900 tx=0x7f6d4002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 -- 172.21.15.33:0/2400319675 shutdown_connections 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 --2- 172.21.15.33:0/2400319675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50105550 0x7f6d5010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 --2- 172.21.15.33:0/2400319675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50104b90 0x7f6d50105010 secure :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009900 tx=0x7f6d4002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 --2- 172.21.15.33:0/2400319675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d501039a0 0x7f6d50103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 -- 172.21.15.33:0/2400319675 >> 172.21.15.33:0/2400319675 conn(0x7f6d500feef0 msgr2=0x7f6d50101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 -- 172.21.15.33:0/2400319675 shutdown_connections 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.578+0000 7f6d572d2640 1 -- 172.21.15.33:0/2400319675 wait complete. 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 Processor -- start 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 -- start start 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 0x7f6d50199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50105550 0x7f6d5019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d5019f440 0x7f6d501a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010f670 con 0x7f6d501039a0 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010f4f0 con 0x7f6d5019f440 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.579+0000 7f6d572d2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010f7f0 con 0x7f6d50105550 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 0x7f6d50199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 0x7f6d50199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54192/0 (socket says 172.21.15.33:54192) 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 -- 172.21.15.33:0/4054968398 learned_addr learned my addr 172.21.15.33:0/4054968398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5586f640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d5019f440 0x7f6d501a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 -- 172.21.15.33:0/4054968398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50105550 msgr2=0x7f6d5019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50105550 0x7f6d5019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 -- 172.21.15.33:0/4054968398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d5019f440 msgr2=0x7f6d501a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d5019f440 0x7f6d501a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.580+0000 7f6d5506e640 1 -- 172.21.15.33:0/4054968398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d501a1ea0 con 0x7f6d501039a0 2024-04-24T23:31:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.581+0000 7f6d5506e640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 0x7f6d50199ea0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e9c0 tx=0x7f6d4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.581+0000 7f6d467fc640 1 -- 172.21.15.33:0/4054968398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00cd80 con 0x7f6d501039a0 2024-04-24T23:31:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.581+0000 7f6d467fc640 1 -- 172.21.15.33:0/4054968398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d4c0035f0 con 0x7f6d501039a0 2024-04-24T23:31:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.581+0000 7f6d467fc640 1 -- 172.21.15.33:0/4054968398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c010690 con 0x7f6d501039a0 2024-04-24T23:31:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.581+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d501a2190 con 0x7f6d501039a0 2024-04-24T23:31:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.581+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d501a2690 con 0x7f6d501039a0 2024-04-24T23:31:37.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 Processor -- start 2024-04-24T23:31:37.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 -- start start 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098ed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098f0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55570984c670 con 0x5557098e8230 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557097f4970 con 0x5557098edd90 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d59fde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555709848630 con 0x5557098e8c20 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54194/0 (socket says 172.21.15.33:54194) 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d5586f640 1 -- 172.21.15.33:0/634948326 learned_addr learned my addr 172.21.15.33:0/634948326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d5486d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d467fc640 1 -- 172.21.15.33:0/4054968398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6d4c0037a0 con 0x7f6d501039a0 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.586+0000 7f6d5506e640 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5586f640 1 -- 172.21.15.33:0/634948326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 msgr2=0x5557098ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5586f640 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5586f640 1 -- 172.21.15.33:0/634948326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 msgr2=0x5557098f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5586f640 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5586f640 1 -- 172.21.15.33:0/634948326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555709818400 con 0x5557098e8230 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5506e640 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5486d640 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5586f640 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e8650 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009fc0 tx=0x7f6d48010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e84bc2f5eb5679 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d467fc640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d3c0a2ce0 0x7f6d3c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d3b7fe640 1 -- 172.21.15.33:0/634948326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48003b70 con 0x5557098e8230 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d3b7fe640 1 -- 172.21.15.33:0/634948326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d48003d10 con 0x5557098e8230 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d3b7fe640 1 -- 172.21.15.33:0/634948326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48011aa0 con 0x5557098e8230 2024-04-24T23:31:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d467fc640 1 -- 172.21.15.33:0/4054968398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d4c0cf7e0 con 0x7f6d501039a0 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.587+0000 7f6d5506e640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d3c0a2ce0 0x7f6d3c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d5506e640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d3c0a2ce0 0x7f6d3c0a51a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cd2c0 tx=0x7f6d4c009790 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 -- 172.21.15.33:0/634948326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 msgr2=0x5557098e8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e8650 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009fc0 tx=0x7f6d48010860 comp rx=0 tx=0).stop 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 -- 172.21.15.33:0/634948326 shutdown_connections 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 --2- 172.21.15.33:0/634948326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e8650 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 -- 172.21.15.33:0/634948326 >> 172.21.15.33:0/634948326 conn(0x5557098e38a0 msgr2=0x5557098e3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 -- 172.21.15.33:0/634948326 shutdown_connections 2024-04-24T23:31:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.588+0000 7f6d59fde740 1 -- 172.21.15.33:0/634948326 wait complete. 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 -- start start 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 Processor -- start 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098fb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098edd90 0x5557099020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557098f0d40 con 0x5557098edd90 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555709808050 con 0x5557098e8230 2024-04-24T23:31:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557098f0ec0 con 0x5557098e8c20 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d59fde740 10 client.0 _ll_register_callbacks cb 0x55570955fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098fab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48212/0 (socket says 172.21.15.33:48212) 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d5586f640 1 -- 172.21.15.33:0/1955917507 learned_addr learned my addr 172.21.15.33:0/1955917507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098edd90 0x5557099020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.589+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d5586f640 1 -- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 msgr2=0x5557098fb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098fb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d5586f640 1 -- 172.21.15.33:0/1955917507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098edd90 msgr2=0x5557099020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098edd90 0x5557099020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d5586f640 1 -- 172.21.15.33:0/1955917507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557097d9dc0 con 0x5557098e8230 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098fab40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009a30 tx=0x7f6d48011500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2377351640a04209 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:31:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1955917507 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48024040 con 0x5557098e8230 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d48003d10 con 0x5557098e8230 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.590+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1955917507 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d48004c40 con 0x5557098e8230 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.591+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1955917507 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6d48033050 con 0x5557098e8230 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.591+0000 7f6d59fde740 10 client.28988 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.591+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5557095603f0 con 0x5557098e8230 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.594+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1955917507 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d48003d10 con 0x5557098e8230 2024-04-24T23:31:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1955917507 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6d48005030 con 0x5557098e8230 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d59fde740 10 client.28988 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d59fde740 10 client.28988 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d59fde740 20 client.28988 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x555709906620 0x555709908a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d59fde740 4 client.28988 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555709859bb0 con 0x555709906620 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.595+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x555709906620 0x555709908a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.596+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x555709906620 0x555709908a30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a79b71a8fe3329f server_cookie=2d7e01afa149fd31 in_seq=0 out_seq=0 2024-04-24T23:31:37.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.596+0000 7f6d38ff9640 10 client.28988 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1955917507 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555709859bb0 con 0x555709906620 2024-04-24T23:31:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d38ff9640 10 client.28988 handle_command_reply: tid=0 2024-04-24T23:31:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 client.28988 shutdown 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 msgr2=0x5557098fab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098fab40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009a30 tx=0x7f6d48011500 comp rx=0 tx=0).stop 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 shutdown_connections 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x555709906620 0x555709908a30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098edd90 0x5557099020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x5557098fb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1955917507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098fab40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d38ff9640 0 client.28988 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 >> 172.21.15.33:0/1955917507 conn(0x5557098e38a0 msgr2=0x5557098e2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 shutdown_connections 2024-04-24T23:31:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 1 -- 172.21.15.33:0/1955917507 wait complete. 2024-04-24T23:31:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.597+0000 7f6d59fde740 20 client.28988 trim_cache size 0 max 16384 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.613+0000 7f6d59fde740 1 Processor -- start 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 -- start start 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x555709902050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098e27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501ab140 con 0x5557098e8230 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501a8a10 con 0x5557098edd90 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d59fde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501a6880 con 0x5557098e8c20 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54214/0 (socket says 172.21.15.33:54214) 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5486d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x555709902050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5586f640 1 -- 172.21.15.33:0/1090036796 learned_addr learned my addr 172.21.15.33:0/1090036796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098e27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5486d640 1 -- 172.21.15.33:0/1090036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 msgr2=0x5557098e27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098e27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5486d640 1 -- 172.21.15.33:0/1090036796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 msgr2=0x5557098e22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5486d640 1 -- 172.21.15.33:0/1090036796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570962a7d0 con 0x5557098e8c20 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.614+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.615+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x555709902050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d40008000 tx=0x7f6d40031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63041465b59f83a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.615+0000 7f6d3affd640 1 -- 172.21.15.33:0/1090036796 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40035040 con 0x5557098e8c20 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.615+0000 7f6d3affd640 1 -- 172.21.15.33:0/1090036796 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d40045e20 con 0x5557098e8c20 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.615+0000 7f6d3affd640 1 -- 172.21.15.33:0/1090036796 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400454c0 con 0x5557098e8c20 2024-04-24T23:31:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 -- 172.21.15.33:0/1090036796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 msgr2=0x555709902050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x555709902050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d40008000 tx=0x7f6d40031400 comp rx=0 tx=0).stop 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 -- 172.21.15.33:0/1090036796 shutdown_connections 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098edd90 0x5557098e27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098e8c20 0x555709902050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1090036796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8230 0x5557098e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 -- 172.21.15.33:0/1090036796 >> 172.21.15.33:0/1090036796 conn(0x5557098e38a0 msgr2=0x5557098e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 -- 172.21.15.33:0/1090036796 shutdown_connections 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 -- 172.21.15.33:0/1090036796 wait complete. 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 -- start start 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 Processor -- start 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.616+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8c20 0x5557098ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d59fde740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098edd90 0x555709900920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d59fde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501a6ed0 con 0x5557098e8c20 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d59fde740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501a8340 con 0x5557098e8230 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d59fde740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501a75a0 con 0x5557098edd90 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d59fde740 10 client.0 _ll_register_callbacks cb 0x7f6d4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098edd90 0x555709900920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8c20 0x5557098ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5586f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48228/0 (socket says 172.21.15.33:48228) 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5586f640 1 -- 172.21.15.33:0/1434508553 learned_addr learned my addr 172.21.15.33:0/1434508553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 -- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098edd90 msgr2=0x555709900920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098edd90 0x555709900920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 -- 172.21.15.33:0/1434508553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 msgr2=0x5557098ff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098ff3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 -- 172.21.15.33:0/1434508553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555709900f10 con 0x5557098e8c20 2024-04-24T23:31:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d5486d640 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8c20 0x5557098ff910 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x5557098ed6c0 tx=0x7f6d40009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d703d1e71b874ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.617+0000 7f6d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1434508553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40035030 con 0x5557098e8c20 2024-04-24T23:31:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d40045e20 con 0x5557098e8c20 2024-04-24T23:31:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1434508553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d40005200 con 0x5557098e8c20 2024-04-24T23:31:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1434508553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6d40058050 con 0x5557098e8c20 2024-04-24T23:31:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d59fde740 10 client.19509 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555709902210 con 0x5557098e8c20 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.618+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1434508553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d40045e20 con 0x5557098e8c20 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1434508553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6d400316b0 con 0x5557098e8c20 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 10 client.19509 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 10 client.19509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 20 client.19509 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55570996d040 0x55570996f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 4 client.19509 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555709853f00 con 0x55570996d040 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55570996d040 0x55570996f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d38ff9640 10 client.19509 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d5586f640 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55570996d040 0x55570996f460 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=818626da16ab196b server_cookie=407d4d73c2dbc6a2 in_seq=0 out_seq=0 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d38ff9640 1 -- 172.21.15.33:0/1434508553 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555709853f00 con 0x55570996d040 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d38ff9640 10 client.19509 handle_command_reply: tid=0 2024-04-24T23:31:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 1 client.19509 shutdown 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.619+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8c20 msgr2=0x5557098ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8c20 0x5557098ff910 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x5557098ed6c0 tx=0x7f6d40009680 comp rx=0 tx=0).stop 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 shutdown_connections 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55570996d040 0x55570996f460 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d38ff9640 0 client.19509 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557098edd90 0x555709900920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557098e8c20 0x5557098ff910 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 --2- 172.21.15.33:0/1434508553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557098e8230 0x5557098ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 >> 172.21.15.33:0/1434508553 conn(0x5557098e38a0 msgr2=0x5557098e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 shutdown_connections 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 1 -- 172.21.15.33:0/1434508553 wait complete. 2024-04-24T23:31:37.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.620+0000 7f6d59fde740 20 client.19509 trim_cache size 0 max 16384 2024-04-24T23:31:37.628 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d3c0a2ce0 msgr2=0x7f6d3c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d3c0a2ce0 0x7f6d3c0a51a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cd2c0 tx=0x7f6d4c009790 comp rx=0 tx=0).stop 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 msgr2=0x7f6d50199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 0x7f6d50199ea0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e9c0 tx=0x7f6d4c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 shutdown_connections 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d3c0a2ce0 0x7f6d3c0a51a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d5019f440 0x7f6d501a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50105550 0x7f6d5019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 --2- 172.21.15.33:0/4054968398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d501039a0 0x7f6d50199ea0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 >> 172.21.15.33:0/4054968398 conn(0x7f6d500feef0 msgr2=0x7f6d50107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 shutdown_connections 2024-04-24T23:31:37.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.621+0000 7f6d572d2640 1 -- 172.21.15.33:0/4054968398 wait complete. 2024-04-24T23:31:37.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 -- 172.21.15.33:0/3431550837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 msgr2=0x7f03a8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3431550837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 0x7f03a8109bd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009ca0 tx=0x7f03a002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 -- 172.21.15.33:0/3431550837 shutdown_connections 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3431550837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 0x7f03a8109bd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3431550837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a8102980 0x7f03a8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3431550837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8101790 0x7f03a8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 -- 172.21.15.33:0/3431550837 >> 172.21.15.33:0/3431550837 conn(0x7f03a80fcd20 msgr2=0x7f03a80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 -- 172.21.15.33:0/3431550837 shutdown_connections 2024-04-24T23:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.765+0000 7f03acbc1640 1 -- 172.21.15.33:0/3431550837 wait complete. 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.766+0000 7f03acbc1640 1 Processor -- start 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.766+0000 7f03acbc1640 1 -- start start 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.766+0000 7f03acbc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a8101790 0x7f03a8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.766+0000 7f03acbc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 0x7f03a8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.766+0000 7f03acbc1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 0x7f03a819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.766+0000 7f03acbc1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03a810d410 con 0x7f03a8103340 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f03acbc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03a810d290 con 0x7f03a8102980 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f03acbc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03a810d590 con 0x7f03a8101790 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 0x7f03a8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f03a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a8101790 0x7f03a8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 0x7f03a8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48232/0 (socket says 172.21.15.33:48232) 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 -- 172.21.15.33:0/3764017181 learned_addr learned my addr 172.21.15.33:0/3764017181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 0x7f03a819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 -- 172.21.15.33:0/3764017181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a8101790 msgr2=0x7f03a8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a8101790 0x7f03a8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 -- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 msgr2=0x7f03a819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 0x7f03a819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 -- 172.21.15.33:0/3764017181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a819f6a0 con 0x7f03a8102980 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 0x7f03a8197fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f039000e9f0 tx=0x7f039000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.767+0000 7f039ffff640 1 -- 172.21.15.33:0/3764017181 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039000cd80 con 0x7f03a8102980 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.768+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a80739d0 con 0x7f03a8102980 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.768+0000 7f039ffff640 1 -- 172.21.15.33:0/3764017181 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0390005500 con 0x7f03a8102980 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.768+0000 7f039ffff640 1 -- 172.21.15.33:0/3764017181 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390010690 con 0x7f03a8102980 2024-04-24T23:31:37.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.768+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a8073f20 con 0x7f03a8102980 2024-04-24T23:31:37.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.770+0000 7f039ffff640 1 -- 172.21.15.33:0/3764017181 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f03900046c0 con 0x7f03a8102980 2024-04-24T23:31:37.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 Processor -- start 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 -- start start 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa926650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa92b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa92e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa88a670 con 0x55d9fa926c20 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa832970 con 0x55d9fa926230 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03af9aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa886630 con 0x55d9fa92bd90 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03a6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa926650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03a659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa92b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03a6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa926650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48244/0 (socket says 172.21.15.33:48244) 2024-04-24T23:31:37.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f03a6d9d640 1 -- 172.21.15.33:0/3982192334 learned_addr learned my addr 172.21.15.33:0/3982192334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f039ffff640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03940a2b90 0x7f03940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.771+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa92e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03940a2b90 0x7f03940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 -- 172.21.15.33:0/3982192334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 msgr2=0x55d9fa92e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa92e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 -- 172.21.15.33:0/3982192334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 msgr2=0x55d9fa92b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa92b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 -- 172.21.15.33:0/3982192334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9fa856400 con 0x55d9fa926230 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f039ffff640 1 -- 172.21.15.33:0/3764017181 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03900cf860 con 0x7f03a8102980 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a659c640 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa92b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa926650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005cc0 tx=0x7f03a0031e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47553834ba666c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03940a2b90 0x7f03940a5050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03a0047000 tx=0x7f03a0049000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f039cd99640 1 -- 172.21.15.33:0/3982192334 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a004e070 con 0x55d9fa926230 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.772+0000 7f039cd99640 1 -- 172.21.15.33:0/3982192334 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a0036070 con 0x55d9fa926230 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f039cd99640 1 -- 172.21.15.33:0/3982192334 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a002fa00 con 0x55d9fa926230 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 -- 172.21.15.33:0/3982192334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 msgr2=0x55d9fa926650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa926650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005cc0 tx=0x7f03a0031e00 comp rx=0 tx=0).stop 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 -- 172.21.15.33:0/3982192334 shutdown_connections 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa92e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa92b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3982192334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa926650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.773+0000 7f03af9aa740 1 -- 172.21.15.33:0/3982192334 >> 172.21.15.33:0/3982192334 conn(0x55d9fa9218a0 msgr2=0x55d9fa921ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.774+0000 7f03af9aa740 1 -- 172.21.15.33:0/3982192334 shutdown_connections 2024-04-24T23:31:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.774+0000 7f03af9aa740 1 -- 172.21.15.33:0/3982192334 wait complete. 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.774+0000 7f03af9aa740 1 -- start start 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.774+0000 7f03af9aa740 1 Processor -- start 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa930200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa930740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa937780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa92ebb0 con 0x55d9fa926c20 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa930200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa930200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48246/0 (socket says 172.21.15.33:48246) 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03a659c640 1 -- 172.21.15.33:0/3798165554 learned_addr learned my addr 172.21.15.33:0/3798165554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa846050 con 0x55d9fa926230 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa930740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa92ed30 con 0x55d9fa92bd90 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03af9aa740 10 client.0 _ll_register_callbacks cb 0x55d9fa59dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.775+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa937780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f03a659c640 1 -- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 msgr2=0x55d9fa937780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f03a659c640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa937780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f03a659c640 1 -- 172.21.15.33:0/3798165554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 msgr2=0x55d9fa930740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f03a659c640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa930740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f03a659c640 1 -- 172.21.15.33:0/3798165554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9fa817dc0 con 0x55d9fa926230 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f03a659c640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa930200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0398009900 tx=0x7f0398009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11dd94040ccb2922 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.776+0000 7f038dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f038dffb640 1 -- 172.21.15.33:0/3798165554 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800bcf0 con 0x55d9fa926230 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f038dffb640 1 -- 172.21.15.33:0/3798165554 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0398004030 con 0x55d9fa926230 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a0036070 con 0x55d9fa926230 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f038dffb640 1 -- 172.21.15.33:0/3798165554 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f039802a050 con 0x55d9fa926230 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f03af9aa740 10 client.29006 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d9fa59e3f0 con 0x55d9fa926230 2024-04-24T23:31:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.777+0000 7f038dffb640 1 -- 172.21.15.33:0/3798165554 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a0036070 con 0x55d9fa926230 2024-04-24T23:31:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f038dffb640 1 -- 172.21.15.33:0/3798165554 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f03980035f0 con 0x55d9fa926230 2024-04-24T23:31:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f03af9aa740 10 client.29006 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f03af9aa740 10 client.29006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f03af9aa740 20 client.29006 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa93c130 0x55d9fa93e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f03af9aa740 4 client.29006 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9fa897bb0 con 0x55d9fa93c130 2024-04-24T23:31:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.779+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa93c130 0x55d9fa93e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.780+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa93c130 0x55d9fa93e5a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4651739e04b4d2 server_cookie=aade4cc2a58c742e in_seq=0 out_seq=0 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.780+0000 7f038dffb640 10 client.29006 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f038dffb640 1 -- 172.21.15.33:0/3798165554 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9fa897bb0 con 0x55d9fa93c130 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f038dffb640 10 client.29006 handle_command_reply: tid=0 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 client.29006 shutdown 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 msgr2=0x55d9fa930200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa930200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0398009900 tx=0x7f0398009d00 comp rx=0 tx=0).stop 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 shutdown_connections 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa93c130 0x55d9fa93e5a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa92bd90 0x55d9fa937780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926c20 0x55d9fa930740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f038dffb640 0 client.29006 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3798165554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926230 0x55d9fa930200 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.781+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 >> 172.21.15.33:0/3798165554 conn(0x55d9fa9218a0 msgr2=0x55d9fa920b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.782+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 shutdown_connections 2024-04-24T23:31:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.782+0000 7f03af9aa740 1 -- 172.21.15.33:0/3798165554 wait complete. 2024-04-24T23:31:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.782+0000 7f03af9aa740 20 client.29006 trim_cache size 0 max 16384 2024-04-24T23:31:37.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.799+0000 7f03af9aa740 1 Processor -- start 2024-04-24T23:31:37.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 -- start start 2024-04-24T23:31:37.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 0x55d9fa9202b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926c20 0x55d9fa937740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa92bd90 0x55d9fa92f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81aaa60 con 0x55d9fa926230 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81a8330 con 0x55d9fa92bd90 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03af9aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81a6180 con 0x55d9fa926c20 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03a659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 0x55d9fa9202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03a659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 0x55d9fa9202b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54276/0 (socket says 172.21.15.33:54276) 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926c20 0x55d9fa937740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03a659c640 1 -- 172.21.15.33:0/3389162192 learned_addr learned my addr 172.21.15.33:0/3389162192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.802+0000 7f03a6d9d640 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa92bd90 0x55d9fa92f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f03a659c640 1 -- 172.21.15.33:0/3389162192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926c20 msgr2=0x55d9fa937740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f03a659c640 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926c20 0x55d9fa937740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f03a659c640 1 -- 172.21.15.33:0/3389162192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa92bd90 msgr2=0x55d9fa92f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f03a659c640 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa92bd90 0x55d9fa92f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f03a659c640 1 -- 172.21.15.33:0/3389162192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9fa6687d0 con 0x55d9fa926230 2024-04-24T23:31:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f039dd9b640 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926c20 0x55d9fa937740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f03a659c640 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 0x55d9fa9202b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f039800a010 tx=0x7f039800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2be5d0794a8165 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f038ffff640 1 -- 172.21.15.33:0/3389162192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980035f0 con 0x55d9fa926230 2024-04-24T23:31:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f038ffff640 1 -- 172.21.15.33:0/3389162192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0398032020 con 0x55d9fa926230 2024-04-24T23:31:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.803+0000 7f038ffff640 1 -- 172.21.15.33:0/3389162192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398003da0 con 0x55d9fa926230 2024-04-24T23:31:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 -- 172.21.15.33:0/3389162192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 msgr2=0x55d9fa9202b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 0x55d9fa9202b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f039800a010 tx=0x7f039800b690 comp rx=0 tx=0).stop 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 -- 172.21.15.33:0/3389162192 shutdown_connections 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa92bd90 0x55d9fa92f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926c20 0x55d9fa937740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 --2- 172.21.15.33:0/3389162192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa926230 0x55d9fa9202b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 -- 172.21.15.33:0/3389162192 >> 172.21.15.33:0/3389162192 conn(0x55d9fa9218a0 msgr2=0x55d9fa927eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 -- 172.21.15.33:0/3389162192 shutdown_connections 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.804+0000 7f03af9aa740 1 -- 172.21.15.33:0/3389162192 wait complete. 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 -- start start 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 Processor -- start 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926230 0x55d9fa9350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926c20 0x55d9fa935630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa92bd90 0x55d9fa9365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81a67f0 con 0x55d9fa92bd90 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81a7c60 con 0x55d9fa926c20 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81a6ec0 con 0x55d9fa926230 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03af9aa740 10 client.0 _ll_register_callbacks cb 0x7f0398010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa92bd90 0x55d9fa9365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa92bd90 0x55d9fa9365f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54292/0 (socket says 172.21.15.33:54292) 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926c20 0x55d9fa935630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03a6d9d640 1 -- 172.21.15.33:0/4225603387 learned_addr learned my addr 172.21.15.33:0/4225603387 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.805+0000 7f03a659c640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926230 0x55d9fa9350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03a659c640 1 -- 172.21.15.33:0/4225603387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926c20 msgr2=0x55d9fa935630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03a659c640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926c20 0x55d9fa935630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03a659c640 1 -- 172.21.15.33:0/4225603387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa92bd90 msgr2=0x55d9fa9365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03a659c640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa92bd90 0x55d9fa9365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03a659c640 1 -- 172.21.15.33:0/4225603387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9fa936be0 con 0x55d9fa926230 2024-04-24T23:31:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f039dd9b640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926c20 0x55d9fa935630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03a659c640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926230 0x55d9fa9350f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55d9fa93eba0 tx=0x7f03980057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c0a3bdf179e8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f038dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f038dffb640 1 -- 172.21.15.33:0/4225603387 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800bd00 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0398032020 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f038dffb640 1 -- 172.21.15.33:0/4225603387 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0398003b40 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f038dffb640 1 -- 172.21.15.33:0/4225603387 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0398046050 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03af9aa740 10 client.28951 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d9fa9a2940 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.806+0000 7f038dffb640 1 -- 172.21.15.33:0/4225603387 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0398032020 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f038dffb640 1 -- 172.21.15.33:0/4225603387 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f039801b890 con 0x55d9fa926230 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f03af9aa740 10 client.28951 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f03af9aa740 10 client.28951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f03af9aa740 20 client.28951 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f03af9aa740 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa9a32b0 0x55d9fa9a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f03af9aa740 4 client.28951 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9fa891f00 con 0x55d9fa9a32b0 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f039dd9b640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa9a32b0 0x55d9fa9a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f038dffb640 10 client.28951 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.807+0000 7f039dd9b640 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa9a32b0 0x55d9fa9a56d0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c065654c002881cf server_cookie=8505dc61f37a2e52 in_seq=0 out_seq=0 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f038dffb640 1 -- 172.21.15.33:0/4225603387 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9fa891f00 con 0x55d9fa9a32b0 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f038dffb640 10 client.28951 handle_command_reply: tid=0 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 client.28951 shutdown 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926230 msgr2=0x55d9fa9350f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926230 0x55d9fa9350f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55d9fa93eba0 tx=0x7f03980057d0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 shutdown_connections 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d9fa9a32b0 0x55d9fa9a56d0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f038dffb640 0 client.28951 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9fa92bd90 0x55d9fa9365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d9fa926c20 0x55d9fa935630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 --2- 172.21.15.33:0/4225603387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9fa926230 0x55d9fa9350f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 >> 172.21.15.33:0/4225603387 conn(0x55d9fa9218a0 msgr2=0x55d9fa924580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 shutdown_connections 2024-04-24T23:31:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 1 -- 172.21.15.33:0/4225603387 wait complete. 2024-04-24T23:31:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.808+0000 7f03af9aa740 20 client.28951 trim_cache size 0 max 16384 2024-04-24T23:31:37.816 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03940a2b90 msgr2=0x7f03940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03940a2b90 0x7f03940a5050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03a0047000 tx=0x7f03a0049000 comp rx=0 tx=0).stop 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 msgr2=0x7f03a8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 0x7f03a8197fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f039000e9f0 tx=0x7f039000edf0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 shutdown_connections 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03940a2b90 0x7f03940a5050 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a8103340 0x7f03a819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03a8102980 0x7f03a8197fc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 --2- 172.21.15.33:0/3764017181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a8101790 0x7f03a8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.809+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 >> 172.21.15.33:0/3764017181 conn(0x7f03a80fcd20 msgr2=0x7f03a8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.810+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 shutdown_connections 2024-04-24T23:31:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:37.810+0000 7f03acbc1640 1 -- 172.21.15.33:0/3764017181 wait complete. 2024-04-24T23:31:37.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 147736 22.58 MB/sec execute 222 sec latency 62.617 ms 2024-04-24T23:31:37.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:37 smithi038 ceph-mon[32450]: pgmap v939: 129 pgs: 129 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.5 MiB/s wr, 434 op/s 2024-04-24T23:31:37.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:37 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2874288905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:37.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:37 smithi117 ceph-mon[32529]: pgmap v939: 129 pgs: 129 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.5 MiB/s wr, 434 op/s 2024-04-24T23:31:37.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:37 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2874288905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:37.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:37 smithi033 ceph-mon[29201]: pgmap v939: 129 pgs: 129 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.5 MiB/s wr, 434 op/s 2024-04-24T23:31:37.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:37 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2874288905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:38.827 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:31:38.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:31:38.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:31:38.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 148289 22.62 MB/sec execute 223 sec latency 75.838 ms 2024-04-24T23:31:38.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 -- 172.21.15.33:0/500378340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc104b30 msgr2=0x7fb7fc106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 --2- 172.21.15.33:0/500378340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc104b30 0x7fb7fc106f20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402a410 comp rx=0 tx=0).stop 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 -- 172.21.15.33:0/500378340 shutdown_connections 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 --2- 172.21.15.33:0/500378340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc107460 0x7fb7fc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 --2- 172.21.15.33:0/500378340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc104b30 0x7fb7fc106f20 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 --2- 172.21.15.33:0/500378340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc102200 0x7fb7fc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 -- 172.21.15.33:0/500378340 >> 172.21.15.33:0/500378340 conn(0x7fb7fc0fbbf0 msgr2=0x7fb7fc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 -- 172.21.15.33:0/500378340 shutdown_connections 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.956+0000 7fb800e53640 1 -- 172.21.15.33:0/500378340 wait complete. 2024-04-24T23:31:38.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 Processor -- start 2024-04-24T23:31:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 -- start start 2024-04-24T23:31:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc102200 0x7fb7fc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 0x7fb7fc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc107460 0x7fb7fc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc10d440 con 0x7fb7fc107460 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc10d2c0 con 0x7fb7fc104b30 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.957+0000 7fb800e53640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc10d5c0 con 0x7fb7fc102200 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 0x7fb7fc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc102200 0x7fb7fc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 0x7fb7fc198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48278/0 (socket says 172.21.15.33:48278) 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/508183482 learned_addr learned my addr 172.21.15.33:0/508183482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc107460 0x7fb7fc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/508183482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc102200 msgr2=0x7fb7fc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc102200 0x7fb7fc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc107460 msgr2=0x7fb7fc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc107460 0x7fb7fc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/508183482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc19f730 con 0x7fb7fc104b30 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.958+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 0x7fb7fc198050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4031440 tx=0x7fb7e4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.959+0000 7fb7f3fff640 1 -- 172.21.15.33:0/508183482 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e403f030 con 0x7fb7fc104b30 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.959+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc19f9c0 con 0x7fb7fc104b30 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.959+0000 7fb7f3fff640 1 -- 172.21.15.33:0/508183482 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e4003ce0 con 0x7fb7fc104b30 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.959+0000 7fb7f3fff640 1 -- 172.21.15.33:0/508183482 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4035070 con 0x7fb7fc104b30 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.959+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc19feb0 con 0x7fb7fc104b30 2024-04-24T23:31:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb7f3fff640 1 -- 172.21.15.33:0/508183482 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb7e4042030 con 0x7fb7fc104b30 2024-04-24T23:31:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 Processor -- start 2024-04-24T23:31:38.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 -- start start 2024-04-24T23:31:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557fb90132c0 con 0x557fb90b41a0 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557fb8fbb910 con 0x557fb90b4b90 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.962+0000 7fb803c35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557fb900f280 con 0x557fb90b9d00 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7f1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54328/0 (socket says 172.21.15.33:54328) 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 -- 172.21.15.33:0/3178476724 learned_addr learned my addr 172.21.15.33:0/3178476724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7f3fff640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7e80a2b90 0x7fb7e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 -- 172.21.15.33:0/3178476724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 msgr2=0x557fb90bc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 -- 172.21.15.33:0/3178476724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 msgr2=0x557fb90b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7fad9d640 1 -- 172.21.15.33:0/3178476724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fb8fccfb0 con 0x557fb90b41a0 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.963+0000 7fb7f3fff640 1 -- 172.21.15.33:0/508183482 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb7e40ef8b0 con 0x7fb7fc104b30 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.964+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90b45c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4011440 tx=0x7fb7f4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b274b61f4f0546d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.964+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7e80a2b90 0x7fb7e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.964+0000 7fb7f0d99640 1 -- 172.21.15.33:0/3178476724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f401e070 con 0x557fb90b41a0 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.964+0000 7fb7f0d99640 1 -- 172.21.15.33:0/3178476724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40045f0 con 0x557fb90b41a0 2024-04-24T23:31:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.964+0000 7fb7f0d99640 1 -- 172.21.15.33:0/3178476724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4019630 con 0x557fb90b41a0 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 -- 172.21.15.33:0/3178476724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 msgr2=0x557fb90b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90b45c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4011440 tx=0x7fb7f4011900 comp rx=0 tx=0).stop 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7e80a2b90 0x7fb7e80a5050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb7f401b000 tx=0x7fb7f4019c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 -- 172.21.15.33:0/3178476724 shutdown_connections 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 --2- 172.21.15.33:0/3178476724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90b45c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 -- 172.21.15.33:0/3178476724 >> 172.21.15.33:0/3178476724 conn(0x557fb9077950 msgr2=0x557fb90ab810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 -- 172.21.15.33:0/3178476724 shutdown_connections 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 -- 172.21.15.33:0/3178476724 wait complete. 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 -- start start 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 Processor -- start 2024-04-24T23:31:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.965+0000 7fb803c35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b4b90 0x557fb90beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb803c35740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b9d00 0x557fb90c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb803c35740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557fb90bccb0 con 0x557fb90b41a0 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb803c35740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557fb8fcee90 con 0x557fb90b9d00 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb803c35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557fb90bce30 con 0x557fb90b4b90 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb803c35740 10 client.0 _ll_register_callbacks cb 0x557fb900a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7f1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b4b90 0x557fb90beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7f1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b4b90 0x557fb90beb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:36878/0 (socket says 172.21.15.33:36878) 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/3995646162 learned_addr learned my addr 172.21.15.33:0/3995646162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b9d00 0x557fb90c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 -- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b4b90 msgr2=0x557fb90beb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b4b90 0x557fb90beb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 -- 172.21.15.33:0/3995646162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b9d00 msgr2=0x557fb90c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b9d00 0x557fb90c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 -- 172.21.15.33:0/3995646162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fb8def7e0 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90be600 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x557fb90b4f90 tx=0x7fb7ec012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892d557959d9f83a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.966+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/3995646162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00f890 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/3995646162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb7ec00fa30 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f40045f0 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/3995646162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec00fc30 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb803c35740 10 client.19539 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557fb8e4ff70 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/3995646162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40045f0 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/3995646162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb7ec010430 con 0x557fb90b41a0 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb803c35740 10 client.19539 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.967+0000 7fb803c35740 10 client.19539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb803c35740 20 client.19539 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb803c35740 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb90ca0f0 0x557fb90cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb803c35740 4 client.19539 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fb8fe75a0 con 0x557fb90ca0f0 2024-04-24T23:31:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb90ca0f0 0x557fb90cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb7e1ffb640 10 client.19539 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.968+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb90ca0f0 0x557fb90cc560 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367f81127fe8820c server_cookie=e9d2c20df03d0407 in_seq=0 out_seq=0 2024-04-24T23:31:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.971+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/3995646162 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fb8fe75a0 con 0x557fb90ca0f0 2024-04-24T23:31:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.971+0000 7fb7e1ffb640 10 client.19539 handle_command_reply: tid=0 2024-04-24T23:31:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.971+0000 7fb803c35740 1 client.19539 shutdown 2024-04-24T23:31:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 msgr2=0x557fb90be600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90be600 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x557fb90b4f90 tx=0x7fb7ec012b60 comp rx=0 tx=0).stop 2024-04-24T23:31:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 shutdown_connections 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb90ca0f0 0x557fb90cc560 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b9d00 0x557fb90c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b4b90 0x557fb90beb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 --2- 172.21.15.33:0/3995646162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90be600 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 >> 172.21.15.33:0/3995646162 conn(0x557fb9077950 msgr2=0x557fb90aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb7e1ffb640 0 client.19539 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 shutdown_connections 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 1 -- 172.21.15.33:0/3995646162 wait complete. 2024-04-24T23:31:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.972+0000 7fb803c35740 20 client.19539 trim_cache size 0 max 16384 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.997+0000 7fb803c35740 1 Processor -- start 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.997+0000 7fb803c35740 1 -- start start 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb803c35740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90bd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb803c35740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90bd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90bd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54350/0 (socket says 172.21.15.33:54350) 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb7fa59c640 1 -- 172.21.15.33:0/1273321753 learned_addr learned my addr 172.21.15.33:0/1273321753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb803c35740 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1aaaf0 con 0x557fb90b41a0 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1a83c0 con 0x557fb90b4b90 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.998+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1a6210 con 0x557fb90b9d00 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/1273321753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 msgr2=0x557fb90bde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/1273321753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 msgr2=0x557fb90bd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90bd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/1273321753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fb90aba60 con 0x557fb90b4b90 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90c5b40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b86f0 tx=0x7fb7e40f5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5beb4195f4921c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7e3fff640 1 -- 172.21.15.33:0/1273321753 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4049040 con 0x557fb90b4b90 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7e3fff640 1 -- 172.21.15.33:0/1273321753 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e40bc420 con 0x557fb90b4b90 2024-04-24T23:31:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:38.999+0000 7fb7e3fff640 1 -- 172.21.15.33:0/1273321753 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ed0d0 con 0x557fb90b4b90 2024-04-24T23:31:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.000+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 msgr2=0x557fb90c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.000+0000 7fb803c35740 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90c5b40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b86f0 tx=0x7fb7e40f5530 comp rx=0 tx=0).stop 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.000+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 shutdown_connections 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b4b90 0x557fb90c5b40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 --2- 172.21.15.33:0/1273321753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b41a0 0x557fb90bd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 >> 172.21.15.33:0/1273321753 conn(0x557fb9077950 msgr2=0x557fb90aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 shutdown_connections 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 -- 172.21.15.33:0/1273321753 wait complete. 2024-04-24T23:31:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.001+0000 7fb803c35740 1 -- start start 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.002+0000 7fb803c35740 1 Processor -- start 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.002+0000 7fb803c35740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 0x557fb90c3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.002+0000 7fb803c35740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b4b90 0x557fb90c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 0x557fb90c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 0x557fb90c3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48306/0 (socket says 172.21.15.33:48306) 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb7fa59c640 1 -- 172.21.15.33:0/1624907274 learned_addr learned my addr 172.21.15.33:0/1624907274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b4b90 0x557fb90c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb7fad9d640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1a6880 con 0x557fb90b4b90 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1a7cf0 con 0x557fb90b41a0 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.003+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1a6f50 con 0x557fb90b9d00 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 msgr2=0x557fb90c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/1624907274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 msgr2=0x557fb90c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 0x557fb90c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7f1d9b640 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fb8ce7170 con 0x557fb90b4b90 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 0x557fb90c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7f1d9b640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b4b90 0x557fb90c3880 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4004600 tx=0x7fb7e4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f417f6eb524baf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb803c35740 10 client.0 _ll_register_callbacks cb 0x7fb7ec009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e40bc420 con 0x557fb90b4b90 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/1624907274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e409e070 con 0x557fb90b4b90 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/1624907274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb7e40ed0d0 con 0x557fb90b4b90 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/1624907274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb7e4049070 con 0x557fb90b4b90 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.004+0000 7fb803c35740 10 client.19545 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=33}) -- 0x557fb91306a0 con 0x557fb90b4b90 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/1624907274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb7e40a1030 con 0x557fb90b4b90 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 10 client.19545 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 10 client.19545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 20 client.19545 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb9130d70 0x557fb9133190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 4 client.19545 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.005+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557fb9020800 con 0x557fb9130d70 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.006+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb9130d70 0x557fb9133190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.006+0000 7fb7fa59c640 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb9130d70 0x557fb9133190 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7b3ff3eb5d2a22 server_cookie=caa9b521ecc22233 in_seq=0 out_seq=0 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.006+0000 7fb7e1ffb640 10 client.19545 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb7e1ffb640 1 -- 172.21.15.33:0/1624907274 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557fb9020800 con 0x557fb9130d70 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb7e1ffb640 10 client.19545 handle_command_reply: tid=0 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 client.19545 shutdown 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b4b90 msgr2=0x557fb90c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b4b90 0x557fb90c3880 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4004600 tx=0x7fb7e4009560 comp rx=0 tx=0).stop 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 shutdown_connections 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557fb9130d70 0x557fb9133190 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fb90b9d00 0x557fb90c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557fb90b4b90 0x557fb90c3880 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb7e1ffb640 0 client.19545 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 --2- 172.21.15.33:0/1624907274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557fb90b41a0 0x557fb90c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 >> 172.21.15.33:0/1624907274 conn(0x557fb9077950 msgr2=0x557fb90aa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 shutdown_connections 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.009+0000 7fb803c35740 1 -- 172.21.15.33:0/1624907274 wait complete. 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb803c35740 20 client.19545 trim_cache size 0 max 16384 2024-04-24T23:31:39.012 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:31:39.013 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:31:39.013 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "aeefc8b1-0210-45f5-b1f2-1d16c679d908", 2024-04-24T23:31:39.013 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:31:39.013 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7e80a2b90 msgr2=0x7fb7e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7e80a2b90 0x7fb7e80a5050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb7f401b000 tx=0x7fb7f4019c60 comp rx=0 tx=0).stop 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 msgr2=0x7fb7fc198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 0x7fb7fc198050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4031440 tx=0x7fb7e4032ce0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 shutdown_connections 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb7e80a2b90 0x7fb7e80a5050 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc107460 0x7fb7fc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7fc104b30 0x7fb7fc198050 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 --2- 172.21.15.33:0/508183482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc102200 0x7fb7fc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.010+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 >> 172.21.15.33:0/508183482 conn(0x7fb7fc0fbbf0 msgr2=0x7fb7fc102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.011+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 shutdown_connections 2024-04-24T23:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.011+0000 7fb800e53640 1 -- 172.21.15.33:0/508183482 wait complete. 2024-04-24T23:31:39.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:39.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 -- 172.21.15.33:0/4041540955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8096800 msgr2=0x7f65b8098bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 --2- 172.21.15.33:0/4041540955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8096800 0x7f65b8098bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 -- 172.21.15.33:0/4041540955 shutdown_connections 2024-04-24T23:31:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 --2- 172.21.15.33:0/4041540955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b809ba60 0x7f65b809deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 --2- 172.21.15.33:0/4041540955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8099130 0x7f65b809b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 --2- 172.21.15.33:0/4041540955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8096800 0x7f65b8098bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.180+0000 7f65bf59e640 1 -- 172.21.15.33:0/4041540955 >> 172.21.15.33:0/4041540955 conn(0x7f65b80901d0 msgr2=0x7f65b8092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.181+0000 7f65bf59e640 1 -- 172.21.15.33:0/4041540955 shutdown_connections 2024-04-24T23:31:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.181+0000 7f65bf59e640 1 -- 172.21.15.33:0/4041540955 wait complete. 2024-04-24T23:31:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 Processor -- start 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 -- start start 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 0x7f65b8129ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b8099130 0x7f65b812a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b809ba60 0x7f65b8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80a1a30 con 0x7f65b8099130 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80a18b0 con 0x7f65b8096800 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.182+0000 7f65bf59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80a1bb0 con 0x7f65b809ba60 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 0x7f65b8129ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 0x7f65b8129ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48324/0 (socket says 172.21.15.33:48324) 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 -- 172.21.15.33:0/2529649911 learned_addr learned my addr 172.21.15.33:0/2529649911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65bed9d640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b809ba60 0x7f65b8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b8099130 0x7f65b812a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 -- 172.21.15.33:0/2529649911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b809ba60 msgr2=0x7f65b8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b809ba60 0x7f65b8139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 -- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b8099130 msgr2=0x7f65b812a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b8099130 0x7f65b812a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.183+0000 7f65be59c640 1 -- 172.21.15.33:0/2529649911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b814b330 con 0x7f65b8096800 2024-04-24T23:31:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.184+0000 7f65be59c640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 0x7f65b8129ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009230 tx=0x7f65b00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.187+0000 7f65af7fe640 1 -- 172.21.15.33:0/2529649911 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b003f030 con 0x7f65b8096800 2024-04-24T23:31:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.188+0000 7f65af7fe640 1 -- 172.21.15.33:0/2529649911 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0003ce0 con 0x7f65b8096800 2024-04-24T23:31:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.188+0000 7f65af7fe640 1 -- 172.21.15.33:0/2529649911 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0035070 con 0x7f65b8096800 2024-04-24T23:31:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.188+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b814b5c0 con 0x7f65b8096800 2024-04-24T23:31:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.188+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b814ba60 con 0x7f65b8096800 2024-04-24T23:31:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 Processor -- start 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 -- start start 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06e00 0x557607e0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e0e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557607d6a6b0 con 0x557607e0bf70 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557607d12880 con 0x557607e06410 2024-04-24T23:31:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65c79c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557607d66670 con 0x557607e06e00 2024-04-24T23:31:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.192+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e06830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48334/0 (socket says 172.21.15.33:48334) 2024-04-24T23:31:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 -- 172.21.15.33:0/2239218132 learned_addr learned my addr 172.21.15.33:0/2239218132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65af7fe640 1 -- 172.21.15.33:0/2529649911 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f65b0042030 con 0x7f65b8096800 2024-04-24T23:31:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65be59c640 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06e00 0x557607e0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 -- 172.21.15.33:0/2239218132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06e00 msgr2=0x557607e0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06e00 0x557607e0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 -- 172.21.15.33:0/2239218132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 msgr2=0x557607e0e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e0e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 -- 172.21.15.33:0/2239218132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557607b487d0 con 0x557607e06410 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65be59c640 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06e00 0x557607e0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.193+0000 7f65bed9d640 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e06830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f65c0050ce0 tx=0x7f65c0050470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b8bbb2717601bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.194+0000 7f65af7fe640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65a80a2d60 0x7f65a80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.194+0000 7f65af7fe640 1 -- 172.21.15.33:0/2529649911 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f65b00ef820 con 0x7f65b8096800 2024-04-24T23:31:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.194+0000 7f65be59c640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65a80a2d60 0x7f65a80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.194+0000 7f65acff9640 1 -- 172.21.15.33:0/2239218132 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0061b80 con 0x557607e06410 2024-04-24T23:31:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.194+0000 7f65acff9640 1 -- 172.21.15.33:0/2239218132 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0061d20 con 0x557607e06410 2024-04-24T23:31:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.194+0000 7f65acff9640 1 -- 172.21.15.33:0/2239218132 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0062020 con 0x557607e06410 2024-04-24T23:31:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 -- 172.21.15.33:0/2239218132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 msgr2=0x557607e06830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e06830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f65c0050ce0 tx=0x7f65c0050470 comp rx=0 tx=0).stop 2024-04-24T23:31:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65be59c640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65a80a2d60 0x7f65a80a5220 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f65b0031440 tx=0x7f65b002a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 -- 172.21.15.33:0/2239218132 shutdown_connections 2024-04-24T23:31:39.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e0e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06e00 0x557607e0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 --2- 172.21.15.33:0/2239218132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e06830 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.195+0000 7f65c79c7740 1 -- 172.21.15.33:0/2239218132 >> 172.21.15.33:0/2239218132 conn(0x557607e01a80 msgr2=0x557607e01e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.196+0000 7f65c79c7740 1 -- 172.21.15.33:0/2239218132 shutdown_connections 2024-04-24T23:31:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.203+0000 7f65c79c7740 1 -- 172.21.15.33:0/2239218132 wait complete. 2024-04-24T23:31:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.203+0000 7f65c79c7740 1 -- start start 2024-04-24T23:31:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.203+0000 7f65c79c7740 1 Processor -- start 2024-04-24T23:31:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.205+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e10860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.205+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e10da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.205+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e0bf70 0x557607e17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e10860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e10860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48338/0 (socket says 172.21.15.33:48338) 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65bed9d640 1 -- 172.21.15.33:0/3877460379 learned_addr learned my addr 172.21.15.33:0/3877460379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557607e0ef20 con 0x557607e06e00 2024-04-24T23:31:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557607d25f50 con 0x557607e06410 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557607e0f0a0 con 0x557607e0bf70 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.206+0000 7f65bed9d640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e0bf70 0x557607e17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e10da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65c79c7740 10 client.0 _ll_register_callbacks cb 0x557607d61000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bed9d640 1 -- 172.21.15.33:0/3877460379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 msgr2=0x557607e10860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bed9d640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e10860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bed9d640 1 -- 172.21.15.33:0/3877460379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 msgr2=0x557607e10da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bed9d640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e10da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bed9d640 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557607a7e3f0 con 0x557607e0bf70 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65bed9d640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e0bf70 0x557607e17de0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f65c0077010 tx=0x7f65c0077040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f3953c3dc70eaee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.207+0000 7f65a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:31:39.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.208+0000 7f65a5ffb640 1 -- 172.21.15.33:0/3877460379 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00692d0 con 0x557607e0bf70 2024-04-24T23:31:39.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.208+0000 7f65a5ffb640 1 -- 172.21.15.33:0/3877460379 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f65c0069470 con 0x557607e0bf70 2024-04-24T23:31:39.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.208+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c0061d20 con 0x557607e0bf70 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.209+0000 7f65a5ffb640 1 -- 172.21.15.33:0/3877460379 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f65c0093520 con 0x557607e0bf70 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.209+0000 7f65c79c7740 10 client.28978 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.209+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557607a3e0f0 con 0x557607e0bf70 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.209+0000 7f65a5ffb640 1 -- 172.21.15.33:0/3877460379 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0089040 con 0x557607e0bf70 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65a5ffb640 1 -- 172.21.15.33:0/3877460379 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f65c007f480 con 0x557607e0bf70 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65c79c7740 10 client.28978 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65c79c7740 10 client.28978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65c79c7740 20 client.28978 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65c79c7740 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e1c4f0 0x557607e1e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65c79c7740 4 client.28978 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f65c0061d20 con 0x557607e1c4f0 2024-04-24T23:31:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.210+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e1c4f0 0x557607e1e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.211+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e1c4f0 0x557607e1e910 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f99515216ef1b6a server_cookie=1ef83c307a8e98f0 in_seq=0 out_seq=0 2024-04-24T23:31:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.211+0000 7f65a5ffb640 10 client.28978 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.215+0000 7f65a5ffb640 1 -- 172.21.15.33:0/3877460379 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f65c0061d20 con 0x557607e1c4f0 2024-04-24T23:31:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.215+0000 7f65a5ffb640 10 client.28978 handle_command_reply: tid=0 2024-04-24T23:31:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.215+0000 7f65c79c7740 1 client.28978 shutdown 2024-04-24T23:31:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e0bf70 msgr2=0x557607e17de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e0bf70 0x557607e17de0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f65c0077010 tx=0x7f65c0077040 comp rx=0 tx=0).stop 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 shutdown_connections 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e1c4f0 0x557607e1e910 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e0bf70 0x557607e17de0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e10da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 --2- 172.21.15.33:0/3877460379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06410 0x557607e10860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 >> 172.21.15.33:0/3877460379 conn(0x557607e01a80 msgr2=0x557607e00ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65a5ffb640 0 client.28978 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:39.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 shutdown_connections 2024-04-24T23:31:39.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.216+0000 7f65c79c7740 1 -- 172.21.15.33:0/3877460379 wait complete. 2024-04-24T23:31:39.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.217+0000 7f65c79c7740 20 client.28978 trim_cache size 0 max 16384 2024-04-24T23:31:39.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.246+0000 7f65c79c7740 1 Processor -- start 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.246+0000 7f65c79c7740 1 -- start start 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e04a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e17da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e0bf70 0x557607e04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65c79c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8158900 con 0x557607e06e00 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65c79c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81561d0 con 0x557607e0bf70 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65c79c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8154020 con 0x557607e06410 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e17da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e17da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54408/0 (socket says 172.21.15.33:54408) 2024-04-24T23:31:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 -- 172.21.15.33:0/1419727436 learned_addr learned my addr 172.21.15.33:0/1419727436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bed9d640 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e04a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bed9d640 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e0bf70 0x557607e04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 -- 172.21.15.33:0/1419727436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 msgr2=0x557607e04a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e04a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 -- 172.21.15.33:0/1419727436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e0bf70 msgr2=0x557607e04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e0bf70 0x557607e04f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.247+0000 7f65bdd9b640 1 -- 172.21.15.33:0/1419727436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557607e0f240 con 0x557607e06e00 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.248+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e17da0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x557607e029f0 tx=0x7f65b4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e571d27592a6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.248+0000 7f65a7fff640 1 -- 172.21.15.33:0/1419727436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4003a40 con 0x557607e06e00 2024-04-24T23:31:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.248+0000 7f65a7fff640 1 -- 172.21.15.33:0/1419727436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0061d20 con 0x557607e06e00 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.249+0000 7f65c79c7740 1 -- 172.21.15.33:0/1419727436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 msgr2=0x557607e17da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.249+0000 7f65c79c7740 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e17da0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x557607e029f0 tx=0x7f65b4009620 comp rx=0 tx=0).stop 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 -- 172.21.15.33:0/1419727436 shutdown_connections 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e0bf70 0x557607e04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e06e00 0x557607e17da0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 --2- 172.21.15.33:0/1419727436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e04a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 -- 172.21.15.33:0/1419727436 >> 172.21.15.33:0/1419727436 conn(0x557607e01a80 msgr2=0x557607e10860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 -- 172.21.15.33:0/1419727436 shutdown_connections 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 -- 172.21.15.33:0/1419727436 wait complete. 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.250+0000 7f65c79c7740 1 -- start start 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 Processor -- start 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06e00 0x557607e14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8154690 con 0x557607e0bf70 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8155b00 con 0x557607e06e00 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8154d60 con 0x557607e06410 2024-04-24T23:31:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65c79c7740 10 client.0 _ll_register_callbacks cb 0x7f65c0070df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54422/0 (socket says 172.21.15.33:54422) 2024-04-24T23:31:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.251+0000 7f65bed9d640 1 -- 172.21.15.33:0/422405249 learned_addr learned my addr 172.21.15.33:0/422405249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:31:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 -- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 msgr2=0x557607e17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e17c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 -- 172.21.15.33:0/422405249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06e00 msgr2=0x557607e14710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06e00 0x557607e14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 -- 172.21.15.33:0/422405249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557607e91a90 con 0x557607e0bf70 2024-04-24T23:31:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.252+0000 7f65bed9d640 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e14c50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f65c007f880 tx=0x7f65c0077040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e191d5a6d49225e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65a5ffb640 1 -- 172.21.15.33:0/422405249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0099040 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65a5ffb640 1 -- 172.21.15.33:0/422405249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f65c00690b0 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c0061d20 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65a5ffb640 1 -- 172.21.15.33:0/422405249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f65c00936c0 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65c79c7740 10 client.19560 fetch_fsmap learned FSMap version 33 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.253+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557607e92d20 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65a5ffb640 1 -- 172.21.15.33:0/422405249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0061d20 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65a5ffb640 1 -- 172.21.15.33:0/422405249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f65c009f040 con 0x557607e0bf70 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65c79c7740 10 client.19560 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:31:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65c79c7740 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65c79c7740 20 client.19560 populate_metadata read hostname 'smithi033' 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65c79c7740 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e935c0 0x557607e959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65c79c7740 4 client.19560 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557607d77bf0 con 0x557607e935c0 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.254+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e935c0 0x557607e959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.255+0000 7f65bdd9b640 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e935c0 0x557607e959e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f791d667b0e30225 server_cookie=7498f65bfa4acf7 in_seq=0 out_seq=0 2024-04-24T23:31:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.255+0000 7f65a5ffb640 10 client.19560 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65a5ffb640 1 -- 172.21.15.33:0/422405249 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557607d77bf0 con 0x557607e935c0 2024-04-24T23:31:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65a5ffb640 10 client.19560 handle_command_reply: tid=0 2024-04-24T23:31:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 client.19560 shutdown 2024-04-24T23:31:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 msgr2=0x557607e14c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e14c50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f65c007f880 tx=0x7f65c0077040 comp rx=0 tx=0).stop 2024-04-24T23:31:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 shutdown_connections 2024-04-24T23:31:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557607e935c0 0x557607e959e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557607e0bf70 0x557607e14c50 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557607e06e00 0x557607e14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 --2- 172.21.15.33:0/422405249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557607e06410 0x557607e17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 >> 172.21.15.33:0/422405249 conn(0x557607e01a80 msgr2=0x557607e04a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65a5ffb640 0 client.19560 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:31:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 shutdown_connections 2024-04-24T23:31:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 1 -- 172.21.15.33:0/422405249 wait complete. 2024-04-24T23:31:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.256+0000 7f65c79c7740 20 client.19560 trim_cache size 0 max 16384 2024-04-24T23:31:39.262 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:31:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:31:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:31:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "aeefc8b1-0210-45f5-b1f2-1d16c679d908": { 2024-04-24T23:31:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:31:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "aeefc8b1-0210-45f5-b1f2-1d16c679d908", 2024-04-24T23:31:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:31:39.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:31:39.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:31:39.264 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65a80a2d60 msgr2=0x7f65a80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65a80a2d60 0x7f65a80a5220 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f65b0031440 tx=0x7f65b002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 msgr2=0x7f65b8129ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 0x7f65b8129ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009230 tx=0x7f65b00042f0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 shutdown_connections 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f65a80a2d60 0x7f65a80a5220 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b809ba60 0x7f65b8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b8099130 0x7f65b812a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 --2- 172.21.15.33:0/2529649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65b8096800 0x7f65b8129ee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 >> 172.21.15.33:0/2529649911 conn(0x7f65b80901d0 msgr2=0x7f65b80975b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 shutdown_connections 2024-04-24T23:31:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:31:39.257+0000 7f65bf59e640 1 -- 172.21.15.33:0/2529649911 wait complete. 2024-04-24T23:31:39.272 INFO:tasks.cephfs.filesystem:scrub status for tag:aeefc8b1-0210-45f5-b1f2-1d16c679d908 - {'path': '/', 'tag': 'aeefc8b1-0210-45f5-b1f2-1d16c679d908', 'options': 'recursive,force'} 2024-04-24T23:31:39.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 148716 22.63 MB/sec execute 224 sec latency 116.136 ms 2024-04-24T23:31:39.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: pgmap v940: 129 pgs: 129 active+clean; 576 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 426 op/s 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:39.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:39 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: pgmap v940: 129 pgs: 129 active+clean; 576 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 426 op/s 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:39.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:39 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: pgmap v940: 129 pgs: 129 active+clean; 576 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 426 op/s 2024-04-24T23:31:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:31:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:39.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:39.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:39.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:39 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:40.892 INFO:tasks.workunit.client.0.smithi033.stdout: 10 148885 22.56 MB/sec execute 225 sec latency 308.943 ms 2024-04-24T23:31:40.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:31:40.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:40.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:31:40.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:31:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:41.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 149059 22.48 MB/sec execute 226 sec latency 215.924 ms 2024-04-24T23:31:41.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:41 smithi038 ceph-mon[32450]: pgmap v941: 129 pgs: 129 active+clean; 576 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.7 MiB/s wr, 406 op/s 2024-04-24T23:31:41.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:41 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:41.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:41 smithi117 ceph-mon[32529]: pgmap v941: 129 pgs: 129 active+clean; 576 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.7 MiB/s wr, 406 op/s 2024-04-24T23:31:41.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:41 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:41.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:41 smithi033 ceph-mon[29201]: pgmap v941: 129 pgs: 129 active+clean; 576 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.7 MiB/s wr, 406 op/s 2024-04-24T23:31:41.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:41 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:42.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 149410 22.46 MB/sec execute 227 sec latency 86.161 ms 2024-04-24T23:31:43.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 149943 22.45 MB/sec execute 228 sec latency 85.442 ms 2024-04-24T23:31:43.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:43 smithi038 ceph-mon[32450]: pgmap v942: 129 pgs: 129 active+clean; 570 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 469 op/s 2024-04-24T23:31:43.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:43 smithi117 ceph-mon[32529]: pgmap v942: 129 pgs: 129 active+clean; 570 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 469 op/s 2024-04-24T23:31:43.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:43 smithi033 ceph-mon[29201]: pgmap v942: 129 pgs: 129 active+clean; 570 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 469 op/s 2024-04-24T23:31:44.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 150559 22.49 MB/sec execute 229 sec latency 78.442 ms 2024-04-24T23:31:45.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 151152 22.55 MB/sec execute 230 sec latency 268.591 ms 2024-04-24T23:31:45.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:45 smithi038 ceph-mon[32450]: pgmap v943: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 442 op/s 2024-04-24T23:31:45.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:45 smithi117 ceph-mon[32529]: pgmap v943: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 442 op/s 2024-04-24T23:31:45.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:45 smithi033 ceph-mon[29201]: pgmap v943: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 442 op/s 2024-04-24T23:31:46.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 151507 22.55 MB/sec execute 231 sec latency 345.481 ms 2024-04-24T23:31:47.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 151887 22.58 MB/sec execute 232 sec latency 90.739 ms 2024-04-24T23:31:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:47 smithi038 ceph-mon[32450]: pgmap v944: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-04-24T23:31:47.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:47 smithi117 ceph-mon[32529]: pgmap v944: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-04-24T23:31:47.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:47 smithi033 ceph-mon[29201]: pgmap v944: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-04-24T23:31:48.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 152373 22.60 MB/sec execute 233 sec latency 86.610 ms 2024-04-24T23:31:49.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 152690 22.58 MB/sec execute 234 sec latency 98.959 ms 2024-04-24T23:31:49.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:49 smithi038 ceph-mon[32450]: pgmap v945: 129 pgs: 129 active+clean; 578 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 426 op/s 2024-04-24T23:31:49.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:49 smithi117 ceph-mon[32529]: pgmap v945: 129 pgs: 129 active+clean; 578 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 426 op/s 2024-04-24T23:31:49.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:49 smithi033 ceph-mon[29201]: pgmap v945: 129 pgs: 129 active+clean; 578 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 426 op/s 2024-04-24T23:31:50.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 152891 22.50 MB/sec execute 235 sec latency 209.443 ms 2024-04-24T23:31:51.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 153141 22.45 MB/sec execute 236 sec latency 380.118 ms 2024-04-24T23:31:51.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:51 smithi038 ceph-mon[32450]: pgmap v946: 129 pgs: 129 active+clean; 580 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 391 op/s 2024-04-24T23:31:51.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:51 smithi117 ceph-mon[32529]: pgmap v946: 129 pgs: 129 active+clean; 580 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 391 op/s 2024-04-24T23:31:51.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:51 smithi033 ceph-mon[29201]: pgmap v946: 129 pgs: 129 active+clean; 580 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 391 op/s 2024-04-24T23:31:52.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 153551 22.43 MB/sec execute 237 sec latency 96.587 ms 2024-04-24T23:31:52.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:31:52.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:31:52.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:31:52.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:31:53.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:31:53.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:31:53.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 154140 22.47 MB/sec execute 238 sec latency 87.645 ms 2024-04-24T23:31:53.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:53 smithi117 ceph-mon[32529]: pgmap v947: 129 pgs: 129 active+clean; 587 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.6 MiB/s wr, 480 op/s 2024-04-24T23:31:53.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:53 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:31:53.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:53 smithi033 ceph-mon[29201]: pgmap v947: 129 pgs: 129 active+clean; 587 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.6 MiB/s wr, 480 op/s 2024-04-24T23:31:53.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:53 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:31:54.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:53 smithi038 ceph-mon[32450]: pgmap v947: 129 pgs: 129 active+clean; 587 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.6 MiB/s wr, 480 op/s 2024-04-24T23:31:54.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:53 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:31:54.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 154727 22.50 MB/sec execute 239 sec latency 101.632 ms 2024-04-24T23:31:55.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 155256 22.54 MB/sec execute 240 sec latency 59.766 ms 2024-04-24T23:31:55.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:55 smithi033 ceph-mon[29201]: pgmap v948: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 405 op/s 2024-04-24T23:31:56.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:55 smithi038 ceph-mon[32450]: pgmap v948: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 405 op/s 2024-04-24T23:31:56.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:55 smithi117 ceph-mon[32529]: pgmap v948: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 405 op/s 2024-04-24T23:31:56.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 155467 22.48 MB/sec execute 241 sec latency 424.718 ms 2024-04-24T23:31:57.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 155888 22.52 MB/sec execute 242 sec latency 103.023 ms 2024-04-24T23:31:57.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:57 smithi033 ceph-mon[29201]: pgmap v949: 129 pgs: 129 active+clean; 588 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 402 op/s 2024-04-24T23:31:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:57 smithi038 ceph-mon[32450]: pgmap v949: 129 pgs: 129 active+clean; 588 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 402 op/s 2024-04-24T23:31:58.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:57 smithi117 ceph-mon[32529]: pgmap v949: 129 pgs: 129 active+clean; 588 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 402 op/s 2024-04-24T23:31:58.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 156552 22.58 MB/sec execute 243 sec latency 58.360 ms 2024-04-24T23:31:59.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 156897 22.59 MB/sec execute 244 sec latency 84.599 ms 2024-04-24T23:31:59.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:31:59 smithi033 ceph-mon[29201]: pgmap v950: 129 pgs: 129 active+clean; 593 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 400 op/s 2024-04-24T23:32:00.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:31:59 smithi038 ceph-mon[32450]: pgmap v950: 129 pgs: 129 active+clean; 593 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 400 op/s 2024-04-24T23:32:00.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:31:59 smithi117 ceph-mon[32529]: pgmap v950: 129 pgs: 129 active+clean; 593 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.2 MiB/s wr, 400 op/s 2024-04-24T23:32:00.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 157170 22.53 MB/sec execute 245 sec latency 96.303 ms 2024-04-24T23:32:01.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 157271 22.45 MB/sec execute 246 sec latency 434.823 ms 2024-04-24T23:32:01.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:01 smithi038 ceph-mon[32450]: pgmap v951: 129 pgs: 129 active+clean; 593 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-04-24T23:32:01.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:01 smithi033 ceph-mon[29201]: pgmap v951: 129 pgs: 129 active+clean; 593 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-04-24T23:32:02.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:01 smithi117 ceph-mon[32529]: pgmap v951: 129 pgs: 129 active+clean; 593 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-04-24T23:32:02.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 157624 22.42 MB/sec execute 247 sec latency 170.001 ms 2024-04-24T23:32:03.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 158111 22.47 MB/sec execute 248 sec latency 87.716 ms 2024-04-24T23:32:03.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:03 smithi033 ceph-mon[29201]: pgmap v952: 129 pgs: 129 active+clean; 596 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.5 MiB/s wr, 456 op/s 2024-04-24T23:32:04.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:03 smithi038 ceph-mon[32450]: pgmap v952: 129 pgs: 129 active+clean; 596 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.5 MiB/s wr, 456 op/s 2024-04-24T23:32:04.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:03 smithi117 ceph-mon[32529]: pgmap v952: 129 pgs: 129 active+clean; 596 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.5 MiB/s wr, 456 op/s 2024-04-24T23:32:04.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 158654 22.46 MB/sec execute 249 sec latency 99.081 ms 2024-04-24T23:32:05.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 159316 22.48 MB/sec execute 250 sec latency 52.642 ms 2024-04-24T23:32:05.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:05 smithi033 ceph-mon[29201]: pgmap v953: 129 pgs: 129 active+clean; 578 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 383 op/s 2024-04-24T23:32:06.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:05 smithi038 ceph-mon[32450]: pgmap v953: 129 pgs: 129 active+clean; 578 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 383 op/s 2024-04-24T23:32:06.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:05 smithi117 ceph-mon[32529]: pgmap v953: 129 pgs: 129 active+clean; 578 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 383 op/s 2024-04-24T23:32:06.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 159638 22.49 MB/sec execute 251 sec latency 498.896 ms 2024-04-24T23:32:07.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 160093 22.50 MB/sec execute 252 sec latency 158.626 ms 2024-04-24T23:32:07.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:07 smithi033 ceph-mon[29201]: pgmap v954: 129 pgs: 129 active+clean; 580 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 353 op/s 2024-04-24T23:32:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:07 smithi038 ceph-mon[32450]: pgmap v954: 129 pgs: 129 active+clean; 580 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 353 op/s 2024-04-24T23:32:08.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:07 smithi117 ceph-mon[32529]: pgmap v954: 129 pgs: 129 active+clean; 580 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 353 op/s 2024-04-24T23:32:08.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 160520 22.57 MB/sec execute 253 sec latency 117.559 ms 2024-04-24T23:32:09.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:32:09.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:32:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 -- 172.21.15.33:0/3928005815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0104b30 msgr2=0x7ff7d0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 --2- 172.21.15.33:0/3928005815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0104b30 0x7ff7d0106f20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009900 tx=0x7ff7c402a410 comp rx=0 tx=0).stop 2024-04-24T23:32:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 -- 172.21.15.33:0/3928005815 shutdown_connections 2024-04-24T23:32:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 --2- 172.21.15.33:0/3928005815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d0107460 0x7ff7d01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 --2- 172.21.15.33:0/3928005815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0104b30 0x7ff7d0106f20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 --2- 172.21.15.33:0/3928005815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d0102200 0x7ff7d01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 -- 172.21.15.33:0/3928005815 >> 172.21.15.33:0/3928005815 conn(0x7ff7d00fbbf0 msgr2=0x7ff7d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 -- 172.21.15.33:0/3928005815 shutdown_connections 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 -- 172.21.15.33:0/3928005815 wait complete. 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.422+0000 7ff7d559e640 1 Processor -- start 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 -- start start 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 0x7ff7d0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d0104b30 0x7ff7d019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d0107460 0x7ff7d01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010d400 con 0x7ff7d0102200 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010d280 con 0x7ff7d0107460 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7d559e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010d580 con 0x7ff7d0104b30 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 0x7ff7d0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 0x7ff7d0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57336/0 (socket says 172.21.15.33:57336) 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 -- 172.21.15.33:0/2443410819 learned_addr learned my addr 172.21.15.33:0/2443410819 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d0104b30 0x7ff7d019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d0107460 0x7ff7d01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 -- 172.21.15.33:0/2443410819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d0104b30 msgr2=0x7ff7d019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d0104b30 0x7ff7d019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 -- 172.21.15.33:0/2443410819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d0107460 msgr2=0x7ff7d01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d0107460 0x7ff7d01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 -- 172.21.15.33:0/2443410819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d01a1920 con 0x7ff7d0102200 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 0x7ff7d0199d10 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000ea60 tx=0x7ff7c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.423+0000 7ff7b7fff640 1 -- 172.21.15.33:0/2443410819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000cde0 con 0x7ff7d0102200 2024-04-24T23:32:09.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.424+0000 7ff7b7fff640 1 -- 172.21.15.33:0/2443410819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c00035f0 con 0x7ff7d0102200 2024-04-24T23:32:09.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.424+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d01a1c10 con 0x7ff7d0102200 2024-04-24T23:32:09.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.424+0000 7ff7b7fff640 1 -- 172.21.15.33:0/2443410819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0010670 con 0x7ff7d0102200 2024-04-24T23:32:09.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.424+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d01a2160 con 0x7ff7d0102200 2024-04-24T23:32:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.426+0000 7ff7b7fff640 1 -- 172.21.15.33:0/2443410819 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff7c00037a0 con 0x7ff7d0102200 2024-04-24T23:32:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.426+0000 7ff7d8386740 1 Processor -- start 2024-04-24T23:32:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.426+0000 7ff7d8386740 1 -- start start 2024-04-24T23:32:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.426+0000 7ff7b7fff640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff7b80a2b90 0x7ff7b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7b7fff640 1 -- 172.21.15.33:0/2443410819 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff7c00cf730 con 0x7ff7d0102200 2024-04-24T23:32:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 0x5620256f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 0x5620256fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025700230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7d8386740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56202565c6b0 con 0x5620256fdda0 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7d8386740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562025604880 con 0x5620256f8c30 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7d8386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562025658670 con 0x5620256f8240 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff7b80a2b90 0x7ff7b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 0x5620256fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ceffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 0x5620256f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 0x5620256fd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60426/0 (socket says 172.21.15.33:60426) 2024-04-24T23:32:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ceffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 0x5620256f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38082/0 (socket says 172.21.15.33:38082) 2024-04-24T23:32:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 -- 172.21.15.33:0/3216865798 learned_addr learned my addr 172.21.15.33:0/3216865798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025700230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 -- 172.21.15.33:0/3216865798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 msgr2=0x5620256f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 0x5620256f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 -- 172.21.15.33:0/3216865798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 msgr2=0x562025700230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025700230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 -- 172.21.15.33:0/3216865798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202543a7d0 con 0x5620256f8c30 2024-04-24T23:32:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 0x5620256f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff7b80a2b90 0x7ff7b80a5050 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005fd0 tx=0x7ff7c8013b60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.427+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 0x5620256fd730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4031610 tx=0x7ff7c4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf96dca034850bac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7b57fa640 1 -- 172.21.15.33:0/3216865798 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4040070 con 0x5620256f8c30 2024-04-24T23:32:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7b57fa640 1 -- 172.21.15.33:0/3216865798 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4004030 con 0x5620256f8c30 2024-04-24T23:32:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7b57fa640 1 -- 172.21.15.33:0/3216865798 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4035040 con 0x5620256f8c30 2024-04-24T23:32:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 -- 172.21.15.33:0/3216865798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 msgr2=0x5620256fd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 0x5620256fd730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4031610 tx=0x7ff7c4005de0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 -- 172.21.15.33:0/3216865798 shutdown_connections 2024-04-24T23:32:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025700230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8c30 0x5620256fd730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3216865798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8240 0x5620256f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 -- 172.21.15.33:0/3216865798 >> 172.21.15.33:0/3216865798 conn(0x5620256f38b0 msgr2=0x5620256f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 -- 172.21.15.33:0/3216865798 shutdown_connections 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.428+0000 7ff7d8386740 1 -- 172.21.15.33:0/3216865798 wait complete. 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 -- start start 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 Processor -- start 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x562025702600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025702b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025709b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562025700cf0 con 0x5620256fdda0 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562025617f50 con 0x5620256f8240 2024-04-24T23:32:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562025700e70 con 0x5620256f8c30 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7d8386740 10 client.0 _ll_register_callbacks cb 0x562025653000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025709b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025709b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57366/0 (socket says 172.21.15.33:57366) 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7cf7fe640 1 -- 172.21.15.33:0/3101640985 learned_addr learned my addr 172.21.15.33:0/3101640985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x562025702600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025702b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ceffd640 1 -- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 msgr2=0x562025702b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025702b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ceffd640 1 -- 172.21.15.33:0/3101640985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 msgr2=0x562025709b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025709b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.429+0000 7ff7ceffd640 1 -- 172.21.15.33:0/3101640985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620253703f0 con 0x5620256f8240 2024-04-24T23:32:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025702b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x562025702600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00cd210 tx=0x7ff7c009b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50d32e396deedc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/3101640985 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00b9060 con 0x5620256f8240 2024-04-24T23:32:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c4004030 con 0x5620256f8240 2024-04-24T23:32:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/3101640985 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff7c00cf440 con 0x5620256f8240 2024-04-24T23:32:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/3101640985 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7c009bc00 con 0x5620256f8240 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7d8386740 10 client.29054 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620253300f0 con 0x5620256f8240 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/3101640985 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4004030 con 0x5620256f8240 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/3101640985 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff7c009b0e0 con 0x5620256f8240 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7d8386740 10 client.29054 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7d8386740 10 client.29054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.430+0000 7ff7d8386740 20 client.29054 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.431+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56202570e340 0x5620257107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.431+0000 7ff7d8386740 4 client.29054 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.431+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562025669bf0 con 0x56202570e340 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.431+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56202570e340 0x5620257107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.431+0000 7ff7b2ff5640 10 client.29054 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.431+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56202570e340 0x5620257107d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff68910656772b5 server_cookie=211eb603ca050cc8 in_seq=0 out_seq=0 2024-04-24T23:32:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/3101640985 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562025669bf0 con 0x56202570e340 2024-04-24T23:32:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7b2ff5640 10 client.29054 handle_command_reply: tid=0 2024-04-24T23:32:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 client.29054 shutdown 2024-04-24T23:32:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 msgr2=0x562025702600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x562025702600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00cd210 tx=0x7ff7c009b830 comp rx=0 tx=0).stop 2024-04-24T23:32:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 shutdown_connections 2024-04-24T23:32:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56202570e340 0x5620257107d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025709b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025702b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 --2- 172.21.15.33:0/3101640985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x562025702600 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7b2ff5640 0 client.29054 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 >> 172.21.15.33:0/3101640985 conn(0x5620256f38b0 msgr2=0x5620256f2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 shutdown_connections 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 1 -- 172.21.15.33:0/3101640985 wait complete. 2024-04-24T23:32:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.432+0000 7ff7d8386740 20 client.29054 trim_cache size 0 max 16384 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 Processor -- start 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 -- start start 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 0x562025702aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025709b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256fdda0 0x562025701210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01aef10 con 0x5620256f8240 2024-04-24T23:32:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01ac7e0 con 0x5620256fdda0 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7d8386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01aa630 con 0x5620256f8c30 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 0x562025702aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 0x562025702aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57380/0 (socket says 172.21.15.33:57380) 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256fdda0 0x562025701210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7ceffd640 1 -- 172.21.15.33:0/532988264 learned_addr learned my addr 172.21.15.33:0/532988264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7ce7fc640 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025709b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7ceffd640 1 -- 172.21.15.33:0/532988264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 msgr2=0x562025709b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.450+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025709b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7ceffd640 1 -- 172.21.15.33:0/532988264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256fdda0 msgr2=0x562025701210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256fdda0 0x562025701210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7ceffd640 1 -- 172.21.15.33:0/532988264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620256e5090 con 0x5620256f8240 2024-04-24T23:32:09.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 0x562025702aa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0003ee0 tx=0x7ff7c0097780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c832a25fc81f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7b4ff9640 1 -- 172.21.15.33:0/532988264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00a9070 con 0x5620256f8240 2024-04-24T23:32:09.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7b4ff9640 1 -- 172.21.15.33:0/532988264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c00a4870 con 0x5620256f8240 2024-04-24T23:32:09.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7b4ff9640 1 -- 172.21.15.33:0/532988264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00b9050 con 0x5620256f8240 2024-04-24T23:32:09.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 -- 172.21.15.33:0/532988264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 msgr2=0x562025702aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 0x562025702aa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0003ee0 tx=0x7ff7c0097780 comp rx=0 tx=0).stop 2024-04-24T23:32:09.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 -- 172.21.15.33:0/532988264 shutdown_connections 2024-04-24T23:32:09.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256fdda0 0x562025701210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x562025709b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 --2- 172.21.15.33:0/532988264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256f8240 0x562025702aa0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.451+0000 7ff7d8386740 1 -- 172.21.15.33:0/532988264 >> 172.21.15.33:0/532988264 conn(0x5620256f38b0 msgr2=0x5620256f2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 -- 172.21.15.33:0/532988264 shutdown_connections 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 -- 172.21.15.33:0/532988264 wait complete. 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 -- start start 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 Processor -- start 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x5620257072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x5620257077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025708750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01aaca0 con 0x5620256fdda0 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01ac110 con 0x5620256f8240 2024-04-24T23:32:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01ab370 con 0x5620256f8c30 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7d8386740 10 client.0 _ll_register_callbacks cb 0x7ff7c0097e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025708750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7ce7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x5620257077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025708750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57382/0 (socket says 172.21.15.33:57382) 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 -- 172.21.15.33:0/2507970605 learned_addr learned my addr 172.21.15.33:0/2507970605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x5620257072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 -- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 msgr2=0x5620257077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x5620257077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 -- 172.21.15.33:0/2507970605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 msgr2=0x5620257072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x5620257072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7cf7fe640 1 -- 172.21.15.33:0/2507970605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562025708d40 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.452+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x5620257072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7cf7fe640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025708750 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8003bd0 tx=0x7ff7c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899904b193a4887e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/2507970605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80044c0 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/2507970605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff7c8004660 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c00a4870 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/2507970605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7c802b440 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 10 client.19578 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620257748e0 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/2507970605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c00a4870 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/2507970605 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff7c8029070 con 0x5620256fdda0 2024-04-24T23:32:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 10 client.19578 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 20 client.19578 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5620257752b0 0x5620257776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 4 client.19578 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.453+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562025663f40 con 0x5620257752b0 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5620257752b0 0x5620257776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7b2ff5640 10 client.19578 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7ceffd640 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5620257752b0 0x5620257776d0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7529e12c5005cc server_cookie=5b915ef2db9d2fc4 in_seq=0 out_seq=0 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7b2ff5640 1 -- 172.21.15.33:0/2507970605 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562025663f40 con 0x5620257752b0 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7b2ff5640 10 client.19578 handle_command_reply: tid=0 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 client.19578 shutdown 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 msgr2=0x562025708750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025708750 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8003bd0 tx=0x7ff7c8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 shutdown_connections 2024-04-24T23:32:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5620257752b0 0x5620257776d0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620256fdda0 0x562025708750 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5620256f8c30 0x5620257077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7b2ff5640 0 client.19578 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 --2- 172.21.15.33:0/2507970605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5620256f8240 0x5620257072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 >> 172.21.15.33:0/2507970605 conn(0x5620256f38b0 msgr2=0x562025702890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 shutdown_connections 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.454+0000 7ff7d8386740 1 -- 172.21.15.33:0/2507970605 wait complete. 2024-04-24T23:32:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d8386740 20 client.19578 trim_cache size 0 max 16384 2024-04-24T23:32:09.471 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:32:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:32:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:32:09.471 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:32:09.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff7b80a2b90 msgr2=0x7ff7b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff7b80a2b90 0x7ff7b80a5050 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005fd0 tx=0x7ff7c8013b60 comp rx=0 tx=0).stop 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 msgr2=0x7ff7d0199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 0x7ff7d0199d10 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000ea60 tx=0x7ff7c000ef20 comp rx=0 tx=0).stop 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 shutdown_connections 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff7b80a2b90 0x7ff7b80a5050 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7d0107460 0x7ff7d01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d0104b30 0x7ff7d019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 --2- 172.21.15.33:0/2443410819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7d0102200 0x7ff7d0199d10 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 >> 172.21.15.33:0/2443410819 conn(0x7ff7d00fbbf0 msgr2=0x7ff7d0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 shutdown_connections 2024-04-24T23:32:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.455+0000 7ff7d559e640 1 -- 172.21.15.33:0/2443410819 wait complete. 2024-04-24T23:32:09.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:32:09.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:32:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 -- 172.21.15.33:0/1123214562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35401017a0 msgr2=0x7f3540101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 --2- 172.21.15.33:0/1123214562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35401017a0 0x7f3540101bc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3530009900 tx=0x7f353002a410 comp rx=0 tx=0).stop 2024-04-24T23:32:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 -- 172.21.15.33:0/1123214562 shutdown_connections 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 --2- 172.21.15.33:0/1123214562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3540103350 0x7f3540109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 --2- 172.21.15.33:0/1123214562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3540102990 0x7f3540102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 --2- 172.21.15.33:0/1123214562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35401017a0 0x7f3540101bc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 -- 172.21.15.33:0/1123214562 >> 172.21.15.33:0/1123214562 conn(0x7f35400fccf0 msgr2=0x7f35400ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 -- 172.21.15.33:0/1123214562 shutdown_connections 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 -- 172.21.15.33:0/1123214562 wait complete. 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 Processor -- start 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.586+0000 7f3544c88640 1 -- start start 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35401017a0 0x7f3540197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3540102990 0x7f3540198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 0x7f354019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f354010d410 con 0x7f3540103350 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f354010d290 con 0x7f3540102990 2024-04-24T23:32:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f354010d590 con 0x7f35401017a0 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 0x7f354019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 0x7f354019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57400/0 (socket says 172.21.15.33:57400) 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 -- 172.21.15.33:0/2606433609 learned_addr learned my addr 172.21.15.33:0/2606433609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3537fff640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3540102990 0x7f3540198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353e59c640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35401017a0 0x7f3540197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 -- 172.21.15.33:0/2606433609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35401017a0 msgr2=0x7f3540197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35401017a0 0x7f3540197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 -- 172.21.15.33:0/2606433609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3540102990 msgr2=0x7f3540198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3540102990 0x7f3540198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 -- 172.21.15.33:0/2606433609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354019f710 con 0x7f3540103350 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f353ed9d640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 0x7f354019f0a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f353800eb10 tx=0x7f353800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538004b20 con 0x7f3540103350 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354019fa00 con 0x7f3540103350 2024-04-24T23:32:09.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3538004cc0 con 0x7f3540103350 2024-04-24T23:32:09.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538010760 con 0x7f3540103350 2024-04-24T23:32:09.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.587+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354019ff50 con 0x7f3540103350 2024-04-24T23:32:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.588+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3540108b40 con 0x7f3540103350 2024-04-24T23:32:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.590+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3538010900 con 0x7f3540103350 2024-04-24T23:32:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.591+0000 7f35377fe640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f352c0a2b90 0x7f352c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.591+0000 7f353e59c640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f352c0a2b90 0x7f352c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.591+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35380cfb50 con 0x7f3540103350 2024-04-24T23:32:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.591+0000 7f353e59c640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f352c0a2b90 0x7f352c0a5050 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3530005ab0 tx=0x7f3530000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.595+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3538053740 con 0x7f3540103350 2024-04-24T23:32:09.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.835+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354002d170 con 0x7f3540103350 2024-04-24T23:32:09.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.835+0000 7f35377fe640 1 -- 172.21.15.33:0/2606433609 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f353809c720 con 0x7f3540103350 2024-04-24T23:32:09.835 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:32:09.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:32:09.835 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:32:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.837+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f352c0a2b90 msgr2=0x7f352c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.837+0000 7f3544c88640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f352c0a2b90 0x7f352c0a5050 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3530005ab0 tx=0x7f3530000bc0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.837+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 msgr2=0x7f354019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.837+0000 7f3544c88640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 0x7f354019f0a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f353800eb10 tx=0x7f353800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 shutdown_connections 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f352c0a2b90 0x7f352c0a5050 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3540103350 0x7f354019f0a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3540102990 0x7f3540198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 --2- 172.21.15.33:0/2606433609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35401017a0 0x7f3540197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 >> 172.21.15.33:0/2606433609 conn(0x7f35400fccf0 msgr2=0x7f3540105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 shutdown_connections 2024-04-24T23:32:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.838+0000 7f3544c88640 1 -- 172.21.15.33:0/2606433609 wait complete. 2024-04-24T23:32:09.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:32:09.893 INFO:tasks.workunit.client.0.smithi033.stdout: 10 160767 22.54 MB/sec execute 254 sec latency 141.574 ms 2024-04-24T23:32:09.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:09 smithi033 ceph-mon[29201]: pgmap v955: 129 pgs: 129 active+clean; 584 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 -- 172.21.15.33:0/630381948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44101870 msgr2=0x7fcf44101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/630381948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44101870 0x7fcf44101cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009860 tx=0x7fcf2c02a340 comp rx=0 tx=0).stop 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 -- 172.21.15.33:0/630381948 shutdown_connections 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/630381948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf44102230 0x7fcf44108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/630381948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44101870 0x7fcf44101cf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/630381948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44100680 0x7fcf44100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 -- 172.21.15.33:0/630381948 >> 172.21.15.33:0/630381948 conn(0x7fcf440fbbf0 msgr2=0x7fcf440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 -- 172.21.15.33:0/630381948 shutdown_connections 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.960+0000 7fcf48ad2640 1 -- 172.21.15.33:0/630381948 wait complete. 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 Processor -- start 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 -- start start 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf44100680 0x7fcf44069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 0x7fcf4419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44102230 0x7fcf4419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410c300 con 0x7fcf44101870 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410c180 con 0x7fcf44102230 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf48ad2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410c480 con 0x7fcf44100680 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 0x7fcf4419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 0x7fcf4419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57410/0 (socket says 172.21.15.33:57410) 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf3bfff640 1 -- 172.21.15.33:0/2890509064 learned_addr learned my addr 172.21.15.33:0/2890509064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44102230 0x7fcf4419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.961+0000 7fcf4259c640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf44100680 0x7fcf44069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3bfff640 1 -- 172.21.15.33:0/2890509064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf44100680 msgr2=0x7fcf44069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf44100680 0x7fcf44069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3bfff640 1 -- 172.21.15.33:0/2890509064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44102230 msgr2=0x7fcf4419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44102230 0x7fcf4419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3bfff640 1 -- 172.21.15.33:0/2890509064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4419f6f0 con 0x7fcf44101870 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 0x7fcf4419c7f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005940 tx=0x7fcf2c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3b7fe640 1 -- 172.21.15.33:0/2890509064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c03f020 con 0x7fcf44101870 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3b7fe640 1 -- 172.21.15.33:0/2890509064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c030850 con 0x7fcf44101870 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4419f980 con 0x7fcf44101870 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf3b7fe640 1 -- 172.21.15.33:0/2890509064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c037040 con 0x7fcf44101870 2024-04-24T23:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.962+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4419fe70 con 0x7fcf44101870 2024-04-24T23:32:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.965+0000 7fcf4b7dd740 1 Processor -- start 2024-04-24T23:32:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.965+0000 7fcf4b7dd740 1 -- start start 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c115b0 0x559b89c13a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559b89b67670 con 0x559b89c115b0 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b89b0f970 con 0x559b89c021c0 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b89b63630 con 0x559b89c0eb50 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf3b7fe640 1 -- 172.21.15.33:0/2890509064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcf2c042020 con 0x7fcf44101870 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c0e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60506/0 (socket says 172.21.15.33:60506) 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1605660378 learned_addr learned my addr 172.21.15.33:0/1605660378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4259c640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c115b0 0x559b89c13a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1605660378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 msgr2=0x559b89c10f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c10f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1605660378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c115b0 msgr2=0x559b89c13a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c115b0 0x559b89c13a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1605660378 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b89b33400 con 0x559b89c021c0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf4259c640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.966+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c115b0 0x559b89c13a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c0e580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009fd0 tx=0x7fcf3c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4860d99cfb5b8f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf3b7fe640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf300a2b90 0x7fcf300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf38ff9640 1 -- 172.21.15.33:0/1605660378 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c004d20 con 0x559b89c021c0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf38ff9640 1 -- 172.21.15.33:0/1605660378 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c004050 con 0x559b89c021c0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf38ff9640 1 -- 172.21.15.33:0/1605660378 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c005280 con 0x559b89c021c0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf4259c640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf300a2b90 0x7fcf300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.967+0000 7fcf3b7fe640 1 -- 172.21.15.33:0/2890509064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcf2c0ef720 con 0x7fcf44101870 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4259c640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf300a2b90 0x7fcf300a5050 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559b89c01f60 tx=0x7fcf34009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1605660378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 msgr2=0x559b89c0e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c0e580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009fd0 tx=0x7fcf3c0118d0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1605660378 shutdown_connections 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c115b0 0x559b89c13a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1605660378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c0e580 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1605660378 >> 172.21.15.33:0/1605660378 conn(0x559b89c02cb0 msgr2=0x559b89c030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1605660378 shutdown_connections 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1605660378 wait complete. 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 -- start start 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.968+0000 7fcf4b7dd740 1 Processor -- start 2024-04-24T23:32:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 0x559b89c160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89c1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559b89c14540 con 0x559b89c0eb50 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b89b23050 con 0x559b89c021c0 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b89c146c0 con 0x559b89c115b0 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4b7dd740 10 client.0 _ll_register_callbacks cb 0x559b8987abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 0x559b89c160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 0x559b89c160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57436/0 (socket says 172.21.15.33:57436) 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf3bfff640 1 -- 172.21.15.33:0/1756063555 learned_addr learned my addr 172.21.15.33:0/1756063555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf4259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89c1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 msgr2=0x559b89c15bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c15bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1756063555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 msgr2=0x559b89c160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 0x559b89c160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 -- 172.21.15.33:0/1756063555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b89af4dc0 con 0x559b89c115b0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 0x559b89c160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89c1d130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c01b040 tx=0x7fcf3c003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1c17e354591545 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.969+0000 7fcf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf29ffb640 1 -- 172.21.15.33:0/1756063555 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c029020 con 0x559b89c115b0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c004050 con 0x559b89c115b0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf29ffb640 1 -- 172.21.15.33:0/1756063555 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcf3c00f4e0 con 0x559b89c115b0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf29ffb640 1 -- 172.21.15.33:0/1756063555 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcf3c034050 con 0x559b89c115b0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf4b7dd740 10 client.29020 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b8987b3f0 con 0x559b89c115b0 2024-04-24T23:32:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.970+0000 7fcf29ffb640 1 -- 172.21.15.33:0/1756063555 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c004050 con 0x559b89c115b0 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.971+0000 7fcf29ffb640 1 -- 172.21.15.33:0/1756063555 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcf3c0257d0 con 0x559b89c115b0 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.973+0000 7fcf4b7dd740 10 client.29020 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.973+0000 7fcf4b7dd740 10 client.29020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.973+0000 7fcf4b7dd740 20 client.29020 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.973+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c21800 0x559b89c23c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.973+0000 7fcf4b7dd740 4 client.29020 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.973+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b89b74bb0 con 0x559b89c21800 2024-04-24T23:32:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.976+0000 7fcf4259c640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c21800 0x559b89c23c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:09.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.979+0000 7fcf4259c640 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c21800 0x559b89c23c10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b265e4ff35a572e4 server_cookie=ae972b58d2e3d301 in_seq=0 out_seq=0 2024-04-24T23:32:09.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.979+0000 7fcf29ffb640 10 client.29020 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.983+0000 7fcf29ffb640 1 -- 172.21.15.33:0/1756063555 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b89b74bb0 con 0x559b89c21800 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.983+0000 7fcf29ffb640 10 client.29020 handle_command_reply: tid=0 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.983+0000 7fcf4b7dd740 1 client.29020 shutdown 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 msgr2=0x559b89c1d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89c1d130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c01b040 tx=0x7fcf3c003c60 comp rx=0 tx=0).stop 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 shutdown_connections 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c21800 0x559b89c23c10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89c1d130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf29ffb640 0 client.29020 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c0eb50 0x559b89c160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/1756063555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c021c0 0x559b89c15bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 >> 172.21.15.33:0/1756063555 conn(0x559b89c02cb0 msgr2=0x559b89c0f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:09.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 shutdown_connections 2024-04-24T23:32:09.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/1756063555 wait complete. 2024-04-24T23:32:09.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.984+0000 7fcf4b7dd740 20 client.29020 trim_cache size 0 max 16384 2024-04-24T23:32:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 Processor -- start 2024-04-24T23:32:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 -- start start 2024-04-24T23:32:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89bfe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c0eb50 0x559b89c1d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89bfe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441aaa50 con 0x559b89c021c0 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441a8320 con 0x559b89c0eb50 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441a6170 con 0x559b89c115b0 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89bfe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89bfe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57444/0 (socket says 172.21.15.33:57444) 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf4259c640 1 -- 172.21.15.33:0/3682241328 learned_addr learned my addr 172.21.15.33:0/3682241328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c0eb50 0x559b89c1d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:09.999+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89bfe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf4259c640 1 -- 172.21.15.33:0/3682241328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 msgr2=0x559b89bfe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf4259c640 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89bfe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf4259c640 1 -- 172.21.15.33:0/3682241328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c0eb50 msgr2=0x559b89c1d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf4259c640 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c0eb50 0x559b89c1d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf4259c640 1 -- 172.21.15.33:0/3682241328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b899457d0 con 0x559b89c021c0 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf4259c640 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89bfe2a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcf441016d0 tx=0x7fcf3401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f6eda45d35ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf2bfff640 1 -- 172.21.15.33:0/3682241328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34032030 con 0x559b89c021c0 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf2bfff640 1 -- 172.21.15.33:0/3682241328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf34005500 con 0x559b89c021c0 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.000+0000 7fcf2bfff640 1 -- 172.21.15.33:0/3682241328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3402d0f0 con 0x559b89c021c0 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/3682241328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 msgr2=0x559b89bfe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89bfe2a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcf441016d0 tx=0x7fcf3401b000 comp rx=0 tx=0).stop 2024-04-24T23:32:10.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/3682241328 shutdown_connections 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c115b0 0x559b89bfe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c0eb50 0x559b89c1d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/3682241328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89bfe2a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/3682241328 >> 172.21.15.33:0/3682241328 conn(0x559b89c02cb0 msgr2=0x559b89c284a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/3682241328 shutdown_connections 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/3682241328 wait complete. 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 -- start start 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.001+0000 7fcf4b7dd740 1 Processor -- start 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89c1c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c1c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c115b0 0x559b89c86130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441a67e0 con 0x559b89c021c0 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441a7c50 con 0x559b89c115b0 2024-04-24T23:32:10.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441a6eb0 con 0x559b89c0eb50 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 10 client.0 _ll_register_callbacks cb 0x7fcf3c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89c1c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89c1c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57446/0 (socket says 172.21.15.33:57446) 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 -- 172.21.15.33:0/599580613 learned_addr learned my addr 172.21.15.33:0/599580613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf42d9d640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c115b0 0x559b89c86130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c1c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 -- 172.21.15.33:0/599580613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 msgr2=0x559b89c1c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c1c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 -- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c115b0 msgr2=0x559b89c86130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c115b0 0x559b89c86130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 -- 172.21.15.33:0/599580613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b89c86720 con 0x559b89c021c0 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4259c640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89c1c460 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcf34030040 tx=0x7fcf3401d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8353ad8547fd89b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf29ffb640 1 -- 172.21.15.33:0/599580613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34032040 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf29ffb640 1 -- 172.21.15.33:0/599580613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcf340051c0 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.002+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf34005500 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf29ffb640 1 -- 172.21.15.33:0/599580613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcf34047050 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 10 client.19599 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b89c87a20 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf29ffb640 1 -- 172.21.15.33:0/599580613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf34005500 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf29ffb640 1 -- 172.21.15.33:0/599580613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcf3402d720 con 0x559b89c021c0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 10 client.19599 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 10 client.19599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 20 client.19599 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c883f0 0x559b89c8a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 4 client.19599 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b89c159a0 con 0x559b89c883f0 2024-04-24T23:32:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c883f0 0x559b89c8a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf3bfff640 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c883f0 0x559b89c8a810 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd1d890b87adf31f server_cookie=32ad263d409f3d8f in_seq=0 out_seq=0 2024-04-24T23:32:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.003+0000 7fcf29ffb640 10 client.19599 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.005+0000 7fcf29ffb640 1 -- 172.21.15.33:0/599580613 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b89c159a0 con 0x559b89c883f0 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.005+0000 7fcf29ffb640 10 client.19599 handle_command_reply: tid=0 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.005+0000 7fcf4b7dd740 1 client.19599 shutdown 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 msgr2=0x559b89c1c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89c1c460 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcf34030040 tx=0x7fcf3401d610 comp rx=0 tx=0).stop 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 shutdown_connections 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x559b89c883f0 0x559b89c8a810 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b89c115b0 0x559b89c86130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf29ffb640 0 client.19599 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b89c0eb50 0x559b89c1c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 --2- 172.21.15.33:0/599580613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b89c021c0 0x559b89c1c460 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 >> 172.21.15.33:0/599580613 conn(0x559b89c02cb0 msgr2=0x559b89c28d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 shutdown_connections 2024-04-24T23:32:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 1 -- 172.21.15.33:0/599580613 wait complete. 2024-04-24T23:32:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf4b7dd740 20 client.19599 trim_cache size 0 max 16384 2024-04-24T23:32:10.011 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf300a2b90 msgr2=0x7fcf300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf300a2b90 0x7fcf300a5050 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559b89c01f60 tx=0x7fcf34009260 comp rx=0 tx=0).stop 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 msgr2=0x7fcf4419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 0x7fcf4419c7f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005940 tx=0x7fcf2c033d90 comp rx=0 tx=0).stop 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 shutdown_connections 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcf300a2b90 0x7fcf300a5050 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44102230 0x7fcf4419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf44101870 0x7fcf4419c7f0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 --2- 172.21.15.33:0/2890509064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf44100680 0x7fcf44069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 >> 172.21.15.33:0/2890509064 conn(0x7fcf440fbbf0 msgr2=0x7fcf44104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 shutdown_connections 2024-04-24T23:32:10.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.006+0000 7fcf48ad2640 1 -- 172.21.15.33:0/2890509064 wait complete. 2024-04-24T23:32:10.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:32:10.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:09 smithi038 ceph-mon[32450]: pgmap v955: 129 pgs: 129 active+clean; 584 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-04-24T23:32:10.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1260339490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4102230 msgr2=0x7f94e4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1260339490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4102230 0x7f94e4108ac0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009900 tx=0x7f94dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1260339490 shutdown_connections 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1260339490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4102230 0x7f94e4108ac0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1260339490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94e4101870 0x7f94e4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1260339490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1260339490 >> 172.21.15.33:0/1260339490 conn(0x7f94e40fbbf0 msgr2=0x7f94e40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1260339490 shutdown_connections 2024-04-24T23:32:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.172+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1260339490 wait complete. 2024-04-24T23:32:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 Processor -- start 2024-04-24T23:32:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 -- start start 2024-04-24T23:32:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 0x7f94e4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94e4102230 0x7f94e419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94e410c2e0 con 0x7f94e4101870 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94e410c160 con 0x7f94e4100680 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e8ee7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94e410c460 con 0x7f94e4102230 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 0x7f94e4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60558/0 (socket says 172.21.15.33:60558) 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 -- 172.21.15.33:0/1323120431 learned_addr learned my addr 172.21.15.33:0/1323120431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 0x7f94e4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57474/0 (socket says 172.21.15.33:57474) 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94e4102230 0x7f94e419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 -- 172.21.15.33:0/1323120431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94e4102230 msgr2=0x7f94e419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94e4102230 0x7f94e419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 -- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 msgr2=0x7f94e4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 0x7f94e4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e259c640 1 -- 172.21.15.33:0/1323120431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e419f6b0 con 0x7f94e4100680 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.173+0000 7f94e1d9b640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 0x7f94e4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.174+0000 7f94e259c640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4197ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f94d400ebc0 tx=0x7f94d400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.174+0000 7f94d37fe640 1 -- 172.21.15.33:0/1323120431 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4003b70 con 0x7f94e4100680 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.174+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e419f9a0 con 0x7f94e4100680 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.174+0000 7f94d37fe640 1 -- 172.21.15.33:0/1323120431 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d4003d10 con 0x7f94e4100680 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.174+0000 7f94d37fe640 1 -- 172.21.15.33:0/1323120431 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4019980 con 0x7f94e4100680 2024-04-24T23:32:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.174+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e419fef0 con 0x7f94e4100680 2024-04-24T23:32:10.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:09 smithi117 ceph-mon[32529]: pgmap v955: 129 pgs: 129 active+clean; 584 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-04-24T23:32:10.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94d37fe640 1 -- 172.21.15.33:0/1323120431 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f94d4019b20 con 0x7f94e4100680 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 Processor -- start 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 -- start start 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 0x55c89e56e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e573720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e573d90 0x55c89e576220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c89e4d2670 con 0x55c89e56e230 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c89e47a970 con 0x55c89e56ec20 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c89e4ce630 con 0x55c89e573d90 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 0x55c89e56e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 0x55c89e56e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57478/0 (socket says 172.21.15.33:57478) 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94e259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e573720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2718040333 learned_addr learned my addr 172.21.15.33:0/2718040333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94e1d9b640 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e573d90 0x55c89e576220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.177+0000 7f94d37fe640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94cc0a2b90 0x7f94cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2718040333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e573d90 msgr2=0x55c89e576220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e573d90 0x55c89e576220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2718040333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 msgr2=0x55c89e573720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e573720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2718040333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89e49e400 con 0x55c89e56e230 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e259c640 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e573720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94d37fe640 1 -- 172.21.15.33:0/1323120431 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94d40cfbe0 con 0x7f94e4100680 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94cc0a2b90 0x7f94cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 0x55c89e56e650 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009900 tx=0x7f94dc031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a526c7b09567c11d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94cc0a2b90 0x7f94cc0a5050 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009230 tx=0x7f94dc048000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94d0ff9640 1 -- 172.21.15.33:0/2718040333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc035040 con 0x55c89e56e230 2024-04-24T23:32:10.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94d0ff9640 1 -- 172.21.15.33:0/2718040333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc004480 con 0x55c89e56e230 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.178+0000 7f94d0ff9640 1 -- 172.21.15.33:0/2718040333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc046970 con 0x55c89e56e230 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2718040333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 msgr2=0x55c89e56e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 0x55c89e56e650 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009900 tx=0x7f94dc031730 comp rx=0 tx=0).stop 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2718040333 shutdown_connections 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e573d90 0x55c89e576220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e573720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2718040333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e56e230 0x55c89e56e650 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2718040333 >> 172.21.15.33:0/2718040333 conn(0x55c89e5698a0 msgr2=0x55c89e569ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2718040333 shutdown_connections 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2718040333 wait complete. 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 -- start start 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.179+0000 7f94ebbf0740 1 Processor -- start 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e578350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e578890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c89e576d40 con 0x55c89e573d90 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c89e48e050 con 0x55c89e56e230 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c89e576ec0 con 0x55c89e56ec20 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 10 client.0 _ll_register_callbacks cb 0x55c89e1e5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e578350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57488/0 (socket says 172.21.15.33:57488) 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 -- 172.21.15.33:0/475625917 learned_addr learned my addr 172.21.15.33:0/475625917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e1d9b640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e578890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 -- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 msgr2=0x55c89e578890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e578890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 -- 172.21.15.33:0/475625917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 msgr2=0x55c89e578350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e578350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 -- 172.21.15.33:0/475625917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89e45fdc0 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e259c640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e578350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57f8d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005c40 tx=0x7f94dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc00dc8169401ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94c9ffb640 1 -- 172.21.15.33:0/475625917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc05a060 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94c9ffb640 1 -- 172.21.15.33:0/475625917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94dc046d10 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.180+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc004480 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94c9ffb640 1 -- 172.21.15.33:0/475625917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94dc0035f0 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 10 client.19611 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c89e1e63f0 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94c9ffb640 1 -- 172.21.15.33:0/475625917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc004480 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94c9ffb640 1 -- 172.21.15.33:0/475625917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f94dc051780 con 0x55c89e573d90 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 10 client.19611 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 10 client.19611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 20 client.19611 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e583dc0 0x55c89e5861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 4 client.19611 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c89e4dfbb0 con 0x55c89e583dc0 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.181+0000 7f94e259c640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e583dc0 0x55c89e5861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94e259c640 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e583dc0 0x55c89e5861b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8310a03be4cf15ae server_cookie=7204b49eddcb5029 in_seq=0 out_seq=0 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94c9ffb640 10 client.19611 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94c9ffb640 1 -- 172.21.15.33:0/475625917 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c89e4dfbb0 con 0x55c89e583dc0 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94c9ffb640 10 client.19611 handle_command_reply: tid=0 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94ebbf0740 1 client.19611 shutdown 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 msgr2=0x55c89e57f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.182+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57f8d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005c40 tx=0x7f94dc009260 comp rx=0 tx=0).stop 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 shutdown_connections 2024-04-24T23:32:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e583dc0 0x55c89e5861b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57f8d0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e578890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94c9ffb640 0 client.19611 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/475625917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e578350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 >> 172.21.15.33:0/475625917 conn(0x55c89e5698a0 msgr2=0x55c89e568b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 shutdown_connections 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 1 -- 172.21.15.33:0/475625917 wait complete. 2024-04-24T23:32:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.183+0000 7f94ebbf0740 20 client.19611 trim_cache size 0 max 16384 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.196+0000 7f94ebbf0740 1 Processor -- start 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.196+0000 7f94ebbf0740 1 -- start start 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.196+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56e230 0x55c89e5649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e57f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e5682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41acc40 con 0x55c89e573d90 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41aa510 con 0x55c89e56ec20 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41a8360 con 0x55c89e56e230 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e5682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e5682b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57492/0 (socket says 172.21.15.33:57492) 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 -- 172.21.15.33:0/3084267434 learned_addr learned my addr 172.21.15.33:0/3084267434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e1d9b640 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e57f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e259c640 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56e230 0x55c89e5649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 -- 172.21.15.33:0/3084267434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56e230 msgr2=0x55c89e5649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56e230 0x55c89e5649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 -- 172.21.15.33:0/3084267434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 msgr2=0x55c89e57f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e57f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 -- 172.21.15.33:0/3084267434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89e2b07d0 con 0x55c89e573d90 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e5682b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005ab0 tx=0x7f94dc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4e75120899923 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94cbfff640 1 -- 172.21.15.33:0/3084267434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc05a070 con 0x55c89e573d90 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94cbfff640 1 -- 172.21.15.33:0/3084267434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc0464b0 con 0x55c89e573d90 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.197+0000 7f94cbfff640 1 -- 172.21.15.33:0/3084267434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc051e60 con 0x55c89e573d90 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 -- 172.21.15.33:0/3084267434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 msgr2=0x55c89e5682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e5682b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005ab0 tx=0x7f94dc031400 comp rx=0 tx=0).stop 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 -- 172.21.15.33:0/3084267434 shutdown_connections 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e5682b0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56ec20 0x55c89e57f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/3084267434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56e230 0x55c89e5649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 -- 172.21.15.33:0/3084267434 >> 172.21.15.33:0/3084267434 conn(0x55c89e5698a0 msgr2=0x55c89e56ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 -- 172.21.15.33:0/3084267434 shutdown_connections 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 -- 172.21.15.33:0/3084267434 wait complete. 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 -- start start 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.198+0000 7f94ebbf0740 1 Processor -- start 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e57cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e57d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41a89d0 con 0x55c89e573d90 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41a9e40 con 0x55c89e56e230 2024-04-24T23:32:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41a90a0 con 0x55c89e56ec20 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94ebbf0740 10 client.0 _ll_register_callbacks cb 0x7f94dc04a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e57cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e57d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e57cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60606/0 (socket says 172.21.15.33:60606) 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e259c640 1 -- 172.21.15.33:0/2010908879 learned_addr learned my addr 172.21.15.33:0/2010908879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 msgr2=0x55c89e57d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e57d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2010908879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 msgr2=0x55c89e57cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e57cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 -- 172.21.15.33:0/2010908879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89e57ea80 con 0x55c89e573d90 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e259c640 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e57cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94e2d9d640 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57e490 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005ab0 tx=0x7f94dc0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673b8fc651cb935e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.199+0000 7f94c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94c9ffb640 1 -- 172.21.15.33:0/2010908879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc035030 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc0464b0 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94c9ffb640 1 -- 172.21.15.33:0/2010908879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94dc004480 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94c9ffb640 1 -- 172.21.15.33:0/2010908879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94dc074050 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 10 client.19617 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c89e5ea250 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94c9ffb640 1 -- 172.21.15.33:0/2010908879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc0464b0 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94c9ffb640 1 -- 172.21.15.33:0/2010908879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f94dc047880 con 0x55c89e573d90 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 10 client.19617 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 10 client.19617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 20 client.19617 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e5eaa90 0x55c89e5eceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 4 client.19617 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:32:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c89e4d9f00 con 0x55c89e5eaa90 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.200+0000 7f94e259c640 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e5eaa90 0x55c89e5eceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94e259c640 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e5eaa90 0x55c89e5eceb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f9f7f0e222142e9 server_cookie=54f3430e89a32b6d in_seq=0 out_seq=0 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94c9ffb640 10 client.19617 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94c9ffb640 1 -- 172.21.15.33:0/2010908879 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c89e4d9f00 con 0x55c89e5eaa90 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94c9ffb640 10 client.19617 handle_command_reply: tid=0 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 client.19617 shutdown 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 msgr2=0x55c89e57e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57e490 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005ab0 tx=0x7f94dc0315b0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 shutdown_connections 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55c89e5eaa90 0x55c89e5eceb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c89e573d90 0x55c89e57e490 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c89e56ec20 0x55c89e57d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94c9ffb640 0 client.19617 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 --2- 172.21.15.33:0/2010908879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89e56e230 0x55c89e57cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 >> 172.21.15.33:0/2010908879 conn(0x55c89e5698a0 msgr2=0x55c89e56f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 shutdown_connections 2024-04-24T23:32:10.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 1 -- 172.21.15.33:0/2010908879 wait complete. 2024-04-24T23:32:10.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.201+0000 7f94ebbf0740 20 client.19617 trim_cache size 0 max 16384 2024-04-24T23:32:10.208 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94cc0a2b90 msgr2=0x7f94cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94cc0a2b90 0x7f94cc0a5050 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009230 tx=0x7f94dc048000 comp rx=0 tx=0).stop 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 msgr2=0x7f94e4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4197ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f94d400ebc0 tx=0x7f94d400c6a0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 shutdown_connections 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f94cc0a2b90 0x7f94cc0a5050 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94e4102230 0x7f94e419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4101870 0x7f94e4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 --2- 172.21.15.33:0/1323120431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e4100680 0x7f94e4197ae0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 >> 172.21.15.33:0/1323120431 conn(0x7f94e40fbbf0 msgr2=0x7f94e40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 shutdown_connections 2024-04-24T23:32:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.202+0000 7f94e8ee7640 1 -- 172.21.15.33:0/1323120431 wait complete. 2024-04-24T23:32:10.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:32:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 -- 172.21.15.33:0/581802227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 msgr2=0x7fac7c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 --2- 172.21.15.33:0/581802227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c101cf0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fac70009900 tx=0x7fac7002a410 comp rx=0 tx=0).stop 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 -- 172.21.15.33:0/581802227 shutdown_connections 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 --2- 172.21.15.33:0/581802227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7c102230 0x7fac7c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 --2- 172.21.15.33:0/581802227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c101cf0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 --2- 172.21.15.33:0/581802227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 -- 172.21.15.33:0/581802227 >> 172.21.15.33:0/581802227 conn(0x7fac7c0fbbf0 msgr2=0x7fac7c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 -- 172.21.15.33:0/581802227 shutdown_connections 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.324+0000 7fac80f77640 1 -- 172.21.15.33:0/581802227 wait complete. 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 Processor -- start 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 -- start start 2024-04-24T23:32:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7c102230 0x7fac7c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c10c350 con 0x7fac7c101870 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c10c1d0 con 0x7fac7c100680 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac80f77640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c10c4d0 con 0x7fac7c102230 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60630/0 (socket says 172.21.15.33:60630) 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac7a59c640 1 -- 172.21.15.33:0/1449362957 learned_addr learned my addr 172.21.15.33:0/1449362957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57516/0 (socket says 172.21.15.33:57516) 2024-04-24T23:32:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7c102230 0x7fac7c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 -- 172.21.15.33:0/1449362957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7c102230 msgr2=0x7fac7c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7c102230 0x7fac7c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 -- 172.21.15.33:0/1449362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 msgr2=0x7fac7c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac79d9b640 1 -- 172.21.15.33:0/1449362957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c19f700 con 0x7fac7c101870 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.325+0000 7fac7a59c640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.326+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c198020 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fac7002a930 tx=0x7fac70032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.326+0000 7fac6b7fe640 1 -- 172.21.15.33:0/1449362957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7003f030 con 0x7fac7c101870 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.326+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c19f990 con 0x7fac7c101870 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.326+0000 7fac6b7fe640 1 -- 172.21.15.33:0/1449362957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac70003ce0 con 0x7fac7c101870 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.326+0000 7fac6b7fe640 1 -- 172.21.15.33:0/1449362957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70035070 con 0x7fac7c101870 2024-04-24T23:32:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.326+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c19fea0 con 0x7fac7c101870 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.329+0000 7fac6b7fe640 1 -- 172.21.15.33:0/1449362957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fac70042030 con 0x7fac7c101870 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.329+0000 7fac83d64740 1 Processor -- start 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.329+0000 7fac83d64740 1 -- start start 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.329+0000 7fac83d64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d52520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac83d64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d57320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac83d64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d59c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac83d64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561124cad670 con 0x561124d52100 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac83d64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561124c55970 con 0x561124d54f30 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac83d64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561124ca9630 con 0x561124d57860 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac79d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d59c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac79d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d59c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38292/0 (socket says 172.21.15.33:38292) 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.330+0000 7fac79d9b640 1 -- 172.21.15.33:0/856737885 learned_addr learned my addr 172.21.15.33:0/856737885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d52520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac6b7fe640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fac640a2c90 0x7fac640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac7ad9d640 1 -- 172.21.15.33:0/856737885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 msgr2=0x561124d59c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d59c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac7ad9d640 1 -- 172.21.15.33:0/856737885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 msgr2=0x561124d57320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d57320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac7ad9d640 1 -- 172.21.15.33:0/856737885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561124c79400 con 0x561124d52100 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.331+0000 7fac6b7fe640 1 -- 172.21.15.33:0/1449362957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fac700ef730 con 0x7fac7c101870 2024-04-24T23:32:10.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.332+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d52520 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fac74009f00 tx=0x7fac74011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fa7db1d4bc7cba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.332+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fac640a2c90 0x7fac640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.332+0000 7fac68ff9640 1 -- 172.21.15.33:0/856737885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74003a70 con 0x561124d52100 2024-04-24T23:32:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.332+0000 7fac68ff9640 1 -- 172.21.15.33:0/856737885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac74003c10 con 0x561124d52100 2024-04-24T23:32:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.332+0000 7fac68ff9640 1 -- 172.21.15.33:0/856737885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7400fb00 con 0x561124d52100 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.333+0000 7fac83d64740 1 -- 172.21.15.33:0/856737885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 msgr2=0x561124d52520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.333+0000 7fac83d64740 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d52520 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fac74009f00 tx=0x7fac74011810 comp rx=0 tx=0).stop 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.335+0000 7fac83d64740 1 -- 172.21.15.33:0/856737885 shutdown_connections 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.335+0000 7fac83d64740 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d59c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.335+0000 7fac83d64740 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d57320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.335+0000 7fac83d64740 1 --2- 172.21.15.33:0/856737885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d52520 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.335+0000 7fac83d64740 1 -- 172.21.15.33:0/856737885 >> 172.21.15.33:0/856737885 conn(0x561124d44820 msgr2=0x561124d44c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.336+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fac640a2c90 0x7fac640a5150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fac74009f00 tx=0x7fac74011810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.337+0000 7fac83d64740 1 -- 172.21.15.33:0/856737885 shutdown_connections 2024-04-24T23:32:10.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.337+0000 7fac83d64740 1 -- 172.21.15.33:0/856737885 wait complete. 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.337+0000 7fac83d64740 1 -- start start 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.337+0000 7fac83d64740 1 Processor -- start 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d5bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d5c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561124d5a5f0 con 0x561124d52100 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561124c69050 con 0x561124d54f30 2024-04-24T23:32:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561124d5a770 con 0x561124d57860 2024-04-24T23:32:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac83d64740 10 client.0 _ll_register_callbacks cb 0x5611249c0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac7ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac7ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38294/0 (socket says 172.21.15.33:38294) 2024-04-24T23:32:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.338+0000 7fac7ad9d640 1 -- 172.21.15.33:0/259642972 learned_addr learned my addr 172.21.15.33:0/259642972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.339+0000 7fac79d9b640 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d5c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.339+0000 7fac7ad9d640 1 -- 172.21.15.33:0/259642972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 msgr2=0x561124d5c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.339+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d5c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.339+0000 7fac7ad9d640 1 -- 172.21.15.33:0/259642972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 msgr2=0x561124d5bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.339+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d5bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.339+0000 7fac7ad9d640 1 -- 172.21.15.33:0/259642972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561124c3adc0 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d630f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac74011e10 tx=0x7fac7400f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b758b7d965c2a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac61ffb640 1 -- 172.21.15.33:0/259642972 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74032020 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac61ffb640 1 -- 172.21.15.33:0/259642972 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fac74004410 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac74003c10 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac61ffb640 1 -- 172.21.15.33:0/259642972 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fac7403d050 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac83d64740 10 client.29056 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.340+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5611249c13f0 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac61ffb640 1 -- 172.21.15.33:0/259642972 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac74005530 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac61ffb640 1 -- 172.21.15.33:0/259642972 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fac74018070 con 0x561124d57860 2024-04-24T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac83d64740 10 client.29056 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac83d64740 10 client.29056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac83d64740 20 client.29056 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac83d64740 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d677e0 0x561124d69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac83d64740 4 client.29056 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.341+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac74003c10 con 0x561124d677e0 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.342+0000 7fac7a59c640 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d677e0 0x561124d69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.342+0000 7fac61ffb640 10 client.29056 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.342+0000 7fac7a59c640 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d677e0 0x561124d69c60 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574fa1d2fcd52026 server_cookie=9059865d09a4cd7e in_seq=0 out_seq=0 2024-04-24T23:32:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.344+0000 7fac61ffb640 1 -- 172.21.15.33:0/259642972 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fac74003c10 con 0x561124d677e0 2024-04-24T23:32:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.344+0000 7fac61ffb640 10 client.29056 handle_command_reply: tid=0 2024-04-24T23:32:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.344+0000 7fac83d64740 1 client.29056 shutdown 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 msgr2=0x561124d630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d630f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac74011e10 tx=0x7fac7400f580 comp rx=0 tx=0).stop 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 shutdown_connections 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d677e0 0x561124d69c60 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d57860 0x561124d630f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d54f30 0x561124d5c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 --2- 172.21.15.33:0/259642972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.346+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 >> 172.21.15.33:0/259642972 conn(0x561124d44820 msgr2=0x561124d43b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.347+0000 7fac61ffb640 0 client.29056 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.349+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 shutdown_connections 2024-04-24T23:32:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.349+0000 7fac83d64740 1 -- 172.21.15.33:0/259642972 wait complete. 2024-04-24T23:32:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.349+0000 7fac83d64740 20 client.29056 trim_cache size 0 max 16384 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 Processor -- start 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 -- start start 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 0x561124d437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1aaad0 con 0x561124d52100 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1a83a0 con 0x561124d44820 2024-04-24T23:32:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac83d64740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1a61f0 con 0x561124d54f30 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac7ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 0x561124d437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac7ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 0x561124d437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38296/0 (socket says 172.21.15.33:38296) 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.371+0000 7fac7ad9d640 1 -- 172.21.15.33:0/1628073224 learned_addr learned my addr 172.21.15.33:0/1628073224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac7ad9d640 1 -- 172.21.15.33:0/1628073224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 msgr2=0x561124d432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac7ad9d640 1 -- 172.21.15.33:0/1628073224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 msgr2=0x561124d630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac7ad9d640 1 -- 172.21.15.33:0/1628073224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561124a8b7d0 con 0x561124d54f30 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 0x561124d437f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac74003f50 tx=0x7fac74004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d98e2c5ea3e97e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac63fff640 1 -- 172.21.15.33:0/1628073224 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74032040 con 0x561124d54f30 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.372+0000 7fac63fff640 1 -- 172.21.15.33:0/1628073224 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac74046070 con 0x561124d54f30 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.373+0000 7fac63fff640 1 -- 172.21.15.33:0/1628073224 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74004470 con 0x561124d54f30 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.373+0000 7fac83d64740 1 -- 172.21.15.33:0/1628073224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 msgr2=0x561124d437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.373+0000 7fac83d64740 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 0x561124d437f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac74003f50 tx=0x7fac74004700 comp rx=0 tx=0).stop 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.373+0000 7fac83d64740 1 -- 172.21.15.33:0/1628073224 shutdown_connections 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.373+0000 7fac83d64740 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d54f30 0x561124d437f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d52100 0x561124d630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 --2- 172.21.15.33:0/1628073224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 -- 172.21.15.33:0/1628073224 >> 172.21.15.33:0/1628073224 conn(0x561124d5b730 msgr2=0x561124d59630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 -- 172.21.15.33:0/1628073224 shutdown_connections 2024-04-24T23:32:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 -- 172.21.15.33:0/1628073224 wait complete. 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 -- start start 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 Processor -- start 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.374+0000 7fac83d64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac83d64740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d52100 0x561124d60b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac83d64740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d54f30 0x561124d61ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac83d64740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1a6860 con 0x561124d54f30 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac83d64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1a7cd0 con 0x561124d44820 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60670/0 (socket says 172.21.15.33:60670) 2024-04-24T23:32:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac7a59c640 1 -- 172.21.15.33:0/1430155303 learned_addr learned my addr 172.21.15.33:0/1430155303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d52100 0x561124d60b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1a6f30 con 0x561124d52100 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac83d64740 10 client.0 _ll_register_callbacks cb 0x7fac74009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d54f30 0x561124d61ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.375+0000 7fac79d9b640 1 -- 172.21.15.33:0/1430155303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 msgr2=0x561124d605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac79d9b640 1 -- 172.21.15.33:0/1430155303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d54f30 msgr2=0x561124d61ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d54f30 0x561124d61ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac79d9b640 1 -- 172.21.15.33:0/1430155303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561124d620b0 con 0x561124d52100 2024-04-24T23:32:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac7ad9d640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d54f30 0x561124d61ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac79d9b640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d52100 0x561124d60b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fac700bbbe0 tx=0x7fac70009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b454cb2d86143f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.376+0000 7fac61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.377+0000 7fac61ffb640 1 -- 172.21.15.33:0/1430155303 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70049040 con 0x561124d52100 2024-04-24T23:32:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.377+0000 7fac61ffb640 1 -- 172.21.15.33:0/1430155303 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fac700ef400 con 0x561124d52100 2024-04-24T23:32:10.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.378+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac74046070 con 0x561124d52100 2024-04-24T23:32:10.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.379+0000 7fac61ffb640 1 -- 172.21.15.33:0/1430155303 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fac700bb5b0 con 0x561124d52100 2024-04-24T23:32:10.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.380+0000 7fac83d64740 10 client.29062 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:10.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.380+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561124d63420 con 0x561124d52100 2024-04-24T23:32:10.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.380+0000 7fac61ffb640 1 -- 172.21.15.33:0/1430155303 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac74046070 con 0x561124d52100 2024-04-24T23:32:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.381+0000 7fac61ffb640 1 -- 172.21.15.33:0/1430155303 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fac700b82c0 con 0x561124d52100 2024-04-24T23:32:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.381+0000 7fac83d64740 10 client.29062 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.382+0000 7fac83d64740 10 client.29062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.382+0000 7fac83d64740 20 client.29062 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.382+0000 7fac83d64740 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d57860 0x561124dd0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.382+0000 7fac83d64740 4 client.29062 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:32:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.382+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561124cbabb0 con 0x561124d57860 2024-04-24T23:32:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.384+0000 7fac7a59c640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d57860 0x561124dd0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.387+0000 7fac7a59c640 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d57860 0x561124dd0570 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a81ca5854108665 server_cookie=ae39857b4bf1881a in_seq=0 out_seq=0 2024-04-24T23:32:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.388+0000 7fac61ffb640 10 client.29062 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.388+0000 7fac61ffb640 1 -- 172.21.15.33:0/1430155303 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561124cbabb0 con 0x561124d57860 2024-04-24T23:32:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.388+0000 7fac61ffb640 10 client.29062 handle_command_reply: tid=0 2024-04-24T23:32:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.388+0000 7fac83d64740 1 client.29062 shutdown 2024-04-24T23:32:10.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.389+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d52100 msgr2=0x561124d60b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.389+0000 7fac83d64740 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d52100 0x561124d60b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fac700bbbe0 tx=0x7fac70009670 comp rx=0 tx=0).stop 2024-04-24T23:32:10.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.389+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 shutdown_connections 2024-04-24T23:32:10.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.389+0000 7fac83d64740 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561124d57860 0x561124dd0570 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.389+0000 7fac83d64740 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561124d54f30 0x561124d61ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.389+0000 7fac83d64740 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561124d52100 0x561124d60b20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.390+0000 7fac83d64740 1 --2- 172.21.15.33:0/1430155303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561124d44820 0x561124d605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.390+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 >> 172.21.15.33:0/1430155303 conn(0x561124d5b730 msgr2=0x561124d432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.390+0000 7fac61ffb640 0 client.29062 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.390+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 shutdown_connections 2024-04-24T23:32:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.390+0000 7fac83d64740 1 -- 172.21.15.33:0/1430155303 wait complete. 2024-04-24T23:32:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.391+0000 7fac83d64740 20 client.29062 trim_cache size 0 max 16384 2024-04-24T23:32:10.392 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:32:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.392+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fac640a2c90 msgr2=0x7fac640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.392+0000 7fac80f77640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fac640a2c90 0x7fac640a5150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fac74009f00 tx=0x7fac74011810 comp rx=0 tx=0).stop 2024-04-24T23:32:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 msgr2=0x7fac7c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c198020 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fac7002a930 tx=0x7fac70032ce0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 shutdown_connections 2024-04-24T23:32:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fac640a2c90 0x7fac640a5150 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7c102230 0x7fac7c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac7c101870 0x7fac7c198020 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 --2- 172.21.15.33:0/1449362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac7c100680 0x7fac7c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:10.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.393+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 >> 172.21.15.33:0/1449362957 conn(0x7fac7c0fbbf0 msgr2=0x7fac7c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.394+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 shutdown_connections 2024-04-24T23:32:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:10.394+0000 7fac80f77640 1 -- 172.21.15.33:0/1449362957 wait complete. 2024-04-24T23:32:10.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 161039 22.47 MB/sec execute 255 sec latency 104.512 ms 2024-04-24T23:32:10.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:10 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2606433609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:11.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:10 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2606433609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:11.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:10 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2606433609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:11.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:32:11.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:11.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:32:11.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 -- 172.21.15.33:0/4071039369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b81017a0 msgr2=0x7f08b8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 --2- 172.21.15.33:0/4071039369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b81017a0 0x7f08b8101bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009900 tx=0x7f08ac02a3f0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 -- 172.21.15.33:0/4071039369 shutdown_connections 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 --2- 172.21.15.33:0/4071039369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b8103350 0x7f08b8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 --2- 172.21.15.33:0/4071039369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 --2- 172.21.15.33:0/4071039369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b81017a0 0x7f08b8101bc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 -- 172.21.15.33:0/4071039369 >> 172.21.15.33:0/4071039369 conn(0x7f08b80fcd10 msgr2=0x7f08b80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.598+0000 7f08bea8c640 1 -- 172.21.15.33:0/4071039369 shutdown_connections 2024-04-24T23:32:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.599+0000 7f08bea8c640 1 -- 172.21.15.33:0/4071039369 wait complete. 2024-04-24T23:32:11.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.599+0000 7f08bea8c640 1 Processor -- start 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.599+0000 7f08bea8c640 1 -- start start 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bea8c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81017a0 0x7f08b8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bea8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bea8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b8103350 0x7f08b819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bea8c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08b810d440 con 0x7f08b8102990 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bea8c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08b810d2c0 con 0x7f08b81017a0 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bea8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08b810d5c0 con 0x7f08b8103350 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bc828640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81017a0 0x7f08b8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bc828640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81017a0 0x7f08b8197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60684/0 (socket says 172.21.15.33:60684) 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57570/0 (socket says 172.21.15.33:57570) 2024-04-24T23:32:11.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bc828640 1 -- 172.21.15.33:0/3753436206 learned_addr learned my addr 172.21.15.33:0/3753436206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.600+0000 7f08bd029640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b8103350 0x7f08b819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.601+0000 7f08b7fff640 1 -- 172.21.15.33:0/3753436206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b8103350 msgr2=0x7f08b819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.601+0000 7f08b7fff640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b8103350 0x7f08b819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.601+0000 7f08b7fff640 1 -- 172.21.15.33:0/3753436206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81017a0 msgr2=0x7f08b8197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.601+0000 7f08b7fff640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81017a0 0x7f08b8197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.601+0000 7f08b7fff640 1 -- 172.21.15.33:0/3753436206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b819f730 con 0x7f08b8102990 2024-04-24T23:32:11.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.601+0000 7f08b7fff640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8198050 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e9f0 tx=0x7f08a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.602+0000 7f08b5ffb640 1 -- 172.21.15.33:0/3753436206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400ccc0 con 0x7f08b8102990 2024-04-24T23:32:11.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.602+0000 7f08b5ffb640 1 -- 172.21.15.33:0/3753436206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a4010910 con 0x7f08b8102990 2024-04-24T23:32:11.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.602+0000 7f08b5ffb640 1 -- 172.21.15.33:0/3753436206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4018960 con 0x7f08b8102990 2024-04-24T23:32:11.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.602+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b819fa20 con 0x7f08b8102990 2024-04-24T23:32:11.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.602+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b819ff20 con 0x7f08b8102990 2024-04-24T23:32:11.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 Processor -- start 2024-04-24T23:32:11.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 -- start start 2024-04-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 0x556d4f4475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f44c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f44cd00 0x556d4f44f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d4f3a62c0 con 0x556d4f447b90 2024-04-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d4f34e910 con 0x556d4f44cd00 2024-04-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.606+0000 7f08c1796740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d4f3a2280 con 0x556d4f4471a0 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 0x556d4f4475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 0x556d4f4475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38326/0 (socket says 172.21.15.33:38326) 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 -- 172.21.15.33:0/3315163510 learned_addr learned my addr 172.21.15.33:0/3315163510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bc828640 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f44c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08b5ffb640 1 -- 172.21.15.33:0/3753436206 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f08a4010ac0 con 0x7f08b8102990 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 -- 172.21.15.33:0/3315163510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f44cd00 msgr2=0x556d4f44f190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f44cd00 0x556d4f44f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 -- 172.21.15.33:0/3315163510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 msgr2=0x556d4f44c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f44c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.607+0000 7f08bd029640 1 -- 172.21.15.33:0/3315163510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4f35ffb0 con 0x556d4f4471a0 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.608+0000 7f08b5ffb640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08a80a2c60 0x7f08a80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.608+0000 7f08b5ffb640 1 -- 172.21.15.33:0/3753436206 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f08a40cf6e0 con 0x7f08b8102990 2024-04-24T23:32:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.608+0000 7f08bc828640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08a80a2c60 0x7f08a80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.609+0000 7f08bd029640 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 0x556d4f4475c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08b000ae90 tx=0x7f08b000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee62b5636b8390a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.609+0000 7f08a2ffd640 1 -- 172.21.15.33:0/3315163510 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003b70 con 0x556d4f4471a0 2024-04-24T23:32:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.609+0000 7f08a2ffd640 1 -- 172.21.15.33:0/3315163510 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0003d10 con 0x556d4f4471a0 2024-04-24T23:32:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.609+0000 7f08a2ffd640 1 -- 172.21.15.33:0/3315163510 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0010c70 con 0x556d4f4471a0 2024-04-24T23:32:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 -- 172.21.15.33:0/3315163510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 msgr2=0x556d4f4475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 0x556d4f4475c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08b000ae90 tx=0x7f08b000f790 comp rx=0 tx=0).stop 2024-04-24T23:32:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 -- 172.21.15.33:0/3315163510 shutdown_connections 2024-04-24T23:32:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f44cd00 0x556d4f44f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f44c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 --2- 172.21.15.33:0/3315163510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f4471a0 0x556d4f4475c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08c1796740 1 -- 172.21.15.33:0/3315163510 >> 172.21.15.33:0/3315163510 conn(0x556d4f40a950 msgr2=0x556d4f43e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.610+0000 7f08bc828640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08a80a2c60 0x7f08a80a5120 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000c00 tx=0x7f08ac005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.611+0000 7f08c1796740 1 -- 172.21.15.33:0/3315163510 shutdown_connections 2024-04-24T23:32:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.611+0000 7f08c1796740 1 -- 172.21.15.33:0/3315163510 wait complete. 2024-04-24T23:32:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.611+0000 7f08c1796740 1 -- start start 2024-04-24T23:32:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.612+0000 7f08c1796740 1 Processor -- start 2024-04-24T23:32:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.612+0000 7f08c1796740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f451680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08c1796740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f451bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08c1796740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f458c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08c1796740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d4f44fcb0 con 0x556d4f44cd00 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08c1796740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d4f361e90 con 0x556d4f4471a0 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08c1796740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d4f44fe30 con 0x556d4f447b90 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08bc828640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f451680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08bc828640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f451680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60700/0 (socket says 172.21.15.33:60700) 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.613+0000 7f08bc828640 1 -- 172.21.15.33:0/1747042380 learned_addr learned my addr 172.21.15.33:0/1747042380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08c1796740 10 client.0 _ll_register_callbacks cb 0x556d4f39d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08bd029640 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f458c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08bd029640 1 -- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 msgr2=0x556d4f451bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08bd029640 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f451bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08bd029640 1 -- 172.21.15.33:0/1747042380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 msgr2=0x556d4f451680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08bd029640 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f451680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.614+0000 7f08bd029640 1 -- 172.21.15.33:0/1747042380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4f1827e0 con 0x556d4f44cd00 2024-04-24T23:32:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.615+0000 7f08bd029640 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f458c00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f08b000ae90 tx=0x7f08b000abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a35e2d666ae7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.615+0000 7f0897fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.615+0000 7f0897fff640 1 -- 172.21.15.33:0/1747042380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b001d040 con 0x556d4f44cd00 2024-04-24T23:32:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.615+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b0003d10 con 0x556d4f44cd00 2024-04-24T23:32:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.617+0000 7f0897fff640 1 -- 172.21.15.33:0/1747042380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f08b0004a40 con 0x556d4f44cd00 2024-04-24T23:32:11.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.619+0000 7f0897fff640 1 -- 172.21.15.33:0/1747042380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08b0030050 con 0x556d4f44cd00 2024-04-24T23:32:11.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f08c1796740 10 client.19641 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:11.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d4f1e2f70 con 0x556d4f44cd00 2024-04-24T23:32:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f0897fff640 1 -- 172.21.15.33:0/1747042380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0003d10 con 0x556d4f44cd00 2024-04-24T23:32:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f0897fff640 1 -- 172.21.15.33:0/1747042380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f08b0005050 con 0x556d4f44cd00 2024-04-24T23:32:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f08c1796740 10 client.19641 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f08c1796740 10 client.19641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f08c1796740 20 client.19641 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.620+0000 7f08c1796740 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f45cd90 0x556d4f45f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.621+0000 7f08c1796740 4 client.19641 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.621+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d4f37a5a0 con 0x556d4f45cd90 2024-04-24T23:32:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.621+0000 7f08bc828640 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f45cd90 0x556d4f45f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.621+0000 7f08bc828640 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f45cd90 0x556d4f45f1b0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5383d9501c521d63 server_cookie=9fdda9f2a40ce8c5 in_seq=0 out_seq=0 2024-04-24T23:32:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.622+0000 7f0897fff640 10 client.19641 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.623+0000 7f0897fff640 1 -- 172.21.15.33:0/1747042380 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d4f37a5a0 con 0x556d4f45cd90 2024-04-24T23:32:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.623+0000 7f0897fff640 10 client.19641 handle_command_reply: tid=0 2024-04-24T23:32:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.623+0000 7f08c1796740 1 client.19641 shutdown 2024-04-24T23:32:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 msgr2=0x556d4f458c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f458c00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f08b000ae90 tx=0x7f08b000abb0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 shutdown_connections 2024-04-24T23:32:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f45cd90 0x556d4f45f1b0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f458c00 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f451bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 --2- 172.21.15.33:0/1747042380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f451680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 >> 172.21.15.33:0/1747042380 conn(0x556d4f40a950 msgr2=0x556d4f43d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.625+0000 7f0897fff640 0 client.19641 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.626+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 shutdown_connections 2024-04-24T23:32:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.626+0000 7f08c1796740 1 -- 172.21.15.33:0/1747042380 wait complete. 2024-04-24T23:32:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.626+0000 7f08c1796740 20 client.19641 trim_cache size 0 max 16384 2024-04-24T23:32:11.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.658+0000 7f08c1796740 1 Processor -- start 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.658+0000 7f08c1796740 1 -- start start 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08c1796740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f441ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08c1796740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f458bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08c1796740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f463a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08c1796740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81aab60 con 0x556d4f44cd00 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08c1796740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81a8430 con 0x556d4f4471a0 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08c1796740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81a6280 con 0x556d4f447b90 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f458bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f458bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:38354/0 (socket says 172.21.15.33:38354) 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08b7fff640 1 -- 172.21.15.33:0/2494220025 learned_addr learned my addr 172.21.15.33:0/2494220025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08bd029640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f463a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.659+0000 7f08bc828640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f441ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08b7fff640 1 -- 172.21.15.33:0/2494220025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 msgr2=0x556d4f441ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08b7fff640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f441ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08b7fff640 1 -- 172.21.15.33:0/2494220025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 msgr2=0x556d4f463a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08b7fff640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f463a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08b7fff640 1 -- 172.21.15.33:0/2494220025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4f45f520 con 0x556d4f447b90 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08bd029640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08bc828640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f441ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08b7fff640 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f458bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e9c0 tx=0x7f08a409b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b2e8d714df00a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08a27fc640 1 -- 172.21.15.33:0/2494220025 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a4820 con 0x556d4f447b90 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08a27fc640 1 -- 172.21.15.33:0/2494220025 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a4099f60 con 0x556d4f447b90 2024-04-24T23:32:11.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.660+0000 7f08a27fc640 1 -- 172.21.15.33:0/2494220025 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a409b0b0 con 0x556d4f447b90 2024-04-24T23:32:11.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 -- 172.21.15.33:0/2494220025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 msgr2=0x556d4f458bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f458bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e9c0 tx=0x7f08a409b860 comp rx=0 tx=0).stop 2024-04-24T23:32:11.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 -- 172.21.15.33:0/2494220025 shutdown_connections 2024-04-24T23:32:11.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f44cd00 0x556d4f463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f447b90 0x556d4f458bc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 --2- 172.21.15.33:0/2494220025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f441ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 -- 172.21.15.33:0/2494220025 >> 172.21.15.33:0/2494220025 conn(0x556d4f40a950 msgr2=0x556d4f4417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 -- 172.21.15.33:0/2494220025 shutdown_connections 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.661+0000 7f08c1796740 1 -- 172.21.15.33:0/2494220025 wait complete. 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 -- start start 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 Processor -- start 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f4564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f456a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f44cd00 0x556d4f457a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81a68f0 con 0x556d4f447b90 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81a7d60 con 0x556d4f4471a0 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81a6fc0 con 0x556d4f44cd00 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08bc828640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f4564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08bc828640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f4564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60720/0 (socket says 172.21.15.33:60720) 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08bc828640 1 -- 172.21.15.33:0/3476904745 learned_addr learned my addr 172.21.15.33:0/3476904745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08c1796740 10 client.0 _ll_register_callbacks cb 0x7f08ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08b7fff640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f456a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.662+0000 7f08bd029640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f44cd00 0x556d4f457a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08bc828640 1 -- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f44cd00 msgr2=0x556d4f457a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08bc828640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f44cd00 0x556d4f457a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08bc828640 1 -- 172.21.15.33:0/3476904745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 msgr2=0x556d4f456a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08bc828640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f456a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08bc828640 1 -- 172.21.15.33:0/3476904745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4f07a170 con 0x556d4f4471a0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08bc828640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f4564d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08ac005ab0 tx=0x7f08ac009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8187c611ec4022 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f0897fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f0897fff640 1 -- 172.21.15.33:0/3476904745 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac035040 con 0x556d4f4471a0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f0897fff640 1 -- 172.21.15.33:0/3476904745 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f08ac032b10 con 0x556d4f4471a0 2024-04-24T23:32:11.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a4099f60 con 0x556d4f4471a0 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f0897fff640 1 -- 172.21.15.33:0/3476904745 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08ac063050 con 0x556d4f4471a0 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08c1796740 10 client.29126 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d4f4c3360 con 0x556d4f4471a0 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.663+0000 7f0897fff640 1 -- 172.21.15.33:0/3476904745 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a4099f60 con 0x556d4f4471a0 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f0897fff640 1 -- 172.21.15.33:0/3476904745 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f08ac0046e0 con 0x556d4f4471a0 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08c1796740 10 client.29126 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08c1796740 10 client.29126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08c1796740 20 client.29126 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08c1796740 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f4c3d50 0x556d4f4c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08c1796740 4 client.29126 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d4f3b3800 con 0x556d4f4c3d50 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08b7fff640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f4c3d50 0x556d4f4c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f0897fff640 10 client.29126 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.664+0000 7f08b7fff640 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f4c3d50 0x556d4f4c6170 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d642fd81b71a39d9 server_cookie=1917d6357381a75b in_seq=0 out_seq=0 2024-04-24T23:32:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f0897fff640 1 -- 172.21.15.33:0/3476904745 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d4f3b3800 con 0x556d4f4c3d50 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f0897fff640 10 client.29126 handle_command_reply: tid=0 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 client.29126 shutdown 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 msgr2=0x556d4f4564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f4564d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08ac005ab0 tx=0x7f08ac009260 comp rx=0 tx=0).stop 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 shutdown_connections 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d4f4c3d50 0x556d4f4c6170 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4f44cd00 0x556d4f457a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d4f447b90 0x556d4f456a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f0897fff640 0 client.29126 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 --2- 172.21.15.33:0/3476904745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d4f4471a0 0x556d4f4564d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.666+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 >> 172.21.15.33:0/3476904745 conn(0x556d4f40a950 msgr2=0x556d4f43d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 shutdown_connections 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08c1796740 1 -- 172.21.15.33:0/3476904745 wait complete. 2024-04-24T23:32:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08c1796740 20 client.29126 trim_cache size 0 max 16384 2024-04-24T23:32:11.670 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:32:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:32:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d44080cf-03d3-4d85-87ed-93a0c88e49a8", 2024-04-24T23:32:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:32:11.670 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08a80a2c60 msgr2=0x7f08a80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08a80a2c60 0x7f08a80a5120 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000c00 tx=0x7f08ac005a40 comp rx=0 tx=0).stop 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 msgr2=0x7f08b8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8198050 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e9f0 tx=0x7f08a400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 shutdown_connections 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f08a80a2c60 0x7f08a80a5120 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b8103350 0x7f08b819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b8102990 0x7f08b8198050 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 --2- 172.21.15.33:0/3753436206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08b81017a0 0x7f08b8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.667+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 >> 172.21.15.33:0/3753436206 conn(0x7f08b80fcd10 msgr2=0x7f08b8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.668+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 shutdown_connections 2024-04-24T23:32:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.668+0000 7f08bea8c640 1 -- 172.21.15.33:0/3753436206 wait complete. 2024-04-24T23:32:11.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:32:11.679 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:32:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:11 smithi033 ceph-mon[29201]: pgmap v956: 129 pgs: 129 active+clean; 586 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 356 op/s 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2369541337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24100680 msgr2=0x7f5c24100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2369541337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24100680 0x7f5c24100aa0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009900 tx=0x7f5c1402a410 comp rx=0 tx=0).stop 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2369541337 shutdown_connections 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2369541337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24102230 0x7f5c24108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2369541337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c24101870 0x7f5c24101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2369541337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24100680 0x7f5c24100aa0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2369541337 >> 172.21.15.33:0/2369541337 conn(0x7f5c240fbbf0 msgr2=0x7f5c240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2369541337 shutdown_connections 2024-04-24T23:32:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.802+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2369541337 wait complete. 2024-04-24T23:32:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 Processor -- start 2024-04-24T23:32:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 -- start start 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24100680 0x7f5c24197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24101870 0x7f5c24198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c24102230 0x7f5c2419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410c2d0 con 0x7f5c24101870 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410c150 con 0x7f5c24102230 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c28ba9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410c450 con 0x7f5c24100680 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24101870 0x7f5c24198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24101870 0x7f5c24198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57628/0 (socket says 172.21.15.33:57628) 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c1bfff640 1 -- 172.21.15.33:0/2333619517 learned_addr learned my addr 172.21.15.33:0/2333619517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c2259c640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24100680 0x7f5c24197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.803+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c24102230 0x7f5c2419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c2259c640 1 -- 172.21.15.33:0/2333619517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c24102230 msgr2=0x7f5c2419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c2259c640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c24102230 0x7f5c2419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c2259c640 1 -- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24101870 msgr2=0x7f5c24198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c2259c640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24101870 0x7f5c24198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c2259c640 1 -- 172.21.15.33:0/2333619517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2419f740 con 0x7f5c24100680 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c2259c640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24100680 0x7f5c24197b20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009230 tx=0x7f5c14031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/2333619517 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1403f070 con 0x7f5c24100680 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/2333619517 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c14003cd0 con 0x7f5c24100680 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2419f9d0 con 0x7f5c24100680 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2419fe90 con 0x7f5c24100680 2024-04-24T23:32:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.804+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/2333619517 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14035070 con 0x7f5c24100680 2024-04-24T23:32:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.807+0000 7f5c2b99a740 1 Processor -- start 2024-04-24T23:32:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.807+0000 7f5c2b99a740 1 -- start start 2024-04-24T23:32:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.807+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/2333619517 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5c1403b460 con 0x7f5c24100680 2024-04-24T23:32:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.807+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 0x556a96359660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a9635e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96361230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 0x556a96359660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 0x556a96359660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60740/0 (socket says 172.21.15.33:60740) 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3850209604 learned_addr learned my addr 172.21.15.33:0/3850209604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96361230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a962bd6b0 con 0x556a9635eda0 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556a96265880 con 0x556a96359240 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556a962b9670 con 0x556a96359c30 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c2259c640 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a9635e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3850209604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 msgr2=0x556a9635e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a9635e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3850209604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 msgr2=0x556a96361230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96361230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3850209604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a9609b7d0 con 0x556a96359240 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.808+0000 7f5c1b7fe640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c100a2b90 0x7f5c100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.809+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 0x556a96359660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0039f0 tx=0x7f5c1c003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d688925c666310c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.809+0000 7f5c2259c640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c100a2b90 0x7f5c100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.809+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/2333619517 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c140ef730 con 0x7f5c24100680 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.809+0000 7f5c18ff9640 1 -- 172.21.15.33:0/3850209604 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c01a470 con 0x556a96359240 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.809+0000 7f5c18ff9640 1 -- 172.21.15.33:0/3850209604 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c015070 con 0x556a96359240 2024-04-24T23:32:11.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.809+0000 7f5c18ff9640 1 -- 172.21.15.33:0/3850209604 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c010900 con 0x556a96359240 2024-04-24T23:32:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 msgr2=0x556a96359660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 0x556a96359660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0039f0 tx=0x7f5c1c003eb0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2259c640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c100a2b90 0x7f5c100a5050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5c140b80b0 tx=0x7f5c14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 shutdown_connections 2024-04-24T23:32:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96361230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a9635e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3850209604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359240 0x556a96359660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 >> 172.21.15.33:0/3850209604 conn(0x556a963548b0 msgr2=0x556a96354cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 shutdown_connections 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3850209604 wait complete. 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 -- start start 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 Processor -- start 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.810+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96363700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96363c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a9635eda0 0x556a9636ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96363700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a96361d50 con 0x556a96359240 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556a96278f50 con 0x556a96359c30 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556a96361ed0 con 0x556a9635eda0 2024-04-24T23:32:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96363700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57640/0 (socket says 172.21.15.33:57640) 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 -- 172.21.15.33:0/556512106 learned_addr learned my addr 172.21.15.33:0/556512106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 10 client.0 _ll_register_callbacks cb 0x556a962b4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96363c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 -- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a9635eda0 msgr2=0x556a9636ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a9635eda0 0x556a9636ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 -- 172.21.15.33:0/556512106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 msgr2=0x556a96363c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96363c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 -- 172.21.15.33:0/556512106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a95fd13f0 con 0x556a96359240 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96363c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96363700 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006010 tx=0x7f5c1c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375019457a169aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c09ffb640 1 -- 172.21.15.33:0/556512106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c02a060 con 0x556a96359240 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c09ffb640 1 -- 172.21.15.33:0/556512106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c1c011650 con 0x556a96359240 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.811+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c1c015070 con 0x556a96359240 2024-04-24T23:32:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c09ffb640 1 -- 172.21.15.33:0/556512106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5c1c031050 con 0x556a96359240 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 10 client.19659 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556a95f910f0 con 0x556a96359240 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c09ffb640 1 -- 172.21.15.33:0/556512106 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c015070 con 0x556a96359240 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c09ffb640 1 -- 172.21.15.33:0/556512106 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5c1c01a470 con 0x556a96359240 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 10 client.19659 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 10 client.19659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 20 client.19659 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a9636f290 0x556a96371680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 4 client.19659 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a962cabf0 con 0x556a9636f290 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.812+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a9636f290 0x556a96371680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.813+0000 7f5c09ffb640 10 client.19659 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.813+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a9636f290 0x556a96371680 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d71667faa3b9144 server_cookie=992ae684ba5f1a1a in_seq=0 out_seq=0 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c09ffb640 1 -- 172.21.15.33:0/556512106 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a962cabf0 con 0x556a9636f290 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c09ffb640 10 client.19659 handle_command_reply: tid=0 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 client.19659 shutdown 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 msgr2=0x556a96363700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96363700 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006010 tx=0x7f5c1c005ae0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 shutdown_connections 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a9636f290 0x556a96371680 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a9635eda0 0x556a9636ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96363c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c09ffb640 0 client.19659 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/556512106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96363700 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 >> 172.21.15.33:0/556512106 conn(0x556a963548b0 msgr2=0x556a96353af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 shutdown_connections 2024-04-24T23:32:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 1 -- 172.21.15.33:0/556512106 wait complete. 2024-04-24T23:32:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.815+0000 7f5c2b99a740 20 client.19659 trim_cache size 0 max 16384 2024-04-24T23:32:11.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.835+0000 7f5c2b99a740 1 Processor -- start 2024-04-24T23:32:11.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.835+0000 7f5c2b99a740 1 -- start start 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359240 0x556a96363a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96357860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96357da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040168a0 con 0x556a9635eda0 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04014170 con 0x556a96359c30 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04011fc0 con 0x556a96359240 2024-04-24T23:32:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96357da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96357da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57650/0 (socket says 172.21.15.33:57650) 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 -- 172.21.15.33:0/1247885513 learned_addr learned my addr 172.21.15.33:0/1247885513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359240 0x556a96363a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96357860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 -- 172.21.15.33:0/1247885513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359240 msgr2=0x556a96363a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359240 0x556a96363a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 -- 172.21.15.33:0/1247885513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 msgr2=0x556a96357860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96357860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 -- 172.21.15.33:0/1247885513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a963719f0 con 0x556a9635eda0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96357da0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x556a96364910 tx=0x7f5c1c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8118f66717503ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.836+0000 7f5c0bfff640 1 -- 172.21.15.33:0/1247885513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c02a040 con 0x556a9635eda0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c0bfff640 1 -- 172.21.15.33:0/1247885513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c015070 con 0x556a9635eda0 2024-04-24T23:32:11.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c0bfff640 1 -- 172.21.15.33:0/1247885513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c010440 con 0x556a9635eda0 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 -- 172.21.15.33:0/1247885513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 msgr2=0x556a96357da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96357da0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x556a96364910 tx=0x7f5c1c009b20 comp rx=0 tx=0).stop 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 -- 172.21.15.33:0/1247885513 shutdown_connections 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a9635eda0 0x556a96357da0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a96359c30 0x556a96357860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/1247885513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359240 0x556a96363a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 -- 172.21.15.33:0/1247885513 >> 172.21.15.33:0/1247885513 conn(0x556a963548b0 msgr2=0x556a96363310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 -- 172.21.15.33:0/1247885513 shutdown_connections 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.837+0000 7f5c2b99a740 1 -- 172.21.15.33:0/1247885513 wait complete. 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 -- start start 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 Processor -- start 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96367be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a96368120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a9635eda0 0x556a96369090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04012630 con 0x556a96359240 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04013aa0 con 0x556a9635eda0 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04012d00 con 0x556a96359c30 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c2b99a740 10 client.0 _ll_register_callbacks cb 0x7f5c0c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96367be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96367be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57654/0 (socket says 172.21.15.33:57654) 2024-04-24T23:32:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3964390885 learned_addr learned my addr 172.21.15.33:0/3964390885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a96368120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.838+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a9635eda0 0x556a96369090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 msgr2=0x556a96368120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a96368120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3964390885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a9635eda0 msgr2=0x556a96369090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a9635eda0 0x556a96369090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c22d9d640 1 -- 172.21.15.33:0/3964390885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a96369680 con 0x556a96359240 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c22d9d640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96367be0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005b60 tx=0x7f5c1c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61a93ea1d623736 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c09ffb640 1 -- 172.21.15.33:0/3964390885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c03a040 con 0x556a96359240 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c1c015070 con 0x556a96359240 2024-04-24T23:32:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c09ffb640 1 -- 172.21.15.33:0/3964390885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c1c011860 con 0x556a96359240 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c09ffb640 1 -- 172.21.15.33:0/3964390885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5c1c02a040 con 0x556a96359240 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c2b99a740 10 client.19665 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.839+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556a963de190 con 0x556a96359240 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c09ffb640 1 -- 172.21.15.33:0/3964390885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c015070 con 0x556a96359240 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c09ffb640 1 -- 172.21.15.33:0/3964390885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5c1c035a00 con 0x556a96359240 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c2b99a740 10 client.19665 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c2b99a740 10 client.19665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c2b99a740 20 client.19665 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a963dea80 0x556a963e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c2b99a740 4 client.19665 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a962c4f40 con 0x556a963dea80 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a963dea80 0x556a963e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c1bfff640 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a963dea80 0x556a963e0ea0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9299732ee6518cb server_cookie=664f6d4a37e42694 in_seq=0 out_seq=0 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.840+0000 7f5c09ffb640 10 client.19665 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c09ffb640 1 -- 172.21.15.33:0/3964390885 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556a962c4f40 con 0x556a963dea80 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c09ffb640 10 client.19665 handle_command_reply: tid=0 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 client.19665 shutdown 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 msgr2=0x556a96367be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96367be0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005b60 tx=0x7f5c1c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 shutdown_connections 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556a963dea80 0x556a963e0ea0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a9635eda0 0x556a96369090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a96359c30 0x556a96368120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 --2- 172.21.15.33:0/3964390885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a96359240 0x556a96367be0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 >> 172.21.15.33:0/3964390885 conn(0x556a963548b0 msgr2=0x556a96353260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c09ffb640 0 client.19665 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 shutdown_connections 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 1 -- 172.21.15.33:0/3964390885 wait complete. 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.841+0000 7f5c2b99a740 20 client.19665 trim_cache size 0 max 16384 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:32:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "d44080cf-03d3-4d85-87ed-93a0c88e49a8": { 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d44080cf-03d3-4d85-87ed-93a0c88e49a8", 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:32:11.848 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c100a2b90 msgr2=0x7f5c100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c100a2b90 0x7f5c100a5050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5c140b80b0 tx=0x7f5c14009260 comp rx=0 tx=0).stop 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24100680 msgr2=0x7f5c24197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24100680 0x7f5c24197b20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009230 tx=0x7f5c14031b60 comp rx=0 tx=0).stop 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 shutdown_connections 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5c100a2b90 0x7f5c100a5050 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c24102230 0x7f5c2419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c24101870 0x7f5c24198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 --2- 172.21.15.33:0/2333619517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24100680 0x7f5c24197b20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 >> 172.21.15.33:0/2333619517 conn(0x7f5c240fbbf0 msgr2=0x7f5c24068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 shutdown_connections 2024-04-24T23:32:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:11.842+0000 7f5c28ba9640 1 -- 172.21.15.33:0/2333619517 wait complete. 2024-04-24T23:32:11.856 INFO:tasks.cephfs.filesystem:scrub status for tag:d44080cf-03d3-4d85-87ed-93a0c88e49a8 - {'path': '/', 'tag': 'd44080cf-03d3-4d85-87ed-93a0c88e49a8', 'options': 'recursive,force'} 2024-04-24T23:32:11.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 161280 22.41 MB/sec execute 256 sec latency 279.124 ms 2024-04-24T23:32:12.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:11 smithi038 ceph-mon[32450]: pgmap v956: 129 pgs: 129 active+clean; 586 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 356 op/s 2024-04-24T23:32:12.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:11 smithi117 ceph-mon[32529]: pgmap v956: 129 pgs: 129 active+clean; 586 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 356 op/s 2024-04-24T23:32:12.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 161748 22.41 MB/sec execute 257 sec latency 187.155 ms 2024-04-24T23:32:13.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:13.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:12 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:13.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:12 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:13.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:12 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:13.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 162215 22.43 MB/sec execute 258 sec latency 103.363 ms 2024-04-24T23:32:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:13 smithi038 ceph-mon[32450]: pgmap v957: 129 pgs: 129 active+clean; 588 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.7 MiB/s wr, 435 op/s 2024-04-24T23:32:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:13 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:14.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:13 smithi117 ceph-mon[32529]: pgmap v957: 129 pgs: 129 active+clean; 588 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.7 MiB/s wr, 435 op/s 2024-04-24T23:32:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:13 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:13 smithi033 ceph-mon[29201]: pgmap v957: 129 pgs: 129 active+clean; 588 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.7 MiB/s wr, 435 op/s 2024-04-24T23:32:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:13 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:14.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 162682 22.44 MB/sec execute 259 sec latency 85.279 ms 2024-04-24T23:32:15.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 163257 22.45 MB/sec execute 260 sec latency 65.485 ms 2024-04-24T23:32:16.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:15 smithi038 ceph-mon[32450]: pgmap v958: 129 pgs: 129 active+clean; 594 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.5 MiB/s wr, 374 op/s 2024-04-24T23:32:16.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:15 smithi117 ceph-mon[32529]: pgmap v958: 129 pgs: 129 active+clean; 594 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.5 MiB/s wr, 374 op/s 2024-04-24T23:32:16.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:15 smithi033 ceph-mon[29201]: pgmap v958: 129 pgs: 129 active+clean; 594 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.5 MiB/s wr, 374 op/s 2024-04-24T23:32:16.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 163523 22.42 MB/sec execute 261 sec latency 509.891 ms 2024-04-24T23:32:17.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 163936 22.44 MB/sec execute 262 sec latency 549.647 ms 2024-04-24T23:32:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:17 smithi038 ceph-mon[32450]: pgmap v959: 129 pgs: 129 active+clean; 594 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 356 op/s 2024-04-24T23:32:18.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:17 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:32:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:17 smithi117 ceph-mon[32529]: pgmap v959: 129 pgs: 129 active+clean; 594 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 356 op/s 2024-04-24T23:32:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:17 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:32:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:17 smithi033 ceph-mon[29201]: pgmap v959: 129 pgs: 129 active+clean; 594 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 356 op/s 2024-04-24T23:32:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:17 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:32:18.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 164378 22.47 MB/sec execute 263 sec latency 109.894 ms 2024-04-24T23:32:19.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:32:19.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:32:19.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:32:19.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:32:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:32:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:32:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:32:19.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:32:19.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:32:19.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 164903 22.51 MB/sec execute 264 sec latency 81.699 ms 2024-04-24T23:32:20.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:19 smithi038 ceph-mon[32450]: pgmap v960: 129 pgs: 129 active+clean; 601 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 524 op/s 2024-04-24T23:32:20.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:19 smithi117 ceph-mon[32529]: pgmap v960: 129 pgs: 129 active+clean; 601 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 524 op/s 2024-04-24T23:32:20.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:19 smithi033 ceph-mon[29201]: pgmap v960: 129 pgs: 129 active+clean; 601 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 524 op/s 2024-04-24T23:32:20.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 165261 22.49 MB/sec execute 265 sec latency 72.177 ms 2024-04-24T23:32:21.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 165554 22.46 MB/sec execute 266 sec latency 231.018 ms 2024-04-24T23:32:22.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:21 smithi038 ceph-mon[32450]: pgmap v961: 129 pgs: 129 active+clean; 600 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 391 op/s 2024-04-24T23:32:22.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:21 smithi117 ceph-mon[32529]: pgmap v961: 129 pgs: 129 active+clean; 600 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 391 op/s 2024-04-24T23:32:22.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:21 smithi033 ceph-mon[29201]: pgmap v961: 129 pgs: 129 active+clean; 600 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 391 op/s 2024-04-24T23:32:22.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 165758 22.41 MB/sec execute 267 sec latency 202.141 ms 2024-04-24T23:32:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:22 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:32:23.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:22 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:32:23.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:22 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:32:23.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:22 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:32:23.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:22 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:32:23.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:22 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:32:23.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:22 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:32:23.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:22 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:32:23.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:22 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:32:23.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 166117 22.39 MB/sec execute 268 sec latency 107.204 ms 2024-04-24T23:32:24.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:23 smithi038 ceph-mon[32450]: pgmap v962: 129 pgs: 129 active+clean; 606 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-04-24T23:32:24.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:23 smithi117 ceph-mon[32529]: pgmap v962: 129 pgs: 129 active+clean; 606 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-04-24T23:32:24.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:23 smithi033 ceph-mon[29201]: pgmap v962: 129 pgs: 129 active+clean; 606 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-04-24T23:32:24.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 166565 22.38 MB/sec execute 269 sec latency 65.105 ms 2024-04-24T23:32:25.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 167438 22.45 MB/sec execute 270 sec latency 52.441 ms 2024-04-24T23:32:26.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:25 smithi038 ceph-mon[32450]: pgmap v963: 129 pgs: 129 active+clean; 605 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-04-24T23:32:26.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:25 smithi117 ceph-mon[32529]: pgmap v963: 129 pgs: 129 active+clean; 605 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-04-24T23:32:26.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:25 smithi033 ceph-mon[29201]: pgmap v963: 129 pgs: 129 active+clean; 605 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-04-24T23:32:26.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 167963 22.50 MB/sec execute 271 sec latency 314.546 ms 2024-04-24T23:32:27.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 168228 22.47 MB/sec execute 272 sec latency 502.543 ms 2024-04-24T23:32:28.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:27 smithi038 ceph-mon[32450]: pgmap v964: 129 pgs: 129 active+clean; 605 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 373 op/s 2024-04-24T23:32:28.178 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:27 smithi117 ceph-mon[32529]: pgmap v964: 129 pgs: 129 active+clean; 605 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 373 op/s 2024-04-24T23:32:28.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:27 smithi033 ceph-mon[29201]: pgmap v964: 129 pgs: 129 active+clean; 605 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 373 op/s 2024-04-24T23:32:28.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 168588 22.49 MB/sec execute 273 sec latency 110.349 ms 2024-04-24T23:32:29.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 168961 22.48 MB/sec execute 274 sec latency 72.209 ms 2024-04-24T23:32:30.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:29 smithi038 ceph-mon[32450]: pgmap v965: 129 pgs: 129 active+clean; 590 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 526 op/s 2024-04-24T23:32:30.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:29 smithi033 ceph-mon[29201]: pgmap v965: 129 pgs: 129 active+clean; 590 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 526 op/s 2024-04-24T23:32:30.219 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:29 smithi117 ceph-mon[32529]: pgmap v965: 129 pgs: 129 active+clean; 590 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 526 op/s 2024-04-24T23:32:30.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 169389 22.48 MB/sec execute 275 sec latency 88.470 ms 2024-04-24T23:32:31.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 169564 22.43 MB/sec execute 276 sec latency 187.971 ms 2024-04-24T23:32:32.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:31 smithi038 ceph-mon[32450]: pgmap v966: 129 pgs: 129 active+clean; 591 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 350 op/s 2024-04-24T23:32:32.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:31 smithi117 ceph-mon[32529]: pgmap v966: 129 pgs: 129 active+clean; 591 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 350 op/s 2024-04-24T23:32:32.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:31 smithi033 ceph-mon[29201]: pgmap v966: 129 pgs: 129 active+clean; 591 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 350 op/s 2024-04-24T23:32:32.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 169803 22.38 MB/sec execute 277 sec latency 313.301 ms 2024-04-24T23:32:33.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 170338 22.39 MB/sec execute 278 sec latency 86.292 ms 2024-04-24T23:32:34.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:33 smithi033 ceph-mon[29201]: pgmap v967: 129 pgs: 129 active+clean; 596 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 474 op/s 2024-04-24T23:32:34.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:33 smithi038 ceph-mon[32450]: pgmap v967: 129 pgs: 129 active+clean; 596 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 474 op/s 2024-04-24T23:32:34.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:33 smithi117 ceph-mon[32529]: pgmap v967: 129 pgs: 129 active+clean; 596 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 474 op/s 2024-04-24T23:32:34.894 INFO:tasks.workunit.client.0.smithi033.stdout: 10 170826 22.41 MB/sec execute 279 sec latency 90.684 ms 2024-04-24T23:32:35.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 171487 22.43 MB/sec execute 280 sec latency 54.747 ms 2024-04-24T23:32:36.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:35 smithi038 ceph-mon[32450]: pgmap v968: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 393 op/s 2024-04-24T23:32:36.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:35 smithi117 ceph-mon[32529]: pgmap v968: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 393 op/s 2024-04-24T23:32:36.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:35 smithi033 ceph-mon[29201]: pgmap v968: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 393 op/s 2024-04-24T23:32:36.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 171881 22.46 MB/sec execute 281 sec latency 82.866 ms 2024-04-24T23:32:37.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 172135 22.41 MB/sec execute 282 sec latency 411.947 ms 2024-04-24T23:32:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:37 smithi038 ceph-mon[32450]: pgmap v969: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 352 op/s 2024-04-24T23:32:38.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:37 smithi117 ceph-mon[32529]: pgmap v969: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 352 op/s 2024-04-24T23:32:38.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:37 smithi033 ceph-mon[29201]: pgmap v969: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 352 op/s 2024-04-24T23:32:38.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 172488 22.40 MB/sec execute 283 sec latency 136.514 ms 2024-04-24T23:32:39.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 173151 22.48 MB/sec execute 284 sec latency 94.198 ms 2024-04-24T23:32:40.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:39 smithi038 ceph-mon[32450]: pgmap v970: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.9 MiB/s wr, 431 op/s 2024-04-24T23:32:40.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:39 smithi117 ceph-mon[32529]: pgmap v970: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.9 MiB/s wr, 431 op/s 2024-04-24T23:32:40.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:39 smithi033 ceph-mon[29201]: pgmap v970: 129 pgs: 129 active+clean; 596 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.9 MiB/s wr, 431 op/s 2024-04-24T23:32:40.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 173624 22.50 MB/sec execute 285 sec latency 84.681 ms 2024-04-24T23:32:41.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:32:41.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:32:41.897 INFO:tasks.workunit.client.0.smithi033.stdout: 10 173870 22.45 MB/sec execute 286 sec latency 109.817 ms 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.014+0000 7fa160129640 1 -- 172.21.15.33:0/1411135765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810b0b0 msgr2=0x7fa15810d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.014+0000 7fa160129640 1 --2- 172.21.15.33:0/1411135765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810b0b0 0x7fa15810d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa150009be0 tx=0x7fa15002a6f0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.015+0000 7fa160129640 1 -- 172.21.15.33:0/1411135765 shutdown_connections 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.015+0000 7fa160129640 1 --2- 172.21.15.33:0/1411135765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa15810da80 0x7fa15810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.015+0000 7fa160129640 1 --2- 172.21.15.33:0/1411135765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810b0b0 0x7fa15810d540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.015+0000 7fa160129640 1 --2- 172.21.15.33:0/1411135765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa158108740 0x7fa15810ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.015+0000 7fa160129640 1 -- 172.21.15.33:0/1411135765 >> 172.21.15.33:0/1411135765 conn(0x7fa15806e5d0 msgr2=0x7fa158070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.020+0000 7fa160129640 1 -- 172.21.15.33:0/1411135765 shutdown_connections 2024-04-24T23:32:42.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.022+0000 7fa160129640 1 -- 172.21.15.33:0/1411135765 wait complete. 2024-04-24T23:32:42.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.023+0000 7fa160129640 1 Processor -- start 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 -- start start 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa158108740 0x7fa158115020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15810b0b0 0x7fa158115560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810da80 0x7fa1581b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa15811c270 con 0x7fa158108740 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa15811c0f0 con 0x7fa15810b0b0 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.031+0000 7fa160129640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa15811c3f0 con 0x7fa15810da80 2024-04-24T23:32:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.032+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15810b0b0 0x7fa158115560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.035+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15810b0b0 0x7fa158115560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32838/0 (socket says 172.21.15.33:32838) 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.035+0000 7fa15d6c4640 1 -- 172.21.15.33:0/3999793630 learned_addr learned my addr 172.21.15.33:0/3999793630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.036+0000 7fa15dec5640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa158108740 0x7fa158115020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.037+0000 7fa15dec5640 1 -- 172.21.15.33:0/3999793630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810da80 msgr2=0x7fa1581b6940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.037+0000 7fa15dec5640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810da80 0x7fa1581b6940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.037+0000 7fa15dec5640 1 -- 172.21.15.33:0/3999793630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15810b0b0 msgr2=0x7fa158115560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.037+0000 7fa15dec5640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15810b0b0 0x7fa158115560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.037+0000 7fa15dec5640 1 -- 172.21.15.33:0/3999793630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1581b6f30 con 0x7fa158108740 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.038+0000 7fa15dec5640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa158108740 0x7fa158115020 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e9f0 tx=0x7fa14800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.038+0000 7fa14effd640 1 -- 172.21.15.33:0/3999793630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14800ccc0 con 0x7fa158108740 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.039+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1581b7220 con 0x7fa158108740 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.039+0000 7fa14effd640 1 -- 172.21.15.33:0/3999793630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa148010910 con 0x7fa158108740 2024-04-24T23:32:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.039+0000 7fa14effd640 1 -- 172.21.15.33:0/3999793630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148018960 con 0x7fa158108740 2024-04-24T23:32:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.043+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1581b77c0 con 0x7fa158108740 2024-04-24T23:32:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.049+0000 7fa14effd640 1 -- 172.21.15.33:0/3999793630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa148010ac0 con 0x7fa158108740 2024-04-24T23:32:42.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.050+0000 7fa162e39740 1 Processor -- start 2024-04-24T23:32:42.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 -- start start 2024-04-24T23:32:42.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e78660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e7d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e80230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564847ddc6b0 con 0x564847e78c30 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564847d84880 con 0x564847e78240 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa162e39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564847dd8670 con 0x564847e7dda0 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e80230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e80230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39422/0 (socket says 172.21.15.33:39422) 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.051+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2530369190 learned_addr learned my addr 172.21.15.33:0/2530369190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa14effd640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1440a2c50 0x7fa1440a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2530369190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 msgr2=0x564847e78660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa15d6c4640 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e78660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2530369190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 msgr2=0x564847e7d730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa15d6c4640 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e7d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2530369190 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564847bba7d0 con 0x564847e7dda0 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa14effd640 1 -- 172.21.15.33:0/3999793630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1480cf520 con 0x7fa158108740 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.052+0000 7fa15d6c4640 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e80230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa150009b30 tx=0x7fa150005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4819222fb3c380c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.054+0000 7fa143fff640 1 -- 172.21.15.33:0/2530369190 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150036040 con 0x564847e7dda0 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.054+0000 7fa143fff640 1 -- 172.21.15.33:0/2530369190 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa150030ac0 con 0x564847e7dda0 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.054+0000 7fa143fff640 1 -- 172.21.15.33:0/2530369190 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150004560 con 0x564847e7dda0 2024-04-24T23:32:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.054+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1440a2c50 0x7fa1440a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.055+0000 7fa162e39740 1 -- 172.21.15.33:0/2530369190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 msgr2=0x564847e80230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.055+0000 7fa162e39740 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e80230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa150009b30 tx=0x7fa150005e30 comp rx=0 tx=0).stop 2024-04-24T23:32:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.058+0000 7fa162e39740 1 -- 172.21.15.33:0/2530369190 shutdown_connections 2024-04-24T23:32:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.058+0000 7fa162e39740 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e80230 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.058+0000 7fa162e39740 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e7d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.058+0000 7fa162e39740 1 --2- 172.21.15.33:0/2530369190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e78660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.058+0000 7fa162e39740 1 -- 172.21.15.33:0/2530369190 >> 172.21.15.33:0/2530369190 conn(0x564847e738b0 msgr2=0x564847e73cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.064+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1440a2c50 0x7fa1440a5110 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa154005dc0 tx=0x7fa154005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.064+0000 7fa162e39740 1 -- 172.21.15.33:0/2530369190 shutdown_connections 2024-04-24T23:32:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.069+0000 7fa162e39740 1 -- 172.21.15.33:0/2530369190 wait complete. 2024-04-24T23:32:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.069+0000 7fa162e39740 1 -- start start 2024-04-24T23:32:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.069+0000 7fa162e39740 1 Processor -- start 2024-04-24T23:32:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e82640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e82b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e89bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564847e80cf0 con 0x564847e78c30 2024-04-24T23:32:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564847d97f50 con 0x564847e78240 2024-04-24T23:32:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564847e80e70 con 0x564847e7dda0 2024-04-24T23:32:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.070+0000 7fa162e39740 10 client.0 _ll_register_callbacks cb 0x564847dd3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.071+0000 7fa15e6c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e89bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.074+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e82b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.074+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e82b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39018/0 (socket says 172.21.15.33:39018) 2024-04-24T23:32:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.074+0000 7fa15d6c4640 1 -- 172.21.15.33:0/1751314330 learned_addr learned my addr 172.21.15.33:0/1751314330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.074+0000 7fa15dec5640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e82640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.078+0000 7fa15e6c6640 1 -- 172.21.15.33:0/1751314330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 msgr2=0x564847e82640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.078+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e82640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.078+0000 7fa15e6c6640 1 -- 172.21.15.33:0/1751314330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 msgr2=0x564847e82b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.078+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e82b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.078+0000 7fa15e6c6640 1 -- 172.21.15.33:0/1751314330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564847af03f0 con 0x564847e7dda0 2024-04-24T23:32:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.081+0000 7fa15dec5640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.087+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e89bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x564847e79490 tx=0x7fa154004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95613e09f10ad57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.088+0000 7fa1417fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.092+0000 7fa1417fa640 1 -- 172.21.15.33:0/1751314330 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154021070 con 0x564847e7dda0 2024-04-24T23:32:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.092+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa150030ac0 con 0x564847e7dda0 2024-04-24T23:32:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.092+0000 7fa1417fa640 1 -- 172.21.15.33:0/1751314330 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa15401d5f0 con 0x564847e7dda0 2024-04-24T23:32:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.100+0000 7fa1417fa640 1 -- 172.21.15.33:0/1751314330 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa15402b050 con 0x564847e7dda0 2024-04-24T23:32:42.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.103+0000 7fa162e39740 10 client.29101 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:42.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.104+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564847ab00f0 con 0x564847e7dda0 2024-04-24T23:32:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.105+0000 7fa1417fa640 1 -- 172.21.15.33:0/1751314330 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa150030ac0 con 0x564847e7dda0 2024-04-24T23:32:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.106+0000 7fa1417fa640 1 -- 172.21.15.33:0/1751314330 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa15402e020 con 0x564847e7dda0 2024-04-24T23:32:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.110+0000 7fa162e39740 10 client.29101 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.110+0000 7fa162e39740 10 client.29101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.110+0000 7fa162e39740 20 client.29101 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.110+0000 7fa162e39740 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847e8e4f0 0x564847e90960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.110+0000 7fa162e39740 4 client.29101 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.110+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564847de9bf0 con 0x564847e8e4f0 2024-04-24T23:32:42.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.112+0000 7fa15dec5640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847e8e4f0 0x564847e90960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.118+0000 7fa15dec5640 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847e8e4f0 0x564847e90960 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d58e4f4d8e289 server_cookie=35c0ed874351ba1b in_seq=0 out_seq=0 2024-04-24T23:32:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.120+0000 7fa1417fa640 10 client.29101 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.122+0000 7fa1417fa640 1 -- 172.21.15.33:0/1751314330 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564847de9bf0 con 0x564847e8e4f0 2024-04-24T23:32:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.122+0000 7fa1417fa640 10 client.29101 handle_command_reply: tid=0 2024-04-24T23:32:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.123+0000 7fa162e39740 1 client.29101 shutdown 2024-04-24T23:32:42.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.134+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 msgr2=0x564847e89bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.134+0000 7fa162e39740 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e89bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x564847e79490 tx=0x7fa154004730 comp rx=0 tx=0).stop 2024-04-24T23:32:42.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 shutdown_connections 2024-04-24T23:32:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa162e39740 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847e8e4f0 0x564847e90960 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa162e39740 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e89bc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa162e39740 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78c30 0x564847e82b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa162e39740 1 --2- 172.21.15.33:0/1751314330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78240 0x564847e82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 >> 172.21.15.33:0/1751314330 conn(0x564847e738b0 msgr2=0x564847e72af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.136+0000 7fa1417fa640 0 client.29101 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.138+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 shutdown_connections 2024-04-24T23:32:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.138+0000 7fa162e39740 1 -- 172.21.15.33:0/1751314330 wait complete. 2024-04-24T23:32:42.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.142+0000 7fa162e39740 20 client.29101 trim_cache size 0 max 16384 2024-04-24T23:32:42.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:41 smithi038 ceph-mon[32450]: pgmap v971: 129 pgs: 129 active+clean; 603 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 354 op/s 2024-04-24T23:32:42.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:41 smithi117 ceph-mon[32529]: pgmap v971: 129 pgs: 129 active+clean; 603 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 354 op/s 2024-04-24T23:32:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.181+0000 7fa162e39740 1 Processor -- start 2024-04-24T23:32:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.182+0000 7fa162e39740 1 -- start start 2024-04-24T23:32:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.182+0000 7fa162e39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.182+0000 7fa162e39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.182+0000 7fa162e39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e72970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.182+0000 7fa162e39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1580805d0 con 0x564847e78240 2024-04-24T23:32:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.182+0000 7fa162e39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa15807dea0 con 0x564847e78c30 2024-04-24T23:32:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.183+0000 7fa162e39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa15807bcf0 con 0x564847e7dda0 2024-04-24T23:32:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.183+0000 7fa15e6c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e72970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.183+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.183+0000 7fa15e6c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e72970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39434/0 (socket says 172.21.15.33:39434) 2024-04-24T23:32:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.183+0000 7fa15e6c6640 1 -- 172.21.15.33:0/633918905 learned_addr learned my addr 172.21.15.33:0/633918905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.184+0000 7fa15e6c6640 1 -- 172.21.15.33:0/633918905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 msgr2=0x564847e89b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.184+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e89b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.184+0000 7fa15e6c6640 1 -- 172.21.15.33:0/633918905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 msgr2=0x564847e90500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.184+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.184+0000 7fa15e6c6640 1 -- 172.21.15.33:0/633918905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564847e7a710 con 0x564847e7dda0 2024-04-24T23:32:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.188+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e72970 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa154012040 tx=0x7fa15400d850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4234e0d4328906f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.191+0000 7fa1437fe640 1 -- 172.21.15.33:0/633918905 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154033070 con 0x564847e7dda0 2024-04-24T23:32:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.191+0000 7fa1437fe640 1 -- 172.21.15.33:0/633918905 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa154017070 con 0x564847e7dda0 2024-04-24T23:32:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.192+0000 7fa162e39740 1 -- 172.21.15.33:0/633918905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 msgr2=0x564847e72970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.192+0000 7fa162e39740 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e72970 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa154012040 tx=0x7fa15400d850 comp rx=0 tx=0).stop 2024-04-24T23:32:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.195+0000 7fa162e39740 1 -- 172.21.15.33:0/633918905 shutdown_connections 2024-04-24T23:32:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.195+0000 7fa162e39740 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e72970 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.195+0000 7fa162e39740 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e89b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.195+0000 7fa162e39740 1 --2- 172.21.15.33:0/633918905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.195+0000 7fa162e39740 1 -- 172.21.15.33:0/633918905 >> 172.21.15.33:0/633918905 conn(0x564847e738b0 msgr2=0x564847e72260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.196+0000 7fa162e39740 1 -- 172.21.15.33:0/633918905 shutdown_connections 2024-04-24T23:32:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.196+0000 7fa162e39740 1 -- 172.21.15.33:0/633918905 wait complete. 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 -- start start 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 Processor -- start 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e87530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e87a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa15807c360 con 0x564847e78240 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa15807d7d0 con 0x564847e78c30 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa15807ca30 con 0x564847e7dda0 2024-04-24T23:32:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.197+0000 7fa162e39740 10 client.0 _ll_register_callbacks cb 0x7fa15400b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e87a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e87a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32862/0 (socket says 172.21.15.33:32862) 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2510001246 learned_addr learned my addr 172.21.15.33:0/2510001246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15dec5640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e87530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15e6c6640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 msgr2=0x564847e889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e889c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2510001246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 msgr2=0x564847e87530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e87530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 -- 172.21.15.33:0/2510001246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564847e90cd0 con 0x564847e78c30 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15dec5640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e87530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.198+0000 7fa15d6c4640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e87a70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x564847e83b20 tx=0x7fa150003f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d166c1781e0392 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa1417fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa1417fa640 1 -- 172.21.15.33:0/2510001246 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150036040 con 0x564847e78c30 2024-04-24T23:32:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa1417fa640 1 -- 172.21.15.33:0/2510001246 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1500309f0 con 0x564847e78c30 2024-04-24T23:32:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa154017070 con 0x564847e78c30 2024-04-24T23:32:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa1417fa640 1 -- 172.21.15.33:0/2510001246 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa150077050 con 0x564847e78c30 2024-04-24T23:32:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa162e39740 10 client.29159 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.199+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564847ef4b40 con 0x564847e78c30 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa1417fa640 1 -- 172.21.15.33:0/2510001246 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa154017070 con 0x564847e78c30 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa1417fa640 1 -- 172.21.15.33:0/2510001246 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa15005d040 con 0x564847e78c30 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa162e39740 10 client.29159 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa162e39740 10 client.29159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa162e39740 20 client.29159 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa162e39740 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847ef5190 0x564847ef7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa162e39740 4 client.29159 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564847de3f40 con 0x564847ef5190 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.200+0000 7fa15dec5640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847ef5190 0x564847ef7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.201+0000 7fa15dec5640 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847ef5190 0x564847ef7580 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bef9171a2ad60e3e server_cookie=d552633e529d16df in_seq=0 out_seq=0 2024-04-24T23:32:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.201+0000 7fa1417fa640 10 client.29159 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.203+0000 7fa1417fa640 1 -- 172.21.15.33:0/2510001246 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564847de3f40 con 0x564847ef5190 2024-04-24T23:32:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.203+0000 7fa1417fa640 10 client.29159 handle_command_reply: tid=0 2024-04-24T23:32:42.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:41 smithi033 ceph-mon[29201]: pgmap v971: 129 pgs: 129 active+clean; 603 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 354 op/s 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.203+0000 7fa162e39740 1 client.29159 shutdown 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.203+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 msgr2=0x564847e87a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.203+0000 7fa162e39740 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e87a70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x564847e83b20 tx=0x7fa150003f40 comp rx=0 tx=0).stop 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 shutdown_connections 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564847ef5190 0x564847ef7580 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564847e7dda0 0x564847e889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564847e78c30 0x564847e87a70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 --2- 172.21.15.33:0/2510001246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564847e78240 0x564847e87530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 >> 172.21.15.33:0/2510001246 conn(0x564847e738b0 msgr2=0x564847e72af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa1417fa640 0 client.29159 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 shutdown_connections 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 1 -- 172.21.15.33:0/2510001246 wait complete. 2024-04-24T23:32:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.204+0000 7fa162e39740 20 client.29159 trim_cache size 0 max 16384 2024-04-24T23:32:42.206 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:32:42.206 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:32:42.206 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:32:42.206 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1440a2c50 msgr2=0x7fa1440a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1440a2c50 0x7fa1440a5110 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa154005dc0 tx=0x7fa154005d50 comp rx=0 tx=0).stop 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa158108740 msgr2=0x7fa158115020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa158108740 0x7fa158115020 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e9f0 tx=0x7fa14800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 shutdown_connections 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa1440a2c50 0x7fa1440a5110 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa15810da80 0x7fa1581b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15810b0b0 0x7fa158115560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 --2- 172.21.15.33:0/3999793630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa158108740 0x7fa158115020 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 >> 172.21.15.33:0/3999793630 conn(0x7fa15806e5d0 msgr2=0x7fa158109ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 shutdown_connections 2024-04-24T23:32:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.205+0000 7fa160129640 1 -- 172.21.15.33:0/3999793630 wait complete. 2024-04-24T23:32:42.220 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:32:42.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:32:42.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 -- 172.21.15.33:0/2130253386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 msgr2=0x7fadc0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 --2- 172.21.15.33:0/2130253386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc0101d30 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009900 tx=0x7fadb402a410 comp rx=0 tx=0).stop 2024-04-24T23:32:42.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 -- 172.21.15.33:0/2130253386 shutdown_connections 2024-04-24T23:32:42.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 --2- 172.21.15.33:0/2130253386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc0102270 0x7fadc0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 --2- 172.21.15.33:0/2130253386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc0101d30 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 --2- 172.21.15.33:0/2130253386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc01006c0 0x7fadc0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 -- 172.21.15.33:0/2130253386 >> 172.21.15.33:0/2130253386 conn(0x7fadc00fbc30 msgr2=0x7fadc00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 -- 172.21.15.33:0/2130253386 shutdown_connections 2024-04-24T23:32:42.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.370+0000 7fadc564d640 1 -- 172.21.15.33:0/2130253386 wait complete. 2024-04-24T23:32:42.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 Processor -- start 2024-04-24T23:32:42.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 -- start start 2024-04-24T23:32:42.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc01006c0 0x7fadc0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0102270 0x7fadc019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010c390 con 0x7fadc01018b0 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010c210 con 0x7fadc01006c0 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadc564d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010c510 con 0x7fadc0102270 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.371+0000 7fadbe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39058/0 (socket says 172.21.15.33:39058) 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 -- 172.21.15.33:0/1662006188 learned_addr learned my addr 172.21.15.33:0/1662006188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbeffd640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc01006c0 0x7fadc0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbf7fe640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0102270 0x7fadc019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 -- 172.21.15.33:0/1662006188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0102270 msgr2=0x7fadc019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0102270 0x7fadc019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 -- 172.21.15.33:0/1662006188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc01006c0 msgr2=0x7fadc0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc01006c0 0x7fadc0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbe7fc640 1 -- 172.21.15.33:0/1662006188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc019f750 con 0x7fadc01018b0 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.372+0000 7fadbeffd640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc01006c0 0x7fadc0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.373+0000 7fadbe7fc640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc01980c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fadb40314e0 tx=0x7fadb4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.373+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb403f030 con 0x7fadc01018b0 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.373+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4003ce0 con 0x7fadc01018b0 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.373+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc0102bc0 con 0x7fadc01018b0 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.373+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4035070 con 0x7fadc01018b0 2024-04-24T23:32:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.373+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc0103100 con 0x7fadc01018b0 2024-04-24T23:32:42.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.374+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc0103360 con 0x7fadc01018b0 2024-04-24T23:32:42.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.375+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fadb4042030 con 0x7fadc01018b0 2024-04-24T23:32:42.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.376+0000 7fada7fff640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fada80a2c10 0x7fada80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.376+0000 7fadbeffd640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fada80a2c10 0x7fada80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.376+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fadb40ef6c0 con 0x7fadc01018b0 2024-04-24T23:32:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.376+0000 7fadbeffd640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fada80a2c10 0x7fada80a50d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fadc0101710 tx=0x7fadb0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.382+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fadb40bbc40 con 0x7fadc01018b0 2024-04-24T23:32:42.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.713+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadc0103b90 con 0x7fadc01018b0 2024-04-24T23:32:42.715 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:32:42.715 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:32:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:32:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.714+0000 7fada7fff640 1 -- 172.21.15.33:0/1662006188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7fadb40bff40 con 0x7fadc01018b0 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fada80a2c10 msgr2=0x7fada80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fada80a2c10 0x7fada80a50d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fadc0101710 tx=0x7fadb0005f90 comp rx=0 tx=0).stop 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 msgr2=0x7fadc01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc01980c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fadb40314e0 tx=0x7fadb4032ce0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 shutdown_connections 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fada80a2c10 0x7fada80a50d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0102270 0x7fadc019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc01018b0 0x7fadc01980c0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 --2- 172.21.15.33:0/1662006188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadc01006c0 0x7fadc0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.718+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 >> 172.21.15.33:0/1662006188 conn(0x7fadc00fbc30 msgr2=0x7fadc0069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.719+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 shutdown_connections 2024-04-24T23:32:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.719+0000 7fadc564d640 1 -- 172.21.15.33:0/1662006188 wait complete. 2024-04-24T23:32:42.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:32:42.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 174056 22.40 MB/sec execute 287 sec latency 251.297 ms 2024-04-24T23:32:42.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 -- 172.21.15.33:0/3517956785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10b0b0 msgr2=0x7fa92c10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 --2- 172.21.15.33:0/3517956785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10b0b0 0x7fa92c10d540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa924009ca0 tx=0x7fa92402a8d0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 -- 172.21.15.33:0/3517956785 shutdown_connections 2024-04-24T23:32:42.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 --2- 172.21.15.33:0/3517956785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10da80 0x7fa92c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 --2- 172.21.15.33:0/3517956785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10b0b0 0x7fa92c10d540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 --2- 172.21.15.33:0/3517956785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 0x7fa92c10ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 -- 172.21.15.33:0/3517956785 >> 172.21.15.33:0/3517956785 conn(0x7fa92c06e5d0 msgr2=0x7fa92c070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.931+0000 7fa933272640 1 -- 172.21.15.33:0/3517956785 shutdown_connections 2024-04-24T23:32:42.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.932+0000 7fa933272640 1 -- 172.21.15.33:0/3517956785 wait complete. 2024-04-24T23:32:42.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.932+0000 7fa933272640 1 Processor -- start 2024-04-24T23:32:42.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.932+0000 7fa933272640 1 -- start start 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa933272640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 0x7fa92c114e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa933272640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 0x7fa92c1153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa933272640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10da80 0x7fa92c1a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa933272640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c11c350 con 0x7fa92c108740 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa933272640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c11c1d0 con 0x7fa92c10b0b0 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa933272640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c11c4d0 con 0x7fa92c10da80 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa93080d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 0x7fa92c1153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa93080d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 0x7fa92c1153b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32892/0 (socket says 172.21.15.33:32892) 2024-04-24T23:32:42.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa93080d640 1 -- 172.21.15.33:0/3573516742 learned_addr learned my addr 172.21.15.33:0/3573516742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.933+0000 7fa93180f640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10da80 0x7fa92c1a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93100e640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 0x7fa92c114e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93080d640 1 -- 172.21.15.33:0/3573516742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10da80 msgr2=0x7fa92c1a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93080d640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10da80 0x7fa92c1a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93080d640 1 -- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 msgr2=0x7fa92c114e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93080d640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 0x7fa92c114e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93080d640 1 -- 172.21.15.33:0/3573516742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c1a5f90 con 0x7fa92c10b0b0 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93100e640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 0x7fa92c114e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.934+0000 7fa93080d640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 0x7fa92c1153b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa92402aec0 tx=0x7fa924003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.938+0000 7fa9227fc640 1 -- 172.21.15.33:0/3573516742 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924004040 con 0x7fa92c10b0b0 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.938+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c1a6220 con 0x7fa92c10b0b0 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.938+0000 7fa9227fc640 1 -- 172.21.15.33:0/3573516742 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9240046c0 con 0x7fa92c10b0b0 2024-04-24T23:32:42.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.939+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c1a6840 con 0x7fa92c10b0b0 2024-04-24T23:32:42.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.940+0000 7fa9227fc640 1 -- 172.21.15.33:0/3573516742 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240305d0 con 0x7fa92c10b0b0 2024-04-24T23:32:42.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.942+0000 7fa935f7f740 1 Processor -- start 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 -- start start 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa9227fc640 1 -- 172.21.15.33:0/3573516742 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa92402f620 con 0x7fa92c10b0b0 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6869650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d686e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6871220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5568d67cd670 con 0x5568d6869230 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5568d6775970 con 0x5568d6869c20 2024-04-24T23:32:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.943+0000 7fa935f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568d67c9630 con 0x5568d686ed90 2024-04-24T23:32:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.944+0000 7fa93080d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6871220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.944+0000 7fa93080d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6871220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39470/0 (socket says 172.21.15.33:39470) 2024-04-24T23:32:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.944+0000 7fa93080d640 1 -- 172.21.15.33:0/1908760786 learned_addr learned my addr 172.21.15.33:0/1908760786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.944+0000 7fa93100e640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d686e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.944+0000 7fa93180f640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6869650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa9227fc640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa9180a2b30 0x7fa9180a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa93080d640 1 -- 172.21.15.33:0/1908760786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 msgr2=0x5568d686e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa93080d640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d686e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa93080d640 1 -- 172.21.15.33:0/1908760786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 msgr2=0x5568d6869650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa93080d640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6869650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa93080d640 1 -- 172.21.15.33:0/1908760786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d6799400 con 0x5568d686ed90 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.957+0000 7fa9227fc640 1 -- 172.21.15.33:0/3573516742 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa9240f0a70 con 0x7fa92c10b0b0 2024-04-24T23:32:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.958+0000 7fa93080d640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6871220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa9240c6e10 tx=0x7fa9240b8c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15a55a89df55307 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.958+0000 7fa9177fe640 1 -- 172.21.15.33:0/1908760786 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240bccb0 con 0x5568d686ed90 2024-04-24T23:32:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.958+0000 7fa9177fe640 1 -- 172.21.15.33:0/1908760786 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924036070 con 0x5568d686ed90 2024-04-24T23:32:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.958+0000 7fa9177fe640 1 -- 172.21.15.33:0/1908760786 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240c5b50 con 0x5568d686ed90 2024-04-24T23:32:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.959+0000 7fa93100e640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d686e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.959+0000 7fa93180f640 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6869650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.959+0000 7fa93180f640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa9180a2b30 0x7fa9180a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.960+0000 7fa935f7f740 1 -- 172.21.15.33:0/1908760786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 msgr2=0x5568d6871220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.960+0000 7fa935f7f740 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6871220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa9240c6e10 tx=0x7fa9240b8c30 comp rx=0 tx=0).stop 2024-04-24T23:32:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.961+0000 7fa935f7f740 1 -- 172.21.15.33:0/1908760786 shutdown_connections 2024-04-24T23:32:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.961+0000 7fa935f7f740 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6871220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.961+0000 7fa935f7f740 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d686e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.961+0000 7fa935f7f740 1 --2- 172.21.15.33:0/1908760786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6869650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.961+0000 7fa935f7f740 1 -- 172.21.15.33:0/1908760786 >> 172.21.15.33:0/1908760786 conn(0x5568d68648a0 msgr2=0x5568d6864ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.961+0000 7fa93180f640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa9180a2b30 0x7fa9180a4ff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5568d686a480 tx=0x7fa91c0047d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.964+0000 7fa935f7f740 1 -- 172.21.15.33:0/1908760786 shutdown_connections 2024-04-24T23:32:42.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.966+0000 7fa935f7f740 1 -- 172.21.15.33:0/1908760786 wait complete. 2024-04-24T23:32:42.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.966+0000 7fa935f7f740 1 -- start start 2024-04-24T23:32:42.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 Processor -- start 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 0x5568d6873010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869c20 0x5568d6873550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d687a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5568d6871b00 con 0x5568d686ed90 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5568d6789050 con 0x5568d6869230 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.967+0000 7fa935f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568d6871c80 con 0x5568d6869c20 2024-04-24T23:32:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.968+0000 7fa935f7f740 10 client.0 _ll_register_callbacks cb 0x5568d64e0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.968+0000 7fa93100e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 0x5568d6873010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.968+0000 7fa93100e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 0x5568d6873010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32904/0 (socket says 172.21.15.33:32904) 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.968+0000 7fa93100e640 1 -- 172.21.15.33:0/2550274753 learned_addr learned my addr 172.21.15.33:0/2550274753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.969+0000 7fa93180f640 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d687a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.969+0000 7fa93100e640 1 -- 172.21.15.33:0/2550274753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869c20 msgr2=0x5568d6873550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.969+0000 7fa93100e640 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869c20 0x5568d6873550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.969+0000 7fa93100e640 1 -- 172.21.15.33:0/2550274753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 msgr2=0x5568d687a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:42.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.969+0000 7fa93100e640 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d687a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.969+0000 7fa93100e640 1 -- 172.21.15.33:0/2550274753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d675adc0 con 0x5568d6869230 2024-04-24T23:32:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.973+0000 7fa93100e640 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 0x5568d6873010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa92800e2f0 tx=0x7fa92800e7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b52d631c98dde22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.973+0000 7fa914ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.973+0000 7fa914ff9640 1 -- 172.21.15.33:0/2550274753 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928018030 con 0x5568d6869230 2024-04-24T23:32:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.974+0000 7fa914ff9640 1 -- 172.21.15.33:0/2550274753 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa92800f040 con 0x5568d6869230 2024-04-24T23:32:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.974+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa924036070 con 0x5568d6869230 2024-04-24T23:32:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.974+0000 7fa914ff9640 1 -- 172.21.15.33:0/2550274753 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa928022050 con 0x5568d6869230 2024-04-24T23:32:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.976+0000 7fa935f7f740 10 client.29174 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:42.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.976+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5568d64e13f0 con 0x5568d6869230 2024-04-24T23:32:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.980+0000 7fa914ff9640 1 -- 172.21.15.33:0/2550274753 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924036070 con 0x5568d6869230 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa914ff9640 1 -- 172.21.15.33:0/2550274753 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa92801d8e0 con 0x5568d6869230 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa935f7f740 10 client.29174 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa935f7f740 10 client.29174 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa935f7f740 20 client.29174 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d687eee0 0x5568d6881380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa935f7f740 4 client.29174 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.985+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568d67dabb0 con 0x5568d687eee0 2024-04-24T23:32:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.986+0000 7fa93080d640 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d687eee0 0x5568d6881380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.992+0000 7fa93080d640 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d687eee0 0x5568d6881380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7842e690ff549c84 server_cookie=776f7da19385b7f0 in_seq=0 out_seq=0 2024-04-24T23:32:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.993+0000 7fa914ff9640 10 client.29174 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.998+0000 7fa914ff9640 1 -- 172.21.15.33:0/2550274753 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568d67dabb0 con 0x5568d687eee0 2024-04-24T23:32:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:42.999+0000 7fa914ff9640 10 client.29174 handle_command_reply: tid=0 2024-04-24T23:32:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.000+0000 7fa935f7f740 1 client.29174 shutdown 2024-04-24T23:32:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 msgr2=0x5568d6873010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 0x5568d6873010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa92800e2f0 tx=0x7fa92800e7b0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 shutdown_connections 2024-04-24T23:32:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d687eee0 0x5568d6881380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d687a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869c20 0x5568d6873550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2550274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869230 0x5568d6873010 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 >> 172.21.15.33:0/2550274753 conn(0x5568d68648a0 msgr2=0x5568d6863b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.005+0000 7fa914ff9640 0 client.29174 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:43.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.007+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 shutdown_connections 2024-04-24T23:32:43.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.008+0000 7fa935f7f740 1 -- 172.21.15.33:0/2550274753 wait complete. 2024-04-24T23:32:43.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.008+0000 7fa935f7f740 20 client.29174 trim_cache size 0 max 16384 2024-04-24T23:32:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.034+0000 7fa935f7f740 1 Processor -- start 2024-04-24T23:32:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.034+0000 7fa935f7f740 1 -- start start 2024-04-24T23:32:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.034+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6872630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.034+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d687a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.034+0000 7fa93100e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6872630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6872630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39100/0 (socket says 172.21.15.33:39100) 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.034+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6872b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 -- 172.21.15.33:0/2490238822 learned_addr learned my addr 172.21.15.33:0/2490238822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1d5310 con 0x5568d6869230 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1b0bd0 con 0x5568d6869c20 2024-04-24T23:32:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1aea20 con 0x5568d686ed90 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 -- 172.21.15.33:0/2490238822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 msgr2=0x5568d6872b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6872b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 -- 172.21.15.33:0/2490238822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 msgr2=0x5568d687a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d687a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 -- 172.21.15.33:0/2490238822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d65ab7d0 con 0x5568d6869230 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.035+0000 7fa93100e640 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6872630 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa92801dcd0 tx=0x7fa92801d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7158d5453b6897 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.036+0000 7fa916ffd640 1 -- 172.21.15.33:0/2490238822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92802a020 con 0x5568d6869230 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.036+0000 7fa916ffd640 1 -- 172.21.15.33:0/2490238822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa928013660 con 0x5568d6869230 2024-04-24T23:32:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.036+0000 7fa916ffd640 1 -- 172.21.15.33:0/2490238822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928018040 con 0x5568d6869230 2024-04-24T23:32:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.037+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 msgr2=0x5568d6872630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.037+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6872630 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa92801dcd0 tx=0x7fa92801d400 comp rx=0 tx=0).stop 2024-04-24T23:32:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 shutdown_connections 2024-04-24T23:32:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d686ed90 0x5568d6872b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d687a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2490238822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d6869230 0x5568d6872630 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 >> 172.21.15.33:0/2490238822 conn(0x5568d68648a0 msgr2=0x5568d6868100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 shutdown_connections 2024-04-24T23:32:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 -- 172.21.15.33:0/2490238822 wait complete. 2024-04-24T23:32:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.039+0000 7fa935f7f740 1 -- start start 2024-04-24T23:32:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 Processor -- start 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869230 0x5568d68778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d6877e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d6878e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1af090 con 0x5568d686ed90 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1b0500 con 0x5568d6869c20 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c1af760 con 0x5568d6869230 2024-04-24T23:32:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa935f7f740 10 client.0 _ll_register_callbacks cb 0x7fa92800c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.040+0000 7fa93080d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d6877e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d6877e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32914/0 (socket says 172.21.15.33:32914) 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 -- 172.21.15.33:0/2519157309 learned_addr learned my addr 172.21.15.33:0/2519157309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93180f640 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d6878e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 -- 172.21.15.33:0/2519157309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869230 msgr2=0x5568d68778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869230 0x5568d68778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 -- 172.21.15.33:0/2519157309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 msgr2=0x5568d6878e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d6878e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 -- 172.21.15.33:0/2519157309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d6879430 con 0x5568d6869c20 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa93080d640 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d6877e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b98b0 tx=0x7fa924003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d1199d648d04f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.041+0000 7fa914ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.042+0000 7fa914ff9640 1 -- 172.21.15.33:0/2519157309 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240bb4a0 con 0x5568d6869c20 2024-04-24T23:32:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.042+0000 7fa914ff9640 1 -- 172.21.15.33:0/2519157309 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa9240bb1e0 con 0x5568d6869c20 2024-04-24T23:32:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.042+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa928013660 con 0x5568d6869c20 2024-04-24T23:32:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.046+0000 7fa914ff9640 1 -- 172.21.15.33:0/2519157309 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9240bb690 con 0x5568d6869c20 2024-04-24T23:32:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.046+0000 7fa935f7f740 10 client.29177 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.046+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5568d687a7a0 con 0x5568d6869c20 2024-04-24T23:32:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.046+0000 7fa914ff9640 1 -- 172.21.15.33:0/2519157309 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa928013660 con 0x5568d6869c20 2024-04-24T23:32:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa914ff9640 1 -- 172.21.15.33:0/2519157309 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa9240b9550 con 0x5568d6869c20 2024-04-24T23:32:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa935f7f740 10 client.29177 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa935f7f740 10 client.29177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:32:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa935f7f740 20 client.29177 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d68e59f0 0x5568d68e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa935f7f740 4 client.29177 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:32:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568d67d4f00 con 0x5568d68e59f0 2024-04-24T23:32:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.047+0000 7fa93100e640 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d68e59f0 0x5568d68e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.049+0000 7fa93100e640 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d68e59f0 0x5568d68e7e10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45aeb2c889fd9588 server_cookie=1066b0ecca38bdfc in_seq=0 out_seq=0 2024-04-24T23:32:43.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.049+0000 7fa914ff9640 10 client.29177 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.051+0000 7fa914ff9640 1 -- 172.21.15.33:0/2519157309 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568d67d4f00 con 0x5568d68e59f0 2024-04-24T23:32:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.051+0000 7fa914ff9640 10 client.29177 handle_command_reply: tid=0 2024-04-24T23:32:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 client.29177 shutdown 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 msgr2=0x5568d6877e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d6877e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b98b0 tx=0x7fa924003f10 comp rx=0 tx=0).stop 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 shutdown_connections 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5568d68e59f0 0x5568d68e7e10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568d686ed90 0x5568d6878e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5568d6869c20 0x5568d6877e30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 --2- 172.21.15.33:0/2519157309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d6869230 0x5568d68778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 >> 172.21.15.33:0/2519157309 conn(0x5568d68648a0 msgr2=0x5568d686a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.052+0000 7fa914ff9640 0 client.29177 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:32:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.053+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 shutdown_connections 2024-04-24T23:32:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.053+0000 7fa935f7f740 1 -- 172.21.15.33:0/2519157309 wait complete. 2024-04-24T23:32:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.053+0000 7fa935f7f740 20 client.29177 trim_cache size 0 max 16384 2024-04-24T23:32:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:32:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa9180a2b30 msgr2=0x7fa9180a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa9180a2b30 0x7fa9180a4ff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5568d686a480 tx=0x7fa91c0047d0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 msgr2=0x7fa92c1153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 0x7fa92c1153b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa92402aec0 tx=0x7fa924003ba0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 shutdown_connections 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa9180a2b30 0x7fa9180a4ff0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa92c10da80 0x7fa92c1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92c10b0b0 0x7fa92c1153b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 --2- 172.21.15.33:0/3573516742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c108740 0x7fa92c114e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.057+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 >> 172.21.15.33:0/3573516742 conn(0x7fa92c06e5d0 msgr2=0x7fa92c108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.058+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 shutdown_connections 2024-04-24T23:32:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.058+0000 7fa933272640 1 -- 172.21.15.33:0/3573516742 wait complete. 2024-04-24T23:32:43.072 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:42 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1662006188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:43.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:32:43.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:42 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1662006188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:43.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:42 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1662006188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.274+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1448684046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 msgr2=0x7fbf9c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.275+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1448684046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c104dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009860 tx=0x7fbf9002a380 comp rx=0 tx=0).stop 2024-04-24T23:32:43.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1448684046 shutdown_connections 2024-04-24T23:32:43.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1448684046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1448684046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c104dd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1448684046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 0x7fbf9c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1448684046 >> 172.21.15.33:0/1448684046 conn(0x7fbf9c0fbc30 msgr2=0x7fbf9c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1448684046 shutdown_connections 2024-04-24T23:32:43.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.276+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1448684046 wait complete. 2024-04-24T23:32:43.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.277+0000 7fbfa22d4640 1 Processor -- start 2024-04-24T23:32:43.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.277+0000 7fbfa22d4640 1 -- start start 2024-04-24T23:32:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.278+0000 7fbfa22d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 0x7fbf9c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.278+0000 7fbfa22d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.278+0000 7fbfa22d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c1a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.278+0000 7fbfa22d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c113a90 con 0x7fbf9c0ffff0 2024-04-24T23:32:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbfa22d4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c113910 con 0x7fbf9c102960 2024-04-24T23:32:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbfa22d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c113c10 con 0x7fbf9c105310 2024-04-24T23:32:43.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32940/0 (socket says 172.21.15.33:32940) 2024-04-24T23:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbfa0871640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c1a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbfa0871640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c1a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39498/0 (socket says 172.21.15.33:39498) 2024-04-24T23:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.279+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1772583895 learned_addr learned my addr 172.21.15.33:0/1772583895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.281+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 0x7fbf9c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.281+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1772583895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 msgr2=0x7fbf9c1a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.281+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c1a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.281+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 msgr2=0x7fbf9c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.281+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 0x7fbf9c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.281+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1772583895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c1a5d50 con 0x7fbf9c102960 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.282+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 0x7fbf9c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:43.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.282+0000 7fbfa0871640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c1a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-24T23:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.282+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c19e670 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009780 tx=0x7fbf90032d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.283+0000 7fbf997fa640 1 -- 172.21.15.33:0/1772583895 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90043070 con 0x7fbf9c102960 2024-04-24T23:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.283+0000 7fbf997fa640 1 -- 172.21.15.33:0/1772583895 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf90038040 con 0x7fbf9c102960 2024-04-24T23:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.284+0000 7fbf997fa640 1 -- 172.21.15.33:0/1772583895 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90037050 con 0x7fbf9c102960 2024-04-24T23:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.284+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c1a5fe0 con 0x7fbf9c102960 2024-04-24T23:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.284+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c1a64c0 con 0x7fbf9c102960 2024-04-24T23:32:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.287+0000 7fbf997fa640 1 -- 172.21.15.33:0/1772583895 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbf90046070 con 0x7fbf9c102960 2024-04-24T23:32:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.288+0000 7fbf997fa640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf840a2d10 0x7fbf840a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.288+0000 7fbf997fa640 1 -- 172.21.15.33:0/1772583895 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbf900ef5a0 con 0x7fbf9c102960 2024-04-24T23:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.289+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf840a2d10 0x7fbf840a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.289+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf840a2d10 0x7fbf840a51d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009d60 tx=0x7fbf8c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.291+0000 7fbfa4fe3740 1 Processor -- start 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.291+0000 7fbfa4fe3740 1 -- start start 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.291+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbe230 0x5641efcbe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.291+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcc3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcc3d90 0x5641efcc6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5641efc22670 con 0x5641efcc3d90 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5641efbca970 con 0x5641efcbec20 2024-04-24T23:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641efc1e630 con 0x5641efcbe230 2024-04-24T23:32:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcc3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcc3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:32950/0 (socket says 172.21.15.33:32950) 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/2252199822 learned_addr learned my addr 172.21.15.33:0/2252199822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/2252199822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbe230 msgr2=0x5641efcbe650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbe230 0x5641efcbe650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/2252199822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcc3d90 msgr2=0x5641efcc6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcc3d90 0x5641efcc6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.292+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/2252199822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641efbee400 con 0x5641efcbec20 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.293+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcc3720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf900c41b0 tx=0x7fbf900053d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7eb044a4d3aa79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.293+0000 7fbf827fc640 1 -- 172.21.15.33:0/2252199822 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90032920 con 0x5641efcbec20 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.293+0000 7fbf827fc640 1 -- 172.21.15.33:0/2252199822 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf900bc170 con 0x5641efcbec20 2024-04-24T23:32:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.293+0000 7fbf827fc640 1 -- 172.21.15.33:0/2252199822 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf900b8050 con 0x5641efcbec20 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2252199822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 msgr2=0x5641efcc3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcc3720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf900c41b0 tx=0x7fbf900053d0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2252199822 shutdown_connections 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcc3d90 0x5641efcc6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcc3720 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2252199822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbe230 0x5641efcbe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.294+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2252199822 >> 172.21.15.33:0/2252199822 conn(0x5641efcb98a0 msgr2=0x5641efcb9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.295+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2252199822 shutdown_connections 2024-04-24T23:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.295+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2252199822 wait complete. 2024-04-24T23:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.295+0000 7fbfa4fe3740 1 -- start start 2024-04-24T23:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.295+0000 7fbfa4fe3740 1 Processor -- start 2024-04-24T23:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcd9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcd9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcc3d90 0x5641efce06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5641efcd7b10 con 0x5641efcbe230 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5641efbde050 con 0x5641efcbec20 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641efcd7c90 con 0x5641efcc3d90 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.296+0000 7fbfa4fe3740 10 client.0 _ll_register_callbacks cb 0x5641ef935be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcd9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcd9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcd9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39152/0 (socket says 172.21.15.33:39152) 2024-04-24T23:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9bfff640 1 -- 172.21.15.33:0/1751974393 learned_addr learned my addr 172.21.15.33:0/1751974393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbfa0871640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcc3d90 0x5641efce06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcc3d90 msgr2=0x5641efce06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcc3d90 0x5641efce06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1751974393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 msgr2=0x5641efcd9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcd9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/1751974393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641efbafdc0 con 0x5641efcbec20 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.297+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcd9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcd9660 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5641efcbf690 tx=0x7fbf90005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f70f378033e833e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbf7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/1751974393 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf900d8070 con 0x5641efcbec20 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf900bc170 con 0x5641efcbec20 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/1751974393 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbf900d4070 con 0x5641efcbec20 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/1751974393 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf90032920 con 0x5641efcbec20 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbfa4fe3740 10 client.29189 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.298+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641ef9363f0 con 0x5641efcbec20 2024-04-24T23:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/1751974393 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf900bc170 con 0x5641efcbec20 2024-04-24T23:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/1751974393 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbf900c4250 con 0x5641efcbec20 2024-04-24T23:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbfa4fe3740 10 client.29189 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbfa4fe3740 10 client.29189 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbfa4fe3740 20 client.29189 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efce4dd0 0x5641efce71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbfa4fe3740 4 client.29189 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.300+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641efc2fbb0 con 0x5641efce4dd0 2024-04-24T23:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.301+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efce4dd0 0x5641efce71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.306+0000 7fbf7b7fe640 10 client.29189 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:43.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.306+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efce4dd0 0x5641efce71c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2571d0ce90c0b0e6 server_cookie=ade39466b8bc18af in_seq=0 out_seq=0 2024-04-24T23:32:43.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.308+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/1751974393 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641efc2fbb0 con 0x5641efce4dd0 2024-04-24T23:32:43.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.308+0000 7fbf7b7fe640 10 client.29189 handle_command_reply: tid=0 2024-04-24T23:32:43.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.309+0000 7fbfa4fe3740 1 client.29189 shutdown 2024-04-24T23:32:43.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.316+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 msgr2=0x5641efcd9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.316+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcd9660 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5641efcbf690 tx=0x7fbf90005260 comp rx=0 tx=0).stop 2024-04-24T23:32:43.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 shutdown_connections 2024-04-24T23:32:43.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efce4dd0 0x5641efce71c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcc3d90 0x5641efce06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcbec20 0x5641efcd9660 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/1751974393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcd9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 >> 172.21.15.33:0/1751974393 conn(0x5641efcb98a0 msgr2=0x5641efcb8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.318+0000 7fbf7b7fe640 0 client.29189 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:43.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.319+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 shutdown_connections 2024-04-24T23:32:43.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.319+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/1751974393 wait complete. 2024-04-24T23:32:43.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.320+0000 7fbfa4fe3740 20 client.29189 trim_cache size 0 max 16384 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbfa4fe3740 1 Processor -- start 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbfa4fe3740 1 -- start start 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcbc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efce0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcbcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcbc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcbc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39156/0 (socket says 172.21.15.33:39156) 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbf9bfff640 1 -- 172.21.15.33:0/2813069561 learned_addr learned my addr 172.21.15.33:0/2813069561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.340+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efce0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1aaab0 con 0x5641efcbe230 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a8380 con 0x5641efcc3d90 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10e8b0 con 0x5641efcbec20 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbfa0871640 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcbcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9bfff640 1 -- 172.21.15.33:0/2813069561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 msgr2=0x5641efce0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efce0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9bfff640 1 -- 172.21.15.33:0/2813069561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 msgr2=0x5641efcbcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcbcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9bfff640 1 -- 172.21.15.33:0/2813069561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641efa007d0 con 0x5641efcbe230 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efce0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcbc890 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009fd0 tx=0x7fbf8c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbbc1f2a288bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf81ffb640 1 -- 172.21.15.33:0/2813069561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c02c030 con 0x5641efcbe230 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.341+0000 7fbf81ffb640 1 -- 172.21.15.33:0/2813069561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c015dd0 con 0x5641efcbe230 2024-04-24T23:32:43.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbf81ffb640 1 -- 172.21.15.33:0/2813069561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c01b040 con 0x5641efcbe230 2024-04-24T23:32:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 msgr2=0x5641efcbc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcbc890 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009fd0 tx=0x7fbf8c011000 comp rx=0 tx=0).stop 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 shutdown_connections 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcbcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efce0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/2813069561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efcbc890 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 >> 172.21.15.33:0/2813069561 conn(0x5641efcb98a0 msgr2=0x5641efcd9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 shutdown_connections 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.342+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/2813069561 wait complete. 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 -- start start 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 Processor -- start 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efce0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efcdce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcdd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a6860 con 0x5641efcbe230 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a7cb0 con 0x5641efcc3d90 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a6f10 con 0x5641efcbec20 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa4fe3740 10 client.0 _ll_register_callbacks cb 0x7fbf900b7d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efce0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efcdce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbfa0871640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcdd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efcdce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39536/0 (socket says 172.21.15.33:39536) 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efce0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39166/0 (socket says 172.21.15.33:39166) 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.343+0000 7fbf9b7fe640 1 -- 172.21.15.33:0/4094737046 learned_addr learned my addr 172.21.15.33:0/4094737046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa0871640 1 -- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 msgr2=0x5641efcdce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa0871640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efcdce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa0871640 1 -- 172.21.15.33:0/4094737046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 msgr2=0x5641efce0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa0871640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efce0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa0871640 1 -- 172.21.15.33:0/4094737046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641efcdda10 con 0x5641efcc3d90 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf9b7fe640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efcdce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efce0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa0871640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcdd3d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5641efcb9870 tx=0x7fbf9400eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a48734d2bf466b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/4094737046 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400dbc0 con 0x5641efcc3d90 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/4094737046 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbf9400dd60 con 0x5641efcc3d90 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c015dd0 con 0x5641efcc3d90 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/4094737046 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf94026050 con 0x5641efcc3d90 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa4fe3740 10 client.29195 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641efd4a450 con 0x5641efcc3d90 2024-04-24T23:32:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.344+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/4094737046 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c015dd0 con 0x5641efcc3d90 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/4094737046 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbf9400b040 con 0x5641efcc3d90 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbfa4fe3740 10 client.29195 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbfa4fe3740 10 client.29195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbfa4fe3740 20 client.29195 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efd4ae80 0x5641efd4d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbfa4fe3740 4 client.29195 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641efc29f00 con 0x5641efd4ae80 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efd4ae80 0x5641efd4d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbf9bfff640 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efd4ae80 0x5641efd4d2a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82f1408c4dc69e53 server_cookie=592611c340198f0e in_seq=0 out_seq=0 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.345+0000 7fbf7b7fe640 10 client.29195 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbf7b7fe640 1 -- 172.21.15.33:0/4094737046 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641efc29f00 con 0x5641efd4ae80 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbf7b7fe640 10 client.29195 handle_command_reply: tid=0 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 client.29195 shutdown 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 msgr2=0x5641efcdd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcdd3d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5641efcb9870 tx=0x7fbf9400eed0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 shutdown_connections 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5641efd4ae80 0x5641efd4d2a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641efcc3d90 0x5641efcdd3d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641efcbec20 0x5641efcdce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 --2- 172.21.15.33:0/4094737046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641efcbe230 0x5641efce0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbf7b7fe640 0 client.29195 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 >> 172.21.15.33:0/4094737046 conn(0x5641efcb98a0 msgr2=0x5641efcbc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 shutdown_connections 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 1 -- 172.21.15.33:0/4094737046 wait complete. 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa4fe3740 20 client.29195 trim_cache size 0 max 16384 2024-04-24T23:32:43.354 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf840a2d10 msgr2=0x7fbf840a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.346+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf840a2d10 0x7fbf840a51d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009d60 tx=0x7fbf8c009110 comp rx=0 tx=0).stop 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 msgr2=0x7fbf9c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c19e670 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009780 tx=0x7fbf90032d00 comp rx=0 tx=0).stop 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 shutdown_connections 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbf840a2d10 0x7fbf840a51d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf9c105310 0x7fbf9c1a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c102960 0x7fbf9c19e670 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 --2- 172.21.15.33:0/1772583895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0ffff0 0x7fbf9c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 >> 172.21.15.33:0/1772583895 conn(0x7fbf9c0fbc30 msgr2=0x7fbf9c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 shutdown_connections 2024-04-24T23:32:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.347+0000 7fbfa22d4640 1 -- 172.21.15.33:0/1772583895 wait complete. 2024-04-24T23:32:43.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:32:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 -- 172.21.15.33:0/2610579721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18102200 msgr2=0x7fab181045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 --2- 172.21.15.33:0/2610579721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18102200 0x7fab181045f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009900 tx=0x7fab0c02a410 comp rx=0 tx=0).stop 2024-04-24T23:32:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 -- 172.21.15.33:0/2610579721 shutdown_connections 2024-04-24T23:32:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 --2- 172.21.15.33:0/2610579721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab18107460 0x7fab181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 --2- 172.21.15.33:0/2610579721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab18104b30 0x7fab18106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 --2- 172.21.15.33:0/2610579721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18102200 0x7fab181045f0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.472+0000 7fab1f24d640 1 -- 172.21.15.33:0/2610579721 >> 172.21.15.33:0/2610579721 conn(0x7fab180fbc30 msgr2=0x7fab180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.473+0000 7fab1f24d640 1 -- 172.21.15.33:0/2610579721 shutdown_connections 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.473+0000 7fab1f24d640 1 -- 172.21.15.33:0/2610579721 wait complete. 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.473+0000 7fab1f24d640 1 Processor -- start 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.473+0000 7fab1f24d640 1 -- start start 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1f24d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab18102200 0x7fab181958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1f24d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab18104b30 0x7fab18195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1f24d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 0x7fab1819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1f24d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810d400 con 0x7fab18107460 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1f24d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810d280 con 0x7fab18104b30 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1f24d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810d580 con 0x7fab18102200 2024-04-24T23:32:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1d7ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 0x7fab1819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1d7ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 0x7fab1819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39186/0 (socket says 172.21.15.33:39186) 2024-04-24T23:32:43.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.474+0000 7fab1d7ea640 1 -- 172.21.15.33:0/3630996078 learned_addr learned my addr 172.21.15.33:0/3630996078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.475+0000 7fab1d7ea640 1 -- 172.21.15.33:0/3630996078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab18102200 msgr2=0x7fab181958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.475+0000 7fab1d7ea640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab18102200 0x7fab181958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.475+0000 7fab1d7ea640 1 -- 172.21.15.33:0/3630996078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab18104b30 msgr2=0x7fab18195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.475+0000 7fab1d7ea640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab18104b30 0x7fab18195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.475+0000 7fab1d7ea640 1 -- 172.21.15.33:0/3630996078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1819d510 con 0x7fab18107460 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.476+0000 7fab1d7ea640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 0x7fab1819cea0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fab1000eb10 tx=0x7fab1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.478+0000 7fab15ffb640 1 -- 172.21.15.33:0/3630996078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10004b20 con 0x7fab18107460 2024-04-24T23:32:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.478+0000 7fab15ffb640 1 -- 172.21.15.33:0/3630996078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab10004cc0 con 0x7fab18107460 2024-04-24T23:32:43.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.478+0000 7fab15ffb640 1 -- 172.21.15.33:0/3630996078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10010760 con 0x7fab18107460 2024-04-24T23:32:43.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.478+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1819d800 con 0x7fab18107460 2024-04-24T23:32:43.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.478+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1819dcf0 con 0x7fab18107460 2024-04-24T23:32:43.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.482+0000 7fab15ffb640 1 -- 172.21.15.33:0/3630996078 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab10010900 con 0x7fab18107460 2024-04-24T23:32:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 Processor -- start 2024-04-24T23:32:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 -- start start 2024-04-24T23:32:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 0x556d8d5ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d5f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d8d551670 con 0x556d8d5f2d90 2024-04-24T23:32:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d8d4f9970 con 0x556d8d5edc20 2024-04-24T23:32:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.483+0000 7fab22039740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d8d54d630 con 0x556d8d5ed230 2024-04-24T23:32:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d5f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 0x556d8d5ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 0x556d8d5ed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39560/0 (socket says 172.21.15.33:39560) 2024-04-24T23:32:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 -- 172.21.15.33:0/1203834505 learned_addr learned my addr 172.21.15.33:0/1203834505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab15ffb640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fab080a2d60 0x7fab080a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 -- 172.21.15.33:0/1203834505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 msgr2=0x556d8d5f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d5f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 -- 172.21.15.33:0/1203834505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 msgr2=0x556d8d5f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.484+0000 7fab17fff640 1 -- 172.21.15.33:0/1203834505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8d51d400 con 0x556d8d5ed230 2024-04-24T23:32:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.485+0000 7fab15ffb640 1 -- 172.21.15.33:0/3630996078 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab100cfb80 con 0x7fab18107460 2024-04-24T23:32:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.485+0000 7fab17fff640 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 0x556d8d5ed650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fab04009bd0 tx=0x7fab040107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8164792cd10f1ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.485+0000 7fab1d7ea640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fab080a2d60 0x7fab080a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.486+0000 7fab1d7ea640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fab080a2d60 0x7fab080a5220 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fab1009cc30 tx=0x7fab10014000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.486+0000 7fab02ffd640 1 -- 172.21.15.33:0/1203834505 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04004050 con 0x556d8d5ed230 2024-04-24T23:32:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.487+0000 7fab02ffd640 1 -- 172.21.15.33:0/1203834505 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab040041f0 con 0x556d8d5ed230 2024-04-24T23:32:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.487+0000 7fab02ffd640 1 -- 172.21.15.33:0/1203834505 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040116a0 con 0x556d8d5ed230 2024-04-24T23:32:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 -- 172.21.15.33:0/1203834505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 msgr2=0x556d8d5ed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 0x556d8d5ed650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fab04009bd0 tx=0x7fab040107a0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 -- 172.21.15.33:0/1203834505 shutdown_connections 2024-04-24T23:32:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d5f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 --2- 172.21.15.33:0/1203834505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5ed230 0x556d8d5ed650 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 -- 172.21.15.33:0/1203834505 >> 172.21.15.33:0/1203834505 conn(0x556d8d5e88a0 msgr2=0x556d8d5e8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 -- 172.21.15.33:0/1203834505 shutdown_connections 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.488+0000 7fab22039740 1 -- 172.21.15.33:0/1203834505 wait complete. 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.489+0000 7fab22039740 1 -- start start 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.489+0000 7fab22039740 1 Processor -- start 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab22039740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5edc20 0x556d8d5f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab22039740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d5f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab22039740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d5f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d5f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39568/0 (socket says 172.21.15.33:39568) 2024-04-24T23:32:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab1cfe9640 1 -- 172.21.15.33:0/1569351170 learned_addr learned my addr 172.21.15.33:0/1569351170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d8d5f5d40 con 0x556d8d5edc20 2024-04-24T23:32:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d8d50d050 con 0x556d8d5fcb70 2024-04-24T23:32:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.490+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d8d5f5ec0 con 0x556d8d5f2d90 2024-04-24T23:32:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab17fff640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5edc20 0x556d8d5f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab22039740 10 client.0 _ll_register_callbacks cb 0x556d8d264be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab17fff640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab1cfe9640 1 -- 172.21.15.33:0/1569351170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 msgr2=0x556d8d5fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5fef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab1cfe9640 1 -- 172.21.15.33:0/1569351170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5edc20 msgr2=0x556d8d5f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5edc20 0x556d8d5f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab1cfe9640 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d8d4dedc0 con 0x556d8d5f2d90 2024-04-24T23:32:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.491+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d5f7b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab0c02a930 tx=0x7fab0c032e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1de077de81a16a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.492+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab040041f0 con 0x556d8d5f2d90 2024-04-24T23:32:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.492+0000 7faaf7fff640 10 client.29143 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7faaf7fff640 1 -- 172.21.15.33:0/1569351170 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c040070 con 0x556d8d5f2d90 2024-04-24T23:32:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7faaf7fff640 1 -- 172.21.15.33:0/1569351170 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab0c004030 con 0x556d8d5f2d90 2024-04-24T23:32:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7faaf7fff640 1 -- 172.21.15.33:0/1569351170 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fab0c035040 con 0x556d8d5f2d90 2024-04-24T23:32:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 10 client.29143 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=33}) -- 0x556d8d2653f0 con 0x556d8d5f2d90 2024-04-24T23:32:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7faaf7fff640 1 -- 172.21.15.33:0/1569351170 <== mon.1 v2:172.21.15.117:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fab0c004270 con 0x556d8d5f2d90 2024-04-24T23:32:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 10 client.29143 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 10 client.29143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 20 client.29143 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d605a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 4 client.29143 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.493+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab040041f0 con 0x556d8d6035c0 2024-04-24T23:32:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.494+0000 7fab17fff640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d605a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.494+0000 7fab17fff640 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d605a40 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b9f6e0f919df73 server_cookie=13f7e4851f987394 in_seq=0 out_seq=0 2024-04-24T23:32:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.494+0000 7faaf7fff640 10 client.29143 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.496+0000 7faaf7fff640 1 -- 172.21.15.33:0/1569351170 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fab040041f0 con 0x556d8d6035c0 2024-04-24T23:32:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.496+0000 7faaf7fff640 10 client.29143 handle_command_reply: tid=0 2024-04-24T23:32:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.496+0000 7fab22039740 1 client.29143 shutdown 2024-04-24T23:32:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 msgr2=0x556d8d5f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d5f7b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab0c02a930 tx=0x7fab0c032e60 comp rx=0 tx=0).stop 2024-04-24T23:32:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 shutdown_connections 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d605a40 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d5f7b40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7faaf7fff640 0 client.29143 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 --2- 172.21.15.33:0/1569351170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5edc20 0x556d8d5f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 >> 172.21.15.33:0/1569351170 conn(0x556d8d5e88a0 msgr2=0x556d8d5e7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.497+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 shutdown_connections 2024-04-24T23:32:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.498+0000 7fab22039740 1 -- 172.21.15.33:0/1569351170 wait complete. 2024-04-24T23:32:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.498+0000 7fab22039740 20 client.29143 trim_cache size 0 max 16384 2024-04-24T23:32:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.524+0000 7fab22039740 1 Processor -- start 2024-04-24T23:32:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.524+0000 7fab22039740 1 -- start start 2024-04-24T23:32:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab22039740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5edc20 0x556d8d5ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab22039740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab22039740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab22039740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faafc012510 con 0x556d8d5f2d90 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab22039740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00fde0 con 0x556d8d5fcb70 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab22039740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00dc30 con 0x556d8d5edc20 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5eec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39206/0 (socket says 172.21.15.33:39206) 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.525+0000 7fab1cfe9640 1 -- 172.21.15.33:0/334503032 learned_addr learned my addr 172.21.15.33:0/334503032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab1cfe9640 1 -- 172.21.15.33:0/334503032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5edc20 msgr2=0x556d8d5ee720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5edc20 0x556d8d5ee720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab1cfe9640 1 -- 172.21.15.33:0/334503032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 msgr2=0x556d8d5ef1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5ef1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab1cfe9640 1 -- 172.21.15.33:0/334503032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8d32f7d0 con 0x556d8d5f2d90 2024-04-24T23:32:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5eec60 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009a20 tx=0x7fab0c037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6475fb5369b7ce48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab027fc640 1 -- 172.21.15.33:0/334503032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c000e60 con 0x556d8d5f2d90 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.526+0000 7fab027fc640 1 -- 172.21.15.33:0/334503032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab0c031980 con 0x556d8d5f2d90 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.527+0000 7fab027fc640 1 -- 172.21.15.33:0/334503032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c004de0 con 0x556d8d5f2d90 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.527+0000 7fab22039740 1 -- 172.21.15.33:0/334503032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 msgr2=0x556d8d5eec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.527+0000 7fab22039740 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5eec60 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009a20 tx=0x7fab0c037000 comp rx=0 tx=0).stop 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.528+0000 7fab22039740 1 -- 172.21.15.33:0/334503032 shutdown_connections 2024-04-24T23:32:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.528+0000 7fab22039740 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5fcb70 0x556d8d5ef1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.528+0000 7fab22039740 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5f2d90 0x556d8d5eec60 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.528+0000 7fab22039740 1 --2- 172.21.15.33:0/334503032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5edc20 0x556d8d5ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.528+0000 7fab22039740 1 -- 172.21.15.33:0/334503032 >> 172.21.15.33:0/334503032 conn(0x556d8d5e88a0 msgr2=0x556d8d5eb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.529+0000 7fab22039740 1 -- 172.21.15.33:0/334503032 shutdown_connections 2024-04-24T23:32:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.529+0000 7fab22039740 1 -- 172.21.15.33:0/334503032 wait complete. 2024-04-24T23:32:43.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.529+0000 7fab22039740 1 -- start start 2024-04-24T23:32:43.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.530+0000 7fab22039740 1 Processor -- start 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.530+0000 7fab22039740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d616ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.531+0000 7fab22039740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d6150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.531+0000 7fab22039740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5fcb70 0x556d8d615630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.531+0000 7fab22039740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00e2a0 con 0x556d8d5fcb70 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.531+0000 7fab22039740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00f710 con 0x556d8d5edc20 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.531+0000 7fab22039740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00e970 con 0x556d8d5f2d90 2024-04-24T23:32:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.531+0000 7fab22039740 10 client.0 _ll_register_callbacks cb 0x7fab04005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.532+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d6150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.532+0000 7fab1cfe9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d6150f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39578/0 (socket says 172.21.15.33:39578) 2024-04-24T23:32:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.532+0000 7fab1cfe9640 1 -- 172.21.15.33:0/3679761104 learned_addr learned my addr 172.21.15.33:0/3679761104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.532+0000 7fab17fff640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d616ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab17fff640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5fcb70 0x556d8d615630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab1cfe9640 1 -- 172.21.15.33:0/3679761104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 msgr2=0x556d8d616ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d616ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab1cfe9640 1 -- 172.21.15.33:0/3679761104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5fcb70 msgr2=0x556d8d615630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5fcb70 0x556d8d615630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab1cfe9640 1 -- 172.21.15.33:0/3679761104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d8d615ba0 con 0x556d8d5f2d90 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.533+0000 7fab1cfe9640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d6150f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab0c005e10 tx=0x7fab0c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88335a0025e85b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.534+0000 7faaf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7faaf7fff640 1 -- 172.21.15.33:0/3679761104 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c05d070 con 0x556d8d5f2d90 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7faaf7fff640 1 -- 172.21.15.33:0/3679761104 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab0c03db30 con 0x556d8d5f2d90 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0c031980 con 0x556d8d5f2d90 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7faaf7fff640 1 -- 172.21.15.33:0/3679761104 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fab0c066050 con 0x556d8d5f2d90 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7fab22039740 10 client.29146 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d8d66a400 con 0x556d8d5f2d90 2024-04-24T23:32:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.535+0000 7faaf7fff640 1 -- 172.21.15.33:0/3679761104 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab0c03dd30 con 0x556d8d5f2d90 2024-04-24T23:32:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.536+0000 7faaf7fff640 1 -- 172.21.15.33:0/3679761104 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fab0c060020 con 0x556d8d5f2d90 2024-04-24T23:32:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.536+0000 7fab22039740 10 client.29146 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.536+0000 7fab22039740 10 client.29146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.536+0000 7fab22039740 20 client.29146 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.536+0000 7fab22039740 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d66cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.537+0000 7fab22039740 4 client.29146 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:32:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.537+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fab0c031980 con 0x556d8d6035c0 2024-04-24T23:32:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.537+0000 7fab17fff640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d66cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.538+0000 7fab17fff640 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d66cdb0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595a0b9967f8491f server_cookie=752f79ea2b1dbc16 in_seq=0 out_seq=0 2024-04-24T23:32:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.538+0000 7faaf7fff640 10 client.29146 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.538+0000 7faaf7fff640 1 -- 172.21.15.33:0/3679761104 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fab0c031980 con 0x556d8d6035c0 2024-04-24T23:32:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.538+0000 7faaf7fff640 10 client.29146 handle_command_reply: tid=0 2024-04-24T23:32:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.538+0000 7fab22039740 1 client.29146 shutdown 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.539+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 msgr2=0x556d8d6150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.539+0000 7fab22039740 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d6150f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab0c005e10 tx=0x7fab0c031400 comp rx=0 tx=0).stop 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.539+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 shutdown_connections 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.539+0000 7fab22039740 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556d8d6035c0 0x556d8d66cdb0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.539+0000 7fab22039740 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d8d5fcb70 0x556d8d615630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.539+0000 7fab22039740 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d8d5f2d90 0x556d8d6150f0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab0c005e10 tx=0x7fab0c031400 comp rx=0 tx=0).stop 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.540+0000 7fab22039740 1 --2- 172.21.15.33:0/3679761104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d8d5edc20 0x556d8d616ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.540+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 >> 172.21.15.33:0/3679761104 conn(0x556d8d5e88a0 msgr2=0x556d8d5e72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.541+0000 7faaf7fff640 0 client.29146 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:43.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.541+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 shutdown_connections 2024-04-24T23:32:43.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.541+0000 7fab22039740 1 -- 172.21.15.33:0/3679761104 wait complete. 2024-04-24T23:32:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.542+0000 7fab22039740 20 client.29146 trim_cache size 0 max 16384 2024-04-24T23:32:43.547 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.549+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fab080a2d60 msgr2=0x7fab080a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.549+0000 7fab1f24d640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fab080a2d60 0x7fab080a5220 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fab1009cc30 tx=0x7fab10014000 comp rx=0 tx=0).stop 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.549+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 msgr2=0x7fab1819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.549+0000 7fab1f24d640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 0x7fab1819cea0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fab1000eb10 tx=0x7fab1000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.551+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 shutdown_connections 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.551+0000 7fab1f24d640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fab080a2d60 0x7fab080a5220 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.551+0000 7fab1f24d640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18107460 0x7fab1819cea0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.551+0000 7fab1f24d640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab18104b30 0x7fab18195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.551+0000 7fab1f24d640 1 --2- 172.21.15.33:0/3630996078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab18102200 0x7fab181958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.551+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 >> 172.21.15.33:0/3630996078 conn(0x7fab180fbc30 msgr2=0x7fab18101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.552+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 shutdown_connections 2024-04-24T23:32:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:43.552+0000 7fab1f24d640 1 -- 172.21.15.33:0/3630996078 wait complete. 2024-04-24T23:32:43.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 174429 22.37 MB/sec execute 288 sec latency 106.481 ms 2024-04-24T23:32:44.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:43 smithi038 ceph-mon[32450]: pgmap v972: 129 pgs: 129 active+clean; 601 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 377 op/s 2024-04-24T23:32:44.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:43 smithi117 ceph-mon[32529]: pgmap v972: 129 pgs: 129 active+clean; 601 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 377 op/s 2024-04-24T23:32:44.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:43 smithi033 ceph-mon[29201]: pgmap v972: 129 pgs: 129 active+clean; 601 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 377 op/s 2024-04-24T23:32:44.572 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:32:44.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:44.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:32:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 -- 172.21.15.33:0/2045198698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af41006a0 msgr2=0x7f0af4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/2045198698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af41006a0 0x7f0af4100ac0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009900 tx=0x7f0af002a410 comp rx=0 tx=0).stop 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 -- 172.21.15.33:0/2045198698 shutdown_connections 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/2045198698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4102250 0x7f0af4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/2045198698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af4101890 0x7f0af4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/2045198698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af41006a0 0x7f0af4100ac0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 -- 172.21.15.33:0/2045198698 >> 172.21.15.33:0/2045198698 conn(0x7f0af40fbc30 msgr2=0x7f0af40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 -- 172.21.15.33:0/2045198698 shutdown_connections 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.695+0000 7f0afc5ee640 1 -- 172.21.15.33:0/2045198698 wait complete. 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 Processor -- start 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 -- start start 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 0x7f0af406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4101890 0x7f0af419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 0x7f0af419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0af410c320 con 0x7f0af4102250 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0af410c1a0 con 0x7f0af41006a0 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afc5ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af410c4a0 con 0x7f0af4101890 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 0x7f0af419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 0x7f0af406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 0x7f0af406a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:33034/0 (socket says 172.21.15.33:33034) 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 0x7f0af419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39218/0 (socket says 172.21.15.33:39218) 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afa38a640 1 -- 172.21.15.33:0/1811389402 learned_addr learned my addr 172.21.15.33:0/1811389402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4101890 0x7f0af419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 -- 172.21.15.33:0/1811389402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4101890 msgr2=0x7f0af419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4101890 0x7f0af419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 -- 172.21.15.33:0/1811389402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 msgr2=0x7f0af406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 0x7f0af406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 -- 172.21.15.33:0/1811389402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af419f730 con 0x7f0af4102250 2024-04-24T23:32:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 0x7f0af406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0afab8b640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 0x7f0af419ee20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009fd0 tx=0x7f0aec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.696+0000 7f0aeb7fe640 1 -- 172.21.15.33:0/1811389402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00ce20 con 0x7f0af4102250 2024-04-24T23:32:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.697+0000 7f0aeb7fe640 1 -- 172.21.15.33:0/1811389402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aec005020 con 0x7f0af4102250 2024-04-24T23:32:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.697+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af419f9c0 con 0x7f0af4102250 2024-04-24T23:32:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.697+0000 7f0aeb7fe640 1 -- 172.21.15.33:0/1811389402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec010720 con 0x7f0af4102250 2024-04-24T23:32:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.697+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af419ff10 con 0x7f0af4102250 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.699+0000 7f0aeb7fe640 1 -- 172.21.15.33:0/1811389402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0aec005540 con 0x7f0af4102250 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.699+0000 7f0aff3d8740 1 Processor -- start 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.699+0000 7f0aff3d8740 1 -- start start 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 0x55a558bb2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bb28f0 0x55a558bb4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a558b102c0 con 0x55a558bacd90 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a558ab8910 con 0x55a558bb28f0 2024-04-24T23:32:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a558b0c280 con 0x55a558bad780 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 0x55a558bb2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 0x55a558bb2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39618/0 (socket says 172.21.15.33:39618) 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0af9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 -- 172.21.15.33:0/1990550306 learned_addr learned my addr 172.21.15.33:0/1990550306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bb28f0 0x55a558bb4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aeb7fe640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ae00a2b90 0x7f0ae00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 -- 172.21.15.33:0/1990550306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bb28f0 msgr2=0x55a558bb4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bb28f0 0x55a558bb4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 -- 172.21.15.33:0/1990550306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 msgr2=0x55a558bad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 -- 172.21.15.33:0/1990550306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a558ac9fb0 con 0x55a558bad780 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bb28f0 0x55a558bb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afab8b640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ae00a2b90 0x7f0ae00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 0x55a558bb2280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0af0031440 tx=0x7f0af0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7bc92d5fbcc93f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0aeb7fe640 1 -- 172.21.15.33:0/1811389402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0aec0cf9d0 con 0x7f0af4102250 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.700+0000 7f0ae8bfe640 1 -- 172.21.15.33:0/1990550306 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0035040 con 0x55a558bad780 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0ae8bfe640 1 -- 172.21.15.33:0/1990550306 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af00319b0 con 0x55a558bad780 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0ae8bfe640 1 -- 172.21.15.33:0/1990550306 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af00049a0 con 0x55a558bad780 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1990550306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 msgr2=0x55a558bb2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0afab8b640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ae00a2b90 0x7f0ae00a5050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0aec01f000 tx=0x7f0aec005b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 0x55a558bb2280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0af0031440 tx=0x7f0af0005de0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1990550306 shutdown_connections 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bb28f0 0x55a558bb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bad780 0x55a558bb2280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1990550306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1990550306 >> 172.21.15.33:0/1990550306 conn(0x55a558b74950 msgr2=0x55a558ba8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1990550306 shutdown_connections 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.701+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1990550306 wait complete. 2024-04-24T23:32:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 -- start start 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 Processor -- start 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bb7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bad780 0x55a558bb76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a558bb5840 con 0x55a558bacd90 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a558acbe90 con 0x55a558bad780 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a558bb59c0 con 0x55a558bb28f0 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0aff3d8740 10 client.0 _ll_register_callbacks cb 0x55a558b07970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bb7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39244/0 (socket says 172.21.15.33:39244) 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 -- 172.21.15.33:0/1716572453 learned_addr learned my addr 172.21.15.33:0/1716572453 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bad780 0x55a558bb76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 -- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 msgr2=0x55a558bbe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 -- 172.21.15.33:0/1716572453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bad780 msgr2=0x55a558bb76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bad780 0x55a558bb76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0af9b89640 1 -- 172.21.15.33:0/1716572453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5588ec7e0 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.702+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bad780 0x55a558bb76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0af9b89640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bb7180 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4012ea0 tx=0x7f0ae4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8524572e9520530e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0addffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0addffb640 1 -- 172.21.15.33:0/1716572453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae401c030 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0addffb640 1 -- 172.21.15.33:0/1716572453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ae4012110 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af00319b0 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0addffb640 1 -- 172.21.15.33:0/1716572453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ae4027050 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 10 client.19734 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a55894cf70 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0addffb640 1 -- 172.21.15.33:0/1716572453 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af00319b0 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0addffb640 1 -- 172.21.15.33:0/1716572453 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0ae4003c90 con 0x55a558bacd90 2024-04-24T23:32:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 10 client.19734 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 20 client.19734 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558bc5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 4 client.19734 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.703+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a558ae45a0 con 0x55a558bc2c10 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.704+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558bc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.704+0000 7f0afa38a640 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558bc5080 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbc4f0c54e49c0b server_cookie=6bc4830635dba885 in_seq=0 out_seq=0 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.704+0000 7f0addffb640 10 client.19734 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0addffb640 1 -- 172.21.15.33:0/1716572453 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a558ae45a0 con 0x55a558bc2c10 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0addffb640 10 client.19734 handle_command_reply: tid=0 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 client.19734 shutdown 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 msgr2=0x55a558bb7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bb7180 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4012ea0 tx=0x7f0ae4005f90 comp rx=0 tx=0).stop 2024-04-24T23:32:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 shutdown_connections 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558bc5080 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0addffb640 0 client.19734 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bad780 0x55a558bb76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/1716572453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bacd90 0x55a558bb7180 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 >> 172.21.15.33:0/1716572453 conn(0x55a558b74950 msgr2=0x55a558ba7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 shutdown_connections 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 1 -- 172.21.15.33:0/1716572453 wait complete. 2024-04-24T23:32:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.705+0000 7f0aff3d8740 20 client.19734 trim_cache size 0 max 16384 2024-04-24T23:32:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 Processor -- start 2024-04-24T23:32:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 -- start start 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558ba77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558ba7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41accd0 con 0x55a558bad780 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41aa5a0 con 0x55a558bacd90 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41a83f0 con 0x55a558bb28f0 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558ba7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558ba7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39252/0 (socket says 172.21.15.33:39252) 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0afa38a640 1 -- 172.21.15.33:0/4099488068 learned_addr learned my addr 172.21.15.33:0/4099488068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0af9b89640 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558ba77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.723+0000 7f0af9b89640 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0afa38a640 1 -- 172.21.15.33:0/4099488068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 msgr2=0x55a558bbe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0afa38a640 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0afa38a640 1 -- 172.21.15.33:0/4099488068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 msgr2=0x55a558ba77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0afa38a640 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558ba77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0afa38a640 1 -- 172.21.15.33:0/4099488068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5587e4170 con 0x55a558bad780 2024-04-24T23:32:44.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0afa38a640 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558ba7d10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0af0006010 tx=0x7f0af0049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda2ce13a086df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0adffff640 1 -- 172.21.15.33:0/4099488068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0035040 con 0x55a558bad780 2024-04-24T23:32:44.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0adffff640 1 -- 172.21.15.33:0/4099488068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af0031bd0 con 0x55a558bad780 2024-04-24T23:32:44.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.724+0000 7f0adffff640 1 -- 172.21.15.33:0/4099488068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af00035f0 con 0x55a558bad780 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- 172.21.15.33:0/4099488068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 msgr2=0x55a558ba7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558ba7d10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0af0006010 tx=0x7f0af0049040 comp rx=0 tx=0).stop 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- 172.21.15.33:0/4099488068 shutdown_connections 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558ba7d10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/4099488068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558ba77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- 172.21.15.33:0/4099488068 >> 172.21.15.33:0/4099488068 conn(0x55a558b74950 msgr2=0x55a558ba70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- 172.21.15.33:0/4099488068 shutdown_connections 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- 172.21.15.33:0/4099488068 wait complete. 2024-04-24T23:32:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- start start 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 Processor -- start 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558bbc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558bbccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41a8a60 con 0x55a558bad780 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41a9ed0 con 0x55a558bacd90 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41a9130 con 0x55a558bb28f0 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0aff3d8740 10 client.0 _ll_register_callbacks cb 0x7f0ae4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558bbccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0af9b89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558bbc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.725+0000 7f0afa38a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558bbccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39258/0 (socket says 172.21.15.33:39258) 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 -- 172.21.15.33:0/309615045 learned_addr learned my addr 172.21.15.33:0/309615045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0af9b89640 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 -- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 msgr2=0x55a558bbdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 -- 172.21.15.33:0/309615045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 msgr2=0x55a558bbc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558bbc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 -- 172.21.15.33:0/309615045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a558bbe280 con 0x55a558bad780 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0afa38a640 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558bbccb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0af0031610 tx=0x7f0af00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5786e8f2d6350f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0addffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0addffb640 1 -- 172.21.15.33:0/309615045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0035070 con 0x55a558bad780 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0addffb640 1 -- 172.21.15.33:0/309615045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0af0052800 con 0x55a558bad780 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af0031bd0 con 0x55a558bad780 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0addffb640 1 -- 172.21.15.33:0/309615045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0af005d470 con 0x55a558bad780 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0aff3d8740 10 client.19740 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:44.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.726+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a558c29ac0 con 0x55a558bad780 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0addffb640 1 -- 172.21.15.33:0/309615045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af0031bd0 con 0x55a558bad780 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0addffb640 1 -- 172.21.15.33:0/309615045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0af0046800 con 0x55a558bad780 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0aff3d8740 10 client.19740 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0aff3d8740 10 client.19740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0aff3d8740 20 client.19740 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558c2bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0aff3d8740 4 client.19740 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a558b1d800 con 0x55a558bc2c10 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0af9b89640 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558c2bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0addffb640 10 client.19740 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.727+0000 7f0af9b89640 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558c2bf60 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c659a69fb631248d server_cookie=4e1e4ef3172a81f8 in_seq=0 out_seq=0 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.728+0000 7f0addffb640 1 -- 172.21.15.33:0/309615045 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a558b1d800 con 0x55a558bc2c10 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0addffb640 10 client.19740 handle_command_reply: tid=0 2024-04-24T23:32:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 client.19740 shutdown 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 msgr2=0x55a558bbccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558bbccb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0af0031610 tx=0x7f0af00058b0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 shutdown_connections 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a558bc2c10 0x55a558c2bf60 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a558bb28f0 0x55a558bbdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0addffb640 0 client.19740 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a558bad780 0x55a558bbccb0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 --2- 172.21.15.33:0/309615045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a558bacd90 0x55a558bbc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 >> 172.21.15.33:0/309615045 conn(0x55a558b74950 msgr2=0x55a558ba7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 shutdown_connections 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 1 -- 172.21.15.33:0/309615045 wait complete. 2024-04-24T23:32:44.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0aff3d8740 20 client.19740 trim_cache size 0 max 16384 2024-04-24T23:32:44.735 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:32:44.735 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:32:44.735 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "00b28a40-b290-45e3-9ac4-f3899e9d3df0", 2024-04-24T23:32:44.735 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:32:44.735 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ae00a2b90 msgr2=0x7f0ae00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ae00a2b90 0x7f0ae00a5050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0aec01f000 tx=0x7f0aec005b40 comp rx=0 tx=0).stop 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 msgr2=0x7f0af419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 0x7f0af419ee20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009fd0 tx=0x7f0aec00ef10 comp rx=0 tx=0).stop 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 shutdown_connections 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0ae00a2b90 0x7f0ae00a5050 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0af4102250 0x7f0af419ee20 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4101890 0x7f0af419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.729+0000 7f0afc5ee640 1 --2- 172.21.15.33:0/1811389402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af41006a0 0x7f0af406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.730+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 >> 172.21.15.33:0/1811389402 conn(0x7f0af40fbc30 msgr2=0x7f0af4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.730+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 shutdown_connections 2024-04-24T23:32:44.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.730+0000 7f0afc5ee640 1 -- 172.21.15.33:0/1811389402 wait complete. 2024-04-24T23:32:44.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:32:44.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 -- 172.21.15.33:0/2289970195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 msgr2=0x7f6d6c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 --2- 172.21.15.33:0/2289970195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c108ae0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009ca0 tx=0x7f6d6402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 -- 172.21.15.33:0/2289970195 shutdown_connections 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 --2- 172.21.15.33:0/2289970195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c108ae0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 --2- 172.21.15.33:0/2289970195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d6c101890 0x7f6d6c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 --2- 172.21.15.33:0/2289970195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c1006a0 0x7f6d6c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 -- 172.21.15.33:0/2289970195 >> 172.21.15.33:0/2289970195 conn(0x7f6d6c0fbc30 msgr2=0x7f6d6c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 -- 172.21.15.33:0/2289970195 shutdown_connections 2024-04-24T23:32:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.863+0000 7f6d71dee640 1 -- 172.21.15.33:0/2289970195 wait complete. 2024-04-24T23:32:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 Processor -- start 2024-04-24T23:32:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 -- start start 2024-04-24T23:32:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c1006a0 0x7f6d6c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d6c101890 0x7f6d6c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c10c320 con 0x7f6d6c102250 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c10c1a0 con 0x7f6d6c101890 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d71dee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c10c4a0 con 0x7f6d6c1006a0 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c1006a0 0x7f6d6c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39276/0 (socket says 172.21.15.33:39276) 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.864+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1547822067 learned_addr learned my addr 172.21.15.33:0/1547822067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6affd640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d6c101890 0x7f6d6c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1547822067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c1006a0 msgr2=0x7f6d6c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c1006a0 0x7f6d6c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1547822067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d6c101890 msgr2=0x7f6d6c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d6c101890 0x7f6d6c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1547822067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c19f680 con 0x7f6d6c102250 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c19f010 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005e30 tx=0x7f6d64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d68ff9640 1 -- 172.21.15.33:0/1547822067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64003b70 con 0x7f6d6c102250 2024-04-24T23:32:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c19f910 con 0x7f6d6c102250 2024-04-24T23:32:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d68ff9640 1 -- 172.21.15.33:0/1547822067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d64003d10 con 0x7f6d6c102250 2024-04-24T23:32:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d68ff9640 1 -- 172.21.15.33:0/1547822067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6402fad0 con 0x7f6d6c102250 2024-04-24T23:32:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.865+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c19fe60 con 0x7f6d6c102250 2024-04-24T23:32:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.868+0000 7f6d68ff9640 1 -- 172.21.15.33:0/1547822067 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6d640045c0 con 0x7f6d6c102250 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 Processor -- start 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d68ff9640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d540a2b90 0x7f6d540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 -- start start 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d6affd640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d540a2b90 0x7f6d540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d68ff9640 1 -- 172.21.15.33:0/1547822067 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d640f0360 con 0x7f6d6c102250 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699e5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e5da0 0x559b699e8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559b699446b0 con 0x559b699e5da0 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d74bda740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b698ec880 con 0x559b699e0c30 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e5da0 0x559b699e8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39684/0 (socket says 172.21.15.33:39684) 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.869+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/3301449194 learned_addr learned my addr 172.21.15.33:0/3301449194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d74bda740 1 -- 172.21.15.33:0/3301449194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b69940670 con 0x559b699e0240 2024-04-24T23:32:44.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6affd640 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/3301449194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 msgr2=0x559b699e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699e5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/3301449194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e5da0 msgr2=0x559b699e8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e5da0 0x559b699e8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/3301449194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b697227d0 con 0x559b699e0240 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6affd640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d540a2b90 0x7f6d540a5050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c198f70 tx=0x7f6d60011810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e5da0 0x559b699e8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e0660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0091e0 tx=0x7f6d5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4089b1491dbaaff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d51ffb640 1 -- 172.21.15.33:0/3301449194 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c004220 con 0x559b699e0240 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d51ffb640 1 -- 172.21.15.33:0/3301449194 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c009e20 con 0x559b699e0240 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.870+0000 7f6d51ffb640 1 -- 172.21.15.33:0/3301449194 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c016780 con 0x559b699e0240 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 -- 172.21.15.33:0/3301449194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 msgr2=0x559b699e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e0660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0091e0 tx=0x7f6d5c011810 comp rx=0 tx=0).stop 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 -- 172.21.15.33:0/3301449194 shutdown_connections 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e5da0 0x559b699e8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 --2- 172.21.15.33:0/3301449194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e0660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 -- 172.21.15.33:0/3301449194 >> 172.21.15.33:0/3301449194 conn(0x559b699db8b0 msgr2=0x559b699dbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 -- 172.21.15.33:0/3301449194 shutdown_connections 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.871+0000 7f6d74bda740 1 -- 172.21.15.33:0/3301449194 wait complete. 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 -- start start 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 Processor -- start 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699ea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699eab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e5da0 0x559b699f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559b699e8d50 con 0x559b699e0240 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b698fff50 con 0x559b699e0c30 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b699e8ed0 con 0x559b699e5da0 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d74bda740 10 client.0 _ll_register_callbacks cb 0x559b6993b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699ea630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39300/0 (socket says 172.21.15.33:39300) 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/1307437290 learned_addr learned my addr 172.21.15.33:0/1307437290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d6affd640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.872+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e5da0 0x559b699f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1307437290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 msgr2=0x559b699eab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699eab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1307437290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 msgr2=0x559b699ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699ea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6bfff640 1 -- 172.21.15.33:0/1307437290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b696583f0 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e5da0 0x559b699f1bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6d640edbc0 tx=0x7f6d640bc4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70bd2d23650156c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d4affd640 1 -- 172.21.15.33:0/1307437290 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640c0c80 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d5c009e20 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d4affd640 1 -- 172.21.15.33:0/1307437290 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d640b8f40 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d4affd640 1 -- 172.21.15.33:0/1307437290 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6d640c5540 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d74bda740 10 client.29176 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.873+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b696180f0 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d4affd640 1 -- 172.21.15.33:0/1307437290 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c009e20 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d4affd640 1 -- 172.21.15.33:0/1307437290 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6d64004b40 con 0x559b699e5da0 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d74bda740 10 client.29176 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d74bda740 10 client.29176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d74bda740 20 client.29176 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d74bda740 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b699f64c0 0x559b699f8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d74bda740 4 client.29176 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b69951bf0 con 0x559b699f64c0 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b699f64c0 0x559b699f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d4affd640 10 client.29176 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.874+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b699f64c0 0x559b699f8960 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5252444f64bdc885 server_cookie=20e75db667d5dda8 in_seq=0 out_seq=0 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d4affd640 1 -- 172.21.15.33:0/1307437290 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b69951bf0 con 0x559b699f64c0 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d4affd640 10 client.29176 handle_command_reply: tid=0 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 client.29176 shutdown 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e5da0 msgr2=0x559b699f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e5da0 0x559b699f1bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6d640edbc0 tx=0x7f6d640bc4d0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 shutdown_connections 2024-04-24T23:32:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b699f64c0 0x559b699f8960 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e5da0 0x559b699f1bb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e0c30 0x559b699eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 --2- 172.21.15.33:0/1307437290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d4affd640 0 client.29176 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 >> 172.21.15.33:0/1307437290 conn(0x559b699db8b0 msgr2=0x559b699daaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 shutdown_connections 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 1 -- 172.21.15.33:0/1307437290 wait complete. 2024-04-24T23:32:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.875+0000 7f6d74bda740 20 client.29176 trim_cache size 0 max 16384 2024-04-24T23:32:44.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 174911 22.40 MB/sec execute 289 sec latency 92.200 ms 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 Processor -- start 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 -- start start 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 0x559b699f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699e2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1aa8e0 con 0x559b699e0c30 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1a81b0 con 0x559b699e5da0 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d74bda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1a6000 con 0x559b699e0240 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 0x559b699f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 0x559b699f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39308/0 (socket says 172.21.15.33:39308) 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 -- 172.21.15.33:0/4160495964 learned_addr learned my addr 172.21.15.33:0/4160495964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699e2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 -- 172.21.15.33:0/4160495964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 msgr2=0x559b699e1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 -- 172.21.15.33:0/4160495964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 msgr2=0x559b699e2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699e2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 -- 172.21.15.33:0/4160495964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b699e1a40 con 0x559b699e0c30 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.895+0000 7f6d6affd640 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 0x559b699f1b70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6d6001e8c0 tx=0x7f6d6001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add323f2ca707cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d517fa640 1 -- 172.21.15.33:0/4160495964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60005020 con 0x559b699e0c30 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d517fa640 1 -- 172.21.15.33:0/4160495964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6001d7c0 con 0x559b699e0c30 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d517fa640 1 -- 172.21.15.33:0/4160495964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60004a00 con 0x559b699e0c30 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 -- 172.21.15.33:0/4160495964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 msgr2=0x559b699f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 0x559b699f1b70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6d6001e8c0 tx=0x7f6d6001d000 comp rx=0 tx=0).stop 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 -- 172.21.15.33:0/4160495964 shutdown_connections 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699e2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0c30 0x559b699f1b70 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 --2- 172.21.15.33:0/4160495964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0240 0x559b699e1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 -- 172.21.15.33:0/4160495964 >> 172.21.15.33:0/4160495964 conn(0x559b699db8b0 msgr2=0x559b699ea630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.896+0000 7f6d74bda740 1 -- 172.21.15.33:0/4160495964 shutdown_connections 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 -- 172.21.15.33:0/4160495964 wait complete. 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 -- start start 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 Processor -- start 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0c30 0x559b699ef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699f0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1a6670 con 0x559b699e0240 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1a7ae0 con 0x559b699e5da0 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1a6d40 con 0x559b699e0c30 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d74bda740 10 client.0 _ll_register_callbacks cb 0x7f6d64009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:32:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39322/0 (socket says 172.21.15.33:39322) 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/745035259 learned_addr learned my addr 172.21.15.33:0/745035259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.897+0000 7f6d6bfff640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6affd640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0c30 0x559b699ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0c30 msgr2=0x559b699ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0c30 0x559b699ef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/745035259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 msgr2=0x559b699f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699f0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6b7fe640 1 -- 172.21.15.33:0/745035259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b699f0830 con 0x559b699e0240 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d6b7fe640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699eecf0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009c70 tx=0x7f6d5c009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185b044e1d83832e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d4affd640 1 -- 172.21.15.33:0/745035259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0043e0 con 0x559b699e0240 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d4affd640 1 -- 172.21.15.33:0/745035259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d5c024040 con 0x559b699e0240 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6001d7c0 con 0x559b699e0240 2024-04-24T23:32:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d4affd640 1 -- 172.21.15.33:0/745035259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c009e20 con 0x559b699e0240 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d74bda740 10 client.19758 fetch_fsmap learned FSMap version 33 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b699f1bb0 con 0x559b699e0240 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d4affd640 1 -- 172.21.15.33:0/745035259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6001d7c0 con 0x559b699e0240 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.898+0000 7f6d4affd640 1 -- 172.21.15.33:0/745035259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6d5c004680 con 0x559b699e0240 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 10 client.19758 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 10 client.19758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 20 client.19758 populate_metadata read hostname 'smithi033' 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b69a5ce70 0x559b69a5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 4 client.19758 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b6994bf40 con 0x559b69a5ce70 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d6affd640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b69a5ce70 0x559b69a5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d4affd640 10 client.19758 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d6affd640 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b69a5ce70 0x559b69a5f260 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0c39ee872f2012 server_cookie=cdc32198549c2322 in_seq=0 out_seq=0 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d4affd640 1 -- 172.21.15.33:0/745035259 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559b6994bf40 con 0x559b69a5ce70 2024-04-24T23:32:44.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d4affd640 10 client.19758 handle_command_reply: tid=0 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 client.19758 shutdown 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 msgr2=0x559b699eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699eecf0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009c70 tx=0x7f6d5c009400 comp rx=0 tx=0).stop 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 shutdown_connections 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559b69a5ce70 0x559b69a5f260 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.899+0000 7f6d74bda740 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b699e5da0 0x559b699f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d74bda740 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b699e0c30 0x559b699ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d74bda740 1 --2- 172.21.15.33:0/745035259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b699e0240 0x559b699eecf0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d4affd640 0 client.19758 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 >> 172.21.15.33:0/745035259 conn(0x559b699db8b0 msgr2=0x559b699ea610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 shutdown_connections 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d74bda740 1 -- 172.21.15.33:0/745035259 wait complete. 2024-04-24T23:32:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d74bda740 20 client.19758 trim_cache size 0 max 16384 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "00b28a40-b290-45e3-9ac4-f3899e9d3df0": { 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "00b28a40-b290-45e3-9ac4-f3899e9d3df0", 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:32:44.907 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d540a2b90 msgr2=0x7f6d540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d540a2b90 0x7f6d540a5050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c198f70 tx=0x7f6d60011810 comp rx=0 tx=0).stop 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 msgr2=0x7f6d6c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c19f010 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005e30 tx=0x7f6d64004ab0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 shutdown_connections 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6d540a2b90 0x7f6d540a5050 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c102250 0x7f6d6c19f010 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d6c101890 0x7f6d6c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 --2- 172.21.15.33:0/1547822067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c1006a0 0x7f6d6c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 >> 172.21.15.33:0/1547822067 conn(0x7f6d6c0fbc30 msgr2=0x7f6d6c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 shutdown_connections 2024-04-24T23:32:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:32:44.900+0000 7f6d71dee640 1 -- 172.21.15.33:0/1547822067 wait complete. 2024-04-24T23:32:44.911 INFO:tasks.cephfs.filesystem:scrub status for tag:00b28a40-b290-45e3-9ac4-f3899e9d3df0 - {'path': '/', 'tag': '00b28a40-b290-45e3-9ac4-f3899e9d3df0', 'options': 'recursive,force'} 2024-04-24T23:32:45.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 175431 22.39 MB/sec execute 290 sec latency 100.817 ms 2024-04-24T23:32:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: pgmap v973: 129 pgs: 129 active+clean; 608 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:32:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:32:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:32:46.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:45 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: pgmap v973: 129 pgs: 129 active+clean; 608 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:32:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:45 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: pgmap v973: 129 pgs: 129 active+clean; 608 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:32:46.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:45 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:46.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 176017 22.44 MB/sec execute 291 sec latency 147.114 ms 2024-04-24T23:32:47.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:46 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:47.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:46 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:47.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:46 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:32:47.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 176259 22.42 MB/sec execute 292 sec latency 383.608 ms 2024-04-24T23:32:48.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:47 smithi038 ceph-mon[32450]: pgmap v974: 129 pgs: 129 active+clean; 608 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-04-24T23:32:48.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:47 smithi117 ceph-mon[32529]: pgmap v974: 129 pgs: 129 active+clean; 608 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-04-24T23:32:48.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:47 smithi033 ceph-mon[29201]: pgmap v974: 129 pgs: 129 active+clean; 608 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-04-24T23:32:48.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 176641 22.42 MB/sec execute 293 sec latency 181.552 ms 2024-04-24T23:32:49.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 177133 22.45 MB/sec execute 294 sec latency 103.536 ms 2024-04-24T23:32:50.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:49 smithi038 ceph-mon[32450]: pgmap v975: 129 pgs: 129 active+clean; 615 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 521 op/s 2024-04-24T23:32:50.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:49 smithi117 ceph-mon[32529]: pgmap v975: 129 pgs: 129 active+clean; 615 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 521 op/s 2024-04-24T23:32:50.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:49 smithi033 ceph-mon[29201]: pgmap v975: 129 pgs: 129 active+clean; 615 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 521 op/s 2024-04-24T23:32:50.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 177446 22.43 MB/sec execute 295 sec latency 82.373 ms 2024-04-24T23:32:51.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 177719 22.39 MB/sec execute 296 sec latency 105.745 ms 2024-04-24T23:32:52.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:51 smithi038 ceph-mon[32450]: pgmap v976: 129 pgs: 129 active+clean; 616 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 452 op/s 2024-04-24T23:32:52.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:51 smithi117 ceph-mon[32529]: pgmap v976: 129 pgs: 129 active+clean; 616 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 452 op/s 2024-04-24T23:32:52.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:51 smithi033 ceph-mon[29201]: pgmap v976: 129 pgs: 129 active+clean; 616 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 452 op/s 2024-04-24T23:32:52.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 177877 22.34 MB/sec execute 297 sec latency 261.594 ms 2024-04-24T23:32:53.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 178273 22.33 MB/sec execute 298 sec latency 145.638 ms 2024-04-24T23:32:54.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:53 smithi038 ceph-mon[32450]: pgmap v977: 129 pgs: 129 active+clean; 618 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 411 op/s 2024-04-24T23:32:54.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:53 smithi117 ceph-mon[32529]: pgmap v977: 129 pgs: 129 active+clean; 618 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 411 op/s 2024-04-24T23:32:54.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:53 smithi033 ceph-mon[29201]: pgmap v977: 129 pgs: 129 active+clean; 618 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 411 op/s 2024-04-24T23:32:54.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 178969 22.38 MB/sec execute 299 sec latency 82.660 ms 2024-04-24T23:32:55.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 179485 22.39 MB/sec execute 300 sec latency 54.407 ms 2024-04-24T23:32:56.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:55 smithi038 ceph-mon[32450]: pgmap v978: 129 pgs: 129 active+clean; 621 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 481 op/s 2024-04-24T23:32:56.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:55 smithi117 ceph-mon[32529]: pgmap v978: 129 pgs: 129 active+clean; 621 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 481 op/s 2024-04-24T23:32:56.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:55 smithi033 ceph-mon[29201]: pgmap v978: 129 pgs: 129 active+clean; 621 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.7 MiB/s wr, 481 op/s 2024-04-24T23:32:56.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 179921 22.39 MB/sec execute 301 sec latency 98.275 ms 2024-04-24T23:32:57.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 180141 22.37 MB/sec execute 302 sec latency 363.824 ms 2024-04-24T23:32:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:57 smithi038 ceph-mon[32450]: pgmap v979: 129 pgs: 129 active+clean; 626 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-04-24T23:32:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:57 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:32:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:57 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:32:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:57 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:32:58.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:57 smithi117 ceph-mon[32529]: pgmap v979: 129 pgs: 129 active+clean; 626 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-04-24T23:32:58.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:57 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:32:58.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:57 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:32:58.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:57 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:32:58.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:57 smithi033 ceph-mon[29201]: pgmap v979: 129 pgs: 129 active+clean; 626 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-04-24T23:32:58.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:57 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:32:58.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:57 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:32:58.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:57 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:32:58.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 180628 22.38 MB/sec execute 303 sec latency 128.081 ms 2024-04-24T23:32:59.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 180981 22.38 MB/sec execute 304 sec latency 101.649 ms 2024-04-24T23:33:00.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:32:59 smithi038 ceph-mon[32450]: pgmap v980: 129 pgs: 129 active+clean; 611 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.2 MiB/s wr, 519 op/s 2024-04-24T23:33:00.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:32:59 smithi117 ceph-mon[32529]: pgmap v980: 129 pgs: 129 active+clean; 611 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.2 MiB/s wr, 519 op/s 2024-04-24T23:33:00.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:32:59 smithi033 ceph-mon[29201]: pgmap v980: 129 pgs: 129 active+clean; 611 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.2 MiB/s wr, 519 op/s 2024-04-24T23:33:00.895 INFO:tasks.workunit.client.0.smithi033.stdout: 10 181459 22.38 MB/sec execute 305 sec latency 84.509 ms 2024-04-24T23:33:01.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 181889 22.39 MB/sec execute 306 sec latency 141.088 ms 2024-04-24T23:33:02.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:01 smithi038 ceph-mon[32450]: pgmap v981: 129 pgs: 129 active+clean; 608 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 342 op/s 2024-04-24T23:33:02.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:01 smithi117 ceph-mon[32529]: pgmap v981: 129 pgs: 129 active+clean; 608 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 342 op/s 2024-04-24T23:33:02.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:01 smithi033 ceph-mon[29201]: pgmap v981: 129 pgs: 129 active+clean; 608 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 342 op/s 2024-04-24T23:33:02.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 182132 22.36 MB/sec execute 307 sec latency 316.882 ms 2024-04-24T23:33:03.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 182622 22.36 MB/sec execute 308 sec latency 96.903 ms 2024-04-24T23:33:04.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:03 smithi038 ceph-mon[32450]: pgmap v982: 129 pgs: 129 active+clean; 608 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.8 MiB/s wr, 356 op/s 2024-04-24T23:33:04.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:03 smithi117 ceph-mon[32529]: pgmap v982: 129 pgs: 129 active+clean; 608 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.8 MiB/s wr, 356 op/s 2024-04-24T23:33:04.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:03 smithi033 ceph-mon[29201]: pgmap v982: 129 pgs: 129 active+clean; 608 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.8 MiB/s wr, 356 op/s 2024-04-24T23:33:04.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 183039 22.37 MB/sec execute 309 sec latency 103.826 ms 2024-04-24T23:33:05.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 183480 22.35 MB/sec execute 310 sec latency 70.467 ms 2024-04-24T23:33:06.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:05 smithi038 ceph-mon[32450]: pgmap v983: 129 pgs: 129 active+clean; 613 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.6 MiB/s wr, 468 op/s 2024-04-24T23:33:06.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:05 smithi117 ceph-mon[32529]: pgmap v983: 129 pgs: 129 active+clean; 613 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.6 MiB/s wr, 468 op/s 2024-04-24T23:33:06.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:05 smithi033 ceph-mon[29201]: pgmap v983: 129 pgs: 129 active+clean; 613 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.6 MiB/s wr, 468 op/s 2024-04-24T23:33:06.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 183948 22.38 MB/sec execute 311 sec latency 75.542 ms 2024-04-24T23:33:07.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 184391 22.38 MB/sec execute 312 sec latency 312.959 ms 2024-04-24T23:33:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:07 smithi038 ceph-mon[32450]: pgmap v984: 129 pgs: 129 active+clean; 613 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:33:08.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:07 smithi117 ceph-mon[32529]: pgmap v984: 129 pgs: 129 active+clean; 613 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:33:08.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:07 smithi033 ceph-mon[29201]: pgmap v984: 129 pgs: 129 active+clean; 613 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:33:08.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 184858 22.40 MB/sec execute 313 sec latency 357.052 ms 2024-04-24T23:33:09.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 185178 22.41 MB/sec execute 314 sec latency 108.961 ms 2024-04-24T23:33:10.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:09 smithi038 ceph-mon[32450]: pgmap v985: 129 pgs: 129 active+clean; 619 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 433 op/s 2024-04-24T23:33:10.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:09 smithi117 ceph-mon[32529]: pgmap v985: 129 pgs: 129 active+clean; 619 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 433 op/s 2024-04-24T23:33:10.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:09 smithi033 ceph-mon[29201]: pgmap v985: 129 pgs: 129 active+clean; 619 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 433 op/s 2024-04-24T23:33:10.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 185469 22.37 MB/sec execute 315 sec latency 91.638 ms 2024-04-24T23:33:11.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 185802 22.34 MB/sec execute 316 sec latency 94.103 ms 2024-04-24T23:33:12.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:11 smithi038 ceph-mon[32450]: pgmap v986: 129 pgs: 129 active+clean; 620 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-04-24T23:33:12.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:11 smithi117 ceph-mon[32529]: pgmap v986: 129 pgs: 129 active+clean; 620 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-04-24T23:33:12.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:11 smithi033 ceph-mon[29201]: pgmap v986: 129 pgs: 129 active+clean; 620 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-04-24T23:33:12.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 186131 22.32 MB/sec execute 317 sec latency 208.549 ms 2024-04-24T23:33:13.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 186611 22.33 MB/sec execute 318 sec latency 220.063 ms 2024-04-24T23:33:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:13 smithi038 ceph-mon[32450]: pgmap v987: 129 pgs: 129 active+clean; 620 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:33:14.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:13 smithi117 ceph-mon[32529]: pgmap v987: 129 pgs: 129 active+clean; 620 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:33:14.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:13 smithi033 ceph-mon[29201]: pgmap v987: 129 pgs: 129 active+clean; 620 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 367 op/s 2024-04-24T23:33:14.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 187062 22.34 MB/sec execute 319 sec latency 91.871 ms 2024-04-24T23:33:14.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:33:14.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/3546968254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08101870 msgr2=0x7f9a08101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/3546968254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08101870 0x7f9a08101cf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009900 tx=0x7f99f402a410 comp rx=0 tx=0).stop 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/3546968254 shutdown_connections 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/3546968254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a08102230 0x7f9a08108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/3546968254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08101870 0x7f9a08101cf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/3546968254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08100680 0x7f9a08100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/3546968254 >> 172.21.15.33:0/3546968254 conn(0x7f9a080fbbf0 msgr2=0x7f9a080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/3546968254 shutdown_connections 2024-04-24T23:33:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.039+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/3546968254 wait complete. 2024-04-24T23:33:15.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 Processor -- start 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 -- start start 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 0x7f9a08199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08101870 0x7f9a0819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a08102230 0x7f9a081a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810b200 con 0x7f9a08101870 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810b080 con 0x7f9a08100680 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0eb9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810b380 con 0x7f9a08102230 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 0x7f9a08199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08101870 0x7f9a0819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 0x7f9a08199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44510/0 (socket says 172.21.15.33:44510) 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0c939640 1 -- 172.21.15.33:0/145096687 learned_addr learned my addr 172.21.15.33:0/145096687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0d13a640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a08102230 0x7f9a081a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a07fff640 1 -- 172.21.15.33:0/145096687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a08102230 msgr2=0x7f9a081a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a07fff640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a08102230 0x7f9a081a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a07fff640 1 -- 172.21.15.33:0/145096687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 msgr2=0x7f9a08199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a07fff640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 0x7f9a08199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a07fff640 1 -- 172.21.15.33:0/145096687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a081a18f0 con 0x7f9a08101870 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.040+0000 7f9a0c939640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 0x7f9a08199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.041+0000 7f9a07fff640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08101870 0x7f9a0819a1f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f99f4031440 tx=0x7f99f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.041+0000 7f9a05ffb640 1 -- 172.21.15.33:0/145096687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f403f070 con 0x7f9a08101870 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.041+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a081a1b80 con 0x7f9a08101870 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.041+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081a2090 con 0x7f9a08101870 2024-04-24T23:33:15.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.041+0000 7f9a05ffb640 1 -- 172.21.15.33:0/145096687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f4003cd0 con 0x7f9a08101870 2024-04-24T23:33:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.041+0000 7f9a05ffb640 1 -- 172.21.15.33:0/145096687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4048060 con 0x7f9a08101870 2024-04-24T23:33:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.044+0000 7f9a05ffb640 1 -- 172.21.15.33:0/145096687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f99f403a3c0 con 0x7f9a08101870 2024-04-24T23:33:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.044+0000 7f9a11981740 1 Processor -- start 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.044+0000 7f9a11981740 1 -- start start 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a11981740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a11981740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a11981740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a11981740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e0311a6b0 con 0x557e031bbda0 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a05ffb640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99f80a2b90 0x7f99f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a11981740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e030c2880 con 0x557e031b6c30 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a11981740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557e03116670 con 0x557e031b6240 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0d13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44512/0 (socket says 172.21.15.33:44512) 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 -- 172.21.15.33:0/3417343486 learned_addr learned my addr 172.21.15.33:0/3417343486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0d13a640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99f80a2b90 0x7f99f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a05ffb640 1 -- 172.21.15.33:0/145096687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f40ef810 con 0x7f9a08101870 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 -- 172.21.15.33:0/3417343486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 msgr2=0x557e031b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 -- 172.21.15.33:0/3417343486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 msgr2=0x557e031be230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031be230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.045+0000 7f9a0c939640 1 -- 172.21.15.33:0/3417343486 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e02ef87d0 con 0x557e031b6c30 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031bb730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0041d0 tx=0x7f99fc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d674d185c503265 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f9a0d13a640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99f80a2b90 0x7f99f80a5050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x557e031b7490 tx=0x7f9a00009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f99f2ffd640 1 -- 172.21.15.33:0/3417343486 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004440 con 0x557e031b6c30 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f99f2ffd640 1 -- 172.21.15.33:0/3417343486 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99fc005500 con 0x557e031b6c30 2024-04-24T23:33:15.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f99f2ffd640 1 -- 172.21.15.33:0/3417343486 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0166d0 con 0x557e031b6c30 2024-04-24T23:33:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f9a11981740 1 -- 172.21.15.33:0/3417343486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 msgr2=0x557e031bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.046+0000 7f9a11981740 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031bb730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0041d0 tx=0x7f99fc005f90 comp rx=0 tx=0).stop 2024-04-24T23:33:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 -- 172.21.15.33:0/3417343486 shutdown_connections 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031bb730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 --2- 172.21.15.33:0/3417343486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 -- 172.21.15.33:0/3417343486 >> 172.21.15.33:0/3417343486 conn(0x557e031b18b0 msgr2=0x557e031b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 -- 172.21.15.33:0/3417343486 shutdown_connections 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 -- 172.21.15.33:0/3417343486 wait complete. 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 -- start start 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 Processor -- start 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.047+0000 7f9a11981740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a11981740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031b6c30 0x557e031c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a11981740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031bbda0 0x557e031c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a11981740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e031bed50 con 0x557e031b6c30 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a11981740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e030d5f50 con 0x557e031b6240 2024-04-24T23:33:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a11981740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557e031beed0 con 0x557e031bbda0 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0d13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031bbda0 0x557e031c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031b6c30 0x557e031c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:44526/0 (socket says 172.21.15.33:44526) 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 -- 172.21.15.33:0/3271287740 learned_addr learned my addr 172.21.15.33:0/3271287740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0d13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031bbda0 0x557e031c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54468/0 (socket says 172.21.15.33:54468) 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a11981740 10 client.0 _ll_register_callbacks cb 0x557e03111000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 -- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031bbda0 msgr2=0x557e031c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031bbda0 0x557e031c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 -- 172.21.15.33:0/3271287740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031b6c30 msgr2=0x557e031c0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031b6c30 0x557e031c0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 -- 172.21.15.33:0/3271287740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e02e2e3f0 con 0x557e031b6240 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031c06c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b800 tx=0x7f99fc00cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c64b87aff98832 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.048+0000 7f99e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f99e7fff640 1 -- 172.21.15.33:0/3271287740 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc005100 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f99e7fff640 1 -- 172.21.15.33:0/3271287740 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99fc0052a0 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc005500 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f99e7fff640 1 -- 172.21.15.33:0/3271287740 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f99fc031050 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 10 client.29252 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e02dee0f0 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f99e7fff640 1 -- 172.21.15.33:0/3271287740 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99fc005500 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f99e7fff640 1 -- 172.21.15.33:0/3271287740 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f99fc020680 con 0x557e031b6240 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 10 client.29252 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 10 client.29252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 20 client.29252 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e031cc220 0x557e031ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 4 client.29252 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.049+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e03127bf0 con 0x557e031cc220 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.050+0000 7f9a07fff640 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e031cc220 0x557e031ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.050+0000 7f99e7fff640 10 client.29252 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.050+0000 7f9a07fff640 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e031cc220 0x557e031ce610 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47991d960ca4247 server_cookie=bc0b4eb0a48c203e in_seq=0 out_seq=0 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f99e7fff640 1 -- 172.21.15.33:0/3271287740 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e03127bf0 con 0x557e031cc220 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f99e7fff640 10 client.29252 handle_command_reply: tid=0 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 client.29252 shutdown 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 msgr2=0x557e031c06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031c06c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b800 tx=0x7f99fc00cf20 comp rx=0 tx=0).stop 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 shutdown_connections 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e031cc220 0x557e031ce610 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031bbda0 0x557e031c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f99e7fff640 0 client.29252 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031b6c30 0x557e031c0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 --2- 172.21.15.33:0/3271287740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031c06c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 >> 172.21.15.33:0/3271287740 conn(0x557e031b18b0 msgr2=0x557e031b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 shutdown_connections 2024-04-24T23:33:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 1 -- 172.21.15.33:0/3271287740 wait complete. 2024-04-24T23:33:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.051+0000 7f9a11981740 20 client.29252 trim_cache size 0 max 16384 2024-04-24T23:33:15.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 Processor -- start 2024-04-24T23:33:15.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 -- start start 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6c30 0x557e031c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081aaa90 con 0x557e031bbda0 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a8360 con 0x557e031b6240 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a11981740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a61b0 con 0x557e031b6c30 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a0d13a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a07fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6c30 0x557e031c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a0d13a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031bfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52332/0 (socket says 172.21.15.33:52332) 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a0d13a640 1 -- 172.21.15.33:0/947646425 learned_addr learned my addr 172.21.15.33:0/947646425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.078+0000 7f9a0c939640 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a07fff640 1 -- 172.21.15.33:0/947646425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 msgr2=0x557e031b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a07fff640 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a07fff640 1 -- 172.21.15.33:0/947646425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 msgr2=0x557e031bfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a07fff640 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031bfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a07fff640 1 -- 172.21.15.33:0/947646425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e031a3090 con 0x557e031b6c30 2024-04-24T23:33:15.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a0d13a640 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:15.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f9a07fff640 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6c30 0x557e031c7c00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99f40ecfd0 tx=0x7f99f4035be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6331b4d69be9b77f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f99f27fc640 1 -- 172.21.15.33:0/947646425 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4035030 con 0x557e031b6c30 2024-04-24T23:33:15.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f99f27fc640 1 -- 172.21.15.33:0/947646425 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f40bc3e0 con 0x557e031b6c30 2024-04-24T23:33:15.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.079+0000 7f99f27fc640 1 -- 172.21.15.33:0/947646425 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40b9fe0 con 0x557e031b6c30 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 -- 172.21.15.33:0/947646425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6c30 msgr2=0x557e031c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6c30 0x557e031c7c00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99f40ecfd0 tx=0x7f99f4035be0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 -- 172.21.15.33:0/947646425 shutdown_connections 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6c30 0x557e031c7c00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 --2- 172.21.15.33:0/947646425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6240 0x557e031b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 -- 172.21.15.33:0/947646425 >> 172.21.15.33:0/947646425 conn(0x557e031b18b0 msgr2=0x557e031b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 -- 172.21.15.33:0/947646425 shutdown_connections 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.080+0000 7f9a11981740 1 -- 172.21.15.33:0/947646425 wait complete. 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.081+0000 7f9a11981740 1 -- start start 2024-04-24T23:33:15.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.081+0000 7f9a11981740 1 Processor -- start 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a6820 con 0x557e031bbda0 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a0c939640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54486/0 (socket says 172.21.15.33:54486) 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a0c939640 1 -- 172.21.15.33:0/3244997649 learned_addr learned my addr 172.21.15.33:0/3244997649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a7c90 con 0x557e031b6c30 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a6ef0 con 0x557e031b6240 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a07fff640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a11981740 10 client.0 _ll_register_callbacks cb 0x7f9a00009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a0d13a640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a07fff640 1 -- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 msgr2=0x557e031c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a07fff640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a07fff640 1 -- 172.21.15.33:0/3244997649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 msgr2=0x557e031c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.082+0000 7f9a07fff640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f9a07fff640 1 -- 172.21.15.33:0/3244997649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e031c6d60 con 0x557e031b6c30 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f9a07fff640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031c5760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f99f40bbea0 tx=0x7f99f403b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a442f2c7c02497 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f99e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f99e7fff640 1 -- 172.21.15.33:0/3244997649 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4035070 con 0x557e031b6c30 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f99e7fff640 1 -- 172.21.15.33:0/3244997649 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f40b8350 con 0x557e031b6c30 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f40bc3e0 con 0x557e031b6c30 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f99e7fff640 1 -- 172.21.15.33:0/3244997649 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f99f403b460 con 0x557e031b6c30 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f9a11981740 10 client.29258 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.083+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e03232700 con 0x557e031b6c30 2024-04-24T23:33:15.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f99e7fff640 1 -- 172.21.15.33:0/3244997649 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f403b600 con 0x557e031b6c30 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f99e7fff640 1 -- 172.21.15.33:0/3244997649 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f99f404b060 con 0x557e031b6c30 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a11981740 10 client.29258 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a11981740 10 client.29258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a11981740 20 client.29258 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a11981740 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e03232f20 0x557e03235340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a11981740 4 client.29258 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f99f40bc3e0 con 0x557e03232f20 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.084+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e03232f20 0x557e03235340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f9a0c939640 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e03232f20 0x557e03235340 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7230e79c08a98478 server_cookie=2ff0a48cffc46c3b in_seq=0 out_seq=0 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f99e7fff640 10 client.29258 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f99e7fff640 1 -- 172.21.15.33:0/3244997649 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f99f40bc3e0 con 0x557e03232f20 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f99e7fff640 10 client.29258 handle_command_reply: tid=0 2024-04-24T23:33:15.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f9a11981740 1 client.29258 shutdown 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 msgr2=0x557e031c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.085+0000 7f9a11981740 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031c5760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f99f40bbea0 tx=0x7f99f403b800 comp rx=0 tx=0).stop 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 shutdown_connections 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557e03232f20 0x557e03235340 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f99e7fff640 0 client.29258 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e031bbda0 0x557e031c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e031b6c30 0x557e031c5760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 --2- 172.21.15.33:0/3244997649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e031b6240 0x557e031c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 >> 172.21.15.33:0/3244997649 conn(0x557e031b18b0 msgr2=0x557e031b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 shutdown_connections 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 1 -- 172.21.15.33:0/3244997649 wait complete. 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.086+0000 7f9a11981740 20 client.29258 trim_cache size 0 max 16384 2024-04-24T23:33:15.088 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:33:15.089 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:33:15.089 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:33:15.089 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:33:15.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99f80a2b90 msgr2=0x7f99f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99f80a2b90 0x7f99f80a5050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x557e031b7490 tx=0x7f9a00009500 comp rx=0 tx=0).stop 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08101870 msgr2=0x7f9a0819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08101870 0x7f9a0819a1f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f99f4031440 tx=0x7f99f4004270 comp rx=0 tx=0).stop 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 shutdown_connections 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f99f80a2b90 0x7f99f80a5050 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a08102230 0x7f9a081a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a08101870 0x7f9a0819a1f0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 --2- 172.21.15.33:0/145096687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a08100680 0x7f9a08199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 >> 172.21.15.33:0/145096687 conn(0x7f9a080fbbf0 msgr2=0x7f9a08104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 shutdown_connections 2024-04-24T23:33:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.087+0000 7f9a0eb9d640 1 -- 172.21.15.33:0/145096687 wait complete. 2024-04-24T23:33:15.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:33:15.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:33:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 -- 172.21.15.33:0/1876157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530100680 msgr2=0x7fd530100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 --2- 172.21.15.33:0/1876157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530100680 0x7fd530100aa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd520009900 tx=0x7fd52002a410 comp rx=0 tx=0).stop 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 -- 172.21.15.33:0/1876157867 shutdown_connections 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 --2- 172.21.15.33:0/1876157867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530102230 0x7fd530108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 --2- 172.21.15.33:0/1876157867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd530101870 0x7fd530101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 --2- 172.21.15.33:0/1876157867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530100680 0x7fd530100aa0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 -- 172.21.15.33:0/1876157867 >> 172.21.15.33:0/1876157867 conn(0x7fd5300fbbf0 msgr2=0x7fd5300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 -- 172.21.15.33:0/1876157867 shutdown_connections 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.217+0000 7fd53584d640 1 -- 172.21.15.33:0/1876157867 wait complete. 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 Processor -- start 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 -- start start 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530100680 0x7fd530197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 0x7fd530197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd530102230 0x7fd53019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010c300 con 0x7fd530101870 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010c180 con 0x7fd530102230 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd53584d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010c480 con 0x7fd530100680 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 0x7fd530197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 0x7fd530197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52356/0 (socket says 172.21.15.33:52356) 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 -- 172.21.15.33:0/679604133 learned_addr learned my addr 172.21.15.33:0/679604133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd52f7fe640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd530102230 0x7fd53019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd52effd640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530100680 0x7fd530197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 -- 172.21.15.33:0/679604133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530100680 msgr2=0x7fd530197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530100680 0x7fd530197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 -- 172.21.15.33:0/679604133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd530102230 msgr2=0x7fd53019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd530102230 0x7fd53019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 -- 172.21.15.33:0/679604133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53019f6c0 con 0x7fd530101870 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.218+0000 7fd527fff640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 0x7fd530197fe0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e9f0 tx=0x7fd51800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.219+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51800cd80 con 0x7fd530101870 2024-04-24T23:33:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.219+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53019f9b0 con 0x7fd530101870 2024-04-24T23:33:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.219+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd518010910 con 0x7fd530101870 2024-04-24T23:33:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.219+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518018990 con 0x7fd530101870 2024-04-24T23:33:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.219+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53019ff00 con 0x7fd530101870 2024-04-24T23:33:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.220+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5301a0130 con 0x7fd530101870 2024-04-24T23:33:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.221+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd518010ac0 con 0x7fd530101870 2024-04-24T23:33:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.222+0000 7fd53484b640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd51c0a2b90 0x7fd51c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.222+0000 7fd52effd640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd51c0a2b90 0x7fd51c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.222+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd5180cf740 con 0x7fd530101870 2024-04-24T23:33:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.222+0000 7fd52effd640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd51c0a2b90 0x7fd51c0a5050 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd520005ab0 tx=0x7fd520000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.226+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd51809bc60 con 0x7fd530101870 2024-04-24T23:33:15.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.500+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5301a0300 con 0x7fd530101870 2024-04-24T23:33:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.501+0000 7fd53484b640 1 -- 172.21.15.33:0/679604133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7fd51809ff60 con 0x7fd530101870 2024-04-24T23:33:15.500 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:33:15.501 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:33:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd51c0a2b90 msgr2=0x7fd51c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd51c0a2b90 0x7fd51c0a5050 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd520005ab0 tx=0x7fd520000bc0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 msgr2=0x7fd530197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 0x7fd530197fe0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e9f0 tx=0x7fd51800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 shutdown_connections 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd51c0a2b90 0x7fd51c0a5050 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd530102230 0x7fd53019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd530101870 0x7fd530197fe0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 --2- 172.21.15.33:0/679604133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530100680 0x7fd530197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 >> 172.21.15.33:0/679604133 conn(0x7fd5300fbbf0 msgr2=0x7fd5300fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 shutdown_connections 2024-04-24T23:33:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.503+0000 7fd53584d640 1 -- 172.21.15.33:0/679604133 wait complete. 2024-04-24T23:33:15.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:33:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 -- 172.21.15.33:0/4125355065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc924101c90 msgr2=0x7fc9241020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 --2- 172.21.15.33:0/4125355065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc924101c90 0x7fc9241020b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc914009860 tx=0x7fc91402a340 comp rx=0 tx=0).stop 2024-04-24T23:33:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 -- 172.21.15.33:0/4125355065 shutdown_connections 2024-04-24T23:33:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 --2- 172.21.15.33:0/4125355065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241012a0 0x7fc924108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 --2- 172.21.15.33:0/4125355065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9241008e0 0x7fc924100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 --2- 172.21.15.33:0/4125355065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc924101c90 0x7fc9241020b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 -- 172.21.15.33:0/4125355065 >> 172.21.15.33:0/4125355065 conn(0x7fc9240fb370 msgr2=0x7fc9240fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 -- 172.21.15.33:0/4125355065 shutdown_connections 2024-04-24T23:33:15.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.640+0000 7fc92964c640 1 -- 172.21.15.33:0/4125355065 wait complete. 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.641+0000 7fc92964c640 1 Processor -- start 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.641+0000 7fc92964c640 1 -- start start 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc92964c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 0x7fc924199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc92964c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9241012a0 0x7fc92419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc92964c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc924101c90 0x7fc9241a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 0x7fc924199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc924101c90 0x7fc9241a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 0x7fc924199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:54504/0 (socket says 172.21.15.33:54504) 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc92964c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410a650 con 0x7fc924101c90 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc92964c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410a4d0 con 0x7fc9241012a0 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc92964c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410a7d0 con 0x7fc9241008e0 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 -- 172.21.15.33:0/4227433426 learned_addr learned my addr 172.21.15.33:0/4227433426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 -- 172.21.15.33:0/4227433426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9241012a0 msgr2=0x7fc92419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9241012a0 0x7fc92419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 -- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc924101c90 msgr2=0x7fc9241a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.642+0000 7fc922ffd640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc924101c90 0x7fc9241a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.643+0000 7fc922ffd640 1 -- 172.21.15.33:0/4227433426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9241a1930 con 0x7fc9241008e0 2024-04-24T23:33:15.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.643+0000 7fc9237fe640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc924101c90 0x7fc9241a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:15.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.644+0000 7fc922ffd640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 0x7fc924199d20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc9140305b0 tx=0x7fc914033b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.644+0000 7fc90bfff640 1 -- 172.21.15.33:0/4227433426 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91403f020 con 0x7fc9241008e0 2024-04-24T23:33:15.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.644+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9241a1bc0 con 0x7fc9241008e0 2024-04-24T23:33:15.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.644+0000 7fc90bfff640 1 -- 172.21.15.33:0/4227433426 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc914033e30 con 0x7fc9241008e0 2024-04-24T23:33:15.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.645+0000 7fc90bfff640 1 -- 172.21.15.33:0/4227433426 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914037040 con 0x7fc9241008e0 2024-04-24T23:33:15.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.646+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9241a2040 con 0x7fc9241008e0 2024-04-24T23:33:15.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.649+0000 7fc92c430740 1 Processor -- start 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.649+0000 7fc90bfff640 1 -- 172.21.15.33:0/4227433426 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc914042020 con 0x7fc9241008e0 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.649+0000 7fc92c430740 1 -- start start 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc92c430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc92c430740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a23c30 0x561432a28730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc92c430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a2b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc92c430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561432976670 con 0x561432a23240 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc92c430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56143291e970 con 0x561432a28da0 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc92c430740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561432972630 con 0x561432a23c30 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc90bfff640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc90c0a2c10 0x7fc90c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.650+0000 7fc90bfff640 1 -- 172.21.15.33:0/4227433426 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc9140eeef0 con 0x7fc9241008e0 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.651+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a23c30 0x561432a28730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.651+0000 7fc9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.651+0000 7fc9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a23660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52392/0 (socket says 172.21.15.33:52392) 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.651+0000 7fc9237fe640 1 -- 172.21.15.33:0/885251597 learned_addr learned my addr 172.21.15.33:0/885251597 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.651+0000 7fc9237fe640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc90c0a2c10 0x7fc90c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc922ffd640 1 -- 172.21.15.33:0/885251597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 msgr2=0x561432a2b230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc922ffd640 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a2b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc922ffd640 1 -- 172.21.15.33:0/885251597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 msgr2=0x561432a23660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc922ffd640 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a23660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc922ffd640 1 -- 172.21.15.33:0/885251597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561432942400 con 0x561432a23c30 2024-04-24T23:33:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc922ffd640 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a23c30 0x561432a28730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc914009830 tx=0x7fc9140036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3bb5bb7744cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc9097fa640 1 -- 172.21.15.33:0/885251597 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9140bba60 con 0x561432a23c30 2024-04-24T23:33:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.653+0000 7fc9097fa640 1 -- 172.21.15.33:0/885251597 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9140bb360 con 0x561432a23c30 2024-04-24T23:33:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.654+0000 7fc9097fa640 1 -- 172.21.15.33:0/885251597 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9140b7ad0 con 0x561432a23c30 2024-04-24T23:33:15.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 -- 172.21.15.33:0/885251597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a23c30 msgr2=0x561432a28730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a23c30 0x561432a28730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc914009830 tx=0x7fc9140036f0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc9237fe640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc90c0a2c10 0x7fc90c0a50d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009fd0 tx=0x7fc91c0134c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 -- 172.21.15.33:0/885251597 shutdown_connections 2024-04-24T23:33:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a2b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a23c30 0x561432a28730 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc914009830 tx=0x7fc9140036f0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 --2- 172.21.15.33:0/885251597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.655+0000 7fc92c430740 1 -- 172.21.15.33:0/885251597 >> 172.21.15.33:0/885251597 conn(0x561432a0d8a0 msgr2=0x561432a0dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.656+0000 7fc92c430740 1 -- 172.21.15.33:0/885251597 shutdown_connections 2024-04-24T23:33:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.656+0000 7fc92c430740 1 -- 172.21.15.33:0/885251597 wait complete. 2024-04-24T23:33:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.656+0000 7fc92c430740 1 -- start start 2024-04-24T23:33:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.656+0000 7fc92c430740 1 Processor -- start 2024-04-24T23:33:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a1bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a1c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561432a11eb0 con 0x561432a23240 2024-04-24T23:33:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561432932050 con 0x561432a28da0 2024-04-24T23:33:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561432a12030 con 0x561432a21540 2024-04-24T23:33:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc92c430740 10 client.0 _ll_register_callbacks cb 0x561432689be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.657+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a1bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a1bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48624/0 (socket says 172.21.15.33:48624) 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc922ffd640 1 -- 172.21.15.33:0/852177560 learned_addr learned my addr 172.21.15.33:0/852177560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9227fc640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a1c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9237fe640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9227fc640 1 -- 172.21.15.33:0/852177560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 msgr2=0x561432a219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9227fc640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a219c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9227fc640 1 -- 172.21.15.33:0/852177560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 msgr2=0x561432a1bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9227fc640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a1bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.658+0000 7fc9227fc640 1 -- 172.21.15.33:0/852177560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561432903dc0 con 0x561432a28da0 2024-04-24T23:33:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.659+0000 7fc9227fc640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a1c510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc9240688a0 tx=0x7fc91800e770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1d62ab49fdbd92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.659+0000 7fc906ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.659+0000 7fc906ff5640 1 -- 172.21.15.33:0/852177560 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918010840 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.659+0000 7fc906ff5640 1 -- 172.21.15.33:0/852177560 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc9180109e0 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.659+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9140bb360 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.659+0000 7fc906ff5640 1 -- 172.21.15.33:0/852177560 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc918010be0 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 10 client.29270 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56143268a3f0 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc906ff5640 1 -- 172.21.15.33:0/852177560 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9140bb360 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc906ff5640 1 -- 172.21.15.33:0/852177560 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc91800f3c0 con 0x561432a28da0 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 10 client.29270 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 10 client.29270 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 20 client.29270 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a31250 0x561432a33690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 4 client.29270 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.660+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561432983bb0 con 0x561432a31250 2024-04-24T23:33:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.663+0000 7fc922ffd640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a31250 0x561432a33690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.664+0000 7fc922ffd640 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a31250 0x561432a33690 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=361260f988f06f62 server_cookie=f8d20be4e8d1708f in_seq=0 out_seq=0 2024-04-24T23:33:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.664+0000 7fc906ff5640 10 client.29270 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.664+0000 7fc906ff5640 1 -- 172.21.15.33:0/852177560 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561432983bb0 con 0x561432a31250 2024-04-24T23:33:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.664+0000 7fc906ff5640 10 client.29270 handle_command_reply: tid=0 2024-04-24T23:33:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.664+0000 7fc92c430740 1 client.29270 shutdown 2024-04-24T23:33:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 msgr2=0x561432a1c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a1c510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc9240688a0 tx=0x7fc91800e770 comp rx=0 tx=0).stop 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 shutdown_connections 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a31250 0x561432a33690 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a1c510 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 --2- 172.21.15.33:0/852177560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a1bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.665+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 >> 172.21.15.33:0/852177560 conn(0x561432a0d8a0 msgr2=0x561432a0cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.666+0000 7fc906ff5640 0 client.29270 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.666+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 shutdown_connections 2024-04-24T23:33:15.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.666+0000 7fc92c430740 1 -- 172.21.15.33:0/852177560 wait complete. 2024-04-24T23:33:15.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.666+0000 7fc92c430740 20 client.29270 trim_cache size 0 max 16384 2024-04-24T23:33:15.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.693+0000 7fc92c430740 1 Processor -- start 2024-04-24T23:33:15.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.693+0000 7fc92c430740 1 -- start start 2024-04-24T23:33:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc92c430740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a0c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc92c430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc92c430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a24d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a0c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a0c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35182/0 (socket says 172.21.15.33:35182) 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc922ffd640 1 -- 172.21.15.33:0/601223742 learned_addr learned my addr 172.21.15.33:0/601223742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc9227fc640 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc9237fe640 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a24d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc92c430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241a23b0 con 0x561432a23240 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240713e0 con 0x561432a28da0 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.694+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc92406f230 con 0x561432a21540 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.695+0000 7fc9227fc640 1 -- 172.21.15.33:0/601223742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 msgr2=0x561432a0c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.695+0000 7fc9227fc640 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a0c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.695+0000 7fc9227fc640 1 -- 172.21.15.33:0/601223742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 msgr2=0x561432a24d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.695+0000 7fc9227fc640 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a24d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.695+0000 7fc9227fc640 1 -- 172.21.15.33:0/601223742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614327547d0 con 0x561432a23240 2024-04-24T23:33:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.695+0000 7fc9227fc640 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a247d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fc91800f9d0 tx=0x7fc918005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eba0e489719ea76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.696+0000 7fc908ff9640 1 -- 172.21.15.33:0/601223742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91802e030 con 0x561432a23240 2024-04-24T23:33:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.696+0000 7fc908ff9640 1 -- 172.21.15.33:0/601223742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91800fa50 con 0x561432a23240 2024-04-24T23:33:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.696+0000 7fc908ff9640 1 -- 172.21.15.33:0/601223742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918004a00 con 0x561432a23240 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 msgr2=0x561432a247d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a247d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fc91800f9d0 tx=0x7fc918005d50 comp rx=0 tx=0).stop 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 shutdown_connections 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a28da0 0x561432a24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a23240 0x561432a247d0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 --2- 172.21.15.33:0/601223742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a21540 0x561432a0c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 >> 172.21.15.33:0/601223742 conn(0x561432a0d8a0 msgr2=0x561432a0c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.697+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 shutdown_connections 2024-04-24T23:33:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.698+0000 7fc92c430740 1 -- 172.21.15.33:0/601223742 wait complete. 2024-04-24T23:33:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.698+0000 7fc92c430740 1 -- start start 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.698+0000 7fc92c430740 1 Processor -- start 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.698+0000 7fc92c430740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 0x561432a42020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.699+0000 7fc92c430740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a23240 0x561432a42560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.699+0000 7fc92c430740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a28da0 0x561432a43570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.699+0000 7fc92c430740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc92406f8a0 con 0x561432a21540 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.699+0000 7fc92c430740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc924070d10 con 0x561432a23240 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.699+0000 7fc92c430740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc92406ff70 con 0x561432a28da0 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.699+0000 7fc92c430740 10 client.0 _ll_register_callbacks cb 0x7fc918009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.700+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 0x561432a42020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.700+0000 7fc922ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 0x561432a42020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48640/0 (socket says 172.21.15.33:48640) 2024-04-24T23:33:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.700+0000 7fc922ffd640 1 -- 172.21.15.33:0/1517710498 learned_addr learned my addr 172.21.15.33:0/1517710498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc9237fe640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a28da0 0x561432a43570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc922ffd640 1 -- 172.21.15.33:0/1517710498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a28da0 msgr2=0x561432a43570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc922ffd640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a28da0 0x561432a43570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc9227fc640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a23240 0x561432a42560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc922ffd640 1 -- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a23240 msgr2=0x561432a42560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc922ffd640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a23240 0x561432a42560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.703+0000 7fc922ffd640 1 -- 172.21.15.33:0/1517710498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561432a43b60 con 0x561432a21540 2024-04-24T23:33:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.704+0000 7fc922ffd640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 0x561432a42020 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc9140b7730 tx=0x7fc914005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601ff310229f1d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.704+0000 7fc906ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.705+0000 7fc906ff5640 1 -- 172.21.15.33:0/1517710498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9140bba60 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.705+0000 7fc906ff5640 1 -- 172.21.15.33:0/1517710498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc914030850 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.705+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc91800fa50 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.705+0000 7fc906ff5640 1 -- 172.21.15.33:0/1517710498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc9140bf710 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.705+0000 7fc92c430740 10 client.19800 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.705+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561432a972b0 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc906ff5640 1 -- 172.21.15.33:0/1517710498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91800fa50 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc906ff5640 1 -- 172.21.15.33:0/1517710498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc9140c7030 con 0x561432a21540 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc92c430740 10 client.19800 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc92c430740 10 client.19800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc92c430740 20 client.19800 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc92c430740 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a97910 0x561432a99d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc92c430740 4 client.19800 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:33:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56143297df00 con 0x561432a97910 2024-04-24T23:33:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.706+0000 7fc9227fc640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a97910 0x561432a99d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.707+0000 7fc906ff5640 10 client.19800 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.707+0000 7fc9227fc640 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a97910 0x561432a99d30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adf053e542692ecd server_cookie=7ab1bd016b1a2d08 in_seq=0 out_seq=0 2024-04-24T23:33:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.709+0000 7fc906ff5640 1 -- 172.21.15.33:0/1517710498 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56143297df00 con 0x561432a97910 2024-04-24T23:33:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.709+0000 7fc906ff5640 10 client.19800 handle_command_reply: tid=0 2024-04-24T23:33:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.709+0000 7fc92c430740 1 client.19800 shutdown 2024-04-24T23:33:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 msgr2=0x561432a42020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 0x561432a42020 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc9140b7730 tx=0x7fc914005f90 comp rx=0 tx=0).stop 2024-04-24T23:33:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 shutdown_connections 2024-04-24T23:33:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x561432a97910 0x561432a99d30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc906ff5640 0 client.19800 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561432a28da0 0x561432a43570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561432a23240 0x561432a42560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 --2- 172.21.15.33:0/1517710498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561432a21540 0x561432a42020 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.710+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 >> 172.21.15.33:0/1517710498 conn(0x561432a0d8a0 msgr2=0x561432a0cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.711+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 shutdown_connections 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.711+0000 7fc92c430740 1 -- 172.21.15.33:0/1517710498 wait complete. 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.711+0000 7fc92c430740 20 client.19800 trim_cache size 0 max 16384 2024-04-24T23:33:15.711 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.711+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc90c0a2c10 msgr2=0x7fc90c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc90c0a2c10 0x7fc90c0a50d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009fd0 tx=0x7fc91c0134c0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 msgr2=0x7fc924199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 0x7fc924199d20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc9140305b0 tx=0x7fc914033b80 comp rx=0 tx=0).stop 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 shutdown_connections 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc90c0a2c10 0x7fc90c0a50d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc924101c90 0x7fc9241a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9241012a0 0x7fc92419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 --2- 172.21.15.33:0/4227433426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9241008e0 0x7fc924199d20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 >> 172.21.15.33:0/4227433426 conn(0x7fc9240fb370 msgr2=0x7fc9240fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 shutdown_connections 2024-04-24T23:33:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.712+0000 7fc92964c640 1 -- 172.21.15.33:0/4227433426 wait complete. 2024-04-24T23:33:15.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:33:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 -- 172.21.15.33:0/3410745666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 msgr2=0x7fdae8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 --2- 172.21.15.33:0/3410745666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae8100e10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009900 tx=0x7fdadc02a400 comp rx=0 tx=0).stop 2024-04-24T23:33:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 -- 172.21.15.33:0/3410745666 shutdown_connections 2024-04-24T23:33:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 --2- 172.21.15.33:0/3410745666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8101350 0x7fdae81017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 --2- 172.21.15.33:0/3410745666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae8100e10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 --2- 172.21.15.33:0/3410745666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8100050 0x7fdae8100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 -- 172.21.15.33:0/3410745666 >> 172.21.15.33:0/3410745666 conn(0x7fdae80fbc30 msgr2=0x7fdae80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 -- 172.21.15.33:0/3410745666 shutdown_connections 2024-04-24T23:33:15.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.845+0000 7fdaecb14640 1 -- 172.21.15.33:0/3410745666 wait complete. 2024-04-24T23:33:15.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.846+0000 7fdaecb14640 1 Processor -- start 2024-04-24T23:33:15.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.846+0000 7fdaecb14640 1 -- start start 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.846+0000 7fdaecb14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8100050 0x7fdae8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.846+0000 7fdaecb14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdaecb14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8101350 0x7fdae81a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48664/0 (socket says 172.21.15.33:48664) 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae5d9b640 1 -- 172.21.15.33:0/1227468674 learned_addr learned my addr 172.21.15.33:0/1227468674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810f670 con 0x7fdae81009b0 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae659c640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8100050 0x7fdae8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810f4f0 con 0x7fdae8101350 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810f7f0 con 0x7fdae8100050 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae6d9d640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8101350 0x7fdae81a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae659c640 1 -- 172.21.15.33:0/1227468674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8101350 msgr2=0x7fdae81a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae659c640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8101350 0x7fdae81a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae659c640 1 -- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 msgr2=0x7fdae819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae659c640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae659c640 1 -- 172.21.15.33:0/1227468674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae81a1960 con 0x7fdae8100050 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae6d9d640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8101350 0x7fdae81a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.847+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.848+0000 7fdae659c640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8100050 0x7fdae8199d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e9f0 tx=0x7fdad800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.848+0000 7fdad77fe640 1 -- 172.21.15.33:0/1227468674 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad800ccc0 con 0x7fdae8100050 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.848+0000 7fdad77fe640 1 -- 172.21.15.33:0/1227468674 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad8010910 con 0x7fdae8100050 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.848+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae81a1c50 con 0x7fdae8100050 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.848+0000 7fdad77fe640 1 -- 172.21.15.33:0/1227468674 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8018960 con 0x7fdae8100050 2024-04-24T23:33:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.848+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae81a2170 con 0x7fdae8100050 2024-04-24T23:33:15.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.850+0000 7fdad77fe640 1 -- 172.21.15.33:0/1227468674 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdad8018b00 con 0x7fdae8100050 2024-04-24T23:33:15.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.851+0000 7fdaef825740 1 Processor -- start 2024-04-24T23:33:15.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.851+0000 7fdaef825740 1 -- start start 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.851+0000 7fdad77fe640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdad00a2b90 0x7fdad00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.851+0000 7fdaef825740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7326650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdaef825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c732b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdaef825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c732e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdaef825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557c728a670 con 0x5557c7326c20 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdaef825740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557c7232970 con 0x5557c7326230 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdaef825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557c7286630 con 0x5557c732bd90 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdad77fe640 1 -- 172.21.15.33:0/1227468674 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdad80cf830 con 0x7fdae8100050 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae6d9d640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdad00a2b90 0x7fdad00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7326650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c732e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7326650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50152/0 (socket says 172.21.15.33:50152) 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 -- 172.21.15.33:0/2861483822 learned_addr learned my addr 172.21.15.33:0/2861483822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c732b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 -- 172.21.15.33:0/2861483822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 msgr2=0x5557c732e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c732e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 -- 172.21.15.33:0/2861483822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 msgr2=0x5557c732b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c732b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 -- 172.21.15.33:0/2861483822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c7256400 con 0x5557c7326230 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae659c640 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7326650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdad809bc90 tx=0x7fdad8010b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e9df82fafb4af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.852+0000 7fdae6d9d640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdad00a2b90 0x7fdad00a5050 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005c40 tx=0x7fdae0009e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.853+0000 7fdad4ff9640 1 -- 172.21.15.33:0/2861483822 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad809bd00 con 0x5557c7326230 2024-04-24T23:33:15.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.853+0000 7fdad4ff9640 1 -- 172.21.15.33:0/2861483822 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad80a00b0 con 0x5557c7326230 2024-04-24T23:33:15.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.853+0000 7fdad4ff9640 1 -- 172.21.15.33:0/2861483822 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad80504d0 con 0x5557c7326230 2024-04-24T23:33:15.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.853+0000 7fdaef825740 1 -- 172.21.15.33:0/2861483822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 msgr2=0x5557c7326650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.853+0000 7fdaef825740 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7326650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdad809bc90 tx=0x7fdad8010b80 comp rx=0 tx=0).stop 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- 172.21.15.33:0/2861483822 shutdown_connections 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c732e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c732b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 --2- 172.21.15.33:0/2861483822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7326650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- 172.21.15.33:0/2861483822 >> 172.21.15.33:0/2861483822 conn(0x5557c73218a0 msgr2=0x5557c7321ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- 172.21.15.33:0/2861483822 shutdown_connections 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- 172.21.15.33:0/2861483822 wait complete. 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- start start 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 Processor -- start 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7330410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c7330950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c7337990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557c732ed40 con 0x5557c7326c20 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557c7246050 con 0x5557c7326230 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdaef825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557c732eec0 con 0x5557c732bd90 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.854+0000 7fdae5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c7330950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c7330950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48680/0 (socket says 172.21.15.33:48680) 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae5d9b640 1 -- 172.21.15.33:0/4271232996 learned_addr learned my addr 172.21.15.33:0/4271232996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7330410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdaef825740 10 client.0 _ll_register_callbacks cb 0x5557c6f9dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 -- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 msgr2=0x5557c7337990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c7337990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 -- 172.21.15.33:0/4271232996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 msgr2=0x5557c7330950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c7330950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 -- 172.21.15.33:0/4271232996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c7217dc0 con 0x5557c7326230 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c7330950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdae659c640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7330410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5557c7327480 tx=0x7fdad804b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9200efaeb079e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdacdffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdacdffb640 1 -- 172.21.15.33:0/4271232996 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad809bd00 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdacdffb640 1 -- 172.21.15.33:0/4271232996 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdad809a000 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad80a00b0 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdacdffb640 1 -- 172.21.15.33:0/4271232996 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdad80a4970 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.855+0000 7fdaef825740 10 client.29285 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5557c6f9e3f0 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdacdffb640 1 -- 172.21.15.33:0/4271232996 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad80a00b0 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdacdffb640 1 -- 172.21.15.33:0/4271232996 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdad8097a50 con 0x5557c7326230 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 10 client.29285 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 10 client.29285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 20 client.29285 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c733be60 0x5557c733e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 4 client.29285 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c7297bb0 con 0x5557c733be60 2024-04-24T23:33:15.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.856+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c733be60 0x5557c733e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.857+0000 7fdacdffb640 10 client.29285 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.857+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c733be60 0x5557c733e250 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8a83c3137722989 server_cookie=2cc0bc8c90331751 in_seq=0 out_seq=0 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.857+0000 7fdacdffb640 1 -- 172.21.15.33:0/4271232996 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557c7297bb0 con 0x5557c733be60 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.857+0000 7fdacdffb640 10 client.29285 handle_command_reply: tid=0 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.857+0000 7fdaef825740 1 client.29285 shutdown 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 msgr2=0x5557c7330410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7330410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5557c7327480 tx=0x7fdad804b660 comp rx=0 tx=0).stop 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 shutdown_connections 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c733be60 0x5557c733e250 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c732bd90 0x5557c7337990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326c20 0x5557c7330950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 --2- 172.21.15.33:0/4271232996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c7326230 0x5557c7330410 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdacdffb640 0 client.29285 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 >> 172.21.15.33:0/4271232996 conn(0x5557c73218a0 msgr2=0x5557c7320b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 shutdown_connections 2024-04-24T23:33:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 1 -- 172.21.15.33:0/4271232996 wait complete. 2024-04-24T23:33:15.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.858+0000 7fdaef825740 20 client.29285 trim_cache size 0 max 16384 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 Processor -- start 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 -- start start 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c73308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7337950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81aee50 con 0x5557c7326230 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.872+0000 7fdaef825740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81ac720 con 0x5557c732bd90 2024-04-24T23:33:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdaef825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81aa570 con 0x5557c7326c20 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c73308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c73308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48696/0 (socket says 172.21.15.33:48696) 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 -- 172.21.15.33:0/1768237645 learned_addr learned my addr 172.21.15.33:0/1768237645 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae6d9d640 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7337950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 -- 172.21.15.33:0/1768237645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 msgr2=0x5557c7337950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7337950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 -- 172.21.15.33:0/1768237645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 msgr2=0x5557c73209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 -- 172.21.15.33:0/1768237645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c70687d0 con 0x5557c7326230 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7337950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdae659c640 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c73308b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fdad809ba00 tx=0x7fdad80980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687b316e3de94736 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdacffff640 1 -- 172.21.15.33:0/1768237645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad80b8070 con 0x5557c7326230 2024-04-24T23:33:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdacffff640 1 -- 172.21.15.33:0/1768237645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad80a00b0 con 0x5557c7326230 2024-04-24T23:33:15.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.873+0000 7fdacffff640 1 -- 172.21.15.33:0/1768237645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad809b150 con 0x5557c7326230 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 -- 172.21.15.33:0/1768237645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 msgr2=0x5557c73308b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c73308b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fdad809ba00 tx=0x7fdad80980d0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 -- 172.21.15.33:0/1768237645 shutdown_connections 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7337950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 --2- 172.21.15.33:0/1768237645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c73308b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 -- 172.21.15.33:0/1768237645 >> 172.21.15.33:0/1768237645 conn(0x5557c73218a0 msgr2=0x5557c73202b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 -- 172.21.15.33:0/1768237645 shutdown_connections 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.874+0000 7fdaef825740 1 -- 172.21.15.33:0/1768237645 wait complete. 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 -- start start 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 Processor -- start 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c7334720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7334c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81aabe0 con 0x5557c7326230 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81ac050 con 0x5557c732bd90 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81ab2b0 con 0x5557c7326c20 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdaef825740 10 client.0 _ll_register_callbacks cb 0x7fdae00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c7334720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c7334720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48706/0 (socket says 172.21.15.33:48706) 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 -- 172.21.15.33:0/3771214662 learned_addr learned my addr 172.21.15.33:0/3771214662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae6d9d640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7334c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 -- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 msgr2=0x5557c7334c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7334c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 -- 172.21.15.33:0/3771214662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 msgr2=0x5557c73a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 -- 172.21.15.33:0/3771214662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c73a0820 con 0x5557c7326230 2024-04-24T23:33:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdae659c640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c7334720 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fdad80a4b30 tx=0x7fdad8098010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dbf8096185fb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.875+0000 7fdacdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdacdffb640 1 -- 172.21.15.33:0/3771214662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad80b8070 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad80a00b0 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdacdffb640 1 -- 172.21.15.33:0/3771214662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdad804b880 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdacdffb640 1 -- 172.21.15.33:0/3771214662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdad8097bf0 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 10 client.19818 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5557c73a0820 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdacdffb640 1 -- 172.21.15.33:0/3771214662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad80a00b0 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdacdffb640 1 -- 172.21.15.33:0/3771214662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdad8086350 con 0x5557c7326230 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 10 client.19818 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 10 client.19818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 20 client.19818 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c73a1f00 0x5557c73a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 4 client.19818 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:33:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.876+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557c7291f00 con 0x5557c73a1f00 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c73a1f00 0x5557c73a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdae5d9b640 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c73a1f00 0x5557c73a42f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fdedee942805b87 server_cookie=29062b5703f6b987 in_seq=0 out_seq=0 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdacdffb640 10 client.19818 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdacdffb640 1 -- 172.21.15.33:0/3771214662 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557c7291f00 con 0x5557c73a1f00 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdacdffb640 10 client.19818 handle_command_reply: tid=0 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 client.19818 shutdown 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 msgr2=0x5557c7334720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c7334720 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fdad80a4b30 tx=0x7fdad8098010 comp rx=0 tx=0).stop 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 shutdown_connections 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5557c73a1f00 0x5557c73a42f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557c732bd90 0x5557c73a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557c7326c20 0x5557c7334c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 --2- 172.21.15.33:0/3771214662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c7326230 0x5557c7334720 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdacdffb640 0 client.19818 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 >> 172.21.15.33:0/3771214662 conn(0x5557c73218a0 msgr2=0x5557c7320b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 shutdown_connections 2024-04-24T23:33:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.877+0000 7fdaef825740 1 -- 172.21.15.33:0/3771214662 wait complete. 2024-04-24T23:33:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaef825740 20 client.19818 trim_cache size 0 max 16384 2024-04-24T23:33:15.883 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdad00a2b90 msgr2=0x7fdad00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdad00a2b90 0x7fdad00a5050 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005c40 tx=0x7fdae0009e50 comp rx=0 tx=0).stop 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8100050 msgr2=0x7fdae8199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8100050 0x7fdae8199d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e9f0 tx=0x7fdad800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 shutdown_connections 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdad00a2b90 0x7fdad00a5050 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae8101350 0x7fdae81a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae81009b0 0x7fdae819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 --2- 172.21.15.33:0/1227468674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8100050 0x7fdae8199d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 >> 172.21.15.33:0/1227468674 conn(0x7fdae80fbc30 msgr2=0x7fdae80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 shutdown_connections 2024-04-24T23:33:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:15.878+0000 7fdaecb14640 1 -- 172.21.15.33:0/1227468674 wait complete. 2024-04-24T23:33:15.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:33:15.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 187603 22.36 MB/sec execute 320 sec latency 86.111 ms 2024-04-24T23:33:16.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.023+0000 7fbae9453640 1 -- 172.21.15.33:0/3440616503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 msgr2=0x7fbae4102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.023+0000 7fbae9453640 1 --2- 172.21.15.33:0/3440616503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4102e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009900 tx=0x7fbacc02a410 comp rx=0 tx=0).stop 2024-04-24T23:33:16.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 -- 172.21.15.33:0/3440616503 shutdown_connections 2024-04-24T23:33:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 --2- 172.21.15.33:0/3440616503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae4103350 0x7fbae4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 --2- 172.21.15.33:0/3440616503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4102e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 --2- 172.21.15.33:0/3440616503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae41017a0 0x7fbae4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 -- 172.21.15.33:0/3440616503 >> 172.21.15.33:0/3440616503 conn(0x7fbae40fcd10 msgr2=0x7fbae40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 -- 172.21.15.33:0/3440616503 shutdown_connections 2024-04-24T23:33:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 -- 172.21.15.33:0/3440616503 wait complete. 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 Processor -- start 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.024+0000 7fbae9453640 1 -- start start 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae9453640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae41017a0 0x7fbae4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae9453640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae9453640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae4103350 0x7fbae419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae9453640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410d400 con 0x7fbae4103350 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae9453640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410d280 con 0x7fbae41017a0 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae9453640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410d580 con 0x7fbae4102990 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae41017a0 0x7fbae4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35254/0 (socket says 172.21.15.33:35254) 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 -- 172.21.15.33:0/2071857176 learned_addr learned my addr 172.21.15.33:0/2071857176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae37fe640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae4103350 0x7fbae419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 -- 172.21.15.33:0/2071857176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae41017a0 msgr2=0x7fbae4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae41017a0 0x7fbae4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 -- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae4103350 msgr2=0x7fbae419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae4103350 0x7fbae419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbadbfff640 1 -- 172.21.15.33:0/2071857176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae419f6f0 con 0x7fbae4102990 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.025+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae41017a0 0x7fbae4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.026+0000 7fbadbfff640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4198010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009230 tx=0x7fbacc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.026+0000 7fbae0ff9640 1 -- 172.21.15.33:0/2071857176 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc03f030 con 0x7fbae4102990 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.026+0000 7fbae0ff9640 1 -- 172.21.15.33:0/2071857176 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc003ce0 con 0x7fbae4102990 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.026+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae419f980 con 0x7fbae4102990 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.026+0000 7fbae0ff9640 1 -- 172.21.15.33:0/2071857176 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc035070 con 0x7fbae4102990 2024-04-24T23:33:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.026+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae419fe00 con 0x7fbae4102990 2024-04-24T23:33:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.029+0000 7fbae0ff9640 1 -- 172.21.15.33:0/2071857176 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbacc042030 con 0x7fbae4102990 2024-04-24T23:33:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.029+0000 7fbaec233740 1 Processor -- start 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 -- start start 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 0x55bb37a2a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a2ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2f5d0 0x55bb37a31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bb37985670 con 0x55bb37a2a090 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bb3792d970 con 0x55bb37a2f5d0 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbaec233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb37981630 con 0x55bb37a2aa80 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbae37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 0x55bb37a2a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbae37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 0x55bb37a2a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48724/0 (socket says 172.21.15.33:48724) 2024-04-24T23:33:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a2ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbae37fe640 1 -- 172.21.15.33:0/2054944652 learned_addr learned my addr 172.21.15.33:0/2054944652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.030+0000 7fbadbfff640 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2f5d0 0x55bb37a31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae0ff9640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbad00a2c10 0x7fbad00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 -- 172.21.15.33:0/2054944652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 msgr2=0x55bb37a2ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a2ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 -- 172.21.15.33:0/2054944652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2f5d0 msgr2=0x55bb37a31a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2f5d0 0x55bb37a31a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 -- 172.21.15.33:0/2054944652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb37951400 con 0x55bb37a2a090 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbadbfff640 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2f5d0 0x55bb37a31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a2ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbad00a2c10 0x7fbad00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.031+0000 7fbae37fe640 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 0x55bb37a2a4b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00f3b0 tx=0x7fbadc00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26921b2f6a25d518 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.032+0000 7fbae0ff9640 1 -- 172.21.15.33:0/2071857176 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbacc0ef860 con 0x7fbae4102990 2024-04-24T23:33:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.032+0000 7fbad97fa640 1 -- 172.21.15.33:0/2054944652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc012870 con 0x55bb37a2a090 2024-04-24T23:33:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.032+0000 7fbad97fa640 1 -- 172.21.15.33:0/2054944652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbadc01c400 con 0x55bb37a2a090 2024-04-24T23:33:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.032+0000 7fbad97fa640 1 -- 172.21.15.33:0/2054944652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc01b600 con 0x55bb37a2a090 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbae37fe640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbad00a2c10 0x7fbad00a50d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbadc020000 tx=0x7fbadc01b8e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 -- 172.21.15.33:0/2054944652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 msgr2=0x55bb37a2a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 0x55bb37a2a4b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00f3b0 tx=0x7fbadc00f870 comp rx=0 tx=0).stop 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 -- 172.21.15.33:0/2054944652 shutdown_connections 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2f5d0 0x55bb37a31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a2ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 --2- 172.21.15.33:0/2054944652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2a090 0x55bb37a2a4b0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 -- 172.21.15.33:0/2054944652 >> 172.21.15.33:0/2054944652 conn(0x55bb37a1c820 msgr2=0x55bb37a1cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 -- 172.21.15.33:0/2054944652 shutdown_connections 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.033+0000 7fbaec233740 1 -- 172.21.15.33:0/2054944652 wait complete. 2024-04-24T23:33:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbaec233740 1 -- start start 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbaec233740 1 Processor -- start 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbaec233740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a44be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbaec233740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a45120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbaec233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a45120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.034+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a44be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a44be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50188/0 (socket says 172.21.15.33:50188) 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a45120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48738/0 (socket says 172.21.15.33:48738) 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 -- 172.21.15.33:0/3437271920 learned_addr learned my addr 172.21.15.33:0/3437271920 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bb37a435a0 con 0x55bb37a2aa80 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bb37941050 con 0x55bb37a2a090 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb37a43720 con 0x55bb37a2f5d0 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbaec233740 10 client.0 _ll_register_callbacks cb 0x55bb37698be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae37fe640 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 -- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 msgr2=0x55bb37a4c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 -- 172.21.15.33:0/3437271920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 msgr2=0x55bb37a45120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a45120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb37912dc0 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a44be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbad4012d20 tx=0x7fbad400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd374e1c7ef7f904 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbaca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.035+0000 7fbaca7fc640 1 -- 172.21.15.33:0/3437271920 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad40120d0 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaca7fc640 1 -- 172.21.15.33:0/3437271920 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbad4004030 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbadc01c400 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaca7fc640 1 -- 172.21.15.33:0/3437271920 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbad4012270 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 10 client.29303 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bb376993f0 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaca7fc640 1 -- 172.21.15.33:0/3437271920 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbadc01c400 con 0x55bb37a2a090 2024-04-24T23:33:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaca7fc640 1 -- 172.21.15.33:0/3437271920 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbad401c570 con 0x55bb37a2a090 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 10 client.29303 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 10 client.29303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 20 client.29303 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37a509d0 0x55bb37a52e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 4 client.29303 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.036+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb37992bb0 con 0x55bb37a509d0 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.037+0000 7fbadbfff640 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37a509d0 0x55bb37a52e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.037+0000 7fbaca7fc640 10 client.29303 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.037+0000 7fbadbfff640 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37a509d0 0x55bb37a52e90 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64fb9c09f195e8b8 server_cookie=97462a8c2210572e in_seq=0 out_seq=0 2024-04-24T23:33:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaca7fc640 1 -- 172.21.15.33:0/3437271920 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb37992bb0 con 0x55bb37a509d0 2024-04-24T23:33:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaca7fc640 10 client.29303 handle_command_reply: tid=0 2024-04-24T23:33:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 client.29303 shutdown 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 msgr2=0x55bb37a44be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a44be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbad4012d20 tx=0x7fbad400f790 comp rx=0 tx=0).stop 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 shutdown_connections 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37a509d0 0x55bb37a52e90 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a45120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 --2- 172.21.15.33:0/3437271920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a44be0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 >> 172.21.15.33:0/3437271920 conn(0x55bb37a1c820 msgr2=0x55bb37a1bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaca7fc640 0 client.29303 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.039+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 shutdown_connections 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.040+0000 7fbaec233740 1 -- 172.21.15.33:0/3437271920 wait complete. 2024-04-24T23:33:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.040+0000 7fbaec233740 20 client.29303 trim_cache size 0 max 16384 2024-04-24T23:33:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 Processor -- start 2024-04-24T23:33:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 -- start start 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a4c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbae41a2290 con 0x55bb37a2aa80 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbae4075650 con 0x55bb37a2a090 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.059+0000 7fbaec233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbae40734a0 con 0x55bb37a2f5d0 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a2ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50196/0 (socket says 172.21.15.33:50196) 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 -- 172.21.15.33:0/4234660696 learned_addr learned my addr 172.21.15.33:0/4234660696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbadbfff640 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a4c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae37fe640 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 -- 172.21.15.33:0/4234660696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 msgr2=0x55bb37a2b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a2b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 -- 172.21.15.33:0/4234660696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 msgr2=0x55bb37a4c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a4c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 -- 172.21.15.33:0/4234660696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb377637d0 con 0x55bb37a2a090 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a2ae60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbad401cbd0 tx=0x7fbad400fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7edf1776d5abce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.060+0000 7fbad8ff9640 1 -- 172.21.15.33:0/4234660696 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad402e070 con 0x55bb37a2a090 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbad8ff9640 1 -- 172.21.15.33:0/4234660696 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad401c520 con 0x55bb37a2a090 2024-04-24T23:33:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbad8ff9640 1 -- 172.21.15.33:0/4234660696 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004680 con 0x55bb37a2a090 2024-04-24T23:33:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 -- 172.21.15.33:0/4234660696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 msgr2=0x55bb37a2ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a2ae60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbad401cbd0 tx=0x7fbad400fc90 comp rx=0 tx=0).stop 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 -- 172.21.15.33:0/4234660696 shutdown_connections 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2f5d0 0x55bb37a2b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2aa80 0x55bb37a4c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 --2- 172.21.15.33:0/4234660696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a2ae60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 -- 172.21.15.33:0/4234660696 >> 172.21.15.33:0/4234660696 conn(0x55bb37a1c820 msgr2=0x55bb37a1bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 -- 172.21.15.33:0/4234660696 shutdown_connections 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.061+0000 7fbaec233740 1 -- 172.21.15.33:0/4234660696 wait complete. 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 -- start start 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 Processor -- start 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a49340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbae4073b10 con 0x55bb37a2f5d0 2024-04-24T23:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbae4074f80 con 0x55bb37a2a090 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbae40741e0 con 0x55bb37a2aa80 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbaec233740 10 client.0 _ll_register_callbacks cb 0x7fbad4012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a49340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbae37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a49880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35286/0 (socket says 172.21.15.33:35286) 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a49340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50206/0 (socket says 172.21.15.33:50206) 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.062+0000 7fbadbfff640 1 -- 172.21.15.33:0/1069529541 learned_addr learned my addr 172.21.15.33:0/1069529541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbadbfff640 1 -- 172.21.15.33:0/1069529541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 msgr2=0x55bb37a49340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbadbfff640 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a49340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbadbfff640 1 -- 172.21.15.33:0/1069529541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2f5d0 msgr2=0x55bb37a4a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbadbfff640 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbadbfff640 1 -- 172.21.15.33:0/1069529541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb37a4ae80 con 0x55bb37a2aa80 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a49340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbadbfff640 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a49880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b8700 tx=0x7fbacc0bb930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f68a8e8c2cbaf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbaca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbaca7fc640 1 -- 172.21.15.33:0/1069529541 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc049040 con 0x55bb37a2aa80 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbaca7fc640 1 -- 172.21.15.33:0/1069529541 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbacc0ba030 con 0x55bb37a2aa80 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad401c520 con 0x55bb37a2aa80 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbaca7fc640 1 -- 172.21.15.33:0/1069529541 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbacc0bbd30 con 0x55bb37a2aa80 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.063+0000 7fbaec233740 10 client.29254 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.064+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bb37a4c250 con 0x55bb37a2aa80 2024-04-24T23:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.064+0000 7fbaca7fc640 1 -- 172.21.15.33:0/1069529541 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc0f2e50 con 0x55bb37a2aa80 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaca7fc640 1 -- 172.21.15.33:0/1069529541 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbacc0d2070 con 0x55bb37a2aa80 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaec233740 10 client.29254 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaec233740 10 client.29254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaec233740 20 client.29254 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaec233740 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37ab74b0 0x55bb37ab98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaec233740 4 client.29254 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbad401c520 con 0x55bb37ab74b0 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37ab74b0 0x55bb37ab98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbae2ffd640 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37ab74b0 0x55bb37ab98d0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e96c53a5e9ae15 server_cookie=cdf4b0213edf3049 in_seq=0 out_seq=0 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.065+0000 7fbaca7fc640 10 client.29254 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaca7fc640 1 -- 172.21.15.33:0/1069529541 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbad401c520 con 0x55bb37ab74b0 2024-04-24T23:33:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaca7fc640 10 client.29254 handle_command_reply: tid=0 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 client.29254 shutdown 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 msgr2=0x55bb37a49880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a49880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b8700 tx=0x7fbacc0bb930 comp rx=0 tx=0).stop 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 shutdown_connections 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bb37ab74b0 0x55bb37ab98d0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb37a2f5d0 0x55bb37a4a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb37a2aa80 0x55bb37a49880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 --2- 172.21.15.33:0/1069529541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb37a2a090 0x55bb37a49340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaca7fc640 0 client.29254 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 >> 172.21.15.33:0/1069529541 conn(0x55bb37a1c820 msgr2=0x55bb37a28e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 shutdown_connections 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 1 -- 172.21.15.33:0/1069529541 wait complete. 2024-04-24T23:33:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.066+0000 7fbaec233740 20 client.29254 trim_cache size 0 max 16384 2024-04-24T23:33:16.070 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:33:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbad00a2c10 msgr2=0x7fbad00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbad00a2c10 0x7fbad00a50d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbadc020000 tx=0x7fbadc01b8e0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 msgr2=0x7fbae4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4198010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009230 tx=0x7fbacc0042f0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 shutdown_connections 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fbad00a2c10 0x7fbad00a50d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae4103350 0x7fbae419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae4102990 0x7fbae4198010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 --2- 172.21.15.33:0/2071857176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae41017a0 0x7fbae4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 >> 172.21.15.33:0/2071857176 conn(0x7fbae40fcd10 msgr2=0x7fbae4105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 shutdown_connections 2024-04-24T23:33:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:16.067+0000 7fbae9453640 1 -- 172.21.15.33:0/2071857176 wait complete. 2024-04-24T23:33:16.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:15 smithi038 ceph-mon[32450]: pgmap v988: 129 pgs: 129 active+clean; 629 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.1 MiB/s wr, 477 op/s 2024-04-24T23:33:16.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:15 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/679604133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:33:16.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:15 smithi117 ceph-mon[32529]: pgmap v988: 129 pgs: 129 active+clean; 629 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.1 MiB/s wr, 477 op/s 2024-04-24T23:33:16.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:15 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/679604133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:33:16.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:15 smithi033 ceph-mon[29201]: pgmap v988: 129 pgs: 129 active+clean; 629 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.1 MiB/s wr, 477 op/s 2024-04-24T23:33:16.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:15 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/679604133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:33:16.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 188039 22.37 MB/sec execute 321 sec latency 445.095 ms 2024-04-24T23:33:17.082 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:33:17.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:33:17.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 -- 172.21.15.33:0/866613014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd4102250 msgr2=0x7f5fd4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 --2- 172.21.15.33:0/866613014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd4102250 0x7f5fd4108ae0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009ca0 tx=0x7f5fcc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 -- 172.21.15.33:0/866613014 shutdown_connections 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 --2- 172.21.15.33:0/866613014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd4102250 0x7f5fd4108ae0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 --2- 172.21.15.33:0/866613014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd4101890 0x7f5fd4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 --2- 172.21.15.33:0/866613014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fd41006a0 0x7f5fd4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 -- 172.21.15.33:0/866613014 >> 172.21.15.33:0/866613014 conn(0x7f5fd40fbc30 msgr2=0x7f5fd40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 -- 172.21.15.33:0/866613014 shutdown_connections 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 -- 172.21.15.33:0/866613014 wait complete. 2024-04-24T23:33:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.201+0000 7f5fda97e640 1 Processor -- start 2024-04-24T23:33:17.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 -- start start 2024-04-24T23:33:17.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 0x7f5fd4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd4101890 0x7f5fd4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fd4102250 0x7f5fd419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410d440 con 0x7f5fd41006a0 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410d2c0 con 0x7f5fd4102250 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fda97e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410d5c0 con 0x7f5fd4101890 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 0x7f5fd4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 0x7f5fd4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48790/0 (socket says 172.21.15.33:48790) 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 -- 172.21.15.33:0/2947725368 learned_addr learned my addr 172.21.15.33:0/2947725368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fd4102250 0x7f5fd419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd4101890 0x7f5fd4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 -- 172.21.15.33:0/2947725368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd4101890 msgr2=0x7f5fd4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd4101890 0x7f5fd4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 -- 172.21.15.33:0/2947725368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fd4102250 msgr2=0x7f5fd419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fd4102250 0x7f5fd419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.202+0000 7f5fd3fff640 1 -- 172.21.15.33:0/2947725368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd419f660 con 0x7f5fd41006a0 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.203+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 0x7f5fd4197a40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400e9c0 tx=0x7f5fc400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.203+0000 7f5fd17fa640 1 -- 172.21.15.33:0/2947725368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400cd80 con 0x7f5fd41006a0 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.203+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd419f950 con 0x7f5fd41006a0 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.203+0000 7f5fd17fa640 1 -- 172.21.15.33:0/2947725368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc40035f0 con 0x7f5fd41006a0 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.203+0000 7f5fd17fa640 1 -- 172.21.15.33:0/2947725368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4010690 con 0x7f5fd41006a0 2024-04-24T23:33:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.203+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd419fea0 con 0x7f5fd41006a0 2024-04-24T23:33:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.205+0000 7f5fd17fa640 1 -- 172.21.15.33:0/2947725368 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5fc40037a0 con 0x7f5fd41006a0 2024-04-24T23:33:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 Processor -- start 2024-04-24T23:33:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 -- start start 2024-04-24T23:33:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fd17fa640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5fbc0a2b90 0x7f5fbc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578d262a2c0 con 0x5578d26cb1a0 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5578d25d2910 con 0x5578d26cbb90 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5578d2626280 con 0x5578d26d0d00 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50246/0 (socket says 172.21.15.33:50246) 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fd37fe640 1 -- 172.21.15.33:0/2715108180 learned_addr learned my addr 172.21.15.33:0/2715108180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.206+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5fbc0a2b90 0x7f5fbc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd17fa640 1 -- 172.21.15.33:0/2947725368 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5fc40cf7b0 con 0x7f5fd41006a0 2024-04-24T23:33:17.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd37fe640 1 -- 172.21.15.33:0/2715108180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 msgr2=0x5578d26d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd37fe640 1 -- 172.21.15.33:0/2715108180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 msgr2=0x5578d26cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd37fe640 1 -- 172.21.15.33:0/2715108180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d25e3fb0 con 0x5578d26cbb90 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d0690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009ca0 tx=0x7f5fc8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c6b2158c92e1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5fbc0a2b90 0x7f5fbc0a5050 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005e30 tx=0x7f5fcc005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fba7fc640 1 -- 172.21.15.33:0/2715108180 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc801e070 con 0x5578d26cbb90 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fba7fc640 1 -- 172.21.15.33:0/2715108180 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8019b10 con 0x5578d26cbb90 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.207+0000 7f5fba7fc640 1 -- 172.21.15.33:0/2715108180 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800eaa0 con 0x5578d26cbb90 2024-04-24T23:33:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 -- 172.21.15.33:0/2715108180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 msgr2=0x5578d26d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d0690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009ca0 tx=0x7f5fc8011810 comp rx=0 tx=0).stop 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 -- 172.21.15.33:0/2715108180 shutdown_connections 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d0690 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/2715108180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 -- 172.21.15.33:0/2715108180 >> 172.21.15.33:0/2715108180 conn(0x5578d268e950 msgr2=0x5578d26c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 -- 172.21.15.33:0/2715108180 shutdown_connections 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 -- 172.21.15.33:0/2715108180 wait complete. 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.208+0000 7f5fdd76f740 1 -- start start 2024-04-24T23:33:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 Processor -- start 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578d26d3cb0 con 0x5578d26cb1a0 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5578d25e5e90 con 0x5578d26cbb90 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5578d26d3e30 con 0x5578d26d0d00 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fdd76f740 10 client.0 _ll_register_callbacks cb 0x5578d2621970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:17.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48806/0 (socket says 172.21.15.33:48806) 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fd3fff640 1 -- 172.21.15.33:0/3933356245 learned_addr learned my addr 172.21.15.33:0/3933356245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.209+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd3fff640 1 -- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 msgr2=0x5578d26dcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26dcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd3fff640 1 -- 172.21.15.33:0/3933356245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 msgr2=0x5578d26d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd3fff640 1 -- 172.21.15.33:0/3933356245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d24067e0 con 0x5578d26cb1a0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26d55c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5fc409bc10 tx=0x7f5fc409b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcf5c926e506e30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fb37fe640 1 -- 172.21.15.33:0/3933356245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc409c380 con 0x5578d26cb1a0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fb37fe640 1 -- 172.21.15.33:0/3933356245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5fc40aeae0 con 0x5578d26cb1a0 2024-04-24T23:33:17.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc8019b10 con 0x5578d26cb1a0 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fb37fe640 1 -- 172.21.15.33:0/3933356245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5fc40ae590 con 0x5578d26cb1a0 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fdd76f740 10 client.19845 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5578d2466f70 con 0x5578d26cb1a0 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.210+0000 7f5fb37fe640 1 -- 172.21.15.33:0/3933356245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8019b10 con 0x5578d26cb1a0 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fb37fe640 1 -- 172.21.15.33:0/3933356245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5fc40a44f0 con 0x5578d26cb1a0 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fdd76f740 10 client.19845 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fdd76f740 10 client.19845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fdd76f740 20 client.19845 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d26e1050 0x5578d26e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fdd76f740 4 client.19845 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d25fe5a0 con 0x5578d26e1050 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d26e1050 0x5578d26e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fb37fe640 10 client.19845 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.211+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d26e1050 0x5578d26e34e0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b60532f94a0051 server_cookie=418be64309462233 in_seq=0 out_seq=0 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.213+0000 7f5fb37fe640 1 -- 172.21.15.33:0/3933356245 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578d25fe5a0 con 0x5578d26e1050 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.213+0000 7f5fb37fe640 10 client.19845 handle_command_reply: tid=0 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.213+0000 7f5fdd76f740 1 client.19845 shutdown 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.213+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 msgr2=0x5578d26d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.213+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26d55c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5fc409bc10 tx=0x7f5fc409b660 comp rx=0 tx=0).stop 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 shutdown_connections 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d26e1050 0x5578d26e34e0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/3933356245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cb1a0 0x5578d26d55c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 >> 172.21.15.33:0/3933356245 conn(0x5578d268e950 msgr2=0x5578d26c1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fb37fe640 0 client.19845 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 shutdown_connections 2024-04-24T23:33:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 1 -- 172.21.15.33:0/3933356245 wait complete. 2024-04-24T23:33:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.214+0000 7f5fdd76f740 20 client.19845 trim_cache size 0 max 16384 2024-04-24T23:33:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 Processor -- start 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 -- start start 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cb1a0 0x5578d26d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 0x5578d26d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41aaa70 con 0x5578d26cbb90 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41a8340 con 0x5578d26cb1a0 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41a6190 con 0x5578d26d0d00 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 0x5578d26d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fd37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 0x5578d26d5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48810/0 (socket says 172.21.15.33:48810) 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fd37fe640 1 -- 172.21.15.33:0/1024330858 learned_addr learned my addr 172.21.15.33:0/1024330858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.237+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cb1a0 0x5578d26d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd8f1b640 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd37fe640 1 -- 172.21.15.33:0/1024330858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 msgr2=0x5578d26c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd37fe640 1 -- 172.21.15.33:0/1024330858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cb1a0 msgr2=0x5578d26d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cb1a0 0x5578d26d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd37fe640 1 -- 172.21.15.33:0/1024330858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d22fe170 con 0x5578d26cbb90 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cb1a0 0x5578d26d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 0x5578d26d5710 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009f50 tx=0x7f5fc8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba8c2211a231676 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fb9ffb640 1 -- 172.21.15.33:0/1024330858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc801e040 con 0x5578d26cbb90 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fb9ffb640 1 -- 172.21.15.33:0/1024330858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc802f070 con 0x5578d26cbb90 2024-04-24T23:33:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.238+0000 7f5fb9ffb640 1 -- 172.21.15.33:0/1024330858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8003c90 con 0x5578d26cbb90 2024-04-24T23:33:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 -- 172.21.15.33:0/1024330858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 msgr2=0x5578d26d5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 0x5578d26d5710 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009f50 tx=0x7f5fc8009660 comp rx=0 tx=0).stop 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 -- 172.21.15.33:0/1024330858 shutdown_connections 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26d0d00 0x5578d26c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26cbb90 0x5578d26d5710 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/1024330858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cb1a0 0x5578d26d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.239+0000 7f5fdd76f740 1 -- 172.21.15.33:0/1024330858 >> 172.21.15.33:0/1024330858 conn(0x5578d268e950 msgr2=0x5578d26ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 -- 172.21.15.33:0/1024330858 shutdown_connections 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 -- 172.21.15.33:0/1024330858 wait complete. 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 -- start start 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 Processor -- start 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 0x5578d26c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.240+0000 7f5fdd76f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26d0d00 0x5578d2749dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41a6800 con 0x5578d26d0d00 2024-04-24T23:33:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41a7c70 con 0x5578d26cbb90 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fdd76f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41a6ed0 con 0x5578d26cb1a0 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fdd76f740 10 client.0 _ll_register_callbacks cb 0x7f5fc4085080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd8f1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26d0d00 0x5578d2749dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 0x5578d26c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd8f1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26d0d00 0x5578d2749dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48818/0 (socket says 172.21.15.33:48818) 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd8f1b640 1 -- 172.21.15.33:0/904495281 learned_addr learned my addr 172.21.15.33:0/904495281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 0x5578d26c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:35346/0 (socket says 172.21.15.33:35346) 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 -- 172.21.15.33:0/904495281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 msgr2=0x5578d26ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 -- 172.21.15.33:0/904495281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26d0d00 msgr2=0x5578d2749dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26d0d00 0x5578d2749dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 -- 172.21.15.33:0/904495281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d26dcea0 con 0x5578d26cb1a0 2024-04-24T23:33:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fd3fff640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 0x5578d26c9df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40a4860 tx=0x7f5fc40ae900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4995da14f41cdb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.241+0000 7f5fb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fb37fe640 1 -- 172.21.15.33:0/904495281 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc409c380 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fb37fe640 1 -- 172.21.15.33:0/904495281 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5fc40979d0 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc802f070 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fb37fe640 1 -- 172.21.15.33:0/904495281 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5fc409b270 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 10 client.29269 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5578d274b430 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fb37fe640 1 -- 172.21.15.33:0/904495281 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc802f070 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fb37fe640 1 -- 172.21.15.33:0/904495281 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5fc4097bd0 con 0x5578d26cb1a0 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 10 client.29269 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 10 client.29269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 20 client.29269 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:17.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d274bc90 0x5578d274e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 4 client.29269 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:33:17.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.242+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5578d2637800 con 0x5578d274bc90 2024-04-24T23:33:17.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.243+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d274bc90 0x5578d274e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.244+0000 7f5fd37fe640 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d274bc90 0x5578d274e0b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1feb3af8996271f server_cookie=94c709e0db996231 in_seq=0 out_seq=0 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.244+0000 7f5fb37fe640 10 client.29269 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fb37fe640 1 -- 172.21.15.33:0/904495281 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5578d2637800 con 0x5578d274bc90 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fb37fe640 10 client.29269 handle_command_reply: tid=0 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 client.29269 shutdown 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 msgr2=0x5578d26c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 0x5578d26c9df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40a4860 tx=0x7f5fc40ae900 comp rx=0 tx=0).stop 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 shutdown_connections 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5578d274bc90 0x5578d274e0b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d26d0d00 0x5578d2749dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fb37fe640 0 client.29269 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5578d26cbb90 0x5578d26ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 --2- 172.21.15.33:0/904495281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578d26cb1a0 0x5578d26c9df0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.247+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 >> 172.21.15.33:0/904495281 conn(0x5578d268e950 msgr2=0x5578d26d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.248+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 shutdown_connections 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.248+0000 7f5fdd76f740 1 -- 172.21.15.33:0/904495281 wait complete. 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.248+0000 7f5fdd76f740 20 client.29269 trim_cache size 0 max 16384 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:33:17.249 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "8cb4fef8-fe25-4827-8cee-3e1dfc58dfa8", 2024-04-24T23:33:17.250 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:33:17.250 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.250+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5fbc0a2b90 msgr2=0x7f5fbc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.250+0000 7f5fda97e640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5fbc0a2b90 0x7f5fbc0a5050 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005e30 tx=0x7f5fcc005ef0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 msgr2=0x7f5fd4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 0x7f5fd4197a40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400e9c0 tx=0x7f5fc400ee80 comp rx=0 tx=0).stop 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 shutdown_connections 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5fbc0a2b90 0x7f5fbc0a5050 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5fd4102250 0x7f5fd419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fd4101890 0x7f5fd4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 --2- 172.21.15.33:0/2947725368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd41006a0 0x7f5fd4197a40 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 >> 172.21.15.33:0/2947725368 conn(0x7f5fd40fbc30 msgr2=0x7f5fd4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 shutdown_connections 2024-04-24T23:33:17.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.251+0000 7f5fda97e640 1 -- 172.21.15.33:0/2947725368 wait complete. 2024-04-24T23:33:17.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:33:17.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:33:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 -- 172.21.15.33:0/2587051798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45640ffa50 msgr2=0x7f45640ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 --2- 172.21.15.33:0/2587051798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45640ffa50 0x7f45640ffe70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4554009900 tx=0x7f455402a400 comp rx=0 tx=0).stop 2024-04-24T23:33:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 -- 172.21.15.33:0/2587051798 shutdown_connections 2024-04-24T23:33:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 --2- 172.21.15.33:0/2587051798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4564103480 0x7f4564109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 --2- 172.21.15.33:0/2587051798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4564102ac0 0x7f4564102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 --2- 172.21.15.33:0/2587051798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45640ffa50 0x7f45640ffe70 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.392+0000 7f4569e64640 1 -- 172.21.15.33:0/2587051798 >> 172.21.15.33:0/2587051798 conn(0x7f45640fb380 msgr2=0x7f45640fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.393+0000 7f4569e64640 1 -- 172.21.15.33:0/2587051798 shutdown_connections 2024-04-24T23:33:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.393+0000 7f4569e64640 1 -- 172.21.15.33:0/2587051798 wait complete. 2024-04-24T23:33:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.393+0000 7f4569e64640 1 Processor -- start 2024-04-24T23:33:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.393+0000 7f4569e64640 1 -- start start 2024-04-24T23:33:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.393+0000 7f4569e64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 0x7f4564102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.393+0000 7f4569e64640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 0x7f4564100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f4569e64640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4564103480 0x7f4564100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f4569e64640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f456410d470 con 0x7f4564102ac0 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f4569e64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f456410d2f0 con 0x7f45640ffa50 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f4569e64640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f456410d5f0 con 0x7f4564103480 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 0x7f4564100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f45637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 0x7f4564102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 0x7f4564100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48836/0 (socket says 172.21.15.33:48836) 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f45637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 0x7f4564102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:50282/0 (socket says 172.21.15.33:50282) 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 -- 172.21.15.33:0/3928602473 learned_addr learned my addr 172.21.15.33:0/3928602473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f4563fff640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4564103480 0x7f4564100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 -- 172.21.15.33:0/3928602473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4564103480 msgr2=0x7f4564100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4564103480 0x7f4564100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 -- 172.21.15.33:0/3928602473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 msgr2=0x7f4564102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 0x7f4564102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 -- 172.21.15.33:0/3928602473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4564101840 con 0x7f4564102ac0 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f45637fe640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 0x7f4564102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f455bfff640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 0x7f4564100a70 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e9f0 tx=0x7f454c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f45617fa640 1 -- 172.21.15.33:0/3928602473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c00cd80 con 0x7f4564102ac0 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f45617fa640 1 -- 172.21.15.33:0/3928602473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c010910 con 0x7f4564102ac0 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f45617fa640 1 -- 172.21.15.33:0/3928602473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018990 con 0x7f4564102ac0 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.394+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45641a1c10 con 0x7f4564102ac0 2024-04-24T23:33:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.395+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a2110 con 0x7f4564102ac0 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f45617fa640 1 -- 172.21.15.33:0/3928602473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f454c010ac0 con 0x7f4564102ac0 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 Processor -- start 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 -- start start 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c393660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c398730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c39b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55645c2f76b0 con 0x55645c393240 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55645c29f880 con 0x55645c393c30 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f456cb74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55645c2f3670 con 0x55645c398da0 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c393660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c393660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48848/0 (socket says 172.21.15.33:48848) 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 -- 172.21.15.33:0/1835662243 learned_addr learned my addr 172.21.15.33:0/1835662243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f45617fa640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45500a2c60 0x7f45500a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f45637fe640 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c398730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f455bfff640 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c39b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 -- 172.21.15.33:0/1835662243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 msgr2=0x55645c39b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c39b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 -- 172.21.15.33:0/1835662243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 msgr2=0x55645c398730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.398+0000 7f4563fff640 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c398730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4563fff640 1 -- 172.21.15.33:0/1835662243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645c0d57d0 con 0x55645c393240 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f455bfff640 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c39b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:33:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f45617fa640 1 -- 172.21.15.33:0/3928602473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f454c0cf650 con 0x7f4564102ac0 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4563fff640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45500a2c60 0x7f45500a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4563fff640 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c393660 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f455c011590 tx=0x7f455c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403c070548f2deee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4559ffb640 1 -- 172.21.15.33:0/1835662243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c017050 con 0x55645c393240 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4559ffb640 1 -- 172.21.15.33:0/1835662243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455c00fd20 con 0x55645c393240 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4559ffb640 1 -- 172.21.15.33:0/1835662243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c01bb00 con 0x55645c393240 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.399+0000 7f4563fff640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45500a2c60 0x7f45500a5120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f455c022000 tx=0x7f455c023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 -- 172.21.15.33:0/1835662243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 msgr2=0x55645c393660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c393660 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f455c011590 tx=0x7f455c011a50 comp rx=0 tx=0).stop 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 -- 172.21.15.33:0/1835662243 shutdown_connections 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c39b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c398730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 --2- 172.21.15.33:0/1835662243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c393660 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 -- 172.21.15.33:0/1835662243 >> 172.21.15.33:0/1835662243 conn(0x55645c38e8b0 msgr2=0x55645c38ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 -- 172.21.15.33:0/1835662243 shutdown_connections 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 -- 172.21.15.33:0/1835662243 wait complete. 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.400+0000 7f456cb74740 1 -- start start 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 Processor -- start 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c39d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c393c30 0x55645c39dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 0x55645c3a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55645c39bd50 con 0x55645c398da0 2024-04-24T23:33:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55645c2b2f50 con 0x55645c393240 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55645c39bed0 con 0x55645c393c30 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f456cb74740 10 client.0 _ll_register_callbacks cb 0x55645c2ee000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 0x55645c3a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f45637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c39d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 0x55645c3a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48864/0 (socket says 172.21.15.33:48864) 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f455bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c393c30 0x55645c39dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 -- 172.21.15.33:0/3737487393 learned_addr learned my addr 172.21.15.33:0/3737487393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 -- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c393c30 msgr2=0x55645c39dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c393c30 0x55645c39dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 -- 172.21.15.33:0/3737487393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 msgr2=0x55645c39d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c39d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 -- 172.21.15.33:0/3737487393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645c00b3f0 con 0x55645c398da0 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f45637fe640 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c39d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4563fff640 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 0x55645c3a4be0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f455c014040 tx=0x7f455c01bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891c0597f8f8be1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.401+0000 7f4542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f4542ffd640 1 -- 172.21.15.33:0/3737487393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c004bf0 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f4542ffd640 1 -- 172.21.15.33:0/3737487393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f455c004d90 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455c00fd20 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f4542ffd640 1 -- 172.21.15.33:0/3737487393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f455c017020 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 10 client.19863 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55645bfcb0f0 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f4542ffd640 1 -- 172.21.15.33:0/3737487393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455c00fd20 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f4542ffd640 1 -- 172.21.15.33:0/3737487393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f455c01a070 con 0x55645c398da0 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 10 client.19863 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 10 client.19863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 20 client.19863 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c3a92f0 0x55645c3ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 4 client.19863 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.402+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55645c304bf0 con 0x55645c3a92f0 2024-04-24T23:33:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.403+0000 7f45637fe640 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c3a92f0 0x55645c3ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.403+0000 7f4542ffd640 10 client.19863 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.403+0000 7f45637fe640 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c3a92f0 0x55645c3ab720 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5934952431862c27 server_cookie=a1f2d93e15634d1 in_seq=0 out_seq=0 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f4542ffd640 1 -- 172.21.15.33:0/3737487393 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55645c304bf0 con 0x55645c3a92f0 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f4542ffd640 10 client.19863 handle_command_reply: tid=0 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 client.19863 shutdown 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 msgr2=0x55645c3a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 0x55645c3a4be0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f455c014040 tx=0x7f455c01bc30 comp rx=0 tx=0).stop 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 shutdown_connections 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c3a92f0 0x55645c3ab720 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c398da0 0x55645c3a4be0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c393c30 0x55645c39dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 --2- 172.21.15.33:0/3737487393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c39d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 >> 172.21.15.33:0/3737487393 conn(0x55645c38e8b0 msgr2=0x55645c38daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f4542ffd640 0 client.19863 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 shutdown_connections 2024-04-24T23:33:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 1 -- 172.21.15.33:0/3737487393 wait complete. 2024-04-24T23:33:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.405+0000 7f456cb74740 20 client.19863 trim_cache size 0 max 16384 2024-04-24T23:33:17.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 Processor -- start 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 -- start start 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c395060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 0x55645c3955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c3a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45641aef60 con 0x55645c393c30 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45641ac830 con 0x55645c393240 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f456cb74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45641aa680 con 0x55645c398da0 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f455bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 0x55645c3955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f455bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 0x55645c3955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48878/0 (socket says 172.21.15.33:48878) 2024-04-24T23:33:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f45637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c395060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f455bfff640 1 -- 172.21.15.33:0/74768901 learned_addr learned my addr 172.21.15.33:0/74768901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.426+0000 7f4563fff640 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c3a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f455bfff640 1 -- 172.21.15.33:0/74768901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 msgr2=0x55645c3a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f455bfff640 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c3a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f455bfff640 1 -- 172.21.15.33:0/74768901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 msgr2=0x55645c395060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f455bfff640 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c395060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f455bfff640 1 -- 172.21.15.33:0/74768901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645c38e450 con 0x55645c393c30 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f45637fe640 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c395060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f455bfff640 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 0x55645c3955a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f454c0985a0 tx=0x7f454c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71daa89b4d7e0c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f45597fa640 1 -- 172.21.15.33:0/74768901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0a4790 con 0x55645c393c30 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f45597fa640 1 -- 172.21.15.33:0/74768901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c09ce50 con 0x55645c393c30 2024-04-24T23:33:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.427+0000 7f45597fa640 1 -- 172.21.15.33:0/74768901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018db0 con 0x55645c393c30 2024-04-24T23:33:17.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 -- 172.21.15.33:0/74768901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 msgr2=0x55645c3955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 0x55645c3955a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f454c0985a0 tx=0x7f454c018420 comp rx=0 tx=0).stop 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 -- 172.21.15.33:0/74768901 shutdown_connections 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c3a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393c30 0x55645c3955a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 --2- 172.21.15.33:0/74768901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393240 0x55645c395060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 -- 172.21.15.33:0/74768901 >> 172.21.15.33:0/74768901 conn(0x55645c38e8b0 msgr2=0x55645c39d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 -- 172.21.15.33:0/74768901 shutdown_connections 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 -- 172.21.15.33:0/74768901 wait complete. 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 -- start start 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.428+0000 7f456cb74740 1 Processor -- start 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c3a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c3a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c40d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45641aacf0 con 0x55645c393240 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45641ac160 con 0x55645c393c30 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45641ab3c0 con 0x55645c398da0 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 10 client.0 _ll_register_callbacks cb 0x7f455c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c3a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c3a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48886/0 (socket says 172.21.15.33:48886) 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f455bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c3a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 -- 172.21.15.33:0/3811070996 learned_addr learned my addr 172.21.15.33:0/3811070996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f4563fff640 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c40d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 -- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 msgr2=0x55645c40d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c40d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 -- 172.21.15.33:0/3811070996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 msgr2=0x55645c3a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c3a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 -- 172.21.15.33:0/3811070996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645c3a4dc0 con 0x55645c393240 2024-04-24T23:33:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f45637fe640 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c3a17b0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4554031610 tx=0x7f4554005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4969855c3ba916 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f4542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f4542ffd640 1 -- 172.21.15.33:0/3811070996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554035040 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f4542ffd640 1 -- 172.21.15.33:0/3811070996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4554047610 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c09ce50 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.429+0000 7f4542ffd640 1 -- 172.21.15.33:0/3811070996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4554055050 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 10 client.19869 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55645c40edf0 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f4542ffd640 1 -- 172.21.15.33:0/3811070996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c09ce50 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f4542ffd640 1 -- 172.21.15.33:0/3811070996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4554031840 con 0x55645c393240 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 10 client.19869 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 10 client.19869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 20 client.19869 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c40f840 0x55645c411c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 4 client.19869 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55645c2fef40 con 0x55645c40f840 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f455bfff640 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c40f840 0x55645c411c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f4542ffd640 10 client.19869 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.430+0000 7f455bfff640 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c40f840 0x55645c411c60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0c352750bd60f9 server_cookie=1e3b4b9114cfad39 in_seq=0 out_seq=0 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f4542ffd640 1 -- 172.21.15.33:0/3811070996 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55645c2fef40 con 0x55645c40f840 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f4542ffd640 10 client.19869 handle_command_reply: tid=0 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 client.19869 shutdown 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 msgr2=0x55645c3a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c3a17b0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4554031610 tx=0x7f4554005400 comp rx=0 tx=0).stop 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 shutdown_connections 2024-04-24T23:33:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55645c40f840 0x55645c411c60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c398da0 0x55645c40d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55645c393c30 0x55645c3a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f4542ffd640 0 client.19869 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 --2- 172.21.15.33:0/3811070996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645c393240 0x55645c3a17b0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 >> 172.21.15.33:0/3811070996 conn(0x55645c38e8b0 msgr2=0x55645c39d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 shutdown_connections 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 1 -- 172.21.15.33:0/3811070996 wait complete. 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.431+0000 7f456cb74740 20 client.19869 trim_cache size 0 max 16384 2024-04-24T23:33:17.437 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: "8cb4fef8-fe25-4827-8cee-3e1dfc58dfa8": { 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "8cb4fef8-fe25-4827-8cee-3e1dfc58dfa8", 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:33:17.438 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:33:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45500a2c60 msgr2=0x7f45500a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45500a2c60 0x7f45500a5120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f455c022000 tx=0x7f455c023000 comp rx=0 tx=0).stop 2024-04-24T23:33:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 msgr2=0x7f4564100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 0x7f4564100a70 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e9f0 tx=0x7f454c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 shutdown_connections 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f45500a2c60 0x7f45500a5120 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4564103480 0x7f4564100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4564102ac0 0x7f4564100a70 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 --2- 172.21.15.33:0/3928602473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45640ffa50 0x7f4564102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 >> 172.21.15.33:0/3928602473 conn(0x7f45640fb380 msgr2=0x7f4564105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 shutdown_connections 2024-04-24T23:33:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:17.432+0000 7f4569e64640 1 -- 172.21.15.33:0/3928602473 wait complete. 2024-04-24T23:33:17.445 INFO:tasks.cephfs.filesystem:scrub status for tag:8cb4fef8-fe25-4827-8cee-3e1dfc58dfa8 - {'path': '/', 'tag': '8cb4fef8-fe25-4827-8cee-3e1dfc58dfa8', 'options': 'recursive,force'} 2024-04-24T23:33:17.896 INFO:tasks.workunit.client.0.smithi033.stdout: 10 188407 22.36 MB/sec execute 322 sec latency 972.969 ms 2024-04-24T23:33:18.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: pgmap v989: 129 pgs: 129 active+clean; 629 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 326 op/s 2024-04-24T23:33:18.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:33:18.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:18.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:18.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:18.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:17 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: pgmap v989: 129 pgs: 129 active+clean; 629 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 326 op/s 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:18.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:17 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: pgmap v989: 129 pgs: 129 active+clean; 629 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 326 op/s 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:18.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:17 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:18.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 188641 22.33 MB/sec execute 323 sec latency 1474.391 ms 2024-04-24T23:33:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:18 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:19.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:18 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:19.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:18 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:33:19.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:18 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:19.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:18 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:19.177 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:18 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:33:19.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:18 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:19.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:18 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:19.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:18 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:33:19.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 188869 22.30 MB/sec execute 324 sec latency 2474.449 ms 2024-04-24T23:33:20.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:19 smithi038 ceph-mon[32450]: pgmap v990: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 468 op/s 2024-04-24T23:33:20.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:19 smithi117 ceph-mon[32529]: pgmap v990: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 468 op/s 2024-04-24T23:33:20.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:19 smithi033 ceph-mon[29201]: pgmap v990: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.7 MiB/s wr, 468 op/s 2024-04-24T23:33:20.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 189191 22.30 MB/sec execute 325 sec latency 3474.507 ms 2024-04-24T23:33:21.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:21 smithi038 ceph-mon[32450]: pgmap v991: 129 pgs: 129 active+clean; 606 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-04-24T23:33:21.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:21 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:21 smithi117 ceph-mon[32529]: pgmap v991: 129 pgs: 129 active+clean; 606 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-04-24T23:33:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:21 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:21 smithi033 ceph-mon[29201]: pgmap v991: 129 pgs: 129 active+clean; 606 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-04-24T23:33:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:21 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:21.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 189688 22.29 MB/sec execute 326 sec latency 4043.695 ms 2024-04-24T23:33:22.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 190165 22.33 MB/sec execute 327 sec latency 81.754 ms 2024-04-24T23:33:23.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: pgmap v992: 129 pgs: 129 active+clean; 607 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 399 op/s 2024-04-24T23:33:23.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:33:23.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:33:23.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: pgmap v992: 129 pgs: 129 active+clean; 607 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 399 op/s 2024-04-24T23:33:23.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:33:23.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:33:23.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: pgmap v992: 129 pgs: 129 active+clean; 607 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 399 op/s 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:33:23.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:33:23.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 190472 22.28 MB/sec execute 328 sec latency 338.013 ms 2024-04-24T23:33:24.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 190840 22.29 MB/sec execute 329 sec latency 131.604 ms 2024-04-24T23:33:25.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:25 smithi038 ceph-mon[32450]: pgmap v993: 129 pgs: 129 active+clean; 609 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 498 op/s 2024-04-24T23:33:25.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:25 smithi117 ceph-mon[32529]: pgmap v993: 129 pgs: 129 active+clean; 609 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 498 op/s 2024-04-24T23:33:25.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:25 smithi033 ceph-mon[29201]: pgmap v993: 129 pgs: 129 active+clean; 609 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 498 op/s 2024-04-24T23:33:25.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 191220 22.29 MB/sec execute 330 sec latency 102.184 ms 2024-04-24T23:33:26.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 191595 22.27 MB/sec execute 331 sec latency 111.810 ms 2024-04-24T23:33:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:27 smithi038 ceph-mon[32450]: pgmap v994: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-04-24T23:33:27.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:27 smithi117 ceph-mon[32529]: pgmap v994: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-04-24T23:33:27.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:27 smithi033 ceph-mon[29201]: pgmap v994: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-04-24T23:33:27.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 191963 22.26 MB/sec execute 332 sec latency 99.277 ms 2024-04-24T23:33:28.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 192286 22.24 MB/sec execute 333 sec latency 413.181 ms 2024-04-24T23:33:29.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:29 smithi038 ceph-mon[32450]: pgmap v995: 129 pgs: 129 active+clean; 615 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 448 op/s 2024-04-24T23:33:29.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:29 smithi117 ceph-mon[32529]: pgmap v995: 129 pgs: 129 active+clean; 615 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 448 op/s 2024-04-24T23:33:29.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:29 smithi033 ceph-mon[29201]: pgmap v995: 129 pgs: 129 active+clean; 615 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 448 op/s 2024-04-24T23:33:29.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 192766 22.25 MB/sec execute 334 sec latency 113.695 ms 2024-04-24T23:33:30.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 193247 22.29 MB/sec execute 335 sec latency 62.379 ms 2024-04-24T23:33:31.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:31 smithi038 ceph-mon[32450]: pgmap v996: 129 pgs: 129 active+clean; 622 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 368 op/s 2024-04-24T23:33:31.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:31 smithi117 ceph-mon[32529]: pgmap v996: 129 pgs: 129 active+clean; 622 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 368 op/s 2024-04-24T23:33:31.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:31 smithi033 ceph-mon[29201]: pgmap v996: 129 pgs: 129 active+clean; 622 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 368 op/s 2024-04-24T23:33:31.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 193635 22.27 MB/sec execute 336 sec latency 110.645 ms 2024-04-24T23:33:32.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 193997 22.26 MB/sec execute 337 sec latency 124.910 ms 2024-04-24T23:33:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:33 smithi038 ceph-mon[32450]: pgmap v997: 129 pgs: 129 active+clean; 625 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 395 op/s 2024-04-24T23:33:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:33:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:33:33.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:33 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:33:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:33 smithi117 ceph-mon[32529]: pgmap v997: 129 pgs: 129 active+clean; 625 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 395 op/s 2024-04-24T23:33:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:33:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:33:33.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:33 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:33:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:33 smithi033 ceph-mon[29201]: pgmap v997: 129 pgs: 129 active+clean; 625 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 395 op/s 2024-04-24T23:33:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:33:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:33:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:33 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:33:33.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 194291 22.25 MB/sec execute 338 sec latency 388.399 ms 2024-04-24T23:33:34.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 194655 22.23 MB/sec execute 339 sec latency 133.722 ms 2024-04-24T23:33:35.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:35 smithi033 ceph-mon[29201]: pgmap v998: 129 pgs: 129 active+clean; 627 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 435 op/s 2024-04-24T23:33:35.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:35 smithi038 ceph-mon[32450]: pgmap v998: 129 pgs: 129 active+clean; 627 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 435 op/s 2024-04-24T23:33:35.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:35 smithi117 ceph-mon[32529]: pgmap v998: 129 pgs: 129 active+clean; 627 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 435 op/s 2024-04-24T23:33:35.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 195152 22.25 MB/sec execute 340 sec latency 63.790 ms 2024-04-24T23:33:36.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 195699 22.26 MB/sec execute 341 sec latency 72.212 ms 2024-04-24T23:33:37.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:37 smithi033 ceph-mon[29201]: pgmap v999: 129 pgs: 129 active+clean; 629 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 346 op/s 2024-04-24T23:33:37.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:37 smithi038 ceph-mon[32450]: pgmap v999: 129 pgs: 129 active+clean; 629 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 346 op/s 2024-04-24T23:33:37.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:37 smithi117 ceph-mon[32529]: pgmap v999: 129 pgs: 129 active+clean; 629 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 346 op/s 2024-04-24T23:33:37.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 196159 22.28 MB/sec execute 342 sec latency 83.947 ms 2024-04-24T23:33:38.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 196355 22.24 MB/sec execute 343 sec latency 408.980 ms 2024-04-24T23:33:39.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:39 smithi038 ceph-mon[32450]: pgmap v1000: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 409 op/s 2024-04-24T23:33:39.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:39 smithi117 ceph-mon[32529]: pgmap v1000: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 409 op/s 2024-04-24T23:33:39.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:39 smithi033 ceph-mon[29201]: pgmap v1000: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 409 op/s 2024-04-24T23:33:39.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 196723 22.24 MB/sec execute 344 sec latency 193.086 ms 2024-04-24T23:33:40.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 197111 22.24 MB/sec execute 345 sec latency 78.801 ms 2024-04-24T23:33:41.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:41 smithi038 ceph-mon[32450]: pgmap v1001: 129 pgs: 129 active+clean; 629 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 379 op/s 2024-04-24T23:33:41.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:41 smithi117 ceph-mon[32529]: pgmap v1001: 129 pgs: 129 active+clean; 629 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 379 op/s 2024-04-24T23:33:41.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:41 smithi033 ceph-mon[29201]: pgmap v1001: 129 pgs: 129 active+clean; 629 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 379 op/s 2024-04-24T23:33:41.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 197438 22.22 MB/sec execute 346 sec latency 91.887 ms 2024-04-24T23:33:42.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 197914 22.22 MB/sec execute 347 sec latency 88.296 ms 2024-04-24T23:33:43.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:43 smithi038 ceph-mon[32450]: pgmap v1002: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 379 op/s 2024-04-24T23:33:43.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:43 smithi117 ceph-mon[32529]: pgmap v1002: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 379 op/s 2024-04-24T23:33:43.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:43 smithi033 ceph-mon[29201]: pgmap v1002: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 379 op/s 2024-04-24T23:33:43.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 198275 22.20 MB/sec execute 348 sec latency 338.587 ms 2024-04-24T23:33:44.899 INFO:tasks.workunit.client.0.smithi033.stdout: 10 198845 22.24 MB/sec execute 349 sec latency 110.155 ms 2024-04-24T23:33:45.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:45 smithi038 ceph-mon[32450]: pgmap v1003: 129 pgs: 129 active+clean; 614 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.2 MiB/s wr, 442 op/s 2024-04-24T23:33:45.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:45 smithi117 ceph-mon[32529]: pgmap v1003: 129 pgs: 129 active+clean; 614 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.2 MiB/s wr, 442 op/s 2024-04-24T23:33:45.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:45 smithi033 ceph-mon[29201]: pgmap v1003: 129 pgs: 129 active+clean; 614 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.2 MiB/s wr, 442 op/s 2024-04-24T23:33:45.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 199219 22.24 MB/sec execute 350 sec latency 88.397 ms 2024-04-24T23:33:46.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 199609 22.24 MB/sec execute 351 sec latency 105.646 ms 2024-04-24T23:33:47.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:33:47.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:33:47.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:47 smithi038 ceph-mon[32450]: pgmap v1004: 129 pgs: 129 active+clean; 615 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-04-24T23:33:47.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:47 smithi117 ceph-mon[32529]: pgmap v1004: 129 pgs: 129 active+clean; 615 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-04-24T23:33:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 -- 172.21.15.33:0/2619327013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5841033f0 msgr2=0x7fc584109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 --2- 172.21.15.33:0/2619327013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5841033f0 0x7fc584109c80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009cd0 tx=0x7fc57c02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 -- 172.21.15.33:0/2619327013 shutdown_connections 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 --2- 172.21.15.33:0/2619327013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5841033f0 0x7fc584109c80 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 --2- 172.21.15.33:0/2619327013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc584102a30 0x7fc584102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 --2- 172.21.15.33:0/2619327013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5840ffa60 0x7fc5840ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 -- 172.21.15.33:0/2619327013 >> 172.21.15.33:0/2619327013 conn(0x7fc5840fb370 msgr2=0x7fc5840fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 -- 172.21.15.33:0/2619327013 shutdown_connections 2024-04-24T23:33:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.696+0000 7fc588d70640 1 -- 172.21.15.33:0/2619327013 wait complete. 2024-04-24T23:33:47.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 Processor -- start 2024-04-24T23:33:47.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 -- start start 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5840ffa60 0x7fc584199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 0x7fc58419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5841033f0 0x7fc5841a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410d440 con 0x7fc584102a30 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410d2c0 con 0x7fc5841033f0 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc588d70640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410d5c0 con 0x7fc5840ffa60 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 0x7fc58419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc58259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5840ffa60 0x7fc584199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 0x7fc58419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47290/0 (socket says 172.21.15.33:47290) 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 -- 172.21.15.33:0/125385252 learned_addr learned my addr 172.21.15.33:0/125385252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc582d9d640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5841033f0 0x7fc5841a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 -- 172.21.15.33:0/125385252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5840ffa60 msgr2=0x7fc584199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5840ffa60 0x7fc584199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 -- 172.21.15.33:0/125385252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5841033f0 msgr2=0x7fc5841a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5841033f0 0x7fc5841a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc57bfff640 1 -- 172.21.15.33:0/125385252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5841a18e0 con 0x7fc584102a30 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.697+0000 7fc58259c640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5840ffa60 0x7fc584199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.698+0000 7fc57bfff640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 0x7fc58419a1e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00e9f0 tx=0x7fc56c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.698+0000 7fc57b7fe640 1 -- 172.21.15.33:0/125385252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00cd80 con 0x7fc584102a30 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.698+0000 7fc57b7fe640 1 -- 172.21.15.33:0/125385252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c005500 con 0x7fc584102a30 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.698+0000 7fc57b7fe640 1 -- 172.21.15.33:0/125385252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c010690 con 0x7fc584102a30 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.698+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841a1bd0 con 0x7fc584102a30 2024-04-24T23:33:47.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.698+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a20d0 con 0x7fc584102a30 2024-04-24T23:33:47.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc57b7fe640 1 -- 172.21.15.33:0/125385252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc56c0046c0 con 0x7fc584102a30 2024-04-24T23:33:47.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 Processor -- start 2024-04-24T23:33:47.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 -- start start 2024-04-24T23:33:47.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 0x55b972df7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df7da0 0x55b972dfa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b972d566b0 con 0x55b972df2c30 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b972cfe880 con 0x55b972df2240 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc58ba82740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b972d52670 con 0x55b972df7da0 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.701+0000 7fc57b7fe640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5700a2b90 0x7fc5700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 0x55b972df7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df7da0 0x55b972dfa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 0x55b972df7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47298/0 (socket says 172.21.15.33:47298) 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 -- 172.21.15.33:0/201803985 learned_addr learned my addr 172.21.15.33:0/201803985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc582d9d640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5700a2b90 0x7fc5700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 -- 172.21.15.33:0/201803985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df7da0 msgr2=0x55b972dfa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df7da0 0x55b972dfa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc57b7fe640 1 -- 172.21.15.33:0/125385252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc56c0cf7f0 con 0x7fc584102a30 2024-04-24T23:33:47.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 -- 172.21.15.33:0/201803985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 msgr2=0x55b972df2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 -- 172.21.15.33:0/201803985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b972b347d0 con 0x55b972df2c30 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc582d9d640 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc58259c640 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 0x55b972df7730 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc5740114f0 tx=0x7fc5740119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ac396b432af689 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc582d9d640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5700a2b90 0x7fc5700a5050 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009430 tx=0x7fc57c004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc578ff9640 1 -- 172.21.15.33:0/201803985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574004310 con 0x55b972df2c30 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc578ff9640 1 -- 172.21.15.33:0/201803985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5740044b0 con 0x55b972df2c30 2024-04-24T23:33:47.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.702+0000 7fc578ff9640 1 -- 172.21.15.33:0/201803985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574004310 con 0x55b972df2c30 2024-04-24T23:33:47.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:47 smithi033 ceph-mon[29201]: pgmap v1004: 129 pgs: 129 active+clean; 615 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 -- 172.21.15.33:0/201803985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 msgr2=0x55b972df7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 0x55b972df7730 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc5740114f0 tx=0x7fc5740119b0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 -- 172.21.15.33:0/201803985 shutdown_connections 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df7da0 0x55b972dfa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df2c30 0x55b972df7730 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 --2- 172.21.15.33:0/201803985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.703+0000 7fc58ba82740 1 -- 172.21.15.33:0/201803985 >> 172.21.15.33:0/201803985 conn(0x55b972ded8b0 msgr2=0x55b972dedcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 -- 172.21.15.33:0/201803985 shutdown_connections 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 -- 172.21.15.33:0/201803985 wait complete. 2024-04-24T23:33:47.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 -- start start 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 Processor -- start 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972dfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972dfcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b972dfad50 con 0x55b972df7da0 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b972d11f50 con 0x55b972df2240 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b972dfaed0 con 0x55b972df2c30 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58ba82740 10 client.0 _ll_register_callbacks cb 0x55b972d4d000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972dfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972dfcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972dfc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41022/0 (socket says 172.21.15.33:41022) 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972dfcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53046/0 (socket says 172.21.15.33:53046) 2024-04-24T23:33:47.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.704+0000 7fc58259c640 1 -- 172.21.15.33:0/3848960076 learned_addr learned my addr 172.21.15.33:0/3848960076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc582d9d640 1 -- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 msgr2=0x55b972dfcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc582d9d640 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972dfcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc582d9d640 1 -- 172.21.15.33:0/3848960076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 msgr2=0x55b972dfc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc582d9d640 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972dfc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc582d9d640 1 -- 172.21.15.33:0/3848960076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b972a6a3f0 con 0x55b972df7da0 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc57bfff640 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972dfcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc582d9d640 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03bc0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009ca0 tx=0x7fc57c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5142e73539a84e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc569ffb640 1 -- 172.21.15.33:0/3848960076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c050070 con 0x55b972df7da0 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc569ffb640 1 -- 172.21.15.33:0/3848960076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc57c0305a0 con 0x55b972df7da0 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5740044b0 con 0x55b972df7da0 2024-04-24T23:33:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc569ffb640 1 -- 172.21.15.33:0/3848960076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc57c0307a0 con 0x55b972df7da0 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc58ba82740 10 client.19881 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b972a2a0f0 con 0x55b972df7da0 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.705+0000 7fc569ffb640 1 -- 172.21.15.33:0/3848960076 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5740044b0 con 0x55b972df7da0 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc569ffb640 1 -- 172.21.15.33:0/3848960076 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc57c048040 con 0x55b972df7da0 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58ba82740 10 client.19881 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58ba82740 10 client.19881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58ba82740 20 client.19881 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58ba82740 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58ba82740 4 client.19881 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b972d63bf0 con 0x55b972e08400 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58259c640 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc58259c640 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e0a860 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d61d3290d57b37a server_cookie=4c2efc5190b3f8cd in_seq=0 out_seq=0 2024-04-24T23:33:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.706+0000 7fc569ffb640 10 client.19881 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc569ffb640 1 -- 172.21.15.33:0/3848960076 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b972d63bf0 con 0x55b972e08400 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc569ffb640 10 client.19881 handle_command_reply: tid=0 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 client.19881 shutdown 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 msgr2=0x55b972e03bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03bc0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009ca0 tx=0x7fc57c02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 shutdown_connections 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e0a860 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03bc0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc569ffb640 0 client.19881 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972dfcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 --2- 172.21.15.33:0/3848960076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972dfc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 >> 172.21.15.33:0/3848960076 conn(0x55b972ded8b0 msgr2=0x55b972decaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 shutdown_connections 2024-04-24T23:33:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 1 -- 172.21.15.33:0/3848960076 wait complete. 2024-04-24T23:33:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.707+0000 7fc58ba82740 20 client.19881 trim_cache size 0 max 16384 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 Processor -- start 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 -- start start 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972df4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841aee50 con 0x55b972df7da0 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ac720 con 0x55b972df2240 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.726+0000 7fc58ba82740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841aa5c0 con 0x55b972df2c30 2024-04-24T23:33:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972df4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41036/0 (socket says 172.21.15.33:41036) 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47330/0 (socket says 172.21.15.33:47330) 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 -- 172.21.15.33:0/64727380 learned_addr learned my addr 172.21.15.33:0/64727380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 -- 172.21.15.33:0/64727380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 msgr2=0x55b972df4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972df4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 -- 172.21.15.33:0/64727380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 msgr2=0x55b972e03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 -- 172.21.15.33:0/64727380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b972ddf090 con 0x55b972df2240 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc58259c640 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df3ff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc5740114c0 tx=0x7fc5740096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79472d5a7acbba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc56bfff640 1 -- 172.21.15.33:0/64727380 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574034070 con 0x55b972df2240 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc56bfff640 1 -- 172.21.15.33:0/64727380 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5740044b0 con 0x55b972df2240 2024-04-24T23:33:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.727+0000 7fc56bfff640 1 -- 172.21.15.33:0/64727380 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574004310 con 0x55b972df2240 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 -- 172.21.15.33:0/64727380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 msgr2=0x55b972df3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df3ff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc5740114c0 tx=0x7fc5740096c0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 -- 172.21.15.33:0/64727380 shutdown_connections 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2c30 0x55b972df4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 --2- 172.21.15.33:0/64727380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2240 0x55b972df3ff0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 -- 172.21.15.33:0/64727380 >> 172.21.15.33:0/64727380 conn(0x55b972ded8b0 msgr2=0x55b972dfc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 -- 172.21.15.33:0/64727380 shutdown_connections 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 -- 172.21.15.33:0/64727380 wait complete. 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.728+0000 7fc58ba82740 1 -- start start 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 Processor -- start 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2240 0x55b972e01010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2c30 0x55b972e01550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e02560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841aac30 con 0x55b972df7da0 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ac050 con 0x55b972df2c30 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ab2b0 con 0x55b972df2240 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58ba82740 10 client.0 _ll_register_callbacks cb 0x7fc57c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e02560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2240 0x55b972e01010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e02560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47338/0 (socket says 172.21.15.33:47338) 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc58259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2240 0x55b972e01010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53062/0 (socket says 172.21.15.33:53062) 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 -- 172.21.15.33:0/89579574 learned_addr learned my addr 172.21.15.33:0/89579574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc57bfff640 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2c30 0x55b972e01550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 -- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2240 msgr2=0x55b972e01010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2240 0x55b972e01010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 -- 172.21.15.33:0/89579574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2c30 msgr2=0x55b972e01550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2c30 0x55b972e01550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 -- 172.21.15.33:0/89579574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b972e02b50 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc582d9d640 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e02560 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02ae00 tx=0x7fc57c030dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d59cdef105785e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.729+0000 7fc569ffb640 1 -- 172.21.15.33:0/89579574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c050040 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc569ffb640 1 -- 172.21.15.33:0/89579574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc57c04a650 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5740044b0 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc569ffb640 1 -- 172.21.15.33:0/89579574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc57c003c80 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 10 client.19887 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b972e03e60 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc569ffb640 1 -- 172.21.15.33:0/89579574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5740044b0 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc569ffb640 1 -- 172.21.15.33:0/89579574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc57c048040 con 0x55b972df7da0 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 10 client.19887 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:47.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 10 client.19887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 20 client.19887 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 4 client.19887 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b972d5df40 con 0x55b972e08400 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.730+0000 7fc58259c640 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58259c640 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e711b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7070d435bd36738b server_cookie=4e9d0c1116938726 in_seq=0 out_seq=0 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc569ffb640 10 client.19887 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc569ffb640 1 -- 172.21.15.33:0/89579574 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b972d5df40 con 0x55b972e08400 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc569ffb640 10 client.19887 handle_command_reply: tid=0 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 client.19887 shutdown 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 msgr2=0x55b972e02560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e02560 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02ae00 tx=0x7fc57c030dc0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 shutdown_connections 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b972e08400 0x55b972e711b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b972df7da0 0x55b972e02560 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b972df2c30 0x55b972e01550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 --2- 172.21.15.33:0/89579574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b972df2240 0x55b972e01010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 >> 172.21.15.33:0/89579574 conn(0x55b972ded8b0 msgr2=0x55b972df3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc569ffb640 0 client.19887 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 shutdown_connections 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 1 -- 172.21.15.33:0/89579574 wait complete. 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.731+0000 7fc58ba82740 20 client.19887 trim_cache size 0 max 16384 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:33:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:33:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:33:47.739 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:33:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5700a2b90 msgr2=0x7fc5700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5700a2b90 0x7fc5700a5050 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009430 tx=0x7fc57c004a40 comp rx=0 tx=0).stop 2024-04-24T23:33:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 msgr2=0x7fc58419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 0x7fc58419a1e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00e9f0 tx=0x7fc56c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 shutdown_connections 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc5700a2b90 0x7fc5700a5050 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5841033f0 0x7fc5841a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc584102a30 0x7fc58419a1e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 --2- 172.21.15.33:0/125385252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5840ffa60 0x7fc584199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 >> 172.21.15.33:0/125385252 conn(0x7fc5840fb370 msgr2=0x7fc584100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 shutdown_connections 2024-04-24T23:33:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.732+0000 7fc588d70640 1 -- 172.21.15.33:0/125385252 wait complete. 2024-04-24T23:33:47.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:33:47.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:33:47.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/2254938885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34101870 msgr2=0x7f6c34101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/2254938885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34101870 0x7f6c34101cf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009900 tx=0x7f6c2402a410 comp rx=0 tx=0).stop 2024-04-24T23:33:47.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/2254938885 shutdown_connections 2024-04-24T23:33:47.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/2254938885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34102230 0x7f6c34108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/2254938885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34101870 0x7f6c34101cf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/2254938885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c34100680 0x7f6c34100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/2254938885 >> 172.21.15.33:0/2254938885 conn(0x7f6c340fbbf0 msgr2=0x7f6c340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/2254938885 shutdown_connections 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/2254938885 wait complete. 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 Processor -- start 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.856+0000 7f6c3b3dc640 1 -- start start 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c3b3dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 0x7f6c34199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c3b3dc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c34101870 0x7f6c3419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c3b3dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34102230 0x7f6c341a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c3b3dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3410c2e0 con 0x7f6c34100680 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c3b3dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3410c160 con 0x7f6c34102230 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c3b3dc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3410c460 con 0x7f6c34101870 2024-04-24T23:33:47.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39178640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 0x7f6c34199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39178640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 0x7f6c34199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47346/0 (socket says 172.21.15.33:47346) 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39178640 1 -- 172.21.15.33:0/899528154 learned_addr learned my addr 172.21.15.33:0/899528154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c38977640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c34101870 0x7f6c3419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34102230 0x7f6c341a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 -- 172.21.15.33:0/899528154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c34101870 msgr2=0x7f6c3419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c34101870 0x7f6c3419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 -- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 msgr2=0x7f6c34199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 0x7f6c34199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 -- 172.21.15.33:0/899528154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c341a18f0 con 0x7f6c34102230 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39178640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 0x7f6c34199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.857+0000 7f6c39979640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34102230 0x7f6c341a1250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009fd0 tx=0x7f6c2c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.858+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00ce20 con 0x7f6c34102230 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.858+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c341a1be0 con 0x7f6c34102230 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.858+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2c005020 con 0x7f6c34102230 2024-04-24T23:33:47.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.858+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c010720 con 0x7f6c34102230 2024-04-24T23:33:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.858+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c341a2100 con 0x7f6c34102230 2024-04-24T23:33:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.859+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c341a2330 con 0x7f6c34102230 2024-04-24T23:33:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.860+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6c2c005540 con 0x7f6c34102230 2024-04-24T23:33:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.861+0000 7f6c2a7fc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6c200a2b90 0x7f6c200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.861+0000 7f6c39178640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6c200a2b90 0x7f6c200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.861+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c2c0cfac0 con 0x7f6c34102230 2024-04-24T23:33:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.861+0000 7f6c39178640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6c200a2b90 0x7f6c200a5050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6c3419ab80 tx=0x7f6c30005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:47.869+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6c2c09c630 con 0x7f6c34102230 2024-04-24T23:33:47.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 200021 22.24 MB/sec execute 352 sec latency 135.363 ms 2024-04-24T23:33:48.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.204+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c341a26c0 con 0x7f6c34102230 2024-04-24T23:33:48.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.205+0000 7f6c2a7fc640 1 -- 172.21.15.33:0/899528154 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f6c2c09bf30 con 0x7f6c34102230 2024-04-24T23:33:48.206 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:33:48.206 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:33:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6c200a2b90 msgr2=0x7f6c200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6c200a2b90 0x7f6c200a5050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6c3419ab80 tx=0x7f6c30005f90 comp rx=0 tx=0).stop 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34102230 msgr2=0x7f6c341a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34102230 0x7f6c341a1250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009fd0 tx=0x7f6c2c00ef10 comp rx=0 tx=0).stop 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 shutdown_connections 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f6c200a2b90 0x7f6c200a5050 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c34102230 0x7f6c341a1250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c34101870 0x7f6c3419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 --2- 172.21.15.33:0/899528154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c34100680 0x7f6c34199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 >> 172.21.15.33:0/899528154 conn(0x7f6c340fbbf0 msgr2=0x7f6c340fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 shutdown_connections 2024-04-24T23:33:48.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.208+0000 7f6c3b3dc640 1 -- 172.21.15.33:0/899528154 wait complete. 2024-04-24T23:33:48.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:33:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 -- 172.21.15.33:0/854264326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0101790 msgr2=0x7f40b0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 --2- 172.21.15.33:0/854264326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0101790 0x7f40b0101bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009900 tx=0x7f40ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 -- 172.21.15.33:0/854264326 shutdown_connections 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 --2- 172.21.15.33:0/854264326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b0103340 0x7f40b0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 --2- 172.21.15.33:0/854264326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0102980 0x7f40b0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 --2- 172.21.15.33:0/854264326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0101790 0x7f40b0101bb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 -- 172.21.15.33:0/854264326 >> 172.21.15.33:0/854264326 conn(0x7f40b00fcd20 msgr2=0x7f40b00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 -- 172.21.15.33:0/854264326 shutdown_connections 2024-04-24T23:33:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.345+0000 7f40b73ab640 1 -- 172.21.15.33:0/854264326 wait complete. 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 Processor -- start 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 -- start start 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0101790 0x7f40b0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 0x7f40b0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b0103340 0x7f40b019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010d450 con 0x7f40b0102980 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010d2d0 con 0x7f40b0103340 2024-04-24T23:33:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b73ab640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010d5d0 con 0x7f40b0101790 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 0x7f40b0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 0x7f40b0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47374/0 (socket says 172.21.15.33:47374) 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 -- 172.21.15.33:0/1569595118 learned_addr learned my addr 172.21.15.33:0/1569595118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b5147640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0101790 0x7f40b0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b5948640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b0103340 0x7f40b019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 -- 172.21.15.33:0/1569595118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0101790 msgr2=0x7f40b0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0101790 0x7f40b0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 -- 172.21.15.33:0/1569595118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b0103340 msgr2=0x7f40b019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b0103340 0x7f40b019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 -- 172.21.15.33:0/1569595118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b019f710 con 0x7f40b0102980 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b5147640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0101790 0x7f40b0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.346+0000 7f40b4946640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 0x7f40b0198030 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e9f0 tx=0x7f40a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.347+0000 7f40a67fc640 1 -- 172.21.15.33:0/1569595118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a000cd80 con 0x7f40b0102980 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.347+0000 7f40a67fc640 1 -- 172.21.15.33:0/1569595118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a0005500 con 0x7f40b0102980 2024-04-24T23:33:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.347+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b019fa00 con 0x7f40b0102980 2024-04-24T23:33:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.347+0000 7f40a67fc640 1 -- 172.21.15.33:0/1569595118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0010690 con 0x7f40b0102980 2024-04-24T23:33:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.347+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b019ff00 con 0x7f40b0102980 2024-04-24T23:33:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.349+0000 7f40a67fc640 1 -- 172.21.15.33:0/1569595118 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f40a00046c0 con 0x7f40b0102980 2024-04-24T23:33:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.349+0000 7f40ba18a740 1 Processor -- start 2024-04-24T23:33:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 -- start start 2024-04-24T23:33:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40a67fc640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f409c0a2b90 0x7f409c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x56307316ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073171b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x563073174630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5630730cc670 con 0x56307316d030 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563073074970 con 0x5630731721a0 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40ba18a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630730c8630 con 0x56307316c640 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b5147640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x56307316ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b5948640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f409c0a2b90 0x7f409c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073171b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40a67fc640 1 -- 172.21.15.33:0/1569595118 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40a00cf810 con 0x7f40b0102980 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b5147640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x56307316ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53106/0 (socket says 172.21.15.33:53106) 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073171b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47378/0 (socket says 172.21.15.33:47378) 2024-04-24T23:33:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b5147640 1 -- 172.21.15.33:0/2479005046 learned_addr learned my addr 172.21.15.33:0/2479005046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b5948640 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x563073174630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 -- 172.21.15.33:0/2479005046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 msgr2=0x56307316ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x56307316ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 -- 172.21.15.33:0/2479005046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 msgr2=0x563073174630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x563073174630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 -- 172.21.15.33:0/2479005046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563073098400 con 0x56307316d030 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.350+0000 7f40b4946640 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073171b30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f40a009b980 tx=0x7f40a009ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ef1b41f4110c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40b5948640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f409c0a2b90 0x7f409c0a5050 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a8009e60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f409b7fe640 1 -- 172.21.15.33:0/2479005046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0061340 con 0x56307316d030 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f409b7fe640 1 -- 172.21.15.33:0/2479005046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a009c380 con 0x56307316d030 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f409b7fe640 1 -- 172.21.15.33:0/2479005046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0066130 con 0x56307316d030 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 -- 172.21.15.33:0/2479005046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 msgr2=0x563073171b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073171b30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f40a009b980 tx=0x7f40a009ba50 comp rx=0 tx=0).stop 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 -- 172.21.15.33:0/2479005046 shutdown_connections 2024-04-24T23:33:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x563073174630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073171b30 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 --2- 172.21.15.33:0/2479005046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x56307316ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.351+0000 7f40ba18a740 1 -- 172.21.15.33:0/2479005046 >> 172.21.15.33:0/2479005046 conn(0x563073167cb0 msgr2=0x5630731680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 -- 172.21.15.33:0/2479005046 shutdown_connections 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 -- 172.21.15.33:0/2479005046 wait complete. 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 -- start start 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 Processor -- start 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x563073176770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073176cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x56307317dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563073175150 con 0x56307316d030 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563073088050 con 0x5630731721a0 2024-04-24T23:33:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630731752d0 con 0x56307316c640 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40ba18a740 10 client.0 _ll_register_callbacks cb 0x563072ddfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40b4946640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073176cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40b4946640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073176cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47386/0 (socket says 172.21.15.33:47386) 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40b4946640 1 -- 172.21.15.33:0/59716944 learned_addr learned my addr 172.21.15.33:0/59716944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40b5948640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x56307317dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.352+0000 7f40b5147640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x563073176770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40b4946640 1 -- 172.21.15.33:0/59716944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 msgr2=0x563073176770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40b4946640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x563073176770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40b4946640 1 -- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 msgr2=0x56307317dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40b4946640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x56307317dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40b4946640 1 -- 172.21.15.33:0/59716944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563073059dc0 con 0x56307316d030 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40b4946640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073176cb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e9c0 tx=0x7f40a009bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993919d102b9d220 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f4098ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f4098ff9640 1 -- 172.21.15.33:0/59716944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a00a0030 con 0x56307316d030 2024-04-24T23:33:48.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f4098ff9640 1 -- 172.21.15.33:0/59716944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40a00cd030 con 0x56307316d030 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a009c380 con 0x56307316d030 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f4098ff9640 1 -- 172.21.15.33:0/59716944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40a00a4460 con 0x56307316d030 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40ba18a740 10 client.19905 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563072de03f0 con 0x56307316d030 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f4098ff9640 1 -- 172.21.15.33:0/59716944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a009c380 con 0x56307316d030 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.353+0000 7f4098ff9640 1 -- 172.21.15.33:0/59716944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f40a009ab40 con 0x56307316d030 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40ba18a740 10 client.19905 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40ba18a740 10 client.19905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40ba18a740 20 client.19905 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40ba18a740 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731821e0 0x5630731845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40ba18a740 4 client.19905 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630730d9bb0 con 0x5630731821e0 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40b5147640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731821e0 0x5630731845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f4098ff9640 10 client.19905 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.354+0000 7f40b5147640 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731821e0 0x5630731845d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=581ff8035528dc81 server_cookie=663945efdb5976d4 in_seq=0 out_seq=0 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f4098ff9640 1 -- 172.21.15.33:0/59716944 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630730d9bb0 con 0x5630731821e0 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f4098ff9640 10 client.19905 handle_command_reply: tid=0 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 client.19905 shutdown 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 msgr2=0x563073176cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073176cb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e9c0 tx=0x7f40a009bb80 comp rx=0 tx=0).stop 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 shutdown_connections 2024-04-24T23:33:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731821e0 0x5630731845d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5630731721a0 0x56307317dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f4098ff9640 0 client.19905 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316d030 0x563073176cb0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 --2- 172.21.15.33:0/59716944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316c640 0x563073176770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 >> 172.21.15.33:0/59716944 conn(0x563073167cb0 msgr2=0x563073168cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 shutdown_connections 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.362+0000 7f40ba18a740 1 -- 172.21.15.33:0/59716944 wait complete. 2024-04-24T23:33:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.363+0000 7f40ba18a740 20 client.19905 trim_cache size 0 max 16384 2024-04-24T23:33:48.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 Processor -- start 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 -- start start 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316c640 0x5630731717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316d030 0x56307317dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 0x563073162b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01aaaa0 con 0x5630731721a0 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a8370 con 0x56307316c640 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40ba18a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a61c0 con 0x56307316d030 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5948640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 0x563073162b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5948640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 0x563073162b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47390/0 (socket says 172.21.15.33:47390) 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5948640 1 -- 172.21.15.33:0/430381748 learned_addr learned my addr 172.21.15.33:0/430381748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316c640 0x5630731717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b4946640 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316d030 0x56307317dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 -- 172.21.15.33:0/430381748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316d030 msgr2=0x56307317dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316d030 0x56307317dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 -- 172.21.15.33:0/430381748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 msgr2=0x563073162b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 0x563073162b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 -- 172.21.15.33:0/430381748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563072eaa7d0 con 0x56307316c640 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5948640 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 0x563073162b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.377+0000 7f40b5147640 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316c640 0x5630731717a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x56307316d890 tx=0x7f40ac031930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412ef1a9dd3ce0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.378+0000 7f409affd640 1 -- 172.21.15.33:0/430381748 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac035040 con 0x56307316c640 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.378+0000 7f409affd640 1 -- 172.21.15.33:0/430381748 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ac04b070 con 0x56307316c640 2024-04-24T23:33:48.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.378+0000 7f409affd640 1 -- 172.21.15.33:0/430381748 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac003970 con 0x56307316c640 2024-04-24T23:33:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- 172.21.15.33:0/430381748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316c640 msgr2=0x5630731717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316c640 0x5630731717a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x56307316d890 tx=0x7f40ac031930 comp rx=0 tx=0).stop 2024-04-24T23:33:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- 172.21.15.33:0/430381748 shutdown_connections 2024-04-24T23:33:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5630731721a0 0x563073162b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307316d030 0x56307317dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- 172.21.15.33:0/430381748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316c640 0x5630731717a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- 172.21.15.33:0/430381748 >> 172.21.15.33:0/430381748 conn(0x563073167cb0 msgr2=0x56307316ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- 172.21.15.33:0/430381748 shutdown_connections 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- 172.21.15.33:0/430381748 wait complete. 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- start start 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 Processor -- start 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316c640 0x56307317b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 0x56307317b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630731721a0 0x56307317c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a6830 con 0x56307316c640 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a7ca0 con 0x56307316d030 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a6f00 con 0x5630731721a0 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.379+0000 7f40ba18a740 10 client.0 _ll_register_callbacks cb 0x7f40a0064070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 0x56307317b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b5147640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316c640 0x56307317b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 0x56307317b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41106/0 (socket says 172.21.15.33:41106) 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b5147640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316c640 0x56307317b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47406/0 (socket says 172.21.15.33:47406) 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b5948640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630731721a0 0x56307317c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 -- 172.21.15.33:0/65115502 learned_addr learned my addr 172.21.15.33:0/65115502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 -- 172.21.15.33:0/65115502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630731721a0 msgr2=0x56307317c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630731721a0 0x56307317c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 -- 172.21.15.33:0/65115502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316c640 msgr2=0x56307317b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316c640 0x56307317b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 -- 172.21.15.33:0/65115502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307317cf00 con 0x56307316d030 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40b4946640 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 0x56307317b900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f40a0061e10 tx=0x7f40a00661e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57abe6c24f00cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f4098ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f4098ff9640 1 -- 172.21.15.33:0/65115502 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0066040 con 0x56307316d030 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40ac04b070 con 0x56307316d030 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f4098ff9640 1 -- 172.21.15.33:0/65115502 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40a00ccfa0 con 0x56307316d030 2024-04-24T23:33:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f4098ff9640 1 -- 172.21.15.33:0/65115502 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40a009c380 con 0x56307316d030 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40ba18a740 10 client.29387 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.380+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56307317cf00 con 0x56307316d030 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f4098ff9640 1 -- 172.21.15.33:0/65115502 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ac04b070 con 0x56307316d030 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f4098ff9640 1 -- 172.21.15.33:0/65115502 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f40a009ab40 con 0x56307316d030 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40ba18a740 10 client.29387 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40ba18a740 10 client.29387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40ba18a740 20 client.29387 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40ba18a740 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731e8bf0 0x5630731eb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40ba18a740 4 client.29387 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630730d3f00 con 0x5630731e8bf0 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40b5147640 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731e8bf0 0x5630731eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f40b5147640 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731e8bf0 0x5630731eb010 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0c39b719118b567 server_cookie=aec6867f1c389e2d in_seq=0 out_seq=0 2024-04-24T23:33:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.381+0000 7f4098ff9640 10 client.29387 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f4098ff9640 1 -- 172.21.15.33:0/65115502 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630730d3f00 con 0x5630731e8bf0 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f4098ff9640 10 client.29387 handle_command_reply: tid=0 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 client.29387 shutdown 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 msgr2=0x56307317b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 0x56307317b900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f40a0061e10 tx=0x7f40a00661e0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 shutdown_connections 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5630731e8bf0 0x5630731eb010 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630731721a0 0x56307317c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56307316d030 0x56307317b900 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 --2- 172.21.15.33:0/65115502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56307316c640 0x56307317b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 >> 172.21.15.33:0/65115502 conn(0x563073167cb0 msgr2=0x5630731629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.384+0000 7f4098ff9640 0 client.29387 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 shutdown_connections 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40ba18a740 1 -- 172.21.15.33:0/65115502 wait complete. 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40ba18a740 20 client.29387 trim_cache size 0 max 16384 2024-04-24T23:33:48.389 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:33:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f409c0a2b90 msgr2=0x7f409c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f409c0a2b90 0x7f409c0a5050 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a8009e60 comp rx=0 tx=0).stop 2024-04-24T23:33:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 msgr2=0x7f40b0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 0x7f40b0198030 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e9f0 tx=0x7f40a000edf0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 shutdown_connections 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f409c0a2b90 0x7f409c0a5050 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b0103340 0x7f40b019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40b0102980 0x7f40b0198030 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 --2- 172.21.15.33:0/1569595118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40b0101790 0x7f40b0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.385+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 >> 172.21.15.33:0/1569595118 conn(0x7f40b00fcd20 msgr2=0x7f40b0105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.386+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 shutdown_connections 2024-04-24T23:33:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.386+0000 7f40b73ab640 1 -- 172.21.15.33:0/1569595118 wait complete. 2024-04-24T23:33:48.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:33:48.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:48 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/899528154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:33:48.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.531+0000 7ff6b23c1640 1 -- 172.21.15.33:0/2376918324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 msgr2=0x7ff6ac10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.531+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/2376918324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 0x7ff6ac10ada0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009ca0 tx=0x7ff6a402a8d0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.539+0000 7ff6b23c1640 1 -- 172.21.15.33:0/2376918324 shutdown_connections 2024-04-24T23:33:48.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.539+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/2376918324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.539+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/2376918324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10b370 0x7ff6ac10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.539+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/2376918324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 0x7ff6ac10ada0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.539+0000 7ff6b23c1640 1 -- 172.21.15.33:0/2376918324 >> 172.21.15.33:0/2376918324 conn(0x7ff6ac06e5d0 msgr2=0x7ff6ac070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.547+0000 7ff6b23c1640 1 -- 172.21.15.33:0/2376918324 shutdown_connections 2024-04-24T23:33:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.549+0000 7ff6b23c1640 1 -- 172.21.15.33:0/2376918324 wait complete. 2024-04-24T23:33:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.549+0000 7ff6b23c1640 1 Processor -- start 2024-04-24T23:33:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 -- start start 2024-04-24T23:33:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 0x7ff6ac19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac10b370 0x7ff6ac19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac113f80 con 0x7ff6ac10b370 2024-04-24T23:33:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac113e00 con 0x7ff6ac1089b0 2024-04-24T23:33:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.550+0000 7ff6b23c1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac114100 con 0x7ff6ac10de30 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.551+0000 7ff6b095e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.551+0000 7ff6b095e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53144/0 (socket says 172.21.15.33:53144) 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.551+0000 7ff6b095e640 1 -- 172.21.15.33:0/1558299695 learned_addr learned my addr 172.21.15.33:0/1558299695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.553+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 0x7ff6ac19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.553+0000 7ff6b095e640 1 -- 172.21.15.33:0/1558299695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 msgr2=0x7ff6ac19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.553+0000 7ff6b095e640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 0x7ff6ac19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.553+0000 7ff6b095e640 1 -- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac10b370 msgr2=0x7ff6ac19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.553+0000 7ff6b095e640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac10b370 0x7ff6ac19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.553+0000 7ff6b095e640 1 -- 172.21.15.33:0/1558299695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac1a5ce0 con 0x7ff6ac10de30 2024-04-24T23:33:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.554+0000 7ff6b095e640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1a5640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b3f0 tx=0x7ff69c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.555+0000 7ff6a97fa640 1 -- 172.21.15.33:0/1558299695 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c015020 con 0x7ff6ac10de30 2024-04-24T23:33:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.555+0000 7ff6a97fa640 1 -- 172.21.15.33:0/1558299695 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c011d10 con 0x7ff6ac10de30 2024-04-24T23:33:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.555+0000 7ff6a97fa640 1 -- 172.21.15.33:0/1558299695 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00f2e0 con 0x7ff6ac10de30 2024-04-24T23:33:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.557+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac1ae8c0 con 0x7ff6ac10de30 2024-04-24T23:33:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.557+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac1aedb0 con 0x7ff6ac10de30 2024-04-24T23:33:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.561+0000 7ff6b51ac740 1 Processor -- start 2024-04-24T23:33:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.561+0000 7ff6b51ac740 1 -- start start 2024-04-24T23:33:48.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.565+0000 7ff6a97fa640 1 -- 172.21.15.33:0/1558299695 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff69c018020 con 0x7ff6ac10de30 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.566+0000 7ff6a97fa640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6940a2d90 0x7ff6940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.566+0000 7ff6a97fa640 1 -- 172.21.15.33:0/1558299695 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff69c0cec80 con 0x7ff6ac10de30 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.566+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff5c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.566+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff5c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff5c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41140/0 (socket says 172.21.15.33:41140) 2024-04-24T23:33:48.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6ab7fe640 1 -- 172.21.15.33:0/496667106 learned_addr learned my addr 172.21.15.33:0/496667106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ec8fec0670 con 0x55ec8ff5cc20 2024-04-24T23:33:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ec8fe68970 con 0x55ec8ff5c230 2024-04-24T23:33:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6b095e640 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.567+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec8febc630 con 0x55ec8ff61d90 2024-04-24T23:33:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.568+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6940a2d90 0x7ff6940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.569+0000 7ff6ab7fe640 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.569+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6940a2d90 0x7ff6940a5250 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009bf0 tx=0x7ff6a4000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.571+0000 7ff6b095e640 1 -- 172.21.15.33:0/496667106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 msgr2=0x55ec8ff5c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.571+0000 7ff6b095e640 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff5c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.571+0000 7ff6b095e640 1 -- 172.21.15.33:0/496667106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 msgr2=0x55ec8ff61720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.571+0000 7ff6b095e640 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff61720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.571+0000 7ff6b095e640 1 -- 172.21.15.33:0/496667106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8fe8c400 con 0x55ec8ff61d90 2024-04-24T23:33:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.573+0000 7ff6b095e640 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff64220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff69c09b080 tx=0x7ff69c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ed1b1df36adefa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.574+0000 7ff6927fc640 1 -- 172.21.15.33:0/496667106 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0b4070 con 0x55ec8ff61d90 2024-04-24T23:33:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.574+0000 7ff6927fc640 1 -- 172.21.15.33:0/496667106 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c09b0f0 con 0x55ec8ff61d90 2024-04-24T23:33:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.574+0000 7ff6927fc640 1 -- 172.21.15.33:0/496667106 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0bb0b0 con 0x55ec8ff61d90 2024-04-24T23:33:48.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.577+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 msgr2=0x55ec8ff64220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.578+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff64220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff69c09b080 tx=0x7ff69c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.578+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 shutdown_connections 2024-04-24T23:33:48.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.578+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff64220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.578+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff61720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.578+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/496667106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff5c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.578+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 >> 172.21.15.33:0/496667106 conn(0x55ec8ff578a0 msgr2=0x55ec8ff57ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.579+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 shutdown_connections 2024-04-24T23:33:48.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.579+0000 7ff6b51ac740 1 -- 172.21.15.33:0/496667106 wait complete. 2024-04-24T23:33:48.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.579+0000 7ff6b51ac740 1 -- start start 2024-04-24T23:33:48.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.593+0000 7ff6b51ac740 1 Processor -- start 2024-04-24T23:33:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.593+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.594+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff66800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.594+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.594+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ec8ff64d40 con 0x55ec8ff5cc20 2024-04-24T23:33:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.594+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ec8fe7c050 con 0x55ec8ff5c230 2024-04-24T23:33:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.594+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec8ff64ec0 con 0x55ec8ff61d90 2024-04-24T23:33:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.594+0000 7ff6b51ac740 10 client.0 _ll_register_callbacks cb 0x55ec8fbd3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.595+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff66800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.596+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff66800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47434/0 (socket says 172.21.15.33:47434) 2024-04-24T23:33:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.596+0000 7ff6ab7fe640 1 -- 172.21.15.33:0/780755191 learned_addr learned my addr 172.21.15.33:0/780755191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.597+0000 7ff6b095e640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.597+0000 7ff6abfff640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.602+0000 7ff6b095e640 1 -- 172.21.15.33:0/780755191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 msgr2=0x55ec8ff662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.602+0000 7ff6b095e640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.602+0000 7ff6b095e640 1 -- 172.21.15.33:0/780755191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 msgr2=0x55ec8ff66800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.602+0000 7ff6b095e640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff66800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.602+0000 7ff6b095e640 1 -- 172.21.15.33:0/780755191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8fe4ddc0 con 0x55ec8ff61d90 2024-04-24T23:33:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.610+0000 7ff6b095e640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff6d840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff69c09b080 tx=0x7ff69c09f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3db2ebe8653b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.611+0000 7ff68b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:33:48.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.616+0000 7ff68b7fe640 1 -- 172.21.15.33:0/780755191 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c09f4a0 con 0x55ec8ff61d90 2024-04-24T23:33:48.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.616+0000 7ff68b7fe640 1 -- 172.21.15.33:0/780755191 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff69c097760 con 0x55ec8ff61d90 2024-04-24T23:33:48.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.617+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff69c09b0f0 con 0x55ec8ff61d90 2024-04-24T23:33:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.624+0000 7ff68b7fe640 1 -- 172.21.15.33:0/780755191 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff69c0a38d0 con 0x55ec8ff61d90 2024-04-24T23:33:48.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.628+0000 7ff6b51ac740 10 client.29338 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:48.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.628+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ec8fbd43f0 con 0x55ec8ff61d90 2024-04-24T23:33:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff68b7fe640 1 -- 172.21.15.33:0/780755191 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c09b0f0 con 0x55ec8ff61d90 2024-04-24T23:33:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff68b7fe640 1 -- 172.21.15.33:0/780755191 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff69c0b5070 con 0x55ec8ff61d90 2024-04-24T23:33:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff6b51ac740 10 client.29338 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff6b51ac740 10 client.29338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:33:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff6b51ac740 20 client.29338 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff71d30 0x55ec8ff74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff6b51ac740 4 client.29338 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.636+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec8fecdbb0 con 0x55ec8ff71d30 2024-04-24T23:33:48.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.638+0000 7ff6abfff640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff71d30 0x55ec8ff74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.650+0000 7ff6abfff640 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff71d30 0x55ec8ff74120 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d588d094bc50100 server_cookie=6415c51c122ac20 in_seq=0 out_seq=0 2024-04-24T23:33:48.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.650+0000 7ff68b7fe640 10 client.29338 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:48.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.655+0000 7ff68b7fe640 1 -- 172.21.15.33:0/780755191 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec8fecdbb0 con 0x55ec8ff71d30 2024-04-24T23:33:48.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.655+0000 7ff68b7fe640 10 client.29338 handle_command_reply: tid=0 2024-04-24T23:33:48.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.656+0000 7ff6b51ac740 1 client.29338 shutdown 2024-04-24T23:33:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.664+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 msgr2=0x55ec8ff6d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.664+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff6d840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff69c09b080 tx=0x7ff69c09f3a0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:48 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/899528154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:33:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 shutdown_connections 2024-04-24T23:33:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff71d30 0x55ec8ff74120 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff61d90 0x55ec8ff6d840 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff66800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/780755191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5c230 0x55ec8ff662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 >> 172.21.15.33:0/780755191 conn(0x55ec8ff578a0 msgr2=0x55ec8ff56b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.670+0000 7ff68b7fe640 0 client.29338 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.673+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 shutdown_connections 2024-04-24T23:33:48.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:48 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/899528154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:33:48.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.677+0000 7ff6b51ac740 1 -- 172.21.15.33:0/780755191 wait complete. 2024-04-24T23:33:48.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.679+0000 7ff6b51ac740 20 client.29338 trim_cache size 0 max 16384 2024-04-24T23:33:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.711+0000 7ff6b51ac740 1 Processor -- start 2024-04-24T23:33:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.713+0000 7ff6b51ac740 1 -- start start 2024-04-24T23:33:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.713+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff578a0 0x55ec8ff5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.713+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1bbc00 con 0x55ec8ff5cc20 2024-04-24T23:33:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1b94d0 con 0x55ec8ff578a0 2024-04-24T23:33:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1b7320 con 0x55ec8ff5c230 2024-04-24T23:33:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53166/0 (socket says 172.21.15.33:53166) 2024-04-24T23:33:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.714+0000 7ff6ab7fe640 1 -- 172.21.15.33:0/1874695449 learned_addr learned my addr 172.21.15.33:0/1874695449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.715+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff578a0 0x55ec8ff5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.715+0000 7ff6b095e640 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.717+0000 7ff6abfff640 1 -- 172.21.15.33:0/1874695449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff5c230 msgr2=0x55ec8ff6d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.717+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.717+0000 7ff6abfff640 1 -- 172.21.15.33:0/1874695449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 msgr2=0x55ec8ff569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.717+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.717+0000 7ff6abfff640 1 -- 172.21.15.33:0/1874695449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8fc9e7d0 con 0x55ec8ff578a0 2024-04-24T23:33:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.722+0000 7ff6abfff640 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff578a0 0x55ec8ff5b040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ec8ff58770 tx=0x7ff6a403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94af4b60ae40d119 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.725+0000 7ff691ffb640 1 -- 172.21.15.33:0/1874695449 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4031d10 con 0x55ec8ff578a0 2024-04-24T23:33:48.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.725+0000 7ff691ffb640 1 -- 172.21.15.33:0/1874695449 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a4057070 con 0x55ec8ff578a0 2024-04-24T23:33:48.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.725+0000 7ff691ffb640 1 -- 172.21.15.33:0/1874695449 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4052740 con 0x55ec8ff578a0 2024-04-24T23:33:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.727+0000 7ff6b51ac740 1 -- 172.21.15.33:0/1874695449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff578a0 msgr2=0x55ec8ff5b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.727+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff578a0 0x55ec8ff5b040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ec8ff58770 tx=0x7ff6a403e000 comp rx=0 tx=0).stop 2024-04-24T23:33:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.731+0000 7ff6b51ac740 1 -- 172.21.15.33:0/1874695449 shutdown_connections 2024-04-24T23:33:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.731+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.731+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.731+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/1874695449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff578a0 0x55ec8ff5b040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.731+0000 7ff6b51ac740 1 -- 172.21.15.33:0/1874695449 >> 172.21.15.33:0/1874695449 conn(0x55ec8ff742e0 msgr2=0x55ec8ff562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.737+0000 7ff6b51ac740 1 -- 172.21.15.33:0/1874695449 shutdown_connections 2024-04-24T23:33:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.738+0000 7ff6b51ac740 1 -- 172.21.15.33:0/1874695449 wait complete. 2024-04-24T23:33:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.738+0000 7ff6b51ac740 1 -- start start 2024-04-24T23:33:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.739+0000 7ff6b51ac740 1 Processor -- start 2024-04-24T23:33:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.739+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff578a0 0x55ec8ff6ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.739+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6b51ac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff6c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1b7990 con 0x55ec8ff5c230 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1b8e00 con 0x55ec8ff5cc20 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6b51ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1b8060 con 0x55ec8ff578a0 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6b51ac740 10 client.0 _ll_register_callbacks cb 0x7ff69c0b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47452/0 (socket says 172.21.15.33:47452) 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6ab7fe640 1 -- 172.21.15.33:0/617104561 learned_addr learned my addr 172.21.15.33:0/617104561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.740+0000 7ff6b095e640 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff6c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.742+0000 7ff6b095e640 1 -- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff578a0 msgr2=0x55ec8ff6ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:33:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.742+0000 7ff6b095e640 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff578a0 0x55ec8ff6ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.742+0000 7ff6b095e640 1 -- 172.21.15.33:0/617104561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5c230 msgr2=0x55ec8ff6b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.742+0000 7ff6b095e640 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.742+0000 7ff6b095e640 1 -- 172.21.15.33:0/617104561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8ff6c8a0 con 0x55ec8ff5cc20 2024-04-24T23:33:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.746+0000 7ff6b095e640 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff6c2b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b3c0 tx=0x7ff69c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80c87453cdac6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:48.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.748+0000 7ff68b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:48.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.748+0000 7ff68b7fe640 1 -- 172.21.15.33:0/617104561 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0b4040 con 0x55ec8ff5cc20 2024-04-24T23:33:48.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.748+0000 7ff68b7fe640 1 -- 172.21.15.33:0/617104561 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff69c09a540 con 0x55ec8ff5cc20 2024-04-24T23:33:48.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.748+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a4057070 con 0x55ec8ff5cc20 2024-04-24T23:33:48.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.750+0000 7ff68b7fe640 1 -- 172.21.15.33:0/617104561 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff69c09f4a0 con 0x55ec8ff5cc20 2024-04-24T23:33:48.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.751+0000 7ff6b51ac740 10 client.29405 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:48.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.751+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ec8ff6c8a0 con 0x55ec8ff5cc20 2024-04-24T23:33:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.755+0000 7ff68b7fe640 1 -- 172.21.15.33:0/617104561 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a4057070 con 0x55ec8ff5cc20 2024-04-24T23:33:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.756+0000 7ff68b7fe640 1 -- 172.21.15.33:0/617104561 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff69c0b5070 con 0x55ec8ff5cc20 2024-04-24T23:33:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6b51ac740 10 client.29405 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6b51ac740 10 client.29405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:33:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6b51ac740 20 client.29405 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff61d90 0x55ec8ffda460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6b51ac740 4 client.29405 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:33:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec8fec7f00 con 0x55ec8ff61d90 2024-04-24T23:33:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.758+0000 7ff6abfff640 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff61d90 0x55ec8ffda460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:48.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.761+0000 7ff6abfff640 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff61d90 0x55ec8ffda460 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=546ec21f955b98cf server_cookie=c240b7064460517a in_seq=0 out_seq=0 2024-04-24T23:33:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.762+0000 7ff68b7fe640 10 client.29405 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:48.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.762+0000 7ff68b7fe640 1 -- 172.21.15.33:0/617104561 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec8fec7f00 con 0x55ec8ff61d90 2024-04-24T23:33:48.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.762+0000 7ff68b7fe640 10 client.29405 handle_command_reply: tid=0 2024-04-24T23:33:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.765+0000 7ff6b51ac740 1 client.29405 shutdown 2024-04-24T23:33:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.770+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5cc20 msgr2=0x55ec8ff6c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.770+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff6c2b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b3c0 tx=0x7ff69c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 shutdown_connections 2024-04-24T23:33:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55ec8ff61d90 0x55ec8ffda460 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec8ff5cc20 0x55ec8ff6c2b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec8ff5c230 0x55ec8ff6b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff6b51ac740 1 --2- 172.21.15.33:0/617104561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec8ff578a0 0x55ec8ff6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 >> 172.21.15.33:0/617104561 conn(0x55ec8ff742e0 msgr2=0x55ec8ff56b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.778+0000 7ff68b7fe640 0 client.29405 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:33:48.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.785+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 shutdown_connections 2024-04-24T23:33:48.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.788+0000 7ff6b51ac740 1 -- 172.21.15.33:0/617104561 wait complete. 2024-04-24T23:33:48.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.789+0000 7ff6b51ac740 20 client.29405 trim_cache size 0 max 16384 2024-04-24T23:33:48.790 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.792+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6940a2d90 msgr2=0x7ff6940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.792+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6940a2d90 0x7ff6940a5250 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009bf0 tx=0x7ff6a4000f40 comp rx=0 tx=0).stop 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.792+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 msgr2=0x7ff6ac1a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.792+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1a5640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b3f0 tx=0x7ff69c00b8b0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.793+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 shutdown_connections 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.793+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff6940a2d90 0x7ff6940a5250 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.793+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6ac10de30 0x7ff6ac1a5640 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.793+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac10b370 0x7ff6ac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.793+0000 7ff6b23c1640 1 --2- 172.21.15.33:0/1558299695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6ac1089b0 0x7ff6ac19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.793+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 >> 172.21.15.33:0/1558299695 conn(0x7ff6ac06e5d0 msgr2=0x7ff6ac10ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.796+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 shutdown_connections 2024-04-24T23:33:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:48.796+0000 7ff6b23c1640 1 -- 172.21.15.33:0/1558299695 wait complete. 2024-04-24T23:33:48.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:33:48.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 200302 22.21 MB/sec execute 353 sec latency 364.599 ms 2024-04-24T23:33:49.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.005+0000 7fe9851af640 1 -- 172.21.15.33:0/3069196182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978032e30 con 0x7fe9801062f0 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.006+0000 7fe988415640 1 -- 172.21.15.33:0/3069196182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 msgr2=0x7fe9801087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.006+0000 7fe988415640 1 --2- 172.21.15.33:0/3069196182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 0x7fe9801087b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe978009be0 tx=0x7fe97802c650 comp rx=0 tx=0).stop 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.007+0000 7fe988415640 1 -- 172.21.15.33:0/3069196182 shutdown_connections 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.007+0000 7fe988415640 1 --2- 172.21.15.33:0/3069196182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 0x7fe9801087b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.007+0000 7fe988415640 1 --2- 172.21.15.33:0/3069196182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe980103850 0x7fe980105c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.007+0000 7fe988415640 1 --2- 172.21.15.33:0/3069196182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.007+0000 7fe988415640 1 -- 172.21.15.33:0/3069196182 >> 172.21.15.33:0/3069196182 conn(0x7fe9800faf40 msgr2=0x7fe9800fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.007+0000 7fe988415640 1 -- 172.21.15.33:0/3069196182 shutdown_connections 2024-04-24T23:33:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.008+0000 7fe988415640 1 -- 172.21.15.33:0/3069196182 wait complete. 2024-04-24T23:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 Processor -- start 2024-04-24T23:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 -- start start 2024-04-24T23:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe980103850 0x7fe980195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 0x7fe98019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe98010b1b0 con 0x7fe980103850 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe98010b030 con 0x7fe980100f20 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.009+0000 7fe988415640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe98010b330 con 0x7fe9801062f0 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41192/0 (socket says 172.21.15.33:41192) 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 -- 172.21.15.33:0/1483135074 learned_addr learned my addr 172.21.15.33:0/1483135074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 -- 172.21.15.33:0/1483135074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 msgr2=0x7fe98019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 0x7fe98019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 -- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe980103850 msgr2=0x7fe980195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe980103850 0x7fe980195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.010+0000 7fe9861b1640 1 -- 172.21.15.33:0/1483135074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9801ae570 con 0x7fe980100f20 2024-04-24T23:33:49.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.012+0000 7fe9861b1640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980195910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9f0 tx=0x7fe97c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.013+0000 7fe9777fe640 1 -- 172.21.15.33:0/1483135074 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00cd80 con 0x7fe980100f20 2024-04-24T23:33:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.013+0000 7fe9777fe640 1 -- 172.21.15.33:0/1483135074 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c010910 con 0x7fe980100f20 2024-04-24T23:33:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.013+0000 7fe9777fe640 1 -- 172.21.15.33:0/1483135074 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0188f0 con 0x7fe980100f20 2024-04-24T23:33:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.014+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9801ae860 con 0x7fe980100f20 2024-04-24T23:33:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.014+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9801aed60 con 0x7fe980100f20 2024-04-24T23:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.018+0000 7fe98b201740 1 Processor -- start 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.018+0000 7fe98b201740 1 -- start start 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe98b201740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c668b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe98b201740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668bc20 0x55c2c6690720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe98b201740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c6690d90 0x55c2c6693220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe98b201740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c2c65ef670 con 0x55c2c668bc20 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe98b201740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c2c6597970 con 0x55c2c668b230 2024-04-24T23:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe98b201740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2c65eb630 con 0x55c2c6690d90 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c668b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9859b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c6690d90 0x55c2c6693220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c668b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41200/0 (socket says 172.21.15.33:41200) 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 -- 172.21.15.33:0/163675086 learned_addr learned my addr 172.21.15.33:0/163675086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9777fe640 1 -- 172.21.15.33:0/1483135074 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe97c010ac0 con 0x7fe980100f20 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9861b1640 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668bc20 0x55c2c6690720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 -- 172.21.15.33:0/163675086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c6690d90 msgr2=0x55c2c6693220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c6690d90 0x55c2c6693220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 -- 172.21.15.33:0/163675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668bc20 msgr2=0x55c2c6690720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668bc20 0x55c2c6690720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9869b2640 1 -- 172.21.15.33:0/163675086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c65bb400 con 0x55c2c668b230 2024-04-24T23:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9859b0640 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c6690d90 0x55c2c6693220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.019+0000 7fe9861b1640 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668bc20 0x55c2c6690720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.020+0000 7fe9777fe640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe96c0a2c90 0x7fe96c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.021+0000 7fe9777fe640 1 -- 172.21.15.33:0/1483135074 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe97c0cfa40 con 0x7fe980100f20 2024-04-24T23:33:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.021+0000 7fe9861b1640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe96c0a2c90 0x7fe96c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.021+0000 7fe9869b2640 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c668b650 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe97802cb70 tx=0x7fe978005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a4924d42704bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.022+0000 7fe974bfe640 1 -- 172.21.15.33:0/163675086 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978038050 con 0x55c2c668b230 2024-04-24T23:33:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.022+0000 7fe974bfe640 1 -- 172.21.15.33:0/163675086 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978042070 con 0x55c2c668b230 2024-04-24T23:33:49.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.022+0000 7fe974bfe640 1 -- 172.21.15.33:0/163675086 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978004040 con 0x55c2c668b230 2024-04-24T23:33:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.024+0000 7fe98b201740 1 -- 172.21.15.33:0/163675086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 msgr2=0x55c2c668b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.024+0000 7fe98b201740 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c668b650 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe97802cb70 tx=0x7fe978005bf0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.024+0000 7fe9861b1640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe96c0a2c90 0x7fe96c0a5150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0a45f0 tx=0x7fe97c009700 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.025+0000 7fe98b201740 1 -- 172.21.15.33:0/163675086 shutdown_connections 2024-04-24T23:33:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.025+0000 7fe98b201740 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c6690d90 0x55c2c6693220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.025+0000 7fe98b201740 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668bc20 0x55c2c6690720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.025+0000 7fe98b201740 1 --2- 172.21.15.33:0/163675086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c668b650 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.025+0000 7fe98b201740 1 -- 172.21.15.33:0/163675086 >> 172.21.15.33:0/163675086 conn(0x55c2c66868a0 msgr2=0x55c2c6686ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:49.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.026+0000 7fe98b201740 1 -- 172.21.15.33:0/163675086 shutdown_connections 2024-04-24T23:33:49.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.026+0000 7fe98b201740 1 -- 172.21.15.33:0/163675086 wait complete. 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.026+0000 7fe98b201740 1 -- start start 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.026+0000 7fe98b201740 1 Processor -- start 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c669c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c2c6693ce0 con 0x55c2c6690d90 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c2c65ab050 con 0x55c2c668bc20 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2c6693e60 con 0x55c2c668b230 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe98b201740 10 client.0 _ll_register_callbacks cb 0x55c2c6302be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9869b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c669c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41206/0 (socket says 172.21.15.33:41206) 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9869b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c669c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47498/0 (socket says 172.21.15.33:47498) 2024-04-24T23:33:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 -- 172.21.15.33:0/1103495785 learned_addr learned my addr 172.21.15.33:0/1103495785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 -- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668b230 msgr2=0x55c2c66953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 -- 172.21.15.33:0/1103495785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 msgr2=0x55c2c669c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c669c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 -- 172.21.15.33:0/1103495785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c657cdc0 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.027+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9700096c0 tx=0x7fe970011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d62d95873cd79a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe969ffb640 1 -- 172.21.15.33:0/1103495785 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97000fd70 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe969ffb640 1 -- 172.21.15.33:0/1103495785 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe970004030 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe978042070 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe969ffb640 1 -- 172.21.15.33:0/1103495785 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe970024050 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe98b201740 10 client.29417 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c2c63033f0 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe969ffb640 1 -- 172.21.15.33:0/1103495785 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978042070 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe969ffb640 1 -- 172.21.15.33:0/1103495785 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe970019420 con 0x55c2c668bc20 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe98b201740 10 client.29417 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe98b201740 10 client.29417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.028+0000 7fe98b201740 20 client.29417 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.029+0000 7fe98b201740 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c66a1250 0x55c2c66a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.029+0000 7fe98b201740 4 client.29417 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.029+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c65fcbb0 con 0x55c2c66a1250 2024-04-24T23:33:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.029+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c66a1250 0x55c2c66a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.030+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c66a1250 0x55c2c66a3670 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875a337c486dac39 server_cookie=e43797d26ac614ee in_seq=0 out_seq=0 2024-04-24T23:33:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.030+0000 7fe969ffb640 10 client.29417 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe969ffb640 1 -- 172.21.15.33:0/1103495785 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2c65fcbb0 con 0x55c2c66a1250 2024-04-24T23:33:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe969ffb640 10 client.29417 handle_command_reply: tid=0 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 client.29417 shutdown 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 msgr2=0x55c2c6695910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9700096c0 tx=0x7fe970011b50 comp rx=0 tx=0).stop 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 shutdown_connections 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c66a1250 0x55c2c66a3670 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c669c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 --2- 172.21.15.33:0/1103495785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.032+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 >> 172.21.15.33:0/1103495785 conn(0x55c2c66868a0 msgr2=0x55c2c6685b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.033+0000 7fe969ffb640 0 client.29417 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.033+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 shutdown_connections 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.033+0000 7fe98b201740 1 -- 172.21.15.33:0/1103495785 wait complete. 2024-04-24T23:33:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.033+0000 7fe98b201740 20 client.29417 trim_cache size 0 max 16384 2024-04-24T23:33:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.070+0000 7fe98b201740 1 Processor -- start 2024-04-24T23:33:49.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.074+0000 7fe98b201740 1 -- start start 2024-04-24T23:33:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.074+0000 7fe98b201740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.074+0000 7fe98b201740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.074+0000 7fe98b201740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c6690d90 0x55c2c669c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.074+0000 7fe98b201740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9801bbb30 con 0x55c2c668b230 2024-04-24T23:33:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.074+0000 7fe98b201740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9801b9400 con 0x55c2c6690d90 2024-04-24T23:33:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.075+0000 7fe98b201740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9801b7250 con 0x55c2c668bc20 2024-04-24T23:33:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.077+0000 7fe9859b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.077+0000 7fe9859b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53222/0 (socket says 172.21.15.33:53222) 2024-04-24T23:33:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.077+0000 7fe9859b0640 1 -- 172.21.15.33:0/2235537928 learned_addr learned my addr 172.21.15.33:0/2235537928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9859b0640 1 -- 172.21.15.33:0/2235537928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c6690d90 msgr2=0x55c2c669c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9869b2640 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c6690d90 0x55c2c669c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9869b2640 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9859b0640 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c6690d90 0x55c2c669c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9859b0640 1 -- 172.21.15.33:0/2235537928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668b230 msgr2=0x55c2c66953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9859b0640 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.078+0000 7fe9859b0640 1 -- 172.21.15.33:0/2235537928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c63cd7d0 con 0x55c2c668bc20 2024-04-24T23:33:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.079+0000 7fe9869b2640 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:33:49.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.079+0000 7fe9859b0640 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe970005ce0 tx=0x7fe970005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296f59b67519000e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:49.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.080+0000 7fe96bfff640 1 -- 172.21.15.33:0/2235537928 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970019e60 con 0x55c2c668bc20 2024-04-24T23:33:49.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.080+0000 7fe96bfff640 1 -- 172.21.15.33:0/2235537928 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe970019420 con 0x55c2c668bc20 2024-04-24T23:33:49.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.080+0000 7fe96bfff640 1 -- 172.21.15.33:0/2235537928 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970030bf0 con 0x55c2c668bc20 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 -- 172.21.15.33:0/2235537928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 msgr2=0x55c2c6695910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe970005ce0 tx=0x7fe970005890 comp rx=0 tx=0).stop 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 -- 172.21.15.33:0/2235537928 shutdown_connections 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c6690d90 0x55c2c669c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6695910 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 --2- 172.21.15.33:0/2235537928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c668b230 0x55c2c66953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 -- 172.21.15.33:0/2235537928 >> 172.21.15.33:0/2235537928 conn(0x55c2c66868a0 msgr2=0x55c2c66949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 -- 172.21.15.33:0/2235537928 shutdown_connections 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 -- 172.21.15.33:0/2235537928 wait complete. 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.081+0000 7fe98b201740 1 -- start start 2024-04-24T23:33:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 Processor -- start 2024-04-24T23:33:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c66995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6699ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c67057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9801b78c0 con 0x55c2c6690d90 2024-04-24T23:33:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9801b8d30 con 0x55c2c668b230 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe98b201740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9801b7f90 con 0x55c2c668bc20 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.082+0000 7fe9869b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c66995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe98b201740 10 client.0 _ll_register_callbacks cb 0x7fe970009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c66995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41234/0 (socket says 172.21.15.33:41234) 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 -- 172.21.15.33:0/1834909562 learned_addr learned my addr 172.21.15.33:0/1834909562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6699ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c67057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 -- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 msgr2=0x55c2c6699ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6699ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 -- 172.21.15.33:0/1834909562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 msgr2=0x55c2c67057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c67057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 -- 172.21.15.33:0/1834909562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c669cb60 con 0x55c2c668b230 2024-04-24T23:33:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6699ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:33:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.083+0000 7fe9869b2640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c66995a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe978032960 tx=0x7fe97804b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393ee08ab914c733 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.084+0000 7fe969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.089+0000 7fe969ffb640 1 -- 172.21.15.33:0/1834909562 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978038070 con 0x55c2c668b230 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.089+0000 7fe969ffb640 1 -- 172.21.15.33:0/1834909562 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe978057ae0 con 0x55c2c668b230 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.089+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe970019420 con 0x55c2c668b230 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.089+0000 7fe969ffb640 1 -- 172.21.15.33:0/1834909562 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe978003ae0 con 0x55c2c668b230 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.089+0000 7fe98b201740 10 client.29420 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.089+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c2c6706ca0 con 0x55c2c668b230 2024-04-24T23:33:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe969ffb640 1 -- 172.21.15.33:0/1834909562 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe970019420 con 0x55c2c668b230 2024-04-24T23:33:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe969ffb640 1 -- 172.21.15.33:0/1834909562 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe978057ce0 con 0x55c2c668b230 2024-04-24T23:33:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe98b201740 10 client.29420 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe98b201740 10 client.29420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe98b201740 20 client.29420 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe98b201740 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c67071e0 0x55c2c6709600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe98b201740 4 client.29420 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:33:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.090+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2c65f6f00 con 0x55c2c67071e0 2024-04-24T23:33:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.091+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c67071e0 0x55c2c6709600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.091+0000 7fe9859b0640 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c67071e0 0x55c2c6709600 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e07b5d3e05ee93 server_cookie=a99bd50cc513b647 in_seq=0 out_seq=0 2024-04-24T23:33:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.091+0000 7fe969ffb640 10 client.29420 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe969ffb640 1 -- 172.21.15.33:0/1834909562 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2c65f6f00 con 0x55c2c67071e0 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe969ffb640 10 client.29420 handle_command_reply: tid=0 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 client.29420 shutdown 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 msgr2=0x55c2c66995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c66995a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe978032960 tx=0x7fe97804b040 comp rx=0 tx=0).stop 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 shutdown_connections 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2c67071e0 0x55c2c6709600 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2c6690d90 0x55c2c67057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2c668bc20 0x55c2c6699ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 --2- 172.21.15.33:0/1834909562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c2c668b230 0x55c2c66995a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 >> 172.21.15.33:0/1834909562 conn(0x55c2c66868a0 msgr2=0x55c2c6685b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe969ffb640 0 client.29420 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 shutdown_connections 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 1 -- 172.21.15.33:0/1834909562 wait complete. 2024-04-24T23:33:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.092+0000 7fe98b201740 20 client.29420 trim_cache size 0 max 16384 2024-04-24T23:33:49.095 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe96c0a2c90 msgr2=0x7fe96c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe96c0a2c90 0x7fe96c0a5150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0a45f0 tx=0x7fe97c009700 comp rx=0 tx=0).stop 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 msgr2=0x7fe980195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980195910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9f0 tx=0x7fe97c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 shutdown_connections 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe96c0a2c90 0x7fe96c0a5150 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9801062f0 0x7fe98019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe980103850 0x7fe980195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 --2- 172.21.15.33:0/1483135074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe980100f20 0x7fe980195910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.094+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 >> 172.21.15.33:0/1483135074 conn(0x7fe9800faf40 msgr2=0x7fe9801018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.095+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 shutdown_connections 2024-04-24T23:33:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:49.095+0000 7fe988415640 1 -- 172.21.15.33:0/1483135074 wait complete. 2024-04-24T23:33:49.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:49 smithi033 ceph-mon[29201]: pgmap v1005: 129 pgs: 129 active+clean; 616 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.5 MiB/s wr, 378 op/s 2024-04-24T23:33:49.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:49 smithi117 ceph-mon[32529]: pgmap v1005: 129 pgs: 129 active+clean; 616 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.5 MiB/s wr, 378 op/s 2024-04-24T23:33:49.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:49 smithi038 ceph-mon[32450]: pgmap v1005: 129 pgs: 129 active+clean; 616 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.5 MiB/s wr, 378 op/s 2024-04-24T23:33:49.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 200688 22.21 MB/sec execute 354 sec latency 483.589 ms 2024-04-24T23:33:50.113 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:33:50.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:33:50.113 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:33:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.291+0000 7f8eedffb640 1 -- 172.21.15.33:0/19028865 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8032930 con 0x7f8ef0102250 2024-04-24T23:33:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.291+0000 7f8ef5a15640 1 -- 172.21.15.33:0/19028865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0102250 msgr2=0x7f8ef0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.291+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/19028865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0102250 0x7f8ef0108ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800bc00 tx=0x7f8ee8009260 comp rx=0 tx=0).stop 2024-04-24T23:33:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.293+0000 7f8ef5a15640 1 -- 172.21.15.33:0/19028865 shutdown_connections 2024-04-24T23:33:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.293+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/19028865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0102250 0x7f8ef0108ae0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.293+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/19028865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0101890 0x7f8ef0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.293+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/19028865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef01006a0 0x7f8ef0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.293+0000 7f8ef5a15640 1 -- 172.21.15.33:0/19028865 >> 172.21.15.33:0/19028865 conn(0x7f8ef00fbc30 msgr2=0x7f8ef00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.297+0000 7f8ef5a15640 1 -- 172.21.15.33:0/19028865 shutdown_connections 2024-04-24T23:33:50.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.298+0000 7f8ef5a15640 1 -- 172.21.15.33:0/19028865 wait complete. 2024-04-24T23:33:50.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.298+0000 7f8ef5a15640 1 Processor -- start 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 -- start start 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01006a0 0x7f8ef0070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101890 0x7f8ef0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0102250 0x7f8ef01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010b2d0 con 0x7f8ef01006a0 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010b150 con 0x7f8ef0102250 2024-04-24T23:33:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.299+0000 7f8ef5a15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010b450 con 0x7f8ef0101890 2024-04-24T23:33:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.300+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0102250 0x7f8ef01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.300+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01006a0 0x7f8ef0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.300+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0102250 0x7f8ef01a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41248/0 (socket says 172.21.15.33:41248) 2024-04-24T23:33:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.300+0000 7f8eef7fe640 1 -- 172.21.15.33:0/243261732 learned_addr learned my addr 172.21.15.33:0/243261732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.300+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101890 0x7f8ef0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.301+0000 7f8eee7fc640 1 -- 172.21.15.33:0/243261732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0102250 msgr2=0x7f8ef01a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.301+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0102250 0x7f8ef01a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.301+0000 7f8eee7fc640 1 -- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01006a0 msgr2=0x7f8ef0070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.301+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01006a0 0x7f8ef0070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.301+0000 7f8eee7fc640 1 -- 172.21.15.33:0/243261732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef00736e0 con 0x7f8ef0101890 2024-04-24T23:33:50.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.303+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101890 0x7f8ef0070560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40097d0 tx=0x7f8ee4009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.305+0000 7f8ef4a13640 1 -- 172.21.15.33:0/243261732 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee401c070 con 0x7f8ef0101890 2024-04-24T23:33:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.305+0000 7f8ef4a13640 1 -- 172.21.15.33:0/243261732 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee4017410 con 0x7f8ef0101890 2024-04-24T23:33:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.305+0000 7f8ef4a13640 1 -- 172.21.15.33:0/243261732 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400c5b0 con 0x7f8ef0101890 2024-04-24T23:33:50.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.305+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a5f80 con 0x7f8ef0101890 2024-04-24T23:33:50.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.305+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a6440 con 0x7f8ef0101890 2024-04-24T23:33:50.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.310+0000 7f8ef8800740 1 Processor -- start 2024-04-24T23:33:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.310+0000 7f8ef8800740 1 -- start start 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.310+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.310+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628247780 0x55f62824c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.311+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62824c8f0 0x55f62824ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.311+0000 7f8ef8800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f6281aa2c0 con 0x55f628246d90 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.311+0000 7f8ef8800740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f628152910 con 0x55f62824c8f0 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.314+0000 7f8ef8800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f6281a6280 con 0x55f628247780 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.311+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.314+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47526/0 (socket says 172.21.15.33:47526) 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.314+0000 7f8eef7fe640 1 -- 172.21.15.33:0/1665230988 learned_addr learned my addr 172.21.15.33:0/1665230988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.313+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628247780 0x55f62824c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8ef4a13640 1 -- 172.21.15.33:0/243261732 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8ee401f070 con 0x7f8ef0101890 2024-04-24T23:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8eef7fe640 1 -- 172.21.15.33:0/1665230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628247780 msgr2=0x55f62824c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8eef7fe640 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628247780 0x55f62824c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8eef7fe640 1 -- 172.21.15.33:0/1665230988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62824c8f0 msgr2=0x55f62824ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8eef7fe640 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62824c8f0 0x55f62824ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8eef7fe640 1 -- 172.21.15.33:0/1665230988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f628163fb0 con 0x55f628246d90 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.315+0000 7f8eef7fe640 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282471b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80325d0 tx=0x7f8ee800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc45ff297de7c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.316+0000 7f8ed5ffb640 1 -- 172.21.15.33:0/1665230988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8041070 con 0x55f628246d90 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.316+0000 7f8ed5ffb640 1 -- 172.21.15.33:0/1665230988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee800b400 con 0x55f628246d90 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.316+0000 7f8ef4a13640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ed80a2d60 0x7f8ed80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.316+0000 7f8ef4a13640 1 -- 172.21.15.33:0/243261732 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ee40ceb80 con 0x7f8ef0101890 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.316+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ed80a2d60 0x7f8ed80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ed5ffb640 1 -- 172.21.15.33:0/1665230988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8003d10 con 0x55f628246d90 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 -- 172.21.15.33:0/1665230988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 msgr2=0x55f6282471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282471b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80325d0 tx=0x7f8ee800b8e0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 -- 172.21.15.33:0/1665230988 shutdown_connections 2024-04-24T23:33:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f62824c8f0 0x55f62824ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628247780 0x55f62824c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 --2- 172.21.15.33:0/1665230988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282471b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.317+0000 7f8ef8800740 1 -- 172.21.15.33:0/1665230988 >> 172.21.15.33:0/1665230988 conn(0x55f62820e950 msgr2=0x55f628242810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.318+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ed80a2d60 0x7f8ed80a5220 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55f628247b80 tx=0x7f8ee0013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.318+0000 7f8ef8800740 1 -- 172.21.15.33:0/1665230988 shutdown_connections 2024-04-24T23:33:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.318+0000 7f8ef8800740 1 -- 172.21.15.33:0/1665230988 wait complete. 2024-04-24T23:33:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.318+0000 7f8ef8800740 1 -- start start 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.318+0000 7f8ef8800740 1 Processor -- start 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628262800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62824c8f0 0x55f628269840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f6282608a0 con 0x55f628246d90 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f628165e90 con 0x55f628247780 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f628260a20 con 0x55f62824c8f0 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8ef8800740 10 client.0 _ll_register_callbacks cb 0x55f6281a1970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8eee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628262800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8eee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628262800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41260/0 (socket says 172.21.15.33:41260) 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8eee7fc640 1 -- 172.21.15.33:0/3614776724 learned_addr learned my addr 172.21.15.33:0/3614776724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.319+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eeeffd640 1 -- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62824c8f0 msgr2=0x55f628269840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62824c8f0 0x55f628269840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eeeffd640 1 -- 172.21.15.33:0/3614776724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 msgr2=0x55f628262800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628262800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eeeffd640 1 -- 172.21.15.33:0/3614776724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f627f867e0 con 0x55f628246d90 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628262800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282622c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x55f6282481f0 tx=0x7f8ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b638f7c3d46f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eceffd640 1 -- 172.21.15.33:0/3614776724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0018070 con 0x55f628246d90 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee800b400 con 0x55f628246d90 2024-04-24T23:33:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.320+0000 7f8eceffd640 1 -- 172.21.15.33:0/3614776724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ee001ebe0 con 0x55f628246d90 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.321+0000 7f8eceffd640 1 -- 172.21.15.33:0/3614776724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ee001ede0 con 0x55f628246d90 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.321+0000 7f8ef8800740 10 client.19944 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.321+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f627fe6f70 con 0x55f628246d90 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.321+0000 7f8eceffd640 1 -- 172.21.15.33:0/3614776724 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee800b400 con 0x55f628246d90 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8eceffd640 1 -- 172.21.15.33:0/3614776724 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8ee001c6f0 con 0x55f628246d90 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8ef8800740 10 client.19944 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8ef8800740 10 client.19944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8ef8800740 20 client.19944 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8ef8800740 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f62826def0 0x55f628270370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8ef8800740 4 client.19944 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.322+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62817e5a0 con 0x55f62826def0 2024-04-24T23:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.324+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f62826def0 0x55f628270370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.325+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f62826def0 0x55f628270370 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbca629858f75766 server_cookie=4af627e0a190bbe2 in_seq=0 out_seq=0 2024-04-24T23:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.325+0000 7f8eceffd640 10 client.19944 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8eceffd640 1 -- 172.21.15.33:0/3614776724 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f62817e5a0 con 0x55f62826def0 2024-04-24T23:33:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8eceffd640 10 client.19944 handle_command_reply: tid=0 2024-04-24T23:33:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 client.19944 shutdown 2024-04-24T23:33:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 msgr2=0x55f6282622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282622c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x55f6282481f0 tx=0x7f8ee00046c0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 shutdown_connections 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f62826def0 0x55f628270370 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f62824c8f0 0x55f628269840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628262800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 --2- 172.21.15.33:0/3614776724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f628246d90 0x55f6282622c0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.329+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 >> 172.21.15.33:0/3614776724 conn(0x55f62820e950 msgr2=0x55f628241950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.330+0000 7f8eceffd640 0 client.19944 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.330+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 shutdown_connections 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.330+0000 7f8ef8800740 1 -- 172.21.15.33:0/3614776724 wait complete. 2024-04-24T23:33:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.330+0000 7f8ef8800740 20 client.19944 trim_cache size 0 max 16384 2024-04-24T23:33:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 Processor -- start 2024-04-24T23:33:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 -- start start 2024-04-24T23:33:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f62826a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628269800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f628261ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc012510 con 0x55f62824c8f0 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc00fde0 con 0x55f628247780 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.358+0000 7f8ef8800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc00dc30 con 0x55f628246d90 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f628261ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f62826a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f628261ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47542/0 (socket says 172.21.15.33:47542) 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eef7fe640 1 -- 172.21.15.33:0/248641528 learned_addr learned my addr 172.21.15.33:0/248641528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628269800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eeeffd640 1 -- 172.21.15.33:0/248641528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 msgr2=0x55f628269800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628269800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eeeffd640 1 -- 172.21.15.33:0/248641528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 msgr2=0x55f628261ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f628261ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.359+0000 7f8eeeffd640 1 -- 172.21.15.33:0/248641528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f627e7e170 con 0x55f628246d90 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.360+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f62826a5e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0000c00 tx=0x7f8ee000dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65956fd8d5a39b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.360+0000 7f8ed57fa640 1 -- 172.21.15.33:0/248641528 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee001cdd0 con 0x55f628246d90 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.360+0000 7f8ed57fa640 1 -- 172.21.15.33:0/248641528 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee003a070 con 0x55f628246d90 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.360+0000 7f8ed57fa640 1 -- 172.21.15.33:0/248641528 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00054a0 con 0x55f628246d90 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 -- 172.21.15.33:0/248641528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 msgr2=0x55f62826a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f62826a5e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0000c00 tx=0x7f8ee000dbf0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 -- 172.21.15.33:0/248641528 shutdown_connections 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f628261ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628269800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 --2- 172.21.15.33:0/248641528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f62826a5e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 -- 172.21.15.33:0/248641528 >> 172.21.15.33:0/248641528 conn(0x55f62820e950 msgr2=0x55f628269ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 -- 172.21.15.33:0/248641528 shutdown_connections 2024-04-24T23:33:50.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.361+0000 7f8ef8800740 1 -- 172.21.15.33:0/248641528 wait complete. 2024-04-24T23:33:50.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 -- start start 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 Processor -- start 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f628266690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628266bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f6282d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc00e2a0 con 0x55f62824c8f0 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc00f710 con 0x55f628247780 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.362+0000 7f8ef8800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc00e970 con 0x55f628246d90 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8ef8800740 10 client.0 _ll_register_callbacks cb 0x7f8ee0008660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f628266690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f628266690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53294/0 (socket says 172.21.15.33:53294) 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 -- 172.21.15.33:0/2356817707 learned_addr learned my addr 172.21.15.33:0/2356817707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628266bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eef7fe640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f6282d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 -- 172.21.15.33:0/2356817707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 msgr2=0x55f628266bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628266bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 -- 172.21.15.33:0/2356817707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 msgr2=0x55f6282d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f6282d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.363+0000 7f8eeeffd640 1 -- 172.21.15.33:0/2356817707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6282d2a20 con 0x55f628246d90 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.364+0000 7f8eeeffd640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f628266690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ee001cc80 tx=0x7f8ee00057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eba6dc66811021c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.364+0000 7f8eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:33:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.364+0000 7f8eceffd640 1 -- 172.21.15.33:0/2356817707 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0009040 con 0x55f628246d90 2024-04-24T23:33:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.364+0000 7f8eceffd640 1 -- 172.21.15.33:0/2356817707 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8ee001ebe0 con 0x55f628246d90 2024-04-24T23:33:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.364+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee003a070 con 0x55f628246d90 2024-04-24T23:33:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.366+0000 7f8eceffd640 1 -- 172.21.15.33:0/2356817707 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ee001c630 con 0x55f628246d90 2024-04-24T23:33:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.366+0000 7f8ef8800740 10 client.29371 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.366+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f6282d3cb0 con 0x55f628246d90 2024-04-24T23:33:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8eceffd640 1 -- 172.21.15.33:0/2356817707 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee003a070 con 0x55f628246d90 2024-04-24T23:33:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8eceffd640 1 -- 172.21.15.33:0/2356817707 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8ee0035480 con 0x55f628246d90 2024-04-24T23:33:50.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8ef8800740 10 client.29371 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:50.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8ef8800740 10 client.29371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:50.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8ef8800740 20 client.29371 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:50.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8ef8800740 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f6282d45f0 0x55f6282d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8ef8800740 4 client.29371 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:33:50.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.367+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f6281b7800 con 0x55f6282d45f0 2024-04-24T23:33:50.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.370+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f6282d45f0 0x55f6282d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.371+0000 7f8eee7fc640 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f6282d45f0 0x55f6282d6a10 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20af19538c93b176 server_cookie=9901ac0b1587c4ab in_seq=0 out_seq=0 2024-04-24T23:33:50.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.371+0000 7f8eceffd640 10 client.29371 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.372+0000 7f8eceffd640 1 -- 172.21.15.33:0/2356817707 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6281b7800 con 0x55f6282d45f0 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.372+0000 7f8eceffd640 10 client.29371 handle_command_reply: tid=0 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.372+0000 7f8ef8800740 1 client.29371 shutdown 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 msgr2=0x55f628266690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f628266690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ee001cc80 tx=0x7f8ee00057e0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 shutdown_connections 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f6282d45f0 0x55f6282d6a10 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f62824c8f0 0x55f6282d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f628247780 0x55f628266bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 --2- 172.21.15.33:0/2356817707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f628246d90 0x55f628266690 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8eceffd640 0 client.29371 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 >> 172.21.15.33:0/2356817707 conn(0x55f62820e950 msgr2=0x55f628245700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 shutdown_connections 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 1 -- 172.21.15.33:0/2356817707 wait complete. 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.373+0000 7f8ef8800740 20 client.29371 trim_cache size 0 max 16384 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "4b0c821c-eaef-44a5-ab24-64c75ee59059", 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:33:50.375 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ed80a2d60 msgr2=0x7f8ed80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ed80a2d60 0x7f8ed80a5220 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55f628247b80 tx=0x7f8ee0013040 comp rx=0 tx=0).stop 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101890 msgr2=0x7f8ef0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101890 0x7f8ef0070560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40097d0 tx=0x7f8ee4009c90 comp rx=0 tx=0).stop 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 shutdown_connections 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f8ed80a2d60 0x7f8ed80a5220 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef0102250 0x7f8ef01a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101890 0x7f8ef0070560 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 --2- 172.21.15.33:0/243261732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01006a0 0x7f8ef0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 >> 172.21.15.33:0/243261732 conn(0x7f8ef00fbc30 msgr2=0x7f8ef0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 shutdown_connections 2024-04-24T23:33:50.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.374+0000 7f8ef5a15640 1 -- 172.21.15.33:0/243261732 wait complete. 2024-04-24T23:33:50.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:33:50.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:33:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 -- 172.21.15.33:0/1636744225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 msgr2=0x7f032c109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 --2- 172.21.15.33:0/1636744225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c109410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0320009900 tx=0x7f032002a410 comp rx=0 tx=0).stop 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 -- 172.21.15.33:0/1636744225 shutdown_connections 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 --2- 172.21.15.33:0/1636744225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 0x7f032c1101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 --2- 172.21.15.33:0/1636744225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c109410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 --2- 172.21.15.33:0/1636744225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 0x7f032c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.556+0000 7f033269e640 1 -- 172.21.15.33:0/1636744225 >> 172.21.15.33:0/1636744225 conn(0x7f032c076d50 msgr2=0x7f032c079170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.557+0000 7f033269e640 1 -- 172.21.15.33:0/1636744225 shutdown_connections 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.557+0000 7f033269e640 1 -- 172.21.15.33:0/1636744225 wait complete. 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.557+0000 7f033269e640 1 Processor -- start 2024-04-24T23:33:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.557+0000 7f033269e640 1 -- start start 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f033269e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 0x7f032c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f033269e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f033269e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 0x7f032c1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f033269e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f032c113a00 con 0x7f032c109950 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f033269e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f032c113880 con 0x7f032c108f90 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f033269e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f032c113b80 con 0x7f032c107da0 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 0x7f032c1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 0x7f032c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41308/0 (socket says 172.21.15.33:41308) 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 -- 172.21.15.33:0/2609555549 learned_addr learned my addr 172.21.15.33:0/2609555549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 -- 172.21.15.33:0/2609555549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 msgr2=0x7f032c19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 0x7f032c19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 -- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 msgr2=0x7f032c1a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 0x7f032c1a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 -- 172.21.15.33:0/2609555549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c1a5cd0 con 0x7f032c108f90 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f0330c3b640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 0x7f032c1a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032bfff640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 0x7f032c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.558+0000 7f032b7fe640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c19e5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0320009230 tx=0x7f03200042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.559+0000 7f03297fa640 1 -- 172.21.15.33:0/2609555549 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032003f030 con 0x7f032c108f90 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.559+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1a5f60 con 0x7f032c108f90 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.559+0000 7f03297fa640 1 -- 172.21.15.33:0/2609555549 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0320003ce0 con 0x7f032c108f90 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.559+0000 7f03297fa640 1 -- 172.21.15.33:0/2609555549 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320035070 con 0x7f032c108f90 2024-04-24T23:33:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.559+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1a63d0 con 0x7f032c108f90 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 Processor -- start 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 -- start start 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e788660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e78d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e790230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b7e6ec6b0 con 0x555b7e788c30 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b7e694880 con 0x555b7e788240 2024-04-24T23:33:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03297fa640 1 -- 172.21.15.33:0/2609555549 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0320042030 con 0x7f032c108f90 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.561+0000 7f03353af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b7e6e8670 con 0x555b7e78dda0 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e790230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e788660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e790230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53318/0 (socket says 172.21.15.33:53318) 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f032b7fe640 1 -- 172.21.15.33:0/3594645007 learned_addr learned my addr 172.21.15.33:0/3594645007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e788660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41322/0 (socket says 172.21.15.33:41322) 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f032bfff640 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e78d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 -- 172.21.15.33:0/3594645007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 msgr2=0x555b7e790230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e790230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 -- 172.21.15.33:0/3594645007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 msgr2=0x555b7e78d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e78d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 -- 172.21.15.33:0/3594645007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7e4ca7d0 con 0x555b7e788240 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f0330c3b640 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e788660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0324011760 tx=0x7f0324011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eca8d938d021735 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f03297fa640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03140a2c10 0x7f03140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f03127fc640 1 -- 172.21.15.33:0/3594645007 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324004d20 con 0x555b7e788240 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f03127fc640 1 -- 172.21.15.33:0/3594645007 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0324003970 con 0x555b7e788240 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.562+0000 7f03297fa640 1 -- 172.21.15.33:0/2609555549 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03200ef7d0 con 0x7f032c108f90 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03127fc640 1 -- 172.21.15.33:0/3594645007 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324004480 con 0x555b7e788240 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f032bfff640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03140a2c10 0x7f03140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 -- 172.21.15.33:0/3594645007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 msgr2=0x555b7e788660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e788660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0324011760 tx=0x7f0324011c20 comp rx=0 tx=0).stop 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f032bfff640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03140a2c10 0x7f03140a50d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x555b7e789030 tx=0x7f031c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 -- 172.21.15.33:0/3594645007 shutdown_connections 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e790230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e78d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 --2- 172.21.15.33:0/3594645007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e788660 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 -- 172.21.15.33:0/3594645007 >> 172.21.15.33:0/3594645007 conn(0x555b7e7838b0 msgr2=0x555b7e783cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 -- 172.21.15.33:0/3594645007 shutdown_connections 2024-04-24T23:33:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 -- 172.21.15.33:0/3594645007 wait complete. 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.563+0000 7f03353af740 1 -- start start 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 Processor -- start 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e792690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e792bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e799c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b7e790da0 con 0x555b7e788c30 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b7e6a7f50 con 0x555b7e788240 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b7e790f20 con 0x555b7e78dda0 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f03353af740 10 client.0 _ll_register_callbacks cb 0x555b7e6e3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e792690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e792690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:41334/0 (socket says 172.21.15.33:41334) 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 -- 172.21.15.33:0/862784234 learned_addr learned my addr 172.21.15.33:0/862784234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f0330c3b640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e799c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032b7fe640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e792bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 -- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 msgr2=0x555b7e799c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e799c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 -- 172.21.15.33:0/862784234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 msgr2=0x555b7e792bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e792bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.564+0000 7f032bfff640 1 -- 172.21.15.33:0/862784234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7e4003f0 con 0x555b7e788240 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f032bfff640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e792690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f031c004620 tx=0x7f031c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e8c14c9c35d1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f030b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f030b7fe640 1 -- 172.21.15.33:0/862784234 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00f8f0 con 0x555b7e788240 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f030b7fe640 1 -- 172.21.15.33:0/862784234 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f031c00fa90 con 0x555b7e788240 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0324003970 con 0x555b7e788240 2024-04-24T23:33:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f030b7fe640 1 -- 172.21.15.33:0/862784234 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f031c00fc90 con 0x555b7e788240 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 10 client.29447 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555b7e3c00f0 con 0x555b7e788240 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f030b7fe640 1 -- 172.21.15.33:0/862784234 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0324003970 con 0x555b7e788240 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f030b7fe640 1 -- 172.21.15.33:0/862784234 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f031c027070 con 0x555b7e788240 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 10 client.29447 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 10 client.29447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 20 client.29447 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e79e590 0x555b7e7a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 4 client.29447 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.565+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b7e6f9bf0 con 0x555b7e79e590 2024-04-24T23:33:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.566+0000 7f032b7fe640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e79e590 0x555b7e7a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.566+0000 7f030b7fe640 10 client.29447 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.566+0000 7f032b7fe640 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e79e590 0x555b7e7a0a10 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2697e178e19f5e server_cookie=bf228e3f6eca6e1a in_seq=0 out_seq=0 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f030b7fe640 1 -- 172.21.15.33:0/862784234 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b7e6f9bf0 con 0x555b7e79e590 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f030b7fe640 10 client.29447 handle_command_reply: tid=0 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 client.29447 shutdown 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 msgr2=0x555b7e792690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e792690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f031c004620 tx=0x7f031c004730 comp rx=0 tx=0).stop 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 shutdown_connections 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e79e590 0x555b7e7a0a10 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e78dda0 0x555b7e799c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e788c30 0x555b7e792bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f030b7fe640 0 client.29447 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 --2- 172.21.15.33:0/862784234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e792690 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 >> 172.21.15.33:0/862784234 conn(0x555b7e7838b0 msgr2=0x555b7e782af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 shutdown_connections 2024-04-24T23:33:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.567+0000 7f03353af740 1 -- 172.21.15.33:0/862784234 wait complete. 2024-04-24T23:33:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.568+0000 7f03353af740 20 client.29447 trim_cache size 0 max 16384 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.588+0000 7f03353af740 1 Processor -- start 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.588+0000 7f03353af740 1 -- start start 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f03353af740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e7922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f03353af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e799bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f03353af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f03353af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b3220 con 0x555b7e78dda0 2024-04-24T23:33:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f03353af740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b0af0 con 0x555b7e788240 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f03353af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1ae940 con 0x555b7e788c30 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f032bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e7922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e799bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7927e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47568/0 (socket says 172.21.15.33:47568) 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 -- 172.21.15.33:0/726576034 learned_addr learned my addr 172.21.15.33:0/726576034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 -- 172.21.15.33:0/726576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 msgr2=0x555b7e799bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e799bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 -- 172.21.15.33:0/726576034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 msgr2=0x555b7e7922a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e7922a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 -- 172.21.15.33:0/726576034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7e775090 con 0x555b7e78dda0 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.589+0000 7f0330c3b640 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7927e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0324005c40 tx=0x7f0324011210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ae256deb80e45f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f0311ffb640 1 -- 172.21.15.33:0/726576034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032401d030 con 0x555b7e78dda0 2024-04-24T23:33:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f0311ffb640 1 -- 172.21.15.33:0/726576034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032401e040 con 0x555b7e78dda0 2024-04-24T23:33:50.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f0311ffb640 1 -- 172.21.15.33:0/726576034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324005550 con 0x555b7e78dda0 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 -- 172.21.15.33:0/726576034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 msgr2=0x555b7e7927e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7927e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0324005c40 tx=0x7f0324011210 comp rx=0 tx=0).stop 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 -- 172.21.15.33:0/726576034 shutdown_connections 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7927e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e799bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 --2- 172.21.15.33:0/726576034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e7922a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 -- 172.21.15.33:0/726576034 >> 172.21.15.33:0/726576034 conn(0x555b7e7838b0 msgr2=0x555b7e789ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.590+0000 7f03353af740 1 -- 172.21.15.33:0/726576034 shutdown_connections 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 -- 172.21.15.33:0/726576034 wait complete. 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 -- start start 2024-04-24T23:33:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 Processor -- start 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e796d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e797290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1aefb0 con 0x555b7e78dda0 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b0420 con 0x555b7e788240 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1af680 con 0x555b7e788c30 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f03353af740 10 client.0 _ll_register_callbacks cb 0x7f031c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f032bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e796d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f0330c3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47574/0 (socket says 172.21.15.33:47574) 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f0330c3b640 1 -- 172.21.15.33:0/3776454742 learned_addr learned my addr 172.21.15.33:0/3776454742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.591+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e797290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f0330c3b640 1 -- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 msgr2=0x555b7e797290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f0330c3b640 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e797290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f0330c3b640 1 -- 172.21.15.33:0/3776454742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 msgr2=0x555b7e796d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f0330c3b640 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e796d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f0330c3b640 1 -- 172.21.15.33:0/3776454742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7e798890 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f032b7fe640 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e797290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f0330c3b640 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7982a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f03240099a0 tx=0x7f0324005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7180c101b236bce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f030b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f030b7fe640 1 -- 172.21.15.33:0/3776454742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032403c060 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f030b7fe640 1 -- 172.21.15.33:0/3776454742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0324003b80 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f032401e040 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f030b7fe640 1 -- 172.21.15.33:0/3776454742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0324043050 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f03353af740 10 client.19968 fetch_fsmap learned FSMap version 33 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555b7e799ba0 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.592+0000 7f030b7fe640 1 -- 172.21.15.33:0/3776454742 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032401e040 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f030b7fe640 1 -- 172.21.15.33:0/3776454742 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f03240195c0 con 0x555b7e78dda0 2024-04-24T23:33:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 10 client.19968 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 10 client.19968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 20 client.19968 populate_metadata read hostname 'smithi033' 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e804e40 0x555b7e807230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 4 client.19968 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b7e6f3f40 con 0x555b7e804e40 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f032bfff640 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e804e40 0x555b7e807230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f032bfff640 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e804e40 0x555b7e807230 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b191a173d9b9cadd server_cookie=e8ca16cab417f70 in_seq=0 out_seq=0 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f030b7fe640 10 client.19968 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f030b7fe640 1 -- 172.21.15.33:0/3776454742 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555b7e6f3f40 con 0x555b7e804e40 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f030b7fe640 10 client.19968 handle_command_reply: tid=0 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.593+0000 7f03353af740 1 client.19968 shutdown 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 msgr2=0x555b7e7982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7982a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f03240099a0 tx=0x7f0324005cf0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 shutdown_connections 2024-04-24T23:33:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b7e804e40 0x555b7e807230 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e78dda0 0x555b7e7982a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b7e788c30 0x555b7e797290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f030b7fe640 0 client.19968 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 --2- 172.21.15.33:0/3776454742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b7e788240 0x555b7e796d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 >> 172.21.15.33:0/3776454742 conn(0x555b7e7838b0 msgr2=0x555b7e7922a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 shutdown_connections 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 1 -- 172.21.15.33:0/3776454742 wait complete. 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f03353af740 20 client.19968 trim_cache size 0 max 16384 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:33:50.600 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout: "4b0c821c-eaef-44a5-ab24-64c75ee59059": { 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "4b0c821c-eaef-44a5-ab24-64c75ee59059", 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:33:50.601 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03140a2c10 msgr2=0x7f03140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03140a2c10 0x7f03140a50d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x555b7e789030 tx=0x7f031c005f90 comp rx=0 tx=0).stop 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 msgr2=0x7f032c19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c19e5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0320009230 tx=0x7f03200042f0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 shutdown_connections 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03140a2c10 0x7f03140a50d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f032c109950 0x7f032c1a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f032c108f90 0x7f032c19e5f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 --2- 172.21.15.33:0/2609555549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032c107da0 0x7f032c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:33:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.594+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 >> 172.21.15.33:0/2609555549 conn(0x7f032c076d50 msgr2=0x7f032c10c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:33:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.595+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 shutdown_connections 2024-04-24T23:33:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:33:50.595+0000 7f033269e640 1 -- 172.21.15.33:0/2609555549 wait complete. 2024-04-24T23:33:50.606 INFO:tasks.cephfs.filesystem:scrub status for tag:4b0c821c-eaef-44a5-ab24-64c75ee59059 - {'path': '/', 'tag': '4b0c821c-eaef-44a5-ab24-64c75ee59059', 'options': 'recursive,force'} 2024-04-24T23:33:50.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 201018 22.20 MB/sec execute 355 sec latency 117.961 ms 2024-04-24T23:33:51.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:51 smithi038 ceph-mon[32450]: pgmap v1006: 129 pgs: 129 active+clean; 621 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-04-24T23:33:51.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:51 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:33:51.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:51 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:51.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:51 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:51 smithi117 ceph-mon[32529]: pgmap v1006: 129 pgs: 129 active+clean; 621 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-04-24T23:33:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:51 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:33:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:51 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:51.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:51 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:51.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:51 smithi033 ceph-mon[29201]: pgmap v1006: 129 pgs: 129 active+clean; 621 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-04-24T23:33:51.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:51 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:33:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:51 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:51.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:51 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:51.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 201495 22.22 MB/sec execute 356 sec latency 92.227 ms 2024-04-24T23:33:52.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:52 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:52.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:52 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:52 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:52.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:52 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:52 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:33:52.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:52 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:52 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:52.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:52 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:52 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:52.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:52 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:52 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:33:52.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:52 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:52 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:52.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:52 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:52 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:52 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:52 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:33:52.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:52 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:52.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 201878 22.21 MB/sec execute 357 sec latency 104.847 ms 2024-04-24T23:33:53.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:53 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:53.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:53 smithi038 ceph-mon[32450]: pgmap v1007: 129 pgs: 129 active+clean; 622 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 403 op/s 2024-04-24T23:33:53.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:53 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:53.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:53 smithi117 ceph-mon[32529]: pgmap v1007: 129 pgs: 129 active+clean; 622 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 403 op/s 2024-04-24T23:33:53.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:53 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:33:53.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:53 smithi033 ceph-mon[29201]: pgmap v1007: 129 pgs: 129 active+clean; 622 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 403 op/s 2024-04-24T23:33:53.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 202294 22.20 MB/sec execute 358 sec latency 234.152 ms 2024-04-24T23:33:54.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 202606 22.19 MB/sec execute 359 sec latency 333.931 ms 2024-04-24T23:33:55.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:55 smithi038 ceph-mon[32450]: pgmap v1008: 129 pgs: 129 active+clean; 628 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.9 MiB/s wr, 498 op/s 2024-04-24T23:33:55.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:55 smithi117 ceph-mon[32529]: pgmap v1008: 129 pgs: 129 active+clean; 628 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.9 MiB/s wr, 498 op/s 2024-04-24T23:33:55.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:55 smithi033 ceph-mon[29201]: pgmap v1008: 129 pgs: 129 active+clean; 628 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.9 MiB/s wr, 498 op/s 2024-04-24T23:33:55.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 202995 22.19 MB/sec execute 360 sec latency 131.589 ms 2024-04-24T23:33:56.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 203381 22.19 MB/sec execute 361 sec latency 104.825 ms 2024-04-24T23:33:57.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:57 smithi038 ceph-mon[32450]: pgmap v1009: 129 pgs: 129 active+clean; 627 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 381 op/s 2024-04-24T23:33:57.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:57 smithi117 ceph-mon[32529]: pgmap v1009: 129 pgs: 129 active+clean; 627 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 381 op/s 2024-04-24T23:33:57.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:57 smithi033 ceph-mon[29201]: pgmap v1009: 129 pgs: 129 active+clean; 627 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 381 op/s 2024-04-24T23:33:57.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 204014 22.21 MB/sec execute 362 sec latency 63.358 ms 2024-04-24T23:33:58.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 204528 22.23 MB/sec execute 363 sec latency 158.754 ms 2024-04-24T23:33:59.563 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:33:59 smithi038 ceph-mon[32450]: pgmap v1010: 129 pgs: 129 active+clean; 630 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 434 op/s 2024-04-24T23:33:59.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:33:59 smithi117 ceph-mon[32529]: pgmap v1010: 129 pgs: 129 active+clean; 630 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 434 op/s 2024-04-24T23:33:59.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:33:59 smithi033 ceph-mon[29201]: pgmap v1010: 129 pgs: 129 active+clean; 630 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 434 op/s 2024-04-24T23:33:59.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 204803 22.22 MB/sec execute 364 sec latency 319.899 ms 2024-04-24T23:34:00.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 205032 22.20 MB/sec execute 365 sec latency 115.557 ms 2024-04-24T23:34:01.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:01 smithi038 ceph-mon[32450]: pgmap v1011: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 468 op/s 2024-04-24T23:34:01.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:01 smithi117 ceph-mon[32529]: pgmap v1011: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 468 op/s 2024-04-24T23:34:01.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:01 smithi033 ceph-mon[29201]: pgmap v1011: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 468 op/s 2024-04-24T23:34:01.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 205358 22.18 MB/sec execute 366 sec latency 119.492 ms 2024-04-24T23:34:02.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 205747 22.16 MB/sec execute 367 sec latency 105.287 ms 2024-04-24T23:34:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:03 smithi038 ceph-mon[32450]: pgmap v1012: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 392 op/s 2024-04-24T23:34:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:34:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:34:03.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:03 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:34:03.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:03 smithi117 ceph-mon[32529]: pgmap v1012: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 392 op/s 2024-04-24T23:34:03.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:34:03.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:34:03.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:03 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:34:03.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:03 smithi033 ceph-mon[29201]: pgmap v1012: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 392 op/s 2024-04-24T23:34:03.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:34:03.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:34:03.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:03 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:34:03.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 206211 22.17 MB/sec execute 368 sec latency 83.588 ms 2024-04-24T23:34:04.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 206515 22.16 MB/sec execute 369 sec latency 438.426 ms 2024-04-24T23:34:05.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:05 smithi038 ceph-mon[32450]: pgmap v1013: 129 pgs: 129 active+clean; 642 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 464 op/s 2024-04-24T23:34:05.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:05 smithi117 ceph-mon[32529]: pgmap v1013: 129 pgs: 129 active+clean; 642 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 464 op/s 2024-04-24T23:34:05.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:05 smithi033 ceph-mon[29201]: pgmap v1013: 129 pgs: 129 active+clean; 642 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 464 op/s 2024-04-24T23:34:05.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 206958 22.15 MB/sec execute 370 sec latency 117.229 ms 2024-04-24T23:34:06.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 207429 22.17 MB/sec execute 371 sec latency 58.930 ms 2024-04-24T23:34:07.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:07 smithi038 ceph-mon[32450]: pgmap v1014: 129 pgs: 129 active+clean; 641 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-04-24T23:34:07.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:07 smithi117 ceph-mon[32529]: pgmap v1014: 129 pgs: 129 active+clean; 641 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-04-24T23:34:07.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:07 smithi033 ceph-mon[29201]: pgmap v1014: 129 pgs: 129 active+clean; 641 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-04-24T23:34:07.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 207988 22.19 MB/sec execute 372 sec latency 105.365 ms 2024-04-24T23:34:08.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 208394 22.20 MB/sec execute 373 sec latency 93.981 ms 2024-04-24T23:34:09.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:09 smithi038 ceph-mon[32450]: pgmap v1015: 129 pgs: 129 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 416 op/s 2024-04-24T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:09 smithi117 ceph-mon[32529]: pgmap v1015: 129 pgs: 129 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 416 op/s 2024-04-24T23:34:09.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:09 smithi033 ceph-mon[29201]: pgmap v1015: 129 pgs: 129 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 416 op/s 2024-04-24T23:34:09.900 INFO:tasks.workunit.client.0.smithi033.stdout: 10 208575 22.17 MB/sec execute 374 sec latency 433.370 ms 2024-04-24T23:34:10.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 208837 22.14 MB/sec execute 375 sec latency 147.205 ms 2024-04-24T23:34:11.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:11 smithi038 ceph-mon[32450]: pgmap v1016: 129 pgs: 129 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 416 op/s 2024-04-24T23:34:11.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:11 smithi117 ceph-mon[32529]: pgmap v1016: 129 pgs: 129 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 416 op/s 2024-04-24T23:34:11.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:11 smithi033 ceph-mon[29201]: pgmap v1016: 129 pgs: 129 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 416 op/s 2024-04-24T23:34:11.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 209287 22.16 MB/sec execute 376 sec latency 59.981 ms 2024-04-24T23:34:12.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 209801 22.15 MB/sec execute 377 sec latency 118.788 ms 2024-04-24T23:34:13.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:13 smithi038 ceph-mon[32450]: pgmap v1017: 129 pgs: 129 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-04-24T23:34:13.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:13 smithi117 ceph-mon[32529]: pgmap v1017: 129 pgs: 129 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-04-24T23:34:13.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:13 smithi033 ceph-mon[29201]: pgmap v1017: 129 pgs: 129 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-04-24T23:34:13.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 210309 22.17 MB/sec execute 378 sec latency 55.436 ms 2024-04-24T23:34:14.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 210496 22.15 MB/sec execute 379 sec latency 499.505 ms 2024-04-24T23:34:15.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:15 smithi038 ceph-mon[32450]: pgmap v1018: 129 pgs: 129 active+clean; 627 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-04-24T23:34:15.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:15 smithi117 ceph-mon[32529]: pgmap v1018: 129 pgs: 129 active+clean; 627 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-04-24T23:34:15.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:15 smithi033 ceph-mon[29201]: pgmap v1018: 129 pgs: 129 active+clean; 627 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-04-24T23:34:15.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 210820 22.13 MB/sec execute 380 sec latency 117.158 ms 2024-04-24T23:34:16.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 211238 22.12 MB/sec execute 381 sec latency 63.597 ms 2024-04-24T23:34:17.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:17 smithi038 ceph-mon[32450]: pgmap v1019: 129 pgs: 129 active+clean; 635 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 361 op/s 2024-04-24T23:34:17.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:17 smithi117 ceph-mon[32529]: pgmap v1019: 129 pgs: 129 active+clean; 635 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 361 op/s 2024-04-24T23:34:17.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:17 smithi033 ceph-mon[29201]: pgmap v1019: 129 pgs: 129 active+clean; 635 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 361 op/s 2024-04-24T23:34:17.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 211734 22.14 MB/sec execute 382 sec latency 92.088 ms 2024-04-24T23:34:18.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 212314 22.18 MB/sec execute 383 sec latency 47.004 ms 2024-04-24T23:34:19.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:19 smithi038 ceph-mon[32450]: pgmap v1020: 129 pgs: 129 active+clean; 636 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 388 op/s 2024-04-24T23:34:19.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:19 smithi117 ceph-mon[32529]: pgmap v1020: 129 pgs: 129 active+clean; 636 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 388 op/s 2024-04-24T23:34:19.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:19 smithi033 ceph-mon[29201]: pgmap v1020: 129 pgs: 129 active+clean; 636 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 388 op/s 2024-04-24T23:34:19.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 212535 22.14 MB/sec execute 384 sec latency 283.116 ms 2024-04-24T23:34:20.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:34:20.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 -- 172.21.15.33:0/2276511375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a81074d0 msgr2=0x7f76a81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 --2- 172.21.15.33:0/2276511375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a81074d0 0x7f76a81098c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ca0 tx=0x7f76a002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 -- 172.21.15.33:0/2276511375 shutdown_connections 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 --2- 172.21.15.33:0/2276511375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a81074d0 0x7f76a81098c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 --2- 172.21.15.33:0/2276511375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a8104ba0 0x7f76a8106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 --2- 172.21.15.33:0/2276511375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a8102270 0x7f76a8104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 -- 172.21.15.33:0/2276511375 >> 172.21.15.33:0/2276511375 conn(0x7f76a80fbc00 msgr2=0x7f76a80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 -- 172.21.15.33:0/2276511375 shutdown_connections 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 -- 172.21.15.33:0/2276511375 wait complete. 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.822+0000 7f76af132640 1 Processor -- start 2024-04-24T23:34:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 -- start start 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 0x7f76a8199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a8104ba0 0x7f76a819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a81074d0 0x7f76a81a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 0x7f76a8199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 0x7f76a8199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51818/0 (socket says 172.21.15.33:51818) 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 -- 172.21.15.33:0/878446662 learned_addr learned my addr 172.21.15.33:0/878446662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810d430 con 0x7f76a8102270 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810d2b0 con 0x7f76a81074d0 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810d5b0 con 0x7f76a8104ba0 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76a7fff640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a8104ba0 0x7f76a819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a81074d0 0x7f76a81a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 -- 172.21.15.33:0/878446662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a8104ba0 msgr2=0x7f76a819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a8104ba0 0x7f76a819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 -- 172.21.15.33:0/878446662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a81074d0 msgr2=0x7f76a81a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a81074d0 0x7f76a81a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 -- 172.21.15.33:0/878446662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a81a18e0 con 0x7f76a8102270 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76acece640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 0x7f76a8199ca0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e9f0 tx=0x7f769c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.823+0000 7f76a5ffb640 1 -- 172.21.15.33:0/878446662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00cd80 con 0x7f76a8102270 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.824+0000 7f76a5ffb640 1 -- 172.21.15.33:0/878446662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c005500 con 0x7f76a8102270 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.824+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a81a1bd0 con 0x7f76a8102270 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.824+0000 7f76a5ffb640 1 -- 172.21.15.33:0/878446662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c010690 con 0x7f76a8102270 2024-04-24T23:34:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.824+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a2120 con 0x7f76a8102270 2024-04-24T23:34:20.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.826+0000 7f76a5ffb640 1 -- 172.21.15.33:0/878446662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f769c0046c0 con 0x7f76a8102270 2024-04-24T23:34:20.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.827+0000 7f76a5ffb640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76980a2c10 0x7f76980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.827+0000 7f76a5ffb640 1 -- 172.21.15.33:0/878446662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f769c0cf750 con 0x7f76a8102270 2024-04-24T23:34:20.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.827+0000 7f76a7fff640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76980a2c10 0x7f76980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.827+0000 7f76a7fff640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76980a2c10 0x7f76980a50d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f76a819b1a0 tx=0x7f7694005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:20.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 212938 22.15 MB/sec execute 385 sec latency 92.967 ms 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 Processor -- start 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 -- start start 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ec240 0x55f90b8ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8f1da0 0x55f90b8f4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f90b8506b0 con 0x55f90b8ecc30 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f90b7f8880 con 0x55f90b8f1da0 2024-04-24T23:34:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.954+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f90b84c670 con 0x55f90b8ec240 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51830/0 (socket says 172.21.15.33:51830) 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 -- 172.21.15.33:0/2839858527 learned_addr learned my addr 172.21.15.33:0/2839858527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76acece640 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ec240 0x55f90b8ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8f1da0 0x55f90b8f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 -- 172.21.15.33:0/2839858527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ec240 msgr2=0x55f90b8ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ec240 0x55f90b8ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 -- 172.21.15.33:0/2839858527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8f1da0 msgr2=0x55f90b8f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8f1da0 0x55f90b8f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 -- 172.21.15.33:0/2839858527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90b62e7d0 con 0x55f90b8ecc30 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f76a7fff640 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f1730 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f769400e630 tx=0x7f7694014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f55afb9a7db8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f7692ffd640 1 -- 172.21.15.33:0/2839858527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694004020 con 0x55f90b8ecc30 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f7692ffd640 1 -- 172.21.15.33:0/2839858527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76940041c0 con 0x55f90b8ecc30 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.955+0000 7f7692ffd640 1 -- 172.21.15.33:0/2839858527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769401c710 con 0x55f90b8ecc30 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 -- 172.21.15.33:0/2839858527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 msgr2=0x55f90b8f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f1730 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f769400e630 tx=0x7f7694014000 comp rx=0 tx=0).stop 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 -- 172.21.15.33:0/2839858527 shutdown_connections 2024-04-24T23:34:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8f1da0 0x55f90b8f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f1730 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/2839858527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ec240 0x55f90b8ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 -- 172.21.15.33:0/2839858527 >> 172.21.15.33:0/2839858527 conn(0x55f90b8e78b0 msgr2=0x55f90b8e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 -- 172.21.15.33:0/2839858527 shutdown_connections 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.956+0000 7f76b1e3d740 1 -- 172.21.15.33:0/2839858527 wait complete. 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 -- start start 2024-04-24T23:34:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 Processor -- start 2024-04-24T23:34:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f90b8f4d50 con 0x55f90b8ecc30 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f90b80bf50 con 0x55f90b8ec240 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f90b8f4ed0 con 0x55f90b8f1da0 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76b1e3d740 10 client.0 _ll_register_callbacks cb 0x55f90b847000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51838/0 (socket says 172.21.15.33:51838) 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76acece640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76a7fff640 1 -- 172.21.15.33:0/4279742094 learned_addr learned my addr 172.21.15.33:0/4279742094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.957+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76a7fff640 1 -- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 msgr2=0x55f90b8fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76a7fff640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76a7fff640 1 -- 172.21.15.33:0/4279742094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 msgr2=0x55f90b8f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76a7fff640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76a7fff640 1 -- 172.21.15.33:0/4279742094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90b5643f0 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76acece640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76a7fff640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f6ba0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7694009730 tx=0x7f7694004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff05515b704435e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f7687fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f7687fff640 1 -- 172.21.15.33:0/4279742094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694005050 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f7687fff640 1 -- 172.21.15.33:0/4279742094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f76940051f0 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76940041c0 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f7687fff640 1 -- 172.21.15.33:0/4279742094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f76940053f0 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76b1e3d740 10 client.19980 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.958+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f90b5240f0 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f7687fff640 1 -- 172.21.15.33:0/4279742094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76940041c0 con 0x55f90b8ecc30 2024-04-24T23:34:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f7687fff640 1 -- 172.21.15.33:0/4279742094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7694004b00 con 0x55f90b8ecc30 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76b1e3d740 10 client.19980 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76b1e3d740 10 client.19980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76b1e3d740 20 client.19980 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b9020d0 0x55f90b9044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76b1e3d740 4 client.19980 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f90b85dbf0 con 0x55f90b9020d0 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76acece640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b9020d0 0x55f90b9044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f76acece640 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b9020d0 0x55f90b9044c0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6adbb525532ad5 server_cookie=61936b2824741de5 in_seq=0 out_seq=0 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.959+0000 7f7687fff640 10 client.19980 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.960+0000 7f7687fff640 1 -- 172.21.15.33:0/4279742094 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f90b85dbf0 con 0x55f90b9020d0 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.960+0000 7f7687fff640 10 client.19980 handle_command_reply: tid=0 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.960+0000 7f76b1e3d740 1 client.19980 shutdown 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.960+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 msgr2=0x55f90b8f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.960+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f6ba0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7694009730 tx=0x7f7694004140 comp rx=0 tx=0).stop 2024-04-24T23:34:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 shutdown_connections 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b9020d0 0x55f90b9044c0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8f6ba0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/4279742094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 >> 172.21.15.33:0/4279742094 conn(0x55f90b8e78b0 msgr2=0x55f90b8e6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f7687fff640 0 client.19980 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 shutdown_connections 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 1 -- 172.21.15.33:0/4279742094 wait complete. 2024-04-24T23:34:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.961+0000 7f76b1e3d740 20 client.19980 trim_cache size 0 max 16384 2024-04-24T23:34:20.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.980+0000 7f76b1e3d740 1 Processor -- start 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.980+0000 7f76b1e3d740 1 -- start start 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b9040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ecc30 0x55f90b8fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 0x55f90b8e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81aeeb0 con 0x55f90b8f1da0 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81ac780 con 0x55f90b8ec240 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81aa5d0 con 0x55f90b8ecc30 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 0x55f90b8e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 0x55f90b8e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51850/0 (socket says 172.21.15.33:51850) 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 -- 172.21.15.33:0/3921508928 learned_addr learned my addr 172.21.15.33:0/3921508928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76acece640 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b9040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76a7fff640 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ecc30 0x55f90b8fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 -- 172.21.15.33:0/3921508928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ecc30 msgr2=0x55f90b8fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ecc30 0x55f90b8fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 -- 172.21.15.33:0/3921508928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 msgr2=0x55f90b9040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b9040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 -- 172.21.15.33:0/3921508928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90b8d9090 con 0x55f90b8f1da0 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 0x55f90b8e6260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ca0 tx=0x7f76a00098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437262ce87a186f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76927fc640 1 -- 172.21.15.33:0/3921508928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a004c070 con 0x55f90b8f1da0 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.981+0000 7f76927fc640 1 -- 172.21.15.33:0/3921508928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0036040 con 0x55f90b8f1da0 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76927fc640 1 -- 172.21.15.33:0/3921508928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00043b0 con 0x55f90b8f1da0 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3921508928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 msgr2=0x55f90b8e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 0x55f90b8e6260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ca0 tx=0x7f76a00098b0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3921508928 shutdown_connections 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8f1da0 0x55f90b8e6260 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8ecc30 0x55f90b8fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3921508928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b9040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3921508928 >> 172.21.15.33:0/3921508928 conn(0x55f90b8e78b0 msgr2=0x55f90b8edff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3921508928 shutdown_connections 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.982+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3921508928 wait complete. 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 -- start start 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 Processor -- start 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8fb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81aac40 con 0x55f90b8ecc30 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81ac0b0 con 0x55f90b8ec240 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81ab310 con 0x55f90b8f1da0 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76b1e3d740 10 client.0 _ll_register_callbacks cb 0x7f769c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76acece640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8fb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51860/0 (socket says 172.21.15.33:51860) 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76acece640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8fb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53222/0 (socket says 172.21.15.33:53222) 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 -- 172.21.15.33:0/3694704427 learned_addr learned my addr 172.21.15.33:0/3694704427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76ad6cf640 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 -- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 msgr2=0x55f90b8fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 -- 172.21.15.33:0/3694704427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 msgr2=0x55f90b8fb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8fb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.983+0000 7f76a7fff640 1 -- 172.21.15.33:0/3694704427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90b8fcd40 con 0x55f90b8ecc30 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76a7fff640 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8fb740 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f769400efd0 tx=0x7f7694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db021c8d637a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f7687fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f7687fff640 1 -- 172.21.15.33:0/3694704427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694044060 con 0x55f90b8ecc30 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f7687fff640 1 -- 172.21.15.33:0/3694704427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7694004790 con 0x55f90b8ecc30 2024-04-24T23:34:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a0036040 con 0x55f90b8ecc30 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f7687fff640 1 -- 172.21.15.33:0/3694704427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f769400eaf0 con 0x55f90b8ecc30 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 10 client.19986 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f90b9685d0 con 0x55f90b8ecc30 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f7687fff640 1 -- 172.21.15.33:0/3694704427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0036040 con 0x55f90b8ecc30 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f7687fff640 1 -- 172.21.15.33:0/3694704427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7694003c60 con 0x55f90b8ecc30 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 10 client.19986 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 10 client.19986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 20 client.19986 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b968eb0 0x55f90b96b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 4 client.19986 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.984+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f90b857f40 con 0x55f90b968eb0 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76acece640 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b968eb0 0x55f90b96b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f7687fff640 10 client.19986 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76acece640 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b968eb0 0x55f90b96b2d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1424617a70663339 server_cookie=171279ed1fbbded3 in_seq=0 out_seq=0 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f7687fff640 1 -- 172.21.15.33:0/3694704427 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f90b857f40 con 0x55f90b968eb0 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f7687fff640 10 client.19986 handle_command_reply: tid=0 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 client.19986 shutdown 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 msgr2=0x55f90b8fb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8fb740 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f769400efd0 tx=0x7f7694009250 comp rx=0 tx=0).stop 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 shutdown_connections 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f90b968eb0 0x55f90b96b2d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f90b8f1da0 0x55f90b8fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90b8ecc30 0x55f90b8fb740 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 --2- 172.21.15.33:0/3694704427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90b8ec240 0x55f90b8fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f7687fff640 0 client.19986 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 >> 172.21.15.33:0/3694704427 conn(0x55f90b8e78b0 msgr2=0x55f90b8e6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 shutdown_connections 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.985+0000 7f76b1e3d740 1 -- 172.21.15.33:0/3694704427 wait complete. 2024-04-24T23:34:20.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76b1e3d740 20 client.19986 trim_cache size 0 max 16384 2024-04-24T23:34:20.992 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:34:20.992 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:34:20.992 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:34:20.992 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76980a2c10 msgr2=0x7f76980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76980a2c10 0x7f76980a50d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f76a819b1a0 tx=0x7f7694005f90 comp rx=0 tx=0).stop 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 msgr2=0x7f76a8199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 0x7f76a8199ca0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e9f0 tx=0x7f769c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 shutdown_connections 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f76980a2c10 0x7f76980a50d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76a81074d0 0x7f76a81a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a8104ba0 0x7f76a819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 --2- 172.21.15.33:0/878446662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76a8102270 0x7f76a8199ca0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 >> 172.21.15.33:0/878446662 conn(0x7f76a80fbc00 msgr2=0x7f76a8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:20.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 shutdown_connections 2024-04-24T23:34:20.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:20.986+0000 7f76af132640 1 -- 172.21.15.33:0/878446662 wait complete. 2024-04-24T23:34:20.997 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:34:20.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 -- 172.21.15.33:0/1676199217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 msgr2=0x7f66bc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 --2- 172.21.15.33:0/1676199217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc101cf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009900 tx=0x7f66b002a410 comp rx=0 tx=0).stop 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 -- 172.21.15.33:0/1676199217 shutdown_connections 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 --2- 172.21.15.33:0/1676199217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 0x7f66bc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 --2- 172.21.15.33:0/1676199217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc101cf0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 --2- 172.21.15.33:0/1676199217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66bc100680 0x7f66bc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 -- 172.21.15.33:0/1676199217 >> 172.21.15.33:0/1676199217 conn(0x7f66bc0fbbf0 msgr2=0x7f66bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 -- 172.21.15.33:0/1676199217 shutdown_connections 2024-04-24T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.112+0000 7f66c109e640 1 -- 172.21.15.33:0/1676199217 wait complete. 2024-04-24T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 Processor -- start 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 -- start start 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66bc100680 0x7f66bc104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 0x7f66bc103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10c350 con 0x7f66bc101870 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10c1d0 con 0x7f66bc100680 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.113+0000 7f66c109e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10c4d0 con 0x7f66bc102230 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66bc100680 0x7f66bc104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51882/0 (socket says 172.21.15.33:51882) 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 0x7f66bc103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66ba59c640 1 -- 172.21.15.33:0/290146877 learned_addr learned my addr 172.21.15.33:0/290146877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 -- 172.21.15.33:0/290146877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66bc100680 msgr2=0x7f66bc104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66bc100680 0x7f66bc104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 -- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 msgr2=0x7f66bc102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 -- 172.21.15.33:0/290146877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc103a20 con 0x7f66bc102230 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66ba59c640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66bb59e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 0x7f66bc103190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009fd0 tx=0x7f66b400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b400ce20 con 0x7f66bc102230 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc1a1bb0 con 0x7f66bc102230 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b4005020 con 0x7f66bc102230 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4010720 con 0x7f66bc102230 2024-04-24T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.114+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1a2150 con 0x7f66bc102230 2024-04-24T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.116+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc1a2380 con 0x7f66bc102230 2024-04-24T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.117+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f66b4005540 con 0x7f66bc102230 2024-04-24T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.117+0000 7f66a3fff640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f66a40a2b90 0x7f66a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.117+0000 7f66bad9d640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f66a40a2b90 0x7f66a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.117+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66b40cfaf0 con 0x7f66bc102230 2024-04-24T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.118+0000 7f66bad9d640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f66a40a2b90 0x7f66a40a5050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f66bc1016d0 tx=0x7f66ac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.122+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66b409c660 con 0x7f66bc102230 2024-04-24T23:34:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.427+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66bc1a2550 con 0x7f66bc102230 2024-04-24T23:34:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.428+0000 7f66a3fff640 1 -- 172.21.15.33:0/290146877 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f66b409bf60 con 0x7f66bc102230 2024-04-24T23:34:21.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:34:21.428 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:34:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f66a40a2b90 msgr2=0x7f66a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f66a40a2b90 0x7f66a40a5050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f66bc1016d0 tx=0x7f66ac005f90 comp rx=0 tx=0).stop 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 msgr2=0x7f66bc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 0x7f66bc103190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009fd0 tx=0x7f66b400ef10 comp rx=0 tx=0).stop 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 shutdown_connections 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f66a40a2b90 0x7f66a40a5050 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66bc102230 0x7f66bc103190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc101870 0x7f66bc102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 --2- 172.21.15.33:0/290146877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66bc100680 0x7f66bc104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.431+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 >> 172.21.15.33:0/290146877 conn(0x7f66bc0fbbf0 msgr2=0x7f66bc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.432+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 shutdown_connections 2024-04-24T23:34:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.432+0000 7f66c109e640 1 -- 172.21.15.33:0/290146877 wait complete. 2024-04-24T23:34:21.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 -- 172.21.15.33:0/3591962319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde601006a0 msgr2=0x7fde60100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 --2- 172.21.15.33:0/3591962319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde601006a0 0x7fde60100ac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde50009900 tx=0x7fde5002a410 comp rx=0 tx=0).stop 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 -- 172.21.15.33:0/3591962319 shutdown_connections 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 --2- 172.21.15.33:0/3591962319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60102250 0x7fde60108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 --2- 172.21.15.33:0/3591962319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde60101890 0x7fde60101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 --2- 172.21.15.33:0/3591962319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde601006a0 0x7fde60100ac0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 -- 172.21.15.33:0/3591962319 >> 172.21.15.33:0/3591962319 conn(0x7fde600fbc10 msgr2=0x7fde600fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 -- 172.21.15.33:0/3591962319 shutdown_connections 2024-04-24T23:34:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.567+0000 7fde65e95640 1 -- 172.21.15.33:0/3591962319 wait complete. 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 Processor -- start 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 -- start start 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde601006a0 0x7fde60199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde60101890 0x7fde6019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde60102250 0x7fde601a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010c320 con 0x7fde601006a0 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010c1a0 con 0x7fde60101890 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.568+0000 7fde65e95640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010c4a0 con 0x7fde60102250 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde601006a0 0x7fde60199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde60101890 0x7fde6019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde601006a0 0x7fde60199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51904/0 (socket says 172.21.15.33:51904) 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5f7fe640 1 -- 172.21.15.33:0/1268583784 learned_addr learned my addr 172.21.15.33:0/1268583784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5ffff640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde60102250 0x7fde601a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 -- 172.21.15.33:0/1268583784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde60102250 msgr2=0x7fde601a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde60102250 0x7fde601a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 -- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde601006a0 msgr2=0x7fde60199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde601006a0 0x7fde60199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 -- 172.21.15.33:0/1268583784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde601a1970 con 0x7fde60101890 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5effd640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde60101890 0x7fde6019a290 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9f0 tx=0x7fde5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5cff9640 1 -- 172.21.15.33:0/1268583784 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5400ccc0 con 0x7fde60101890 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5cff9640 1 -- 172.21.15.33:0/1268583784 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde54005020 con 0x7fde60101890 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde5cff9640 1 -- 172.21.15.33:0/1268583784 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde540049e0 con 0x7fde60101890 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde601a1c60 con 0x7fde60101890 2024-04-24T23:34:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.569+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde601a2160 con 0x7fde60101890 2024-04-24T23:34:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 Processor -- start 2024-04-24T23:34:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 -- start start 2024-04-24T23:34:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0e1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55deef03d670 con 0x55deef0d9c20 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55deeefe5970 con 0x55deef0d9230 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde68b9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55deef039630 con 0x55deef0ded90 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde5cff9640 1 -- 172.21.15.33:0/1268583784 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fde54005560 con 0x7fde60101890 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0d9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53286/0 (socket says 172.21.15.33:53286) 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde5ffff640 1 -- 172.21.15.33:0/4140398264 learned_addr learned my addr 172.21.15.33:0/4140398264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.572+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0de720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51906/0 (socket says 172.21.15.33:51906) 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5effd640 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5ffff640 1 -- 172.21.15.33:0/4140398264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 msgr2=0x55deef0e1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5ffff640 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0e1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5ffff640 1 -- 172.21.15.33:0/4140398264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 msgr2=0x55deef0de720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5ffff640 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0de720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5ffff640 1 -- 172.21.15.33:0/4140398264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deef009400 con 0x55deef0d9230 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5effd640 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5ffff640 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0d9650 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde58009fd0 tx=0x7fde580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7756c875e7e232a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5cff9640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde480a2c10 0x7fde480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde45ffb640 1 -- 172.21.15.33:0/4140398264 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58004d20 con 0x55deef0d9230 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde480a2c10 0x7fde480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde45ffb640 1 -- 172.21.15.33:0/4140398264 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58004050 con 0x55deef0d9230 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde45ffb640 1 -- 172.21.15.33:0/4140398264 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58005260 con 0x55deef0d9230 2024-04-24T23:34:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.573+0000 7fde5cff9640 1 -- 172.21.15.33:0/1268583784 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fde540cf380 con 0x7fde60101890 2024-04-24T23:34:21.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 -- 172.21.15.33:0/4140398264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 msgr2=0x55deef0d9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde480a2c10 0x7fde480a50d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde50000c00 tx=0x7fde500046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0d9650 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde58009fd0 tx=0x7fde580118d0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 -- 172.21.15.33:0/4140398264 shutdown_connections 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 --2- 172.21.15.33:0/4140398264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0d9650 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.574+0000 7fde68b9b740 1 -- 172.21.15.33:0/4140398264 >> 172.21.15.33:0/4140398264 conn(0x55deef0d48a0 msgr2=0x55deef0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 -- 172.21.15.33:0/4140398264 shutdown_connections 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 -- 172.21.15.33:0/4140398264 wait complete. 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 -- start start 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 Processor -- start 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9c20 0x55deef0e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0ded90 0x55deef0ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55deef0e1d40 con 0x55deef0ded90 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55deeeff9050 con 0x55deef0d9230 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55deef0e1ec0 con 0x55deef0d9c20 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.575+0000 7fde68b9b740 10 client.0 _ll_register_callbacks cb 0x55deeed50be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0ded90 0x55deef0ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0ded90 0x55deef0ea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51920/0 (socket says 172.21.15.33:51920) 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53288/0 (socket says 172.21.15.33:53288) 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5ffff640 1 -- 172.21.15.33:0/2712389870 learned_addr learned my addr 172.21.15.33:0/2712389870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9c20 0x55deef0e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2712389870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9c20 msgr2=0x55deef0e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9c20 0x55deef0e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2712389870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0ded90 msgr2=0x55deef0ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0ded90 0x55deef0ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2712389870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deeefcadc0 con 0x55deef0d9230 2024-04-24T23:34:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0e3200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde50005ab0 tx=0x7fde50009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c72abcc471061fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde3effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde3effd640 1 -- 172.21.15.33:0/2712389870 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5004e070 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde58004050 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde3effd640 1 -- 172.21.15.33:0/2712389870 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fde50003a00 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde3effd640 1 -- 172.21.15.33:0/2712389870 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde5005a050 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde68b9b740 10 client.29489 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.576+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55deeed513f0 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde3effd640 1 -- 172.21.15.33:0/2712389870 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58004050 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde3effd640 1 -- 172.21.15.33:0/2712389870 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fde5003d450 con 0x55deef0d9230 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde68b9b740 10 client.29489 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde68b9b740 10 client.29489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde68b9b740 20 client.29489 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef0f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde68b9b740 4 client.29489 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deef04abb0 con 0x55deef0eef10 2024-04-24T23:34:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde5effd640 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef0f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde5effd640 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef0f1340 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bca7c3dd61bbc53b server_cookie=53284d8aa955a56e in_seq=0 out_seq=0 2024-04-24T23:34:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.577+0000 7fde3effd640 10 client.29489 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde3effd640 1 -- 172.21.15.33:0/2712389870 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55deef04abb0 con 0x55deef0eef10 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde3effd640 10 client.29489 handle_command_reply: tid=0 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 client.29489 shutdown 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 msgr2=0x55deef0e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0e3200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde50005ab0 tx=0x7fde50009260 comp rx=0 tx=0).stop 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 shutdown_connections 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef0f1340 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde3effd640 0 client.29489 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0ded90 0x55deef0ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9c20 0x55deef0e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2712389870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9230 0x55deef0e3200 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 >> 172.21.15.33:0/2712389870 conn(0x55deef0d48a0 msgr2=0x55deef0d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 shutdown_connections 2024-04-24T23:34:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 1 -- 172.21.15.33:0/2712389870 wait complete. 2024-04-24T23:34:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.580+0000 7fde68b9b740 20 client.29489 trim_cache size 0 max 16384 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 Processor -- start 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 -- start start 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 0x55deef0d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9c20 0x55deef0ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde601aeec0 con 0x55deef0d9230 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fde601ac790 con 0x55deef0d9c20 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde68b9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde601aa5e0 con 0x55deef0ded90 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 0x55deef0d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 0x55deef0d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51922/0 (socket says 172.21.15.33:51922) 2024-04-24T23:34:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2327530980 learned_addr learned my addr 172.21.15.33:0/2327530980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde5effd640 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9c20 0x55deef0ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.595+0000 7fde5ffff640 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2327530980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 msgr2=0x55deef0d7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0d7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2327530980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9c20 msgr2=0x55deef0ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9c20 0x55deef0ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5f7fe640 1 -- 172.21.15.33:0/2327530980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deeee1b7d0 con 0x55deef0d9230 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5effd640 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9c20 0x55deef0ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 0x55deef0d7580 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fde5003d870 tx=0x7fde50005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fc8aee3419f8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde457fa640 1 -- 172.21.15.33:0/2327530980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5004e070 con 0x55deef0d9230 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde457fa640 1 -- 172.21.15.33:0/2327530980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5003d450 con 0x55deef0d9230 2024-04-24T23:34:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.596+0000 7fde457fa640 1 -- 172.21.15.33:0/2327530980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50035040 con 0x55deef0d9230 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 -- 172.21.15.33:0/2327530980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 msgr2=0x55deef0d7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 0x55deef0d7580 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fde5003d870 tx=0x7fde50005c90 comp rx=0 tx=0).stop 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 -- 172.21.15.33:0/2327530980 shutdown_connections 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0ded90 0x55deef0d7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0d9c20 0x55deef0ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 --2- 172.21.15.33:0/2327530980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9230 0x55deef0d7580 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 -- 172.21.15.33:0/2327530980 >> 172.21.15.33:0/2327530980 conn(0x55deef0d48a0 msgr2=0x55deef0d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 -- 172.21.15.33:0/2327530980 shutdown_connections 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 -- 172.21.15.33:0/2327530980 wait complete. 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.597+0000 7fde68b9b740 1 -- start start 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 Processor -- start 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9230 0x55deef0e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0ded90 0x55deef0e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde601aac50 con 0x55deef0d9c20 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fde601ac0c0 con 0x55deef0ded90 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde601ab320 con 0x55deef0d9230 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde68b9b740 10 client.0 _ll_register_callbacks cb 0x7fde50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0ded90 0x55deef0e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0e8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51934/0 (socket says 172.21.15.33:51934) 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5effd640 1 -- 172.21.15.33:0/3758446199 learned_addr learned my addr 172.21.15.33:0/3758446199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9230 0x55deef0e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5ffff640 1 -- 172.21.15.33:0/3758446199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9230 msgr2=0x55deef0e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5ffff640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9230 0x55deef0e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5ffff640 1 -- 172.21.15.33:0/3758446199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 msgr2=0x55deef0e8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5ffff640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0e8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5ffff640 1 -- 172.21.15.33:0/3758446199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deef0e9b20 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.598+0000 7fde5effd640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde5ffff640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0ded90 0x55deef0e9530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55deef0da640 tx=0x7fde58005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc65a3e89844bcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde3effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde3effd640 1 -- 172.21.15.33:0/3758446199 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58005980 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde3effd640 1 -- 172.21.15.33:0/3758446199 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fde58009be0 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde5003d450 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde3effd640 1 -- 172.21.15.33:0/3758446199 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde5800f4e0 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde68b9b740 10 client.29495 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55deef1555f0 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.599+0000 7fde3effd640 1 -- 172.21.15.33:0/3758446199 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5003d450 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde3effd640 1 -- 172.21.15.33:0/3758446199 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fde58019810 con 0x55deef0ded90 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde68b9b740 10 client.29495 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:21.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde68b9b740 10 client.29495 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde68b9b740 20 client.29495 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde68b9b740 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde68b9b740 4 client.29495 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deef044f00 con 0x55deef0eef10 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde3effd640 10 client.29495 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.600+0000 7fde5f7fe640 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef157e40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=891db6e95732728a server_cookie=6f893028f14ea181 in_seq=0 out_seq=0 2024-04-24T23:34:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde3effd640 1 -- 172.21.15.33:0/3758446199 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deef044f00 con 0x55deef0eef10 2024-04-24T23:34:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde3effd640 10 client.29495 handle_command_reply: tid=0 2024-04-24T23:34:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 client.29495 shutdown 2024-04-24T23:34:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0ded90 msgr2=0x55deef0e9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0ded90 0x55deef0e9530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55deef0da640 tx=0x7fde58005600 comp rx=0 tx=0).stop 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 shutdown_connections 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55deef0eef10 0x55deef157e40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55deef0ded90 0x55deef0e9530 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde3effd640 0 client.29495 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deef0d9c20 0x55deef0e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 --2- 172.21.15.33:0/3758446199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55deef0d9230 0x55deef0e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.606+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 >> 172.21.15.33:0/3758446199 conn(0x55deef0d48a0 msgr2=0x55deef0da670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 shutdown_connections 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde68b9b740 1 -- 172.21.15.33:0/3758446199 wait complete. 2024-04-24T23:34:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde68b9b740 20 client.29495 trim_cache size 0 max 16384 2024-04-24T23:34:21.608 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:34:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde480a2c10 msgr2=0x7fde480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde480a2c10 0x7fde480a50d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde50000c00 tx=0x7fde500046c0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde60101890 msgr2=0x7fde6019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde60101890 0x7fde6019a290 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9f0 tx=0x7fde5400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 shutdown_connections 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fde480a2c10 0x7fde480a50d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde60102250 0x7fde601a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde60101890 0x7fde6019a290 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.607+0000 7fde65e95640 1 --2- 172.21.15.33:0/1268583784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde601006a0 0x7fde60199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.608+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 >> 172.21.15.33:0/1268583784 conn(0x7fde600fbc10 msgr2=0x7fde600fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.608+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 shutdown_connections 2024-04-24T23:34:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.608+0000 7fde65e95640 1 -- 172.21.15.33:0/1268583784 wait complete. 2024-04-24T23:34:21.618 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:21 smithi033 ceph-mon[29201]: pgmap v1021: 129 pgs: 129 active+clean; 640 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 404 op/s 2024-04-24T23:34:21.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:34:21.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:21 smithi038 ceph-mon[32450]: pgmap v1021: 129 pgs: 129 active+clean; 640 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 404 op/s 2024-04-24T23:34:21.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:21 smithi117 ceph-mon[32529]: pgmap v1021: 129 pgs: 129 active+clean; 640 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 404 op/s 2024-04-24T23:34:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 -- 172.21.15.33:0/3955129509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100680 msgr2=0x7f04d8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/3955129509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100680 0x7f04d8100aa0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c802a410 comp rx=0 tx=0).stop 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 -- 172.21.15.33:0/3955129509 shutdown_connections 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/3955129509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d8102230 0x7f04d8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/3955129509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04d8101870 0x7f04d8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/3955129509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8100680 0x7f04d8100aa0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 -- 172.21.15.33:0/3955129509 >> 172.21.15.33:0/3955129509 conn(0x7f04d80fbbf0 msgr2=0x7f04d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 -- 172.21.15.33:0/3955129509 shutdown_connections 2024-04-24T23:34:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.734+0000 7f04dcfc4640 1 -- 172.21.15.33:0/3955129509 wait complete. 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 Processor -- start 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 -- start start 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04d8100680 0x7f04d8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d8101870 0x7f04d8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 0x7f04d819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810c300 con 0x7f04d8102230 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810c180 con 0x7f04d8100680 2024-04-24T23:34:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04dcfc4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810c480 con 0x7f04d8101870 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 0x7f04d819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 0x7f04d819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51950/0 (socket says 172.21.15.33:51950) 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04d8100680 0x7f04d8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 -- 172.21.15.33:0/4184602286 learned_addr learned my addr 172.21.15.33:0/4184602286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d8101870 0x7f04d8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 -- 172.21.15.33:0/4184602286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d8101870 msgr2=0x7f04d8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d8101870 0x7f04d8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 -- 172.21.15.33:0/4184602286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04d8100680 msgr2=0x7f04d8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04d8100680 0x7f04d8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.735+0000 7f04d6d9d640 1 -- 172.21.15.33:0/4184602286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d819f6f0 con 0x7f04d8102230 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.736+0000 7f04d6d9d640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 0x7f04d819f080 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f04d000eb10 tx=0x7f04d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.736+0000 7f04c77fe640 1 -- 172.21.15.33:0/4184602286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0004b20 con 0x7f04d8102230 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.736+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d819f9e0 con 0x7f04d8102230 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.736+0000 7f04c77fe640 1 -- 172.21.15.33:0/4184602286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0004cc0 con 0x7f04d8102230 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.736+0000 7f04c77fe640 1 -- 172.21.15.33:0/4184602286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0010760 con 0x7f04d8102230 2024-04-24T23:34:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.736+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d819ff80 con 0x7f04d8102230 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.738+0000 7f04c77fe640 1 -- 172.21.15.33:0/4184602286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f04d0005020 con 0x7f04d8102230 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.738+0000 7f04dfda8740 1 Processor -- start 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.738+0000 7f04dfda8740 1 -- start start 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab14e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 0x5556ab153720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab153d90 0x5556ab156220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04dfda8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5556ab0b2670 con 0x5556ab14ec20 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04dfda8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5556ab05a970 con 0x5556ab14e230 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04dfda8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5556ab0ae630 con 0x5556ab153d90 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04c77fe640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04c00a2b90 0x7f04c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 0x5556ab153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 0x5556ab153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51952/0 (socket says 172.21.15.33:51952) 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 -- 172.21.15.33:0/209700983 learned_addr learned my addr 172.21.15.33:0/209700983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab153d90 0x5556ab156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04c77fe640 1 -- 172.21.15.33:0/4184602286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04d00cfa70 con 0x7f04d8102230 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d6d9d640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04c00a2b90 0x7f04c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 -- 172.21.15.33:0/209700983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab153d90 msgr2=0x5556ab156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab153d90 0x5556ab156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 -- 172.21.15.33:0/209700983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 msgr2=0x5556ab14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d659c640 1 -- 172.21.15.33:0/209700983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ab07e400 con 0x5556ab14ec20 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.739+0000 7f04d6d9d640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04c00a2b90 0x7f04c00a5050 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005ba0 tx=0x7f04d000efd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04d659c640 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 0x5556ab153720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f04c8031610 tx=0x7f04c8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec912b40ab49691 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04c4ff9640 1 -- 172.21.15.33:0/209700983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8035040 con 0x5556ab14ec20 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04c4ff9640 1 -- 172.21.15.33:0/209700983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c80318b0 con 0x5556ab14ec20 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04c4ff9640 1 -- 172.21.15.33:0/209700983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80049e0 con 0x5556ab14ec20 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 -- 172.21.15.33:0/209700983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 msgr2=0x5556ab153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 0x5556ab153720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f04c8031610 tx=0x7f04c8005de0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 -- 172.21.15.33:0/209700983 shutdown_connections 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab153d90 0x5556ab156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14ec20 0x5556ab153720 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 --2- 172.21.15.33:0/209700983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab14e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.740+0000 7f04dfda8740 1 -- 172.21.15.33:0/209700983 >> 172.21.15.33:0/209700983 conn(0x5556ab1498a0 msgr2=0x5556ab149ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 -- 172.21.15.33:0/209700983 shutdown_connections 2024-04-24T23:34:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 -- 172.21.15.33:0/209700983 wait complete. 2024-04-24T23:34:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 -- start start 2024-04-24T23:34:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 Processor -- start 2024-04-24T23:34:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5556ab156ce0 con 0x5556ab153d90 2024-04-24T23:34:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5556ab06e050 con 0x5556ab14e230 2024-04-24T23:34:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5556ab156e60 con 0x5556ab14ec20 2024-04-24T23:34:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04dfda8740 10 client.0 _ll_register_callbacks cb 0x5556aadc5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51958/0 (socket says 172.21.15.33:51958) 2024-04-24T23:34:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04d5d9b640 1 -- 172.21.15.33:0/427183542 learned_addr learned my addr 172.21.15.33:0/427183542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.741+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d5d9b640 1 -- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 msgr2=0x5556ab1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d5d9b640 1 -- 172.21.15.33:0/427183542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 msgr2=0x5556ab1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d5d9b640 1 -- 172.21.15.33:0/427183542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ab03fdc0 con 0x5556ab153d90 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d659c640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15f820 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00f480 tx=0x7f04cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef9a0d63178b01e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04bdffb640 1 -- 172.21.15.33:0/427183542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc011260 con 0x5556ab153d90 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c80318b0 con 0x5556ab153d90 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04bdffb640 1 -- 172.21.15.33:0/427183542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04cc01a040 con 0x5556ab153d90 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04bdffb640 1 -- 172.21.15.33:0/427183542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04cc004040 con 0x5556ab153d90 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04dfda8740 10 client.20022 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5556aadc63f0 con 0x5556ab153d90 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.742+0000 7f04bdffb640 1 -- 172.21.15.33:0/427183542 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c80318b0 con 0x5556ab153d90 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04bdffb640 1 -- 172.21.15.33:0/427183542 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f04cc0047a0 con 0x5556ab153d90 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04dfda8740 10 client.20022 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04dfda8740 10 client.20022 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04dfda8740 20 client.20022 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04dfda8740 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1640a0 0x5556ab166560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04dfda8740 4 client.20022 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ab0bfbb0 con 0x5556ab1640a0 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04d659c640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1640a0 0x5556ab166560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04d659c640 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1640a0 0x5556ab166560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f20f0bdbab8eda2 server_cookie=6f7ad46f1437efae in_seq=0 out_seq=0 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.743+0000 7f04bdffb640 10 client.20022 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04bdffb640 1 -- 172.21.15.33:0/427183542 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556ab0bfbb0 con 0x5556ab1640a0 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04bdffb640 10 client.20022 handle_command_reply: tid=0 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 client.20022 shutdown 2024-04-24T23:34:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 msgr2=0x5556ab15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15f820 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00f480 tx=0x7f04cc004730 comp rx=0 tx=0).stop 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 shutdown_connections 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1640a0 0x5556ab166560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15f820 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 --2- 172.21.15.33:0/427183542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 >> 172.21.15.33:0/427183542 conn(0x5556ab1498a0 msgr2=0x5556ab148b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04bdffb640 0 client.20022 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 shutdown_connections 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 1 -- 172.21.15.33:0/427183542 wait complete. 2024-04-24T23:34:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.744+0000 7f04dfda8740 20 client.20022 trim_cache size 0 max 16384 2024-04-24T23:34:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 Processor -- start 2024-04-24T23:34:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 -- start start 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 0x5556ab1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab153d90 0x5556ab14c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81acd70 con 0x5556ab14e230 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81aa640 con 0x5556ab153d90 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04dfda8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81a8490 con 0x5556ab14ec20 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 0x5556ab1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 0x5556ab1660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51970/0 (socket says 172.21.15.33:51970) 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.758+0000 7f04d5d9b640 1 -- 172.21.15.33:0/3164855936 learned_addr learned my addr 172.21.15.33:0/3164855936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d659c640 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab153d90 0x5556ab14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 -- 172.21.15.33:0/3164855936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 msgr2=0x5556ab15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 -- 172.21.15.33:0/3164855936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab153d90 msgr2=0x5556ab14c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab153d90 0x5556ab14c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 -- 172.21.15.33:0/3164855936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556aae907d0 con 0x5556ab14e230 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d659c640 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 0x5556ab1660b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006010 tx=0x7f04cc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c8dfb9408da5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04bffff640 1 -- 172.21.15.33:0/3164855936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0122b0 con 0x5556ab14e230 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04bffff640 1 -- 172.21.15.33:0/3164855936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc0047a0 con 0x5556ab14e230 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.759+0000 7f04bffff640 1 -- 172.21.15.33:0/3164855936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc005650 con 0x5556ab14e230 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 -- 172.21.15.33:0/3164855936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 msgr2=0x5556ab1660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 0x5556ab1660b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006010 tx=0x7f04cc005850 comp rx=0 tx=0).stop 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 -- 172.21.15.33:0/3164855936 shutdown_connections 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab153d90 0x5556ab14c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- 172.21.15.33:0/3164855936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab14e230 0x5556ab1660b0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 -- 172.21.15.33:0/3164855936 >> 172.21.15.33:0/3164855936 conn(0x5556ab1498a0 msgr2=0x5556ab148b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 -- 172.21.15.33:0/3164855936 shutdown_connections 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 -- 172.21.15.33:0/3164855936 wait complete. 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 -- start start 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 Processor -- start 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.760+0000 7f04dfda8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04dfda8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81a8b00 con 0x5556ab153d90 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04dfda8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81a9f70 con 0x5556ab14e230 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04dfda8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81a91d0 con 0x5556ab14ec20 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04dfda8740 10 client.0 _ll_register_callbacks cb 0x7f04cc012580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51986/0 (socket says 172.21.15.33:51986) 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 -- 172.21.15.33:0/4150884479 learned_addr learned my addr 172.21.15.33:0/4150884479 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d659c640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 -- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 msgr2=0x5556ab15dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 -- 172.21.15.33:0/4150884479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 msgr2=0x5556ab15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 -- 172.21.15.33:0/4150884479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ab15f240 con 0x5556ab153d90 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04d5d9b640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15ec50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f04cc004770 tx=0x7f04cc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83c54b840ce0311 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04bdffb640 1 -- 172.21.15.33:0/4150884479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc03e040 con 0x5556ab153d90 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04bdffb640 1 -- 172.21.15.33:0/4150884479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04cc0053d0 con 0x5556ab153d90 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04cc0047a0 con 0x5556ab153d90 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.761+0000 7f04bdffb640 1 -- 172.21.15.33:0/4150884479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04cc0055d0 con 0x5556ab153d90 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 10 client.20028 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5556ab1caec0 con 0x5556ab153d90 2024-04-24T23:34:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04bdffb640 1 -- 172.21.15.33:0/4150884479 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc0047a0 con 0x5556ab153d90 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04bdffb640 1 -- 172.21.15.33:0/4150884479 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f04cc003a40 con 0x5556ab153d90 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 10 client.20028 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 10 client.20028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 20 client.20028 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1cb790 0x5556ab1cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 4 client.20028 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556ab0b9f00 con 0x5556ab1cb790 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04d659c640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1cb790 0x5556ab1cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04bdffb640 10 client.20028 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.762+0000 7f04d659c640 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1cb790 0x5556ab1cdbb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22fb6b0bc4747b01 server_cookie=738edb0b8164f94 in_seq=0 out_seq=0 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04bdffb640 1 -- 172.21.15.33:0/4150884479 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556ab0b9f00 con 0x5556ab1cb790 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04bdffb640 10 client.20028 handle_command_reply: tid=0 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 client.20028 shutdown 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 msgr2=0x5556ab15ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15ec50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f04cc004770 tx=0x7f04cc009fd0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 shutdown_connections 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5556ab1cb790 0x5556ab1cdbb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556ab153d90 0x5556ab15ec50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ab14ec20 0x5556ab15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 --2- 172.21.15.33:0/4150884479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556ab14e230 0x5556ab15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 >> 172.21.15.33:0/4150884479 conn(0x5556ab1498a0 msgr2=0x5556ab1482b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04bdffb640 0 client.20028 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 shutdown_connections 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 1 -- 172.21.15.33:0/4150884479 wait complete. 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dfda8740 20 client.20028 trim_cache size 0 max 16384 2024-04-24T23:34:21.768 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:34:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04c00a2b90 msgr2=0x7f04c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04c00a2b90 0x7f04c00a5050 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005ba0 tx=0x7f04d000efd0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 msgr2=0x7f04d819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 0x7f04d819f080 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f04d000eb10 tx=0x7f04d000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 shutdown_connections 2024-04-24T23:34:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04c00a2b90 0x7f04c00a5050 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8102230 0x7f04d819f080 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d8101870 0x7f04d8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 --2- 172.21.15.33:0/4184602286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04d8100680 0x7f04d8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.763+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 >> 172.21.15.33:0/4184602286 conn(0x7f04d80fbbf0 msgr2=0x7f04d80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.764+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 shutdown_connections 2024-04-24T23:34:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.764+0000 7f04dcfc4640 1 -- 172.21.15.33:0/4184602286 wait complete. 2024-04-24T23:34:21.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:34:21.901 INFO:tasks.workunit.client.0.smithi033.stdout: 10 213278 22.14 MB/sec execute 386 sec latency 90.426 ms 2024-04-24T23:34:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/1528993357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0103470 msgr2=0x7fcdc0109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/1528993357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0103470 0x7fcdc0109d00 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009ca0 tx=0x7fcdb802a8d0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/1528993357 shutdown_connections 2024-04-24T23:34:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/1528993357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0103470 0x7fcdc0109d00 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/1528993357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0102ab0 0x7fcdc0102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/1528993357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc00ffa40 0x7fcdc00ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.925+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/1528993357 >> 172.21.15.33:0/1528993357 conn(0x7fcdc00fb370 msgr2=0x7fcdc00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.926+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/1528993357 shutdown_connections 2024-04-24T23:34:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.926+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/1528993357 wait complete. 2024-04-24T23:34:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.926+0000 7fcdc7bdc640 1 Processor -- start 2024-04-24T23:34:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.926+0000 7fcdc7bdc640 1 -- start start 2024-04-24T23:34:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc7bdc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc00ffa40 0x7fcdc01a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc7bdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 0x7fcdc01a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc7bdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0103470 0x7fcdc01a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc7bdc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc006bbb0 con 0x7fcdc0102ab0 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc7bdc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc006ba30 con 0x7fcdc0103470 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc7bdc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc006bd30 con 0x7fcdc00ffa40 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 0x7fcdc01a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5978640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc00ffa40 0x7fcdc01a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 0x7fcdc01a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52002/0 (socket says 172.21.15.33:52002) 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 -- 172.21.15.33:0/217063249 learned_addr learned my addr 172.21.15.33:0/217063249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc6179640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0103470 0x7fcdc01a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 -- 172.21.15.33:0/217063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc00ffa40 msgr2=0x7fcdc01a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc00ffa40 0x7fcdc01a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 -- 172.21.15.33:0/217063249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0103470 msgr2=0x7fcdc01a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0103470 0x7fcdc01a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.927+0000 7fcdc5177640 1 -- 172.21.15.33:0/217063249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc01aa050 con 0x7fcdc0102ab0 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.928+0000 7fcdc6179640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0103470 0x7fcdc01a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.928+0000 7fcdc5177640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 0x7fcdc01a2e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00058c0 tx=0x7fcdb0007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.928+0000 7fcdb6ffd640 1 -- 172.21.15.33:0/217063249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0013020 con 0x7fcdc0102ab0 2024-04-24T23:34:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.928+0000 7fcdb6ffd640 1 -- 172.21.15.33:0/217063249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb00138a0 con 0x7fcdc0102ab0 2024-04-24T23:34:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.928+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc0103d30 con 0x7fcdc0102ab0 2024-04-24T23:34:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.929+0000 7fcdb6ffd640 1 -- 172.21.15.33:0/217063249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000f670 con 0x7fcdc0102ab0 2024-04-24T23:34:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.929+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc0104220 con 0x7fcdc0102ab0 2024-04-24T23:34:21.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.935+0000 7fcdca9ca740 1 Processor -- start 2024-04-24T23:34:21.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.935+0000 7fcdb6ffd640 1 -- 172.21.15.33:0/217063249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcdb000c070 con 0x7fcdc0102ab0 2024-04-24T23:34:21.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdb6ffd640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdac0a2d60 0x7fcdac0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 -- start start 2024-04-24T23:34:21.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042d230 0x55ce9042d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042dc20 0x55ce90432720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce90435220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce90391670 con 0x55ce9042dc20 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ce90339970 con 0x55ce90432d90 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce9038d630 con 0x55ce9042d230 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.936+0000 7fcdc6179640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdac0a2d60 0x7fcdac0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042dc20 0x55ce90432720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042dc20 0x55ce90432720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52012/0 (socket says 172.21.15.33:52012) 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5177640 1 -- 172.21.15.33:0/1094303883 learned_addr learned my addr 172.21.15.33:0/1094303883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5978640 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042d230 0x55ce9042d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdb6ffd640 1 -- 172.21.15.33:0/217063249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcdb00cec60 con 0x7fcdc0102ab0 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5978640 1 -- 172.21.15.33:0/1094303883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 msgr2=0x55ce90435220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5978640 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce90435220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5978640 1 -- 172.21.15.33:0/1094303883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042dc20 msgr2=0x55ce90432720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5978640 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042dc20 0x55ce90432720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.937+0000 7fcdc5978640 1 -- 172.21.15.33:0/1094303883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9035d400 con 0x55ce9042d230 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.938+0000 7fcdc5978640 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042d230 0x55ce9042d650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005fd0 tx=0x7fcdbc00bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7173dca0156562de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.938+0000 7fcdc6179640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdac0a2d60 0x7fcdac0a5220 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8004af0 tx=0x7fcdb80093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.938+0000 7fcdabfff640 1 -- 172.21.15.33:0/1094303883 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc003da0 con 0x55ce9042d230 2024-04-24T23:34:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.938+0000 7fcdabfff640 1 -- 172.21.15.33:0/1094303883 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc015070 con 0x55ce9042d230 2024-04-24T23:34:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.939+0000 7fcdabfff640 1 -- 172.21.15.33:0/1094303883 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0116a0 con 0x55ce9042d230 2024-04-24T23:34:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.939+0000 7fcdca9ca740 1 -- 172.21.15.33:0/1094303883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042d230 msgr2=0x55ce9042d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.939+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042d230 0x55ce9042d650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005fd0 tx=0x7fcdbc00bbe0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.942+0000 7fcdca9ca740 1 -- 172.21.15.33:0/1094303883 shutdown_connections 2024-04-24T23:34:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.942+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce90435220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.942+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042dc20 0x55ce90432720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.942+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/1094303883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042d230 0x55ce9042d650 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.942+0000 7fcdca9ca740 1 -- 172.21.15.33:0/1094303883 >> 172.21.15.33:0/1094303883 conn(0x55ce904288a0 msgr2=0x55ce90428ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.942+0000 7fcdca9ca740 1 -- 172.21.15.33:0/1094303883 shutdown_connections 2024-04-24T23:34:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.943+0000 7fcdca9ca740 1 -- 172.21.15.33:0/1094303883 wait complete. 2024-04-24T23:34:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.943+0000 7fcdca9ca740 1 -- start start 2024-04-24T23:34:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.943+0000 7fcdca9ca740 1 Processor -- start 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.945+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 0x55ce90437350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.945+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042dc20 0x55ce90437890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.945+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce9043e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.945+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce90435d40 con 0x55ce9042d230 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.945+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ce9034d050 con 0x55ce90432d90 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.945+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce90435ec0 con 0x55ce9042dc20 2024-04-24T23:34:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdca9ca740 10 client.0 _ll_register_callbacks cb 0x55ce900a4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 0x55ce90437350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc6179640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce9043e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 0x55ce90437350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52028/0 (socket says 172.21.15.33:52028) 2024-04-24T23:34:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 -- 172.21.15.33:0/3301622378 learned_addr learned my addr 172.21.15.33:0/3301622378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042dc20 0x55ce90437890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 -- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042dc20 msgr2=0x55ce90437890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042dc20 0x55ce90437890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 -- 172.21.15.33:0/3301622378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 msgr2=0x55ce9043e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce9043e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.946+0000 7fcdc5978640 1 -- 172.21.15.33:0/3301622378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce9031edc0 con 0x55ce9042d230 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.947+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 0x55ce90437350 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0043b0 tx=0x7fcdbc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a517fa177deb07d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.947+0000 7fcda97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.947+0000 7fcda97fa640 1 -- 172.21.15.33:0/3301622378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc01a880 con 0x55ce9042d230 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.948+0000 7fcda97fa640 1 -- 172.21.15.33:0/3301622378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcdbc005040 con 0x55ce9042d230 2024-04-24T23:34:21.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.948+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdbc015070 con 0x55ce9042d230 2024-04-24T23:34:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.949+0000 7fcda97fa640 1 -- 172.21.15.33:0/3301622378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc031050 con 0x55ce9042d230 2024-04-24T23:34:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.949+0000 7fcdca9ca740 10 client.20040 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:21.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.949+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ce900a53f0 con 0x55ce9042d230 2024-04-24T23:34:21.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.949+0000 7fcda97fa640 1 -- 172.21.15.33:0/3301622378 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc028070 con 0x55ce9042d230 2024-04-24T23:34:21.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.950+0000 7fcda97fa640 1 -- 172.21.15.33:0/3301622378 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcdbc004b70 con 0x55ce9042d230 2024-04-24T23:34:21.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdca9ca740 10 client.20040 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:21.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdca9ca740 10 client.20040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:21.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdca9ca740 20 client.20040 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:21.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce90442e70 0x55ce90445260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdca9ca740 4 client.20040 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:21.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcdbc015070 con 0x55ce90442e70 2024-04-24T23:34:21.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce90442e70 0x55ce90445260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce90442e70 0x55ce90445260 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85b4c45826e66e2 server_cookie=c2ca634fdf55bbc6 in_seq=0 out_seq=0 2024-04-24T23:34:21.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.951+0000 7fcda97fa640 10 client.20040 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:21.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.953+0000 7fcda97fa640 1 -- 172.21.15.33:0/3301622378 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fcdbc015070 con 0x55ce90442e70 2024-04-24T23:34:21.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.953+0000 7fcda97fa640 10 client.20040 handle_command_reply: tid=0 2024-04-24T23:34:21.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.953+0000 7fcdca9ca740 1 client.20040 shutdown 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 msgr2=0x55ce90437350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 0x55ce90437350 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0043b0 tx=0x7fcdbc00bfd0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 shutdown_connections 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce90442e70 0x55ce90445260 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce90432d90 0x55ce9043e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9042dc20 0x55ce90437890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3301622378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce9042d230 0x55ce90437350 secure :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0043b0 tx=0x7fcdbc00bfd0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 >> 172.21.15.33:0/3301622378 conn(0x55ce904288a0 msgr2=0x55ce90427b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.954+0000 7fcda97fa640 0 client.20040 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:21.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.955+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 shutdown_connections 2024-04-24T23:34:21.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.955+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3301622378 wait complete. 2024-04-24T23:34:21.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.955+0000 7fcdca9ca740 20 client.20040 trim_cache size 0 max 16384 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 Processor -- start 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 -- start start 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9042be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9042c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce904369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40124d0 con 0x55ce90432d90 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400fda0 con 0x55ce9042dc20 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.972+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400dbf0 con 0x55ce9043e890 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9042c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9042c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52040/0 (socket says 172.21.15.33:52040) 2024-04-24T23:34:21.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 -- 172.21.15.33:0/3495583524 learned_addr learned my addr 172.21.15.33:0/3495583524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9042be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc6179640 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce904369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 -- 172.21.15.33:0/3495583524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 msgr2=0x55ce904369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce904369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 -- 172.21.15.33:0/3495583524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 msgr2=0x55ce9042be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9042be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 -- 172.21.15.33:0/3495583524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9016f7d0 con 0x55ce90432d90 2024-04-24T23:34:21.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9042c3d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009fd0 tx=0x7fcdb0080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8261087b5765b5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdab7fe640 1 -- 172.21.15.33:0/3495583524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb009b0d0 con 0x55ce90432d90 2024-04-24T23:34:21.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdab7fe640 1 -- 172.21.15.33:0/3495583524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb009f480 con 0x55ce90432d90 2024-04-24T23:34:21.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.973+0000 7fcdab7fe640 1 -- 172.21.15.33:0/3495583524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0084cb0 con 0x55ce90432d90 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3495583524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 msgr2=0x55ce9042c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9042c3d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009fd0 tx=0x7fcdb0080040 comp rx=0 tx=0).stop 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3495583524 shutdown_connections 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce904369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9042c3d0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3495583524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9042be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3495583524 >> 172.21.15.33:0/3495583524 conn(0x55ce904288a0 msgr2=0x55ce9042b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3495583524 shutdown_connections 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.974+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3495583524 wait complete. 2024-04-24T23:34:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.975+0000 7fcdca9ca740 1 -- start start 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.975+0000 7fcdca9ca740 1 Processor -- start 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.975+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9043b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.975+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9043baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdca9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce9043cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400e260 con 0x55ce90432d90 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400f6d0 con 0x55ce9042dc20 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdca9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400e930 con 0x55ce9043e890 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9043baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9043baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52046/0 (socket says 172.21.15.33:52046) 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdca9ca740 10 client.0 _ll_register_callbacks cb 0x7fcdbc004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 -- 172.21.15.33:0/3912681944 learned_addr learned my addr 172.21.15.33:0/3912681944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9043b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc6179640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce9043cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 -- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 msgr2=0x55ce9043cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce9043cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 -- 172.21.15.33:0/3912681944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 msgr2=0x55ce9043b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9043b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 -- 172.21.15.33:0/3912681944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce9043d0a0 con 0x55ce90432d90 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9043b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdc5177640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9043baf0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00bb120 tx=0x7fcdb0007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628d91126e29d289 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcda97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcda97fa640 1 -- 172.21.15.33:0/3912681944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb009b0d0 con 0x55ce90432d90 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcda97fa640 1 -- 172.21.15.33:0/3912681944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcdb00976b0 con 0x55ce90432d90 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.976+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb009f480 con 0x55ce90432d90 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcda97fa640 1 -- 172.21.15.33:0/3912681944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcdb00a3d40 con 0x55ce90432d90 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 10 client.20046 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ce9043e3a0 con 0x55ce90432d90 2024-04-24T23:34:21.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcda97fa640 1 -- 172.21.15.33:0/3912681944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb009f480 con 0x55ce90432d90 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcda97fa640 1 -- 172.21.15.33:0/3912681944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcdb0095bb0 con 0x55ce90432d90 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 10 client.20046 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 10 client.20046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 20 client.20046 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce904a9760 0x55ce904abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 4 client.20046 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce9039ebb0 con 0x55ce904a9760 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.977+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce904a9760 0x55ce904abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.978+0000 7fcdc5978640 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce904a9760 0x55ce904abb80 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ea26fdf3a59247 server_cookie=9ce1c780625cd91c in_seq=0 out_seq=0 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.978+0000 7fcda97fa640 10 client.20046 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.979+0000 7fcda97fa640 1 -- 172.21.15.33:0/3912681944 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce9039ebb0 con 0x55ce904a9760 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.979+0000 7fcda97fa640 10 client.20046 handle_command_reply: tid=0 2024-04-24T23:34:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 client.20046 shutdown 2024-04-24T23:34:21.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 msgr2=0x55ce9043baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9043baf0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00bb120 tx=0x7fcdb0007bf0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 shutdown_connections 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55ce904a9760 0x55ce904abb80 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce9043e890 0x55ce9043cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce90432d90 0x55ce9043baf0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 --2- 172.21.15.33:0/3912681944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ce9042dc20 0x55ce9043b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 >> 172.21.15.33:0/3912681944 conn(0x55ce904288a0 msgr2=0x55ce9042c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcda97fa640 0 client.20046 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 shutdown_connections 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 1 -- 172.21.15.33:0/3912681944 wait complete. 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdca9ca740 20 client.20046 trim_cache size 0 max 16384 2024-04-24T23:34:21.984 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:34:21.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdac0a2d60 msgr2=0x7fcdac0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.980+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdac0a2d60 0x7fcdac0a5220 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8004af0 tx=0x7fcdb80093f0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 msgr2=0x7fcdc01a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 0x7fcdc01a2e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00058c0 tx=0x7fcdb0007590 comp rx=0 tx=0).stop 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 shutdown_connections 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcdac0a2d60 0x7fcdac0a5220 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc0103470 0x7fcdc01a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdc0102ab0 0x7fcdc01a2e20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 --2- 172.21.15.33:0/217063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc00ffa40 0x7fcdc01a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 >> 172.21.15.33:0/217063249 conn(0x7fcdc00fb370 msgr2=0x7fcdc01008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 shutdown_connections 2024-04-24T23:34:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:21.981+0000 7fcdc7bdc640 1 -- 172.21.15.33:0/217063249 wait complete. 2024-04-24T23:34:22.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:22 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/290146877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:34:22.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:22 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/290146877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:34:22.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:22 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/290146877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:34:22.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 213660 22.12 MB/sec execute 387 sec latency 124.754 ms 2024-04-24T23:34:22.993 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:34:22.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:34:22.994 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 -- 172.21.15.33:0/21349107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 msgr2=0x7fcc0c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 --2- 172.21.15.33:0/21349107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c101bb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009900 tx=0x7fcbfc02a410 comp rx=0 tx=0).stop 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 -- 172.21.15.33:0/21349107 shutdown_connections 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 --2- 172.21.15.33:0/21349107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc0c103340 0x7fcc0c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 --2- 172.21.15.33:0/21349107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 0x7fcc0c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 --2- 172.21.15.33:0/21349107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c101bb0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 -- 172.21.15.33:0/21349107 >> 172.21.15.33:0/21349107 conn(0x7fcc0c0fcd20 msgr2=0x7fcc0c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 -- 172.21.15.33:0/21349107 shutdown_connections 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.117+0000 7fcc108d6640 1 -- 172.21.15.33:0/21349107 wait complete. 2024-04-24T23:34:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 Processor -- start 2024-04-24T23:34:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 -- start start 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 0x7fcc0c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc0c103340 0x7fcc0c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c0758b0 con 0x7fcc0c101790 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c075730 con 0x7fcc0c103340 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc108d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c10f8b0 con 0x7fcc0c102980 2024-04-24T23:34:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 0x7fcc0c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52070/0 (socket says 172.21.15.33:52070) 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc0a59c640 1 -- 172.21.15.33:0/2303956491 learned_addr learned my addr 172.21.15.33:0/2303956491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc0c103340 0x7fcc0c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc09d9b640 1 -- 172.21.15.33:0/2303956491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc0c103340 msgr2=0x7fcc0c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc0c103340 0x7fcc0c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc09d9b640 1 -- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 msgr2=0x7fcc0c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc09d9b640 1 -- 172.21.15.33:0/2303956491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c1a1900 con 0x7fcc0c102980 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.118+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.119+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 0x7fcc0c19a230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000e9f0 tx=0x7fcc0000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.119+0000 7fcbfb7fe640 1 -- 172.21.15.33:0/2303956491 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000ccc0 con 0x7fcc0c102980 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.119+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c1a1bf0 con 0x7fcc0c102980 2024-04-24T23:34:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.119+0000 7fcbfb7fe640 1 -- 172.21.15.33:0/2303956491 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc00010910 con 0x7fcc0c102980 2024-04-24T23:34:23.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.119+0000 7fcbfb7fe640 1 -- 172.21.15.33:0/2303956491 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00018960 con 0x7fcc0c102980 2024-04-24T23:34:23.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.119+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c1a2190 con 0x7fcc0c102980 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.121+0000 7fcbfb7fe640 1 -- 172.21.15.33:0/2303956491 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcc00010ac0 con 0x7fcc0c102980 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.121+0000 7fcc135e2740 1 Processor -- start 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 -- start start 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf2911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf296280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf298d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557bcf1f42c0 con 0x557bcf290d90 2024-04-24T23:34:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557bcf19c910 con 0x557bcf291780 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc135e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557bcf1f0280 con 0x557bcf2968f0 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf2911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf298d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf2911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52082/0 (socket says 172.21.15.33:52082) 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc0ad9d640 1 -- 172.21.15.33:0/1161773268 learned_addr learned my addr 172.21.15.33:0/1161773268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf296280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.122+0000 7fcbfb7fe640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcbf40a2b90 0x7fcbf40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc09d9b640 1 -- 172.21.15.33:0/1161773268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 msgr2=0x557bcf296280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf296280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc09d9b640 1 -- 172.21.15.33:0/1161773268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 msgr2=0x557bcf2911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf2911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc09d9b640 1 -- 172.21.15.33:0/1161773268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bcf1adfb0 con 0x557bcf2968f0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcbf40a2b90 0x7fcbf40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcbfb7fe640 1 -- 172.21.15.33:0/2303956491 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcc000cf850 con 0x7fcc0c102980 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf298d80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc000ccf00 tx=0x7fcc00066960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ccff10e1c8d7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcbf8ff9640 1 -- 172.21.15.33:0/1161773268 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0009b8b0 con 0x557bcf2968f0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcbf8ff9640 1 -- 172.21.15.33:0/1161773268 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0006aa70 con 0x557bcf2968f0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.123+0000 7fcbf8ff9640 1 -- 172.21.15.33:0/1161773268 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00099eb0 con 0x557bcf2968f0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 -- 172.21.15.33:0/1161773268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 msgr2=0x557bcf298d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcbf40a2b90 0x7fcbf40a5050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x557bcf291fe0 tx=0x7fcc04004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf298d80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc000ccf00 tx=0x7fcc00066960 comp rx=0 tx=0).stop 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 -- 172.21.15.33:0/1161773268 shutdown_connections 2024-04-24T23:34:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf298d80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf296280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 --2- 172.21.15.33:0/1161773268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf2911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 -- 172.21.15.33:0/1161773268 >> 172.21.15.33:0/1161773268 conn(0x557bcf258950 msgr2=0x557bcf28c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 -- 172.21.15.33:0/1161773268 shutdown_connections 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 -- 172.21.15.33:0/1161773268 wait complete. 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 -- start start 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.124+0000 7fcc135e2740 1 Processor -- start 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf290d90 0x557bcf29b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 0x557bcf2a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557bcf299840 con 0x557bcf2968f0 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557bcf1afe90 con 0x557bcf290d90 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557bcf2999c0 con 0x557bcf291780 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc135e2740 10 client.0 _ll_register_callbacks cb 0x557bcf1eb970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 0x557bcf2a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf290d90 0x557bcf29b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 0x557bcf2a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52094/0 (socket says 172.21.15.33:52094) 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf290d90 0x557bcf29b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53450/0 (socket says 172.21.15.33:53450) 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 -- 172.21.15.33:0/2519123182 learned_addr learned my addr 172.21.15.33:0/2519123182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 -- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 msgr2=0x557bcf29b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 -- 172.21.15.33:0/2519123182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf290d90 msgr2=0x557bcf29b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf290d90 0x557bcf29b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 -- 172.21.15.33:0/2519123182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bcefd07e0 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 0x557bcf2a26a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004650 tx=0x7fcc04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dd4d1f6674b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.125+0000 7fcbf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/2519123182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0401a070 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/2519123182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcc04004d20 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0006aa70 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/2519123182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcc04020e50 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 10 client.20058 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557bcf030f70 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/2519123182 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0006aa70 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/2519123182 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcc04021430 con 0x557bcf2968f0 2024-04-24T23:34:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 10 client.20058 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 10 client.20058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 20 client.20058 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf2a6d90 0x557bcf2a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 4 client.20058 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bcf1c85a0 con 0x557bcf2a6d90 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.126+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf2a6d90 0x557bcf2a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.127+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf2a6d90 0x557bcf2a91c0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6593b108f04a780a server_cookie=641f8e5c16f9e98a in_seq=0 out_seq=0 2024-04-24T23:34:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.127+0000 7fcbf1ffb640 10 client.20058 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/2519123182 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bcf1c85a0 con 0x557bcf2a6d90 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcbf1ffb640 10 client.20058 handle_command_reply: tid=0 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 client.20058 shutdown 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 msgr2=0x557bcf2a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 0x557bcf2a26a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004650 tx=0x7fcc04004a40 comp rx=0 tx=0).stop 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 shutdown_connections 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf2a6d90 0x557bcf2a91c0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf2968f0 0x557bcf2a26a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 --2- 172.21.15.33:0/2519123182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf290d90 0x557bcf29b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 >> 172.21.15.33:0/2519123182 conn(0x557bcf258950 msgr2=0x557bcf28b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcbf1ffb640 0 client.20058 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 shutdown_connections 2024-04-24T23:34:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.128+0000 7fcc135e2740 1 -- 172.21.15.33:0/2519123182 wait complete. 2024-04-24T23:34:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.129+0000 7fcc135e2740 20 client.20058 trim_cache size 0 max 16384 2024-04-24T23:34:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 Processor -- start 2024-04-24T23:34:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 -- start start 2024-04-24T23:34:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf292bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf2930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf2a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1aee70 con 0x557bcf290d90 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1ac740 con 0x557bcf291780 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc135e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1aa590 con 0x557bcf2968f0 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf292bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf292bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52096/0 (socket says 172.21.15.33:52096) 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 -- 172.21.15.33:0/108352137 learned_addr learned my addr 172.21.15.33:0/108352137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf2930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf2a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 -- 172.21.15.33:0/108352137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 msgr2=0x557bcf2a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf2a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 -- 172.21.15.33:0/108352137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 msgr2=0x557bcf2930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf2930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.148+0000 7fcc0a59c640 1 -- 172.21.15.33:0/108352137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bceec8170 con 0x557bcf290d90 2024-04-24T23:34:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.149+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf292bb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0314e0 tx=0x7fcbfc032a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea8db3e4501c343 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.149+0000 7fcbf3fff640 1 -- 172.21.15.33:0/108352137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc035040 con 0x557bcf290d90 2024-04-24T23:34:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.149+0000 7fcbf3fff640 1 -- 172.21.15.33:0/108352137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc04a3d0 con 0x557bcf290d90 2024-04-24T23:34:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.149+0000 7fcbf3fff640 1 -- 172.21.15.33:0/108352137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0053a0 con 0x557bcf290d90 2024-04-24T23:34:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- 172.21.15.33:0/108352137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 msgr2=0x557bcf292bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf292bb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0314e0 tx=0x7fcbfc032a00 comp rx=0 tx=0).stop 2024-04-24T23:34:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- 172.21.15.33:0/108352137 shutdown_connections 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf2968f0 0x557bcf2a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf291780 0x557bcf2930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- 172.21.15.33:0/108352137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf292bb0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- 172.21.15.33:0/108352137 >> 172.21.15.33:0/108352137 conn(0x557bcf258950 msgr2=0x557bcf29b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- 172.21.15.33:0/108352137 shutdown_connections 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- 172.21.15.33:0/108352137 wait complete. 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- start start 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 Processor -- start 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf29f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf2968f0 0x557bcf30b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1aac00 con 0x557bcf290d90 2024-04-24T23:34:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1ac070 con 0x557bcf2968f0 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.150+0000 7fcc135e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1ab2d0 con 0x557bcf291780 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc135e2740 10 client.0 _ll_register_callbacks cb 0x7fcc04011ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf29f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf29f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52106/0 (socket says 172.21.15.33:52106) 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 -- 172.21.15.33:0/3503964058 learned_addr learned my addr 172.21.15.33:0/3503964058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf2968f0 0x557bcf30b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 -- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 msgr2=0x557bcf29f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 -- 172.21.15.33:0/3503964058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf2968f0 msgr2=0x557bcf30b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf2968f0 0x557bcf30b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 -- 172.21.15.33:0/3503964058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bcf29fff0 con 0x557bcf290d90 2024-04-24T23:34:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0ad9d640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf2968f0 0x557bcf30b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc0a59c640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf29f440 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc04c040 tx=0x7fcbfc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65833bc0ff84b929 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcbf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/3503964058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc057070 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbfc04a3d0 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.151+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/3503964058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcbfc004390 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/3503964058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcbfc035070 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 10 client.20064 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557bcf30c970 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/3503964058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc04a3d0 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/3503964058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcbfc032280 con 0x557bcf290d90 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 10 client.20064 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 10 client.20064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 20 client.20064 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf30d350 0x557bcf30f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 4 client.20064 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:34:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557bcf201800 con 0x557bcf30d350 2024-04-24T23:34:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.152+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf30d350 0x557bcf30f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.153+0000 7fcc09d9b640 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf30d350 0x557bcf30f770 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9554cd080f70c108 server_cookie=939c032f00bc5b8a in_seq=0 out_seq=0 2024-04-24T23:34:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.153+0000 7fcbf1ffb640 10 client.20064 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.154+0000 7fcbf1ffb640 1 -- 172.21.15.33:0/3503964058 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557bcf201800 con 0x557bcf30d350 2024-04-24T23:34:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.154+0000 7fcbf1ffb640 10 client.20064 handle_command_reply: tid=0 2024-04-24T23:34:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.154+0000 7fcc135e2740 1 client.20064 shutdown 2024-04-24T23:34:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 msgr2=0x557bcf29f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf29f440 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc04c040 tx=0x7fcbfc009260 comp rx=0 tx=0).stop 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 shutdown_connections 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557bcf30d350 0x557bcf30f770 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcbf1ffb640 0 client.20064 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557bcf2968f0 0x557bcf30b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bcf291780 0x557bcf29f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 --2- 172.21.15.33:0/3503964058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bcf290d90 0x557bcf29f440 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 >> 172.21.15.33:0/3503964058 conn(0x557bcf258950 msgr2=0x557bcf29b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 shutdown_connections 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 1 -- 172.21.15.33:0/3503964058 wait complete. 2024-04-24T23:34:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.155+0000 7fcc135e2740 20 client.20064 trim_cache size 0 max 16384 2024-04-24T23:34:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:34:23.160 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:34:23.160 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f88da3c4-0b8f-49a3-a5da-b58d4005d3d2", 2024-04-24T23:34:23.160 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:34:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcbf40a2b90 msgr2=0x7fcbf40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcbf40a2b90 0x7fcbf40a5050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x557bcf291fe0 tx=0x7fcc04004ab0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 msgr2=0x7fcc0c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 0x7fcc0c19a230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000e9f0 tx=0x7fcc0000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 shutdown_connections 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcbf40a2b90 0x7fcbf40a5050 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc0c103340 0x7fcc0c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc0c102980 0x7fcc0c19a230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 --2- 172.21.15.33:0/2303956491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc0c101790 0x7fcc0c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 >> 172.21.15.33:0/2303956491 conn(0x7fcc0c0fcd20 msgr2=0x7fcc0c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 shutdown_connections 2024-04-24T23:34:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.156+0000 7fcc108d6640 1 -- 172.21.15.33:0/2303956491 wait complete. 2024-04-24T23:34:23.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:34:23.168 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:34:23.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 -- 172.21.15.33:0/415387444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c10a1b0 msgr2=0x7f045c110a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 --2- 172.21.15.33:0/415387444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c10a1b0 0x7f045c110a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f044c009860 tx=0x7f044c02a340 comp rx=0 tx=0).stop 2024-04-24T23:34:23.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 -- 172.21.15.33:0/415387444 shutdown_connections 2024-04-24T23:34:23.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 --2- 172.21.15.33:0/415387444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c10a1b0 0x7f045c110a40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 --2- 172.21.15.33:0/415387444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045c1097f0 0x7f045c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 --2- 172.21.15.33:0/415387444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c108b90 0x7f045c108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.319+0000 7f0463d7e640 1 -- 172.21.15.33:0/415387444 >> 172.21.15.33:0/415387444 conn(0x7f045c06e970 msgr2=0x7f045c070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.320+0000 7f0463d7e640 1 -- 172.21.15.33:0/415387444 shutdown_connections 2024-04-24T23:34:23.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.320+0000 7f0463d7e640 1 -- 172.21.15.33:0/415387444 wait complete. 2024-04-24T23:34:23.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 Processor -- start 2024-04-24T23:34:23.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 -- start start 2024-04-24T23:34:23.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045c108b90 0x7f045c1a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c1097f0 0x7f045c1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c10a1b0 0x7f045c1a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f045c114270 con 0x7f045c108b90 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1140f0 con 0x7f045c1097f0 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.321+0000 7f0463d7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1143f0 con 0x7f045c10a1b0 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f0461b1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045c108b90 0x7f045c1a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f0461319640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c1097f0 0x7f045c1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f0461319640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c1097f0 0x7f045c1a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53482/0 (socket says 172.21.15.33:53482) 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f0461319640 1 -- 172.21.15.33:0/413552428 learned_addr learned my addr 172.21.15.33:0/413552428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f046231b640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c10a1b0 0x7f045c1a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f046231b640 1 -- 172.21.15.33:0/413552428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c1097f0 msgr2=0x7f045c1a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f046231b640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c1097f0 0x7f045c1a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f046231b640 1 -- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045c108b90 msgr2=0x7f045c1a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f046231b640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045c108b90 0x7f045c1a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.322+0000 7f046231b640 1 -- 172.21.15.33:0/413552428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c1aa0e0 con 0x7f045c10a1b0 2024-04-24T23:34:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.323+0000 7f046231b640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c10a1b0 0x7f045c1a9a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f044c009780 tx=0x7f044c033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.324+0000 7f0452ffd640 1 -- 172.21.15.33:0/413552428 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c037040 con 0x7f045c10a1b0 2024-04-24T23:34:23.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.324+0000 7f0452ffd640 1 -- 172.21.15.33:0/413552428 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c030850 con 0x7f045c10a1b0 2024-04-24T23:34:23.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.324+0000 7f0452ffd640 1 -- 172.21.15.33:0/413552428 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c004920 con 0x7f045c10a1b0 2024-04-24T23:34:23.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.325+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c1aa370 con 0x7f045c10a1b0 2024-04-24T23:34:23.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.325+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1aa810 con 0x7f045c10a1b0 2024-04-24T23:34:23.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.329+0000 7f0466b69740 1 Processor -- start 2024-04-24T23:34:23.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:23 smithi033 ceph-mon[29201]: pgmap v1022: 129 pgs: 129 active+clean; 639 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 373 op/s 2024-04-24T23:34:23.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:23 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:34:23.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.330+0000 7f0452ffd640 1 -- 172.21.15.33:0/413552428 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f044c005030 con 0x7f045c10a1b0 2024-04-24T23:34:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.330+0000 7f0466b69740 1 -- start start 2024-04-24T23:34:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.331+0000 7f0466b69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa23be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.331+0000 7f0466b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa240f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.331+0000 7f0466b69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa243a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.331+0000 7f0466b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b7aa1976b0 con 0x55b7aa23c440 2024-04-24T23:34:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.331+0000 7f0466b69740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b7aa13f880 con 0x55b7aa23ba50 2024-04-24T23:34:23.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.331+0000 7f0466b69740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b7aa193670 con 0x55b7aa2415b0 2024-04-24T23:34:23.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.332+0000 7f0452ffd640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04480a2d10 0x7f04480a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.332+0000 7f0452ffd640 1 -- 172.21.15.33:0/413552428 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f044c0eed80 con 0x7f045c10a1b0 2024-04-24T23:34:23.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.332+0000 7f0461b1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa240f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.332+0000 7f0461b1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa240f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52116/0 (socket says 172.21.15.33:52116) 2024-04-24T23:34:23.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.332+0000 7f0461b1a640 1 -- 172.21.15.33:0/444763679 learned_addr learned my addr 172.21.15.33:0/444763679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.332+0000 7f046231b640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04480a2d10 0x7f04480a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.333+0000 7f0461b1a640 1 -- 172.21.15.33:0/444763679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 msgr2=0x55b7aa243a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.333+0000 7f046231b640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04480a2d10 0x7f04480a51d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f044c0c3990 tx=0x7f044c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.333+0000 7f0461b1a640 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa243a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.333+0000 7f0461b1a640 1 -- 172.21.15.33:0/444763679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 msgr2=0x55b7aa23be70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.333+0000 7f0461b1a640 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa23be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.333+0000 7f0461b1a640 1 -- 172.21.15.33:0/444763679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a9f757d0 con 0x55b7aa23c440 2024-04-24T23:34:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.334+0000 7f0461b1a640 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa240f40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0454009ca0 tx=0x7f0454011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83af4d86c55a8eff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.334+0000 7f0447fff640 1 -- 172.21.15.33:0/444763679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454004220 con 0x55b7aa23c440 2024-04-24T23:34:23.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.334+0000 7f0447fff640 1 -- 172.21.15.33:0/444763679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04540043c0 con 0x55b7aa23c440 2024-04-24T23:34:23.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.334+0000 7f0447fff640 1 -- 172.21.15.33:0/444763679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454004220 con 0x55b7aa23c440 2024-04-24T23:34:23.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.335+0000 7f0466b69740 1 -- 172.21.15.33:0/444763679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 msgr2=0x55b7aa240f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.335+0000 7f0466b69740 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa240f40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0454009ca0 tx=0x7f0454011810 comp rx=0 tx=0).stop 2024-04-24T23:34:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 -- 172.21.15.33:0/444763679 shutdown_connections 2024-04-24T23:34:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa243a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa240f40 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 --2- 172.21.15.33:0/444763679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa23be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 -- 172.21.15.33:0/444763679 >> 172.21.15.33:0/444763679 conn(0x55b7aa2370c0 msgr2=0x55b7aa2374c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 -- 172.21.15.33:0/444763679 shutdown_connections 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 -- 172.21.15.33:0/444763679 wait complete. 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 -- start start 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.336+0000 7f0466b69740 1 Processor -- start 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa2300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa230600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa235630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b7aa22e720 con 0x55b7aa23c440 2024-04-24T23:34:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b7aa152f50 con 0x55b7aa23ba50 2024-04-24T23:34:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b7aa22e8a0 con 0x55b7aa2415b0 2024-04-24T23:34:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0466b69740 10 client.0 _ll_register_callbacks cb 0x55b7aa18e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0461319640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa2300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0461319640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa2300c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:53498/0 (socket says 172.21.15.33:53498) 2024-04-24T23:34:23.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0461319640 1 -- 172.21.15.33:0/3612510046 learned_addr learned my addr 172.21.15.33:0/3612510046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.337+0000 7f0461319640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa235630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.338+0000 7f0461b1a640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa230600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.338+0000 7f0461b1a640 1 -- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 msgr2=0x55b7aa235630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.338+0000 7f0461b1a640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa235630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.338+0000 7f0461b1a640 1 -- 172.21.15.33:0/3612510046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 msgr2=0x55b7aa2300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.338+0000 7f0461b1a640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa2300c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.338+0000 7f0461b1a640 1 -- 172.21.15.33:0/3612510046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a9eab3f0 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f0461b1a640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa230600 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x55b7aa238030 tx=0x7f0454003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38032e8dd1e659b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f04457fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f04457fa640 1 -- 172.21.15.33:0/3612510046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454028070 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f04457fa640 1 -- 172.21.15.33:0/3612510046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0454004020 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04540043c0 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f04457fa640 1 -- 172.21.15.33:0/3612510046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0454034050 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f0466b69740 10 client.20076 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.339+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7a9e6b0f0 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f04457fa640 1 -- 172.21.15.33:0/3612510046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04540043c0 con 0x55b7aa23c440 2024-04-24T23:34:23.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f04457fa640 1 -- 172.21.15.33:0/3612510046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0454004c60 con 0x55b7aa23c440 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0466b69740 10 client.20076 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0466b69740 10 client.20076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0466b69740 20 client.20076 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0466b69740 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa25a780 0x55b7aa25cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0466b69740 4 client.20076 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7aa1a4bf0 con 0x55b7aa25a780 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.340+0000 7f0461319640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa25a780 0x55b7aa25cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.341+0000 7f04457fa640 10 client.20076 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.341+0000 7f0461319640 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa25a780 0x55b7aa25cc00 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7525e80b42b4b63c server_cookie=9b56ee8fe3371082 in_seq=0 out_seq=0 2024-04-24T23:34:23.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.343+0000 7f04457fa640 1 -- 172.21.15.33:0/3612510046 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7aa1a4bf0 con 0x55b7aa25a780 2024-04-24T23:34:23.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.343+0000 7f04457fa640 10 client.20076 handle_command_reply: tid=0 2024-04-24T23:34:23.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.343+0000 7f0466b69740 1 client.20076 shutdown 2024-04-24T23:34:23.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.343+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 msgr2=0x55b7aa230600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.343+0000 7f0466b69740 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa230600 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x55b7aa238030 tx=0x7f0454003be0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 shutdown_connections 2024-04-24T23:34:23.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa25a780 0x55b7aa25cc00 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2415b0 0x55b7aa235630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23c440 0x55b7aa230600 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 --2- 172.21.15.33:0/3612510046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa2300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 >> 172.21.15.33:0/3612510046 conn(0x55b7aa2370c0 msgr2=0x55b7aa22daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f04457fa640 0 client.20076 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 shutdown_connections 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 1 -- 172.21.15.33:0/3612510046 wait complete. 2024-04-24T23:34:23.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.344+0000 7f0466b69740 20 client.20076 trim_cache size 0 max 16384 2024-04-24T23:34:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.379+0000 7f0466b69740 1 Processor -- start 2024-04-24T23:34:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.379+0000 7f0466b69740 1 -- start start 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.380+0000 7f0466b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa2370c0 0x55b7aa2300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.380+0000 7f0466b69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa230600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.380+0000 7f0466b69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 0x55b7aa23d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.380+0000 7f0466b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b3270 con 0x55b7aa2370c0 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.380+0000 7f0466b69740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b0b40 con 0x55b7aa23ba50 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.381+0000 7f0461319640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 0x55b7aa23d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.381+0000 7f0461319640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 0x55b7aa23d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46924/0 (socket says 172.21.15.33:46924) 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.381+0000 7f0461319640 1 -- 172.21.15.33:0/3713887561 learned_addr learned my addr 172.21.15.33:0/3713887561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.381+0000 7f0466b69740 1 -- 172.21.15.33:0/3713887561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ae9b0 con 0x55b7aa23c440 2024-04-24T23:34:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.381+0000 7f0461319640 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa2370c0 0x55b7aa2300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.382+0000 7f0461319640 1 -- 172.21.15.33:0/3713887561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 msgr2=0x55b7aa230600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:34:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.382+0000 7f0461319640 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa230600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.382+0000 7f0461319640 1 -- 172.21.15.33:0/3713887561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa2370c0 msgr2=0x55b7aa2300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.382+0000 7f0461319640 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa2370c0 0x55b7aa2300c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.382+0000 7f0461319640 1 -- 172.21.15.33:0/3713887561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7aa220090 con 0x55b7aa23c440 2024-04-24T23:34:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.382+0000 7f0461319640 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 0x55b7aa23d900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0458010650 tx=0x7f0458009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6b494a9d880344 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.383+0000 7f04477fe640 1 -- 172.21.15.33:0/3713887561 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045801a040 con 0x55b7aa23c440 2024-04-24T23:34:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.383+0000 7f04477fe640 1 -- 172.21.15.33:0/3713887561 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0458011ae0 con 0x55b7aa23c440 2024-04-24T23:34:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.383+0000 7f04477fe640 1 -- 172.21.15.33:0/3713887561 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458004670 con 0x55b7aa23c440 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.384+0000 7f0466b69740 1 -- 172.21.15.33:0/3713887561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 msgr2=0x55b7aa23d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.384+0000 7f0466b69740 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 0x55b7aa23d900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0458010650 tx=0x7f0458009b70 comp rx=0 tx=0).stop 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.385+0000 7f0466b69740 1 -- 172.21.15.33:0/3713887561 shutdown_connections 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.385+0000 7f0466b69740 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa23c440 0x55b7aa23d900 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.385+0000 7f0466b69740 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23ba50 0x55b7aa230600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.385+0000 7f0466b69740 1 --2- 172.21.15.33:0/3713887561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa2370c0 0x55b7aa2300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.385+0000 7f0466b69740 1 -- 172.21.15.33:0/3713887561 >> 172.21.15.33:0/3713887561 conn(0x55b7aa229e70 msgr2=0x55b7aa2374a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.386+0000 7f0466b69740 1 -- 172.21.15.33:0/3713887561 shutdown_connections 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.386+0000 7f0466b69740 1 -- 172.21.15.33:0/3713887561 wait complete. 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.387+0000 7f0466b69740 1 -- start start 2024-04-24T23:34:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.387+0000 7f0466b69740 1 Processor -- start 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.388+0000 7f0466b69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 0x55b7aa26b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.388+0000 7f0466b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23ba50 0x55b7aa26b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 0x55b7aa26b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 0x55b7aa26b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46940/0 (socket says 172.21.15.33:46940) 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 -- 172.21.15.33:0/1795927404 learned_addr learned my addr 172.21.15.33:0/1795927404 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23c440 0x55b7aa2be850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1af000 con 0x55b7aa23ba50 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b0470 con 0x55b7aa23c440 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1af6d0 con 0x55b7aa2370c0 2024-04-24T23:34:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23c440 0x55b7aa2be850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 -- 172.21.15.33:0/1795927404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23c440 msgr2=0x55b7aa2be850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23c440 0x55b7aa2be850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 -- 172.21.15.33:0/1795927404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23ba50 msgr2=0x55b7aa26b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23ba50 0x55b7aa26b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.389+0000 7f0461319640 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7aa2bee40 con 0x55b7aa2370c0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.390+0000 7f0466b69740 10 client.0 _ll_register_callbacks cb 0x7f0454009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.390+0000 7f0461319640 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 0x55b7aa26b300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f04580105c0 tx=0x7f0458009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42390bc6d5854567 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.390+0000 7f04457fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.390+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0458011ae0 con 0x55b7aa2370c0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.390+0000 7f04457fa640 1 -- 172.21.15.33:0/1795927404 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458039050 con 0x55b7aa2370c0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.390+0000 7f04457fa640 1 -- 172.21.15.33:0/1795927404 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04580045f0 con 0x55b7aa2370c0 2024-04-24T23:34:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f04457fa640 1 -- 172.21.15.33:0/1795927404 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04580047f0 con 0x55b7aa2370c0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 10 client.29497 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55b7aa2c0050 con 0x55b7aa2370c0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f04457fa640 1 -- 172.21.15.33:0/1795927404 <== mon.1 v2:172.21.15.117:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f045800b050 con 0x55b7aa2370c0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 10 client.29497 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 10 client.29497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 20 client.29497 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa2415b0 0x55b7aa2c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 4 client.29497 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.391+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0458011ae0 con 0x55b7aa2415b0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.392+0000 7f0461b1a640 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa2415b0 0x55b7aa2c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.392+0000 7f0461b1a640 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa2415b0 0x55b7aa2c2850 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7c54b6bb4d427e server_cookie=e5cbeffd48c593fb in_seq=0 out_seq=0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.392+0000 7f04457fa640 10 client.29497 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.392+0000 7f04457fa640 1 -- 172.21.15.33:0/1795927404 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f0458011ae0 con 0x55b7aa2415b0 2024-04-24T23:34:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.392+0000 7f04457fa640 10 client.29497 handle_command_reply: tid=0 2024-04-24T23:34:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.392+0000 7f0466b69740 1 client.29497 shutdown 2024-04-24T23:34:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 msgr2=0x55b7aa26b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 0x55b7aa26b300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f04580105c0 tx=0x7f0458009820 comp rx=0 tx=0).stop 2024-04-24T23:34:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 shutdown_connections 2024-04-24T23:34:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b7aa2415b0 0x55b7aa2c2850 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7aa23c440 0x55b7aa2be850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f04457fa640 0 client.29497 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7aa23ba50 0x55b7aa26b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 --2- 172.21.15.33:0/1795927404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7aa2370c0 0x55b7aa26b300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.394+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 >> 172.21.15.33:0/1795927404 conn(0x55b7aa229e70 msgr2=0x55b7aa2300a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 shutdown_connections 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0466b69740 1 -- 172.21.15.33:0/1795927404 wait complete. 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0466b69740 20 client.29497 trim_cache size 0 max 16384 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:34:23.396 INFO:teuthology.orchestra.run.smithi033.stdout: "f88da3c4-0b8f-49a3-a5da-b58d4005d3d2": { 2024-04-24T23:34:23.397 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:34:23.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "f88da3c4-0b8f-49a3-a5da-b58d4005d3d2", 2024-04-24T23:34:23.397 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:34:23.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:34:23.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:34:23.397 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04480a2d10 msgr2=0x7f04480a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0463d7e640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04480a2d10 0x7f04480a51d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f044c0c3990 tx=0x7f044c005f90 comp rx=0 tx=0).stop 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c10a1b0 msgr2=0x7f045c1a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.395+0000 7f0463d7e640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c10a1b0 0x7f045c1a9a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f044c009780 tx=0x7f044c033d10 comp rx=0 tx=0).stop 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 shutdown_connections 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f04480a2d10 0x7f04480a51d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f045c10a1b0 0x7f045c1a9a70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f045c1097f0 0x7f045c1a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 --2- 172.21.15.33:0/413552428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045c108b90 0x7f045c1a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 >> 172.21.15.33:0/413552428 conn(0x7f045c06e970 msgr2=0x7f045c070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 shutdown_connections 2024-04-24T23:34:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:23.396+0000 7f0463d7e640 1 -- 172.21.15.33:0/413552428 wait complete. 2024-04-24T23:34:23.413 INFO:tasks.cephfs.filesystem:scrub status for tag:f88da3c4-0b8f-49a3-a5da-b58d4005d3d2 - {'path': '/', 'tag': 'f88da3c4-0b8f-49a3-a5da-b58d4005d3d2', 'options': 'recursive,force'} 2024-04-24T23:34:23.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:23 smithi038 ceph-mon[32450]: pgmap v1022: 129 pgs: 129 active+clean; 639 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 373 op/s 2024-04-24T23:34:23.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:23 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:34:23.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:23 smithi117 ceph-mon[32529]: pgmap v1022: 129 pgs: 129 active+clean; 639 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 373 op/s 2024-04-24T23:34:23.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:23 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:34:23.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 214057 22.12 MB/sec execute 388 sec latency 114.678 ms 2024-04-24T23:34:24.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.671 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:24 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:34:24.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:24.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:24.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.676 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:24 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:24.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:24 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:24.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 214194 22.09 MB/sec execute 389 sec latency 424.778 ms 2024-04-24T23:34:25.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:25 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:25.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:25 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:25.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:25 smithi038 ceph-mon[32450]: pgmap v1023: 129 pgs: 129 active+clean; 644 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 428 op/s 2024-04-24T23:34:25.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:25 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:25.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:25 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:25.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:25 smithi117 ceph-mon[32529]: pgmap v1023: 129 pgs: 129 active+clean; 644 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 428 op/s 2024-04-24T23:34:25.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:25 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:25.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:25 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:25.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:25 smithi033 ceph-mon[29201]: pgmap v1023: 129 pgs: 129 active+clean; 644 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 428 op/s 2024-04-24T23:34:25.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 214417 22.07 MB/sec execute 390 sec latency 202.150 ms 2024-04-24T23:34:26.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:26.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:34:26.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:34:26.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:34:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:26.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:34:26.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:34:26.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:34:26.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 214940 22.07 MB/sec execute 391 sec latency 91.844 ms 2024-04-24T23:34:27.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 215491 22.08 MB/sec execute 392 sec latency 63.426 ms 2024-04-24T23:34:28.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:27 smithi038 ceph-mon[32450]: pgmap v1024: 129 pgs: 129 active+clean; 648 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-04-24T23:34:28.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:27 smithi117 ceph-mon[32529]: pgmap v1024: 129 pgs: 129 active+clean; 648 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-04-24T23:34:28.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:27 smithi033 ceph-mon[29201]: pgmap v1024: 129 pgs: 129 active+clean; 648 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-04-24T23:34:28.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 216162 22.13 MB/sec execute 393 sec latency 59.476 ms 2024-04-24T23:34:29.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 216404 22.12 MB/sec execute 394 sec latency 437.331 ms 2024-04-24T23:34:30.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:29 smithi038 ceph-mon[32450]: pgmap v1025: 129 pgs: 129 active+clean; 650 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 435 op/s 2024-04-24T23:34:30.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:29 smithi117 ceph-mon[32529]: pgmap v1025: 129 pgs: 129 active+clean; 650 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 435 op/s 2024-04-24T23:34:30.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:29 smithi033 ceph-mon[29201]: pgmap v1025: 129 pgs: 129 active+clean; 650 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 435 op/s 2024-04-24T23:34:30.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 216629 22.09 MB/sec execute 395 sec latency 480.948 ms 2024-04-24T23:34:31.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 216937 22.07 MB/sec execute 396 sec latency 108.151 ms 2024-04-24T23:34:32.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:31 smithi038 ceph-mon[32450]: pgmap v1026: 129 pgs: 129 active+clean; 657 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.5 MiB/s wr, 498 op/s 2024-04-24T23:34:32.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:31 smithi117 ceph-mon[32529]: pgmap v1026: 129 pgs: 129 active+clean; 657 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.5 MiB/s wr, 498 op/s 2024-04-24T23:34:32.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:31 smithi033 ceph-mon[29201]: pgmap v1026: 129 pgs: 129 active+clean; 657 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.5 MiB/s wr, 498 op/s 2024-04-24T23:34:32.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 217308 22.08 MB/sec execute 397 sec latency 84.272 ms 2024-04-24T23:34:33.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 217732 22.07 MB/sec execute 398 sec latency 110.504 ms 2024-04-24T23:34:34.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:33 smithi038 ceph-mon[32450]: pgmap v1027: 129 pgs: 129 active+clean; 657 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-04-24T23:34:34.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:33 smithi117 ceph-mon[32529]: pgmap v1027: 129 pgs: 129 active+clean; 657 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-04-24T23:34:34.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:33 smithi033 ceph-mon[29201]: pgmap v1027: 129 pgs: 129 active+clean; 657 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-04-24T23:34:34.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 218106 22.05 MB/sec execute 399 sec latency 280.774 ms 2024-04-24T23:34:35.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 218424 22.05 MB/sec execute 400 sec latency 130.786 ms 2024-04-24T23:34:36.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:35 smithi038 ceph-mon[32450]: pgmap v1028: 129 pgs: 129 active+clean; 655 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 464 op/s 2024-04-24T23:34:36.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:35 smithi117 ceph-mon[32529]: pgmap v1028: 129 pgs: 129 active+clean; 655 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 464 op/s 2024-04-24T23:34:36.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:35 smithi033 ceph-mon[29201]: pgmap v1028: 129 pgs: 129 active+clean; 655 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 464 op/s 2024-04-24T23:34:36.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 218801 22.04 MB/sec execute 401 sec latency 90.950 ms 2024-04-24T23:34:37.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 219305 22.05 MB/sec execute 402 sec latency 73.597 ms 2024-04-24T23:34:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:37 smithi038 ceph-mon[32450]: pgmap v1029: 129 pgs: 129 active+clean; 639 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 443 op/s 2024-04-24T23:34:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:37 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:34:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:37 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:34:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:37 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:34:38.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:37 smithi117 ceph-mon[32529]: pgmap v1029: 129 pgs: 129 active+clean; 639 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 443 op/s 2024-04-24T23:34:38.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:37 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:34:38.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:37 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:34:38.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:37 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:34:38.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:37 smithi033 ceph-mon[29201]: pgmap v1029: 129 pgs: 129 active+clean; 639 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 443 op/s 2024-04-24T23:34:38.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:37 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:34:38.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:37 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:34:38.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:37 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:34:38.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 219831 22.07 MB/sec execute 403 sec latency 111.588 ms 2024-04-24T23:34:39.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 220180 22.06 MB/sec execute 404 sec latency 151.064 ms 2024-04-24T23:34:40.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:39 smithi038 ceph-mon[32450]: pgmap v1030: 129 pgs: 129 active+clean; 638 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 438 op/s 2024-04-24T23:34:40.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:39 smithi117 ceph-mon[32529]: pgmap v1030: 129 pgs: 129 active+clean; 638 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 438 op/s 2024-04-24T23:34:40.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:39 smithi033 ceph-mon[29201]: pgmap v1030: 129 pgs: 129 active+clean; 638 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 438 op/s 2024-04-24T23:34:40.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 220436 22.04 MB/sec execute 405 sec latency 351.823 ms 2024-04-24T23:34:41.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 220814 22.04 MB/sec execute 406 sec latency 86.005 ms 2024-04-24T23:34:42.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:41 smithi038 ceph-mon[32450]: pgmap v1031: 129 pgs: 129 active+clean; 640 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 422 op/s 2024-04-24T23:34:42.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:41 smithi117 ceph-mon[32529]: pgmap v1031: 129 pgs: 129 active+clean; 640 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 422 op/s 2024-04-24T23:34:42.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:41 smithi033 ceph-mon[29201]: pgmap v1031: 129 pgs: 129 active+clean; 640 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 422 op/s 2024-04-24T23:34:42.902 INFO:tasks.workunit.client.0.smithi033.stdout: 10 221422 22.07 MB/sec execute 407 sec latency 50.666 ms 2024-04-24T23:34:43.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 221833 22.08 MB/sec execute 408 sec latency 164.572 ms 2024-04-24T23:34:44.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:43 smithi038 ceph-mon[32450]: pgmap v1032: 129 pgs: 129 active+clean; 642 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 335 op/s 2024-04-24T23:34:44.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:43 smithi117 ceph-mon[32529]: pgmap v1032: 129 pgs: 129 active+clean; 642 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 335 op/s 2024-04-24T23:34:44.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:43 smithi033 ceph-mon[29201]: pgmap v1032: 129 pgs: 129 active+clean; 642 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 335 op/s 2024-04-24T23:34:44.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 222159 22.06 MB/sec execute 409 sec latency 109.991 ms 2024-04-24T23:34:45.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 222317 22.02 MB/sec execute 410 sec latency 272.581 ms 2024-04-24T23:34:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:45 smithi038 ceph-mon[32450]: pgmap v1033: 129 pgs: 129 active+clean; 649 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 412 op/s 2024-04-24T23:34:46.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:45 smithi117 ceph-mon[32529]: pgmap v1033: 129 pgs: 129 active+clean; 649 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 412 op/s 2024-04-24T23:34:46.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:45 smithi033 ceph-mon[29201]: pgmap v1033: 129 pgs: 129 active+clean; 649 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 412 op/s 2024-04-24T23:34:46.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 222631 22.01 MB/sec execute 411 sec latency 112.004 ms 2024-04-24T23:34:47.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 223158 22.01 MB/sec execute 412 sec latency 114.428 ms 2024-04-24T23:34:48.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:47 smithi038 ceph-mon[32450]: pgmap v1034: 129 pgs: 129 active+clean; 647 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 380 op/s 2024-04-24T23:34:48.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:47 smithi117 ceph-mon[32529]: pgmap v1034: 129 pgs: 129 active+clean; 647 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 380 op/s 2024-04-24T23:34:48.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:47 smithi033 ceph-mon[29201]: pgmap v1034: 129 pgs: 129 active+clean; 647 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 380 op/s 2024-04-24T23:34:48.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 223763 22.05 MB/sec execute 413 sec latency 97.880 ms 2024-04-24T23:34:49.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 224351 22.08 MB/sec execute 414 sec latency 60.193 ms 2024-04-24T23:34:50.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:49 smithi038 ceph-mon[32450]: pgmap v1035: 129 pgs: 129 active+clean; 649 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 378 op/s 2024-04-24T23:34:50.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:49 smithi117 ceph-mon[32529]: pgmap v1035: 129 pgs: 129 active+clean; 649 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 378 op/s 2024-04-24T23:34:50.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:49 smithi033 ceph-mon[29201]: pgmap v1035: 129 pgs: 129 active+clean; 649 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 378 op/s 2024-04-24T23:34:50.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 224549 22.05 MB/sec execute 415 sec latency 472.479 ms 2024-04-24T23:34:51.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 224837 22.03 MB/sec execute 416 sec latency 142.493 ms 2024-04-24T23:34:52.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:51 smithi038 ceph-mon[32450]: pgmap v1036: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 345 op/s 2024-04-24T23:34:52.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:51 smithi117 ceph-mon[32529]: pgmap v1036: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 345 op/s 2024-04-24T23:34:52.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:51 smithi033 ceph-mon[29201]: pgmap v1036: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 345 op/s 2024-04-24T23:34:52.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 225193 22.03 MB/sec execute 417 sec latency 93.597 ms 2024-04-24T23:34:53.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:34:53.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 -- 172.21.15.33:0/3659782110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8102200 msgr2=0x7f3bc81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/3659782110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8102200 0x7f3bc81045f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009900 tx=0x7f3bb802a3f0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 -- 172.21.15.33:0/3659782110 shutdown_connections 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/3659782110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bc8107460 0x7f3bc81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/3659782110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8104b30 0x7f3bc8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/3659782110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8102200 0x7f3bc81045f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 -- 172.21.15.33:0/3659782110 >> 172.21.15.33:0/3659782110 conn(0x7f3bc80fbbf0 msgr2=0x7f3bc80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 -- 172.21.15.33:0/3659782110 shutdown_connections 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 -- 172.21.15.33:0/3659782110 wait complete. 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.583+0000 7f3bcdb42640 1 Processor -- start 2024-04-24T23:34:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 -- start start 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 0x7f3bc8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8104b30 0x7f3bc81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bc8107460 0x7f3bc819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc810d400 con 0x7f3bc8102200 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc810d280 con 0x7f3bc8104b30 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bcdb42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc810d580 con 0x7f3bc8107460 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 0x7f3bc8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 0x7f3bc8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35800/0 (socket says 172.21.15.33:35800) 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 -- 172.21.15.33:0/4035031357 learned_addr learned my addr 172.21.15.33:0/4035031357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc6ffd640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8104b30 0x7f3bc81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bc8107460 0x7f3bc819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 -- 172.21.15.33:0/4035031357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bc8107460 msgr2=0x7f3bc819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bc8107460 0x7f3bc819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 -- 172.21.15.33:0/4035031357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8104b30 msgr2=0x7f3bc81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8104b30 0x7f3bc81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc77fe640 1 -- 172.21.15.33:0/4035031357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc819f750 con 0x7f3bc8102200 2024-04-24T23:34:53.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.584+0000 7f3bc6ffd640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8104b30 0x7f3bc81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:53.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.585+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 0x7f3bc8197b80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009230 tx=0x7f3bb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.585+0000 7f3bc4ff9640 1 -- 172.21.15.33:0/4035031357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb803f030 con 0x7f3bc8102200 2024-04-24T23:34:53.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.585+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc80ffff0 con 0x7f3bc8102200 2024-04-24T23:34:53.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.585+0000 7f3bc4ff9640 1 -- 172.21.15.33:0/4035031357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb8003ce0 con 0x7f3bc8102200 2024-04-24T23:34:53.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.585+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc8100530 con 0x7f3bc8102200 2024-04-24T23:34:53.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.585+0000 7f3bc4ff9640 1 -- 172.21.15.33:0/4035031357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8049060 con 0x7f3bc8102200 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bc4ff9640 1 -- 172.21.15.33:0/4035031357 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3bb8042030 con 0x7f3bc8102200 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 Processor -- start 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 -- start start 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b37ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b383f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b386a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e8b2da6b0 con 0x555e8b37f440 2024-04-24T23:34:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e8b282880 con 0x555e8b37ea50 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.588+0000 7f3bd084d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555e8b2d6670 con 0x555e8b3845b0 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b37ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b383f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b37ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57860/0 (socket says 172.21.15.33:57860) 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 -- 172.21.15.33:0/751553916 learned_addr learned my addr 172.21.15.33:0/751553916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc6ffd640 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b386a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc4ff9640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bb00a2c10 0x7f3bb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc4ff9640 1 -- 172.21.15.33:0/4035031357 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3bb80ef2a0 con 0x7f3bc8102200 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 -- 172.21.15.33:0/751553916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 msgr2=0x555e8b386a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b386a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 -- 172.21.15.33:0/751553916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 msgr2=0x555e8b383f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b383f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 -- 172.21.15.33:0/751553916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8b0b87d0 con 0x555e8b37ea50 2024-04-24T23:34:53.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bb00a2c10 0x7f3bb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b37ee70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0011570 tx=0x7f3bc0011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=203ff8270e7a54a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.589+0000 7f3badffb640 1 -- 172.21.15.33:0/751553916 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0017070 con 0x555e8b37ea50 2024-04-24T23:34:53.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3badffb640 1 -- 172.21.15.33:0/751553916 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bc0004760 con 0x555e8b37ea50 2024-04-24T23:34:53.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3badffb640 1 -- 172.21.15.33:0/751553916 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0026c00 con 0x555e8b37ea50 2024-04-24T23:34:53.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 -- 172.21.15.33:0/751553916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 msgr2=0x555e8b37ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bb00a2c10 0x7f3bb00a50d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0021000 tx=0x7f3bc0022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b37ee70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0011570 tx=0x7f3bc0011a30 comp rx=0 tx=0).stop 2024-04-24T23:34:53.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 -- 172.21.15.33:0/751553916 shutdown_connections 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b386a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b383f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 --2- 172.21.15.33:0/751553916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b37ee70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 -- 172.21.15.33:0/751553916 >> 172.21.15.33:0/751553916 conn(0x555e8b3718b0 msgr2=0x555e8b371cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 -- 172.21.15.33:0/751553916 shutdown_connections 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.590+0000 7f3bd084d740 1 -- 172.21.15.33:0/751553916 wait complete. 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 -- start start 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 Processor -- start 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b388e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e8b387560 con 0x555e8b37f440 2024-04-24T23:34:53.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e8b295f50 con 0x555e8b37ea50 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555e8b3876e0 con 0x555e8b3845b0 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bd084d740 10 client.0 _ll_register_callbacks cb 0x555e8b2d1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.591+0000 7f3bc6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b388e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3903f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53430/0 (socket says 172.21.15.33:53430) 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3295228605 learned_addr learned my addr 172.21.15.33:0/3295228605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b388e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57870/0 (socket says 172.21.15.33:57870) 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3295228605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 msgr2=0x555e8b388e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b388e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3295228605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 msgr2=0x555e8b3893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3295228605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8afee3f0 con 0x555e8b3845b0 2024-04-24T23:34:53.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3903f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0011f30 tx=0x7f3bc0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234ba7a6eaa95121 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3ba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3295228605 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0017030 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc0004760 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.592+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3295228605 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3bc0004050 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3295228605 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3bc003d050 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 10 client.29509 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555e8afae0f0 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3295228605 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bc0004760 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3295228605 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3bc002e070 con 0x555e8b3845b0 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 10 client.29509 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 10 client.29509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 20 client.29509 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b396e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 4 client.29509 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e8b2e7bf0 con 0x555e8b394a50 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.593+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b396e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.594+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b396e80 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a6fa003433e41a server_cookie=fe8c53fae38d8ecb in_seq=0 out_seq=0 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.594+0000 7f3ba6ffd640 10 client.29509 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.594+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3295228605 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e8b2e7bf0 con 0x555e8b394a50 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.594+0000 7f3ba6ffd640 10 client.29509 handle_command_reply: tid=0 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.594+0000 7f3bd084d740 1 client.29509 shutdown 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 msgr2=0x555e8b3903f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3903f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0011f30 tx=0x7f3bc0009500 comp rx=0 tx=0).stop 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 shutdown_connections 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b396e80 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3903f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3ba6ffd640 0 client.29509 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:53.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3295228605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b388e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 >> 172.21.15.33:0/3295228605 conn(0x555e8b3718b0 msgr2=0x555e8b370af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 shutdown_connections 2024-04-24T23:34:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 1 -- 172.21.15.33:0/3295228605 wait complete. 2024-04-24T23:34:53.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.595+0000 7f3bd084d740 20 client.29509 trim_cache size 0 max 16384 2024-04-24T23:34:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 Processor -- start 2024-04-24T23:34:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 -- start start 2024-04-24T23:34:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b371c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc81aaae0 con 0x555e8b37f440 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc81a83b0 con 0x555e8b37ea50 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bd084d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc81a6200 con 0x555e8b3845b0 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b371c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3808a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53440/0 (socket says 172.21.15.33:53440) 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc7fff640 1 -- 172.21.15.33:0/1117762906 learned_addr learned my addr 172.21.15.33:0/1117762906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b371c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57872/0 (socket says 172.21.15.33:57872) 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc6ffd640 1 -- 172.21.15.33:0/1117762906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 msgr2=0x555e8b3808a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc6ffd640 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3808a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc6ffd640 1 -- 172.21.15.33:0/1117762906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 msgr2=0x555e8b371c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc6ffd640 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b371c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc6ffd640 1 -- 172.21.15.33:0/1117762906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8b372620 con 0x555e8b37f440 2024-04-24T23:34:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3808a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.616+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b371c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.617+0000 7f3bc6ffd640 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3903b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x555e8b3727f0 tx=0x7f3bbc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d573b075ea2c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.617+0000 7f3bad7fa640 1 -- 172.21.15.33:0/1117762906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc01e030 con 0x555e8b37f440 2024-04-24T23:34:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.617+0000 7f3bad7fa640 1 -- 172.21.15.33:0/1117762906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc005500 con 0x555e8b37f440 2024-04-24T23:34:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.617+0000 7f3bad7fa640 1 -- 172.21.15.33:0/1117762906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004e20 con 0x555e8b37f440 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 -- 172.21.15.33:0/1117762906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 msgr2=0x555e8b3903b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3903b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x555e8b3727f0 tx=0x7f3bbc005f90 comp rx=0 tx=0).stop 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 -- 172.21.15.33:0/1117762906 shutdown_connections 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b3845b0 0x555e8b3808a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b37f440 0x555e8b3903b0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 --2- 172.21.15.33:0/1117762906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b371c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 -- 172.21.15.33:0/1117762906 >> 172.21.15.33:0/1117762906 conn(0x555e8b3718b0 msgr2=0x555e8b388e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 -- 172.21.15.33:0/1117762906 shutdown_connections 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 -- 172.21.15.33:0/1117762906 wait complete. 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 -- start start 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.618+0000 7f3bd084d740 1 Processor -- start 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b38d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b37f440 0x555e8b38dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 0x555e8b38eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc81a6870 con 0x555e8b3845b0 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc81a7ce0 con 0x555e8b37ea50 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc81a6f40 con 0x555e8b37f440 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bd084d740 10 client.0 _ll_register_callbacks cb 0x7f3bb80b7a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b38d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 0x555e8b38eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b37f440 0x555e8b38dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 0x555e8b38eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35840/0 (socket says 172.21.15.33:35840) 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b37f440 0x555e8b38dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53450/0 (socket says 172.21.15.33:53450) 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b38d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57888/0 (socket says 172.21.15.33:57888) 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3133115667 learned_addr learned my addr 172.21.15.33:0/3133115667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b37f440 msgr2=0x555e8b38dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b37f440 0x555e8b38dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3133115667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 msgr2=0x555e8b38d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b38d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 -- 172.21.15.33:0/3133115667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8b38f290 con 0x555e8b3845b0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3bc7fff640 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 0x555e8b38eca0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0026c90 tx=0x7f3bc0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218d04a2dc23242f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.619+0000 7f3ba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3133115667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0017040 con 0x555e8b3845b0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3133115667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3bc00040b0 con 0x555e8b3845b0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bbc005500 con 0x555e8b3845b0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3133115667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3bc002b040 con 0x555e8b3845b0 2024-04-24T23:34:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 10 client.20097 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555e8b3905a0 con 0x555e8b3845b0 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3133115667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc005500 con 0x555e8b3845b0 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3133115667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3bc0049030 con 0x555e8b3845b0 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 10 client.20097 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 10 client.20097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 20 client.20097 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b3fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 4 client.20097 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.620+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e8b2e1f40 con 0x555e8b394a50 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.621+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b3fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.621+0000 7f3ba6ffd640 10 client.20097 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.621+0000 7f3bc77fe640 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b3fd620 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74284ebcb3f33907 server_cookie=4a158dc7f1c9a667 in_seq=0 out_seq=0 2024-04-24T23:34:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3ba6ffd640 1 -- 172.21.15.33:0/3133115667 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e8b2e1f40 con 0x555e8b394a50 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3ba6ffd640 10 client.20097 handle_command_reply: tid=0 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 client.20097 shutdown 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 msgr2=0x555e8b38eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 0x555e8b38eca0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0026c90 tx=0x7f3bc0005f90 comp rx=0 tx=0).stop 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 shutdown_connections 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555e8b394a50 0x555e8b3fd620 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e8b3845b0 0x555e8b38eca0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e8b37f440 0x555e8b38dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3ba6ffd640 0 client.20097 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 --2- 172.21.15.33:0/3133115667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e8b37ea50 0x555e8b38d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 >> 172.21.15.33:0/3133115667 conn(0x555e8b3718b0 msgr2=0x555e8b3808a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 shutdown_connections 2024-04-24T23:34:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 1 -- 172.21.15.33:0/3133115667 wait complete. 2024-04-24T23:34:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.624+0000 7f3bd084d740 20 client.20097 trim_cache size 0 max 16384 2024-04-24T23:34:53.628 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:34:53.628 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:34:53.628 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:34:53.628 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bb00a2c10 msgr2=0x7f3bb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bb00a2c10 0x7f3bb00a50d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0021000 tx=0x7f3bc0022000 comp rx=0 tx=0).stop 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 msgr2=0x7f3bc8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 0x7f3bc8197b80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009230 tx=0x7f3bb80042f0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 shutdown_connections 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3bb00a2c10 0x7f3bb00a50d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bc8107460 0x7f3bc819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc8104b30 0x7f3bc81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 --2- 172.21.15.33:0/4035031357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bc8102200 0x7f3bc8197b80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 >> 172.21.15.33:0/4035031357 conn(0x7f3bc80fbbf0 msgr2=0x7f3bc8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 shutdown_connections 2024-04-24T23:34:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.625+0000 7f3bcdb42640 1 -- 172.21.15.33:0/4035031357 wait complete. 2024-04-24T23:34:53.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:34:53.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:34:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.749+0000 7ff1cfe12640 1 -- 172.21.15.33:0/2236242060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c81017b0 msgr2=0x7ff1c8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.749+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/2236242060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c81017b0 0x7ff1c8101bd0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009900 tx=0x7ff1c402a410 comp rx=0 tx=0).stop 2024-04-24T23:34:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- 172.21.15.33:0/2236242060 shutdown_connections 2024-04-24T23:34:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/2236242060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1c8103360 0x7ff1c8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/2236242060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c81029a0 0x7ff1c8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/2236242060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c81017b0 0x7ff1c8101bd0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- 172.21.15.33:0/2236242060 >> 172.21.15.33:0/2236242060 conn(0x7ff1c80fcd40 msgr2=0x7ff1c80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- 172.21.15.33:0/2236242060 shutdown_connections 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- 172.21.15.33:0/2236242060 wait complete. 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 Processor -- start 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- start start 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c81017b0 0x7ff1c8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1c81029a0 0x7ff1c819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 0x7ff1c81a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c810d470 con 0x7ff1c8103360 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c810d2f0 con 0x7ff1c81029a0 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.750+0000 7ff1cfe12640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c810d5f0 con 0x7ff1c81017b0 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 0x7ff1c81a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 0x7ff1c81a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35852/0 (socket says 172.21.15.33:35852) 2024-04-24T23:34:53.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 -- 172.21.15.33:0/1972044962 learned_addr learned my addr 172.21.15.33:0/1972044962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1cd3ad640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1c81029a0 0x7ff1c819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1cdbae640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c81017b0 0x7ff1c8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 -- 172.21.15.33:0/1972044962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c81017b0 msgr2=0x7ff1c8199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c81017b0 0x7ff1c8199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 -- 172.21.15.33:0/1972044962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1c81029a0 msgr2=0x7ff1c819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1c81029a0 0x7ff1c819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 -- 172.21.15.33:0/1972044962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c81a1960 con 0x7ff1c8103360 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1ce3af640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 0x7ff1c81a12c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000eb10 tx=0x7ff1c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0004b20 con 0x7ff1c8103360 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0004cc0 con 0x7ff1c8103360 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c81a1c50 con 0x7ff1c8103360 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0010760 con 0x7ff1c8103360 2024-04-24T23:34:53.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.751+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c81a2120 con 0x7ff1c8103360 2024-04-24T23:34:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.752+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c81a22f0 con 0x7ff1c8103360 2024-04-24T23:34:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.753+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff1c0005020 con 0x7ff1c8103360 2024-04-24T23:34:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.754+0000 7ff1beffd640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1b40a2b90 0x7ff1b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.754+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff1c00cfa10 con 0x7ff1c8103360 2024-04-24T23:34:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.754+0000 7ff1cdbae640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1b40a2b90 0x7ff1b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.755+0000 7ff1cdbae640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1b40a2b90 0x7ff1b40a5050 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff1c402aa00 tx=0x7ff1c40315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:53.759+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff1c009c5e0 con 0x7ff1c8103360 2024-04-24T23:34:53.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 225641 22.04 MB/sec execute 418 sec latency 139.895 ms 2024-04-24T23:34:54.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.019+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1c81a24c0 con 0x7ff1c8103360 2024-04-24T23:34:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.020+0000 7ff1beffd640 1 -- 172.21.15.33:0/1972044962 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7ff1c009bee0 con 0x7ff1c8103360 2024-04-24T23:34:54.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:34:54.020 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:34:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:34:54.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1b40a2b90 msgr2=0x7ff1b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1b40a2b90 0x7ff1b40a5050 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff1c402aa00 tx=0x7ff1c40315b0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 msgr2=0x7ff1c81a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 0x7ff1c81a12c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000eb10 tx=0x7ff1c000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 shutdown_connections 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff1b40a2b90 0x7ff1b40a5050 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c8103360 0x7ff1c81a12c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1c81029a0 0x7ff1c819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 --2- 172.21.15.33:0/1972044962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c81017b0 0x7ff1c8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 >> 172.21.15.33:0/1972044962 conn(0x7ff1c80fcd40 msgr2=0x7ff1c8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 shutdown_connections 2024-04-24T23:34:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.022+0000 7ff1cfe12640 1 -- 172.21.15.33:0/1972044962 wait complete. 2024-04-24T23:34:54.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:34:54.055 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:53 smithi033 ceph-mon[29201]: pgmap v1037: 129 pgs: 129 active+clean; 657 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 369 op/s 2024-04-24T23:34:54.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 -- 172.21.15.33:0/1891528049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b701006a0 msgr2=0x7f7b70100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 --2- 172.21.15.33:0/1891528049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b701006a0 0x7f7b70100ac0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009900 tx=0x7f7b6002a410 comp rx=0 tx=0).stop 2024-04-24T23:34:54.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 -- 172.21.15.33:0/1891528049 shutdown_connections 2024-04-24T23:34:54.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 --2- 172.21.15.33:0/1891528049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b70102250 0x7f7b70108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 --2- 172.21.15.33:0/1891528049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b70101890 0x7f7b70101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 --2- 172.21.15.33:0/1891528049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b701006a0 0x7f7b70100ac0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 -- 172.21.15.33:0/1891528049 >> 172.21.15.33:0/1891528049 conn(0x7f7b700fbc30 msgr2=0x7f7b700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 -- 172.21.15.33:0/1891528049 shutdown_connections 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 -- 172.21.15.33:0/1891528049 wait complete. 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.147+0000 7f7b75f32640 1 Processor -- start 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 -- start start 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b701006a0 0x7f7b70197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b70101890 0x7f7b70198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 0x7f7b7019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010d440 con 0x7f7b70102250 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010d2c0 con 0x7f7b701006a0 2024-04-24T23:34:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b75f32640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010d5c0 con 0x7f7b70101890 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 0x7f7b7019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 0x7f7b7019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35866/0 (socket says 172.21.15.33:35866) 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 -- 172.21.15.33:0/347573874 learned_addr learned my addr 172.21.15.33:0/347573874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b701006a0 0x7f7b70197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6effd640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b70101890 0x7f7b70198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 -- 172.21.15.33:0/347573874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b70101890 msgr2=0x7f7b70198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b70101890 0x7f7b70198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 -- 172.21.15.33:0/347573874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b701006a0 msgr2=0x7f7b70197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b701006a0 0x7f7b70197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 -- 172.21.15.33:0/347573874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7019f740 con 0x7f7b70102250 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6ffff640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 0x7f7b7019f0d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800eb10 tx=0x7f7b6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.148+0000 7f7b6cff9640 1 -- 172.21.15.33:0/347573874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004b20 con 0x7f7b70102250 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.149+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7019fa30 con 0x7f7b70102250 2024-04-24T23:34:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.149+0000 7f7b6cff9640 1 -- 172.21.15.33:0/347573874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68004cc0 con 0x7f7b70102250 2024-04-24T23:34:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.149+0000 7f7b6cff9640 1 -- 172.21.15.33:0/347573874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68010760 con 0x7f7b70102250 2024-04-24T23:34:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.149+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7019ff30 con 0x7f7b70102250 2024-04-24T23:34:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.151+0000 7f7b6cff9640 1 -- 172.21.15.33:0/347573874 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7b68005020 con 0x7f7b70102250 2024-04-24T23:34:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 Processor -- start 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 -- start start 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf05740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55951ce65670 con 0x55951cf05db0 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55951ce0d970 con 0x55951cf00870 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.152+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55951ce61630 con 0x55951cf01260 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6cff9640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b580a2b90 0x7f7b580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf00c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:57944/0 (socket says 172.21.15.33:57944) 2024-04-24T23:34:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf05740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6effd640 1 -- 172.21.15.33:0/1602941326 learned_addr learned my addr 172.21.15.33:0/1602941326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6cff9640 1 -- 172.21.15.33:0/347573874 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b680cfa30 con 0x7f7b70102250 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6ffff640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b580a2b90 0x7f7b580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/1602941326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 msgr2=0x55951cf00c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf00c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/1602941326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 msgr2=0x55951cf08240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf08240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/1602941326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951ce31400 con 0x55951cf01260 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6effd640 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf08240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.153+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf05740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b6002a930 tx=0x7f7b60005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9bd1ad5f819b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.154+0000 7f7b6ffff640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b580a2b90 0x7f7b580a5050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7b68005b40 tx=0x7f7b680058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.154+0000 7f7b55ffb640 1 -- 172.21.15.33:0/1602941326 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60035040 con 0x55951cf01260 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.154+0000 7f7b55ffb640 1 -- 172.21.15.33:0/1602941326 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60004400 con 0x55951cf01260 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.154+0000 7f7b55ffb640 1 -- 172.21.15.33:0/1602941326 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60004e30 con 0x55951cf01260 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 -- 172.21.15.33:0/1602941326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 msgr2=0x55951cf05740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf05740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b6002a930 tx=0x7f7b60005dd0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 -- 172.21.15.33:0/1602941326 shutdown_connections 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf08240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf05740 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/1602941326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf00c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 -- 172.21.15.33:0/1602941326 >> 172.21.15.33:0/1602941326 conn(0x55951cefc820 msgr2=0x55951cefcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 -- 172.21.15.33:0/1602941326 shutdown_connections 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 -- 172.21.15.33:0/1602941326 wait complete. 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 -- start start 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.155+0000 7f7b78c3d740 1 Processor -- start 2024-04-24T23:34:54.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf1c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf1c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf23900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55951cf08ce0 con 0x55951cf05db0 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55951ce21050 con 0x55951cf00870 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55951cf1aeb0 con 0x55951cf01260 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b78c3d740 10 client.0 _ll_register_callbacks cb 0x55951cb78be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf23900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf1c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf23900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35874/0 (socket says 172.21.15.33:35874) 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf1c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53498/0 (socket says 172.21.15.33:53498) 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6effd640 1 -- 172.21.15.33:0/3250193495 learned_addr learned my addr 172.21.15.33:0/3250193495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6effd640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf1c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 msgr2=0x55951cf1c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf1c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/3250193495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 msgr2=0x55951cf23900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf23900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/3250193495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951cdf2dc0 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.156+0000 7f7b6effd640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf23900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf1c8c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009900 tx=0x7f7b60031470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87f861a167edbf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b4effd640 1 -- 172.21.15.33:0/3250193495 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60035040 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b4effd640 1 -- 172.21.15.33:0/3250193495 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b60031bf0 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b60004400 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b4effd640 1 -- 172.21.15.33:0/3250193495 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b60004770 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 10 client.29533 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55951cb793f0 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b4effd640 1 -- 172.21.15.33:0/3250193495 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60004400 con 0x55951cf01260 2024-04-24T23:34:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b4effd640 1 -- 172.21.15.33:0/3250193495 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7b60055030 con 0x55951cf01260 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 10 client.29533 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 10 client.29533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 20 client.29533 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf27df0 0x55951cf2a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 4 client.29533 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.157+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55951ce72bb0 con 0x55951cf27df0 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.158+0000 7f7b6effd640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf27df0 0x55951cf2a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.158+0000 7f7b6effd640 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf27df0 0x55951cf2a1e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8aad6d86d532ea0 server_cookie=8b067fbc8340bc23 in_seq=0 out_seq=0 2024-04-24T23:34:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.158+0000 7f7b4effd640 10 client.29533 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.160+0000 7f7b4effd640 1 -- 172.21.15.33:0/3250193495 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55951ce72bb0 con 0x55951cf27df0 2024-04-24T23:34:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.160+0000 7f7b4effd640 10 client.29533 handle_command_reply: tid=0 2024-04-24T23:34:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.160+0000 7f7b78c3d740 1 client.29533 shutdown 2024-04-24T23:34:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 msgr2=0x55951cf1c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf1c8c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009900 tx=0x7f7b60031470 comp rx=0 tx=0).stop 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 shutdown_connections 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf27df0 0x55951cf2a1e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf05db0 0x55951cf23900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf1c8c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b4effd640 0 client.29533 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/3250193495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cf1c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 >> 172.21.15.33:0/3250193495 conn(0x55951cefc820 msgr2=0x55951cefbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 shutdown_connections 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 1 -- 172.21.15.33:0/3250193495 wait complete. 2024-04-24T23:34:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.161+0000 7f7b78c3d740 20 client.29533 trim_cache size 0 max 16384 2024-04-24T23:34:54.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:53 smithi038 ceph-mon[32450]: pgmap v1037: 129 pgs: 129 active+clean; 657 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 369 op/s 2024-04-24T23:34:54.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:53 smithi117 ceph-mon[32529]: pgmap v1037: 129 pgs: 129 active+clean; 657 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 369 op/s 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 Processor -- start 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 -- start start 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cefb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 0x55951cf238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf05db0 0x55951cefb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701aaad0 con 0x55951cf01260 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701a83a0 con 0x55951cf00870 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701a61f0 con 0x55951cf05db0 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 0x55951cf238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 0x55951cf238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35886/0 (socket says 172.21.15.33:35886) 2024-04-24T23:34:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/874458794 learned_addr learned my addr 172.21.15.33:0/874458794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b6effd640 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cefb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.177+0000 7f7b6effd640 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf05db0 0x55951cefb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/874458794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf05db0 msgr2=0x55951cefb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf05db0 0x55951cefb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/874458794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 msgr2=0x55951cefb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cefb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/874458794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951cc437d0 con 0x55951cf01260 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6effd640 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf05db0 0x55951cefb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 0x55951cf238c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7b6003c6f0 tx=0x7f7b60009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f01a15c2c32c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b557fa640 1 -- 172.21.15.33:0/874458794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60005d50 con 0x55951cf01260 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b557fa640 1 -- 172.21.15.33:0/874458794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6003cdc0 con 0x55951cf01260 2024-04-24T23:34:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.178+0000 7f7b557fa640 1 -- 172.21.15.33:0/874458794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60003850 con 0x55951cf01260 2024-04-24T23:34:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 -- 172.21.15.33:0/874458794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 msgr2=0x55951cf238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 0x55951cf238c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7b6003c6f0 tx=0x7f7b60009660 comp rx=0 tx=0).stop 2024-04-24T23:34:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 -- 172.21.15.33:0/874458794 shutdown_connections 2024-04-24T23:34:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf05db0 0x55951cefb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf01260 0x55951cf238c0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/874458794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf00870 0x55951cefb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 -- 172.21.15.33:0/874458794 >> 172.21.15.33:0/874458794 conn(0x55951cefc820 msgr2=0x55951cf1c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 -- 172.21.15.33:0/874458794 shutdown_connections 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 -- 172.21.15.33:0/874458794 wait complete. 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.179+0000 7f7b78c3d740 1 -- start start 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 Processor -- start 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 0x55951cf20ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf21020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf05db0 0x55951cf22030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701a6860 con 0x55951cf00870 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701a7cd0 con 0x55951cf05db0 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701a6f30 con 0x55951cf01260 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b78c3d740 10 client.0 _ll_register_callbacks cb 0x7f7b64009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 0x55951cf20ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b6f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf21020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 0x55951cf20ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35900/0 (socket says 172.21.15.33:35900) 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b6effd640 1 -- 172.21.15.33:0/2676453960 learned_addr learned my addr 172.21.15.33:0/2676453960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.180+0000 7f7b6effd640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf05db0 0x55951cf22030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf05db0 msgr2=0x55951cf22030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf05db0 0x55951cf22030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/2676453960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 msgr2=0x55951cf20ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 0x55951cf20ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6f7fe640 1 -- 172.21.15.33:0/2676453960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951cf22620 con 0x55951cf01260 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6effd640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 0x55951cf20ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b6f7fe640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf21020 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55951cef12c0 tx=0x7f7b60005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7823639ad634df48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b4effd640 1 -- 172.21.15.33:0/2676453960 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6006d060 con 0x55951cf01260 2024-04-24T23:34:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6003cdc0 con 0x55951cf01260 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b4effd640 1 -- 172.21.15.33:0/2676453960 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b600037f0 con 0x55951cf01260 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b4effd640 1 -- 172.21.15.33:0/2676453960 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b60074050 con 0x55951cf01260 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b78c3d740 10 client.29539 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.181+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55951cf23920 con 0x55951cf01260 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b4effd640 1 -- 172.21.15.33:0/2676453960 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6003cdc0 con 0x55951cf01260 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b4effd640 1 -- 172.21.15.33:0/2676453960 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7b60055040 con 0x55951cf01260 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b78c3d740 10 client.29539 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b78c3d740 10 client.29539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b78c3d740 20 client.29539 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf8e430 0x55951cf90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b78c3d740 4 client.29539 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55951ce6cf00 con 0x55951cf8e430 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b6effd640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf8e430 0x55951cf90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b4effd640 10 client.29539 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.182+0000 7f7b6effd640 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf8e430 0x55951cf90820 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c58498aaf276fd86 server_cookie=fc70932413da0b21 in_seq=0 out_seq=0 2024-04-24T23:34:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b4effd640 1 -- 172.21.15.33:0/2676453960 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55951ce6cf00 con 0x55951cf8e430 2024-04-24T23:34:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b4effd640 10 client.29539 handle_command_reply: tid=0 2024-04-24T23:34:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 client.29539 shutdown 2024-04-24T23:34:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 msgr2=0x55951cf21020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf21020 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55951cef12c0 tx=0x7f7b60005c90 comp rx=0 tx=0).stop 2024-04-24T23:34:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 shutdown_connections 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55951cf8e430 0x55951cf90820 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55951cf05db0 0x55951cf22030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b4effd640 0 client.29539 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55951cf01260 0x55951cf21020 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 --2- 172.21.15.33:0/2676453960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951cf00870 0x55951cf20ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 >> 172.21.15.33:0/2676453960 conn(0x55951cefc820 msgr2=0x55951cf08e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 shutdown_connections 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.187+0000 7f7b78c3d740 1 -- 172.21.15.33:0/2676453960 wait complete. 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b78c3d740 20 client.29539 trim_cache size 0 max 16384 2024-04-24T23:34:54.190 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:34:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b580a2b90 msgr2=0x7f7b580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b580a2b90 0x7f7b580a5050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7b68005b40 tx=0x7f7b680058d0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 msgr2=0x7f7b7019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 0x7f7b7019f0d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800eb10 tx=0x7f7b6800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 shutdown_connections 2024-04-24T23:34:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7b580a2b90 0x7f7b580a5050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b70102250 0x7f7b7019f0d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b70101890 0x7f7b70198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 --2- 172.21.15.33:0/347573874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b701006a0 0x7f7b70197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 >> 172.21.15.33:0/347573874 conn(0x7f7b700fbc30 msgr2=0x7f7b70104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 shutdown_connections 2024-04-24T23:34:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.188+0000 7f7b75f32640 1 -- 172.21.15.33:0/347573874 wait complete. 2024-04-24T23:34:54.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:34:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.362+0000 7f191e343640 1 -- 172.21.15.33:0/1349222487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918072b80 msgr2=0x7f1918072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.362+0000 7f191e343640 1 --2- 172.21.15.33:0/1349222487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918072b80 0x7f1918072fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1914009900 tx=0x7f191402a3f0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.366+0000 7f191e343640 1 -- 172.21.15.33:0/1349222487 shutdown_connections 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.366+0000 7f191e343640 1 --2- 172.21.15.33:0/1349222487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19181097c0 0x7f191810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.366+0000 7f191e343640 1 --2- 172.21.15.33:0/1349222487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1918108e20 0x7f1918109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.366+0000 7f191e343640 1 --2- 172.21.15.33:0/1349222487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918072b80 0x7f1918072fa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.366+0000 7f191e343640 1 -- 172.21.15.33:0/1349222487 >> 172.21.15.33:0/1349222487 conn(0x7f191806e650 msgr2=0x7f1918070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.370+0000 7f191e343640 1 -- 172.21.15.33:0/1349222487 shutdown_connections 2024-04-24T23:34:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.370+0000 7f191e343640 1 -- 172.21.15.33:0/1349222487 wait complete. 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 Processor -- start 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 -- start start 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1918072b80 0x7f1918110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918108e20 0x7f1918110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19181097c0 0x7f1918117800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f191811c0e0 con 0x7f19181097c0 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f191811bf60 con 0x7f1918072b80 2024-04-24T23:34:54.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.371+0000 7f191e343640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f191811c260 con 0x7f1918108e20 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191cb40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918108e20 0x7f1918110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191cb40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918108e20 0x7f1918110790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53526/0 (socket says 172.21.15.33:53526) 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191cb40640 1 -- 172.21.15.33:0/3202345637 learned_addr learned my addr 172.21.15.33:0/3202345637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19181097c0 0x7f1918117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 -- 172.21.15.33:0/3202345637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918108e20 msgr2=0x7f1918110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918108e20 0x7f1918110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 -- 172.21.15.33:0/3202345637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1918072b80 msgr2=0x7f1918110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1918072b80 0x7f1918110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 -- 172.21.15.33:0/3202345637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1918117e70 con 0x7f19181097c0 2024-04-24T23:34:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.372+0000 7f191db42640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19181097c0 0x7f1918117800 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1910009fd0 tx=0x7f191000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.373+0000 7f190e7fc640 1 -- 172.21.15.33:0/3202345637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191000ce20 con 0x7f19181097c0 2024-04-24T23:34:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.373+0000 7f190e7fc640 1 -- 172.21.15.33:0/3202345637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1910005020 con 0x7f19181097c0 2024-04-24T23:34:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.373+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1918118160 con 0x7f19181097c0 2024-04-24T23:34:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.373+0000 7f190e7fc640 1 -- 172.21.15.33:0/3202345637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910010720 con 0x7f19181097c0 2024-04-24T23:34:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.373+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19181a6550 con 0x7f19181097c0 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.378+0000 7f190e7fc640 1 -- 172.21.15.33:0/3202345637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1910005540 con 0x7f19181097c0 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.378+0000 7f192238f740 1 Processor -- start 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 -- start start 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bddb0 0x5559cb2c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2c2f20 0x5559cb2c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5559cb221670 con 0x5559cb2c2f20 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5559cb1c9970 con 0x5559cb2bddb0 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.381+0000 7f192238f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559cb21d630 con 0x5559cb2bd3c0 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191cb40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191cb40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53532/0 (socket says 172.21.15.33:53532) 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191cb40640 1 -- 172.21.15.33:0/2608690891 learned_addr learned my addr 172.21.15.33:0/2608690891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191cb40640 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2c2f20 0x5559cb2c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191d341640 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bddb0 0x5559cb2c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191d341640 1 -- 172.21.15.33:0/2608690891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bd3c0 msgr2=0x5559cb2bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191d341640 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191d341640 1 -- 172.21.15.33:0/2608690891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2c2f20 msgr2=0x5559cb2c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191d341640 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2c2f20 0x5559cb2c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.382+0000 7f191d341640 1 -- 172.21.15.33:0/2608690891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cb1ed400 con 0x5559cb2bddb0 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.383+0000 7f191d341640 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bddb0 0x5559cb2c28b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19140314e0 tx=0x7f1914000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=790d744860667adf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.383+0000 7f19037fe640 1 -- 172.21.15.33:0/2608690891 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191403e070 con 0x5559cb2bddb0 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.383+0000 7f19037fe640 1 -- 172.21.15.33:0/2608690891 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1914005530 con 0x5559cb2bddb0 2024-04-24T23:34:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.383+0000 7f19037fe640 1 -- 172.21.15.33:0/2608690891 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914035040 con 0x5559cb2bddb0 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f190e7fc640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f19040a2c90 0x7f19040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 -- 172.21.15.33:0/2608690891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bddb0 msgr2=0x5559cb2c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bddb0 0x5559cb2c28b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19140314e0 tx=0x7f1914000bc0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 -- 172.21.15.33:0/2608690891 shutdown_connections 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2c2f20 0x5559cb2c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bddb0 0x5559cb2c28b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 --2- 172.21.15.33:0/2608690891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f192238f740 1 -- 172.21.15.33:0/2608690891 >> 172.21.15.33:0/2608690891 conn(0x5559cb2b8a30 msgr2=0x5559cb2b8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f191cb40640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f19040a2c90 0x7f19040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.384+0000 7f190e7fc640 1 -- 172.21.15.33:0/3202345637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f19100cfb60 con 0x7f19181097c0 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.385+0000 7f191cb40640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f19040a2c90 0x7f19040a5150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x5559cb2be610 tx=0x7f190802d490 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.386+0000 7f192238f740 1 -- 172.21.15.33:0/2608690891 shutdown_connections 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.386+0000 7f192238f740 1 -- 172.21.15.33:0/2608690891 wait complete. 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 -- start start 2024-04-24T23:34:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 Processor -- start 2024-04-24T23:34:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5559cb2c5e70 con 0x5559cb2bddb0 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5559cb1dd050 con 0x5559cb2bd3c0 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559cb2c5ff0 con 0x5559cb2c2f20 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.387+0000 7f192238f740 10 client.0 _ll_register_callbacks cb 0x5559caf34be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191db42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35942/0 (socket says 172.21.15.33:35942) 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 -- 172.21.15.33:0/2387181814 learned_addr learned my addr 172.21.15.33:0/2387181814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191d341640 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 -- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 msgr2=0x5559cb2cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 -- 172.21.15.33:0/2387181814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 msgr2=0x5559cb2c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.388+0000 7f191cb40640 1 -- 172.21.15.33:0/2387181814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cb1aedc0 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f191cb40640 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2c79e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1908033b50 tx=0x7f1908004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c702e6a2052f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f1900ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f1900ff9640 1 -- 172.21.15.33:0/2387181814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908033e40 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1914005530 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f1900ff9640 1 -- 172.21.15.33:0/2387181814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1908032960 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f1900ff9640 1 -- 172.21.15.33:0/2387181814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1908054050 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f192238f740 10 client.20133 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.389+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5559caf353f0 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f1900ff9640 1 -- 172.21.15.33:0/2387181814 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1914005530 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f1900ff9640 1 -- 172.21.15.33:0/2387181814 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1908030040 con 0x5559cb2bddb0 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f192238f740 10 client.20133 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f192238f740 10 client.20133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:34:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f192238f740 20 client.20133 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f192238f740 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb2d3390 0x5559cb2d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f192238f740 4 client.20133 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559cb22ebb0 con 0x5559cb2d3390 2024-04-24T23:34:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.390+0000 7f191d341640 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb2d3390 0x5559cb2d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.391+0000 7f191d341640 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb2d3390 0x5559cb2d5850 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfb4ab0b1dc3c9c8 server_cookie=238c474c8f096960 in_seq=0 out_seq=0 2024-04-24T23:34:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.391+0000 7f1900ff9640 10 client.20133 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.392+0000 7f1900ff9640 1 -- 172.21.15.33:0/2387181814 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559cb22ebb0 con 0x5559cb2d3390 2024-04-24T23:34:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.392+0000 7f1900ff9640 10 client.20133 handle_command_reply: tid=0 2024-04-24T23:34:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.393+0000 7f192238f740 1 client.20133 shutdown 2024-04-24T23:34:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 msgr2=0x5559cb2c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2c79e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1908033b50 tx=0x7f1908004730 comp rx=0 tx=0).stop 2024-04-24T23:34:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 shutdown_connections 2024-04-24T23:34:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb2d3390 0x5559cb2d5850 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2c79e0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 --2- 172.21.15.33:0/2387181814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 >> 172.21.15.33:0/2387181814 conn(0x5559cb2b8a30 msgr2=0x5559cb2b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.394+0000 7f1900ff9640 0 client.20133 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:54.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.395+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 shutdown_connections 2024-04-24T23:34:54.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.395+0000 7f192238f740 1 -- 172.21.15.33:0/2387181814 wait complete. 2024-04-24T23:34:54.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.395+0000 7f192238f740 20 client.20133 trim_cache size 0 max 16384 2024-04-24T23:34:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.420+0000 7f192238f740 1 Processor -- start 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.420+0000 7f192238f740 1 -- start start 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f192238f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f192238f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bddb0 0x5559cb2bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f192238f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2c2f20 0x5559cb2ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f191d341640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f191d341640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35946/0 (socket says 172.21.15.33:35946) 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f191cb40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bddb0 0x5559cb2bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f191cb40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bddb0 0x5559cb2bc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53558/0 (socket says 172.21.15.33:53558) 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f191d341640 1 -- 172.21.15.33:0/797820358 learned_addr learned my addr 172.21.15.33:0/797820358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19181b2f90 con 0x5559cb2bd3c0 2024-04-24T23:34:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f191db42640 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2c2f20 0x5559cb2ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19181b0860 con 0x5559cb2c2f20 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.421+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19181ae6b0 con 0x5559cb2bddb0 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f191d341640 1 -- 172.21.15.33:0/797820358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bddb0 msgr2=0x5559cb2bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f191d341640 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bddb0 0x5559cb2bc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f191d341640 1 -- 172.21.15.33:0/797820358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2c2f20 msgr2=0x5559cb2ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f191d341640 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2c2f20 0x5559cb2ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f191d341640 1 -- 172.21.15.33:0/797820358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cafff7d0 con 0x5559cb2bd3c0 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f191d341640 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bc050 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1914009900 tx=0x7f191402ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6030322277978ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f1902ffd640 1 -- 172.21.15.33:0/797820358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191403e040 con 0x5559cb2bd3c0 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f1902ffd640 1 -- 172.21.15.33:0/797820358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1914031b90 con 0x5559cb2bd3c0 2024-04-24T23:34:54.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.422+0000 7f1902ffd640 1 -- 172.21.15.33:0/797820358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914035040 con 0x5559cb2bd3c0 2024-04-24T23:34:54.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 msgr2=0x5559cb2bc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bc050 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1914009900 tx=0x7f191402ab20 comp rx=0 tx=0).stop 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 shutdown_connections 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2c2f20 0x5559cb2ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2bddb0 0x5559cb2bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 --2- 172.21.15.33:0/797820358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bd3c0 0x5559cb2bc050 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 >> 172.21.15.33:0/797820358 conn(0x5559cb2b8a30 msgr2=0x5559cb2bb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.423+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 shutdown_connections 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.424+0000 7f192238f740 1 -- 172.21.15.33:0/797820358 wait complete. 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.424+0000 7f192238f740 1 -- start start 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.424+0000 7f192238f740 1 Processor -- start 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f192238f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f192238f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f192238f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f192238f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19181aed20 con 0x5559cb2bddb0 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f191db42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f191db42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53570/0 (socket says 172.21.15.33:53570) 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f191db42640 1 -- 172.21.15.33:0/2848136538 learned_addr learned my addr 172.21.15.33:0/2848136538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19181b0190 con 0x5559cb2bd3c0 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f191cb40640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.425+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19181af3f0 con 0x5559cb2c2f20 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f192238f740 10 client.0 _ll_register_callbacks cb 0x7f1908003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191d341640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191db42640 1 -- 172.21.15.33:0/2848136538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 msgr2=0x5559cb2cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191db42640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191db42640 1 -- 172.21.15.33:0/2848136538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 msgr2=0x5559cb2cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191db42640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191db42640 1 -- 172.21.15.33:0/2848136538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cb2cdd00 con 0x5559cb2c2f20 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f191db42640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cd710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19100989a0 tx=0x7f191009c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e0e885ca01dbf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f1900ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f1900ff9640 1 -- 172.21.15.33:0/2848136538 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19100a4c40 con 0x5559cb2c2f20 2024-04-24T23:34:54.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f1900ff9640 1 -- 172.21.15.33:0/2848136538 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f19100051d0 con 0x5559cb2c2f20 2024-04-24T23:34:54.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.426+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1914031b90 con 0x5559cb2c2f20 2024-04-24T23:34:54.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.427+0000 7f1900ff9640 1 -- 172.21.15.33:0/2848136538 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1910084050 con 0x5559cb2c2f20 2024-04-24T23:34:54.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.427+0000 7f192238f740 10 client.29557 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:54.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.427+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5559cb339a60 con 0x5559cb2c2f20 2024-04-24T23:34:54.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.427+0000 7f1900ff9640 1 -- 172.21.15.33:0/2848136538 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1910085240 con 0x5559cb2c2f20 2024-04-24T23:34:54.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f1900ff9640 1 -- 172.21.15.33:0/2848136538 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1910096cc0 con 0x5559cb2c2f20 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f192238f740 10 client.29557 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f192238f740 10 client.29557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f192238f740 20 client.29557 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f192238f740 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb33a460 0x5559cb33c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f192238f740 4 client.29557 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1914031b90 con 0x5559cb33a460 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f191d341640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb33a460 0x5559cb33c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f191d341640 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb33a460 0x5559cb33c880 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca8d05003655d241 server_cookie=e2fa705f43e8d36b in_seq=0 out_seq=0 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.428+0000 7f1900ff9640 10 client.29557 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f1900ff9640 1 -- 172.21.15.33:0/2848136538 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1914031b90 con 0x5559cb33a460 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f1900ff9640 10 client.29557 handle_command_reply: tid=0 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 client.29557 shutdown 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 msgr2=0x5559cb2cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cd710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19100989a0 tx=0x7f191009c750 comp rx=0 tx=0).stop 2024-04-24T23:34:54.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 shutdown_connections 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5559cb33a460 0x5559cb33c880 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559cb2c2f20 0x5559cb2cd710 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559cb2bddb0 0x5559cb2cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 --2- 172.21.15.33:0/2848136538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5559cb2bd3c0 0x5559cb2cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 >> 172.21.15.33:0/2848136538 conn(0x5559cb2b8a30 msgr2=0x5559cb2bc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f1900ff9640 0 client.29557 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 shutdown_connections 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 1 -- 172.21.15.33:0/2848136538 wait complete. 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.429+0000 7f192238f740 20 client.29557 trim_cache size 0 max 16384 2024-04-24T23:34:54.433 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:34:54.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f19040a2c90 msgr2=0x7f19040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f19040a2c90 0x7f19040a5150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x5559cb2be610 tx=0x7f190802d490 comp rx=0 tx=0).stop 2024-04-24T23:34:54.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19181097c0 msgr2=0x7f1918117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19181097c0 0x7f1918117800 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1910009fd0 tx=0x7f191000ef10 comp rx=0 tx=0).stop 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 shutdown_connections 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f19040a2c90 0x7f19040a5150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19181097c0 0x7f1918117800 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1918108e20 0x7f1918110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 --2- 172.21.15.33:0/3202345637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1918072b80 0x7f1918110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 >> 172.21.15.33:0/3202345637 conn(0x7f191806e650 msgr2=0x7f1918070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 shutdown_connections 2024-04-24T23:34:54.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.430+0000 7f191e343640 1 -- 172.21.15.33:0/3202345637 wait complete. 2024-04-24T23:34:54.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 -- 172.21.15.33:0/2600277435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 msgr2=0x7f627c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 --2- 172.21.15.33:0/2600277435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c108ac0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6274009ca0 tx=0x7f627402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 -- 172.21.15.33:0/2600277435 shutdown_connections 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 --2- 172.21.15.33:0/2600277435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c108ac0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 --2- 172.21.15.33:0/2600277435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c101870 0x7f627c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 --2- 172.21.15.33:0/2600277435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 0x7f627c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 -- 172.21.15.33:0/2600277435 >> 172.21.15.33:0/2600277435 conn(0x7f627c0fbbf0 msgr2=0x7f627c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 -- 172.21.15.33:0/2600277435 shutdown_connections 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.570+0000 7f6282be7640 1 -- 172.21.15.33:0/2600277435 wait complete. 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 Processor -- start 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 -- start start 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 0x7f627c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c101870 0x7f627c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f627c10d420 con 0x7f627c102230 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f627c10d2a0 con 0x7f627c101870 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6282be7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f627c10d5a0 con 0x7f627c100680 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6280983640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 0x7f627c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c1a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35974/0 (socket says 172.21.15.33:35974) 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 -- 172.21.15.33:0/3727107636 learned_addr learned my addr 172.21.15.33:0/3727107636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f627bfff640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c101870 0x7f627c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 -- 172.21.15.33:0/3727107636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 msgr2=0x7f627c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 0x7f627c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 -- 172.21.15.33:0/3727107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c101870 msgr2=0x7f627c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c101870 0x7f627c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6281184640 1 -- 172.21.15.33:0/3727107636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c1a1940 con 0x7f627c102230 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.571+0000 7f6280983640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 0x7f627c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.572+0000 7f6281184640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c1a12a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6274005e30 tx=0x7f6274031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.572+0000 7f6279ffb640 1 -- 172.21.15.33:0/3727107636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274003b70 con 0x7f627c102230 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.572+0000 7f6279ffb640 1 -- 172.21.15.33:0/3727107636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274003d10 con 0x7f627c102230 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.572+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c1a1bd0 con 0x7f627c102230 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.572+0000 7f6279ffb640 1 -- 172.21.15.33:0/3727107636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627402fb70 con 0x7f627c102230 2024-04-24T23:34:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.572+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c1a2170 con 0x7f627c102230 2024-04-24T23:34:54.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.574+0000 7f62859d5740 1 Processor -- start 2024-04-24T23:34:54.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.574+0000 7f6279ffb640 1 -- 172.21.15.33:0/3727107636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f627402fd10 con 0x7f627c102230 2024-04-24T23:34:54.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 -- start start 2024-04-24T23:34:54.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aeae31b670 con 0x55aeae3b7230 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aeae2c3970 con 0x55aeae3b7c20 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f62859d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aeae317630 con 0x55aeae3bcd90 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35978/0 (socket says 172.21.15.33:35978) 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 -- 172.21.15.33:0/3242349048 learned_addr learned my addr 172.21.15.33:0/3242349048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f6280983640 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 -- 172.21.15.33:0/3242349048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 msgr2=0x55aeae3bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 -- 172.21.15.33:0/3242349048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 msgr2=0x55aeae3bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f6279ffb640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f626c0a2c10 0x7f626c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 -- 172.21.15.33:0/3242349048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeae2e7400 con 0x55aeae3b7230 2024-04-24T23:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f6279ffb640 1 -- 172.21.15.33:0/3727107636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62740f04f0 con 0x7f627c102230 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3b7650 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6268013610 tx=0x7f6268009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc22c8ed2ee371eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.575+0000 7f627bfff640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f626c0a2c10 0x7f626c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f6266ffd640 1 -- 172.21.15.33:0/3242349048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626800f600 con 0x55aeae3b7230 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f6266ffd640 1 -- 172.21.15.33:0/3242349048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f626800fd00 con 0x55aeae3b7230 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f6266ffd640 1 -- 172.21.15.33:0/3242349048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268010690 con 0x55aeae3b7230 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 -- 172.21.15.33:0/3242349048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 msgr2=0x55aeae3b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f627bfff640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f626c0a2c10 0x7f626c0a50d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f626801d000 tx=0x7f6268010a30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3b7650 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6268013610 tx=0x7f6268009890 comp rx=0 tx=0).stop 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 -- 172.21.15.33:0/3242349048 shutdown_connections 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 --2- 172.21.15.33:0/3242349048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3b7650 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 -- 172.21.15.33:0/3242349048 >> 172.21.15.33:0/3242349048 conn(0x55aeae3b28a0 msgr2=0x55aeae3b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 -- 172.21.15.33:0/3242349048 shutdown_connections 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.576+0000 7f62859d5740 1 -- 172.21.15.33:0/3242349048 wait complete. 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 -- start start 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 Processor -- start 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 0x55aeae3c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 0x55aeae3c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aeae3bfd40 con 0x55aeae3bcd90 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aeae2d7050 con 0x55aeae3b7c20 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aeae3bfec0 con 0x55aeae3b7230 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f62859d5740 10 client.0 _ll_register_callbacks cb 0x55aeae02ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f6280983640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 0x55aeae3c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f6281184640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 0x55aeae3c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f6281184640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 0x55aeae3c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35990/0 (socket says 172.21.15.33:35990) 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f6281184640 1 -- 172.21.15.33:0/3803495714 learned_addr learned my addr 172.21.15.33:0/3803495714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f6280983640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 0x55aeae3c1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53598/0 (socket says 172.21.15.33:53598) 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.577+0000 7f627bfff640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6281184640 1 -- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 msgr2=0x55aeae3c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6281184640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 0x55aeae3c1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6281184640 1 -- 172.21.15.33:0/3803495714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 msgr2=0x55aeae3c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6281184640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6281184640 1 -- 172.21.15.33:0/3803495714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeae2a8dc0 con 0x55aeae3bcd90 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6280983640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 0x55aeae3c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f6281184640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 0x55aeae3c8880 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f62740c5100 tx=0x7f62740bc850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279c402667feea8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f625bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f625bfff640 1 -- 172.21.15.33:0/3803495714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740d5060 con 0x55aeae3bcd90 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f626800fd00 con 0x55aeae3bcd90 2024-04-24T23:34:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f625bfff640 1 -- 172.21.15.33:0/3803495714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62740b8fa0 con 0x55aeae3bcd90 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f625bfff640 1 -- 172.21.15.33:0/3803495714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62740bd0c0 con 0x55aeae3bcd90 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f62859d5740 10 client.20151 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aeae02f3f0 con 0x55aeae3bcd90 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.578+0000 7f625bfff640 1 -- 172.21.15.33:0/3803495714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f626800fd00 con 0x55aeae3bcd90 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f625bfff640 1 -- 172.21.15.33:0/3803495714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f62740db170 con 0x55aeae3bcd90 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f62859d5740 10 client.20151 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f62859d5740 10 client.20151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f62859d5740 20 client.20151 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f62859d5740 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae3cd160 0x55aeae3cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f62859d5740 4 client.20151 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeae328bb0 con 0x55aeae3cd160 2024-04-24T23:34:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f6280983640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae3cd160 0x55aeae3cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f625bfff640 10 client.20151 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.579+0000 7f6280983640 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae3cd160 0x55aeae3cf600 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba26e3b9b5cef58 server_cookie=5ae9616a34510c23 in_seq=0 out_seq=0 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f625bfff640 1 -- 172.21.15.33:0/3803495714 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aeae328bb0 con 0x55aeae3cd160 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f625bfff640 10 client.20151 handle_command_reply: tid=0 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 client.20151 shutdown 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 msgr2=0x55aeae3c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 0x55aeae3c8880 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f62740c5100 tx=0x7f62740bc850 comp rx=0 tx=0).stop 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 shutdown_connections 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae3cd160 0x55aeae3cf600 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3bcd90 0x55aeae3c8880 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f625bfff640 0 client.20151 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 --2- 172.21.15.33:0/3803495714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7230 0x55aeae3c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 >> 172.21.15.33:0/3803495714 conn(0x55aeae3b28a0 msgr2=0x55aeae3b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 shutdown_connections 2024-04-24T23:34:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 1 -- 172.21.15.33:0/3803495714 wait complete. 2024-04-24T23:34:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.580+0000 7f62859d5740 20 client.20151 trim_cache size 0 max 16384 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 Processor -- start 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 -- start start 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1aee00 con 0x55aeae3b7230 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ac6d0 con 0x55aeae3b7c20 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f62859d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1aa520 con 0x55aeae3bcd90 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f6280983640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f6280983640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3bc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35994/0 (socket says 172.21.15.33:35994) 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.596+0000 7f6280983640 1 -- 172.21.15.33:0/2004004489 learned_addr learned my addr 172.21.15.33:0/2004004489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6281184640 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f627bfff640 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6280983640 1 -- 172.21.15.33:0/2004004489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 msgr2=0x55aeae3b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6280983640 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6280983640 1 -- 172.21.15.33:0/2004004489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 msgr2=0x55aeae3c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6280983640 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6280983640 1 -- 172.21.15.33:0/2004004489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeae0f97d0 con 0x55aeae3b7230 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f627bfff640 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f6280983640 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3bc2a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f627c1016d0 tx=0x7f6270005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23da735726527ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f62667fc640 1 -- 172.21.15.33:0/2004004489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627000fcb0 con 0x55aeae3b7230 2024-04-24T23:34:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f62667fc640 1 -- 172.21.15.33:0/2004004489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627000fe50 con 0x55aeae3b7230 2024-04-24T23:34:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.597+0000 7f62667fc640 1 -- 172.21.15.33:0/2004004489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270010b40 con 0x55aeae3b7230 2024-04-24T23:34:54.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 -- 172.21.15.33:0/2004004489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 msgr2=0x55aeae3bc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3bc2a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f627c1016d0 tx=0x7f6270005f90 comp rx=0 tx=0).stop 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 -- 172.21.15.33:0/2004004489 shutdown_connections 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3bcd90 0x55aeae3b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3b7c20 0x55aeae3c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 --2- 172.21.15.33:0/2004004489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3bc2a0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 -- 172.21.15.33:0/2004004489 >> 172.21.15.33:0/2004004489 conn(0x55aeae3b28a0 msgr2=0x55aeae3b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 -- 172.21.15.33:0/2004004489 shutdown_connections 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 -- 172.21.15.33:0/2004004489 wait complete. 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.598+0000 7f62859d5740 1 -- start start 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 Processor -- start 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7c20 0x55aeae3c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3bcd90 0x55aeae3c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1aab90 con 0x55aeae3b7230 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ac000 con 0x55aeae3bcd90 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ab260 con 0x55aeae3b7c20 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f62859d5740 10 client.0 _ll_register_callbacks cb 0x7f62740095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f6280983640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f627bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7c20 0x55aeae3c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f6280983640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36004/0 (socket says 172.21.15.33:36004) 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f6280983640 1 -- 172.21.15.33:0/2795810396 learned_addr learned my addr 172.21.15.33:0/2795810396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f6281184640 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3bcd90 0x55aeae3c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f627bfff640 1 -- 172.21.15.33:0/2795810396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3bcd90 msgr2=0x55aeae3c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f627bfff640 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3bcd90 0x55aeae3c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f627bfff640 1 -- 172.21.15.33:0/2795810396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 msgr2=0x55aeae3c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f627bfff640 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.599+0000 7f627bfff640 1 -- 172.21.15.33:0/2795810396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeae3c7b30 con 0x55aeae3b7c20 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f627bfff640 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7c20 0x55aeae3c65a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6268022850 tx=0x7f6268027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5054d694a2eadf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f625bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f625bfff640 1 -- 172.21.15.33:0/2795810396 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62680097a0 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f627000fe50 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f625bfff640 1 -- 172.21.15.33:0/2795810396 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6268028030 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f625bfff640 1 -- 172.21.15.33:0/2795810396 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f626803e050 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 10 client.29575 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aeae4337f0 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f625bfff640 1 -- 172.21.15.33:0/2795810396 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627000fe50 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f625bfff640 1 -- 172.21.15.33:0/2795810396 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6268022040 con 0x55aeae3b7c20 2024-04-24T23:34:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 10 client.29575 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 10 client.29575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 20 client.29575 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.600+0000 7f62859d5740 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae4340d0 0x55aeae4364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f62859d5740 4 client.29575 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:34:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aeae322f00 con 0x55aeae4340d0 2024-04-24T23:34:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f6280983640 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae4340d0 0x55aeae4364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f625bfff640 10 client.29575 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f6280983640 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae4340d0 0x55aeae4364f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6d5b48c355dd9c server_cookie=d31aa82f29c8139d in_seq=0 out_seq=0 2024-04-24T23:34:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f625bfff640 1 -- 172.21.15.33:0/2795810396 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aeae322f00 con 0x55aeae4340d0 2024-04-24T23:34:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f625bfff640 10 client.29575 handle_command_reply: tid=0 2024-04-24T23:34:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.601+0000 7f62859d5740 1 client.29575 shutdown 2024-04-24T23:34:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7c20 msgr2=0x55aeae3c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7c20 0x55aeae3c65a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6268022850 tx=0x7f6268027000 comp rx=0 tx=0).stop 2024-04-24T23:34:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 shutdown_connections 2024-04-24T23:34:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55aeae4340d0 0x55aeae4364f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f625bfff640 0 client.29575 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aeae3bcd90 0x55aeae3c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aeae3b7c20 0x55aeae3c65a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 --2- 172.21.15.33:0/2795810396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aeae3b7230 0x55aeae3c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 >> 172.21.15.33:0/2795810396 conn(0x55aeae3b28a0 msgr2=0x55aeae3b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 shutdown_connections 2024-04-24T23:34:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 1 -- 172.21.15.33:0/2795810396 wait complete. 2024-04-24T23:34:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.602+0000 7f62859d5740 20 client.29575 trim_cache size 0 max 16384 2024-04-24T23:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:34:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f626c0a2c10 msgr2=0x7f626c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f626c0a2c10 0x7f626c0a50d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f626801d000 tx=0x7f6268010a30 comp rx=0 tx=0).stop 2024-04-24T23:34:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 msgr2=0x7f627c1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c1a12a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6274005e30 tx=0x7f6274031ee0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 shutdown_connections 2024-04-24T23:34:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f626c0a2c10 0x7f626c0a50d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f627c102230 0x7f627c1a12a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c101870 0x7f627c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 --2- 172.21.15.33:0/3727107636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c100680 0x7f627c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 >> 172.21.15.33:0/3727107636 conn(0x7f627c0fbbf0 msgr2=0x7f627c109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 shutdown_connections 2024-04-24T23:34:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:54.603+0000 7f6282be7640 1 -- 172.21.15.33:0/3727107636 wait complete. 2024-04-24T23:34:54.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 225940 22.02 MB/sec execute 419 sec latency 82.258 ms 2024-04-24T23:34:55.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:54 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1972044962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:34:55.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:54 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1972044962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:34:55.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:54 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1972044962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:34:55.618 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:34:55.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:34:55.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:34:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.748+0000 7f117a790640 1 -- 172.21.15.33:0/4134975836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 msgr2=0x7f1174101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.748+0000 7f117a790640 1 --2- 172.21.15.33:0/4134975836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 0x7f1174101bb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f1164009860 tx=0x7f116402a340 comp rx=0 tx=0).stop 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 -- 172.21.15.33:0/4134975836 shutdown_connections 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 --2- 172.21.15.33:0/4134975836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174103340 0x7f1174109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 --2- 172.21.15.33:0/4134975836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174102980 0x7f1174102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 --2- 172.21.15.33:0/4134975836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 0x7f1174101bb0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 -- 172.21.15.33:0/4134975836 >> 172.21.15.33:0/4134975836 conn(0x7f11740fcd20 msgr2=0x7f11740ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 -- 172.21.15.33:0/4134975836 shutdown_connections 2024-04-24T23:34:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.749+0000 7f117a790640 1 -- 172.21.15.33:0/4134975836 wait complete. 2024-04-24T23:34:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 Processor -- start 2024-04-24T23:34:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 -- start start 2024-04-24T23:34:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 0x7f1174197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 0x7f1174198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174103340 0x7f117419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f117410d410 con 0x7f1174101790 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f117410d290 con 0x7f1174103340 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.750+0000 7f117a790640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f117410d590 con 0x7f1174102980 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 0x7f1174198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 0x7f1174198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:56984/0 (socket says 172.21.15.33:56984) 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 -- 172.21.15.33:0/3339604837 learned_addr learned my addr 172.21.15.33:0/3339604837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f1173fff640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 0x7f1174197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174103340 0x7f117419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 -- 172.21.15.33:0/3339604837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174103340 msgr2=0x7f117419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174103340 0x7f117419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 -- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 msgr2=0x7f1174197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 0x7f1174197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 -- 172.21.15.33:0/3339604837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1174073a70 con 0x7f1174102980 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174103340 0x7f117419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.751+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 0x7f1174198010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9c0 tx=0x7f116800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.752+0000 7f11717fa640 1 -- 172.21.15.33:0/3339604837 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800cde0 con 0x7f1174102980 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.752+0000 7f11717fa640 1 -- 172.21.15.33:0/3339604837 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11680035f0 con 0x7f1174102980 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.752+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1174073d60 con 0x7f1174102980 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.752+0000 7f11717fa640 1 -- 172.21.15.33:0/3339604837 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168010690 con 0x7f1174102980 2024-04-24T23:34:55.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.752+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1174074300 con 0x7f1174102980 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 Processor -- start 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 -- start start 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273244280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273246d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f2731a22c0 con 0x55f27323f780 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f27314a910 con 0x55f27323ed90 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f117d57d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f27319e280 con 0x55f2732448f0 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273244280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273244280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46196/0 (socket says 172.21.15.33:46196) 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 -- 172.21.15.33:0/3814136494 learned_addr learned my addr 172.21.15.33:0/3814136494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 -- 172.21.15.33:0/3814136494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 msgr2=0x55f273246d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273246d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 -- 172.21.15.33:0/3814136494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 msgr2=0x55f27323f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1173fff640 1 -- 172.21.15.33:0/3814136494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27315bfb0 con 0x55f27323f780 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.755+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.756+0000 7f11717fa640 1 -- 172.21.15.33:0/3339604837 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f11680037a0 con 0x7f1174102980 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.756+0000 7f1173fff640 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273244280 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1164005940 tx=0x7f1164005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4effb4ea2f7f6145 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.756+0000 7f115a7fc640 1 -- 172.21.15.33:0/3814136494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037040 con 0x55f27323f780 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.756+0000 7f115a7fc640 1 -- 172.21.15.33:0/3814136494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1164004030 con 0x55f27323f780 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.756+0000 7f115a7fc640 1 -- 172.21.15.33:0/3814136494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640055b0 con 0x55f27323f780 2024-04-24T23:34:55.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.757+0000 7f11717fa640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f115c0a2c10 0x7f115c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.757+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f115c0a2c10 0x7f115c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.757+0000 7f117d57d740 1 -- 172.21.15.33:0/3814136494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 msgr2=0x55f273244280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.757+0000 7f117d57d740 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273244280 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1164005940 tx=0x7f1164005c90 comp rx=0 tx=0).stop 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.757+0000 7f11717fa640 1 -- 172.21.15.33:0/3339604837 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11680cf800 con 0x7f1174102980 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 -- 172.21.15.33:0/3814136494 shutdown_connections 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273246d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273244280 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 --2- 172.21.15.33:0/3814136494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 -- 172.21.15.33:0/3814136494 >> 172.21.15.33:0/3814136494 conn(0x55f273206950 msgr2=0x55f27323a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 -- 172.21.15.33:0/3814136494 shutdown_connections 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f115c0a2c10 0x7f115c0a50d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55f27323ffe0 tx=0x7f116c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 -- 172.21.15.33:0/3814136494 wait complete. 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 -- start start 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.758+0000 7f117d57d740 1 Processor -- start 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f273249210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273249750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273250720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f273247840 con 0x55f27323f780 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f27315de90 con 0x55f27323ed90 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2732479c0 con 0x55f2732448f0 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f11737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273249750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f273249210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f273249210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49114/0 (socket says 172.21.15.33:49114) 2024-04-24T23:34:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1178d2d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273250720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 -- 172.21.15.33:0/2385048143 learned_addr learned my addr 172.21.15.33:0/2385048143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f117d57d740 10 client.0 _ll_register_callbacks cb 0x55f273199970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 -- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 msgr2=0x55f273250720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273250720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 -- 172.21.15.33:0/2385048143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 msgr2=0x55f273249750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273249750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1173fff640 1 -- 172.21.15.33:0/2385048143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f272f7e7e0 con 0x55f27323ed90 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f1178d2d640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273250720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.759+0000 7f11737fe640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273249750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f1173fff640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f273249210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1164000c00 tx=0x7f1164004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96aa7af24a3407b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f11537fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:34:55.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f11537fe640 1 -- 172.21.15.33:0/2385048143 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037050 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f11537fe640 1 -- 172.21.15.33:0/2385048143 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f11640052f0 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1164004030 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f11537fe640 1 -- 172.21.15.33:0/2385048143 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f11640054f0 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f117d57d740 10 client.29639 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f272fdef70 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f11537fe640 1 -- 172.21.15.33:0/2385048143 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1164004030 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f11537fe640 1 -- 172.21.15.33:0/2385048143 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1164003770 con 0x55f27323ed90 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f117d57d740 10 client.29639 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f117d57d740 10 client.29639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.760+0000 7f117d57d740 20 client.29639 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.761+0000 7f117d57d740 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f273256c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.761+0000 7f117d57d740 4 client.29639 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.761+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2731765a0 con 0x55f273254860 2024-04-24T23:34:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.761+0000 7f11737fe640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f273256c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.761+0000 7f11737fe640 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f273256c50 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86039b21956f5131 server_cookie=f2bd2e5b796c0413 in_seq=0 out_seq=0 2024-04-24T23:34:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.761+0000 7f11537fe640 10 client.29639 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.763+0000 7f11537fe640 1 -- 172.21.15.33:0/2385048143 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2731765a0 con 0x55f273254860 2024-04-24T23:34:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.763+0000 7f11537fe640 10 client.29639 handle_command_reply: tid=0 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.763+0000 7f117d57d740 1 client.29639 shutdown 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 msgr2=0x55f273249210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f273249210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1164000c00 tx=0x7f1164004600 comp rx=0 tx=0).stop 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 shutdown_connections 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f273256c50 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f273250720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f11537fe640 0 client.29639 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f273249750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 --2- 172.21.15.33:0/2385048143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f273249210 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 >> 172.21.15.33:0/2385048143 conn(0x55f273206950 msgr2=0x55f273239950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 shutdown_connections 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 1 -- 172.21.15.33:0/2385048143 wait complete. 2024-04-24T23:34:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.764+0000 7f117d57d740 20 client.29639 trim_cache size 0 max 16384 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 Processor -- start 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 -- start start 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f2732506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f2732397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11741aee80 con 0x55f27323f780 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ac750 con 0x55f27323ed90 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f117d57d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11741aa5a0 con 0x55f2732448f0 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f11737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f2732506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f11737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f2732506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46200/0 (socket says 172.21.15.33:46200) 2024-04-24T23:34:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f11737fe640 1 -- 172.21.15.33:0/2117692609 learned_addr learned my addr 172.21.15.33:0/2117692609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.787+0000 7f1173fff640 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f1178d2d640 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f2732397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f11737fe640 1 -- 172.21.15.33:0/2117692609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 msgr2=0x55f2732397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f11737fe640 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f2732397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f11737fe640 1 -- 172.21.15.33:0/2117692609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 msgr2=0x55f27323dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f11737fe640 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f11737fe640 1 -- 172.21.15.33:0/2117692609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f272e76170 con 0x55f27323f780 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f1173fff640 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f11737fe640 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f2732506e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f11680ccf10 tx=0x7f1168097890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d124ae0cb9ae1c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f1159ffb640 1 -- 172.21.15.33:0/2117692609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116809bc70 con 0x55f27323f780 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f1159ffb640 1 -- 172.21.15.33:0/2117692609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11680a0020 con 0x55f27323f780 2024-04-24T23:34:55.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.788+0000 7f1159ffb640 1 -- 172.21.15.33:0/2117692609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116809b0c0 con 0x55f27323f780 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 -- 172.21.15.33:0/2117692609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 msgr2=0x55f2732506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f2732506e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f11680ccf10 tx=0x7f1168097890 comp rx=0 tx=0).stop 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 -- 172.21.15.33:0/2117692609 shutdown_connections 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2732448f0 0x55f2732397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323f780 0x55f2732506e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 --2- 172.21.15.33:0/2117692609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f27323ed90 0x55f27323dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 -- 172.21.15.33:0/2117692609 >> 172.21.15.33:0/2117692609 conn(0x55f273206950 msgr2=0x55f2732390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 -- 172.21.15.33:0/2117692609 shutdown_connections 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 -- 172.21.15.33:0/2117692609 wait complete. 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.789+0000 7f117d57d740 1 -- start start 2024-04-24T23:34:55.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 Processor -- start 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 0x55f27324e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27323f780 0x55f27324e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 0x55f27324e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2732448f0 0x55f27324f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 0x55f27324e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46208/0 (socket says 172.21.15.33:46208) 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 -- 172.21.15.33:0/3339360720 learned_addr learned my addr 172.21.15.33:0/3339360720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11741aac10 con 0x55f27323ed90 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27323f780 0x55f27324e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ac080 con 0x55f2732448f0 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11741ab2e0 con 0x55f27323f780 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f117d57d740 10 client.0 _ll_register_callbacks cb 0x7f116c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1178d2d640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2732448f0 0x55f27324f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 -- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27323f780 msgr2=0x55f27324e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27323f780 0x55f27324e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 -- 172.21.15.33:0/3339360720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2732448f0 msgr2=0x55f27324f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2732448f0 0x55f27324f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27324ffc0 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27323f780 0x55f27324e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f1173fff640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 0x55f27324e480 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f116402a860 tx=0x7f116402a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf26ee064d11865b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.790+0000 7f11537fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f11537fe640 1 -- 172.21.15.33:0/3339360720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037070 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f11537fe640 1 -- 172.21.15.33:0/3339360720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f116403d470 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11680a0020 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f11537fe640 1 -- 172.21.15.33:0/3339360720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1164060050 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 10 client.20175 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2732bb290 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f11537fe640 1 -- 172.21.15.33:0/3339360720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11680a0020 con 0x55f27323ed90 2024-04-24T23:34:55.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f11537fe640 1 -- 172.21.15.33:0/3339360720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1164032440 con 0x55f27323ed90 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 10 client.20175 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 10 client.20175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 20 client.20175 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f2732bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 4 client.20175 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f2731af800 con 0x55f273254860 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.791+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f2732bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.792+0000 7f11737fe640 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f2732bdd50 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d171def1d0149c2 server_cookie=5c46b1425925a9f9 in_seq=0 out_seq=0 2024-04-24T23:34:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.792+0000 7f11537fe640 10 client.20175 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f11537fe640 1 -- 172.21.15.33:0/3339360720 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2731af800 con 0x55f273254860 2024-04-24T23:34:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f11537fe640 10 client.20175 handle_command_reply: tid=0 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 client.20175 shutdown 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 msgr2=0x55f27324e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 0x55f27324e480 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f116402a860 tx=0x7f116402a9c0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 shutdown_connections 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55f273254860 0x55f2732bdd50 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2732448f0 0x55f27324f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27323f780 0x55f27324e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 --2- 172.21.15.33:0/3339360720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f27323ed90 0x55f27324e480 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 >> 172.21.15.33:0/3339360720 conn(0x55f273206950 msgr2=0x55f273239950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f11537fe640 0 client.20175 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 shutdown_connections 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 1 -- 172.21.15.33:0/3339360720 wait complete. 2024-04-24T23:34:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.795+0000 7f117d57d740 20 client.20175 trim_cache size 0 max 16384 2024-04-24T23:34:55.800 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:34:55.800 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:34:55.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "55273722-0347-4170-a0ba-719d7968bb41", 2024-04-24T23:34:55.800 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:34:55.800 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.796+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f115c0a2c10 msgr2=0x7f115c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.796+0000 7f117a790640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f115c0a2c10 0x7f115c0a50d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55f27323ffe0 tx=0x7f116c009500 comp rx=0 tx=0).stop 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.796+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 msgr2=0x7f1174198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.796+0000 7f117a790640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 0x7f1174198010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9c0 tx=0x7f116800ee80 comp rx=0 tx=0).stop 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 shutdown_connections 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f115c0a2c10 0x7f115c0a50d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1174103340 0x7f117419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1174102980 0x7f1174198010 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 --2- 172.21.15.33:0/3339604837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1174101790 0x7f1174197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 >> 172.21.15.33:0/3339604837 conn(0x7f11740fcd20 msgr2=0x7f1174105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 shutdown_connections 2024-04-24T23:34:55.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.797+0000 7f117a790640 1 -- 172.21.15.33:0/3339604837 wait complete. 2024-04-24T23:34:55.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:34:55.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:34:55.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 226238 22.00 MB/sec execute 420 sec latency 341.685 ms 2024-04-24T23:34:55.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:55 smithi033 ceph-mon[29201]: pgmap v1038: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 397 op/s 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 -- 172.21.15.33:0/1913579805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 msgr2=0x7f30ec10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 --2- 172.21.15.33:0/1913579805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec10d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009900 tx=0x7f30dc02a440 comp rx=0 tx=0).stop 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 -- 172.21.15.33:0/1913579805 shutdown_connections 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 --2- 172.21.15.33:0/1913579805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 0x7f30ec10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 --2- 172.21.15.33:0/1913579805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec10d540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 --2- 172.21.15.33:0/1913579805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ec108740 0x7f30ec10ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.958+0000 7f30f32a5640 1 -- 172.21.15.33:0/1913579805 >> 172.21.15.33:0/1913579805 conn(0x7f30ec06e5d0 msgr2=0x7f30ec070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.959+0000 7f30f32a5640 1 -- 172.21.15.33:0/1913579805 shutdown_connections 2024-04-24T23:34:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.959+0000 7f30f32a5640 1 -- 172.21.15.33:0/1913579805 wait complete. 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 Processor -- start 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 -- start start 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ec108740 0x7f30ec1a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec1a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 0x7f30ec1ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec11c330 con 0x7f30ec10da80 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec11c1b0 con 0x7f30ec108740 2024-04-24T23:34:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.960+0000 7f30f32a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec11c4b0 con 0x7f30ec10b0b0 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f0840640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec1a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f0840640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec1a7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57008/0 (socket says 172.21.15.33:57008) 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f0840640 1 -- 172.21.15.33:0/3943033146 learned_addr learned my addr 172.21.15.33:0/3943033146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1842640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 0x7f30ec1ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1041640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ec108740 0x7f30ec1a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1842640 1 -- 172.21.15.33:0/3943033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 msgr2=0x7f30ec1a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1842640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec1a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1842640 1 -- 172.21.15.33:0/3943033146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ec108740 msgr2=0x7f30ec1a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1842640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ec108740 0x7f30ec1a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.961+0000 7f30f1842640 1 -- 172.21.15.33:0/3943033146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec1ae690 con 0x7f30ec10da80 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.962+0000 7f30f1842640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 0x7f30ec1ae020 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b3e0 tx=0x7f30e800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.962+0000 7f30e27fc640 1 -- 172.21.15.33:0/3943033146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8015020 con 0x7f30ec10da80 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.962+0000 7f30e27fc640 1 -- 172.21.15.33:0/3943033146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e8011d10 con 0x7f30ec10da80 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.962+0000 7f30e27fc640 1 -- 172.21.15.33:0/3943033146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800f2e0 con 0x7f30ec10da80 2024-04-24T23:34:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.963+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec1ae980 con 0x7f30ec10da80 2024-04-24T23:34:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.963+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1aef00 con 0x7f30ec10da80 2024-04-24T23:34:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.968+0000 7f30e27fc640 1 -- 172.21.15.33:0/3943033146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f30e8018020 con 0x7f30ec10da80 2024-04-24T23:34:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.969+0000 7f30e27fc640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30d80a2d60 0x7f30d80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.969+0000 7f30e27fc640 1 -- 172.21.15.33:0/3943033146 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30e80cebc0 con 0x7f30ec10da80 2024-04-24T23:34:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.969+0000 7f30f5fb1740 1 Processor -- start 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 -- start start 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f1041640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30d80a2d60 0x7f30d80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad83730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 0x555b4ad86230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b4ace26b0 con 0x555b4ad7ec30 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b4ac8a880 con 0x555b4ad7e240 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b4acde670 con 0x555b4ad83da0 2024-04-24T23:34:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.970+0000 7f30f1041640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad7e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49174/0 (socket says 172.21.15.33:49174) 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 -- 172.21.15.33:0/706878896 learned_addr learned my addr 172.21.15.33:0/706878896 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30d80a2d60 0x7f30d80a5220 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c550 tx=0x7f30e400c950 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 -- 172.21.15.33:0/706878896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 msgr2=0x555b4ad86230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 0x555b4ad86230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 -- 172.21.15.33:0/706878896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 msgr2=0x555b4ad83730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad83730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.971+0000 7f30f1041640 1 -- 172.21.15.33:0/706878896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4aac07d0 con 0x555b4ad7e240 2024-04-24T23:34:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.972+0000 7f30f1041640 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad7e660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f30e401a6d0 tx=0x7f30e401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff247f696df568b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.975+0000 7f30d77fe640 1 -- 172.21.15.33:0/706878896 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4027070 con 0x555b4ad7e240 2024-04-24T23:34:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.975+0000 7f30d77fe640 1 -- 172.21.15.33:0/706878896 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e401ae40 con 0x555b4ad7e240 2024-04-24T23:34:55.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.975+0000 7f30d77fe640 1 -- 172.21.15.33:0/706878896 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4022b30 con 0x555b4ad7e240 2024-04-24T23:34:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.976+0000 7f30f5fb1740 1 -- 172.21.15.33:0/706878896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 msgr2=0x555b4ad7e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.976+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad7e660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f30e401a6d0 tx=0x7f30e401a000 comp rx=0 tx=0).stop 2024-04-24T23:34:55.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 -- 172.21.15.33:0/706878896 shutdown_connections 2024-04-24T23:34:55.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 0x555b4ad86230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad83730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/706878896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad7e660 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 -- 172.21.15.33:0/706878896 >> 172.21.15.33:0/706878896 conn(0x555b4ad798b0 msgr2=0x555b4ad79cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 -- 172.21.15.33:0/706878896 shutdown_connections 2024-04-24T23:34:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.981+0000 7f30f5fb1740 1 -- 172.21.15.33:0/706878896 wait complete. 2024-04-24T23:34:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.982+0000 7f30f5fb1740 1 -- start start 2024-04-24T23:34:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.982+0000 7f30f5fb1740 1 Processor -- start 2024-04-24T23:34:55.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.982+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad88a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f1041640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f1041640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad88540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49186/0 (socket says 172.21.15.33:49186) 2024-04-24T23:34:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f1041640 1 -- 172.21.15.33:0/3025581476 learned_addr learned my addr 172.21.15.33:0/3025581476 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 0x555b4ad8fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b4ad86d50 con 0x555b4ad7ec30 2024-04-24T23:34:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f0840640 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad88a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b4ac9df50 con 0x555b4ad7e240 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.983+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b4ad86ed0 con 0x555b4ad83da0 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.984+0000 7f30f5fb1740 10 client.0 _ll_register_callbacks cb 0x555b4acd9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.984+0000 7f30f0840640 1 -- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 msgr2=0x555b4ad8fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.984+0000 7f30f0840640 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 0x555b4ad8fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.984+0000 7f30f0840640 1 -- 172.21.15.33:0/3025581476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 msgr2=0x555b4ad88540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.984+0000 7f30f0840640 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad88540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.984+0000 7f30f0840640 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b4a9f63f0 con 0x555b4ad7ec30 2024-04-24T23:34:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.985+0000 7f30f0840640 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad88a80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f30dc000c00 tx=0x7f30dc0318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7559f1e6f69ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.985+0000 7f30d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:34:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.985+0000 7f30d4ff9640 1 -- 172.21.15.33:0/3025581476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc03f070 con 0x555b4ad7ec30 2024-04-24T23:34:55.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.985+0000 7f30d4ff9640 1 -- 172.21.15.33:0/3025581476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30dc031de0 con 0x555b4ad7ec30 2024-04-24T23:34:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.987+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b4acefbf0 con 0x555b4ad7ec30 2024-04-24T23:34:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.987+0000 7f30d4ff9640 1 -- 172.21.15.33:0/3025581476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f30dc035070 con 0x555b4ad7ec30 2024-04-24T23:34:55.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.989+0000 7f30f5fb1740 10 client.20181 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:55.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.989+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555b4a9b60f0 con 0x555b4ad7ec30 2024-04-24T23:34:55.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.990+0000 7f30d4ff9640 1 -- 172.21.15.33:0/3025581476 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555b4acefbf0 con 0x555b4ad7ec30 2024-04-24T23:34:55.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.990+0000 7f30d4ff9640 1 -- 172.21.15.33:0/3025581476 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f30dc044730 con 0x555b4ad7ec30 2024-04-24T23:34:55.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.991+0000 7f30f5fb1740 10 client.20181 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:55.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.991+0000 7f30f5fb1740 10 client.20181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:55.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.991+0000 7f30f5fb1740 20 client.20181 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.992+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ad94300 0x555b4ad96770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.992+0000 7f30f5fb1740 4 client.20181 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:34:55.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.992+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b4ace9f40 con 0x555b4ad94300 2024-04-24T23:34:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.998+0000 7f30f1041640 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ad94300 0x555b4ad96770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.999+0000 7f30f1041640 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ad94300 0x555b4ad96770 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea49957360f9f12 server_cookie=a4b737ac5e75d1dd in_seq=0 out_seq=0 2024-04-24T23:34:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:55.999+0000 7f30d4ff9640 10 client.20181 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.000+0000 7f30d4ff9640 1 -- 172.21.15.33:0/3025581476 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b4ace9f40 con 0x555b4ad94300 2024-04-24T23:34:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.000+0000 7f30d4ff9640 10 client.20181 handle_command_reply: tid=0 2024-04-24T23:34:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.000+0000 7f30f5fb1740 1 client.20181 shutdown 2024-04-24T23:34:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 msgr2=0x555b4ad88a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad88a80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f30dc000c00 tx=0x7f30dc0318e0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 shutdown_connections 2024-04-24T23:34:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ad94300 0x555b4ad96770 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad83da0 0x555b4ad8fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad88a80 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.002+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/3025581476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad7e240 0x555b4ad88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.003+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 >> 172.21.15.33:0/3025581476 conn(0x555b4ad798b0 msgr2=0x555b4ad78af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.003+0000 7f30d4ff9640 0 client.20181 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:56.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.005+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 shutdown_connections 2024-04-24T23:34:56.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.007+0000 7f30f5fb1740 1 -- 172.21.15.33:0/3025581476 wait complete. 2024-04-24T23:34:56.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.007+0000 7f30f5fb1740 20 client.20181 trim_cache size 0 max 16384 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.044+0000 7f30f5fb1740 1 Processor -- start 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.044+0000 7f30f5fb1740 1 -- start start 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.044+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad78970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.044+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.044+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad9b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.044+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1bbc90 con 0x555b4ad7ec30 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.045+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b9560 con 0x555b4ad83da0 2024-04-24T23:34:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.045+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b73b0 con 0x555b4ad7e240 2024-04-24T23:34:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.047+0000 7f30f1842640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad9b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.047+0000 7f30f1041640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad78970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.047+0000 7f30f1842640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad9b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:49194/0 (socket says 172.21.15.33:49194) 2024-04-24T23:34:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.047+0000 7f30f1041640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad78970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57024/0 (socket says 172.21.15.33:57024) 2024-04-24T23:34:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.047+0000 7f30f1842640 1 -- 172.21.15.33:0/1527004627 learned_addr learned my addr 172.21.15.33:0/1527004627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.048+0000 7f30f0840640 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.048+0000 7f30f1041640 1 -- 172.21.15.33:0/1527004627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 msgr2=0x555b4ad9b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.048+0000 7f30f1041640 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad9b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.048+0000 7f30f1041640 1 -- 172.21.15.33:0/1527004627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 msgr2=0x555b4ad8fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.048+0000 7f30f1041640 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.048+0000 7f30f1041640 1 -- 172.21.15.33:0/1527004627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4ad96ae0 con 0x555b4ad7e240 2024-04-24T23:34:56.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.050+0000 7f30f1041640 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad78970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x555b4ad96fb0 tx=0x7f30e4022430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c5e6ca3851845 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:56.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.054+0000 7f30d6ffd640 1 -- 172.21.15.33:0/1527004627 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4027040 con 0x555b4ad7e240 2024-04-24T23:34:56.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.054+0000 7f30d6ffd640 1 -- 172.21.15.33:0/1527004627 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4036070 con 0x555b4ad7e240 2024-04-24T23:34:56.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.055+0000 7f30d6ffd640 1 -- 172.21.15.33:0/1527004627 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4007d40 con 0x555b4ad7e240 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.055+0000 7f30f5fb1740 1 -- 172.21.15.33:0/1527004627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 msgr2=0x555b4ad78970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.055+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad78970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x555b4ad96fb0 tx=0x7f30e4022430 comp rx=0 tx=0).stop 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.056+0000 7f30f5fb1740 1 -- 172.21.15.33:0/1527004627 shutdown_connections 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.056+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad9b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.056+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.056+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/1527004627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad78970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.056+0000 7f30f5fb1740 1 -- 172.21.15.33:0/1527004627 >> 172.21.15.33:0/1527004627 conn(0x555b4ad798b0 msgr2=0x555b4ad78260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.060+0000 7f30f5fb1740 1 -- 172.21.15.33:0/1527004627 shutdown_connections 2024-04-24T23:34:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.061+0000 7f30f5fb1740 1 -- 172.21.15.33:0/1527004627 wait complete. 2024-04-24T23:34:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.061+0000 7f30f5fb1740 1 -- start start 2024-04-24T23:34:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.062+0000 7f30f5fb1740 1 Processor -- start 2024-04-24T23:34:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad8d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad8ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b7a20 con 0x555b4ad7ec30 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b8e90 con 0x555b4ad83da0 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b80f0 con 0x555b4ad7e240 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f5fb1740 10 client.0 _ll_register_callbacks cb 0x7f30dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f1041640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad8d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f1041640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad8d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:57040/0 (socket says 172.21.15.33:57040) 2024-04-24T23:34:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.066+0000 7f30f1041640 1 -- 172.21.15.33:0/353289791 learned_addr learned my addr 172.21.15.33:0/353289791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.067+0000 7f30f1842640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad8ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.067+0000 7f30f0840640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.068+0000 7f30f1041640 1 -- 172.21.15.33:0/353289791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 msgr2=0x555b4ad8ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.068+0000 7f30f1041640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad8ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.068+0000 7f30f1041640 1 -- 172.21.15.33:0/353289791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 msgr2=0x555b4ad8da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.068+0000 7f30f1041640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.068+0000 7f30f1041640 1 -- 172.21.15.33:0/353289791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b4ad8f070 con 0x555b4ad7e240 2024-04-24T23:34:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.068+0000 7f30f1842640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad8ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:34:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.069+0000 7f30f1041640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad8d530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30e401c010 tx=0x7f30e4009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf94ab151488987c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:34:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.069+0000 7f30d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:34:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.069+0000 7f30d4ff9640 1 -- 172.21.15.33:0/353289791 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e403f070 con 0x555b4ad7e240 2024-04-24T23:34:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.069+0000 7f30d4ff9640 1 -- 172.21.15.33:0/353289791 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30e4007600 con 0x555b4ad7e240 2024-04-24T23:34:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.069+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4036070 con 0x555b4ad7e240 2024-04-24T23:34:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.070+0000 7f30d4ff9640 1 -- 172.21.15.33:0/353289791 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f30e4007800 con 0x555b4ad7e240 2024-04-24T23:34:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.070+0000 7f30f5fb1740 10 client.29602 fetch_fsmap learned FSMap version 33 2024-04-24T23:34:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.070+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555b4ae0bc80 con 0x555b4ad7e240 2024-04-24T23:34:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.070+0000 7f30d4ff9640 1 -- 172.21.15.33:0/353289791 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4036070 con 0x555b4ad7e240 2024-04-24T23:34:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30d4ff9640 1 -- 172.21.15.33:0/353289791 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f30e404c020 con 0x555b4ad7e240 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f5fb1740 10 client.29602 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f5fb1740 10 client.29602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f5fb1740 20 client.29602 populate_metadata read hostname 'smithi033' 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ae0c410 0x555b4ae0e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f5fb1740 4 client.29602 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b4ac676f0 con 0x555b4ae0c410 2024-04-24T23:34:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.071+0000 7f30f0840640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ae0c410 0x555b4ae0e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:34:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.072+0000 7f30f0840640 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ae0c410 0x555b4ae0e830 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da835f0a63c57ed server_cookie=de4a6737d456f2b1 in_seq=0 out_seq=0 2024-04-24T23:34:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.072+0000 7f30d4ff9640 10 client.29602 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.072+0000 7f30d4ff9640 1 -- 172.21.15.33:0/353289791 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555b4ac676f0 con 0x555b4ae0c410 2024-04-24T23:34:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.072+0000 7f30d4ff9640 10 client.29602 handle_command_reply: tid=0 2024-04-24T23:34:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 client.29602 shutdown 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 msgr2=0x555b4ad8d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad8d530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30e401c010 tx=0x7f30e4009860 comp rx=0 tx=0).stop 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 shutdown_connections 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x555b4ae0c410 0x555b4ae0e830 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b4ad83da0 0x555b4ad8ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4ad7ec30 0x555b4ad8da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 --2- 172.21.15.33:0/353289791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b4ad7e240 0x555b4ad8d530 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30e401c010 tx=0x7f30e4009860 comp rx=0 tx=0).stop 2024-04-24T23:34:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 >> 172.21.15.33:0/353289791 conn(0x555b4ad798b0 msgr2=0x555b4ad7cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.073+0000 7f30d4ff9640 0 client.29602 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:34:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.074+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 shutdown_connections 2024-04-24T23:34:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.074+0000 7f30f5fb1740 1 -- 172.21.15.33:0/353289791 wait complete. 2024-04-24T23:34:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.074+0000 7f30f5fb1740 20 client.29602 trim_cache size 0 max 16384 2024-04-24T23:34:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "55273722-0347-4170-a0ba-719d7968bb41": { 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "55273722-0347-4170-a0ba-719d7968bb41", 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:34:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.075+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30d80a2d60 msgr2=0x7f30d80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.075+0000 7f30f32a5640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30d80a2d60 0x7f30d80a5220 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c550 tx=0x7f30e400c950 comp rx=0 tx=0).stop 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.075+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 msgr2=0x7f30ec1ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.075+0000 7f30f32a5640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 0x7f30ec1ae020 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b3e0 tx=0x7f30e800b8a0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.075+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 shutdown_connections 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.075+0000 7f30f32a5640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f30d80a2d60 0x7f30d80a5220 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.076+0000 7f30f32a5640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30ec10da80 0x7f30ec1ae020 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.076+0000 7f30f32a5640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30ec10b0b0 0x7f30ec1a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.076+0000 7f30f32a5640 1 --2- 172.21.15.33:0/3943033146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ec108740 0x7f30ec1a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.076+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 >> 172.21.15.33:0/3943033146 conn(0x7f30ec06e5d0 msgr2=0x7f30ec108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.076+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 shutdown_connections 2024-04-24T23:34:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:34:56.076+0000 7f30f32a5640 1 -- 172.21.15.33:0/3943033146 wait complete. 2024-04-24T23:34:56.094 INFO:tasks.cephfs.filesystem:scrub status for tag:55273722-0347-4170-a0ba-719d7968bb41 - {'path': '/', 'tag': '55273722-0347-4170-a0ba-719d7968bb41', 'options': 'recursive,force'} 2024-04-24T23:34:56.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:55 smithi038 ceph-mon[32450]: pgmap v1038: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 397 op/s 2024-04-24T23:34:56.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:55 smithi117 ceph-mon[32529]: pgmap v1038: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 397 op/s 2024-04-24T23:34:56.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 226589 21.98 MB/sec execute 421 sec latency 108.638 ms 2024-04-24T23:34:57.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:34:57.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:57.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:57.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:57.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:34:57.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:56 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:34:57.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:56 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:34:57.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:34:57.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:56 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:57.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 227146 22.00 MB/sec execute 422 sec latency 78.277 ms 2024-04-24T23:34:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:57 smithi038 ceph-mon[32450]: pgmap v1039: 129 pgs: 129 active+clean; 661 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 376 op/s 2024-04-24T23:34:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:57 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:58.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:57 smithi117 ceph-mon[32529]: pgmap v1039: 129 pgs: 129 active+clean; 661 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 376 op/s 2024-04-24T23:34:58.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:57 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:58.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:57 smithi033 ceph-mon[29201]: pgmap v1039: 129 pgs: 129 active+clean; 661 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 376 op/s 2024-04-24T23:34:58.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:57 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:34:58.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 227714 22.04 MB/sec execute 423 sec latency 85.520 ms 2024-04-24T23:34:59.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 228163 22.04 MB/sec execute 424 sec latency 79.328 ms 2024-04-24T23:35:00.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:34:59 smithi038 ceph-mon[32450]: pgmap v1040: 129 pgs: 129 active+clean; 664 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2024-04-24T23:35:00.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:34:59 smithi117 ceph-mon[32529]: pgmap v1040: 129 pgs: 129 active+clean; 664 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2024-04-24T23:35:00.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:34:59 smithi033 ceph-mon[29201]: pgmap v1040: 129 pgs: 129 active+clean; 664 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2024-04-24T23:35:00.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 228371 22.02 MB/sec execute 425 sec latency 476.547 ms 2024-04-24T23:35:01.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 228603 21.99 MB/sec execute 426 sec latency 143.108 ms 2024-04-24T23:35:02.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:01 smithi038 ceph-mon[32450]: pgmap v1041: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 415 op/s 2024-04-24T23:35:02.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:01 smithi117 ceph-mon[32529]: pgmap v1041: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 415 op/s 2024-04-24T23:35:02.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:01 smithi033 ceph-mon[29201]: pgmap v1041: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 415 op/s 2024-04-24T23:35:02.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 229033 21.98 MB/sec execute 427 sec latency 78.466 ms 2024-04-24T23:35:03.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 229535 22.01 MB/sec execute 428 sec latency 69.665 ms 2024-04-24T23:35:04.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:03 smithi038 ceph-mon[32450]: pgmap v1042: 129 pgs: 129 active+clean; 657 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.1 MiB/s wr, 453 op/s 2024-04-24T23:35:04.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:03 smithi117 ceph-mon[32529]: pgmap v1042: 129 pgs: 129 active+clean; 657 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.1 MiB/s wr, 453 op/s 2024-04-24T23:35:04.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:03 smithi033 ceph-mon[29201]: pgmap v1042: 129 pgs: 129 active+clean; 657 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.1 MiB/s wr, 453 op/s 2024-04-24T23:35:04.903 INFO:tasks.workunit.client.0.smithi033.stdout: 10 230025 22.03 MB/sec execute 429 sec latency 92.784 ms 2024-04-24T23:35:05.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 230323 22.01 MB/sec execute 430 sec latency 321.951 ms 2024-04-24T23:35:06.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:05 smithi038 ceph-mon[32450]: pgmap v1043: 129 pgs: 129 active+clean; 649 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 403 op/s 2024-04-24T23:35:06.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:05 smithi117 ceph-mon[32529]: pgmap v1043: 129 pgs: 129 active+clean; 649 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 403 op/s 2024-04-24T23:35:06.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:05 smithi033 ceph-mon[29201]: pgmap v1043: 129 pgs: 129 active+clean; 649 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 403 op/s 2024-04-24T23:35:06.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 230592 21.99 MB/sec execute 431 sec latency 94.825 ms 2024-04-24T23:35:07.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 231056 21.98 MB/sec execute 432 sec latency 80.665 ms 2024-04-24T23:35:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:07 smithi038 ceph-mon[32450]: pgmap v1044: 129 pgs: 129 active+clean; 654 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 486 op/s 2024-04-24T23:35:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:07 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:35:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:07 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:35:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:07 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:35:08.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:07 smithi117 ceph-mon[32529]: pgmap v1044: 129 pgs: 129 active+clean; 654 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 486 op/s 2024-04-24T23:35:08.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:07 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:35:08.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:07 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:35:08.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:07 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:35:08.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:07 smithi033 ceph-mon[29201]: pgmap v1044: 129 pgs: 129 active+clean; 654 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 486 op/s 2024-04-24T23:35:08.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:07 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:35:08.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:07 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:35:08.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:07 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:35:08.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 231493 22.00 MB/sec execute 433 sec latency 87.240 ms 2024-04-24T23:35:09.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 231893 22.00 MB/sec execute 434 sec latency 105.132 ms 2024-04-24T23:35:10.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:09 smithi038 ceph-mon[32450]: pgmap v1045: 129 pgs: 129 active+clean; 651 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 440 op/s 2024-04-24T23:35:10.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:09 smithi117 ceph-mon[32529]: pgmap v1045: 129 pgs: 129 active+clean; 651 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 440 op/s 2024-04-24T23:35:10.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:09 smithi033 ceph-mon[29201]: pgmap v1045: 129 pgs: 129 active+clean; 651 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 440 op/s 2024-04-24T23:35:10.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 232248 21.99 MB/sec execute 435 sec latency 381.928 ms 2024-04-24T23:35:11.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:11 smithi038 ceph-mon[32450]: pgmap v1046: 129 pgs: 129 active+clean; 654 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 421 op/s 2024-04-24T23:35:11.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:11 smithi117 ceph-mon[32529]: pgmap v1046: 129 pgs: 129 active+clean; 654 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 421 op/s 2024-04-24T23:35:11.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:11 smithi033 ceph-mon[29201]: pgmap v1046: 129 pgs: 129 active+clean; 654 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 421 op/s 2024-04-24T23:35:11.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 232640 21.98 MB/sec execute 436 sec latency 119.841 ms 2024-04-24T23:35:12.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 233100 21.98 MB/sec execute 437 sec latency 105.163 ms 2024-04-24T23:35:13.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:13 smithi038 ceph-mon[32450]: pgmap v1047: 129 pgs: 129 active+clean; 649 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 440 op/s 2024-04-24T23:35:13.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:13 smithi117 ceph-mon[32529]: pgmap v1047: 129 pgs: 129 active+clean; 649 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 440 op/s 2024-04-24T23:35:13.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:13 smithi033 ceph-mon[29201]: pgmap v1047: 129 pgs: 129 active+clean; 649 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 440 op/s 2024-04-24T23:35:13.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 233573 22.00 MB/sec execute 438 sec latency 93.793 ms 2024-04-24T23:35:14.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 233960 22.01 MB/sec execute 439 sec latency 58.221 ms 2024-04-24T23:35:15.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:15 smithi038 ceph-mon[32450]: pgmap v1048: 129 pgs: 129 active+clean; 645 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 353 op/s 2024-04-24T23:35:15.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:15 smithi117 ceph-mon[32529]: pgmap v1048: 129 pgs: 129 active+clean; 645 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 353 op/s 2024-04-24T23:35:15.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:15 smithi033 ceph-mon[29201]: pgmap v1048: 129 pgs: 129 active+clean; 645 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 353 op/s 2024-04-24T23:35:15.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 234262 21.99 MB/sec execute 440 sec latency 206.531 ms 2024-04-24T23:35:16.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 234601 21.98 MB/sec execute 441 sec latency 143.852 ms 2024-04-24T23:35:17.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:17 smithi038 ceph-mon[32450]: pgmap v1049: 129 pgs: 129 active+clean; 636 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 411 op/s 2024-04-24T23:35:17.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:17 smithi117 ceph-mon[32529]: pgmap v1049: 129 pgs: 129 active+clean; 636 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 411 op/s 2024-04-24T23:35:17.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:17 smithi033 ceph-mon[29201]: pgmap v1049: 129 pgs: 129 active+clean; 636 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 411 op/s 2024-04-24T23:35:17.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 235082 21.97 MB/sec execute 442 sec latency 84.418 ms 2024-04-24T23:35:18.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 235689 22.02 MB/sec execute 443 sec latency 68.915 ms 2024-04-24T23:35:19.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:19 smithi038 ceph-mon[32450]: pgmap v1050: 129 pgs: 129 active+clean; 637 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-04-24T23:35:19.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:19 smithi117 ceph-mon[32529]: pgmap v1050: 129 pgs: 129 active+clean; 637 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-04-24T23:35:19.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:19 smithi033 ceph-mon[29201]: pgmap v1050: 129 pgs: 129 active+clean; 637 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-04-24T23:35:19.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 236092 22.01 MB/sec execute 444 sec latency 119.170 ms 2024-04-24T23:35:20.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 236411 22.01 MB/sec execute 445 sec latency 215.489 ms 2024-04-24T23:35:21.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:21 smithi038 ceph-mon[32450]: pgmap v1051: 129 pgs: 129 active+clean; 644 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-04-24T23:35:21.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:21 smithi117 ceph-mon[32529]: pgmap v1051: 129 pgs: 129 active+clean; 644 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-04-24T23:35:21.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:21 smithi033 ceph-mon[29201]: pgmap v1051: 129 pgs: 129 active+clean; 644 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-04-24T23:35:21.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 236680 21.99 MB/sec execute 446 sec latency 440.863 ms 2024-04-24T23:35:22.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 237093 21.97 MB/sec execute 447 sec latency 107.024 ms 2024-04-24T23:35:23.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:23 smithi038 ceph-mon[32450]: pgmap v1052: 129 pgs: 129 active+clean; 649 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 421 op/s 2024-04-24T23:35:23.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:23 smithi117 ceph-mon[32529]: pgmap v1052: 129 pgs: 129 active+clean; 649 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 421 op/s 2024-04-24T23:35:23.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:23 smithi033 ceph-mon[29201]: pgmap v1052: 129 pgs: 129 active+clean; 649 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 421 op/s 2024-04-24T23:35:23.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 237485 21.98 MB/sec execute 448 sec latency 119.904 ms 2024-04-24T23:35:24.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 237927 21.98 MB/sec execute 449 sec latency 149.151 ms 2024-04-24T23:35:25.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:25 smithi038 ceph-mon[32450]: pgmap v1053: 129 pgs: 129 active+clean; 652 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.9 MiB/s wr, 378 op/s 2024-04-24T23:35:25.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:25 smithi117 ceph-mon[32529]: pgmap v1053: 129 pgs: 129 active+clean; 652 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.9 MiB/s wr, 378 op/s 2024-04-24T23:35:25.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:25 smithi033 ceph-mon[29201]: pgmap v1053: 129 pgs: 129 active+clean; 652 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.9 MiB/s wr, 378 op/s 2024-04-24T23:35:25.904 INFO:tasks.workunit.client.0.smithi033.stdout: 10 238421 22.00 MB/sec execute 450 sec latency 119.789 ms 2024-04-24T23:35:26.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:35:26.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:35:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.261+0000 7fc3108fd640 1 -- 172.21.15.33:0/1193664316 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080319b0 con 0x7fc30c100680 2024-04-24T23:35:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.261+0000 7fc313b63640 1 -- 172.21.15.33:0/1193664316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c100680 msgr2=0x7fc30c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.261+0000 7fc313b63640 1 --2- 172.21.15.33:0/1193664316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c100680 0x7fc30c100aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc308009900 tx=0x7fc30802a410 comp rx=0 tx=0).stop 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.261+0000 7fc313b63640 1 -- 172.21.15.33:0/1193664316 shutdown_connections 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.261+0000 7fc313b63640 1 --2- 172.21.15.33:0/1193664316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c102230 0x7fc30c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.262+0000 7fc313b63640 1 --2- 172.21.15.33:0/1193664316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c101870 0x7fc30c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.262+0000 7fc313b63640 1 --2- 172.21.15.33:0/1193664316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c100680 0x7fc30c100aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.262+0000 7fc313b63640 1 -- 172.21.15.33:0/1193664316 >> 172.21.15.33:0/1193664316 conn(0x7fc30c0fbc30 msgr2=0x7fc30c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.262+0000 7fc313b63640 1 -- 172.21.15.33:0/1193664316 shutdown_connections 2024-04-24T23:35:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.262+0000 7fc313b63640 1 -- 172.21.15.33:0/1193664316 wait complete. 2024-04-24T23:35:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.262+0000 7fc313b63640 1 Processor -- start 2024-04-24T23:35:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 -- start start 2024-04-24T23:35:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 0x7fc30c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c101870 0x7fc30c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c102230 0x7fc30c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc3118ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 0x7fc30c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c10d440 con 0x7fc30c100680 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc3118ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 0x7fc30c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42920/0 (socket says 172.21.15.33:42920) 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c10d2c0 con 0x7fc30c102230 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc3118ff640 1 -- 172.21.15.33:0/3329162352 learned_addr learned my addr 172.21.15.33:0/3329162352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c10d5c0 con 0x7fc30c101870 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc3110fe640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c101870 0x7fc30c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.263+0000 7fc312100640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c102230 0x7fc30c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc3118ff640 1 -- 172.21.15.33:0/3329162352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c101870 msgr2=0x7fc30c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc3118ff640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c101870 0x7fc30c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc3118ff640 1 -- 172.21.15.33:0/3329162352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c102230 msgr2=0x7fc30c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc3118ff640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c102230 0x7fc30c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc3118ff640 1 -- 172.21.15.33:0/3329162352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c19f710 con 0x7fc30c100680 2024-04-24T23:35:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc312100640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c102230 0x7fc30c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc3118ff640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 0x7fc30c197af0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc308005940 tx=0x7fc308032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc302ffd640 1 -- 172.21.15.33:0/3329162352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30803f030 con 0x7fc30c100680 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c19f9a0 con 0x7fc30c100680 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc302ffd640 1 -- 172.21.15.33:0/3329162352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc308003ce0 con 0x7fc30c100680 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc302ffd640 1 -- 172.21.15.33:0/3329162352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308035070 con 0x7fc30c100680 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.264+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c109660 con 0x7fc30c100680 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.267+0000 7fc302ffd640 1 -- 172.21.15.33:0/3329162352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc308042030 con 0x7fc30c100680 2024-04-24T23:35:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.267+0000 7fc316875740 1 Processor -- start 2024-04-24T23:35:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 -- start start 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e3c30 0x55b4880e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e8da0 0x55b4880eb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b4880476b0 con 0x55b4880e8da0 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b487fef880 con 0x55b4880e3240 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc316875740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b488043670 con 0x55b4880e3c30 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc3118ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e3c30 0x55b4880e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880e3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39546/0 (socket says 172.21.15.33:39546) 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 -- 172.21.15.33:0/3752821861 learned_addr learned my addr 172.21.15.33:0/3752821861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc3110fe640 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e8da0 0x55b4880eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc302ffd640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc2f80a6fe0 0x7fc2f80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 -- 172.21.15.33:0/3752821861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e3c30 msgr2=0x55b4880e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e3c30 0x55b4880e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 -- 172.21.15.33:0/3752821861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e8da0 msgr2=0x55b4880eb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.268+0000 7fc312100640 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e8da0 0x55b4880eb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc312100640 1 -- 172.21.15.33:0/3752821861 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b487e257d0 con 0x55b4880e3240 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc302ffd640 1 -- 172.21.15.33:0/3329162352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc3080ef2f0 con 0x7fc30c100680 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc312100640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc2f80a6fe0 0x7fc2f80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc312100640 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880e3660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc3040105d0 tx=0x7fc304010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4c76777abc3154 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc2f7fff640 1 -- 172.21.15.33:0/3752821861 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304019070 con 0x55b4880e3240 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc2f7fff640 1 -- 172.21.15.33:0/3752821861 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304022070 con 0x55b4880e3240 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.269+0000 7fc2f7fff640 1 -- 172.21.15.33:0/3752821861 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304011c40 con 0x55b4880e3240 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 -- 172.21.15.33:0/3752821861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 msgr2=0x55b4880e3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc312100640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc2f80a6fe0 0x7fc2f80a94a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc304021000 tx=0x7fc30401f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880e3660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc3040105d0 tx=0x7fc304010a90 comp rx=0 tx=0).stop 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 -- 172.21.15.33:0/3752821861 shutdown_connections 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e8da0 0x55b4880eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e3c30 0x55b4880e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 --2- 172.21.15.33:0/3752821861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880e3660 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 -- 172.21.15.33:0/3752821861 >> 172.21.15.33:0/3752821861 conn(0x55b4880de8b0 msgr2=0x55b4880decb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 -- 172.21.15.33:0/3752821861 shutdown_connections 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 -- 172.21.15.33:0/3752821861 wait complete. 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.270+0000 7fc316875740 1 -- start start 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 Processor -- start 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b4880ebd50 con 0x55b4880e3240 2024-04-24T23:35:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b488002f50 con 0x55b4880e3c30 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b4880ebed0 con 0x55b4880e8da0 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc316875740 10 client.0 _ll_register_callbacks cb 0x55b48803e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42930/0 (socket says 172.21.15.33:42930) 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 -- 172.21.15.33:0/1989051755 learned_addr learned my addr 172.21.15.33:0/1989051755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3110fe640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc312100640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 -- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 msgr2=0x55b4880f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 -- 172.21.15.33:0/1989051755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 msgr2=0x55b4880edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.271+0000 7fc3118ff640 1 -- 172.21.15.33:0/1989051755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b487d5b3f0 con 0x55b4880e3240 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc3118ff640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880ed690 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc3080bb4c0 tx=0x7fc3080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4234e6cc35676c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc2f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc2f57fa640 1 -- 172.21.15.33:0/1989051755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308049040 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc2f57fa640 1 -- 172.21.15.33:0/1989051755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc3080a60b0 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc304022070 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc2f57fa640 1 -- 172.21.15.33:0/1989051755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc3080bb300 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc316875740 10 client.20199 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b487d1b0f0 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc2f57fa640 1 -- 172.21.15.33:0/1989051755 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304022070 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.272+0000 7fc2f57fa640 1 -- 172.21.15.33:0/1989051755 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc3080cfce0 con 0x55b4880e3240 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc316875740 10 client.20199 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc316875740 10 client.20199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc316875740 20 client.20199 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc316875740 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b4880fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc316875740 4 client.20199 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b488054bf0 con 0x55b4880f9310 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc3110fe640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b4880fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc3110fe640 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b4880fb760 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45a66d284c2ca3f server_cookie=bf0046aada40bf8a in_seq=0 out_seq=0 2024-04-24T23:35:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.273+0000 7fc2f57fa640 10 client.20199 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc2f57fa640 1 -- 172.21.15.33:0/1989051755 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b488054bf0 con 0x55b4880f9310 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc2f57fa640 10 client.20199 handle_command_reply: tid=0 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 client.20199 shutdown 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 msgr2=0x55b4880ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880ed690 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc3080bb4c0 tx=0x7fc3080045e0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 shutdown_connections 2024-04-24T23:35:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b4880fb760 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc316875740 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.276+0000 7fc2f57fa640 0 client.20199 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.277+0000 7fc316875740 1 --2- 172.21.15.33:0/1989051755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880ed690 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.277+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 >> 172.21.15.33:0/1989051755 conn(0x55b4880de8b0 msgr2=0x55b4880ddaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.277+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 shutdown_connections 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.277+0000 7fc316875740 1 -- 172.21.15.33:0/1989051755 wait complete. 2024-04-24T23:35:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.277+0000 7fc316875740 20 client.20199 trim_cache size 0 max 16384 2024-04-24T23:35:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 Processor -- start 2024-04-24T23:35:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 -- start start 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3c30 0x55b4880f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1aaa30 con 0x55b4880e3c30 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1a8300 con 0x55b4880e3240 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc316875740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1a6150 con 0x55b4880e8da0 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc3110fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3c30 0x55b4880f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.301+0000 7fc3118ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3110fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3c30 0x55b4880f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42946/0 (socket says 172.21.15.33:42946) 2024-04-24T23:35:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3110fe640 1 -- 172.21.15.33:0/1333365699 learned_addr learned my addr 172.21.15.33:0/1333365699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc312100640 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3118ff640 1 -- 172.21.15.33:0/1333365699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 msgr2=0x55b4880edb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3118ff640 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880edb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3118ff640 1 -- 172.21.15.33:0/1333365699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3c30 msgr2=0x55b4880f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3118ff640 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3c30 0x55b4880f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3118ff640 1 -- 172.21.15.33:0/1333365699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4880d0090 con 0x55b4880e3240 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc3118ff640 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880dec90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd4090793eacb26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc2f77fe640 1 -- 172.21.15.33:0/1333365699 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080d8070 con 0x55b4880e3240 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc2f77fe640 1 -- 172.21.15.33:0/1333365699 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3080bbec0 con 0x55b4880e3240 2024-04-24T23:35:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.302+0000 7fc2f77fe640 1 -- 172.21.15.33:0/1333365699 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308049070 con 0x55b4880e3240 2024-04-24T23:35:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 -- 172.21.15.33:0/1333365699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 msgr2=0x55b4880dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880dec90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308032f90 comp rx=0 tx=0).stop 2024-04-24T23:35:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 -- 172.21.15.33:0/1333365699 shutdown_connections 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3c30 0x55b4880f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 --2- 172.21.15.33:0/1333365699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3240 0x55b4880dec90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 -- 172.21.15.33:0/1333365699 >> 172.21.15.33:0/1333365699 conn(0x55b4880de8b0 msgr2=0x55b4880dd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.303+0000 7fc316875740 1 -- 172.21.15.33:0/1333365699 shutdown_connections 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 -- 172.21.15.33:0/1333365699 wait complete. 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 -- start start 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 Processor -- start 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880f2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1a67c0 con 0x55b4880e3240 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1a7c30 con 0x55b4880e3c30 2024-04-24T23:35:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.304+0000 7fc316875740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1a6e90 con 0x55b4880e8da0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc316875740 10 client.0 _ll_register_callbacks cb 0x7fc3080b73a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39588/0 (socket says 172.21.15.33:39588) 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 -- 172.21.15.33:0/3809274152 learned_addr learned my addr 172.21.15.33:0/3809274152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3118ff640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc312100640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 -- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 msgr2=0x55b4880f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 -- 172.21.15.33:0/3809274152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 msgr2=0x55b4880f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 -- 172.21.15.33:0/3809274152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4880f3ef0 con 0x55b4880e3c30 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc3110fe640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880f2960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55b4880df820 tx=0x7fc2fc0207e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57ac41967d8d547 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc2f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:35:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.305+0000 7fc2f57fa640 1 -- 172.21.15.33:0/3809274152 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc011d80 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3080bbec0 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc2f57fa640 1 -- 172.21.15.33:0/3809274152 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc2fc005020 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc2f57fa640 1 -- 172.21.15.33:0/3809274152 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc031050 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 10 client.29678 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b48815f940 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc2f57fa640 1 -- 172.21.15.33:0/3809274152 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3080bbec0 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc2f57fa640 1 -- 172.21.15.33:0/3809274152 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc2fc01f480 con 0x55b4880e3c30 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 10 client.29678 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 10 client.29678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 20 client.29678 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b488162390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 4 client.29678 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.306+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b48804ef40 con 0x55b4880f9310 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.307+0000 7fc3118ff640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b488162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.307+0000 7fc2f57fa640 10 client.29678 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.307+0000 7fc3118ff640 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b488162390 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0ccec6c7845e7f server_cookie=59ec53586cf7c3eb in_seq=0 out_seq=0 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc2f57fa640 1 -- 172.21.15.33:0/3809274152 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b48804ef40 con 0x55b4880f9310 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc2f57fa640 10 client.29678 handle_command_reply: tid=0 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 client.29678 shutdown 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 msgr2=0x55b4880f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880f2960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55b4880df820 tx=0x7fc2fc0207e0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 shutdown_connections 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b4880f9310 0x55b488162390 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4880e8da0 0x55b4880f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4880e3c30 0x55b4880f2960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc2f57fa640 0 client.29678 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 --2- 172.21.15.33:0/3809274152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b4880e3240 0x55b4880f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.308+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 >> 172.21.15.33:0/3809274152 conn(0x55b4880de8b0 msgr2=0x55b4880ddaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 shutdown_connections 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc316875740 1 -- 172.21.15.33:0/3809274152 wait complete. 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc316875740 20 client.29678 trim_cache size 0 max 16384 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:35:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:35:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc2f80a6fe0 msgr2=0x7fc2f80a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc313b63640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc2f80a6fe0 0x7fc2f80a94a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc304021000 tx=0x7fc30401f000 comp rx=0 tx=0).stop 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 msgr2=0x7fc30c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.309+0000 7fc313b63640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 0x7fc30c197af0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc308005940 tx=0x7fc308032ce0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 shutdown_connections 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc2f80a6fe0 0x7fc2f80a94a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30c102230 0x7fc30c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c101870 0x7fc30c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 --2- 172.21.15.33:0/3329162352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c100680 0x7fc30c197af0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 >> 172.21.15.33:0/3329162352 conn(0x7fc30c0fbc30 msgr2=0x7fc30c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 shutdown_connections 2024-04-24T23:35:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.310+0000 7fc313b63640 1 -- 172.21.15.33:0/3329162352 wait complete. 2024-04-24T23:35:26.323 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:35:26.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 -- 172.21.15.33:0/1033740221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 msgr2=0x7f0ac0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 --2- 172.21.15.33:0/1033740221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac0108ac0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009ca0 tx=0x7f0ab802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 -- 172.21.15.33:0/1033740221 shutdown_connections 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 --2- 172.21.15.33:0/1033740221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac0108ac0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 --2- 172.21.15.33:0/1033740221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0101870 0x7f0ac0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 --2- 172.21.15.33:0/1033740221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ac0100680 0x7f0ac0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 -- 172.21.15.33:0/1033740221 >> 172.21.15.33:0/1033740221 conn(0x7f0ac00fbbf0 msgr2=0x7f0ac00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 -- 172.21.15.33:0/1033740221 shutdown_connections 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.448+0000 7f0ac6876640 1 -- 172.21.15.33:0/1033740221 wait complete. 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 Processor -- start 2024-04-24T23:35:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 -- start start 2024-04-24T23:35:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0100680 0x7f0ac010cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ac0101870 0x7f0ac010b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac010baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0069830 con 0x7f0ac0102230 2024-04-24T23:35:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac00696b0 con 0x7f0ac0101870 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac6876640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010d5b0 con 0x7f0ac0100680 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac010baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac010baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42968/0 (socket says 172.21.15.33:42968) 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 -- 172.21.15.33:0/426525974 learned_addr learned my addr 172.21.15.33:0/426525974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0abffff640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0100680 0x7f0ac010cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0abf7fe640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ac0101870 0x7f0ac010b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 -- 172.21.15.33:0/426525974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0100680 msgr2=0x7f0ac010cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0100680 0x7f0ac010cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 -- 172.21.15.33:0/426525974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ac0101870 msgr2=0x7f0ac010b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ac0101870 0x7f0ac010b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0ac4e13640 1 -- 172.21.15.33:0/426525974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac010c330 con 0x7f0ac0102230 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.449+0000 7f0abffff640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0100680 0x7f0ac010cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.450+0000 7f0ac4e13640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac010baa0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8031ae0 tx=0x7f0ab8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.450+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003b70 con 0x7f0ac0102230 2024-04-24T23:35:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.450+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac01a1b40 con 0x7f0ac0102230 2024-04-24T23:35:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.450+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01a20e0 con 0x7f0ac0102230 2024-04-24T23:35:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.450+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab8003d10 con 0x7f0ac0102230 2024-04-24T23:35:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.450+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab802fc50 con 0x7f0ac0102230 2024-04-24T23:35:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.451+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac01079a0 con 0x7f0ac0102230 2024-04-24T23:35:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.453+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0ab8004b40 con 0x7f0ac0102230 2024-04-24T23:35:26.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:26 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:35:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.454+0000 7f0abd7fa640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0aa80a2c10 0x7f0aa80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.454+0000 7f0abffff640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0aa80a2c10 0x7f0aa80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.454+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ab80f04f0 con 0x7f0ac0102230 2024-04-24T23:35:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.455+0000 7f0abffff640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0aa80a2c10 0x7f0aa80a50d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0ac01016d0 tx=0x7f0ab0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:26.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.458+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0ab80bd060 con 0x7f0ac0102230 2024-04-24T23:35:26.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:26 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:35:26.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:26 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:35:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.774+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac002d170 con 0x7f0ac0102230 2024-04-24T23:35:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.776+0000 7f0abd7fa640 1 -- 172.21.15.33:0/426525974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11859 (secure 0 0 0) 0x7f0ab80bc960 con 0x7f0ac0102230 2024-04-24T23:35:26.776 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:35:26.776 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:28:22.221595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24479,24481,24508]},"id":1}]} 2024-04-24T23:35:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-24T23:35:26.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.780+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0aa80a2c10 msgr2=0x7f0aa80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.780+0000 7f0ac6876640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0aa80a2c10 0x7f0aa80a50d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0ac01016d0 tx=0x7f0ab0005f90 comp rx=0 tx=0).stop 2024-04-24T23:35:26.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.780+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 msgr2=0x7f0ac010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.780+0000 7f0ac6876640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac010baa0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8031ae0 tx=0x7f0ab8031ee0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.785+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 shutdown_connections 2024-04-24T23:35:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.785+0000 7f0ac6876640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0aa80a2c10 0x7f0aa80a50d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.785+0000 7f0ac6876640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0102230 0x7f0ac010baa0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.785+0000 7f0ac6876640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ac0101870 0x7f0ac010b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.785+0000 7f0ac6876640 1 --2- 172.21.15.33:0/426525974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ac0100680 0x7f0ac010cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.785+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 >> 172.21.15.33:0/426525974 conn(0x7f0ac00fbbf0 msgr2=0x7f0ac0104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.788+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 shutdown_connections 2024-04-24T23:35:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.789+0000 7f0ac6876640 1 -- 172.21.15.33:0/426525974 wait complete. 2024-04-24T23:35:26.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:35:26.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 238695 21.98 MB/sec execute 451 sec latency 245.043 ms 2024-04-24T23:35:26.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.987+0000 7fc7984bd640 1 -- 172.21.15.33:0/2679034946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 msgr2=0x7fc790100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:26.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.987+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2679034946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc790100bb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc78c01a910 tx=0x7fc78c03b450 comp rx=0 tx=0).stop 2024-04-24T23:35:26.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.991+0000 7fc7984bd640 1 -- 172.21.15.33:0/2679034946 shutdown_connections 2024-04-24T23:35:26.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.991+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2679034946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7901010f0 0x7fc7901059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.991+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2679034946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc790100bb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.991+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2679034946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ff620 0x7fc7900ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:26.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.991+0000 7fc7984bd640 1 -- 172.21.15.33:0/2679034946 >> 172.21.15.33:0/2679034946 conn(0x7fc7900faf40 msgr2=0x7fc7900fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:26.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.994+0000 7fc7984bd640 1 -- 172.21.15.33:0/2679034946 shutdown_connections 2024-04-24T23:35:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.996+0000 7fc7984bd640 1 -- 172.21.15.33:0/2679034946 wait complete. 2024-04-24T23:35:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 Processor -- start 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 -- start start 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ff620 0x7fc790071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc7900723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7901010f0 0x7fc7900728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc790109300 con 0x7fc7900ff620 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc790109180 con 0x7fc7901010f0 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc7984bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc790109480 con 0x7fc790100750 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc795a58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc7900723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc795a58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc7900723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41430/0 (socket says 172.21.15.33:41430) 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc795a58640 1 -- 172.21.15.33:0/2404261255 learned_addr learned my addr 172.21.15.33:0/2404261255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.997+0000 7fc796a5a640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7901010f0 0x7fc7900728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.998+0000 7fc795a58640 1 -- 172.21.15.33:0/2404261255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7901010f0 msgr2=0x7fc7900728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.998+0000 7fc795a58640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7901010f0 0x7fc7900728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.998+0000 7fc795a58640 1 -- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ff620 msgr2=0x7fc790071e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:35:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.998+0000 7fc795a58640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ff620 0x7fc790071e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:26.998+0000 7fc795a58640 1 -- 172.21.15.33:0/2404261255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc790194ce0 con 0x7fc790100750 2024-04-24T23:35:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.000+0000 7fc795a58640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc7900723a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc78c042a70 tx=0x7fc78c042770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.000+0000 7fc7877fe640 1 -- 172.21.15.33:0/2404261255 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c050030 con 0x7fc790100750 2024-04-24T23:35:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.001+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc790194f70 con 0x7fc790100750 2024-04-24T23:35:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.001+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901954b0 con 0x7fc790100750 2024-04-24T23:35:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.001+0000 7fc7877fe640 1 -- 172.21.15.33:0/2404261255 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78c01a530 con 0x7fc790100750 2024-04-24T23:35:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.001+0000 7fc7877fe640 1 -- 172.21.15.33:0/2404261255 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c040880 con 0x7fc790100750 2024-04-24T23:35:27.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.004+0000 7fc7877fe640 1 -- 172.21.15.33:0/2404261255 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc78c053030 con 0x7fc790100750 2024-04-24T23:35:27.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.005+0000 7fc7877fe640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc77c0a2ce0 0x7fc77c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.005+0000 7fc7877fe640 1 -- 172.21.15.33:0/2404261255 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc78c100590 con 0x7fc790100750 2024-04-24T23:35:27.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.006+0000 7fc796259640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc77c0a2ce0 0x7fc77c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.010+0000 7fc796259640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc77c0a2ce0 0x7fc77c0a51a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc790073590 tx=0x7fc780005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.015+0000 7fc79b1d1740 1 Processor -- start 2024-04-24T23:35:27.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 -- start start 2024-04-24T23:35:27.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 0x55cb7af9f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afa4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb7af03670 con 0x55cb7af9f230 2024-04-24T23:35:27.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cb7aeab970 con 0x55cb7af9fc20 2024-04-24T23:35:27.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.016+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cb7aeff630 con 0x55cb7afa4d90 2024-04-24T23:35:27.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.017+0000 7fc796259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 0x55cb7af9f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.017+0000 7fc796259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 0x55cb7af9f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42984/0 (socket says 172.21.15.33:42984) 2024-04-24T23:35:27.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.017+0000 7fc796259640 1 -- 172.21.15.33:0/1527356678 learned_addr learned my addr 172.21.15.33:0/1527356678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.019+0000 7fc796259640 1 -- 172.21.15.33:0/1527356678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 msgr2=0x55cb7afa7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.019+0000 7fc796259640 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.019+0000 7fc796259640 1 -- 172.21.15.33:0/1527356678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 msgr2=0x55cb7afa4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.019+0000 7fc796259640 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afa4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.019+0000 7fc796259640 1 -- 172.21.15.33:0/1527356678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7aecf400 con 0x55cb7af9f230 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.020+0000 7fc796259640 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 0x55cb7af9f650 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc78000ed30 tx=0x7fc780011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc17baea2385277a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.021+0000 7fc784bfe640 1 -- 172.21.15.33:0/1527356678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780020070 con 0x55cb7af9f230 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.021+0000 7fc784bfe640 1 -- 172.21.15.33:0/1527356678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780010e40 con 0x55cb7af9f230 2024-04-24T23:35:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.021+0000 7fc784bfe640 1 -- 172.21.15.33:0/1527356678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78001c8c0 con 0x55cb7af9f230 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.023+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1527356678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 msgr2=0x55cb7af9f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.023+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 0x55cb7af9f650 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc78000ed30 tx=0x7fc780011000 comp rx=0 tx=0).stop 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.024+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1527356678 shutdown_connections 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.024+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.024+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afa4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.024+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1527356678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9f230 0x55cb7af9f650 secure :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc78000ed30 tx=0x7fc780011000 comp rx=0 tx=0).stop 2024-04-24T23:35:27.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.024+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1527356678 >> 172.21.15.33:0/1527356678 conn(0x55cb7af9a8a0 msgr2=0x55cb7af9aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.029+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1527356678 shutdown_connections 2024-04-24T23:35:27.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.032+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1527356678 wait complete. 2024-04-24T23:35:27.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.032+0000 7fc79b1d1740 1 -- start start 2024-04-24T23:35:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.032+0000 7fc79b1d1740 1 Processor -- start 2024-04-24T23:35:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.033+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7afa96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.033+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afa4d90 0x55cb7afa9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.033+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afb1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb7afa7d40 con 0x55cb7af9fc20 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cb7aebf050 con 0x55cb7afa4d90 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cb7afa7ec0 con 0x55cb7afaec30 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc79b1d1740 10 client.0 _ll_register_callbacks cb 0x55cb7ac16be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc796259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7afa96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc796259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7afa96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42994/0 (socket says 172.21.15.33:42994) 2024-04-24T23:35:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.034+0000 7fc796259640 1 -- 172.21.15.33:0/543944055 learned_addr learned my addr 172.21.15.33:0/543944055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.035+0000 7fc796259640 1 -- 172.21.15.33:0/543944055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 msgr2=0x55cb7afb1020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.035+0000 7fc796259640 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afb1020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.035+0000 7fc796259640 1 -- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afa4d90 msgr2=0x55cb7afa9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.035+0000 7fc796259640 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afa4d90 0x55cb7afa9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.035+0000 7fc796259640 1 -- 172.21.15.33:0/543944055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7ae90dc0 con 0x55cb7af9fc20 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.036+0000 7fc796259640 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7afa96c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e720 tx=0x7fc780025c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a03af8fe7e7cbf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.036+0000 7fc779ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.036+0000 7fc779ffb640 1 -- 172.21.15.33:0/543944055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780020070 con 0x55cb7af9fc20 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.036+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cb7af10bb0 con 0x55cb7af9fc20 2024-04-24T23:35:27.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.037+0000 7fc779ffb640 1 -- 172.21.15.33:0/543944055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc78001c860 con 0x55cb7af9fc20 2024-04-24T23:35:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.043+0000 7fc779ffb640 1 -- 172.21.15.33:0/543944055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc78001ca60 con 0x55cb7af9fc20 2024-04-24T23:35:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.044+0000 7fc79b1d1740 10 client.20220 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.044+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cb7ac173f0 con 0x55cb7af9fc20 2024-04-24T23:35:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc779ffb640 1 -- 172.21.15.33:0/543944055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cb7af10bb0 con 0x55cb7af9fc20 2024-04-24T23:35:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc779ffb640 1 -- 172.21.15.33:0/543944055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc78002ebf0 con 0x55cb7af9fc20 2024-04-24T23:35:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc79b1d1740 10 client.20220 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc79b1d1740 10 client.20220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:35:27.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc79b1d1740 20 client.20220 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7afb5c10 0x55cb7afb8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc79b1d1740 4 client.20220 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.051+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb7af0af00 con 0x55cb7afb5c10 2024-04-24T23:35:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.054+0000 7fc795a58640 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7afb5c10 0x55cb7afb8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.067+0000 7fc779ffb640 10 client.20220 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.066+0000 7fc795a58640 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7afb5c10 0x55cb7afb8070 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25b1fec97ad56cb6 server_cookie=d011a2ca2b69d24e in_seq=0 out_seq=0 2024-04-24T23:35:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.078+0000 7fc779ffb640 1 -- 172.21.15.33:0/543944055 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb7af0af00 con 0x55cb7afb5c10 2024-04-24T23:35:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.078+0000 7fc779ffb640 10 client.20220 handle_command_reply: tid=0 2024-04-24T23:35:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.078+0000 7fc79b1d1740 1 client.20220 shutdown 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 msgr2=0x55cb7afa96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7afa96c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e720 tx=0x7fc780025c40 comp rx=0 tx=0).stop 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 shutdown_connections 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7afb5c10 0x55cb7afb8070 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afb1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afa4d90 0x55cb7afa9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/543944055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7afa96c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 >> 172.21.15.33:0/543944055 conn(0x55cb7af9a8a0 msgr2=0x55cb7af99b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc779ffb640 0 client.20220 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 shutdown_connections 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 1 -- 172.21.15.33:0/543944055 wait complete. 2024-04-24T23:35:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.082+0000 7fc79b1d1740 20 client.20220 trim_cache size 0 max 16384 2024-04-24T23:35:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.115+0000 7fc79b1d1740 1 Processor -- start 2024-04-24T23:35:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 -- start start 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7af9e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afaec30 0x55cb7afa92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7740125b0 con 0x55cb7af9fc20 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc77400fe80 con 0x55cb7afaec30 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc77400dcd0 con 0x55cb7afa4d90 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc796259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7af9e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc796259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7af9e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43010/0 (socket says 172.21.15.33:43010) 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc795a58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.119+0000 7fc796259640 1 -- 172.21.15.33:0/287548011 learned_addr learned my addr 172.21.15.33:0/287548011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.121+0000 7fc795a58640 1 -- 172.21.15.33:0/287548011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afaec30 msgr2=0x55cb7afa92d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:35:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.121+0000 7fc795a58640 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afaec30 0x55cb7afa92d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.121+0000 7fc795a58640 1 -- 172.21.15.33:0/287548011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 msgr2=0x55cb7af9e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.121+0000 7fc795a58640 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7af9e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.121+0000 7fc795a58640 1 -- 172.21.15.33:0/287548011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7ace17d0 con 0x55cb7afa4d90 2024-04-24T23:35:27.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.124+0000 7fc795a58640 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55cb7afa0690 tx=0x7fc78c0b5f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344b75fcbb314b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.126+0000 7fc77bfff640 1 -- 172.21.15.33:0/287548011 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c05a040 con 0x55cb7afa4d90 2024-04-24T23:35:27.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.126+0000 7fc77bfff640 1 -- 172.21.15.33:0/287548011 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78c0d0db0 con 0x55cb7afa4d90 2024-04-24T23:35:27.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.126+0000 7fc77bfff640 1 -- 172.21.15.33:0/287548011 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c100240 con 0x55cb7afa4d90 2024-04-24T23:35:27.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.127+0000 7fc79b1d1740 1 -- 172.21.15.33:0/287548011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 msgr2=0x55cb7afa7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.127+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55cb7afa0690 tx=0x7fc78c0b5f90 comp rx=0 tx=0).stop 2024-04-24T23:35:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.129+0000 7fc79b1d1740 1 -- 172.21.15.33:0/287548011 shutdown_connections 2024-04-24T23:35:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.130+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7afaec30 0x55cb7afa92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.130+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afa4d90 0x55cb7afa7c20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.130+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/287548011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7af9fc20 0x55cb7af9e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.130+0000 7fc79b1d1740 1 -- 172.21.15.33:0/287548011 >> 172.21.15.33:0/287548011 conn(0x55cb7af9a8a0 msgr2=0x55cb7af9ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.135+0000 7fc79b1d1740 1 -- 172.21.15.33:0/287548011 shutdown_connections 2024-04-24T23:35:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.137+0000 7fc79b1d1740 1 -- 172.21.15.33:0/287548011 wait complete. 2024-04-24T23:35:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.137+0000 7fc79b1d1740 1 -- start start 2024-04-24T23:35:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.137+0000 7fc79b1d1740 1 Processor -- start 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.138+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afc65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.138+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7afa4d90 0x55cb7afc6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.138+0000 7fc79b1d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afc7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.138+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc77400e340 con 0x55cb7afa4d90 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.139+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc77400f7b0 con 0x55cb7af9fc20 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.139+0000 7fc79b1d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc77400ea10 con 0x55cb7afaec30 2024-04-24T23:35:27.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.139+0000 7fc79b1d1740 10 client.0 _ll_register_callbacks cb 0x7fc78000ec30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.140+0000 7fc795a58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7afa4d90 0x55cb7afc6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.140+0000 7fc795a58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7afa4d90 0x55cb7afc6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43018/0 (socket says 172.21.15.33:43018) 2024-04-24T23:35:27.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.140+0000 7fc795a58640 1 -- 172.21.15.33:0/1675032662 learned_addr learned my addr 172.21.15.33:0/1675032662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.140+0000 7fc796a5a640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afc7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.141+0000 7fc796259640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afc65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.145+0000 7fc796259640 1 -- 172.21.15.33:0/1675032662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 msgr2=0x55cb7afc7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.145+0000 7fc796259640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afc7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.145+0000 7fc796259640 1 -- 172.21.15.33:0/1675032662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7afa4d90 msgr2=0x55cb7afc6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.145+0000 7fc796259640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7afa4d90 0x55cb7afc6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.145+0000 7fc796259640 1 -- 172.21.15.33:0/1675032662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7afc80d0 con 0x55cb7af9fc20 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.146+0000 7fc796259640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afc65c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc780025810 tx=0x7fc780010d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf07e0db405a2317 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.146+0000 7fc779ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.146+0000 7fc779ffb640 1 -- 172.21.15.33:0/1675032662 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780041030 con 0x55cb7af9fc20 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.146+0000 7fc779ffb640 1 -- 172.21.15.33:0/1675032662 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc780009040 con 0x55cb7af9fc20 2024-04-24T23:35:27.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.146+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78c0d0db0 con 0x55cb7af9fc20 2024-04-24T23:35:27.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.147+0000 7fc779ffb640 1 -- 172.21.15.33:0/1675032662 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc78004b050 con 0x55cb7af9fc20 2024-04-24T23:35:27.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.147+0000 7fc79b1d1740 10 client.29690 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:27.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.147+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cb7afc93d0 con 0x55cb7af9fc20 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc779ffb640 1 -- 172.21.15.33:0/1675032662 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78002dd80 con 0x55cb7af9fc20 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc779ffb640 1 -- 172.21.15.33:0/1675032662 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc780004680 con 0x55cb7af9fc20 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc79b1d1740 10 client.29690 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc79b1d1740 10 client.29690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc79b1d1740 20 client.29690 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7b01be70 0x55cb7b01e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.148+0000 7fc79b1d1740 4 client.29690 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:35:27.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.149+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc78c0d0db0 con 0x55cb7b01be70 2024-04-24T23:35:27.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.154+0000 7fc795a58640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7b01be70 0x55cb7b01e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.155+0000 7fc795a58640 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7b01be70 0x55cb7b01e260 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4d5c9242bbd5dae server_cookie=1786e1c1d283e1dd in_seq=0 out_seq=0 2024-04-24T23:35:27.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.155+0000 7fc779ffb640 10 client.29690 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.164+0000 7fc779ffb640 1 -- 172.21.15.33:0/1675032662 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc78c0d0db0 con 0x55cb7b01be70 2024-04-24T23:35:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.164+0000 7fc779ffb640 10 client.29690 handle_command_reply: tid=0 2024-04-24T23:35:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.164+0000 7fc79b1d1740 1 client.29690 shutdown 2024-04-24T23:35:27.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.169+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 msgr2=0x55cb7afc65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.169+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afc65c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc780025810 tx=0x7fc780010d10 comp rx=0 tx=0).stop 2024-04-24T23:35:27.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.170+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 shutdown_connections 2024-04-24T23:35:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.170+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55cb7b01be70 0x55cb7b01e260 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.170+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb7afaec30 0x55cb7afc7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.170+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb7afa4d90 0x55cb7afc6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.170+0000 7fc79b1d1740 1 --2- 172.21.15.33:0/1675032662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb7af9fc20 0x55cb7afc65c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.170+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 >> 172.21.15.33:0/1675032662 conn(0x55cb7af9a8a0 msgr2=0x55cb7afa4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.171+0000 7fc779ffb640 0 client.29690 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:27.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.172+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 shutdown_connections 2024-04-24T23:35:27.172 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:35:27.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.172+0000 7fc79b1d1740 1 -- 172.21.15.33:0/1675032662 wait complete. 2024-04-24T23:35:27.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.172+0000 7fc79b1d1740 20 client.29690 trim_cache size 0 max 16384 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc77c0a2ce0 msgr2=0x7fc77c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc77c0a2ce0 0x7fc77c0a51a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc790073590 tx=0x7fc780005f90 comp rx=0 tx=0).stop 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 msgr2=0x7fc7900723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc7900723a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc78c042a70 tx=0x7fc78c042770 comp rx=0 tx=0).stop 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 shutdown_connections 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fc77c0a2ce0 0x7fc77c0a51a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7901010f0 0x7fc7900728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc790100750 0x7fc7900723a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 --2- 172.21.15.33:0/2404261255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7900ff620 0x7fc790071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.173+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 >> 172.21.15.33:0/2404261255 conn(0x7fc7900faf40 msgr2=0x7fc790103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.174+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 shutdown_connections 2024-04-24T23:35:27.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.174+0000 7fc7984bd640 1 -- 172.21.15.33:0/2404261255 wait complete. 2024-04-24T23:35:27.194 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:35:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 -- 172.21.15.33:0/3143528913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 msgr2=0x7f0444105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 --2- 172.21.15.33:0/3143528913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f0444105030 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0438009900 tx=0x7f043802a410 comp rx=0 tx=0).stop 2024-04-24T23:35:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 -- 172.21.15.33:0/3143528913 shutdown_connections 2024-04-24T23:35:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 --2- 172.21.15.33:0/3143528913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0444105570 0x7f044410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 --2- 172.21.15.33:0/3143528913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f0444105030 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 --2- 172.21.15.33:0/3143528913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04441039c0 0x7f0444103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 -- 172.21.15.33:0/3143528913 >> 172.21.15.33:0/3143528913 conn(0x7f04440fef30 msgr2=0x7f0444101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.351+0000 7f0449888640 1 -- 172.21.15.33:0/3143528913 shutdown_connections 2024-04-24T23:35:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.352+0000 7f0449888640 1 -- 172.21.15.33:0/3143528913 wait complete. 2024-04-24T23:35:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.352+0000 7f0449888640 1 Processor -- start 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.352+0000 7f0449888640 1 -- start start 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0449888640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04441039c0 0x7f0444199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0449888640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f044419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0449888640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 0x7f04441a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0449888640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f044410f640 con 0x7f0444104bb0 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0449888640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f044410f4c0 con 0x7f04441039c0 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0449888640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f044410f7c0 con 0x7f0444105570 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 0x7f04441a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04441039c0 0x7f0444199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04441039c0 0x7f0444199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39670/0 (socket says 172.21.15.33:39670) 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f0442ffd640 1 -- 172.21.15.33:0/1868206858 learned_addr learned my addr 172.21.15.33:0/1868206858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 0x7f04441a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41482/0 (socket says 172.21.15.33:41482) 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04427fc640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f044419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04427fc640 1 -- 172.21.15.33:0/1868206858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 msgr2=0x7f04441a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04427fc640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 0x7f04441a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04427fc640 1 -- 172.21.15.33:0/1868206858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04441039c0 msgr2=0x7f0444199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04427fc640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04441039c0 0x7f0444199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04427fc640 1 -- 172.21.15.33:0/1868206858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04441a1920 con 0x7f0444104bb0 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.353+0000 7f04437fe640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 0x7f04441a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.354+0000 7f04427fc640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f044419a250 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0438031440 tx=0x7f0438004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.354+0000 7f0448886640 1 -- 172.21.15.33:0/1868206858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043803f070 con 0x7f0444104bb0 2024-04-24T23:35:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.354+0000 7f0448886640 1 -- 172.21.15.33:0/1868206858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0438003cd0 con 0x7f0444104bb0 2024-04-24T23:35:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.354+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04441a1bb0 con 0x7f0444104bb0 2024-04-24T23:35:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.354+0000 7f0448886640 1 -- 172.21.15.33:0/1868206858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438035070 con 0x7f0444104bb0 2024-04-24T23:35:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.354+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04441a20c0 con 0x7f0444104bb0 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 Processor -- start 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 -- start start 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f0448886640 1 -- 172.21.15.33:0/1868206858 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f043803b4f0 con 0x7f0444104bb0 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e5f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 0x561101e64720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e67220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561101dc3670 con 0x561101e64d90 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561101d6b970 con 0x561101e5fc20 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f044c590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561101dbf630 con 0x561101e5f230 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 0x561101e64720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f04437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e5f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 0x561101e64720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39672/0 (socket says 172.21.15.33:39672) 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f0442ffd640 1 -- 172.21.15.33:0/2963548732 learned_addr learned my addr 172.21.15.33:0/2963548732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.357+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e67220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0442ffd640 1 -- 172.21.15.33:0/2963548732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 msgr2=0x561101e5f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0442ffd640 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e5f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0442ffd640 1 -- 172.21.15.33:0/2963548732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 msgr2=0x561101e67220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0442ffd640 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e67220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0442ffd640 1 -- 172.21.15.33:0/2963548732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561101d8f400 con 0x561101e5fc20 2024-04-24T23:35:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f04437fe640 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e5f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0448886640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f042c0a2c10 0x7f042c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0442ffd640 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 0x561101e64720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0434009d20 tx=0x7f0434011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8376a4bdb69ae9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f04437fe640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f042c0a2c10 0x7f042c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0448886640 1 -- 172.21.15.33:0/1868206858 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04380ef7a0 con 0x7f0444104bb0 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0429ffb640 1 -- 172.21.15.33:0/2963548732 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434004220 con 0x561101e5fc20 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.358+0000 7f0429ffb640 1 -- 172.21.15.33:0/2963548732 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04340043c0 con 0x561101e5fc20 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f0429ffb640 1 -- 172.21.15.33:0/2963548732 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434019900 con 0x561101e5fc20 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f044c590740 1 -- 172.21.15.33:0/2963548732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 msgr2=0x561101e64720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f04437fe640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f042c0a2c10 0x7f042c0a50d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x561101e60480 tx=0x7f043c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f044c590740 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 0x561101e64720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0434009d20 tx=0x7f0434011810 comp rx=0 tx=0).stop 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f044c590740 1 -- 172.21.15.33:0/2963548732 shutdown_connections 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f044c590740 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f044c590740 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5fc20 0x561101e64720 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.359+0000 7f044c590740 1 --2- 172.21.15.33:0/2963548732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e5f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.360+0000 7f044c590740 1 -- 172.21.15.33:0/2963548732 >> 172.21.15.33:0/2963548732 conn(0x561101e5a8a0 msgr2=0x561101e5aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.360+0000 7f044c590740 1 -- 172.21.15.33:0/2963548732 shutdown_connections 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.360+0000 7f044c590740 1 -- 172.21.15.33:0/2963548732 wait complete. 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.360+0000 7f044c590740 1 -- start start 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.360+0000 7f044c590740 1 Processor -- start 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 0x561101e69870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e64d90 0x561101e708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561101e67d90 con 0x561101e5fc20 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561101d7f050 con 0x561101e5f230 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561101e67f10 con 0x561101e64d90 2024-04-24T23:35:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f044c590740 10 client.0 _ll_register_callbacks cb 0x561101ad6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 0x561101e69870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e69330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39674/0 (socket says 172.21.15.33:39674) 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f0442ffd640 1 -- 172.21.15.33:0/2564750295 learned_addr learned my addr 172.21.15.33:0/2564750295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 0x561101e69870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43068/0 (socket says 172.21.15.33:43068) 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04437fe640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e64d90 0x561101e708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 -- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e64d90 msgr2=0x561101e708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e64d90 0x561101e708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 -- 172.21.15.33:0/2564750295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 msgr2=0x561101e69330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e69330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 -- 172.21.15.33:0/2564750295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561101d50dc0 con 0x561101e5fc20 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04437fe640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e64d90 0x561101e708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f0442ffd640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f04427fc640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 0x561101e69870 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f04380ecf60 tx=0x7f0438003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24820b1d1947b621 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.361+0000 7f0422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f0422ffd640 1 -- 172.21.15.33:0/2564750295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04380efe60 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f0422ffd640 1 -- 172.21.15.33:0/2564750295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04380bbf80 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04340043c0 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f0422ffd640 1 -- 172.21.15.33:0/2564750295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0438048070 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f044c590740 10 client.20238 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561101ad73f0 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f0422ffd640 1 -- 172.21.15.33:0/2564750295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04340043c0 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.362+0000 7f0422ffd640 1 -- 172.21.15.33:0/2564750295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f043804b060 con 0x561101e5fc20 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f044c590740 10 client.20238 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f044c590740 10 client.20238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f044c590740 20 client.20238 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f044c590740 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101e75020 0x561101e77470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f044c590740 4 client.20238 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:27.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561101dd0bb0 con 0x561101e75020 2024-04-24T23:35:27.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f0442ffd640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101e75020 0x561101e77470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f0442ffd640 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101e75020 0x561101e77470 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22c753a60d6cc6f5 server_cookie=1d25d40a48710a16 in_seq=0 out_seq=0 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.363+0000 7f0422ffd640 10 client.20238 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.364+0000 7f0422ffd640 1 -- 172.21.15.33:0/2564750295 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561101dd0bb0 con 0x561101e75020 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.364+0000 7f0422ffd640 10 client.20238 handle_command_reply: tid=0 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.364+0000 7f044c590740 1 client.20238 shutdown 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 msgr2=0x561101e69870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 0x561101e69870 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f04380ecf60 tx=0x7f0438003f40 comp rx=0 tx=0).stop 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 shutdown_connections 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101e75020 0x561101e77470 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f0422ffd640 0 client.20238 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e64d90 0x561101e708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e5fc20 0x561101e69870 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 --2- 172.21.15.33:0/2564750295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 >> 172.21.15.33:0/2564750295 conn(0x561101e5a8a0 msgr2=0x561101e59b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 shutdown_connections 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 1 -- 172.21.15.33:0/2564750295 wait complete. 2024-04-24T23:35:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.365+0000 7f044c590740 20 client.20238 trim_cache size 0 max 16384 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 Processor -- start 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 -- start start 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e75020 0x561101e604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04441aee40 con 0x561101e64d90 2024-04-24T23:35:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04441ac710 con 0x561101e75020 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f044c590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04441aa560 con 0x561101e5f230 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e75020 0x561101e604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43078/0 (socket says 172.21.15.33:43078) 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 -- 172.21.15.33:0/1008564808 learned_addr learned my addr 172.21.15.33:0/1008564808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 -- 172.21.15.33:0/1008564808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 msgr2=0x561101e599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 -- 172.21.15.33:0/1008564808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e75020 msgr2=0x561101e604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e75020 0x561101e604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 -- 172.21.15.33:0/1008564808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561101ba17d0 con 0x561101e64d90 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.380+0000 7f04427fc640 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e707d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f04380314e0 tx=0x7f043803b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c2a256e358dde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f04297fa640 1 -- 172.21.15.33:0/1008564808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04380b9e60 con 0x561101e64d90 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f04297fa640 1 -- 172.21.15.33:0/1008564808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04380bbd20 con 0x561101e64d90 2024-04-24T23:35:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f04297fa640 1 -- 172.21.15.33:0/1008564808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04380bb680 con 0x561101e64d90 2024-04-24T23:35:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f044c590740 1 -- 172.21.15.33:0/1008564808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 msgr2=0x561101e707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f044c590740 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e707d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f04380314e0 tx=0x7f043803b890 comp rx=0 tx=0).stop 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f044c590740 1 -- 172.21.15.33:0/1008564808 shutdown_connections 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f044c590740 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e75020 0x561101e604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f044c590740 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e707d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.381+0000 7f044c590740 1 --2- 172.21.15.33:0/1008564808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e5f230 0x561101e599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- 172.21.15.33:0/1008564808 >> 172.21.15.33:0/1008564808 conn(0x561101e5a8a0 msgr2=0x561101e592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- 172.21.15.33:0/1008564808 shutdown_connections 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- 172.21.15.33:0/1008564808 wait complete. 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- start start 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 Processor -- start 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e6dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e75020 0x561101e6ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04441aabd0 con 0x561101e64d90 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04441ac040 con 0x561101e5f230 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04441ab2a0 con 0x561101e75020 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.382+0000 7f044c590740 10 client.0 _ll_register_callbacks cb 0x7f0438005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e6dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f04427fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e6dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43092/0 (socket says 172.21.15.33:43092) 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f04437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e75020 0x561101e6ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e6d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39688/0 (socket says 172.21.15.33:39688) 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f04427fc640 1 -- 172.21.15.33:0/3036916778 learned_addr learned my addr 172.21.15.33:0/3036916778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 -- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e75020 msgr2=0x561101e6ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e75020 0x561101e6ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 -- 172.21.15.33:0/3036916778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 msgr2=0x561101e6dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e6dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 -- 172.21.15.33:0/3036916778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561101e6f380 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f04437fe640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e75020 0x561101e6ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f04427fc640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e6dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0442ffd640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e6d840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0434009600 tx=0x7f0434011e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab8d2928d31679 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.383+0000 7f0422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f0422ffd640 1 -- 172.21.15.33:0/3036916778 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434033060 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f0422ffd640 1 -- 172.21.15.33:0/3036916778 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0434005190 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04380bbd20 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f0422ffd640 1 -- 172.21.15.33:0/3036916778 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f043403a050 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f044c590740 10 client.29708 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561101e706f0 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.384+0000 7f0422ffd640 1 -- 172.21.15.33:0/3036916778 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04380bbd20 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f0422ffd640 1 -- 172.21.15.33:0/3036916778 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f043402d1d0 con 0x561101e5f230 2024-04-24T23:35:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f044c590740 10 client.29708 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f044c590740 10 client.29708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f044c590740 20 client.29708 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f044c590740 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101edb790 0x561101eddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f044c590740 4 client.29708 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561101dcaf00 con 0x561101edb790 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f04427fc640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101edb790 0x561101eddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f0422ffd640 10 client.29708 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.385+0000 7f04427fc640 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101edb790 0x561101eddb80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b4735dd3328034a server_cookie=296bf57fb7d5e324 in_seq=0 out_seq=0 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f0422ffd640 1 -- 172.21.15.33:0/3036916778 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561101dcaf00 con 0x561101edb790 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f0422ffd640 10 client.29708 handle_command_reply: tid=0 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 client.29708 shutdown 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 msgr2=0x561101e6d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e6d840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0434009600 tx=0x7f0434011e10 comp rx=0 tx=0).stop 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 shutdown_connections 2024-04-24T23:35:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x561101edb790 0x561101eddb80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561101e75020 0x561101e6ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561101e64d90 0x561101e6dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f0422ffd640 0 client.29708 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 --2- 172.21.15.33:0/3036916778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561101e5f230 0x561101e6d840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 >> 172.21.15.33:0/3036916778 conn(0x561101e5a8a0 msgr2=0x561101e59b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 shutdown_connections 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 1 -- 172.21.15.33:0/3036916778 wait complete. 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.386+0000 7f044c590740 20 client.29708 trim_cache size 0 max 16384 2024-04-24T23:35:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:35:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f042c0a2c10 msgr2=0x7f042c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f042c0a2c10 0x7f042c0a50d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x561101e60480 tx=0x7f043c009500 comp rx=0 tx=0).stop 2024-04-24T23:35:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 msgr2=0x7f044419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f044419a250 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0438031440 tx=0x7f0438004270 comp rx=0 tx=0).stop 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 shutdown_connections 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f042c0a2c10 0x7f042c0a50d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0444105570 0x7f04441a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0444104bb0 0x7f044419a250 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 --2- 172.21.15.33:0/1868206858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04441039c0 0x7f0444199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 >> 172.21.15.33:0/1868206858 conn(0x7f04440fef30 msgr2=0x7f0444107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 shutdown_connections 2024-04-24T23:35:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.387+0000 7f0449888640 1 -- 172.21.15.33:0/1868206858 wait complete. 2024-04-24T23:35:27.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:35:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:27 smithi033 ceph-mon[29201]: pgmap v1054: 129 pgs: 129 active+clean; 646 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 436 op/s 2024-04-24T23:35:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/426525974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:35:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:35:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:35:27.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:35:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 -- 172.21.15.33:0/2338162406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c100680 msgr2=0x7f0c8c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 --2- 172.21.15.33:0/2338162406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c100680 0x7f0c8c100aa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009900 tx=0x7f0c7c02a410 comp rx=0 tx=0).stop 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 -- 172.21.15.33:0/2338162406 shutdown_connections 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 --2- 172.21.15.33:0/2338162406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c102230 0x7f0c8c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 --2- 172.21.15.33:0/2338162406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c8c101870 0x7f0c8c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 --2- 172.21.15.33:0/2338162406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c100680 0x7f0c8c100aa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 -- 172.21.15.33:0/2338162406 >> 172.21.15.33:0/2338162406 conn(0x7f0c8c0fbbf0 msgr2=0x7f0c8c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 -- 172.21.15.33:0/2338162406 shutdown_connections 2024-04-24T23:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.518+0000 7f0c90970640 1 -- 172.21.15.33:0/2338162406 wait complete. 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 Processor -- start 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 -- start start 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c8c100680 0x7f0c8c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c101870 0x7f0c8c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 0x7f0c8c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c069320 con 0x7f0c8c101870 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0691a0 con 0x7f0c8c102230 2024-04-24T23:35:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c90970640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0694a0 con 0x7f0c8c100680 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c8ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 0x7f0c8c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c101870 0x7f0c8c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c8ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 0x7f0c8c1a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39712/0 (socket says 172.21.15.33:39712) 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.519+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/3476203617 learned_addr learned my addr 172.21.15.33:0/3476203617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c83fff640 1 -- 172.21.15.33:0/3476203617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c8c100680 msgr2=0x7f0c8c19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c83fff640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c8c100680 0x7f0c8c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c83fff640 1 -- 172.21.15.33:0/3476203617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 msgr2=0x7f0c8c1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c83fff640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 0x7f0c8c1a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c83fff640 1 -- 172.21.15.33:0/3476203617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c1a5ce0 con 0x7f0c8c101870 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 0x7f0c8c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c83fff640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c101870 0x7f0c8c19e600 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9f0 tx=0x7f0c7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c837fe640 1 -- 172.21.15.33:0/3476203617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400cd80 con 0x7f0c8c101870 2024-04-24T23:35:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c837fe640 1 -- 172.21.15.33:0/3476203617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c740035f0 con 0x7f0c8c101870 2024-04-24T23:35:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c837fe640 1 -- 172.21.15.33:0/3476203617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74010690 con 0x7f0c8c101870 2024-04-24T23:35:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.520+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c1a5fd0 con 0x7f0c8c101870 2024-04-24T23:35:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.521+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c1a63e0 con 0x7f0c8c101870 2024-04-24T23:35:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.524+0000 7f0c93684740 1 Processor -- start 2024-04-24T23:35:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.524+0000 7f0c93684740 1 -- start start 2024-04-24T23:35:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c93684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 0x55de5d034650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c93684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034c20 0x55de5d039720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c93684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d039d90 0x55de5d03c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c93684740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55de5cf98670 con 0x55de5d034230 2024-04-24T23:35:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c93684740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55de5cf40970 con 0x55de5d034c20 2024-04-24T23:35:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c93684740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de5cf94630 con 0x55de5d039d90 2024-04-24T23:35:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 0x55de5d034650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 0x55de5d034650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43136/0 (socket says 172.21.15.33:43136) 2024-04-24T23:35:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/751561151 learned_addr learned my addr 172.21.15.33:0/751561151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034c20 0x55de5d039720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/751561151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d039d90 msgr2=0x55de5d03c220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:35:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d039d90 0x55de5d03c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/751561151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034c20 msgr2=0x55de5d039720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034c20 0x55de5d039720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/751561151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5cf64400 con 0x55de5d034230 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.525+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034c20 0x55de5d039720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.526+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 0x55de5d034650 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0c840118b0 tx=0x7f0c84011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c98cdf77592959f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.526+0000 7f0c837fe640 1 -- 172.21.15.33:0/3476203617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0c74003b30 con 0x7f0c8c101870 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.526+0000 7f0c80ff9640 1 -- 172.21.15.33:0/751561151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401e070 con 0x55de5d034230 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.526+0000 7f0c80ff9640 1 -- 172.21.15.33:0/751561151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c84011310 con 0x55de5d034230 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.526+0000 7f0c80ff9640 1 -- 172.21.15.33:0/751561151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84019dd0 con 0x55de5d034230 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.527+0000 7f0c837fe640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0c780a2c10 0x7f0c780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.527+0000 7f0c837fe640 1 -- 172.21.15.33:0/3476203617 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0c740cf7b0 con 0x7f0c8c101870 2024-04-24T23:35:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.527+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0c780a2c10 0x7f0c780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.527+0000 7f0c93684740 1 -- 172.21.15.33:0/751561151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 msgr2=0x55de5d034650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.527+0000 7f0c93684740 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 0x55de5d034650 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0c840118b0 tx=0x7f0c84011d70 comp rx=0 tx=0).stop 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.528+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0c780a2c10 0x7f0c780a50d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0314e0 tx=0x7f0c7c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.529+0000 7f0c93684740 1 -- 172.21.15.33:0/751561151 shutdown_connections 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.529+0000 7f0c93684740 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d039d90 0x55de5d03c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.529+0000 7f0c93684740 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034c20 0x55de5d039720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.529+0000 7f0c93684740 1 --2- 172.21.15.33:0/751561151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034230 0x55de5d034650 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.529+0000 7f0c93684740 1 -- 172.21.15.33:0/751561151 >> 172.21.15.33:0/751561151 conn(0x55de5d02f8a0 msgr2=0x55de5d02fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.531+0000 7f0c93684740 1 -- 172.21.15.33:0/751561151 shutdown_connections 2024-04-24T23:35:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.531+0000 7f0c93684740 1 -- 172.21.15.33:0/751561151 wait complete. 2024-04-24T23:35:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.532+0000 7f0c93684740 1 -- start start 2024-04-24T23:35:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.532+0000 7f0c93684740 1 Processor -- start 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d03e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d03e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55de5d03cd40 con 0x55de5d034c20 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55de5cf54050 con 0x55de5d039d90 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de5d03cec0 con 0x55de5d034230 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c93684740 10 client.0 _ll_register_callbacks cb 0x55de5ccabbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d03e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d03e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41542/0 (socket says 172.21.15.33:41542) 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c8a59c640 1 -- 172.21.15.33:0/2302425697 learned_addr learned my addr 172.21.15.33:0/2302425697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.533+0000 7f0c83fff640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d03e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8a59c640 1 -- 172.21.15.33:0/2302425697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 msgr2=0x55de5d0458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8a59c640 1 -- 172.21.15.33:0/2302425697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 msgr2=0x55de5d03e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d03e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8a59c640 1 -- 172.21.15.33:0/2302425697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5cf25dc0 con 0x55de5d034230 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:35:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c83fff640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d03e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d03e330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009900 tx=0x7f0c7c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea86226c6e92508 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2302425697 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c04c070 con 0x55de5d034230 2024-04-24T23:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c84011310 con 0x55de5d034230 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.534+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2302425697 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0c7c004250 con 0x55de5d034230 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2302425697 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c7c058050 con 0x55de5d034230 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 10 client.29662 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55de5ccac3f0 con 0x55de5d034230 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2302425697 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c84011310 con 0x55de5d034230 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2302425697 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0c7c03c440 con 0x55de5d034230 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 10 client.29662 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 10 client.29662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 20 client.29662 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d04c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 4 client.29662 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.535+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de5cfa5bb0 con 0x55de5d04a0e0 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.536+0000 7f0c83fff640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d04c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.537+0000 7f0c83fff640 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d04c580 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=befa3c22411dda8f server_cookie=4af73d85aff46c95 in_seq=0 out_seq=0 2024-04-24T23:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.537+0000 7f0c71ffb640 10 client.29662 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:27.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.537+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2302425697 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de5cfa5bb0 con 0x55de5d04a0e0 2024-04-24T23:35:27.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.537+0000 7f0c71ffb640 10 client.29662 handle_command_reply: tid=0 2024-04-24T23:35:27.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 client.29662 shutdown 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 msgr2=0x55de5d03e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d03e330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009900 tx=0x7f0c7c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 shutdown_connections 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d04c580 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c71ffb640 0 client.29662 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d03e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 --2- 172.21.15.33:0/2302425697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d03e330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 >> 172.21.15.33:0/2302425697 conn(0x55de5d02f8a0 msgr2=0x55de5d02eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.538+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 shutdown_connections 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.539+0000 7f0c93684740 1 -- 172.21.15.33:0/2302425697 wait complete. 2024-04-24T23:35:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.539+0000 7f0c93684740 20 client.29662 trim_cache size 0 max 16384 2024-04-24T23:35:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 Processor -- start 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 -- start start 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d0354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d045870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1b3350 con 0x55de5d034c20 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1b0c20 con 0x55de5d039d90 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c93684740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1aea70 con 0x55de5d034230 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d045870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d045870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43156/0 (socket says 172.21.15.33:43156) 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c83fff640 1 -- 172.21.15.33:0/71585160 learned_addr learned my addr 172.21.15.33:0/71585160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.558+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d0354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c83fff640 1 -- 172.21.15.33:0/71585160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 msgr2=0x55de5d0354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c83fff640 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d0354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c83fff640 1 -- 172.21.15.33:0/71585160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 msgr2=0x55de5d0359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c83fff640 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c83fff640 1 -- 172.21.15.33:0/71585160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5cd767d0 con 0x55de5d034c20 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c83fff640 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d045870 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0c74098080 tx=0x7f0c74003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477b63b3cdb0fc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c73fff640 1 -- 172.21.15.33:0/71585160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7409ffd0 con 0x55de5d034c20 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c73fff640 1 -- 172.21.15.33:0/71585160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c740a4890 con 0x55de5d034c20 2024-04-24T23:35:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.559+0000 7f0c73fff640 1 -- 172.21.15.33:0/71585160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7409b070 con 0x55de5d034c20 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 -- 172.21.15.33:0/71585160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 msgr2=0x55de5d045870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d045870 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0c74098080 tx=0x7f0c74003e20 comp rx=0 tx=0).stop 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 -- 172.21.15.33:0/71585160 shutdown_connections 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d039d90 0x55de5d0359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d034c20 0x55de5d045870 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 --2- 172.21.15.33:0/71585160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034230 0x55de5d0354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 -- 172.21.15.33:0/71585160 >> 172.21.15.33:0/71585160 conn(0x55de5d02f8a0 msgr2=0x55de5d02eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 -- 172.21.15.33:0/71585160 shutdown_connections 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 -- 172.21.15.33:0/71585160 wait complete. 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 -- start start 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.560+0000 7f0c93684740 1 Processor -- start 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034230 0x55de5d042b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034c20 0x55de5d0430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d039d90 0x55de5d044040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1af0e0 con 0x55de5d039d90 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1b0550 con 0x55de5d034230 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1af7b0 con 0x55de5d034c20 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034230 0x55de5d042b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c93684740 10 client.0 _ll_register_callbacks cb 0x7f0c7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d039d90 0x55de5d044040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034230 0x55de5d042b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39766/0 (socket says 172.21.15.33:39766) 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8a59c640 1 -- 172.21.15.33:0/2427897096 learned_addr learned my addr 172.21.15.33:0/2427897096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c83fff640 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034c20 0x55de5d0430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034c20 msgr2=0x55de5d0430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034c20 0x55de5d0430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/2427897096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034230 msgr2=0x55de5d042b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034230 0x55de5d042b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 -- 172.21.15.33:0/2427897096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5d044630 con 0x55de5d039d90 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c8ad9d640 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d039d90 0x55de5d044040 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0c8401c040 tx=0x7f0c84004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d89cd750df9eb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.561+0000 7f0c71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2427897096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401e040 con 0x55de5d039d90 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2427897096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0c8400f9c0 con 0x55de5d039d90 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c740a4890 con 0x55de5d039d90 2024-04-24T23:35:27.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2427897096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c84033050 con 0x55de5d039d90 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 10 client.20262 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55de5d0459a0 con 0x55de5d039d90 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2427897096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c740a4890 con 0x55de5d039d90 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2427897096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0c84024450 con 0x55de5d039d90 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 10 client.20262 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 10 client.20262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 20 client.20262 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 4 client.20262 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de5cf9ff00 con 0x55de5d04a0e0 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.562+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c8a59c640 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d0b2be0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3109c8c72e92941f server_cookie=2d6bb2b5bd953ada in_seq=0 out_seq=0 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c71ffb640 10 client.20262 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c71ffb640 1 -- 172.21.15.33:0/2427897096 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de5cf9ff00 con 0x55de5d04a0e0 2024-04-24T23:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c71ffb640 10 client.20262 handle_command_reply: tid=0 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 client.20262 shutdown 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d039d90 msgr2=0x55de5d044040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d039d90 0x55de5d044040 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0c8401c040 tx=0x7f0c84004c00 comp rx=0 tx=0).stop 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 shutdown_connections 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55de5d04a0e0 0x55de5d0b2be0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55de5d039d90 0x55de5d044040 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de5d034c20 0x55de5d0430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 --2- 172.21.15.33:0/2427897096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de5d034230 0x55de5d042b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 >> 172.21.15.33:0/2427897096 conn(0x55de5d02f8a0 msgr2=0x55de5d032780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c71ffb640 0 client.20262 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 shutdown_connections 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.563+0000 7f0c93684740 1 -- 172.21.15.33:0/2427897096 wait complete. 2024-04-24T23:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c93684740 20 client.20262 trim_cache size 0 max 16384 2024-04-24T23:35:27.569 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:35:27.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0c780a2c10 msgr2=0x7f0c780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0c780a2c10 0x7f0c780a50d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0314e0 tx=0x7f0c7c031400 comp rx=0 tx=0).stop 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c101870 msgr2=0x7f0c8c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c101870 0x7f0c8c19e600 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9f0 tx=0x7f0c7400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 shutdown_connections 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f0c780a2c10 0x7f0c780a50d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c102230 0x7f0c8c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c8c101870 0x7f0c8c19e600 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 --2- 172.21.15.33:0/3476203617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c8c100680 0x7f0c8c19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 >> 172.21.15.33:0/3476203617 conn(0x7f0c8c0fbbf0 msgr2=0x7f0c8c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 shutdown_connections 2024-04-24T23:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:27.564+0000 7f0c90970640 1 -- 172.21.15.33:0/3476203617 wait complete. 2024-04-24T23:35:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:27 smithi038 ceph-mon[32450]: pgmap v1054: 129 pgs: 129 active+clean; 646 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 436 op/s 2024-04-24T23:35:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/426525974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:35:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:35:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:35:27.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:35:27.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:27 smithi117 ceph-mon[32529]: pgmap v1054: 129 pgs: 129 active+clean; 646 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 436 op/s 2024-04-24T23:35:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/426525974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:35:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:35:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:35:27.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:35:27.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 239082 21.97 MB/sec execute 452 sec latency 82.176 ms 2024-04-24T23:35:28.576 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:35:28.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:35:28.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 -- 172.21.15.33:0/4202672332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 msgr2=0x7fb2a41057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 --2- 172.21.15.33:0/4202672332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 0x7fb2a41057b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb29c01acb0 tx=0x7fb29c03b8e0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 -- 172.21.15.33:0/4202672332 shutdown_connections 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 --2- 172.21.15.33:0/4202672332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 0x7fb2a41057b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 --2- 172.21.15.33:0/4202672332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a4102980 0x7fb2a4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 --2- 172.21.15.33:0/4202672332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4100010 0x7fb2a4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 -- 172.21.15.33:0/4202672332 >> 172.21.15.33:0/4202672332 conn(0x7fb2a40fbc30 msgr2=0x7fb2a40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 -- 172.21.15.33:0/4202672332 shutdown_connections 2024-04-24T23:35:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.740+0000 7fb2a9610640 1 -- 172.21.15.33:0/4202672332 wait complete. 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.741+0000 7fb2a9610640 1 Processor -- start 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.741+0000 7fb2a9610640 1 -- start start 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a9610640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a4100010 0x7fb2a4071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a9610640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 0x7fb2a4072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a9610640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 0x7fb2a41a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a9610640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a4109270 con 0x7fb2a4105330 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a9610640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41090f0 con 0x7fb2a4102980 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a9610640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41093f0 con 0x7fb2a4100010 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 0x7fb2a4072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 0x7fb2a4072480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39792/0 (socket says 172.21.15.33:39792) 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 -- 172.21.15.33:0/1027764993 learned_addr learned my addr 172.21.15.33:0/1027764993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a37fe640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 0x7fb2a41a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a4100010 0x7fb2a4071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 -- 172.21.15.33:0/1027764993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a4100010 msgr2=0x7fb2a4071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a4100010 0x7fb2a4071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 -- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 msgr2=0x7fb2a41a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 0x7fb2a41a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.742+0000 7fb2a27fc640 1 -- 172.21.15.33:0/1027764993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a41a5e70 con 0x7fb2a4102980 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.743+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 0x7fb2a4072480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e9f0 tx=0x7fb29800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.743+0000 7fb28bfff640 1 -- 172.21.15.33:0/1027764993 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29800cd80 con 0x7fb2a4102980 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.743+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41a6040 con 0x7fb2a4102980 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.743+0000 7fb28bfff640 1 -- 172.21.15.33:0/1027764993 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb298010910 con 0x7fb2a4102980 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.743+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a41a6550 con 0x7fb2a4102980 2024-04-24T23:35:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.743+0000 7fb28bfff640 1 -- 172.21.15.33:0/1027764993 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298018990 con 0x7fb2a4102980 2024-04-24T23:35:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.746+0000 7fb28bfff640 1 -- 172.21.15.33:0/1027764993 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb298010ac0 con 0x7fb2a4102980 2024-04-24T23:35:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.746+0000 7fb2ac3f5740 1 Processor -- start 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 -- start start 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561beb1092c0 con 0x561beb1ab8f0 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561beb0b1910 con 0x561beb1a6780 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561beb105280 con 0x561beb1a5d90 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39802/0 (socket says 172.21.15.33:39802) 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1157219118 learned_addr learned my addr 172.21.15.33:0/1157219118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a37fe640 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb28bfff640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb28c0a2c10 0x7fb28c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1157219118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 msgr2=0x561beb1a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1157219118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 msgr2=0x561beb1add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.747+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1157219118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561beb0c2fb0 con 0x561beb1a6780 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb2a37fe640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb28c0a2c10 0x7fb28c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb28bfff640 1 -- 172.21.15.33:0/1027764993 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2980cf6f0 con 0x7fb2a4102980 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1ab280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb29400bcb0 tx=0x7fb29400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3920a47b379b7c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb2897fa640 1 -- 172.21.15.33:0/1157219118 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294015070 con 0x561beb1a6780 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb2897fa640 1 -- 172.21.15.33:0/1157219118 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb294011920 con 0x561beb1a6780 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb2897fa640 1 -- 172.21.15.33:0/1157219118 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294010930 con 0x561beb1a6780 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.748+0000 7fb2a37fe640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb28c0a2c10 0x7fb28c0a50d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x561beb1a6fe0 tx=0x7fb29c01a5d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1157219118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 msgr2=0x561beb1ab280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1ab280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb29400bcb0 tx=0x7fb29400f730 comp rx=0 tx=0).stop 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1157219118 shutdown_connections 2024-04-24T23:35:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1ab280 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1157219118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1157219118 >> 172.21.15.33:0/1157219118 conn(0x561beb16d950 msgr2=0x561beb1a1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1157219118 shutdown_connections 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1157219118 wait complete. 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- start start 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 Processor -- start 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561beb1ae8a0 con 0x561beb1ab8f0 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561beb0c4e90 con 0x561beb1a6780 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561beb1aea20 con 0x561beb1a5d90 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.749+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41594/0 (socket says 172.21.15.33:41594) 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1564690019 learned_addr learned my addr 172.21.15.33:0/1564690019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2ac3f5740 10 client.0 _ll_register_callbacks cb 0x561beb100970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1564690019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 msgr2=0x561beb1b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1564690019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 msgr2=0x561beb1b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/1564690019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561beaee57e0 con 0x561beb1a5d90 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b0240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb294019000 tx=0x7fb29400b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1523b5f562ad1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb286ff5640 1 -- 172.21.15.33:0/1564690019 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294028060 con 0x561beb1a5d90 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb294011920 con 0x561beb1a5d90 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb286ff5640 1 -- 172.21.15.33:0/1564690019 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb294011650 con 0x561beb1a5d90 2024-04-24T23:35:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb286ff5640 1 -- 172.21.15.33:0/1564690019 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb294015070 con 0x561beb1a5d90 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2ac3f5740 10 client.29680 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.750+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561beaf45f70 con 0x561beb1a5d90 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb286ff5640 1 -- 172.21.15.33:0/1564690019 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb294011920 con 0x561beb1a5d90 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb286ff5640 1 -- 172.21.15.33:0/1564690019 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb2940047e0 con 0x561beb1a5d90 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2ac3f5740 10 client.29680 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2ac3f5740 10 client.29680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2ac3f5740 20 client.29680 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb1bb900 0x561beb1bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2ac3f5740 4 client.29680 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561beb0dd5a0 con 0x561beb1bb900 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb1bb900 0x561beb1bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb286ff5640 10 client.29680 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.751+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb1bb900 0x561beb1bdcf0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c555d785fade6f server_cookie=14929175f2cdc4dd in_seq=0 out_seq=0 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb286ff5640 1 -- 172.21.15.33:0/1564690019 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561beb0dd5a0 con 0x561beb1bb900 2024-04-24T23:35:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb286ff5640 10 client.29680 handle_command_reply: tid=0 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 client.29680 shutdown 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 msgr2=0x561beb1b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b0240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb294019000 tx=0x7fb29400b7c0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 shutdown_connections 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb1bb900 0x561beb1bdcf0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1ab8f0 0x561beb1b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb286ff5640 0 client.29680 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1a6780 0x561beb1b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/1564690019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b0240 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 >> 172.21.15.33:0/1564690019 conn(0x561beb16d950 msgr2=0x561beb1a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 shutdown_connections 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/1564690019 wait complete. 2024-04-24T23:35:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.752+0000 7fb2ac3f5740 20 client.29680 trim_cache size 0 max 16384 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 Processor -- start 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 -- start start 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 0x561beb1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a6780 0x561beb1b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41b3380 con 0x561beb1a5d90 2024-04-24T23:35:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41b0c50 con 0x561beb1ab8f0 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.770+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41aeaa0 con 0x561beb1a6780 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 0x561beb1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a6780 0x561beb1b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 0x561beb1a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43208/0 (socket says 172.21.15.33:43208) 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/828487264 learned_addr learned my addr 172.21.15.33:0/828487264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/828487264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a6780 msgr2=0x561beb1b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a6780 0x561beb1b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/828487264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 msgr2=0x561beb1c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 -- 172.21.15.33:0/828487264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561beaddd170 con 0x561beb1a5d90 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 0x561beb1a0950 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb29400fc30 tx=0x7fb29400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34075238c931a825 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb288ff9640 1 -- 172.21.15.33:0/828487264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294028070 con 0x561beb1a5d90 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb288ff9640 1 -- 172.21.15.33:0/828487264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29400b830 con 0x561beb1a5d90 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.771+0000 7fb288ff9640 1 -- 172.21.15.33:0/828487264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294004250 con 0x561beb1a5d90 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/828487264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 msgr2=0x561beb1a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 0x561beb1a0950 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb29400fc30 tx=0x7fb29400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/828487264 shutdown_connections 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a6780 0x561beb1b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/828487264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a5d90 0x561beb1a0950 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/828487264 >> 172.21.15.33:0/828487264 conn(0x561beb16d950 msgr2=0x561beb1a4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/828487264 shutdown_connections 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/828487264 wait complete. 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 -- start start 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.772+0000 7fb2ac3f5740 1 Processor -- start 2024-04-24T23:35:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 0x561beb1b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41af110 con 0x561beb1a6780 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41b0580 con 0x561beb1ab8f0 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41af7e0 con 0x561beb1a5d90 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 10 client.0 _ll_register_callbacks cb 0x7fb298097e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 0x561beb1b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 0x561beb1b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43212/0 (socket says 172.21.15.33:43212) 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 -- 172.21.15.33:0/4180017719 learned_addr learned my addr 172.21.15.33:0/4180017719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a37fe640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 -- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 msgr2=0x561beb1b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 -- 172.21.15.33:0/4180017719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 msgr2=0x561beb1b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 -- 172.21.15.33:0/4180017719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561beb1b6d00 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2a27fc640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 0x561beb1b57f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb298097f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bd79d6a020c009 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb286ff5640 1 -- 172.21.15.33:0/4180017719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2980d5040 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29400b830 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb286ff5640 1 -- 172.21.15.33:0/4180017719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb298098230 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb286ff5640 1 -- 172.21.15.33:0/4180017719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb29809ff70 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 10 client.20280 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.773+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561beb2221e0 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb286ff5640 1 -- 172.21.15.33:0/4180017719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29400b830 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb286ff5640 1 -- 172.21.15.33:0/4180017719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb2980d2ce0 con 0x561beb1a6780 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2ac3f5740 10 client.20280 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2ac3f5740 10 client.20280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2ac3f5740 20 client.20280 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb222b80 0x561beb224f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2ac3f5740 4 client.20280 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:35:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561beb116800 con 0x561beb222b80 2024-04-24T23:35:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb222b80 0x561beb224f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb286ff5640 10 client.20280 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.774+0000 7fb2a2ffd640 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb222b80 0x561beb224f70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27e048b14c43c0d server_cookie=6539dddc5f4365ab in_seq=0 out_seq=0 2024-04-24T23:35:28.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb286ff5640 1 -- 172.21.15.33:0/4180017719 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561beb116800 con 0x561beb222b80 2024-04-24T23:35:28.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb286ff5640 10 client.20280 handle_command_reply: tid=0 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 client.20280 shutdown 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 msgr2=0x561beb1b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 0x561beb1b57f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb298097f90 comp rx=0 tx=0).stop 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 shutdown_connections 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x561beb222b80 0x561beb224f70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb286ff5640 0 client.20280 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561beb1ab8f0 0x561beb1b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561beb1a6780 0x561beb1b57f0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 --2- 172.21.15.33:0/4180017719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561beb1a5d90 0x561beb1b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.775+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 >> 172.21.15.33:0/4180017719 conn(0x561beb16d950 msgr2=0x561beb1a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 shutdown_connections 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2ac3f5740 1 -- 172.21.15.33:0/4180017719 wait complete. 2024-04-24T23:35:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2ac3f5740 20 client.20280 trim_cache size 0 max 16384 2024-04-24T23:35:28.782 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:35:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:35:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "fe28872c-1fcb-479c-8416-ec7bad3a7e72", 2024-04-24T23:35:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:35:28.782 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb28c0a2c10 msgr2=0x7fb28c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb28c0a2c10 0x7fb28c0a50d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x561beb1a6fe0 tx=0x7fb29c01a5d0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 msgr2=0x7fb2a4072480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 0x7fb2a4072480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e9f0 tx=0x7fb29800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 shutdown_connections 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb28c0a2c10 0x7fb28c0a50d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2a4105330 0x7fb2a41a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2a4102980 0x7fb2a4072480 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 --2- 172.21.15.33:0/1027764993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a4100010 0x7fb2a4071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 >> 172.21.15.33:0/1027764993 conn(0x7fb2a40fbc30 msgr2=0x7fb2a40fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 shutdown_connections 2024-04-24T23:35:28.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.776+0000 7fb2a9610640 1 -- 172.21.15.33:0/1027764993 wait complete. 2024-04-24T23:35:28.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:35:28.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:35:28.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 -- 172.21.15.33:0/135454611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 msgr2=0x7fb5c4109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/135454611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c4109d20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009cd0 tx=0x7fb5bc02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 -- 172.21.15.33:0/135454611 shutdown_connections 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/135454611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c4109d20 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/135454611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5c4102ad0 0x7fb5c4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/135454611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5c40ffa60 0x7fb5c40ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 -- 172.21.15.33:0/135454611 >> 172.21.15.33:0/135454611 conn(0x7fb5c40fb370 msgr2=0x7fb5c40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 -- 172.21.15.33:0/135454611 shutdown_connections 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 -- 172.21.15.33:0/135454611 wait complete. 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 Processor -- start 2024-04-24T23:35:28.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.902+0000 7fb5c9e15640 1 -- start start 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c9e15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5c40ffa60 0x7fb5c4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c9e15640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5c4102ad0 0x7fb5c4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c9e15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c9e15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d410 con 0x7fb5c4103490 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c9e15640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d290 con 0x7fb5c4102ad0 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c9e15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d590 con 0x7fb5c40ffa60 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43240/0 (socket says 172.21.15.33:43240) 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 -- 172.21.15.33:0/926434487 learned_addr learned my addr 172.21.15.33:0/926434487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5c4102ad0 0x7fb5c4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5c40ffa60 0x7fb5c4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 -- 172.21.15.33:0/926434487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5c40ffa60 msgr2=0x7fb5c4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5c40ffa60 0x7fb5c4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 -- 172.21.15.33:0/926434487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5c4102ad0 msgr2=0x7fb5c4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5c4102ad0 0x7fb5c4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 -- 172.21.15.33:0/926434487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c419f6d0 con 0x7fb5c4103490 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.903+0000 7fb5c3fff640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c419f060 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0317f0 tx=0x7fb5bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.904+0000 7fb5c17fa640 1 -- 172.21.15.33:0/926434487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc004040 con 0x7fb5c4103490 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.904+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c419f960 con 0x7fb5c4103490 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.904+0000 7fb5c17fa640 1 -- 172.21.15.33:0/926434487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5bc0046c0 con 0x7fb5c4103490 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.904+0000 7fb5c17fa640 1 -- 172.21.15.33:0/926434487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc03a420 con 0x7fb5c4103490 2024-04-24T23:35:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.904+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c419ff00 con 0x7fb5c4103490 2024-04-24T23:35:28.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 239518 21.98 MB/sec execute 453 sec latency 90.255 ms 2024-04-24T23:35:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5c17fa640 1 -- 172.21.15.33:0/926434487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb5bc0305a0 con 0x7fb5c4103490 2024-04-24T23:35:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 Processor -- start 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 -- start start 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494502660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494507730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c49450a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4944666b0 con 0x55c494507da0 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c49440e880 con 0x55c494502240 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c494462670 con 0x55c494502c30 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494507730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494502660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494507730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41632/0 (socket says 172.21.15.33:41632) 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1875555644 learned_addr learned my addr 172.21.15.33:0/1875555644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.906+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c49450a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5c17fa640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5b00a2b90 0x7fb5b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1875555644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 msgr2=0x55c494502660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494502660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1875555644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 msgr2=0x55c49450a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c49450a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1875555644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4942447d0 con 0x55c494502c30 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5c3fff640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5b00a2b90 0x7fb5b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5c17fa640 1 -- 172.21.15.33:0/926434487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5bc02e030 con 0x7fb5c4103490 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494507730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009ca0 tx=0x7fb5b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5b7c2938fb280b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5b9ffb640 1 -- 172.21.15.33:0/1875555644 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4009e50 con 0x55c494502c30 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5b9ffb640 1 -- 172.21.15.33:0/1875555644 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b40035f0 con 0x55c494502c30 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.907+0000 7fb5b9ffb640 1 -- 172.21.15.33:0/1875555644 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4019670 con 0x55c494502c30 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1875555644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 msgr2=0x55c494507730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5c3fff640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5b00a2b90 0x7fb5b00a5050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0b8df0 tx=0x7fb5bc087ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494507730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009ca0 tx=0x7fb5b4011810 comp rx=0 tx=0).stop 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1875555644 shutdown_connections 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c49450a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494507730 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1875555644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494502660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1875555644 >> 172.21.15.33:0/1875555644 conn(0x55c4944fd8b0 msgr2=0x55c4944fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1875555644 shutdown_connections 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1875555644 wait complete. 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.908+0000 7fb5ccc03740 1 -- start start 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 Processor -- start 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c49450c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c49450cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c494513c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c49450ad50 con 0x55c494507da0 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c494421f50 con 0x55c494502240 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c49450aed0 con 0x55c494502c30 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5ccc03740 10 client.0 _ll_register_callbacks cb 0x55c49445d000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c494513c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c49450cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c49450cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41640/0 (socket says 172.21.15.33:41640) 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2412225042 learned_addr learned my addr 172.21.15.33:0/2412225042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c49450c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2412225042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 msgr2=0x55c49450c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c49450c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2412225042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 msgr2=0x55c494513c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c494513c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2412225042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49417a3f0 con 0x55c494502c30 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.909+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c494513c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c49450cc00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009730 tx=0x7fb5b40039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9bf721d48c974c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:35:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/2412225042 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4028020 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/2412225042 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5b4016040 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b40035f0 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/2412225042 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb5b4033050 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 10 client.29698 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c49413a0f0 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/2412225042 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b40035f0 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/2412225042 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb5b4004880 con 0x55c494502c30 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 10 client.29698 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 10 client.29698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 20 client.29698 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c494518350 0x55c49451a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 4 client.29698 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.910+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c494473bf0 con 0x55c494518350 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.911+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c494518350 0x55c49451a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.911+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c494518350 0x55c49451a790 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211f113578c9bd97 server_cookie=45a8b76bf8a29745 in_seq=0 out_seq=0 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.911+0000 7fb5a2ffd640 10 client.29698 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/2412225042 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c494473bf0 con 0x55c494518350 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5a2ffd640 10 client.29698 handle_command_reply: tid=0 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 client.29698 shutdown 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 msgr2=0x55c49450cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c49450cc00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009730 tx=0x7fb5b40039d0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 shutdown_connections 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c494518350 0x55c49451a790 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494507da0 0x55c494513c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5a2ffd640 0 client.29698 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c49450cc00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/2412225042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c49450c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 >> 172.21.15.33:0/2412225042 conn(0x55c4944fd8b0 msgr2=0x55c4944fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 shutdown_connections 2024-04-24T23:35:28.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 1 -- 172.21.15.33:0/2412225042 wait complete. 2024-04-24T23:35:28.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.912+0000 7fb5ccc03740 20 client.29698 trim_cache size 0 max 16384 2024-04-24T23:35:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 Processor -- start 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 -- start start 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494504090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 0x55c494513c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494507da0 0x55c4945045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41acce0 con 0x55c494502c30 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41aa5b0 con 0x55c494502240 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a8400 con 0x55c494507da0 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 0x55c494513c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 0x55c494513c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43262/0 (socket says 172.21.15.33:43262) 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1146124106 learned_addr learned my addr 172.21.15.33:0/1146124106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494504090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494507da0 0x55c4945045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1146124106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494507da0 msgr2=0x55c4945045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494507da0 0x55c4945045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1146124106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 msgr2=0x55c494504090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494504090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.930+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/1146124106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49450b070 con 0x55c494502c30 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.931+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 0x55c494513c00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4012fd0 tx=0x7fb5b4011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3ad86119d613c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.931+0000 7fb5b97fa640 1 -- 172.21.15.33:0/1146124106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b403c070 con 0x55c494502c30 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.931+0000 7fb5b97fa640 1 -- 172.21.15.33:0/1146124106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b4016030 con 0x55c494502c30 2024-04-24T23:35:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.931+0000 7fb5b97fa640 1 -- 172.21.15.33:0/1146124106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4028040 con 0x55c494502c30 2024-04-24T23:35:28.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1146124106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 msgr2=0x55c494513c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 0x55c494513c00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4012fd0 tx=0x7fb5b4011d10 comp rx=0 tx=0).stop 2024-04-24T23:35:28.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1146124106 shutdown_connections 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494507da0 0x55c4945045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502c30 0x55c494513c00 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/1146124106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494502240 0x55c494504090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1146124106 >> 172.21.15.33:0/1146124106 conn(0x55c4944fd8b0 msgr2=0x55c49450c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1146124106 shutdown_connections 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- 172.21.15.33:0/1146124106 wait complete. 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- start start 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 Processor -- start 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502240 0x55c494511150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494511690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494507da0 0x55c4945126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a8a70 con 0x55c494502240 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a9ee0 con 0x55c494507da0 2024-04-24T23:35:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a9140 con 0x55c494502c30 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.932+0000 7fb5ccc03740 10 client.0 _ll_register_callbacks cb 0x7fb5ac009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494511690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494511690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:41660/0 (socket says 172.21.15.33:41660) 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/186424117 learned_addr learned my addr 172.21.15.33:0/186424117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502240 0x55c494511150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494507da0 0x55c4945126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/186424117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494507da0 msgr2=0x55c4945126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494507da0 0x55c4945126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/186424117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502240 msgr2=0x55c494511150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502240 0x55c494511150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/186424117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c494512c90 con 0x55c494502c30 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494511690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4015040 tx=0x7fb5b4009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17bdf01d0ef91e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:35:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/186424117 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b403c070 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b4016030 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.933+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/186424117 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5b4005450 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.934+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/186424117 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb5b4048050 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.934+0000 7fb5ccc03740 10 client.29704 fetch_fsmap learned FSMap version 33 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.934+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c49457e8d0 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.934+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/186424117 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b4004e40 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.934+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/186424117 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb5b40375f0 con 0x55c494502c30 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 10 client.29704 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 10 client.29704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 20 client.29704 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49457efc0 0x55c4945813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 4 client.29704 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb5b4016030 con 0x55c49457efc0 2024-04-24T23:35:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49457efc0 0x55c4945813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5c37fe640 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49457efc0 0x55c4945813b0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d13875085e4004e server_cookie=3b85e31637e8e85c in_seq=0 out_seq=0 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5a2ffd640 10 client.29704 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5a2ffd640 1 -- 172.21.15.33:0/186424117 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fb5b4016030 con 0x55c49457efc0 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5a2ffd640 10 client.29704 handle_command_reply: tid=0 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.935+0000 7fb5ccc03740 1 client.29704 shutdown 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 msgr2=0x55c494511690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494511690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4015040 tx=0x7fb5b4009340 comp rx=0 tx=0).stop 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 shutdown_connections 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c49457efc0 0x55c4945813b0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c494507da0 0x55c4945126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c494502c30 0x55c494511690 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5a2ffd640 0 client.29704 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 --2- 172.21.15.33:0/186424117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c494502240 0x55c494511150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 >> 172.21.15.33:0/186424117 conn(0x55c4944fd8b0 msgr2=0x55c49450c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 shutdown_connections 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 1 -- 172.21.15.33:0/186424117 wait complete. 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.936+0000 7fb5ccc03740 20 client.29704 trim_cache size 0 max 16384 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "fe28872c-1fcb-479c-8416-ec7bad3a7e72": { 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "fe28872c-1fcb-479c-8416-ec7bad3a7e72", 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:35:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:35:28.942 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:35:28.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5b00a2b90 msgr2=0x7fb5b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5b00a2b90 0x7fb5b00a5050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0b8df0 tx=0x7fb5bc087ab0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 msgr2=0x7fb5c419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c419f060 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0317f0 tx=0x7fb5bc003ba0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 shutdown_connections 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb5b00a2b90 0x7fb5b00a5050 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4103490 0x7fb5c419f060 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5c4102ad0 0x7fb5c4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 --2- 172.21.15.33:0/926434487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5c40ffa60 0x7fb5c4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 >> 172.21.15.33:0/926434487 conn(0x7fb5c40fb370 msgr2=0x7fb5c4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 shutdown_connections 2024-04-24T23:35:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:28.937+0000 7fb5c9e15640 1 -- 172.21.15.33:0/926434487 wait complete. 2024-04-24T23:35:28.947 INFO:tasks.cephfs.filesystem:scrub status for tag:fe28872c-1fcb-479c-8416-ec7bad3a7e72 - {'path': '/', 'tag': 'fe28872c-1fcb-479c-8416-ec7bad3a7e72', 'options': 'recursive,force'} 2024-04-24T23:35:29.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:29 smithi038 ceph-mon[32450]: pgmap v1055: 129 pgs: 129 active+clean; 636 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-04-24T23:35:29.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:29 smithi117 ceph-mon[32529]: pgmap v1055: 129 pgs: 129 active+clean; 636 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-04-24T23:35:29.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:29 smithi033 ceph-mon[29201]: pgmap v1055: 129 pgs: 129 active+clean; 636 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-04-24T23:35:29.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 239803 21.97 MB/sec execute 454 sec latency 106.409 ms 2024-04-24T23:35:30.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:35:30.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:35:30.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:30 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:35:30.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:30 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:35:30.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:35:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:30 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:30.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 240226 21.97 MB/sec execute 455 sec latency 84.332 ms 2024-04-24T23:35:31.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:31 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:31.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:31 smithi038 ceph-mon[32450]: pgmap v1056: 129 pgs: 129 active+clean; 635 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 421 op/s 2024-04-24T23:35:31.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:31 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:31.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:31 smithi117 ceph-mon[32529]: pgmap v1056: 129 pgs: 129 active+clean; 635 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 421 op/s 2024-04-24T23:35:31.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:31 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:35:31.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:31 smithi033 ceph-mon[29201]: pgmap v1056: 129 pgs: 129 active+clean; 635 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 421 op/s 2024-04-24T23:35:31.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 240409 21.95 MB/sec execute 456 sec latency 531.098 ms 2024-04-24T23:35:32.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 240755 21.95 MB/sec execute 457 sec latency 156.618 ms 2024-04-24T23:35:33.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:33 smithi038 ceph-mon[32450]: pgmap v1057: 129 pgs: 129 active+clean; 634 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.0 MiB/s wr, 449 op/s 2024-04-24T23:35:33.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:33 smithi117 ceph-mon[32529]: pgmap v1057: 129 pgs: 129 active+clean; 634 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.0 MiB/s wr, 449 op/s 2024-04-24T23:35:33.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:33 smithi033 ceph-mon[29201]: pgmap v1057: 129 pgs: 129 active+clean; 634 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.0 MiB/s wr, 449 op/s 2024-04-24T23:35:33.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 241279 21.98 MB/sec execute 458 sec latency 86.631 ms 2024-04-24T23:35:34.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 241645 21.98 MB/sec execute 459 sec latency 84.670 ms 2024-04-24T23:35:35.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:35 smithi038 ceph-mon[32450]: pgmap v1058: 129 pgs: 129 active+clean; 634 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 405 op/s 2024-04-24T23:35:35.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:35 smithi117 ceph-mon[32529]: pgmap v1058: 129 pgs: 129 active+clean; 634 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 405 op/s 2024-04-24T23:35:35.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:35 smithi033 ceph-mon[29201]: pgmap v1058: 129 pgs: 129 active+clean; 634 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 405 op/s 2024-04-24T23:35:35.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 242148 21.99 MB/sec execute 460 sec latency 73.833 ms 2024-04-24T23:35:36.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 242355 21.97 MB/sec execute 461 sec latency 319.218 ms 2024-04-24T23:35:37.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:37 smithi038 ceph-mon[32450]: pgmap v1059: 129 pgs: 129 active+clean; 632 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 398 op/s 2024-04-24T23:35:37.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:37 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:35:37.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:37 smithi117 ceph-mon[32529]: pgmap v1059: 129 pgs: 129 active+clean; 632 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 398 op/s 2024-04-24T23:35:37.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:37 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:35:37.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:37 smithi033 ceph-mon[29201]: pgmap v1059: 129 pgs: 129 active+clean; 632 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 398 op/s 2024-04-24T23:35:37.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:37 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:35:37.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 242596 21.96 MB/sec execute 462 sec latency 172.837 ms 2024-04-24T23:35:38.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:38 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:35:38.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:38 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:35:38.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:38 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:35:38.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 243093 21.95 MB/sec execute 463 sec latency 72.016 ms 2024-04-24T23:35:39.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:39 smithi038 ceph-mon[32450]: pgmap v1060: 129 pgs: 129 active+clean; 630 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-04-24T23:35:39.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:39 smithi117 ceph-mon[32529]: pgmap v1060: 129 pgs: 129 active+clean; 630 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-04-24T23:35:39.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:39 smithi033 ceph-mon[29201]: pgmap v1060: 129 pgs: 129 active+clean; 630 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-04-24T23:35:39.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 243532 21.96 MB/sec execute 464 sec latency 113.224 ms 2024-04-24T23:35:40.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 244209 22.00 MB/sec execute 465 sec latency 66.696 ms 2024-04-24T23:35:41.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:41 smithi038 ceph-mon[32450]: pgmap v1061: 129 pgs: 129 active+clean; 631 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 451 op/s 2024-04-24T23:35:41.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:41 smithi117 ceph-mon[32529]: pgmap v1061: 129 pgs: 129 active+clean; 631 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 451 op/s 2024-04-24T23:35:41.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:41 smithi033 ceph-mon[29201]: pgmap v1061: 129 pgs: 129 active+clean; 631 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 451 op/s 2024-04-24T23:35:41.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 244424 21.98 MB/sec execute 466 sec latency 310.132 ms 2024-04-24T23:35:42.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 244705 21.97 MB/sec execute 467 sec latency 147.184 ms 2024-04-24T23:35:43.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:43 smithi038 ceph-mon[32450]: pgmap v1062: 129 pgs: 129 active+clean; 632 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 442 op/s 2024-04-24T23:35:43.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:35:43.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:35:43.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:43 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:35:43.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:43 smithi117 ceph-mon[32529]: pgmap v1062: 129 pgs: 129 active+clean; 632 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 442 op/s 2024-04-24T23:35:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:35:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:35:43.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:43 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:35:43.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:43 smithi033 ceph-mon[29201]: pgmap v1062: 129 pgs: 129 active+clean; 632 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 442 op/s 2024-04-24T23:35:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:35:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:35:43.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:43 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:35:43.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 245070 21.95 MB/sec execute 468 sec latency 105.573 ms 2024-04-24T23:35:44.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 245400 21.94 MB/sec execute 469 sec latency 126.454 ms 2024-04-24T23:35:45.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:45 smithi038 ceph-mon[32450]: pgmap v1063: 129 pgs: 129 active+clean; 638 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 384 op/s 2024-04-24T23:35:45.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:45 smithi117 ceph-mon[32529]: pgmap v1063: 129 pgs: 129 active+clean; 638 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 384 op/s 2024-04-24T23:35:45.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:45 smithi033 ceph-mon[29201]: pgmap v1063: 129 pgs: 129 active+clean; 638 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 384 op/s 2024-04-24T23:35:45.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 245913 21.94 MB/sec execute 470 sec latency 82.162 ms 2024-04-24T23:35:46.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 246152 21.93 MB/sec execute 471 sec latency 405.299 ms 2024-04-24T23:35:47.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:47 smithi038 ceph-mon[32450]: pgmap v1064: 129 pgs: 129 active+clean; 637 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 358 op/s 2024-04-24T23:35:47.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:47 smithi117 ceph-mon[32529]: pgmap v1064: 129 pgs: 129 active+clean; 637 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 358 op/s 2024-04-24T23:35:47.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:47 smithi033 ceph-mon[29201]: pgmap v1064: 129 pgs: 129 active+clean; 637 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 358 op/s 2024-04-24T23:35:47.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 246581 21.94 MB/sec execute 472 sec latency 130.386 ms 2024-04-24T23:35:48.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 247094 21.95 MB/sec execute 473 sec latency 112.505 ms 2024-04-24T23:35:49.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:49 smithi038 ceph-mon[32450]: pgmap v1065: 129 pgs: 129 active+clean; 641 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 458 op/s 2024-04-24T23:35:49.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:49 smithi117 ceph-mon[32529]: pgmap v1065: 129 pgs: 129 active+clean; 641 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 458 op/s 2024-04-24T23:35:49.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:49 smithi033 ceph-mon[29201]: pgmap v1065: 129 pgs: 129 active+clean; 641 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 458 op/s 2024-04-24T23:35:49.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 247478 21.95 MB/sec execute 474 sec latency 63.436 ms 2024-04-24T23:35:50.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 247939 21.95 MB/sec execute 475 sec latency 107.794 ms 2024-04-24T23:35:51.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:51 smithi038 ceph-mon[32450]: pgmap v1066: 129 pgs: 129 active+clean; 641 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 328 op/s 2024-04-24T23:35:51.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:51 smithi117 ceph-mon[32529]: pgmap v1066: 129 pgs: 129 active+clean; 641 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 328 op/s 2024-04-24T23:35:51.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:51 smithi033 ceph-mon[29201]: pgmap v1066: 129 pgs: 129 active+clean; 641 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 328 op/s 2024-04-24T23:35:51.905 INFO:tasks.workunit.client.0.smithi033.stdout: 10 248165 21.94 MB/sec execute 476 sec latency 284.662 ms 2024-04-24T23:35:52.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 248437 21.93 MB/sec execute 477 sec latency 146.042 ms 2024-04-24T23:35:53.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:53 smithi038 ceph-mon[32450]: pgmap v1067: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 391 op/s 2024-04-24T23:35:53.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:53 smithi117 ceph-mon[32529]: pgmap v1067: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 391 op/s 2024-04-24T23:35:53.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:53 smithi033 ceph-mon[29201]: pgmap v1067: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 391 op/s 2024-04-24T23:35:53.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 248698 21.91 MB/sec execute 478 sec latency 146.862 ms 2024-04-24T23:35:54.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 249124 21.89 MB/sec execute 479 sec latency 84.558 ms 2024-04-24T23:35:55.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:55 smithi038 ceph-mon[32450]: pgmap v1068: 129 pgs: 129 active+clean; 628 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 376 op/s 2024-04-24T23:35:55.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:55 smithi117 ceph-mon[32529]: pgmap v1068: 129 pgs: 129 active+clean; 628 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 376 op/s 2024-04-24T23:35:55.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:55 smithi033 ceph-mon[29201]: pgmap v1068: 129 pgs: 129 active+clean; 628 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 376 op/s 2024-04-24T23:35:55.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 249594 21.90 MB/sec execute 480 sec latency 91.240 ms 2024-04-24T23:35:56.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 250036 21.89 MB/sec execute 481 sec latency 367.257 ms 2024-04-24T23:35:57.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:57 smithi038 ceph-mon[32450]: pgmap v1069: 129 pgs: 129 active+clean; 625 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-24T23:35:57.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:57 smithi117 ceph-mon[32529]: pgmap v1069: 129 pgs: 129 active+clean; 625 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-24T23:35:57.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:57 smithi033 ceph-mon[29201]: pgmap v1069: 129 pgs: 129 active+clean; 625 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-24T23:35:57.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 250457 21.92 MB/sec execute 482 sec latency 382.417 ms 2024-04-24T23:35:58.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 250841 21.92 MB/sec execute 483 sec latency 104.914 ms 2024-04-24T23:35:58.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:35:58.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 -- 172.21.15.33:0/2398915119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 msgr2=0x7f5af0102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 --2- 172.21.15.33:0/2398915119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 0x7f5af0102050 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009900 tx=0x7f5ae002a410 comp rx=0 tx=0).stop 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 -- 172.21.15.33:0/2398915119 shutdown_connections 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 --2- 172.21.15.33:0/2398915119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0101220 0x7f5af0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 --2- 172.21.15.33:0/2398915119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5af0100880 0x7f5af0100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 --2- 172.21.15.33:0/2398915119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 0x7f5af0102050 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 -- 172.21.15.33:0/2398915119 >> 172.21.15.33:0/2398915119 conn(0x7f5af00fb370 msgr2=0x7f5af00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 -- 172.21.15.33:0/2398915119 shutdown_connections 2024-04-24T23:35:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.063+0000 7f5af59a9640 1 -- 172.21.15.33:0/2398915119 wait complete. 2024-04-24T23:35:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 Processor -- start 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 -- start start 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 0x7f5af0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5af0101220 0x7f5af0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 0x7f5af019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5af010c2f0 con 0x7f5af0101c30 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5af010c170 con 0x7f5af0100880 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.064+0000 7f5af59a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5af010c470 con 0x7f5af0101220 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 0x7f5af0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 0x7f5af0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51286/0 (socket says 172.21.15.33:51286) 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 -- 172.21.15.33:0/454492111 learned_addr learned my addr 172.21.15.33:0/454492111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5af0101220 0x7f5af0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 0x7f5af019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 -- 172.21.15.33:0/454492111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5af0101220 msgr2=0x7f5af0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5af0101220 0x7f5af0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 -- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 msgr2=0x7f5af019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 0x7f5af019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 -- 172.21.15.33:0/454492111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af019f6e0 con 0x7f5af0100880 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 0x7f5af0197ac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009230 tx=0x7f5ae00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.065+0000 7f5af49a7640 1 -- 172.21.15.33:0/454492111 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae003f030 con 0x7f5af0100880 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.066+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af019f970 con 0x7f5af0100880 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.066+0000 7f5af49a7640 1 -- 172.21.15.33:0/454492111 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0003ce0 con 0x7f5af0100880 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.066+0000 7f5af49a7640 1 -- 172.21.15.33:0/454492111 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0035070 con 0x7f5af0100880 2024-04-24T23:35:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.066+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af019feb0 con 0x7f5af0100880 2024-04-24T23:35:59.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.068+0000 7f5af49a7640 1 -- 172.21.15.33:0/454492111 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5ae0042030 con 0x7f5af0100880 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 Processor -- start 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 -- start start 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dc2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dc5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e791d216b0 con 0x55e791dbd240 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e791cc9880 con 0x55e791dc2da0 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af879a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e791d1d670 con 0x55e791dbdc30 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5af49a7640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ad80a2b90 0x7f5ad80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dc2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5aee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dc5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dc2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42164/0 (socket says 172.21.15.33:42164) 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1303826038 learned_addr learned my addr 172.21.15.33:0/1303826038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.069+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5af49a7640 1 -- 172.21.15.33:0/454492111 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ae00ef920 con 0x7f5af0100880 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ad80a2b90 0x7f5ad80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1303826038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 msgr2=0x55e791dc5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dc5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1303826038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 msgr2=0x55e791dbd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1303826038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e791aff7d0 con 0x55e791dbdc30 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dc2730 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00b8250 tx=0x7f5ae0031b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c42655f851280ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5ad5ffb640 1 -- 172.21.15.33:0/1303826038 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0049070 con 0x55e791dbdc30 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ad80a2b90 0x7f5ad80a5050 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8005c40 tx=0x7f5ae8009bb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5ad5ffb640 1 -- 172.21.15.33:0/1303826038 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae00830c0 con 0x55e791dbdc30 2024-04-24T23:35:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.070+0000 7f5ad5ffb640 1 -- 172.21.15.33:0/1303826038 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00b84b0 con 0x55e791dbdc30 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 -- 172.21.15.33:0/1303826038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 msgr2=0x55e791dc2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dc2730 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00b8250 tx=0x7f5ae0031b40 comp rx=0 tx=0).stop 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 -- 172.21.15.33:0/1303826038 shutdown_connections 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dc5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dc2730 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 --2- 172.21.15.33:0/1303826038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 -- 172.21.15.33:0/1303826038 >> 172.21.15.33:0/1303826038 conn(0x55e791db88b0 msgr2=0x55e791db8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 -- 172.21.15.33:0/1303826038 shutdown_connections 2024-04-24T23:35:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.071+0000 7f5af879a740 1 -- 172.21.15.33:0/1303826038 wait complete. 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 -- start start 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 Processor -- start 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dc7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 0x55e791dc78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 0x55e791dce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e791dc5b10 con 0x55e791dbd240 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e791cdcf50 con 0x55e791dbdc30 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e791dc5c90 con 0x55e791dc2da0 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5af879a740 10 client.0 _ll_register_callbacks cb 0x55e791d18000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dc7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 0x55e791dce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 0x55e791dc78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 0x55e791dce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42180/0 (socket says 172.21.15.33:42180) 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aef7fe640 1 -- 172.21.15.33:0/2089953451 learned_addr learned my addr 172.21.15.33:0/2089953451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 0x55e791dc78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51310/0 (socket says 172.21.15.33:51310) 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.072+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dc7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54676/0 (socket says 172.21.15.33:54676) 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aef7fe640 1 -- 172.21.15.33:0/2089953451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 msgr2=0x55e791dc78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 0x55e791dc78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aef7fe640 1 -- 172.21.15.33:0/2089953451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 msgr2=0x55e791dc7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dc7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aef7fe640 1 -- 172.21.15.33:0/2089953451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e791a353f0 con 0x55e791dc2da0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 0x55e791dc78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 0x55e791dce900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8003e40 tx=0x7f5ae8005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3143d4ae73717bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aceffd640 1 -- 172.21.15.33:0/2089953451 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8004d20 con 0x55e791dc2da0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aceffd640 1 -- 172.21.15.33:0/2089953451 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ae8005020 con 0x55e791dc2da0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e791d2ebf0 con 0x55e791dc2da0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5aceffd640 1 -- 172.21.15.33:0/2089953451 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ae8005220 con 0x55e791dc2da0 2024-04-24T23:35:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5af879a740 10 client.29716 fetch_fsmap learned FSMap version 35 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.073+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7919f50f0 con 0x55e791dc2da0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5aceffd640 1 -- 172.21.15.33:0/2089953451 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e791d2ebf0 con 0x55e791dc2da0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5aceffd640 1 -- 172.21.15.33:0/2089953451 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5ae8029760 con 0x55e791dc2da0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5af879a740 10 client.29716 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5af879a740 10 client.29716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5af879a740 20 client.29716 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5af879a740 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791dd32d0 0x55e791dd5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5af879a740 4 client.29716 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e791d28f40 con 0x55e791dd32d0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.074+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791dd32d0 0x55e791dd5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.075+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791dd32d0 0x55e791dd5720 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cb76344a4e1538 server_cookie=488ddaa58fe5771 in_seq=0 out_seq=0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.075+0000 7f5aceffd640 10 client.29716 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.075+0000 7f5aceffd640 1 -- 172.21.15.33:0/2089953451 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e791d28f40 con 0x55e791dd32d0 2024-04-24T23:35:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.075+0000 7f5aceffd640 10 client.29716 handle_command_reply: tid=0 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.075+0000 7f5af879a740 1 client.29716 shutdown 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 msgr2=0x55e791dce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 0x55e791dce900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8003e40 tx=0x7f5ae8005bd0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 shutdown_connections 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791dd32d0 0x55e791dd5720 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dc2da0 0x55e791dce900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5aceffd640 0 client.29716 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dbdc30 0x55e791dc78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 --2- 172.21.15.33:0/2089953451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dc7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 >> 172.21.15.33:0/2089953451 conn(0x55e791db88b0 msgr2=0x55e791db7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 shutdown_connections 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 1 -- 172.21.15.33:0/2089953451 wait complete. 2024-04-24T23:35:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.076+0000 7f5af879a740 20 client.29716 trim_cache size 0 max 16384 2024-04-24T23:35:59.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 Processor -- start 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 -- start start 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dbc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01aaa90 con 0x55e791dbd240 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01a8330 con 0x55e791dc2da0 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5af879a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01a6180 con 0x55e791dbdc30 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5aee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dbc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51312/0 (socket says 172.21.15.33:51312) 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54684/0 (socket says 172.21.15.33:54684) 2024-04-24T23:35:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.095+0000 7f5aef7fe640 1 -- 172.21.15.33:0/438657775 learned_addr learned my addr 172.21.15.33:0/438657775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aee7fc640 1 -- 172.21.15.33:0/438657775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 msgr2=0x55e791dce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aee7fc640 1 -- 172.21.15.33:0/438657775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 msgr2=0x55e791dbbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aee7fc640 1 -- 172.21.15.33:0/438657775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e791daa090 con 0x55e791dbdc30 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dbc4b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55e791dc8860 tx=0x7f5ae4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed6626f0e5785a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5ad57fa640 1 -- 172.21.15.33:0/438657775 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4021070 con 0x55e791dbdc30 2024-04-24T23:35:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5ad57fa640 1 -- 172.21.15.33:0/438657775 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae40035f0 con 0x55e791dbdc30 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.096+0000 7f5ad57fa640 1 -- 172.21.15.33:0/438657775 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae401c610 con 0x55e791dbdc30 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 -- 172.21.15.33:0/438657775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 msgr2=0x55e791dbc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dbc4b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55e791dc8860 tx=0x7f5ae4010800 comp rx=0 tx=0).stop 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 -- 172.21.15.33:0/438657775 shutdown_connections 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dbc4b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 --2- 172.21.15.33:0/438657775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dbbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 -- 172.21.15.33:0/438657775 >> 172.21.15.33:0/438657775 conn(0x55e791db88b0 msgr2=0x55e791dbb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 -- 172.21.15.33:0/438657775 shutdown_connections 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.097+0000 7f5af879a740 1 -- 172.21.15.33:0/438657775 wait complete. 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.098+0000 7f5af879a740 1 -- start start 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.098+0000 7f5af879a740 1 Processor -- start 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.098+0000 7f5af879a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dcc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.098+0000 7f5af879a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dcc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.098+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dcc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dcc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54700/0 (socket says 172.21.15.33:54700) 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1410093756 learned_addr learned my addr 172.21.15.33:0/1410093756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dcd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01a67f0 con 0x55e791dbd240 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01a7c60 con 0x55e791dc2da0 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01a6ec0 con 0x55e791dbdc30 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5af879a740 10 client.0 _ll_register_callbacks cb 0x7f5ae8011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aef7fe640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dcd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dcc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 msgr2=0x55e791dcc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dcc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1410093756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 msgr2=0x55e791dcd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dcd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aeeffd640 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e791dcdee0 con 0x55e791dbd240 2024-04-24T23:35:59.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.099+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dcc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5aeeffd640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dcc3a0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00b82a0 tx=0x7f5ae00b81b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1866757cfbbd47d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5aceffd640 1 -- 172.21.15.33:0/1410093756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00d4020 con 0x55e791dbd240 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5aceffd640 1 -- 172.21.15.33:0/1410093756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ae00b84b0 con 0x55e791dbd240 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae40035f0 con 0x55e791dbd240 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5aceffd640 1 -- 172.21.15.33:0/1410093756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ae0049040 con 0x55e791dbd240 2024-04-24T23:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5af879a740 10 client.20316 fetch_fsmap learned FSMap version 35 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.100+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e791e39c10 con 0x55e791dbd240 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5aceffd640 1 -- 172.21.15.33:0/1410093756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae40035f0 con 0x55e791dbd240 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5aceffd640 1 -- 172.21.15.33:0/1410093756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5ae00bc0a0 con 0x55e791dbd240 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5af879a740 10 client.20316 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5af879a740 10 client.20316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5af879a740 20 client.20316 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791e3a4a0 0x55e791e3c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5af879a740 4 client.20316 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e791ca66f0 con 0x55e791e3a4a0 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.101+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791e3a4a0 0x55e791e3c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5aee7fc640 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791e3a4a0 0x55e791e3c8c0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4162f8d28bf0ee04 server_cookie=1e026abbe9516ec7 in_seq=0 out_seq=0 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5aceffd640 10 client.20316 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5aceffd640 1 -- 172.21.15.33:0/1410093756 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e791ca66f0 con 0x55e791e3a4a0 2024-04-24T23:35:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5aceffd640 10 client.20316 handle_command_reply: tid=0 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 client.20316 shutdown 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 msgr2=0x55e791dcc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dcc3a0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00b82a0 tx=0x7f5ae00b81b0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 shutdown_connections 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55e791e3a4a0 0x55e791e3c8c0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e791dc2da0 0x55e791dcd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e791dbdc30 0x55e791dcc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 --2- 172.21.15.33:0/1410093756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e791dbd240 0x55e791dcc3a0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5aceffd640 0 client.20316 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 >> 172.21.15.33:0/1410093756 conn(0x55e791db88b0 msgr2=0x55e791dbc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.102+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 shutdown_connections 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af879a740 1 -- 172.21.15.33:0/1410093756 wait complete. 2024-04-24T23:35:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af879a740 20 client.20316 trim_cache size 0 max 16384 2024-04-24T23:35:59.108 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:35:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:35:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:35:59.108 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ad80a2b90 msgr2=0x7f5ad80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ad80a2b90 0x7f5ad80a5050 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8005c40 tx=0x7f5ae8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 msgr2=0x7f5af0197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 0x7f5af0197ac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009230 tx=0x7f5ae00042f0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 shutdown_connections 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5ad80a2b90 0x7f5ad80a5050 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af0101c30 0x7f5af019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5af0101220 0x7f5af0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 --2- 172.21.15.33:0/454492111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af0100880 0x7f5af0197ac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.103+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 >> 172.21.15.33:0/454492111 conn(0x7f5af00fb370 msgr2=0x7f5af00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.104+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 shutdown_connections 2024-04-24T23:35:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.104+0000 7f5af59a9640 1 -- 172.21.15.33:0/454492111 wait complete. 2024-04-24T23:35:59.116 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:35:59.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 -- 172.21.15.33:0/1847216000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c104b30 msgr2=0x7f519c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 --2- 172.21.15.33:0/1847216000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c104b30 0x7f519c106f20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f518c009900 tx=0x7f518c02a410 comp rx=0 tx=0).stop 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 -- 172.21.15.33:0/1847216000 shutdown_connections 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 --2- 172.21.15.33:0/1847216000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f519c107460 0x7f519c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 --2- 172.21.15.33:0/1847216000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c104b30 0x7f519c106f20 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 --2- 172.21.15.33:0/1847216000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f519c102200 0x7f519c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 -- 172.21.15.33:0/1847216000 >> 172.21.15.33:0/1847216000 conn(0x7f519c0fbc30 msgr2=0x7f519c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 -- 172.21.15.33:0/1847216000 shutdown_connections 2024-04-24T23:35:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.231+0000 7f51a3dee640 1 -- 172.21.15.33:0/1847216000 wait complete. 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 Processor -- start 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 -- start start 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f519c102200 0x7f519c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f519c104b30 0x7f519c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 0x7f519c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d3f0 con 0x7f519c107460 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d270 con 0x7f519c104b30 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a3dee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d570 con 0x7f519c102200 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a238b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 0x7f519c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a1b8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f519c102200 0x7f519c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a238b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 0x7f519c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54714/0 (socket says 172.21.15.33:54714) 2024-04-24T23:35:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a238b640 1 -- 172.21.15.33:0/2765459172 learned_addr learned my addr 172.21.15.33:0/2765459172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.232+0000 7f51a1389640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f519c104b30 0x7f519c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a238b640 1 -- 172.21.15.33:0/2765459172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f519c102200 msgr2=0x7f519c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a238b640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f519c102200 0x7f519c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a238b640 1 -- 172.21.15.33:0/2765459172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f519c104b30 msgr2=0x7f519c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a238b640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f519c104b30 0x7f519c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a238b640 1 -- 172.21.15.33:0/2765459172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c19f680 con 0x7f519c107460 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a238b640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 0x7f519c19f010 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f519400eb10 tx=0x7f519400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194004b20 con 0x7f519c107460 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c19f970 con 0x7f519c107460 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5194004cc0 con 0x7f519c107460 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194010760 con 0x7f519c107460 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.233+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c19fe40 con 0x7f519c107460 2024-04-24T23:35:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.234+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c0fd720 con 0x7f519c107460 2024-04-24T23:35:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.236+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5194005000 con 0x7f519c107460 2024-04-24T23:35:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.236+0000 7f5192ffd640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51880a2b90 0x7f51880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.237+0000 7f51a1b8a640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51880a2b90 0x7f51880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.237+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f51940cfb70 con 0x7f519c107460 2024-04-24T23:35:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.237+0000 7f51a1b8a640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51880a2b90 0x7f51880a5050 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5198009bd0 tx=0x7f5198005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.241+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5194053740 con 0x7f519c107460 2024-04-24T23:35:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.541+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f519c062340 con 0x7f519c107460 2024-04-24T23:35:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.542+0000 7f5192ffd640 1 -- 172.21.15.33:0/2765459172 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11861 (secure 0 0 0) 0x7f519409c740 con 0x7f519c107460 2024-04-24T23:35:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:35:59.543 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:35:37.265197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:35:59.543 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-24T23:35:59.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51880a2b90 msgr2=0x7f51880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51880a2b90 0x7f51880a5050 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5198009bd0 tx=0x7f5198005f90 comp rx=0 tx=0).stop 2024-04-24T23:35:59.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 msgr2=0x7f519c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 0x7f519c19f010 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f519400eb10 tx=0x7f519400c2d0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 shutdown_connections 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f51880a2b90 0x7f51880a5050 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f519c107460 0x7f519c19f010 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f519c104b30 0x7f519c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 --2- 172.21.15.33:0/2765459172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f519c102200 0x7f519c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 >> 172.21.15.33:0/2765459172 conn(0x7f519c0fbc30 msgr2=0x7f519c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 shutdown_connections 2024-04-24T23:35:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.546+0000 7f51a3dee640 1 -- 172.21.15.33:0/2765459172 wait complete. 2024-04-24T23:35:59.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:35:59.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:35:59 smithi038 ceph-mon[32450]: pgmap v1070: 129 pgs: 129 active+clean; 625 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 421 op/s 2024-04-24T23:35:59.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:35:59 smithi117 ceph-mon[32529]: pgmap v1070: 129 pgs: 129 active+clean; 625 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 421 op/s 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 -- 172.21.15.33:0/1156384059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c102260 msgr2=0x7f870c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 --2- 172.21.15.33:0/1156384059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c102260 0x7f870c104650 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009900 tx=0x7f86fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 -- 172.21.15.33:0/1156384059 shutdown_connections 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 --2- 172.21.15.33:0/1156384059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c1074c0 0x7f870c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 --2- 172.21.15.33:0/1156384059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c104b90 0x7f870c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 --2- 172.21.15.33:0/1156384059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c102260 0x7f870c104650 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 -- 172.21.15.33:0/1156384059 >> 172.21.15.33:0/1156384059 conn(0x7f870c0fbc30 msgr2=0x7f870c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.688+0000 7f8710929640 1 -- 172.21.15.33:0/1156384059 shutdown_connections 2024-04-24T23:35:59.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.689+0000 7f8710929640 1 -- 172.21.15.33:0/1156384059 wait complete. 2024-04-24T23:35:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.689+0000 7f8710929640 1 Processor -- start 2024-04-24T23:35:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 -- start start 2024-04-24T23:35:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 0x7f870c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c1074c0 0x7f870c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 0x7f870c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10d420 con 0x7f870c19d2b0 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10d2a0 con 0x7f870c104b90 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f870a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 0x7f870c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8710929640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10d5a0 con 0x7f870c1074c0 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 0x7f870c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 0x7f870c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54718/0 (socket says 172.21.15.33:54718) 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f8701d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c1074c0 0x7f870c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f870ad9d640 1 -- 172.21.15.33:0/2561213683 learned_addr learned my addr 172.21.15.33:0/2561213683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.690+0000 7f870a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 0x7f870c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51352/0 (socket says 172.21.15.33:51352) 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870a59c640 1 -- 172.21.15.33:0/2561213683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c1074c0 msgr2=0x7f870c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870a59c640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c1074c0 0x7f870c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870a59c640 1 -- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 msgr2=0x7f870c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870a59c640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 0x7f870c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870a59c640 1 -- 172.21.15.33:0/2561213683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c19fd40 con 0x7f870c104b90 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870ad9d640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 0x7f870c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f8701d9b640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c1074c0 0x7f870c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f870a59c640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 0x7f870c197d50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f86fc008000 tx=0x7f86fc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f8703fff640 1 -- 172.21.15.33:0/2561213683 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc03f070 con 0x7f870c104b90 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f8703fff640 1 -- 172.21.15.33:0/2561213683 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc031a20 con 0x7f870c104b90 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f8703fff640 1 -- 172.21.15.33:0/2561213683 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc035070 con 0x7f870c104b90 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c19ffd0 con 0x7f870c104b90 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.691+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a0380 con 0x7f870c104b90 2024-04-24T23:35:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.694+0000 7f8703fff640 1 -- 172.21.15.33:0/2561213683 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f86fc03b460 con 0x7f870c104b90 2024-04-24T23:35:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f8713636740 1 Processor -- start 2024-04-24T23:35:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f8703fff640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86f80a2c10 0x7f86f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f870ad9d640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86f80a2c10 0x7f86f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f8713636740 1 -- start start 2024-04-24T23:35:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f8703fff640 1 -- 172.21.15.33:0/2561213683 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f86fc0ef7d0 con 0x7f870c104b90 2024-04-24T23:35:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f8713636740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d826650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f8713636740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d826c20 0x56383d82b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f870a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d826650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.695+0000 7f870a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d826650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54732/0 (socket says 172.21.15.33:54732) 2024-04-24T23:35:59.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870ad9d640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86f80a2c10 0x7f86f80a50d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87040049d0 tx=0x7f8704009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8701d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d826c20 0x56383d82b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8701d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d826c20 0x56383d82b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42218/0 (socket says 172.21.15.33:42218) 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 -- 172.21.15.33:0/2029566058 learned_addr learned my addr 172.21.15.33:0/2029566058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8713636740 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d82bd90 0x56383d82e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56383d78a670 con 0x56383d826230 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56383d732970 con 0x56383d82bd90 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56383d786630 con 0x56383d826c20 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870ad9d640 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d82bd90 0x56383d82e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 -- 172.21.15.33:0/2029566058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d826c20 msgr2=0x56383d82b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d826c20 0x56383d82b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 -- 172.21.15.33:0/2029566058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d82bd90 msgr2=0x56383d82e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d82bd90 0x56383d82e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 -- 172.21.15.33:0/2029566058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383d756400 con 0x56383d826230 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f870a59c640 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d826650 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f86fc032c00 tx=0x7f86fc03b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc13e44dcbaa54d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8700d99640 1 -- 172.21.15.33:0/2029566058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc048040 con 0x56383d826230 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8700d99640 1 -- 172.21.15.33:0/2029566058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc0c0050 con 0x56383d826230 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.696+0000 7f8700d99640 1 -- 172.21.15.33:0/2029566058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0c46e0 con 0x56383d826230 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.697+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 msgr2=0x56383d826650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.697+0000 7f8713636740 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d826650 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f86fc032c00 tx=0x7f86fc03b800 comp rx=0 tx=0).stop 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.697+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 shutdown_connections 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d82bd90 0x56383d82e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d826c20 0x56383d82b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 --2- 172.21.15.33:0/2029566058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d826650 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 >> 172.21.15.33:0/2029566058 conn(0x56383d8218a0 msgr2=0x56383d821ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 shutdown_connections 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- 172.21.15.33:0/2029566058 wait complete. 2024-04-24T23:35:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- start start 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 Processor -- start 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d8300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826c20 0x56383d8305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d837620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56383d82eb00 con 0x56383d826230 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56383d746050 con 0x56383d826c20 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56383d82ec80 con 0x56383d82bd90 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f870a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d8300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.698+0000 7f8713636740 10 client.0 _ll_register_callbacks cb 0x56383d49dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d8300a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54748/0 (socket says 172.21.15.33:54748) 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 -- 172.21.15.33:0/1845837255 learned_addr learned my addr 172.21.15.33:0/1845837255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f8701d9b640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826c20 0x56383d8305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870ad9d640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d837620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 -- 172.21.15.33:0/1845837255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 msgr2=0x56383d837620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d837620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 -- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826c20 msgr2=0x56383d8305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826c20 0x56383d8305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 -- 172.21.15.33:0/1845837255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383d717dc0 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f8701d9b640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826c20 0x56383d8305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f870a59c640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d8300a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0ed2b0 tx=0x7f86fc0eceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b18d0a7e80861c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f86f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f86f1ffb640 1 -- 172.21.15.33:0/1845837255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc048030 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f86f1ffb640 1 -- 172.21.15.33:0/1845837255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f86fc0c4690 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86fc0c0050 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f86f1ffb640 1 -- 172.21.15.33:0/1845837255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86fc09ab30 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f8713636740 10 client.20334 fetch_fsmap learned FSMap version 35 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.699+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56383d49e3f0 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f86f1ffb640 1 -- 172.21.15.33:0/1845837255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc0c0050 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f86f1ffb640 1 -- 172.21.15.33:0/1845837255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f86fc04b060 con 0x56383d826230 2024-04-24T23:35:59.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8713636740 10 client.20334 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8713636740 10 client.20334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8713636740 20 client.20334 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8713636740 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d842310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8713636740 4 client.20334 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56383d797bb0 con 0x56383d83ff20 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8701d9b640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d842310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f86f1ffb640 10 client.20334 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:59.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.700+0000 7f8701d9b640 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d842310 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=937555732aa6ef0a server_cookie=74d477c8feb83b2f in_seq=0 out_seq=0 2024-04-24T23:35:59.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:35:59 smithi033 ceph-mon[29201]: pgmap v1070: 129 pgs: 129 active+clean; 625 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 421 op/s 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.703+0000 7f86f1ffb640 1 -- 172.21.15.33:0/1845837255 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56383d797bb0 con 0x56383d83ff20 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.703+0000 7f86f1ffb640 10 client.20334 handle_command_reply: tid=0 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.703+0000 7f8713636740 1 client.20334 shutdown 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.703+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 msgr2=0x56383d8300a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.703+0000 7f8713636740 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d8300a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0ed2b0 tx=0x7f86fc0eceb0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 shutdown_connections 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d842310 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d837620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f86f1ffb640 0 client.20334 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826c20 0x56383d8305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 --2- 172.21.15.33:0/1845837255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826230 0x56383d8300a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 >> 172.21.15.33:0/1845837255 conn(0x56383d8218a0 msgr2=0x56383d820b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 shutdown_connections 2024-04-24T23:35:59.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 1 -- 172.21.15.33:0/1845837255 wait complete. 2024-04-24T23:35:59.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.704+0000 7f8713636740 20 client.20334 trim_cache size 0 max 16384 2024-04-24T23:35:59.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 Processor -- start 2024-04-24T23:35:59.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 -- start start 2024-04-24T23:35:59.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d825040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d8375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ab020 con 0x56383d826c20 2024-04-24T23:35:59.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1a88c0 con 0x56383d826230 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8713636740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1a6710 con 0x56383d82bd90 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8701d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d8375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8701d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d8375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54754/0 (socket says 172.21.15.33:54754) 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f8701d9b640 1 -- 172.21.15.33:0/537383680 learned_addr learned my addr 172.21.15.33:0/537383680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f870a59c640 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d825040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.722+0000 7f870ad9d640 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f8701d9b640 1 -- 172.21.15.33:0/537383680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 msgr2=0x56383d8209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f8701d9b640 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f8701d9b640 1 -- 172.21.15.33:0/537383680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 msgr2=0x56383d825040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f8701d9b640 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d825040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f8701d9b640 1 -- 172.21.15.33:0/537383680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383d5687d0 con 0x56383d826c20 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f8701d9b640 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d8375e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x56383d827020 tx=0x7f86f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5823dcb1d47850d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f86f3fff640 1 -- 172.21.15.33:0/537383680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4012110 con 0x56383d826c20 2024-04-24T23:35:59.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f86f3fff640 1 -- 172.21.15.33:0/537383680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4005020 con 0x56383d826c20 2024-04-24T23:35:59.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.723+0000 7f86f3fff640 1 -- 172.21.15.33:0/537383680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40122b0 con 0x56383d826c20 2024-04-24T23:35:59.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.724+0000 7f8713636740 1 -- 172.21.15.33:0/537383680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 msgr2=0x56383d8375e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.724+0000 7f8713636740 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d8375e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x56383d827020 tx=0x7f86f4005f90 comp rx=0 tx=0).stop 2024-04-24T23:35:59.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.726+0000 7f8713636740 1 -- 172.21.15.33:0/537383680 shutdown_connections 2024-04-24T23:35:59.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.726+0000 7f8713636740 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.726+0000 7f8713636740 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d8375e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.727+0000 7f8713636740 1 --2- 172.21.15.33:0/537383680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d825040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.727+0000 7f8713636740 1 -- 172.21.15.33:0/537383680 >> 172.21.15.33:0/537383680 conn(0x56383d8218a0 msgr2=0x56383d8202b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.728+0000 7f8713636740 1 -- 172.21.15.33:0/537383680 shutdown_connections 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.728+0000 7f8713636740 1 -- 172.21.15.33:0/537383680 wait complete. 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.728+0000 7f8713636740 1 -- start start 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.729+0000 7f8713636740 1 Processor -- start 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.729+0000 7f8713636740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d8345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.729+0000 7f8713636740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d834b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.729+0000 7f8713636740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f8713636740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1a6d80 con 0x56383d826c20 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f8713636740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1a81f0 con 0x56383d826230 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f8713636740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1a7450 con 0x56383d82bd90 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f8713636740 10 client.0 _ll_register_callbacks cb 0x7f8704009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42240/0 (socket says 172.21.15.33:42240) 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 -- 172.21.15.33:0/917795326 learned_addr learned my addr 172.21.15.33:0/917795326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 -- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 msgr2=0x56383d8345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d8345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 -- 172.21.15.33:0/917795326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 msgr2=0x56383d834b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:35:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d834b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.730+0000 7f870ad9d640 1 -- 172.21.15.33:0/917795326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383d8b5950 con 0x56383d82bd90 2024-04-24T23:35:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.731+0000 7f870ad9d640 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8a43d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f870401e7a0 tx=0x7f8704004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c187f1b081fc8586 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f86f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:35:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f86f1ffb640 1 -- 172.21.15.33:0/917795326 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870400e8b0 con 0x56383d82bd90 2024-04-24T23:35:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4005020 con 0x56383d82bd90 2024-04-24T23:35:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f86f1ffb640 1 -- 172.21.15.33:0/917795326 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f870400ea50 con 0x56383d82bd90 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f86f1ffb640 1 -- 172.21.15.33:0/917795326 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8704037050 con 0x56383d82bd90 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f8713636740 10 client.29746 fetch_fsmap learned FSMap version 35 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.732+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56383d8b6d50 con 0x56383d82bd90 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f86f1ffb640 1 -- 172.21.15.33:0/917795326 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4005020 con 0x56383d82bd90 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f86f1ffb640 1 -- 172.21.15.33:0/917795326 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f87040325f0 con 0x56383d82bd90 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f8713636740 10 client.29746 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f8713636740 10 client.29746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f8713636740 20 client.29746 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f8713636740 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d8b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f8713636740 4 client.29746 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.733+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56383d791f00 con 0x56383d83ff20 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.734+0000 7f870a59c640 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d8b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.734+0000 7f86f1ffb640 10 client.29746 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:59.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.734+0000 7f870a59c640 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d8b9850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9983e239dce57bc7 server_cookie=1355c4defa895af2 in_seq=0 out_seq=0 2024-04-24T23:35:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.739+0000 7f86f1ffb640 1 -- 172.21.15.33:0/917795326 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56383d791f00 con 0x56383d83ff20 2024-04-24T23:35:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.739+0000 7f86f1ffb640 10 client.29746 handle_command_reply: tid=0 2024-04-24T23:35:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.739+0000 7f8713636740 1 client.29746 shutdown 2024-04-24T23:35:59.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.740+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 msgr2=0x56383d8a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.740+0000 7f8713636740 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8a43d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f870401e7a0 tx=0x7f8704004ab0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 shutdown_connections 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x56383d83ff20 0x56383d8b9850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56383d82bd90 0x56383d8a43d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56383d826c20 0x56383d834b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 --2- 172.21.15.33:0/917795326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56383d826230 0x56383d8345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 >> 172.21.15.33:0/917795326 conn(0x56383d8218a0 msgr2=0x56383d820b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f86f1ffb640 0 client.29746 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 shutdown_connections 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 1 -- 172.21.15.33:0/917795326 wait complete. 2024-04-24T23:35:59.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8713636740 20 client.29746 trim_cache size 0 max 16384 2024-04-24T23:35:59.743 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:35:59.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86f80a2c10 msgr2=0x7f86f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8710929640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86f80a2c10 0x7f86f80a50d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87040049d0 tx=0x7f8704009500 comp rx=0 tx=0).stop 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 msgr2=0x7f870c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.741+0000 7f8710929640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 0x7f870c197d50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f86fc008000 tx=0x7f86fc0042f0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 shutdown_connections 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f86f80a2c10 0x7f86f80a50d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c19d2b0 0x7f870c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870c1074c0 0x7f870c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 --2- 172.21.15.33:0/2561213683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f870c104b90 0x7f870c197d50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 >> 172.21.15.33:0/2561213683 conn(0x7f870c0fbc30 msgr2=0x7f870c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 shutdown_connections 2024-04-24T23:35:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.742+0000 7f8710929640 1 -- 172.21.15.33:0/2561213683 wait complete. 2024-04-24T23:35:59.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:35:59.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 -- 172.21.15.33:0/3521535289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c100680 msgr2=0x7fa48c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 --2- 172.21.15.33:0/3521535289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c100680 0x7fa48c100aa0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009900 tx=0x7fa47c02a410 comp rx=0 tx=0).stop 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 -- 172.21.15.33:0/3521535289 shutdown_connections 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 --2- 172.21.15.33:0/3521535289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c102230 0x7fa48c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 --2- 172.21.15.33:0/3521535289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa48c101870 0x7fa48c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 --2- 172.21.15.33:0/3521535289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c100680 0x7fa48c100aa0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 -- 172.21.15.33:0/3521535289 >> 172.21.15.33:0/3521535289 conn(0x7fa48c0fbbf0 msgr2=0x7fa48c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 -- 172.21.15.33:0/3521535289 shutdown_connections 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 -- 172.21.15.33:0/3521535289 wait complete. 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 Processor -- start 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.903+0000 7fa4927ee640 1 -- start start 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4927ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100680 0x7fa48c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4927ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 0x7fa48c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4927ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa48c102230 0x7fa48c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4927ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c10c300 con 0x7fa48c101870 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4927ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c10c180 con 0x7fa48c102230 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4927ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c10c480 con 0x7fa48c100680 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 0x7fa48c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 0x7fa48c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54770/0 (socket says 172.21.15.33:54770) 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2908478638 learned_addr learned my addr 172.21.15.33:0/2908478638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48bfff640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100680 0x7fa48c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa490d8b640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa48c102230 0x7fa48c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2908478638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100680 msgr2=0x7fa48c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100680 0x7fa48c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2908478638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa48c102230 msgr2=0x7fa48c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa48c102230 0x7fa48c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2908478638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c19f750 con 0x7fa48c101870 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 0x7fa48c19c820 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e9f0 tx=0x7fa48000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.904+0000 7fa4897fa640 1 -- 172.21.15.33:0/2908478638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48000cd80 con 0x7fa48c101870 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.905+0000 7fa4897fa640 1 -- 172.21.15.33:0/2908478638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa480010910 con 0x7fa48c101870 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.905+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c19f9e0 con 0x7fa48c101870 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.905+0000 7fa4897fa640 1 -- 172.21.15.33:0/2908478638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480018990 con 0x7fa48c101870 2024-04-24T23:35:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.905+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c19ff30 con 0x7fa48c101870 2024-04-24T23:35:59.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 251274 21.92 MB/sec execute 484 sec latency 72.658 ms 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4897fa640 1 -- 172.21.15.33:0/2908478638 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa480010ac0 con 0x7fa48c101870 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 Processor -- start 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 -- start start 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd82720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4897fa640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4740a2b90 0x7fa4740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd85220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d41cce1670 con 0x55d41cd82d90 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d41cc89970 con 0x55d41cd7d230 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.907+0000 7fa4955db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d41ccdd630 con 0x55d41cd7dc20 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd82720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd85220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd82720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42260/0 (socket says 172.21.15.33:42260) 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2788744506 learned_addr learned my addr 172.21.15.33:0/2788744506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48bfff640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4740a2b90 0x7fa4740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa4897fa640 1 -- 172.21.15.33:0/2908478638 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4800cf6e0 con 0x7fa48c101870 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2788744506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 msgr2=0x55d41cd7d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2788744506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 msgr2=0x55d41cd85220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd85220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 -- 172.21.15.33:0/2788744506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41ccad400 con 0x55d41cd7dc20 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa490d8b640 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd82720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa480098630 tx=0x7fa480018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a979b33643fecf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa48bfff640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4740a2b90 0x7fa4740a5050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005ab0 tx=0x7fa47c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa4727fc640 1 -- 172.21.15.33:0/2788744506 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480026040 con 0x55d41cd7dc20 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa4727fc640 1 -- 172.21.15.33:0/2788744506 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48009ff60 con 0x55d41cd7dc20 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.908+0000 7fa4727fc640 1 -- 172.21.15.33:0/2788744506 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4800c0930 con 0x55d41cd7dc20 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 -- 172.21.15.33:0/2788744506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 msgr2=0x55d41cd82720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd82720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa480098630 tx=0x7fa480018420 comp rx=0 tx=0).stop 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 -- 172.21.15.33:0/2788744506 shutdown_connections 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd85220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd82720 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 --2- 172.21.15.33:0/2788744506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 -- 172.21.15.33:0/2788744506 >> 172.21.15.33:0/2788744506 conn(0x55d41cd788a0 msgr2=0x55d41cd78ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 -- 172.21.15.33:0/2788744506 shutdown_connections 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 -- 172.21.15.33:0/2788744506 wait complete. 2024-04-24T23:35:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.909+0000 7fa4955db740 1 -- start start 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 Processor -- start 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7d230 0x55d41cd87380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7dc20 0x55d41cd878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d41cd85ce0 con 0x55d41cd82d90 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d41cc9d050 con 0x55d41cd7dc20 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d41cd85e60 con 0x55d41cd7d230 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa4955db740 10 client.0 _ll_register_callbacks cb 0x55d41c9f4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7d230 0x55d41cd87380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7dc20 0x55d41cd878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7d230 0x55d41cd87380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42268/0 (socket says 172.21.15.33:42268) 2024-04-24T23:35:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd8e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54794/0 (socket says 172.21.15.33:54794) 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa48bfff640 1 -- 172.21.15.33:0/3573628044 learned_addr learned my addr 172.21.15.33:0/3573628044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 -- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7d230 msgr2=0x55d41cd87380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7d230 0x55d41cd87380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 -- 172.21.15.33:0/3573628044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7dc20 msgr2=0x55d41cd878c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7dc20 0x55d41cd878c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa490d8b640 1 -- 172.21.15.33:0/3573628044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41cc6edc0 con 0x55d41cd82d90 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.910+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7dc20 0x55d41cd878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa490d8b640 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd8e900 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca9636076d2810a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa46b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3573628044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484019040 con 0x55d41cd82d90 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3573628044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa484004b20 con 0x55d41cd82d90 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48009ff60 con 0x55d41cd82d90 2024-04-24T23:35:59.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3573628044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa484029050 con 0x55d41cd82d90 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 10 client.20349 fetch_fsmap learned FSMap version 35 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d41c9f53f0 con 0x55d41cd82d90 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3573628044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48009ff60 con 0x55d41cd82d90 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3573628044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fa48401d560 con 0x55d41cd82d90 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 10 client.20349 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 10 client.20349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 20 client.20349 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd932a0 0x55d41cd95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 4 client.20349 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.911+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d41cceebb0 con 0x55d41cd932a0 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.912+0000 7fa48bfff640 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd932a0 0x55d41cd95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.912+0000 7fa48bfff640 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd932a0 0x55d41cd95720 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f85eda9c5556530 server_cookie=aafb77de9b0a3e64 in_seq=0 out_seq=0 2024-04-24T23:35:59.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.912+0000 7fa46b7fe640 10 client.20349 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3573628044 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d41cceebb0 con 0x55d41cd932a0 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa46b7fe640 10 client.20349 handle_command_reply: tid=0 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 client.20349 shutdown 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 msgr2=0x55d41cd8e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd8e900 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48400e780 comp rx=0 tx=0).stop 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 shutdown_connections 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd932a0 0x55d41cd95720 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd82d90 0x55d41cd8e900 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7dc20 0x55d41cd878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa46b7fe640 0 client.20349 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 --2- 172.21.15.33:0/3573628044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7d230 0x55d41cd87380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 >> 172.21.15.33:0/3573628044 conn(0x55d41cd788a0 msgr2=0x55d41cd77b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 shutdown_connections 2024-04-24T23:35:59.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 1 -- 172.21.15.33:0/3573628044 wait complete. 2024-04-24T23:35:59.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.913+0000 7fa4955db740 20 client.20349 trim_cache size 0 max 16384 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 Processor -- start 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 -- start start 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd788a0 0x55d41cd7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd8e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1aabf0 con 0x55d41cd788a0 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a8490 con 0x55d41cd7d230 2024-04-24T23:35:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.928+0000 7fa4955db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a62e0 con 0x55d41cd7dc20 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd8e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd788a0 0x55d41cd7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd8e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42272/0 (socket says 172.21.15.33:42272) 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51434/0 (socket says 172.21.15.33:51434) 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 -- 172.21.15.33:0/3225823720 learned_addr learned my addr 172.21.15.33:0/3225823720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa48bfff640 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd788a0 0x55d41cd7edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54802/0 (socket says 172.21.15.33:54802) 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 -- 172.21.15.33:0/3225823720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 msgr2=0x55d41cd7f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 -- 172.21.15.33:0/3225823720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd788a0 msgr2=0x55d41cd7edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd788a0 0x55d41cd7edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 -- 172.21.15.33:0/3225823720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41cabf7d0 con 0x55d41cd7dc20 2024-04-24T23:35:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa490d8b640 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd8e8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa484000f80 tx=0x7fa4840105e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11908a030a67c189 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa471ffb640 1 -- 172.21.15.33:0/3225823720 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48401dc40 con 0x55d41cd7dc20 2024-04-24T23:35:59.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa471ffb640 1 -- 172.21.15.33:0/3225823720 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484032070 con 0x55d41cd7dc20 2024-04-24T23:35:59.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.929+0000 7fa471ffb640 1 -- 172.21.15.33:0/3225823720 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48402d7a0 con 0x55d41cd7dc20 2024-04-24T23:35:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 -- 172.21.15.33:0/3225823720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 msgr2=0x55d41cd8e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd8e8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa484000f80 tx=0x7fa4840105e0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 -- 172.21.15.33:0/3225823720 shutdown_connections 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd7dc20 0x55d41cd8e8c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd7f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 --2- 172.21.15.33:0/3225823720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd788a0 0x55d41cd7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 -- 172.21.15.33:0/3225823720 >> 172.21.15.33:0/3225823720 conn(0x55d41cd86f40 msgr2=0x55d41cd7e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 -- 172.21.15.33:0/3225823720 shutdown_connections 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 -- 172.21.15.33:0/3225823720 wait complete. 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.930+0000 7fa4955db740 1 -- start start 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 Processor -- start 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd788a0 0x55d41cd8c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 0x55d41cd8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a6950 con 0x55d41cd7dc20 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a7dc0 con 0x55d41cd7d230 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a7020 con 0x55d41cd788a0 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa4955db740 10 client.0 _ll_register_callbacks cb 0x7fa484010bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 0x55d41cd8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 0x55d41cd8d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54804/0 (socket says 172.21.15.33:54804) 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 -- 172.21.15.33:0/537167427 learned_addr learned my addr 172.21.15.33:0/537167427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa48b7fe640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa48bfff640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd788a0 0x55d41cd8c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 -- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd788a0 msgr2=0x55d41cd8c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd788a0 0x55d41cd8c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 -- 172.21.15.33:0/537167427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 msgr2=0x55d41cd8c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:35:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd8c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 -- 172.21.15.33:0/537167427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41cd8dfc0 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa490d8b640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 0x55d41cd8d9d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa48401daf0 tx=0x7fa484010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0605cb34b4d687f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.931+0000 7fa46b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/537167427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48403a030 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/537167427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa48401d3a0 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa484032070 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/537167427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa484044050 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 10 client.20355 fetch_fsmap learned FSMap version 35 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d41cdf9d20 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/537167427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484032070 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/537167427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fa484004040 con 0x55d41cd7dc20 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 10 client.20355 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:35:59.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 10 client.20355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 20 client.20355 populate_metadata read hostname 'smithi033' 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd82d90 0x55d41cdfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 4 client.20355 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d41cce8f00 con 0x55d41cd82d90 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.932+0000 7fa48bfff640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd82d90 0x55d41cdfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa48bfff640 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd82d90 0x55d41cdfc640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=768e4e2c97252037 server_cookie=87d0414e53a02166 in_seq=0 out_seq=0 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa46b7fe640 10 client.20355 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa46b7fe640 1 -- 172.21.15.33:0/537167427 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d41cce8f00 con 0x55d41cd82d90 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa46b7fe640 10 client.20355 handle_command_reply: tid=0 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 client.20355 shutdown 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 msgr2=0x55d41cd8d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 0x55d41cd8d9d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa48401daf0 tx=0x7fa484010c80 comp rx=0 tx=0).stop 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 shutdown_connections 2024-04-24T23:35:59.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d41cd82d90 0x55d41cdfc640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d41cd7dc20 0x55d41cd8d9d0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa46b7fe640 0 client.20355 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d41cd7d230 0x55d41cd8c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 --2- 172.21.15.33:0/537167427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d41cd788a0 0x55d41cd8c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 >> 172.21.15.33:0/537167427 conn(0x55d41cd86f40 msgr2=0x55d41cd7ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 shutdown_connections 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 1 -- 172.21.15.33:0/537167427 wait complete. 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.933+0000 7fa4955db740 20 client.20355 trim_cache size 0 max 16384 2024-04-24T23:35:59.940 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:35:59.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4740a2b90 msgr2=0x7fa4740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4740a2b90 0x7fa4740a5050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005ab0 tx=0x7fa47c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 msgr2=0x7fa48c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 0x7fa48c19c820 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e9f0 tx=0x7fa48000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 shutdown_connections 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa4740a2b90 0x7fa4740a5050 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa48c102230 0x7fa48c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48c101870 0x7fa48c19c820 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 --2- 172.21.15.33:0/2908478638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100680 0x7fa48c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 >> 172.21.15.33:0/2908478638 conn(0x7fa48c0fbbf0 msgr2=0x7fa48c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 shutdown_connections 2024-04-24T23:35:59.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:35:59.934+0000 7fa4927ee640 1 -- 172.21.15.33:0/2908478638 wait complete. 2024-04-24T23:35:59.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:36:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 -- 172.21.15.33:0/820022317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790103350 msgr2=0x7f3790109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 --2- 172.21.15.33:0/820022317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790103350 0x7f3790109be0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3788009860 tx=0x7f378802a340 comp rx=0 tx=0).stop 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 -- 172.21.15.33:0/820022317 shutdown_connections 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 --2- 172.21.15.33:0/820022317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790103350 0x7f3790109be0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 --2- 172.21.15.33:0/820022317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3790102990 0x7f3790102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 --2- 172.21.15.33:0/820022317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37901017a0 0x7f3790101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 -- 172.21.15.33:0/820022317 >> 172.21.15.33:0/820022317 conn(0x7f37900fcd10 msgr2=0x7f37900ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 -- 172.21.15.33:0/820022317 shutdown_connections 2024-04-24T23:36:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.067+0000 7f3798464640 1 -- 172.21.15.33:0/820022317 wait complete. 2024-04-24T23:36:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 Processor -- start 2024-04-24T23:36:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 -- start start 2024-04-24T23:36:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 0x7f37900754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3790102990 0x7f3790073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3790103350 0x7f3790074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f379010d400 con 0x7f37901017a0 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f379010d280 con 0x7f3790102990 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3798464640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f379010d580 con 0x7f3790103350 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 0x7f37900754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 0x7f37900754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54822/0 (socket says 172.21.15.33:54822) 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 -- 172.21.15.33:0/2670298802 learned_addr learned my addr 172.21.15.33:0/2670298802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f37959ff640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3790102990 0x7f3790073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796a01640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3790103350 0x7f3790074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 -- 172.21.15.33:0/2670298802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3790103350 msgr2=0x7f3790074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3790103350 0x7f3790074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 -- 172.21.15.33:0/2670298802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3790102990 msgr2=0x7f3790073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3790102990 0x7f3790073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.068+0000 7f3796200640 1 -- 172.21.15.33:0/2670298802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3790074900 con 0x7f37901017a0 2024-04-24T23:36:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.069+0000 7f3796200640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 0x7f37900754f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f378c009d20 tx=0x7f378c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.069+0000 7f37877fe640 1 -- 172.21.15.33:0/2670298802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c010630 con 0x7f37901017a0 2024-04-24T23:36:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.069+0000 7f37877fe640 1 -- 172.21.15.33:0/2670298802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378c010cb0 con 0x7f37901017a0 2024-04-24T23:36:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.069+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37901a1b60 con 0x7f37901017a0 2024-04-24T23:36:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.069+0000 7f37877fe640 1 -- 172.21.15.33:0/2670298802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c018de0 con 0x7f37901017a0 2024-04-24T23:36:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.069+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37901a20b0 con 0x7f37901017a0 2024-04-24T23:36:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.072+0000 7f37877fe640 1 -- 172.21.15.33:0/2670298802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f378c0107d0 con 0x7f37901017a0 2024-04-24T23:36:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 Processor -- start 2024-04-24T23:36:00.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 -- start start 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f37877fe640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f377c0a2b90 0x7f377c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x5593461bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559346118670 con 0x5593461b9d90 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5593460c0970 con 0x5593461b4230 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f379b174740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559346114630 con 0x5593461b4c20 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f3796a01640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f377c0a2b90 0x7f377c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f37877fe640 1 -- 172.21.15.33:0/2670298802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f378c0cfbf0 con 0x7f37901017a0 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f3796a01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x5593461bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f3796a01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x5593461bc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54832/0 (socket says 172.21.15.33:54832) 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f3796a01640 1 -- 172.21.15.33:0/3377334317 learned_addr learned my addr 172.21.15.33:0/3377334317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f37959ff640 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.073+0000 7f3796200640 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f37959ff640 1 -- 172.21.15.33:0/3377334317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 msgr2=0x5593461b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f37959ff640 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f37959ff640 1 -- 172.21.15.33:0/3377334317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 msgr2=0x5593461bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f37959ff640 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x5593461bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f37959ff640 1 -- 172.21.15.33:0/3377334317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593460e4400 con 0x5593461b4c20 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f3796a01640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f377c0a2b90 0x7f377c0a5050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3788030410 tx=0x7f3788005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f37959ff640 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461b9720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3780009530 tx=0x7f3780011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef0ffbffc8969f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f3784bfe640 1 -- 172.21.15.33:0/3377334317 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780004020 con 0x5593461b4c20 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f3784bfe640 1 -- 172.21.15.33:0/3377334317 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37800041c0 con 0x5593461b4c20 2024-04-24T23:36:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.074+0000 7f3784bfe640 1 -- 172.21.15.33:0/3377334317 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780019890 con 0x5593461b4c20 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 -- 172.21.15.33:0/3377334317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 msgr2=0x5593461b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461b9720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3780009530 tx=0x7f3780011810 comp rx=0 tx=0).stop 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 -- 172.21.15.33:0/3377334317 shutdown_connections 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x5593461bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461b9720 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 --2- 172.21.15.33:0/3377334317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 -- 172.21.15.33:0/3377334317 >> 172.21.15.33:0/3377334317 conn(0x5593461af8a0 msgr2=0x5593461afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 -- 172.21.15.33:0/3377334317 shutdown_connections 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.075+0000 7f379b174740 1 -- 172.21.15.33:0/3377334317 wait complete. 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 -- start start 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 Processor -- start 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 0x5593461be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b9d90 0x5593461c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593461bcd40 con 0x5593461b4c20 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5593460d4050 con 0x5593461b4230 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5593461bcec0 con 0x5593461b9d90 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f379b174740 10 client.0 _ll_register_callbacks cb 0x559345e2bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f37959ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 0x5593461be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f3796200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f37959ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 0x5593461be890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54846/0 (socket says 172.21.15.33:54846) 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f37959ff640 1 -- 172.21.15.33:0/3580805150 learned_addr learned my addr 172.21.15.33:0/3580805150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f3796200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461be350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51476/0 (socket says 172.21.15.33:51476) 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f3796a01640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b9d90 0x5593461c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f37959ff640 1 -- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b9d90 msgr2=0x5593461c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.076+0000 7f37959ff640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b9d90 0x5593461c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f37959ff640 1 -- 172.21.15.33:0/3580805150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 msgr2=0x5593461be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f37959ff640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f37959ff640 1 -- 172.21.15.33:0/3580805150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593460a5dc0 con 0x5593461b4c20 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f3796a01640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b9d90 0x5593461c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f3796200640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f37959ff640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 0x5593461be890 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3780000c00 tx=0x7f3780005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1792dfc978f92694 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f3779ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f3779ffb640 1 -- 172.21.15.33:0/3580805150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780028070 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f3779ffb640 1 -- 172.21.15.33:0/3580805150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3780003fc0 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37800041c0 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f3779ffb640 1 -- 172.21.15.33:0/3580805150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3780034050 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f379b174740 10 client.20367 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.077+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559345e2c3f0 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f3779ffb640 1 -- 172.21.15.33:0/3580805150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37800041c0 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f3779ffb640 1 -- 172.21.15.33:0/3580805150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f3780004c20 con 0x5593461b4c20 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f379b174740 10 client.20367 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f379b174740 10 client.20367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f379b174740 20 client.20367 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f379b174740 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5593461c9ff0 0x5593461cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f379b174740 4 client.20367 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559346125bb0 con 0x5593461c9ff0 2024-04-24T23:36:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f3796200640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5593461c9ff0 0x5593461cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f3779ffb640 10 client.20367 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.078+0000 7f3796200640 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5593461c9ff0 0x5593461cc450 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe8b90aaff73519 server_cookie=632f7fcd7a22c7e3 in_seq=0 out_seq=0 2024-04-24T23:36:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.080+0000 7f3779ffb640 1 -- 172.21.15.33:0/3580805150 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559346125bb0 con 0x5593461c9ff0 2024-04-24T23:36:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.080+0000 7f3779ffb640 10 client.20367 handle_command_reply: tid=0 2024-04-24T23:36:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.080+0000 7f379b174740 1 client.20367 shutdown 2024-04-24T23:36:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 msgr2=0x5593461be890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 0x5593461be890 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3780000c00 tx=0x7f3780005e10 comp rx=0 tx=0).stop 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 shutdown_connections 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5593461c9ff0 0x5593461cc450 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b9d90 0x5593461c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f3779ffb640 0 client.20367 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4c20 0x5593461be890 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 --2- 172.21.15.33:0/3580805150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 >> 172.21.15.33:0/3580805150 conn(0x5593461af8a0 msgr2=0x5593461aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 shutdown_connections 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 1 -- 172.21.15.33:0/3580805150 wait complete. 2024-04-24T23:36:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.081+0000 7f379b174740 20 client.20367 trim_cache size 0 max 16384 2024-04-24T23:36:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 Processor -- start 2024-04-24T23:36:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 -- start start 2024-04-24T23:36:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 0x5593461b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b9d90 0x5593461b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37901aef00 con 0x5593461b4230 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37901ac7a0 con 0x5593461b9d90 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.104+0000 7f379b174740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37901aa5f0 con 0x5593461b4c20 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 0x5593461b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f37959ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796a01640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b9d90 0x5593461b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 0x5593461b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54852/0 (socket says 172.21.15.33:54852) 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 -- 172.21.15.33:0/1084285489 learned_addr learned my addr 172.21.15.33:0/1084285489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 -- 172.21.15.33:0/1084285489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 msgr2=0x5593461c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 -- 172.21.15.33:0/1084285489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b9d90 msgr2=0x5593461b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b9d90 0x5593461b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 -- 172.21.15.33:0/1084285489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559345ef67d0 con 0x5593461b4230 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f3796200640 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 0x5593461b2780 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f378c0cd6d0 tx=0x7f378c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d6e3ccc4b33f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f377bfff640 1 -- 172.21.15.33:0/1084285489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0a0470 con 0x5593461b4230 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f377bfff640 1 -- 172.21.15.33:0/1084285489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378c0a4d30 con 0x5593461b4230 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.105+0000 7f377bfff640 1 -- 172.21.15.33:0/1084285489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0a0120 con 0x5593461b4230 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 -- 172.21.15.33:0/1084285489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 msgr2=0x5593461b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 0x5593461b2780 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f378c0cd6d0 tx=0x7f378c01a040 comp rx=0 tx=0).stop 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 -- 172.21.15.33:0/1084285489 shutdown_connections 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b9d90 0x5593461b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 --2- 172.21.15.33:0/1084285489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b4230 0x5593461b2780 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 -- 172.21.15.33:0/1084285489 >> 172.21.15.33:0/1084285489 conn(0x5593461af8a0 msgr2=0x5593461aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 -- 172.21.15.33:0/1084285489 shutdown_connections 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.106+0000 7f379b174740 1 -- 172.21.15.33:0/1084285489 wait complete. 2024-04-24T23:36:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 -- start start 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 Processor -- start 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x55934622e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37901aac60 con 0x5593461b9d90 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37901ac0d0 con 0x5593461b4230 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37901ab330 con 0x5593461b4c20 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f379b174740 10 client.0 _ll_register_callbacks cb 0x7f3780011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x55934622e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x55934622e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54862/0 (socket says 172.21.15.33:54862) 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 -- 172.21.15.33:0/4247737481 learned_addr learned my addr 172.21.15.33:0/4247737481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796200640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f37959ff640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 -- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 msgr2=0x5593461c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 -- 172.21.15.33:0/4247737481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 msgr2=0x5593461c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.107+0000 7f3796a01640 1 -- 172.21.15.33:0/4247737481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934622eaf0 con 0x5593461b9d90 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3796a01640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x55934622e580 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3788009860 tx=0x7f3788044040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4287794a8eb650 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3779ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3779ffb640 1 -- 172.21.15.33:0/4247737481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788037040 con 0x5593461b9d90 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3779ffb640 1 -- 172.21.15.33:0/4247737481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3788004c50 con 0x5593461b9d90 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f378c0a4d30 con 0x5593461b9d90 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3779ffb640 1 -- 172.21.15.33:0/4247737481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f378805c050 con 0x5593461b9d90 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 10 client.20373 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55934622fe40 con 0x5593461b9d90 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3779ffb640 1 -- 172.21.15.33:0/4247737481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378c0a4d30 con 0x5593461b9d90 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f3779ffb640 1 -- 172.21.15.33:0/4247737481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f378803d050 con 0x5593461b9d90 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 10 client.20373 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 10 client.20373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 20 client.20373 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559346230870 0x559346232c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 4 client.20373 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.108+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593461be140 con 0x559346230870 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f3796200640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559346230870 0x559346232c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f3796200640 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559346230870 0x559346232c90 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4823700c7c9619b5 server_cookie=25c50b043f98e532 in_seq=0 out_seq=0 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f3779ffb640 10 client.20373 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f3779ffb640 1 -- 172.21.15.33:0/4247737481 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593461be140 con 0x559346230870 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f3779ffb640 10 client.20373 handle_command_reply: tid=0 2024-04-24T23:36:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 client.20373 shutdown 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 msgr2=0x55934622e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x55934622e580 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3788009860 tx=0x7f3788044040 comp rx=0 tx=0).stop 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 shutdown_connections 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559346230870 0x559346232c90 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593461b9d90 0x55934622e580 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5593461b4c20 0x5593461c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f3779ffb640 0 client.20373 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 --2- 172.21.15.33:0/4247737481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593461b4230 0x5593461c28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 >> 172.21.15.33:0/4247737481 conn(0x5593461af8a0 msgr2=0x5593461b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 shutdown_connections 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.109+0000 7f379b174740 1 -- 172.21.15.33:0/4247737481 wait complete. 2024-04-24T23:36:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f379b174740 20 client.20373 trim_cache size 0 max 16384 2024-04-24T23:36:00.118 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f377c0a2b90 msgr2=0x7f377c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f377c0a2b90 0x7f377c0a5050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3788030410 tx=0x7f3788005d40 comp rx=0 tx=0).stop 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 msgr2=0x7f37900754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 0x7f37900754f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f378c009d20 tx=0x7f378c00c6a0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 shutdown_connections 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f377c0a2b90 0x7f377c0a5050 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3790103350 0x7f3790074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3790102990 0x7f3790073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 --2- 172.21.15.33:0/2670298802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37901017a0 0x7f37900754f0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 >> 172.21.15.33:0/2670298802 conn(0x7f37900fcd10 msgr2=0x7f3790105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 shutdown_connections 2024-04-24T23:36:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:00.110+0000 7f3798464640 1 -- 172.21.15.33:0/2670298802 wait complete. 2024-04-24T23:36:00.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:00 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2765459172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:36:00.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:00 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2765459172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:36:00.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:00 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2765459172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:36:00.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 251658 21.91 MB/sec execute 485 sec latency 91.427 ms 2024-04-24T23:36:01.126 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:36:01.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:36:01.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 -- 172.21.15.33:0/1845743727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac80ffe00 msgr2=0x7fcac8100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 --2- 172.21.15.33:0/1845743727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac80ffe00 0x7fcac8100220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009860 tx=0x7fcac402a340 comp rx=0 tx=0).stop 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 -- 172.21.15.33:0/1845743727 shutdown_connections 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 --2- 172.21.15.33:0/1845743727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac81019b0 0x7fcac8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 --2- 172.21.15.33:0/1845743727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac8100ff0 0x7fcac8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 --2- 172.21.15.33:0/1845743727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac80ffe00 0x7fcac8100220 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009860 tx=0x7fcac402a340 comp rx=0 tx=0).stop 2024-04-24T23:36:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 -- 172.21.15.33:0/1845743727 >> 172.21.15.33:0/1845743727 conn(0x7fcac80fb370 msgr2=0x7fcac80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 -- 172.21.15.33:0/1845743727 shutdown_connections 2024-04-24T23:36:01.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.253+0000 7fcacf464640 1 -- 172.21.15.33:0/1845743727 wait complete. 2024-04-24T23:36:01.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.254+0000 7fcacf464640 1 Processor -- start 2024-04-24T23:36:01.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 -- start start 2024-04-24T23:36:01.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 0x7fcac8110e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac81019b0 0x7fcac8111360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac8116370 0x7fcac81ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810acc0 con 0x7fcac81019b0 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810ab40 con 0x7fcac8116370 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacf464640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810ae40 con 0x7fcac8100ff0 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 0x7fcac8110e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacc9ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac81019b0 0x7fcac8111360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 0x7fcac8110e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42362/0 (socket says 172.21.15.33:42362) 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacd200640 1 -- 172.21.15.33:0/2090023923 learned_addr learned my addr 172.21.15.33:0/2090023923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacda01640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac8116370 0x7fcac81ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacc9ff640 1 -- 172.21.15.33:0/2090023923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 msgr2=0x7fcac8110e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 0x7fcac8110e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacc9ff640 1 -- 172.21.15.33:0/2090023923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac8116370 msgr2=0x7fcac81ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac8116370 0x7fcac81ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.255+0000 7fcacc9ff640 1 -- 172.21.15.33:0/2090023923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81188b0 con 0x7fcac81019b0 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcacd200640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 0x7fcac8110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac81019b0 0x7fcac8111360 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fcab800ea60 tx=0x7fcab800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcabe7fc640 1 -- 172.21.15.33:0/2090023923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab800cd40 con 0x7fcac81019b0 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcabe7fc640 1 -- 172.21.15.33:0/2090023923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab80035f0 con 0x7fcac81019b0 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcabe7fc640 1 -- 172.21.15.33:0/2090023923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8016670 con 0x7fcac81019b0 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81aee10 con 0x7fcac81019b0 2024-04-24T23:36:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.256+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81af220 con 0x7fcac81019b0 2024-04-24T23:36:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.260+0000 7fcad2174740 1 Processor -- start 2024-04-24T23:36:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.260+0000 7fcabe7fc640 1 -- 172.21.15.33:0/2090023923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcab801d020 con 0x7fcac81019b0 2024-04-24T23:36:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.261+0000 7fcad2174740 1 -- start start 2024-04-24T23:36:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcad2174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a024ff71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcabe7fc640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcab40a2c10 0x7fcab40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcad2174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a024ffc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcad2174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a024ffed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcad2174740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a024f5a2c0 con 0x55a024ff7780 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcad2174740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a024f02910 con 0x55a024ff6d90 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcad2174740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a024f56280 con 0x55a024ffc8f0 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a024ff71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacda01640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcab40a2c10 0x7fcab40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a024ff71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51518/0 (socket says 172.21.15.33:51518) 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 -- 172.21.15.33:0/2052716371 learned_addr learned my addr 172.21.15.33:0/2052716371 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a024ffc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcabe7fc640 1 -- 172.21.15.33:0/2090023923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcab80cee10 con 0x7fcac81019b0 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacda01640 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a024ffed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 -- 172.21.15.33:0/2052716371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 msgr2=0x55a024ffed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a024ffed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 -- 172.21.15.33:0/2052716371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 msgr2=0x55a024ffc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a024ffc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 -- 172.21.15.33:0/2052716371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a024f13fb0 con 0x55a024ff6d90 2024-04-24T23:36:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.262+0000 7fcacd200640 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a024ff71b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcac8100e50 tx=0x7fcac4009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f3b99e08733b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.263+0000 7fcacda01640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcab40a2c10 0x7fcab40a50d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcac00098b0 tx=0x7fcac0009e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.263+0000 7fcab37fe640 1 -- 172.21.15.33:0/2052716371 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037070 con 0x55a024ff6d90 2024-04-24T23:36:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.263+0000 7fcab37fe640 1 -- 172.21.15.33:0/2052716371 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac4038040 con 0x55a024ff6d90 2024-04-24T23:36:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.263+0000 7fcab37fe640 1 -- 172.21.15.33:0/2052716371 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40046c0 con 0x55a024ff6d90 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 -- 172.21.15.33:0/2052716371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 msgr2=0x55a024ff71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a024ff71b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcac8100e50 tx=0x7fcac4009510 comp rx=0 tx=0).stop 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 -- 172.21.15.33:0/2052716371 shutdown_connections 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a024ffed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a024ffc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 --2- 172.21.15.33:0/2052716371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a024ff71b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 -- 172.21.15.33:0/2052716371 >> 172.21.15.33:0/2052716371 conn(0x55a024fbe950 msgr2=0x55a024ff2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 -- 172.21.15.33:0/2052716371 shutdown_connections 2024-04-24T23:36:01.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 -- 172.21.15.33:0/2052716371 wait complete. 2024-04-24T23:36:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 -- start start 2024-04-24T23:36:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.264+0000 7fcad2174740 1 Processor -- start 2024-04-24T23:36:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025000f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 0x55a025001480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a0250084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a024fff840 con 0x55a024ffc8f0 2024-04-24T23:36:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a024f15e90 con 0x55a024ff6d90 2024-04-24T23:36:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a024fff9c0 con 0x55a024ff7780 2024-04-24T23:36:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcad2174740 10 client.0 _ll_register_callbacks cb 0x55a024f51970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 0x55a025001480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 0x55a025001480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42382/0 (socket says 172.21.15.33:42382) 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 -- 172.21.15.33:0/657129682 learned_addr learned my addr 172.21.15.33:0/657129682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacda01640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a0250084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacd200640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025000f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 -- 172.21.15.33:0/657129682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 msgr2=0x55a025000f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025000f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 -- 172.21.15.33:0/657129682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 msgr2=0x55a0250084c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a0250084c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.265+0000 7fcacc9ff640 1 -- 172.21.15.33:0/657129682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a024d367e0 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 0x55a025001480 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcab8004770 tx=0x7fcab8099790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=675e4b3721c0952c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcab0ff9640 1 -- 172.21.15.33:0/657129682 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab806b8f0 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac4038040 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcab0ff9640 1 -- 172.21.15.33:0/657129682 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcab80978f0 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcab0ff9640 1 -- 172.21.15.33:0/657129682 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcab805f110 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcad2174740 10 client.29794 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a024d96f70 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.266+0000 7fcab0ff9640 1 -- 172.21.15.33:0/657129682 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac4038040 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcab0ff9640 1 -- 172.21.15.33:0/657129682 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fcab800b040 con 0x55a024ff7780 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcad2174740 10 client.29794 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcad2174740 10 client.29794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcad2174740 20 client.29794 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcad2174740 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a02500ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcad2174740 4 client.29794 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a024f2e5a0 con 0x55a02500ca20 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcacd200640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a02500ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcacd200640 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a02500ee60 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfccacf32dad6ad1 server_cookie=cd0484f29359e862 in_seq=0 out_seq=0 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.267+0000 7fcab0ff9640 10 client.29794 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcab0ff9640 1 -- 172.21.15.33:0/657129682 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a024f2e5a0 con 0x55a02500ca20 2024-04-24T23:36:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcab0ff9640 10 client.29794 handle_command_reply: tid=0 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 client.29794 shutdown 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 msgr2=0x55a025001480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 0x55a025001480 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcab8004770 tx=0x7fcab8099790 comp rx=0 tx=0).stop 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 shutdown_connections 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a02500ee60 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a0250084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff7780 0x55a025001480 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcab0ff9640 0 client.29794 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 --2- 172.21.15.33:0/657129682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025000f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.268+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 >> 172.21.15.33:0/657129682 conn(0x55a024fbe950 msgr2=0x55a024ff1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.269+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 shutdown_connections 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.269+0000 7fcad2174740 1 -- 172.21.15.33:0/657129682 wait complete. 2024-04-24T23:36:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.269+0000 7fcad2174740 20 client.29794 trim_cache size 0 max 16384 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcad2174740 1 Processor -- start 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcad2174740 1 -- start start 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcad2174740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 0x55a024ff29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcad2174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff7780 0x55a025008480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcacd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 0x55a024ff29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcacd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 0x55a024ff29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42386/0 (socket says 172.21.15.33:42386) 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcacd200640 1 -- 172.21.15.33:0/1086137380 learned_addr learned my addr 172.21.15.33:0/1086137380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcad2174740 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a024ff5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.300+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81bc170 con 0x55a024ffc8f0 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b9a10 con 0x55a024ff7780 2024-04-24T23:36:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b7860 con 0x55a024ff6d90 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a024ff5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 -- 172.21.15.33:0/1086137380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 msgr2=0x55a024ff29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 0x55a024ff29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 -- 172.21.15.33:0/1086137380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff7780 msgr2=0x55a025008480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff7780 0x55a025008480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 -- 172.21.15.33:0/1086137380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a024c2e170 con 0x55a024ffc8f0 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacd200640 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 0x55a024ff29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcacda01640 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a024ff5740 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009fd0 tx=0x7fcac001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64640d41b7b608a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.301+0000 7fcab2ffd640 1 -- 172.21.15.33:0/1086137380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0004b20 con 0x55a024ffc8f0 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.302+0000 7fcab2ffd640 1 -- 172.21.15.33:0/1086137380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac0004cc0 con 0x55a024ffc8f0 2024-04-24T23:36:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.302+0000 7fcab2ffd640 1 -- 172.21.15.33:0/1086137380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0005360 con 0x55a024ffc8f0 2024-04-24T23:36:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.303+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 msgr2=0x55a024ff5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.303+0000 7fcad2174740 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a024ff5740 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009fd0 tx=0x7fcac001e000 comp rx=0 tx=0).stop 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 shutdown_connections 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ffc8f0 0x55a024ff5740 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff7780 0x55a025008480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 --2- 172.21.15.33:0/1086137380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ff6d90 0x55a024ff29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 >> 172.21.15.33:0/1086137380 conn(0x55a024fbe950 msgr2=0x55a024ff22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 shutdown_connections 2024-04-24T23:36:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 -- 172.21.15.33:0/1086137380 wait complete. 2024-04-24T23:36:01.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 -- start start 2024-04-24T23:36:01.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.305+0000 7fcad2174740 1 Processor -- start 2024-04-24T23:36:01.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.306+0000 7fcad2174740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.306+0000 7fcad2174740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a025006560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.306+0000 7fcacd200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.306+0000 7fcacd200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025006020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51542/0 (socket says 172.21.15.33:51542) 2024-04-24T23:36:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.306+0000 7fcacd200640 1 -- 172.21.15.33:0/139627424 learned_addr learned my addr 172.21.15.33:0/139627424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a025007570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a025006560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b7ed0 con 0x55a024ff7780 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b9340 con 0x55a024ff6d90 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b85a0 con 0x55a024ffc8f0 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcad2174740 10 client.0 _ll_register_callbacks cb 0x7fcab80975d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacda01640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a025007570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacd200640 1 -- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 msgr2=0x55a025007570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacd200640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a025007570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacd200640 1 -- 172.21.15.33:0/139627424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 msgr2=0x55a025006560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacd200640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a025006560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacd200640 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a025007b60 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcacd200640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025006020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009250 tx=0x7fcac4009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8cdcf889ced029 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.307+0000 7fcab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcab0ff9640 1 -- 172.21.15.33:0/139627424 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac404e070 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcab0ff9640 1 -- 172.21.15.33:0/139627424 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcac40306d0 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac0004cc0 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcab0ff9640 1 -- 172.21.15.33:0/139627424 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcac405a050 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcad2174740 10 client.29849 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a0250734d0 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcab0ff9640 1 -- 172.21.15.33:0/139627424 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac0004cc0 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.308+0000 7fcab0ff9640 1 -- 172.21.15.33:0/139627424 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fcac4003ae0 con 0x55a024ff6d90 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcad2174740 10 client.29849 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcad2174740 10 client.29849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcad2174740 20 client.29849 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a025075b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcad2174740 4 client.29849 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:36:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a024f67800 con 0x55a02500ca20 2024-04-24T23:36:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.309+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a025075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.310+0000 7fcacc9ff640 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a025075b00 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cf79d831551122 server_cookie=a0114d571f99221 in_seq=0 out_seq=0 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.310+0000 7fcab0ff9640 10 client.29849 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.310+0000 7fcab0ff9640 1 -- 172.21.15.33:0/139627424 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a024f67800 con 0x55a02500ca20 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.310+0000 7fcab0ff9640 10 client.29849 handle_command_reply: tid=0 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.310+0000 7fcad2174740 1 client.29849 shutdown 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 msgr2=0x55a025006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025006020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009250 tx=0x7fcac4009490 comp rx=0 tx=0).stop 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 shutdown_connections 2024-04-24T23:36:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55a02500ca20 0x55a025075b00 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a024ffc8f0 0x55a025007570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcab0ff9640 0 client.29849 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.311+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a024ff7780 0x55a025006560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.312+0000 7fcad2174740 1 --2- 172.21.15.33:0/139627424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a024ff6d90 0x55a025006020 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.312+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 >> 172.21.15.33:0/139627424 conn(0x55a024fbe950 msgr2=0x55a024ff5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.312+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 shutdown_connections 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.313+0000 7fcad2174740 1 -- 172.21.15.33:0/139627424 wait complete. 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.313+0000 7fcad2174740 20 client.29849 trim_cache size 0 max 16384 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:36:01.315 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:36:01.316 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "49571e89-8d92-47c7-8c86-5ff543b14e88", 2024-04-24T23:36:01.316 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:36:01.316 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:36:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.313+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcab40a2c10 msgr2=0x7fcab40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.313+0000 7fcacf464640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcab40a2c10 0x7fcab40a50d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcac00098b0 tx=0x7fcac0009e20 comp rx=0 tx=0).stop 2024-04-24T23:36:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.313+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac81019b0 msgr2=0x7fcac8111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.313+0000 7fcacf464640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac81019b0 0x7fcac8111360 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fcab800ea60 tx=0x7fcab800ef20 comp rx=0 tx=0).stop 2024-04-24T23:36:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 shutdown_connections 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcab40a2c10 0x7fcab40a50d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcac8116370 0x7fcac81ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac81019b0 0x7fcac8111360 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 --2- 172.21.15.33:0/2090023923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8100ff0 0x7fcac8110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 >> 172.21.15.33:0/2090023923 conn(0x7fcac80fb370 msgr2=0x7fcac8104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 shutdown_connections 2024-04-24T23:36:01.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.314+0000 7fcacf464640 1 -- 172.21.15.33:0/2090023923 wait complete. 2024-04-24T23:36:01.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:36:01.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:36:01.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:01 smithi033 ceph-mon[29201]: pgmap v1071: 129 pgs: 129 active+clean; 630 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 340 op/s 2024-04-24T23:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 -- 172.21.15.33:0/945705017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98101890 msgr2=0x7fdd98101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/945705017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98101890 0x7fdd98101d10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009900 tx=0x7fdd8802a410 comp rx=0 tx=0).stop 2024-04-24T23:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 -- 172.21.15.33:0/945705017 shutdown_connections 2024-04-24T23:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/945705017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd98102250 0x7fdd98108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/945705017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98101890 0x7fdd98101d10 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/945705017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd981006a0 0x7fdd98100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 -- 172.21.15.33:0/945705017 >> 172.21.15.33:0/945705017 conn(0x7fdd980fbc30 msgr2=0x7fdd980fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 -- 172.21.15.33:0/945705017 shutdown_connections 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.474+0000 7fdd9f84d640 1 -- 172.21.15.33:0/945705017 wait complete. 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 Processor -- start 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 -- start start 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 0x7fdd9806a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd98101890 0x7fdd9819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd98102250 0x7fdd9819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810c300 con 0x7fdd981006a0 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810c180 con 0x7fdd98102250 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9f84d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810c480 con 0x7fdd98101890 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 0x7fdd9806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 0x7fdd9806a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54924/0 (socket says 172.21.15.33:54924) 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 -- 172.21.15.33:0/630754103 learned_addr learned my addr 172.21.15.33:0/630754103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd98101890 0x7fdd9819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd98102250 0x7fdd9819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 -- 172.21.15.33:0/630754103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd98101890 msgr2=0x7fdd9819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd98101890 0x7fdd9819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 -- 172.21.15.33:0/630754103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd98102250 msgr2=0x7fdd9819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd98102250 0x7fdd9819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9d5e9640 1 -- 172.21.15.33:0/630754103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9819f750 con 0x7fdd981006a0 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.475+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd98101890 0x7fdd9819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.476+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 0x7fdd9806a640 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9c0 tx=0x7fdd9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.476+0000 7fdd8e7fc640 1 -- 172.21.15.33:0/630754103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400cd80 con 0x7fdd981006a0 2024-04-24T23:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.476+0000 7fdd8e7fc640 1 -- 172.21.15.33:0/630754103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd940035f0 con 0x7fdd981006a0 2024-04-24T23:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.476+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9819f9e0 con 0x7fdd981006a0 2024-04-24T23:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.476+0000 7fdd8e7fc640 1 -- 172.21.15.33:0/630754103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94010690 con 0x7fdd981006a0 2024-04-24T23:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.476+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9819ff00 con 0x7fdd981006a0 2024-04-24T23:36:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 Processor -- start 2024-04-24T23:36:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 -- start start 2024-04-24T23:36:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdd8e7fc640 1 -- 172.21.15.33:0/630754103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdd940037a0 con 0x7fdd981006a0 2024-04-24T23:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fbc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54940/0 (socket says 172.21.15.33:54940) 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdd9ddea640 1 -- 172.21.15.33:0/3030684909 learned_addr learned my addr 172.21.15.33:0/3030684909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3f1d6b0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3ec5880 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3f19670 con 0x55d0c3fbcd10 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.479+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fbc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9ddea640 1 -- 172.21.15.33:0/3030684909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fbcd10 msgr2=0x55d0c3fc9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fc9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9ddea640 1 -- 172.21.15.33:0/3030684909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 msgr2=0x55d0c3fbc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fbc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9ddea640 1 -- 172.21.15.33:0/3030684909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c3cfb7d0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fbc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd8e7fc640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd840a2c10 0x7fdd840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb95e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009fd0 tx=0x7fdd900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d069e5d66f45103 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd840a2c10 0x7fdd840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd837fe640 1 -- 172.21.15.33:0/3030684909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90004d20 con 0x55d0c3fb91c0 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd8e7fc640 1 -- 172.21.15.33:0/630754103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd940cf810 con 0x7fdd981006a0 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd837fe640 1 -- 172.21.15.33:0/3030684909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd90004050 con 0x55d0c3fb91c0 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.480+0000 7fdd837fe640 1 -- 172.21.15.33:0/3030684909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90005280 con 0x55d0c3fb91c0 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdd9d5e9640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd840a2c10 0x7fdd840a50d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd940a43c0 tx=0x7fdd94000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 msgr2=0x55d0c3fb95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb95e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009fd0 tx=0x7fdd900118d0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 shutdown_connections 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fc9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fbc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 --2- 172.21.15.33:0/3030684909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb95e0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.481+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 >> 172.21.15.33:0/3030684909 conn(0x55d0c3fb48b0 msgr2=0x55d0c3fb4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.482+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 shutdown_connections 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.482+0000 7fdda263a740 1 -- 172.21.15.33:0/3030684909 wait complete. 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.482+0000 7fdda263a740 1 -- start start 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.482+0000 7fdda263a740 1 Processor -- start 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.482+0000 7fdda263a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fcbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdda263a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fcc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdda263a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdda263a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3fca580 con 0x55d0c3fb91c0 2024-04-24T23:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdda263a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3ed8f50 con 0x55d0c3fbcd10 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdda263a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3fca700 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdda263a740 10 client.0 _ll_register_callbacks cb 0x55d0c3f14000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fcbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fcbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54950/0 (socket says 172.21.15.33:54950) 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9ddea640 1 -- 172.21.15.33:0/1771483770 learned_addr learned my addr 172.21.15.33:0/1771483770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fcc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9cde8640 1 -- 172.21.15.33:0/1771483770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 msgr2=0x55d0c3fd33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9cde8640 1 -- 172.21.15.33:0/1771483770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 msgr2=0x55d0c3fcbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fcbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9cde8640 1 -- 172.21.15.33:0/1771483770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c3c313f0 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.483+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fcbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fcc3a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009230 tx=0x7fdd88031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a430f77284c9d435 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd80ff9640 1 -- 172.21.15.33:0/1771483770 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88035040 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd90004050 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd80ff9640 1 -- 172.21.15.33:0/1771483770 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd88004300 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd80ff9640 1 -- 172.21.15.33:0/1771483770 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd88031b90 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdda263a740 10 client.29812 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d0c3bf10f0 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd80ff9640 1 -- 172.21.15.33:0/1771483770 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd90004050 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.484+0000 7fdd80ff9640 1 -- 172.21.15.33:0/1771483770 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fdd88004880 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdda263a740 10 client.29812 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdda263a740 10 client.29812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdda263a740 20 client.29812 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdda263a740 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c3fd7c60 0x55d0c3fda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdda263a740 4 client.29812 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c3f2abf0 con 0x55d0c3fd7c60 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c3fd7c60 0x55d0c3fda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c3fd7c60 0x55d0c3fda0b0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57ff922d37e6904 server_cookie=f4bc0477f4a67fdc in_seq=0 out_seq=0 2024-04-24T23:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.485+0000 7fdd80ff9640 10 client.29812 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdd80ff9640 1 -- 172.21.15.33:0/1771483770 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0c3f2abf0 con 0x55d0c3fd7c60 2024-04-24T23:36:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdd80ff9640 10 client.29812 handle_command_reply: tid=0 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 client.29812 shutdown 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb9bb0 msgr2=0x55d0c3fcc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fcc3a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009230 tx=0x7fdd88031a20 comp rx=0 tx=0).stop 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 shutdown_connections 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c3fd7c60 0x55d0c3fda0b0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fcc3a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 --2- 172.21.15.33:0/1771483770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fcbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdd80ff9640 0 client.29812 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.487+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 >> 172.21.15.33:0/1771483770 conn(0x55d0c3fb48b0 msgr2=0x55d0c3fb3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.488+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 shutdown_connections 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.488+0000 7fdda263a740 1 -- 172.21.15.33:0/1771483770 wait complete. 2024-04-24T23:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.488+0000 7fdda263a740 20 client.29812 trim_cache size 0 max 16384 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 Processor -- start 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 -- start start 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fb37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981aab10 con 0x55d0c3fbcd10 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981a83b0 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.507+0000 7fdda263a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981a6200 con 0x55d0c3fb91c0 2024-04-24T23:36:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9cde8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42448/0 (socket says 172.21.15.33:42448) 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 -- 172.21.15.33:0/607547808 learned_addr learned my addr 172.21.15.33:0/607547808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9cde8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:51584/0 (socket says 172.21.15.33:51584) 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fb37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 -- 172.21.15.33:0/607547808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 msgr2=0x55d0c3fd33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 -- 172.21.15.33:0/607547808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fbcd10 msgr2=0x55d0c3fb37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fb37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 -- 172.21.15.33:0/607547808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c3fc6ac0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.508+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb3260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd90014040 tx=0x7fdd90009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1a8e8b36a1027e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.509+0000 7fdd82ffd640 1 -- 172.21.15.33:0/607547808 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90003c30 con 0x55d0c3fb91c0 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.509+0000 7fdd82ffd640 1 -- 172.21.15.33:0/607547808 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd90003dd0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.509+0000 7fdd82ffd640 1 -- 172.21.15.33:0/607547808 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd900286b0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 -- 172.21.15.33:0/607547808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 msgr2=0x55d0c3fb3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb3260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd90014040 tx=0x7fdd90009d20 comp rx=0 tx=0).stop 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 -- 172.21.15.33:0/607547808 shutdown_connections 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fb37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 --2- 172.21.15.33:0/607547808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fb3260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 -- 172.21.15.33:0/607547808 >> 172.21.15.33:0/607547808 conn(0x55d0c3fb48b0 msgr2=0x55d0c3fb4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 -- 172.21.15.33:0/607547808 shutdown_connections 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 -- 172.21.15.33:0/607547808 wait complete. 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.510+0000 7fdda263a740 1 -- start start 2024-04-24T23:36:01.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 Processor -- start 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fd0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981a6870 con 0x55d0c3fb9bb0 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981a7ce0 con 0x55d0c3fbcd10 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdda263a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981a6f40 con 0x55d0c3fb91c0 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.511+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fd0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fd0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42458/0 (socket says 172.21.15.33:42458) 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 -- 172.21.15.33:0/933510242 learned_addr learned my addr 172.21.15.33:0/933510242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:01.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdda263a740 10 client.0 _ll_register_callbacks cb 0x7fdd88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 -- 172.21.15.33:0/933510242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 msgr2=0x55d0c3fd1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 -- 172.21.15.33:0/933510242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb9bb0 msgr2=0x55d0c3fd0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 -- 172.21.15.33:0/933510242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c3fd2320 con 0x55d0c3fb91c0 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd9ddea640 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fd0850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009fd0 tx=0x7fdd900096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6fc43f5dbb896 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.512+0000 7fdd80ff9640 1 -- 172.21.15.33:0/933510242 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000f4e0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdd80ff9640 1 -- 172.21.15.33:0/933510242 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd9000f680 con 0x55d0c3fb91c0 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd90003dd0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdd80ff9640 1 -- 172.21.15.33:0/933510242 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd9000f880 con 0x55d0c3fb91c0 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdda263a740 10 client.29818 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d0c3fd3610 con 0x55d0c3fb91c0 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdd80ff9640 1 -- 172.21.15.33:0/933510242 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd90003dd0 con 0x55d0c3fb91c0 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdd80ff9640 1 -- 172.21.15.33:0/933510242 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fdd90025040 con 0x55d0c3fb91c0 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdda263a740 10 client.29818 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.513+0000 7fdda263a740 10 client.29818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdda263a740 20 client.29818 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:01.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdda263a740 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c403e670 0x55d0c4040a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdda263a740 4 client.29818 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0c3f24f40 con 0x55d0c403e670 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c403e670 0x55d0c4040a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdd80ff9640 10 client.29818 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.514+0000 7fdd9cde8640 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c403e670 0x55d0c4040a60 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb6bea2ae61ece5 server_cookie=23186f74a6197c79 in_seq=0 out_seq=0 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdd80ff9640 1 -- 172.21.15.33:0/933510242 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d0c3f24f40 con 0x55d0c403e670 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdd80ff9640 10 client.29818 handle_command_reply: tid=0 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 client.29818 shutdown 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 msgr2=0x55d0c3fd0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fd0850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009fd0 tx=0x7fdd900096c0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 shutdown_connections 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d0c403e670 0x55d0c4040a60 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0c3fbcd10 0x55d0c3fd1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0c3fb9bb0 0x55d0c3fd0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 --2- 172.21.15.33:0/933510242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0c3fb91c0 0x55d0c3fd0850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdd80ff9640 0 client.29818 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.515+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 >> 172.21.15.33:0/933510242 conn(0x55d0c3fb48b0 msgr2=0x55d0c3fb78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.516+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 shutdown_connections 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.516+0000 7fdda263a740 1 -- 172.21.15.33:0/933510242 wait complete. 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.516+0000 7fdda263a740 20 client.29818 trim_cache size 0 max 16384 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:36:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "49571e89-8d92-47c7-8c86-5ff543b14e88": { 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "49571e89-8d92-47c7-8c86-5ff543b14e88", 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:36:01.525 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:36:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd840a2c10 msgr2=0x7fdd840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd840a2c10 0x7fdd840a50d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd940a43c0 tx=0x7fdd94000bc0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 msgr2=0x7fdd9806a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 0x7fdd9806a640 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9c0 tx=0x7fdd9400ee80 comp rx=0 tx=0).stop 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 shutdown_connections 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd840a2c10 0x7fdd840a50d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd98102250 0x7fdd9819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd98101890 0x7fdd9819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 --2- 172.21.15.33:0/630754103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd981006a0 0x7fdd9806a640 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 >> 172.21.15.33:0/630754103 conn(0x7fdd980fbc30 msgr2=0x7fdd98104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 shutdown_connections 2024-04-24T23:36:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:01.517+0000 7fdd9f84d640 1 -- 172.21.15.33:0/630754103 wait complete. 2024-04-24T23:36:01.531 INFO:tasks.cephfs.filesystem:scrub status for tag:49571e89-8d92-47c7-8c86-5ff543b14e88 - {'path': '/', 'tag': '49571e89-8d92-47c7-8c86-5ff543b14e88', 'options': 'recursive,force'} 2024-04-24T23:36:01.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:01 smithi038 ceph-mon[32450]: pgmap v1071: 129 pgs: 129 active+clean; 630 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 340 op/s 2024-04-24T23:36:01.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:01 smithi117 ceph-mon[32529]: pgmap v1071: 129 pgs: 129 active+clean; 630 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 340 op/s 2024-04-24T23:36:01.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 252013 21.90 MB/sec execute 486 sec latency 242.077 ms 2024-04-24T23:36:02.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:36:02.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:02.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:02.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.670 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:02 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:36:02.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:02.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:02.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:02.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.675 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:02 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:02 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:02.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 252287 21.90 MB/sec execute 487 sec latency 163.556 ms 2024-04-24T23:36:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:03 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:03 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:03.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:03 smithi038 ceph-mon[32450]: pgmap v1072: 129 pgs: 129 active+clean; 635 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.7 MiB/s wr, 444 op/s 2024-04-24T23:36:03.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:03 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:03.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:03 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:03.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:03 smithi117 ceph-mon[32529]: pgmap v1072: 129 pgs: 129 active+clean; 635 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.7 MiB/s wr, 444 op/s 2024-04-24T23:36:03.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:03 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:03.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:03 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:03.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:03 smithi033 ceph-mon[29201]: pgmap v1072: 129 pgs: 129 active+clean; 635 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.7 MiB/s wr, 444 op/s 2024-04-24T23:36:03.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 252659 21.89 MB/sec execute 488 sec latency 130.064 ms 2024-04-24T23:36:04.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:04 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:04.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:04 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:04.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:04 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:04.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 253225 21.91 MB/sec execute 489 sec latency 65.518 ms 2024-04-24T23:36:05.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:05 smithi038 ceph-mon[32450]: pgmap v1073: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 403 op/s 2024-04-24T23:36:05.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:05 smithi117 ceph-mon[32529]: pgmap v1073: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 403 op/s 2024-04-24T23:36:05.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:05 smithi033 ceph-mon[29201]: pgmap v1073: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 403 op/s 2024-04-24T23:36:05.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 253633 21.90 MB/sec execute 490 sec latency 77.328 ms 2024-04-24T23:36:06.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 254030 21.89 MB/sec execute 491 sec latency 157.833 ms 2024-04-24T23:36:07.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:07 smithi038 ceph-mon[32450]: pgmap v1074: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 364 op/s 2024-04-24T23:36:07.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:07 smithi117 ceph-mon[32529]: pgmap v1074: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 364 op/s 2024-04-24T23:36:07.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:07 smithi033 ceph-mon[29201]: pgmap v1074: 129 pgs: 129 active+clean; 633 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 364 op/s 2024-04-24T23:36:07.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 254419 21.90 MB/sec execute 492 sec latency 497.896 ms 2024-04-24T23:36:08.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 254706 21.90 MB/sec execute 493 sec latency 169.684 ms 2024-04-24T23:36:09.414 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:09 smithi117 ceph-mon[32529]: pgmap v1075: 129 pgs: 129 active+clean; 656 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.0 MiB/s wr, 500 op/s 2024-04-24T23:36:09.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:09 smithi038 ceph-mon[32450]: pgmap v1075: 129 pgs: 129 active+clean; 656 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.0 MiB/s wr, 500 op/s 2024-04-24T23:36:09.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:09 smithi033 ceph-mon[29201]: pgmap v1075: 129 pgs: 129 active+clean; 656 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.0 MiB/s wr, 500 op/s 2024-04-24T23:36:09.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 255116 21.89 MB/sec execute 494 sec latency 77.328 ms 2024-04-24T23:36:10.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 255558 21.90 MB/sec execute 495 sec latency 69.601 ms 2024-04-24T23:36:11.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:11 smithi038 ceph-mon[32450]: pgmap v1076: 129 pgs: 129 active+clean; 660 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 446 op/s 2024-04-24T23:36:11.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:11 smithi117 ceph-mon[32529]: pgmap v1076: 129 pgs: 129 active+clean; 660 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 446 op/s 2024-04-24T23:36:11.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:11 smithi033 ceph-mon[29201]: pgmap v1076: 129 pgs: 129 active+clean; 660 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 446 op/s 2024-04-24T23:36:11.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 256143 21.92 MB/sec execute 496 sec latency 64.486 ms 2024-04-24T23:36:12.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:12 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:36:12.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 256411 21.91 MB/sec execute 497 sec latency 402.628 ms 2024-04-24T23:36:12.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:12 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:36:12.926 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:12 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:36:13.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:13 smithi033 ceph-mon[29201]: pgmap v1077: 129 pgs: 129 active+clean; 667 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 455 op/s 2024-04-24T23:36:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:36:13.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:13 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:36:13.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 256653 21.89 MB/sec execute 498 sec latency 114.906 ms 2024-04-24T23:36:13.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:13 smithi038 ceph-mon[32450]: pgmap v1077: 129 pgs: 129 active+clean; 667 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 455 op/s 2024-04-24T23:36:13.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:36:13.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:13 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:36:13.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:13 smithi117 ceph-mon[32529]: pgmap v1077: 129 pgs: 129 active+clean; 667 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 455 op/s 2024-04-24T23:36:13.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:36:13.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:13 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:36:14.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 257013 21.88 MB/sec execute 499 sec latency 67.433 ms 2024-04-24T23:36:15.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:15 smithi033 ceph-mon[29201]: pgmap v1078: 129 pgs: 129 active+clean; 667 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.8 MiB/s wr, 378 op/s 2024-04-24T23:36:15.906 INFO:tasks.workunit.client.0.smithi033.stdout: 10 257329 21.86 MB/sec execute 500 sec latency 86.960 ms 2024-04-24T23:36:15.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:15 smithi038 ceph-mon[32450]: pgmap v1078: 129 pgs: 129 active+clean; 667 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.8 MiB/s wr, 378 op/s 2024-04-24T23:36:15.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:15 smithi117 ceph-mon[32529]: pgmap v1078: 129 pgs: 129 active+clean; 667 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.8 MiB/s wr, 378 op/s 2024-04-24T23:36:16.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 257850 21.85 MB/sec execute 501 sec latency 118.654 ms 2024-04-24T23:36:17.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:17 smithi033 ceph-mon[29201]: pgmap v1079: 129 pgs: 129 active+clean; 670 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.0 MiB/s wr, 398 op/s 2024-04-24T23:36:17.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 258238 21.87 MB/sec execute 502 sec latency 337.239 ms 2024-04-24T23:36:17.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:17 smithi038 ceph-mon[32450]: pgmap v1079: 129 pgs: 129 active+clean; 670 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.0 MiB/s wr, 398 op/s 2024-04-24T23:36:17.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:17 smithi117 ceph-mon[32529]: pgmap v1079: 129 pgs: 129 active+clean; 670 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.0 MiB/s wr, 398 op/s 2024-04-24T23:36:18.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 258647 21.88 MB/sec execute 503 sec latency 116.284 ms 2024-04-24T23:36:19.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:19 smithi033 ceph-mon[29201]: pgmap v1080: 129 pgs: 129 active+clean; 656 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.1 MiB/s wr, 455 op/s 2024-04-24T23:36:19.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:19 smithi117 ceph-mon[32529]: pgmap v1080: 129 pgs: 129 active+clean; 656 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.1 MiB/s wr, 455 op/s 2024-04-24T23:36:19.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 259293 21.90 MB/sec execute 504 sec latency 77.512 ms 2024-04-24T23:36:19.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:19 smithi038 ceph-mon[32450]: pgmap v1080: 129 pgs: 129 active+clean; 656 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.1 MiB/s wr, 455 op/s 2024-04-24T23:36:20.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 259688 21.90 MB/sec execute 505 sec latency 96.259 ms 2024-04-24T23:36:21.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:21 smithi033 ceph-mon[29201]: pgmap v1081: 129 pgs: 129 active+clean; 653 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 364 op/s 2024-04-24T23:36:21.721 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:21 smithi038 ceph-mon[32450]: pgmap v1081: 129 pgs: 129 active+clean; 653 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 364 op/s 2024-04-24T23:36:21.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 260199 21.92 MB/sec execute 506 sec latency 104.050 ms 2024-04-24T23:36:21.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:21 smithi117 ceph-mon[32529]: pgmap v1081: 129 pgs: 129 active+clean; 653 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 364 op/s 2024-04-24T23:36:22.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 260366 21.90 MB/sec execute 507 sec latency 386.525 ms 2024-04-24T23:36:23.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:23 smithi033 ceph-mon[29201]: pgmap v1082: 129 pgs: 129 active+clean; 650 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 348 op/s 2024-04-24T23:36:23.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 260531 21.87 MB/sec execute 508 sec latency 172.922 ms 2024-04-24T23:36:23.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:23 smithi038 ceph-mon[32450]: pgmap v1082: 129 pgs: 129 active+clean; 650 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 348 op/s 2024-04-24T23:36:23.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:23 smithi117 ceph-mon[32529]: pgmap v1082: 129 pgs: 129 active+clean; 650 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 348 op/s 2024-04-24T23:36:24.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 260910 21.87 MB/sec execute 509 sec latency 129.587 ms 2024-04-24T23:36:25.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:25 smithi033 ceph-mon[29201]: pgmap v1083: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-04-24T23:36:25.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 261375 21.86 MB/sec execute 510 sec latency 95.248 ms 2024-04-24T23:36:25.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:25 smithi038 ceph-mon[32450]: pgmap v1083: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-04-24T23:36:25.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:25 smithi117 ceph-mon[32529]: pgmap v1083: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-04-24T23:36:26.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 261836 21.86 MB/sec execute 511 sec latency 114.304 ms 2024-04-24T23:36:27.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 262153 21.86 MB/sec execute 512 sec latency 388.468 ms 2024-04-24T23:36:27.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:27 smithi038 ceph-mon[32450]: pgmap v1084: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 369 op/s 2024-04-24T23:36:27.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:27 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:36:27.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:27 smithi117 ceph-mon[32529]: pgmap v1084: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 369 op/s 2024-04-24T23:36:27.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:27 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:36:27.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:27 smithi033 ceph-mon[29201]: pgmap v1084: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 369 op/s 2024-04-24T23:36:27.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:27 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:36:28.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 262374 21.84 MB/sec execute 513 sec latency 178.940 ms 2024-04-24T23:36:28.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:36:28.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:36:28.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:36:28.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:36:28.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:36:28.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:36:28.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:36:28.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:36:28.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:36:29.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 262791 21.84 MB/sec execute 514 sec latency 59.483 ms 2024-04-24T23:36:29.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:29 smithi038 ceph-mon[32450]: pgmap v1085: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2024-04-24T23:36:29.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:29 smithi117 ceph-mon[32529]: pgmap v1085: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2024-04-24T23:36:29.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:29 smithi033 ceph-mon[29201]: pgmap v1085: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2024-04-24T23:36:30.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 263280 21.85 MB/sec execute 515 sec latency 71.776 ms 2024-04-24T23:36:31.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:36:31.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:36:31.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.649+0000 7f03fb909640 1 -- 172.21.15.33:0/2030091568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4103350 msgr2=0x7f03f4109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.649+0000 7f03fb909640 1 --2- 172.21.15.33:0/2030091568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4103350 0x7f03f4109be0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009be0 tx=0x7f03ec02a6c0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- 172.21.15.33:0/2030091568 shutdown_connections 2024-04-24T23:36:31.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 --2- 172.21.15.33:0/2030091568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4103350 0x7f03f4109be0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 --2- 172.21.15.33:0/2030091568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f4102990 0x7f03f4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 --2- 172.21.15.33:0/2030091568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f41017a0 0x7f03f4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- 172.21.15.33:0/2030091568 >> 172.21.15.33:0/2030091568 conn(0x7f03f40fcd10 msgr2=0x7f03f40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- 172.21.15.33:0/2030091568 shutdown_connections 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- 172.21.15.33:0/2030091568 wait complete. 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 Processor -- start 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- start start 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f41017a0 0x7f03f4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4102990 0x7f03f4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 0x7f03f419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410d3d0 con 0x7f03f4103350 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410d250 con 0x7f03f4102990 2024-04-24T23:36:31.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.650+0000 7f03fb909640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410d550 con 0x7f03f41017a0 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 0x7f03f419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f96a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f41017a0 0x7f03f4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 0x7f03f419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43140/0 (socket says 172.21.15.33:43140) 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 -- 172.21.15.33:0/1649282449 learned_addr learned my addr 172.21.15.33:0/1649282449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4102990 0x7f03f4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 -- 172.21.15.33:0/1649282449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f41017a0 msgr2=0x7f03f4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f41017a0 0x7f03f4197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 -- 172.21.15.33:0/1649282449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4102990 msgr2=0x7f03f4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4102990 0x7f03f4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 -- 172.21.15.33:0/1649282449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f419f690 con 0x7f03f4103350 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4102990 0x7f03f4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03f9ea6640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 0x7f03f419f020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0305b0 tx=0x7f03ec0305e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03ea7fc640 1 -- 172.21.15.33:0/1649282449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec041020 con 0x7f03f4103350 2024-04-24T23:36:31.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f419f920 con 0x7f03f4103350 2024-04-24T23:36:31.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03ea7fc640 1 -- 172.21.15.33:0/1649282449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec037040 con 0x7f03f4103350 2024-04-24T23:36:31.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.651+0000 7f03ea7fc640 1 -- 172.21.15.33:0/1649282449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec036070 con 0x7f03f4103350 2024-04-24T23:36:31.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.652+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f419fe60 con 0x7f03f4103350 2024-04-24T23:36:31.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.654+0000 7f03ea7fc640 1 -- 172.21.15.33:0/1649282449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f03ec044020 con 0x7f03f4103350 2024-04-24T23:36:31.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.654+0000 7f03fe60b740 1 Processor -- start 2024-04-24T23:36:31.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.654+0000 7f03fe60b740 1 -- start start 2024-04-24T23:36:31.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fad1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fad6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fad9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03fe60b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560c8fa356b0 con 0x560c8fad1c30 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03fe60b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c8f9dd880 con 0x560c8fad1240 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03fe60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c8fa31670 con 0x560c8fad6da0 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fad1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fad6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fad1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35716/0 (socket says 172.21.15.33:35716) 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f8ea4640 1 -- 172.21.15.33:0/1859284354 learned_addr learned my addr 172.21.15.33:0/1859284354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fad9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03ea7fc640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03e00a2b90 0x7f03e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 -- 172.21.15.33:0/1859284354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 msgr2=0x560c8fad9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fad9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 -- 172.21.15.33:0/1859284354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 msgr2=0x560c8fad1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fad1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 -- 172.21.15.33:0/1859284354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8f8137d0 con 0x560c8fad1c30 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f9ea6640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03e00a2b90 0x7f03e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03ea7fc640 1 -- 172.21.15.33:0/1649282449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03ec0effb0 con 0x7f03f4103350 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fad1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03f96a5640 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fad6730 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f03f41027f0 tx=0x7f03f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc94e01ab1c0125 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.655+0000 7f03df7fe640 1 -- 172.21.15.33:0/1859284354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f001e070 con 0x560c8fad1c30 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03df7fe640 1 -- 172.21.15.33:0/1859284354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0019910 con 0x560c8fad1c30 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03f9ea6640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03e00a2b90 0x7f03e00a5050 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0bc410 tx=0x7f03ec087100 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03df7fe640 1 -- 172.21.15.33:0/1859284354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000e9a0 con 0x560c8fad1c30 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 -- 172.21.15.33:0/1859284354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 msgr2=0x560c8fad6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fad6730 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f03f41027f0 tx=0x7f03f0011810 comp rx=0 tx=0).stop 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 -- 172.21.15.33:0/1859284354 shutdown_connections 2024-04-24T23:36:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fad9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fad6730 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 --2- 172.21.15.33:0/1859284354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fad1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 -- 172.21.15.33:0/1859284354 >> 172.21.15.33:0/1859284354 conn(0x560c8facc8b0 msgr2=0x560c8facccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.656+0000 7f03fe60b740 1 -- 172.21.15.33:0/1859284354 shutdown_connections 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 -- 172.21.15.33:0/1859284354 wait complete. 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 -- start start 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 Processor -- start 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fadb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fae2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560c8fad9d50 con 0x560c8fad1c30 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c8f9f0f50 con 0x560c8fad1240 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c8fad9ed0 con 0x560c8fad6da0 2024-04-24T23:36:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03fe60b740 10 client.0 _ll_register_callbacks cb 0x560c8fa2c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03f96a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fadb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03f96a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43144/0 (socket says 172.21.15.33:43144) 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.657+0000 7f03f96a5640 1 -- 172.21.15.33:0/95983225 learned_addr learned my addr 172.21.15.33:0/95983225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fae2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f96a5640 1 -- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 msgr2=0x560c8fae2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f96a5640 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fae2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f96a5640 1 -- 172.21.15.33:0/95983225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 msgr2=0x560c8fadb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f96a5640 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fadb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f96a5640 1 -- 172.21.15.33:0/95983225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8f7493f0 con 0x560c8fad1c30 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03f96a5640 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009530 tx=0x7f03f00098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26f96ef4e814604 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03dcff9640 1 -- 172.21.15.33:0/95983225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0028070 con 0x560c8fad1c30 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0019910 con 0x560c8fad1c30 2024-04-24T23:36:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03dcff9640 1 -- 172.21.15.33:0/95983225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03f0019630 con 0x560c8fad1c30 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03dcff9640 1 -- 172.21.15.33:0/95983225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03f0034050 con 0x560c8fad1c30 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03fe60b740 10 client.20418 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.658+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c8f7090f0 con 0x560c8fad1c30 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03dcff9640 1 -- 172.21.15.33:0/95983225 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0019910 con 0x560c8fad1c30 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03dcff9640 1 -- 172.21.15.33:0/95983225 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f03f000e6e0 con 0x560c8fad1c30 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03fe60b740 10 client.20418 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03fe60b740 10 client.20418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03fe60b740 20 client.20418 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03fe60b740 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8faeb5c0 0x560c8faeda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03fe60b740 4 client.20418 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c8fa42bf0 con 0x560c8faeb5c0 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.659+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8faeb5c0 0x560c8faeda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.660+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8faeb5c0 0x560c8faeda60 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c46bc2c5ebe8f9d server_cookie=821c9c81405d914b in_seq=0 out_seq=0 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.660+0000 7f03dcff9640 10 client.20418 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.660+0000 7f03dcff9640 1 -- 172.21.15.33:0/95983225 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c8fa42bf0 con 0x560c8faeb5c0 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.660+0000 7f03dcff9640 10 client.20418 handle_command_reply: tid=0 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 client.20418 shutdown 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 msgr2=0x560c8fadbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009530 tx=0x7f03f00098f0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 shutdown_connections 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8faeb5c0 0x560c8faeda60 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad6da0 0x560c8fae2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 --2- 172.21.15.33:0/95983225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8fad1240 0x560c8fadb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 >> 172.21.15.33:0/95983225 conn(0x560c8facc8b0 msgr2=0x560c8facbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03dcff9640 0 client.20418 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 shutdown_connections 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 1 -- 172.21.15.33:0/95983225 wait complete. 2024-04-24T23:36:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.661+0000 7f03fe60b740 20 client.20418 trim_cache size 0 max 16384 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 Processor -- start 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 -- start start 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41aaaa0 con 0x560c8fad1c30 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41a8340 con 0x560c8facc8b0 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03fe60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41a6190 con 0x560c8fad1240 2024-04-24T23:36:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.684+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f96a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43154/0 (socket says 172.21.15.33:43154) 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 -- 172.21.15.33:0/189823430 learned_addr learned my addr 172.21.15.33:0/189823430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f96a5640 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39592/0 (socket says 172.21.15.33:39592) 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 -- 172.21.15.33:0/189823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 msgr2=0x560c8fae2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 -- 172.21.15.33:0/189823430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 msgr2=0x560c8fadb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 -- 172.21.15.33:0/189823430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8faee0e0 con 0x560c8fad1c30 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009be0 tx=0x7f03e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9db5ca179ffdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.685+0000 7f03deffd640 1 -- 172.21.15.33:0/189823430 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4025e60 con 0x560c8fad1c30 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.686+0000 7f03deffd640 1 -- 172.21.15.33:0/189823430 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e4020df0 con 0x560c8fad1c30 2024-04-24T23:36:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.686+0000 7f03deffd640 1 -- 172.21.15.33:0/189823430 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e401a040 con 0x560c8fad1c30 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.686+0000 7f03fe60b740 1 -- 172.21.15.33:0/189823430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 msgr2=0x560c8fadbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.686+0000 7f03fe60b740 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009be0 tx=0x7f03e40046c0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 -- 172.21.15.33:0/189823430 shutdown_connections 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad1c30 0x560c8fadbbe0 secure :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009be0 tx=0x7f03e40046c0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 --2- 172.21.15.33:0/189823430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 -- 172.21.15.33:0/189823430 >> 172.21.15.33:0/189823430 conn(0x560c8fac7e70 msgr2=0x560c8fad3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 -- 172.21.15.33:0/189823430 shutdown_connections 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 -- 172.21.15.33:0/189823430 wait complete. 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 -- start start 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.687+0000 7f03fe60b740 1 Processor -- start 2024-04-24T23:36:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad6da0 0x560c8fae12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad6da0 0x560c8fae12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03f96a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03f8ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35746/0 (socket says 172.21.15.33:35746) 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03f8ea4640 1 -- 172.21.15.33:0/2544485593 learned_addr learned my addr 172.21.15.33:0/2544485593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41a6800 con 0x560c8fad6da0 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41a7c70 con 0x560c8facc8b0 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41a6ed0 con 0x560c8fad1240 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.688+0000 7f03fe60b740 10 client.0 _ll_register_callbacks cb 0x7f03f001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f8ea4640 1 -- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 msgr2=0x560c8fae0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f8ea4640 1 -- 172.21.15.33:0/2544485593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad6da0 msgr2=0x560c8fae12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad6da0 0x560c8fae12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f8ea4640 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8fae18c0 con 0x560c8facc8b0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e4020df0 con 0x560c8facc8b0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f96a5640 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03f8ea4640 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadfdf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x560c8fad6c00 tx=0x7f03e40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ced500d3c9021dee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03dcff9640 10 client.29885 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03dcff9640 1 -- 172.21.15.33:0/2544485593 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4041050 con 0x560c8facc8b0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03dcff9640 1 -- 172.21.15.33:0/2544485593 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03e4005440 con 0x560c8facc8b0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03dcff9640 1 -- 172.21.15.33:0/2544485593 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03e4005640 con 0x560c8facc8b0 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03fe60b740 10 client.29885 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.689+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=35}) -- 0x560c8fae2ab0 con 0x560c8facc8b0 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03dcff9640 1 -- 172.21.15.33:0/2544485593 <== mon.2 v2:172.21.15.38:3300/0 4 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f03e4044020 con 0x560c8facc8b0 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03fe60b740 10 client.29885 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03fe60b740 10 client.29885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03fe60b740 20 client.29885 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03fe60b740 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8fb51cc0 0x560c8fb540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03fe60b740 4 client.29885 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c8fa3cf40 con 0x560c8fb51cc0 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.690+0000 7f03f96a5640 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8fb51cc0 0x560c8fb540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03f96a5640 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8fb51cc0 0x560c8fb540b0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59baee709837ad6 server_cookie=f8715c47d30155ca in_seq=0 out_seq=0 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03dcff9640 10 client.29885 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03dcff9640 1 -- 172.21.15.33:0/2544485593 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560c8fa3cf40 con 0x560c8fb51cc0 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03dcff9640 10 client.29885 handle_command_reply: tid=0 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 client.29885 shutdown 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 msgr2=0x560c8fadfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadfdf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x560c8fad6c00 tx=0x7f03e40096a0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 shutdown_connections 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x560c8fb51cc0 0x560c8fb540b0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c8fad6da0 0x560c8fae12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8fad1240 0x560c8fae0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03dcff9640 0 client.29885 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 --2- 172.21.15.33:0/2544485593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c8facc8b0 0x560c8fadfdf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 >> 172.21.15.33:0/2544485593 conn(0x560c8fac7e70 msgr2=0x560c8fad3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 shutdown_connections 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 1 -- 172.21.15.33:0/2544485593 wait complete. 2024-04-24T23:36:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.691+0000 7f03fe60b740 20 client.29885 trim_cache size 0 max 16384 2024-04-24T23:36:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:36:31.695 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:36:31.695 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:36:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03e00a2b90 msgr2=0x7f03e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03e00a2b90 0x7f03e00a5050 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0bc410 tx=0x7f03ec087100 comp rx=0 tx=0).stop 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 msgr2=0x7f03f419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 0x7f03f419f020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0305b0 tx=0x7f03ec0305e0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 shutdown_connections 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03e00a2b90 0x7f03e00a5050 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f4103350 0x7f03f419f020 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03f4102990 0x7f03f4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 --2- 172.21.15.33:0/1649282449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f41017a0 0x7f03f4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 >> 172.21.15.33:0/1649282449 conn(0x7f03f40fcd10 msgr2=0x7f03f4105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 shutdown_connections 2024-04-24T23:36:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.692+0000 7f03fb909640 1 -- 172.21.15.33:0/1649282449 wait complete. 2024-04-24T23:36:31.704 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:36:31.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:36:31.727 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:31 smithi033 ceph-mon[29201]: pgmap v1086: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 -- 172.21.15.33:0/4152014789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 msgr2=0x7f80e8100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 --2- 172.21.15.33:0/4152014789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8100ab0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009900 tx=0x7f80d802a410 comp rx=0 tx=0).stop 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 -- 172.21.15.33:0/4152014789 shutdown_connections 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 --2- 172.21.15.33:0/4152014789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8102240 0x7f80e8108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 --2- 172.21.15.33:0/4152014789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8101880 0x7f80e8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 --2- 172.21.15.33:0/4152014789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8100ab0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 -- 172.21.15.33:0/4152014789 >> 172.21.15.33:0/4152014789 conn(0x7f80e80fbc20 msgr2=0x7f80e80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 -- 172.21.15.33:0/4152014789 shutdown_connections 2024-04-24T23:36:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.817+0000 7f80ee837640 1 -- 172.21.15.33:0/4152014789 wait complete. 2024-04-24T23:36:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 Processor -- start 2024-04-24T23:36:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 -- start start 2024-04-24T23:36:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8104620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8101880 0x7f80e8102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8102240 0x7f80e81031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80e810c2e0 con 0x7f80e8100690 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f80e810c160 con 0x7f80e8101880 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ee837640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80e810c460 con 0x7f80e8102240 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8101880 0x7f80e8102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80ecdd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8102240 0x7f80e81031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8104620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43182/0 (socket says 172.21.15.33:43182) 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.818+0000 7f80e7fff640 1 -- 172.21.15.33:0/497569781 learned_addr learned my addr 172.21.15.33:0/497569781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e7fff640 1 -- 172.21.15.33:0/497569781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8102240 msgr2=0x7f80e81031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e7fff640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8102240 0x7f80e81031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e7fff640 1 -- 172.21.15.33:0/497569781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8101880 msgr2=0x7f80e8102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e7fff640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8101880 0x7f80e8102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e7fff640 1 -- 172.21.15.33:0/497569781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e8103a30 con 0x7f80e8100690 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80ecdd4640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8102240 0x7f80e81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e77fe640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8101880 0x7f80e8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e7fff640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8104620 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f80d8031610 tx=0x7f80d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d803f030 con 0x7f80e8100690 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e81a1ba0 con 0x7f80e8100690 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d8003ad0 con 0x7f80e8100690 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8035070 con 0x7f80e8100690 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.819+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e81a2140 con 0x7f80e8100690 2024-04-24T23:36:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.820+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e81a2310 con 0x7f80e8100690 2024-04-24T23:36:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.822+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f80d8042030 con 0x7f80e8100690 2024-04-24T23:36:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.822+0000 7f80e57fa640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80d40a2b90 0x7f80d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.822+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f80d80efc20 con 0x7f80e8100690 2024-04-24T23:36:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.822+0000 7f80e77fe640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80d40a2b90 0x7f80d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.823+0000 7f80e77fe640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80d40a2b90 0x7f80d40a5050 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f80e8104390 tx=0x7f80d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:31.826+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80d80bc1a0 con 0x7f80e8100690 2024-04-24T23:36:31.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 263792 21.86 MB/sec execute 516 sec latency 79.173 ms 2024-04-24T23:36:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:31 smithi038 ceph-mon[32450]: pgmap v1086: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-04-24T23:36:31.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:31 smithi117 ceph-mon[32529]: pgmap v1086: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-04-24T23:36:32.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.185+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80e81a24e0 con 0x7f80e8100690 2024-04-24T23:36:32.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.186+0000 7f80e57fa640 1 -- 172.21.15.33:0/497569781 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11861 (secure 0 0 0) 0x7f80d80c04a0 con 0x7f80e8100690 2024-04-24T23:36:32.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:36:32.186 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:35:37.265197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:36:32.186 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80d40a2b90 msgr2=0x7f80d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80d40a2b90 0x7f80d40a5050 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f80e8104390 tx=0x7f80d0005f90 comp rx=0 tx=0).stop 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 msgr2=0x7f80e8104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8104620 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f80d8031610 tx=0x7f80d8004270 comp rx=0 tx=0).stop 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 shutdown_connections 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f80d40a2b90 0x7f80d40a5050 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e8102240 0x7f80e81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80e8101880 0x7f80e8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 --2- 172.21.15.33:0/497569781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e8100690 0x7f80e8104620 secure :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f80d8031610 tx=0x7f80d8004270 comp rx=0 tx=0).stop 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 >> 172.21.15.33:0/497569781 conn(0x7f80e80fbc20 msgr2=0x7f80e8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 shutdown_connections 2024-04-24T23:36:32.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.189+0000 7f80ee837640 1 -- 172.21.15.33:0/497569781 wait complete. 2024-04-24T23:36:32.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:36:32.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.419+0000 7f201e509640 1 -- 172.21.15.33:0/2661396206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018100f60 msgr2=0x7f20181013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.419+0000 7f201e509640 1 --2- 172.21.15.33:0/2661396206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018100f60 0x7f20181013e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2008009900 tx=0x7f200802a410 comp rx=0 tx=0).stop 2024-04-24T23:36:32.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.420+0000 7f201e509640 1 -- 172.21.15.33:0/2661396206 shutdown_connections 2024-04-24T23:36:32.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.420+0000 7f201e509640 1 --2- 172.21.15.33:0/2661396206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2018101920 0x7f2018108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.420+0000 7f201e509640 1 --2- 172.21.15.33:0/2661396206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018100f60 0x7f20181013e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.420+0000 7f201e509640 1 --2- 172.21.15.33:0/2661396206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20180ffe10 0x7f2018100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.420+0000 7f201e509640 1 -- 172.21.15.33:0/2661396206 >> 172.21.15.33:0/2661396206 conn(0x7f20180fb380 msgr2=0x7f20180fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.424+0000 7f201e509640 1 -- 172.21.15.33:0/2661396206 shutdown_connections 2024-04-24T23:36:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.427+0000 7f201e509640 1 -- 172.21.15.33:0/2661396206 wait complete. 2024-04-24T23:36:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.428+0000 7f201e509640 1 Processor -- start 2024-04-24T23:36:32.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 -- start start 2024-04-24T23:36:32.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20180ffe10 0x7f201806e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2018100f60 0x7f201806e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 0x7f201810fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f201810a9c0 con 0x7f2018100f60 2024-04-24T23:36:32.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f201810a840 con 0x7f2018101920 2024-04-24T23:36:32.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201e509640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f201810ab40 con 0x7f20180ffe10 2024-04-24T23:36:32.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201caa6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 0x7f201810fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201caa6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 0x7f201810fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35774/0 (socket says 172.21.15.33:35774) 2024-04-24T23:36:32.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.432+0000 7f201caa6640 1 -- 172.21.15.33:0/2458427509 learned_addr learned my addr 172.21.15.33:0/2458427509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.435+0000 7f20177fe640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2018100f60 0x7f201806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.437+0000 7f2017fff640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20180ffe10 0x7f201806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.438+0000 7f201caa6640 1 -- 172.21.15.33:0/2458427509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20180ffe10 msgr2=0x7f201806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.438+0000 7f201caa6640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20180ffe10 0x7f201806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.438+0000 7f201caa6640 1 -- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2018100f60 msgr2=0x7f201806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.438+0000 7f201caa6640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2018100f60 0x7f201806e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.438+0000 7f201caa6640 1 -- 172.21.15.33:0/2458427509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2018110180 con 0x7f2018101920 2024-04-24T23:36:32.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.441+0000 7f201caa6640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 0x7f201810fb40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f201000eb10 tx=0x7f201000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.445+0000 7f20157fa640 1 -- 172.21.15.33:0/2458427509 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010004b20 con 0x7f2018101920 2024-04-24T23:36:32.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.446+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2018110470 con 0x7f2018101920 2024-04-24T23:36:32.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.446+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2018110990 con 0x7f2018101920 2024-04-24T23:36:32.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.448+0000 7f20157fa640 1 -- 172.21.15.33:0/2458427509 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2010004cc0 con 0x7f2018101920 2024-04-24T23:36:32.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.448+0000 7f20157fa640 1 -- 172.21.15.33:0/2458427509 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010010790 con 0x7f2018101920 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.449+0000 7f202120d740 1 Processor -- start 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 -- start start 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc383f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 0x55fdc3844720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3847220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdc37a3670 con 0x55fdc383f230 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fdc374b970 con 0x55fdc383fc20 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f202120d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fdc379f630 con 0x55fdc3844d90 2024-04-24T23:36:32.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f2017fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 0x55fdc3844720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f2017fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 0x55fdc3844720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35784/0 (socket says 172.21.15.33:35784) 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.450+0000 7f2017fff640 1 -- 172.21.15.33:0/4084673822 learned_addr learned my addr 172.21.15.33:0/4084673822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.451+0000 7f20177fe640 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc383f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.452+0000 7f2017fff640 1 -- 172.21.15.33:0/4084673822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 msgr2=0x55fdc3847220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.452+0000 7f2017fff640 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3847220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.452+0000 7f2017fff640 1 -- 172.21.15.33:0/4084673822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 msgr2=0x55fdc383f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.452+0000 7f2017fff640 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc383f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.453+0000 7f2017fff640 1 -- 172.21.15.33:0/4084673822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc376f400 con 0x55fdc383fc20 2024-04-24T23:36:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.453+0000 7f20157fa640 1 -- 172.21.15.33:0/2458427509 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2010005020 con 0x7f2018101920 2024-04-24T23:36:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.453+0000 7f2017fff640 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 0x55fdc3844720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f200c02dab0 tx=0x7f200c02df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12f65c258c8629a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.463+0000 7f20157fa640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20000a2da0 0x7f20000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.463+0000 7f20157fa640 1 -- 172.21.15.33:0/2458427509 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20100cf990 con 0x7f2018101920 2024-04-24T23:36:32.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.467+0000 7f1ffe7fc640 1 -- 172.21.15.33:0/4084673822 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c031430 con 0x55fdc383fc20 2024-04-24T23:36:32.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.467+0000 7f1ffe7fc640 1 -- 172.21.15.33:0/4084673822 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c031ab0 con 0x55fdc383fc20 2024-04-24T23:36:32.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.467+0000 7f1ffe7fc640 1 -- 172.21.15.33:0/4084673822 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c039b60 con 0x55fdc383fc20 2024-04-24T23:36:32.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.468+0000 7f202120d740 1 -- 172.21.15.33:0/4084673822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 msgr2=0x55fdc3844720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.468+0000 7f202120d740 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 0x55fdc3844720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f200c02dab0 tx=0x7f200c02df70 comp rx=0 tx=0).stop 2024-04-24T23:36:32.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.469+0000 7f20177fe640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20000a2da0 0x7f20000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.471+0000 7f202120d740 1 -- 172.21.15.33:0/4084673822 shutdown_connections 2024-04-24T23:36:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.471+0000 7f202120d740 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3847220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.471+0000 7f202120d740 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383fc20 0x55fdc3844720 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.471+0000 7f202120d740 1 --2- 172.21.15.33:0/4084673822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc383f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.471+0000 7f202120d740 1 -- 172.21.15.33:0/4084673822 >> 172.21.15.33:0/4084673822 conn(0x55fdc383a8a0 msgr2=0x55fdc383aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.473+0000 7f20177fe640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20000a2da0 0x7f20000a5260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f200802aa00 tx=0x7f2008009350 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.478+0000 7f202120d740 1 -- 172.21.15.33:0/4084673822 shutdown_connections 2024-04-24T23:36:32.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:32 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/497569781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.481+0000 7f202120d740 1 -- 172.21.15.33:0/4084673822 wait complete. 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.481+0000 7f202120d740 1 -- start start 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.481+0000 7f202120d740 1 Processor -- start 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3850930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdc3847d40 con 0x55fdc383fc20 2024-04-24T23:36:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fdc375f050 con 0x55fdc383f230 2024-04-24T23:36:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fdc3847ec0 con 0x55fdc3844d90 2024-04-24T23:36:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.482+0000 7f202120d740 10 client.0 _ll_register_callbacks cb 0x55fdc34b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.484+0000 7f2017fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.484+0000 7f201caa6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3850930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.485+0000 7f201caa6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3850930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39656/0 (socket says 172.21.15.33:39656) 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.485+0000 7f201caa6640 1 -- 172.21.15.33:0/95972223 learned_addr learned my addr 172.21.15.33:0/95972223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.486+0000 7f20177fe640 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.488+0000 7f201caa6640 1 -- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 msgr2=0x55fdc38493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.488+0000 7f201caa6640 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.488+0000 7f201caa6640 1 -- 172.21.15.33:0/95972223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 msgr2=0x55fdc38498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.488+0000 7f201caa6640 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.488+0000 7f201caa6640 1 -- 172.21.15.33:0/95972223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc3730dc0 con 0x55fdc3844d90 2024-04-24T23:36:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.493+0000 7f201caa6640 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3850930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f201000cf90 tx=0x7f201009c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddb88c4587579f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.493+0000 7f1ff77fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:36:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.495+0000 7f1ff77fe640 1 -- 172.21.15.33:0/95972223 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20100a4a70 con 0x55fdc3844d90 2024-04-24T23:36:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.495+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200c031ab0 con 0x55fdc3844d90 2024-04-24T23:36:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.495+0000 7f1ff77fe640 1 -- 172.21.15.33:0/95972223 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20100a45e0 con 0x55fdc3844d90 2024-04-24T23:36:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.496+0000 7f1ff77fe640 1 -- 172.21.15.33:0/95972223 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f20100a47e0 con 0x55fdc3844d90 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.496+0000 7f202120d740 10 client.29845 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.496+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fdc34b73f0 con 0x55fdc3844d90 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.499+0000 7f1ff77fe640 1 -- 172.21.15.33:0/95972223 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c031ab0 con 0x55fdc3844d90 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.499+0000 7f1ff77fe640 1 -- 172.21.15.33:0/95972223 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f201009aed0 con 0x55fdc3844d90 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.500+0000 7f202120d740 10 client.29845 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.500+0000 7f202120d740 10 client.29845 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.500+0000 7f202120d740 20 client.29845 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.500+0000 7f202120d740 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38551d0 0x55fdc3857670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.500+0000 7f202120d740 4 client.29845 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.500+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc37b0bb0 con 0x55fdc38551d0 2024-04-24T23:36:32.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.505+0000 7f2017fff640 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38551d0 0x55fdc3857670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.522+0000 7f2017fff640 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38551d0 0x55fdc3857670 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7a11ea787455c77 server_cookie=de5a9dcfa4f682b2 in_seq=0 out_seq=0 2024-04-24T23:36:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.523+0000 7f1ff77fe640 10 client.29845 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:36:32.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.538+0000 7f1ff77fe640 1 -- 172.21.15.33:0/95972223 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdc37b0bb0 con 0x55fdc38551d0 2024-04-24T23:36:32.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.538+0000 7f1ff77fe640 10 client.29845 handle_command_reply: tid=0 2024-04-24T23:36:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.541+0000 7f202120d740 1 client.29845 shutdown 2024-04-24T23:36:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.549+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 msgr2=0x55fdc3850930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.549+0000 7f202120d740 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3850930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f201000cf90 tx=0x7f201009c500 comp rx=0 tx=0).stop 2024-04-24T23:36:32.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 shutdown_connections 2024-04-24T23:36:32.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f202120d740 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38551d0 0x55fdc3857670 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f202120d740 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc3850930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f202120d740 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f202120d740 1 --2- 172.21.15.33:0/95972223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 >> 172.21.15.33:0/95972223 conn(0x55fdc383a8a0 msgr2=0x55fdc3839b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.554+0000 7f1ff77fe640 0 client.29845 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:36:32.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.556+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 shutdown_connections 2024-04-24T23:36:32.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.558+0000 7f202120d740 1 -- 172.21.15.33:0/95972223 wait complete. 2024-04-24T23:36:32.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.559+0000 7f202120d740 20 client.29845 trim_cache size 0 max 16384 2024-04-24T23:36:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.589+0000 7f202120d740 1 Processor -- start 2024-04-24T23:36:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.591+0000 7f202120d740 1 -- start start 2024-04-24T23:36:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.592+0000 7f202120d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.592+0000 7f202120d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.592+0000 7f202120d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc38406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f202120d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20040125e0 con 0x55fdc383fc20 2024-04-24T23:36:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f201caa6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc38406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f201caa6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc38406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39668/0 (socket says 172.21.15.33:39668) 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f201caa6640 1 -- 172.21.15.33:0/2035936157 learned_addr learned my addr 172.21.15.33:0/2035936157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f200400fe80 con 0x55fdc383f230 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f200400dcd0 con 0x55fdc3844d90 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.593+0000 7f2017fff640 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f20177fe640 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f201caa6640 1 -- 172.21.15.33:0/2035936157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 msgr2=0x55fdc38399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f201caa6640 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38399c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f201caa6640 1 -- 172.21.15.33:0/2035936157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 msgr2=0x55fdc38508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f201caa6640 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f201caa6640 1 -- 172.21.15.33:0/2035936157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc35817d0 con 0x55fdc3844d90 2024-04-24T23:36:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f201caa6640 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc38406c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f201009bd90 tx=0x7f2010009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ca80790449d189 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.594+0000 7f1ffdffb640 1 -- 172.21.15.33:0/2035936157 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20100a4a70 con 0x55fdc3844d90 2024-04-24T23:36:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.595+0000 7f1ffdffb640 1 -- 172.21.15.33:0/2035936157 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20100a45e0 con 0x55fdc3844d90 2024-04-24T23:36:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.596+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 msgr2=0x55fdc38406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.596+0000 7f202120d740 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc38406c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f201009bd90 tx=0x7f2010009200 comp rx=0 tx=0).stop 2024-04-24T23:36:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.597+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 shutdown_connections 2024-04-24T23:36:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.597+0000 7f202120d740 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc3844d90 0x55fdc38406c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.597+0000 7f202120d740 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383fc20 0x55fdc38508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.597+0000 7f202120d740 1 --2- 172.21.15.33:0/2035936157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc383f230 0x55fdc38399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.597+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 >> 172.21.15.33:0/2035936157 conn(0x55fdc383a8a0 msgr2=0x55fdc38392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.605+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 shutdown_connections 2024-04-24T23:36:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.609+0000 7f202120d740 1 -- 172.21.15.33:0/2035936157 wait complete. 2024-04-24T23:36:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.609+0000 7f202120d740 1 -- start start 2024-04-24T23:36:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.609+0000 7f202120d740 1 Processor -- start 2024-04-24T23:36:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc384df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc3844d90 0x55fdc384e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc384f4d0 0x55fdc384f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f200400e340 con 0x55fdc383f230 2024-04-24T23:36:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f200400f7b0 con 0x55fdc3844d90 2024-04-24T23:36:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f200400ea10 con 0x55fdc384f4d0 2024-04-24T23:36:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.610+0000 7f202120d740 10 client.0 _ll_register_callbacks cb 0x7f201000cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.612+0000 7f201caa6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc384f4d0 0x55fdc384f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.612+0000 7f201caa6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc384f4d0 0x55fdc384f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39676/0 (socket says 172.21.15.33:39676) 2024-04-24T23:36:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.612+0000 7f201caa6640 1 -- 172.21.15.33:0/3316622969 learned_addr learned my addr 172.21.15.33:0/3316622969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.612+0000 7f2017fff640 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc384df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f2017fff640 1 -- 172.21.15.33:0/3316622969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc384f4d0 msgr2=0x55fdc384f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f2017fff640 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc384f4d0 0x55fdc384f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f2017fff640 1 -- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc3844d90 msgr2=0x55fdc384e4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f2017fff640 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc3844d90 0x55fdc384e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f2017fff640 1 -- 172.21.15.33:0/3316622969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc384fe90 con 0x55fdc383f230 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f2017fff640 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc384df80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x55fdc383b770 tx=0x7f200c02fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f1feb0b54a498f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.613+0000 7f1ff77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.614+0000 7f1ff77fe640 1 -- 172.21.15.33:0/3316622969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c051060 con 0x55fdc383f230 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.614+0000 7f1ff77fe640 1 -- 172.21.15.33:0/3316622969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f200c04cdc0 con 0x55fdc383f230 2024-04-24T23:36:32.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.614+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20100a45e0 con 0x55fdc383f230 2024-04-24T23:36:32.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.617+0000 7f1ff77fe640 1 -- 172.21.15.33:0/3316622969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f200c005250 con 0x55fdc383f230 2024-04-24T23:36:32.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.619+0000 7f202120d740 10 client.20448 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:32.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.619+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fdc38bbaa0 con 0x55fdc383f230 2024-04-24T23:36:32.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.619+0000 7f1ff77fe640 1 -- 172.21.15.33:0/3316622969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20100a45e0 con 0x55fdc383f230 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.619+0000 7f1ff77fe640 1 -- 172.21.15.33:0/3316622969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f200c031780 con 0x55fdc383f230 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.620+0000 7f202120d740 10 client.20448 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.620+0000 7f202120d740 10 client.20448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.620+0000 7f202120d740 20 client.20448 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.620+0000 7f202120d740 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38bc150 0x55fdc38be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.620+0000 7f202120d740 4 client.20448 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:36:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.620+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdc37aaf00 con 0x55fdc38bc150 2024-04-24T23:36:32.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.630+0000 7f20177fe640 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38bc150 0x55fdc38be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.656+0000 7f20177fe640 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38bc150 0x55fdc38be570 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff57ef6acef2cdfc server_cookie=7da038c84d60cc90 in_seq=0 out_seq=0 2024-04-24T23:36:32.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.657+0000 7f1ff77fe640 10 client.20448 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:36:32.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.661+0000 7f1ff77fe640 1 -- 172.21.15.33:0/3316622969 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdc37aaf00 con 0x55fdc38bc150 2024-04-24T23:36:32.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.661+0000 7f1ff77fe640 10 client.20448 handle_command_reply: tid=0 2024-04-24T23:36:32.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.662+0000 7f202120d740 1 client.20448 shutdown 2024-04-24T23:36:32.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.668+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 msgr2=0x55fdc384df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.668+0000 7f202120d740 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc384df80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x55fdc383b770 tx=0x7f200c02fe10 comp rx=0 tx=0).stop 2024-04-24T23:36:32.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 shutdown_connections 2024-04-24T23:36:32.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f202120d740 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x55fdc38bc150 0x55fdc38be570 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f202120d740 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdc384f4d0 0x55fdc384f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f202120d740 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fdc3844d90 0x55fdc384e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f202120d740 1 --2- 172.21.15.33:0/3316622969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdc383f230 0x55fdc384df80 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 >> 172.21.15.33:0/3316622969 conn(0x55fdc383a8a0 msgr2=0x55fdc3839b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.669+0000 7f1ff77fe640 0 client.20448 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:36:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.673+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 shutdown_connections 2024-04-24T23:36:32.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.673+0000 7f202120d740 1 -- 172.21.15.33:0/3316622969 wait complete. 2024-04-24T23:36:32.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.674+0000 7f202120d740 20 client.20448 trim_cache size 0 max 16384 2024-04-24T23:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:36:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.677+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20000a2da0 msgr2=0x7f20000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.677+0000 7f201e509640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20000a2da0 0x7f20000a5260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f200802aa00 tx=0x7f2008009350 comp rx=0 tx=0).stop 2024-04-24T23:36:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.677+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 msgr2=0x7f201810fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.677+0000 7f201e509640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 0x7f201810fb40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f201000eb10 tx=0x7f201000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.679+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 shutdown_connections 2024-04-24T23:36:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.679+0000 7f201e509640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f20000a2da0 0x7f20000a5260 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.679+0000 7f201e509640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2018101920 0x7f201810fb40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.679+0000 7f201e509640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2018100f60 0x7f201806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.679+0000 7f201e509640 1 --2- 172.21.15.33:0/2458427509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20180ffe10 0x7f201806e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.679+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 >> 172.21.15.33:0/2458427509 conn(0x7f20180fb380 msgr2=0x7f20180fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.681+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 shutdown_connections 2024-04-24T23:36:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.682+0000 7f201e509640 1 -- 172.21.15.33:0/2458427509 wait complete. 2024-04-24T23:36:32.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:36:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 -- 172.21.15.33:0/4220991124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04100680 msgr2=0x7f5d04100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/4220991124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04100680 0x7f5d04100aa0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009900 tx=0x7f5d0002a410 comp rx=0 tx=0).stop 2024-04-24T23:36:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 -- 172.21.15.33:0/4220991124 shutdown_connections 2024-04-24T23:36:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/4220991124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d04102230 0x7f5d04108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/4220991124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04101870 0x7f5d04101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/4220991124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04100680 0x7f5d04100aa0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 -- 172.21.15.33:0/4220991124 >> 172.21.15.33:0/4220991124 conn(0x7f5d040fbbf0 msgr2=0x7f5d040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 -- 172.21.15.33:0/4220991124 shutdown_connections 2024-04-24T23:36:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.853+0000 7f5d0c23f640 1 -- 172.21.15.33:0/4220991124 wait complete. 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 Processor -- start 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 -- start start 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04100680 0x7f5d04199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 0x7f5d0419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d04102230 0x7f5d041a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0410f610 con 0x7f5d04101870 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0410f490 con 0x7f5d04100680 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0c23f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0410f790 con 0x7f5d04102230 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 0x7f5d0419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 0x7f5d0419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43268/0 (socket says 172.21.15.33:43268) 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 -- 172.21.15.33:0/306468031 learned_addr learned my addr 172.21.15.33:0/306468031 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04100680 0x7f5d04199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d0a7dc640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d04102230 0x7f5d041a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 -- 172.21.15.33:0/306468031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d04102230 msgr2=0x7f5d041a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d04102230 0x7f5d041a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 -- 172.21.15.33:0/306468031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04100680 msgr2=0x7f5d04199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04100680 0x7f5d04199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d097da640 1 -- 172.21.15.33:0/306468031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d041a1920 con 0x7f5d04101870 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.854+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04100680 0x7f5d04199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.855+0000 7f5d097da640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 0x7f5d0419a250 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ea60 tx=0x7f5cf400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.855+0000 7f5cfaffd640 1 -- 172.21.15.33:0/306468031 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400cde0 con 0x7f5d04101870 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.855+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d041a1c10 con 0x7f5d04101870 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.855+0000 7f5cfaffd640 1 -- 172.21.15.33:0/306468031 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf40035f0 con 0x7f5d04101870 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.855+0000 7f5cfaffd640 1 -- 172.21.15.33:0/306468031 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4010670 con 0x7f5d04101870 2024-04-24T23:36:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.855+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d041a2130 con 0x7f5d04101870 2024-04-24T23:36:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.857+0000 7f5cfaffd640 1 -- 172.21.15.33:0/306468031 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5cf40037a0 con 0x7f5d04101870 2024-04-24T23:36:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.857+0000 7f5d0f02f740 1 Processor -- start 2024-04-24T23:36:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 -- start start 2024-04-24T23:36:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f1b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f20720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606f4e7f670 con 0x5606f4f1b230 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5606f4e27970 con 0x5606f4f20d90 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5606f4e7b630 con 0x5606f4f1bc20 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5cfaffd640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cf00a2b90 0x7f5cf00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f20720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0a7dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f1b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f20720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39704/0 (socket says 172.21.15.33:39704) 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3336778381 learned_addr learned my addr 172.21.15.33:0/3336778381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0a7dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f1b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43276/0 (socket says 172.21.15.33:43276) 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d097da640 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5cfaffd640 1 -- 172.21.15.33:0/306468031 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5cf40cf770 con 0x7f5d04101870 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d0a7dc640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cf00a2b90 0x7f5cf00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3336778381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 msgr2=0x5606f4f23220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f23220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3336778381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 msgr2=0x5606f4f1b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f1b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.858+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3336778381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f4e4b400 con 0x5606f4f1bc20 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f20720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009900 tx=0x7f5d00009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d596640a32adb61d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5d0a7dc640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cf00a2b90 0x7f5cf00a5050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5606f4f1c480 tx=0x7f5cfc004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5ceffff640 1 -- 172.21.15.33:0/3336778381 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00040070 con 0x5606f4f1bc20 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5ceffff640 1 -- 172.21.15.33:0/3336778381 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d000319b0 con 0x5606f4f1bc20 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5ceffff640 1 -- 172.21.15.33:0/3336778381 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00035040 con 0x5606f4f1bc20 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3336778381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 msgr2=0x5606f4f20720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.859+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f20720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009900 tx=0x7f5d00009680 comp rx=0 tx=0).stop 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3336778381 shutdown_connections 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f23220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f20720 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3336778381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f1b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3336778381 >> 172.21.15.33:0/3336778381 conn(0x5606f4f168a0 msgr2=0x5606f4f16ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3336778381 shutdown_connections 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3336778381 wait complete. 2024-04-24T23:36:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- start start 2024-04-24T23:36:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 Processor -- start 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1b230 0x5606f4f25320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 0x5606f4f25860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f2c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606f4f23ce0 con 0x5606f4f1bc20 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5606f4e3b050 con 0x5606f4f20d90 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5606f4f23e60 con 0x5606f4f1b230 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.860+0000 7f5d0f02f740 10 client.0 _ll_register_callbacks cb 0x5606f4b92be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 0x5606f4f25860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 0x5606f4f25860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43278/0 (socket says 172.21.15.33:43278) 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1b230 0x5606f4f25320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 -- 172.21.15.33:0/3391394619 learned_addr learned my addr 172.21.15.33:0/3391394619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d0a7dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f2c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 -- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1b230 msgr2=0x5606f4f25320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1b230 0x5606f4f25320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 -- 172.21.15.33:0/3391394619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 msgr2=0x5606f4f2c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f2c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 -- 172.21.15.33:0/3391394619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f4e0cdc0 con 0x5606f4f1bc20 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d097da640 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 0x5606f4f25860 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40a4320 tx=0x7f5cf406aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bf8bff4e028e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3391394619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4099d20 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3391394619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5cf40981c0 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d000319b0 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.861+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3391394619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5cf409c2e0 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 10 client.20460 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5606f4b933f0 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3391394619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d000319b0 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3391394619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5cf4066ce0 con 0x5606f4f1bc20 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 10 client.20460 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 10 client.20460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 20 client.20460 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f31240 0x5606f4f33650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 4 client.20460 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606f4e8cbb0 con 0x5606f4f31240 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.862+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f31240 0x5606f4f33650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.863+0000 7f5ced7fa640 10 client.20460 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.863+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f31240 0x5606f4f33650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7d4b5175c2add95 server_cookie=734bb433911a2fe5 in_seq=0 out_seq=0 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3391394619 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606f4e8cbb0 con 0x5606f4f31240 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5ced7fa640 10 client.20460 handle_command_reply: tid=0 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 client.20460 shutdown 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 msgr2=0x5606f4f25860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 0x5606f4f25860 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40a4320 tx=0x7f5cf406aad0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 shutdown_connections 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f31240 0x5606f4f33650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f2c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1bc20 0x5606f4f25860 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3391394619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1b230 0x5606f4f25320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5ced7fa640 0 client.20460 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 >> 172.21.15.33:0/3391394619 conn(0x5606f4f168a0 msgr2=0x5606f4f15b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 shutdown_connections 2024-04-24T23:36:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3391394619 wait complete. 2024-04-24T23:36:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.864+0000 7f5d0f02f740 20 client.20460 trim_cache size 0 max 16384 2024-04-24T23:36:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 Processor -- start 2024-04-24T23:36:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 -- start start 2024-04-24T23:36:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f1bc20 0x5606f4f2c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f20d90 0x5606f4f197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041aef60 con 0x5606f4f1b230 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041ac800 con 0x5606f4f1bc20 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041aa650 con 0x5606f4f20d90 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f159c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43288/0 (socket says 172.21.15.33:43288) 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d097da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f1bc20 0x5606f4f2c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 -- 172.21.15.33:0/1626881365 learned_addr learned my addr 172.21.15.33:0/1626881365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0a7dc640 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f20d90 0x5606f4f197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 -- 172.21.15.33:0/1626881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f20d90 msgr2=0x5606f4f197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f20d90 0x5606f4f197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 -- 172.21.15.33:0/1626881365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f1bc20 msgr2=0x5606f4f2c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f1bc20 0x5606f4f2c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d09fdb640 1 -- 172.21.15.33:0/1626881365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f4c5d7d0 con 0x5606f4f1b230 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.878+0000 7f5d0a7dc640 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f20d90 0x5606f4f197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.879+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f159c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5d00033000 tx=0x7f5d00031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7249f118dfc139b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.879+0000 7f5cef7fe640 1 -- 172.21.15.33:0/1626881365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00040070 con 0x5606f4f1b230 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.879+0000 7f5cef7fe640 1 -- 172.21.15.33:0/1626881365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d00005520 con 0x5606f4f1b230 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.879+0000 7f5cef7fe640 1 -- 172.21.15.33:0/1626881365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00035040 con 0x5606f4f1b230 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 -- 172.21.15.33:0/1626881365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 msgr2=0x5606f4f159c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f159c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5d00033000 tx=0x7f5d00031400 comp rx=0 tx=0).stop 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 -- 172.21.15.33:0/1626881365 shutdown_connections 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f20d90 0x5606f4f197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f1bc20 0x5606f4f2c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/1626881365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f159c0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 -- 172.21.15.33:0/1626881365 >> 172.21.15.33:0/1626881365 conn(0x5606f4f168a0 msgr2=0x5606f4f152b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 -- 172.21.15.33:0/1626881365 shutdown_connections 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 -- 172.21.15.33:0/1626881365 wait complete. 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 -- start start 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 Processor -- start 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.880+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f29750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f29c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f95780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041aacc0 con 0x5606f4f1b230 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041ac130 con 0x5606f4f20d90 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0f02f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041ab390 con 0x5606f4f1bc20 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0f02f740 10 client.0 _ll_register_callbacks cb 0x7f5cf400e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f29750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f29750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43298/0 (socket says 172.21.15.33:43298) 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3044846694 learned_addr learned my addr 172.21.15.33:0/3044846694 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d097da640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f29c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d0a7dc640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f95780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 msgr2=0x5606f4f29c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f29c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3044846694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 msgr2=0x5606f4f95780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f95780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 -- 172.21.15.33:0/3044846694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f4f95cc0 con 0x5606f4f1b230 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5d09fdb640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f29750 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5d00031610 tx=0x7f5d0002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c114079910ebbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.881+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3044846694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00035070 con 0x5606f4f1b230 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d00005520 con 0x5606f4f1b230 2024-04-24T23:36:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3044846694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5d00004460 con 0x5606f4f1b230 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3044846694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5d0005d050 con 0x5606f4f1b230 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 10 client.20466 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5606f4f96f20 con 0x5606f4f1b230 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3044846694 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d00005520 con 0x5606f4f1b230 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3044846694 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5d000037b0 con 0x5606f4f1b230 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 10 client.20466 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 10 client.20466 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 20 client.20466 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f97480 0x5606f4f99870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 4 client.20466 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.882+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606f4e86f00 con 0x5606f4f97480 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5d097da640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f97480 0x5606f4f99870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5ced7fa640 10 client.20466 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:36:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5d097da640 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f97480 0x5606f4f99870 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67a070c38e16558d server_cookie=1f842db9bf9e99f in_seq=0 out_seq=0 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5ced7fa640 1 -- 172.21.15.33:0/3044846694 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606f4e86f00 con 0x5606f4f97480 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5ced7fa640 10 client.20466 handle_command_reply: tid=0 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5d0f02f740 1 client.20466 shutdown 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 msgr2=0x5606f4f29750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.883+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f29750 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5d00031610 tx=0x7f5d0002a410 comp rx=0 tx=0).stop 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 shutdown_connections 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x5606f4f97480 0x5606f4f99870 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5ced7fa640 0 client.20466 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606f4f20d90 0x5606f4f95780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606f4f1bc20 0x5606f4f29c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 --2- 172.21.15.33:0/3044846694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606f4f1b230 0x5606f4f29750 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 >> 172.21.15.33:0/3044846694 conn(0x5606f4f168a0 msgr2=0x5606f4f15b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 shutdown_connections 2024-04-24T23:36:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 1 -- 172.21.15.33:0/3044846694 wait complete. 2024-04-24T23:36:32.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0f02f740 20 client.20466 trim_cache size 0 max 16384 2024-04-24T23:36:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cf00a2b90 msgr2=0x7f5cf00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.884+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cf00a2b90 0x7f5cf00a5050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5606f4f1c480 tx=0x7f5cfc004ab0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 msgr2=0x7f5d0419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 0x7f5d0419a250 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ea60 tx=0x7f5cf400ef20 comp rx=0 tx=0).stop 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 shutdown_connections 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f5cf00a2b90 0x7f5cf00a5050 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d04102230 0x7f5d041a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d04101870 0x7f5d0419a250 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 --2- 172.21.15.33:0/306468031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d04100680 0x7f5d04199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 >> 172.21.15.33:0/306468031 conn(0x7f5d040fbbf0 msgr2=0x7f5d04104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 shutdown_connections 2024-04-24T23:36:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:32.885+0000 7f5d0c23f640 1 -- 172.21.15.33:0/306468031 wait complete. 2024-04-24T23:36:32.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:36:32.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 264049 21.84 MB/sec execute 517 sec latency 442.112 ms 2024-04-24T23:36:32.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:32 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/497569781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:36:32.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:32 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/497569781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.037+0000 7f9700ce7640 1 -- 172.21.15.33:0/3331192040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc107480 msgr2=0x7f96fc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.037+0000 7f9700ce7640 1 --2- 172.21.15.33:0/3331192040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc107480 0x7f96fc1098d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009be0 tx=0x7f96f402a6c0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 -- 172.21.15.33:0/3331192040 shutdown_connections 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 --2- 172.21.15.33:0/3331192040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc107480 0x7f96fc1098d0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 --2- 172.21.15.33:0/3331192040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc104b50 0x7f96fc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 --2- 172.21.15.33:0/3331192040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc102220 0x7f96fc104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 -- 172.21.15.33:0/3331192040 >> 172.21.15.33:0/3331192040 conn(0x7f96fc0fbc30 msgr2=0x7f96fc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 -- 172.21.15.33:0/3331192040 shutdown_connections 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.038+0000 7f9700ce7640 1 -- 172.21.15.33:0/3331192040 wait complete. 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.039+0000 7f9700ce7640 1 Processor -- start 2024-04-24T23:36:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f9700ce7640 1 -- start start 2024-04-24T23:36:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f9700ce7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc102220 0x7f96fc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f9700ce7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc104b50 0x7f96fc197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc102220 0x7f96fc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc104b50 0x7f96fc197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc104b50 0x7f96fc197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43316/0 (socket says 172.21.15.33:43316) 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f96f9d9b640 1 -- 172.21.15.33:0/1310966079 learned_addr learned my addr 172.21.15.33:0/1310966079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.040+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc107480 0x7f96fc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10d440 con 0x7f96fc104b50 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10d2c0 con 0x7f96fc107480 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10d5c0 con 0x7f96fc102220 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fad9d640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc107480 0x7f96fc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fad9d640 1 -- 172.21.15.33:0/1310966079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc102220 msgr2=0x7f96fc1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fad9d640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc102220 0x7f96fc1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fad9d640 1 -- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc104b50 msgr2=0x7f96fc197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fad9d640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc104b50 0x7f96fc197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fad9d640 1 -- 172.21.15.33:0/1310966079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc19f4d0 con 0x7f96fc107480 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.041+0000 7f96fa59c640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc102220 0x7f96fc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.042+0000 7f96fad9d640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc107480 0x7f96fc19ee60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009b30 tx=0x7f96f4030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.042+0000 7f96eb7fe640 1 -- 172.21.15.33:0/1310966079 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4041020 con 0x7f96fc107480 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.042+0000 7f96eb7fe640 1 -- 172.21.15.33:0/1310966079 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4031920 con 0x7f96fc107480 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.042+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc19f760 con 0x7f96fc107480 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.042+0000 7f96eb7fe640 1 -- 172.21.15.33:0/1310966079 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4036040 con 0x7f96fc107480 2024-04-24T23:36:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.042+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc19fb90 con 0x7f96fc107480 2024-04-24T23:36:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.045+0000 7f97039f1740 1 Processor -- start 2024-04-24T23:36:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f96eb7fe640 1 -- 172.21.15.33:0/1310966079 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f96f4044020 con 0x7f96fc107480 2024-04-24T23:36:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 -- start start 2024-04-24T23:36:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d44720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d47220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0ca3670 con 0x55cbd0d3f230 2024-04-24T23:36:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0c4b970 con 0x55cbd0d3fc20 2024-04-24T23:36:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f97039f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0c9f630 con 0x55cbd0d44d90 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d44720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43328/0 (socket says 172.21.15.33:43328) 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.046+0000 7f96f9d9b640 1 -- 172.21.15.33:0/2083295360 learned_addr learned my addr 172.21.15.33:0/2083295360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d47220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 -- 172.21.15.33:0/2083295360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 msgr2=0x55cbd0d47220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d47220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 -- 172.21.15.33:0/2083295360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 msgr2=0x55cbd0d44720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d44720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 -- 172.21.15.33:0/2083295360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd0c6f400 con 0x55cbd0d3f230 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3f650 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f96f0013640 tx=0x7f96f00098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6319b0af4ad84d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96eb7fe640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e40a2b90 0x7f96e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96e8ff9640 1 -- 172.21.15.33:0/2083295360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0005020 con 0x55cbd0d3f230 2024-04-24T23:36:33.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96e8ff9640 1 -- 172.21.15.33:0/2083295360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f00047a0 con 0x55cbd0d3f230 2024-04-24T23:36:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96e8ff9640 1 -- 172.21.15.33:0/2083295360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000f860 con 0x55cbd0d3f230 2024-04-24T23:36:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96eb7fe640 1 -- 172.21.15.33:0/1310966079 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96f40f02a0 con 0x7f96fc107480 2024-04-24T23:36:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.047+0000 7f96fa59c640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e40a2b90 0x7f96e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.049+0000 7f97039f1740 1 -- 172.21.15.33:0/2083295360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 msgr2=0x55cbd0d3f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.049+0000 7f97039f1740 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3f650 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f96f0013640 tx=0x7f96f00098a0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.049+0000 7f96fa59c640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e40a2b90 0x7f96e40a5050 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55cbd0d40020 tx=0x7f96ec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.050+0000 7f97039f1740 1 -- 172.21.15.33:0/2083295360 shutdown_connections 2024-04-24T23:36:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.050+0000 7f97039f1740 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d47220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.050+0000 7f97039f1740 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d44720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.050+0000 7f97039f1740 1 --2- 172.21.15.33:0/2083295360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3f650 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.050+0000 7f97039f1740 1 -- 172.21.15.33:0/2083295360 >> 172.21.15.33:0/2083295360 conn(0x55cbd0d3a8a0 msgr2=0x55cbd0d3aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.053+0000 7f97039f1740 1 -- 172.21.15.33:0/2083295360 shutdown_connections 2024-04-24T23:36:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.053+0000 7f97039f1740 1 -- 172.21.15.33:0/2083295360 wait complete. 2024-04-24T23:36:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.053+0000 7f97039f1740 1 -- start start 2024-04-24T23:36:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.053+0000 7f97039f1740 1 Processor -- start 2024-04-24T23:36:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.054+0000 7f97039f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.054+0000 7f97039f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f97039f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f97039f1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0d58d50 con 0x55cbd0d44d90 2024-04-24T23:36:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f97039f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0c5f050 con 0x55cbd0d3fc20 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f97039f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0d58ed0 con 0x55cbd0d3f230 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f97039f1740 10 client.0 _ll_register_callbacks cb 0x55cbd09b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43336/0 (socket says 172.21.15.33:43336) 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 -- 172.21.15.33:0/2500024270 learned_addr learned my addr 172.21.15.33:0/2500024270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fa59c640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 -- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3f230 msgr2=0x55cbd0d5a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 -- 172.21.15.33:0/2500024270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 msgr2=0x55cbd0d5a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fad9d640 1 -- 172.21.15.33:0/2500024270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd0c30dc0 con 0x55cbd0d44d90 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.055+0000 7f96fa59c640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f96fad9d640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d619e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f96f40b8c60 tx=0x7f96f40bc400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb57523dd68ba43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f96e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f96e1ffb640 1 -- 172.21.15.33:0/2500024270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40c30e0 con 0x55cbd0d44d90 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f96e1ffb640 1 -- 172.21.15.33:0/2500024270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96f4003bf0 con 0x55cbd0d44d90 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f00047a0 con 0x55cbd0d44d90 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f96e1ffb640 1 -- 172.21.15.33:0/2500024270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96f40bc800 con 0x55cbd0d44d90 2024-04-24T23:36:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f97039f1740 10 client.20478 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.056+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cbd09b73f0 con 0x55cbd0d44d90 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f96e1ffb640 1 -- 172.21.15.33:0/2500024270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f00047a0 con 0x55cbd0d44d90 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f96e1ffb640 1 -- 172.21.15.33:0/2500024270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f96f40c4f50 con 0x55cbd0d44d90 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f97039f1740 10 client.20478 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f97039f1740 10 client.20478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f97039f1740 20 client.20478 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f97039f1740 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0d66290 0x55cbd0d686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f97039f1740 4 client.20478 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd0cb0bb0 con 0x55cbd0d66290 2024-04-24T23:36:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.057+0000 7f96fa59c640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0d66290 0x55cbd0d686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.060+0000 7f96fa59c640 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0d66290 0x55cbd0d686f0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d67c23c9896414 server_cookie=c93b56760868bf03 in_seq=0 out_seq=0 2024-04-24T23:36:33.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.060+0000 7f96e1ffb640 10 client.20478 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:33.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f96e1ffb640 1 -- 172.21.15.33:0/2500024270 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbd0cb0bb0 con 0x55cbd0d66290 2024-04-24T23:36:33.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f96e1ffb640 10 client.20478 handle_command_reply: tid=0 2024-04-24T23:36:33.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 client.20478 shutdown 2024-04-24T23:36:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 msgr2=0x55cbd0d619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d619e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f96f40b8c60 tx=0x7f96f40bc400 comp rx=0 tx=0).stop 2024-04-24T23:36:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 shutdown_connections 2024-04-24T23:36:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0d66290 0x55cbd0d686f0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d619e0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 --2- 172.21.15.33:0/2500024270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f96e1ffb640 0 client.20478 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.063+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 >> 172.21.15.33:0/2500024270 conn(0x55cbd0d3a8a0 msgr2=0x55cbd0d39b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.064+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 shutdown_connections 2024-04-24T23:36:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.064+0000 7f97039f1740 1 -- 172.21.15.33:0/2500024270 wait complete. 2024-04-24T23:36:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.064+0000 7f97039f1740 20 client.20478 trim_cache size 0 max 16384 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.089+0000 7f97039f1740 1 Processor -- start 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.089+0000 7f97039f1740 1 -- start start 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.089+0000 7f97039f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f97039f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35908/0 (socket says 172.21.15.33:35908) 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f96fa59c640 1 -- 172.21.15.33:0/3687431717 learned_addr learned my addr 172.21.15.33:0/3687431717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f97039f1740 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a2060 con 0x55cbd0d3fc20 2024-04-24T23:36:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0733c0 con 0x55cbd0d3f230 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc071210 con 0x55cbd0d44d90 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.090+0000 7f96fad9d640 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.091+0000 7f96f9d9b640 1 -- 172.21.15.33:0/3687431717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 msgr2=0x55cbd0d59a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.091+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d59a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.091+0000 7f96f9d9b640 1 -- 172.21.15.33:0/3687431717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 msgr2=0x55cbd0d3e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.091+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.091+0000 7f96f9d9b640 1 -- 172.21.15.33:0/3687431717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd0a817d0 con 0x55cbd0d3fc20 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.091+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d619a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f96f0013610 tx=0x7f96f00099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653c9e7b48a1b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.092+0000 7f96e3fff640 1 -- 172.21.15.33:0/3687431717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0005040 con 0x55cbd0d3fc20 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.092+0000 7f96e3fff640 1 -- 172.21.15.33:0/3687431717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0004680 con 0x55cbd0d3fc20 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.092+0000 7f96e3fff640 1 -- 172.21.15.33:0/3687431717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000f820 con 0x55cbd0d3fc20 2024-04-24T23:36:33.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.092+0000 7f96fa59c640 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:36:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.093+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3fc20 msgr2=0x55cbd0d619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.093+0000 7f97039f1740 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d619a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f96f0013610 tx=0x7f96f00099d0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.094+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 shutdown_connections 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.094+0000 7f97039f1740 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d44d90 0x55cbd0d59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.094+0000 7f97039f1740 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d619a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.094+0000 7f97039f1740 1 --2- 172.21.15.33:0/3687431717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d3e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.094+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 >> 172.21.15.33:0/3687431717 conn(0x55cbd0d3a8a0 msgr2=0x55cbd0d392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.096+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 shutdown_connections 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.097+0000 7f97039f1740 1 -- 172.21.15.33:0/3687431717 wait complete. 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.097+0000 7f97039f1740 1 -- start start 2024-04-24T23:36:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.097+0000 7f97039f1740 1 Processor -- start 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.097+0000 7f97039f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.097+0000 7f97039f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f97039f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d60970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35916/0 (socket says 172.21.15.33:35916) 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39774/0 (socket says 172.21.15.33:39774) 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f96fa59c640 1 -- 172.21.15.33:0/1610535185 learned_addr learned my addr 172.21.15.33:0/1610535185 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc071880 con 0x55cbd0d44d90 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc072cf0 con 0x55cbd0d3f230 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc071f50 con 0x55cbd0d3fc20 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.098+0000 7f97039f1740 10 client.0 _ll_register_callbacks cb 0x7f96f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.100+0000 7f96fa59c640 1 -- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3fc20 msgr2=0x55cbd0d5fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.100+0000 7f96fa59c640 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.100+0000 7f96fa59c640 1 -- 172.21.15.33:0/1610535185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 msgr2=0x55cbd0d60970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.100+0000 7f96fa59c640 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d60970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.100+0000 7f96fa59c640 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd0d60f60 con 0x55cbd0d3f230 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.101+0000 7f96fa59c640 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5f500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96ec026700 tx=0x7f96ec01e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbfe10f07a1f58c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.101+0000 7f96e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:36:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.102+0000 7f96e1ffb640 1 -- 172.21.15.33:0/1610535185 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec01c620 con 0x55cbd0d3f230 2024-04-24T23:36:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.102+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f0004680 con 0x55cbd0d3f230 2024-04-24T23:36:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.102+0000 7f96e1ffb640 1 -- 172.21.15.33:0/1610535185 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96ec011960 con 0x55cbd0d3f230 2024-04-24T23:36:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.106+0000 7f96e1ffb640 1 -- 172.21.15.33:0/1610535185 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96ec03b050 con 0x55cbd0d3f230 2024-04-24T23:36:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.111+0000 7f97039f1740 10 client.29936 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.111+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cbd0dccac0 con 0x55cbd0d3f230 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f96e1ffb640 1 -- 172.21.15.33:0/1610535185 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0004680 con 0x55cbd0d3f230 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f96e1ffb640 1 -- 172.21.15.33:0/1610535185 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f96ec023040 con 0x55cbd0d3f230 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f97039f1740 10 client.29936 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f97039f1740 10 client.29936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f97039f1740 20 client.29936 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f97039f1740 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0dcd550 0x55cbd0dcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f97039f1740 4 client.29936 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:36:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.112+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbd0caaf00 con 0x55cbd0dcd550 2024-04-24T23:36:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.113+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0dcd550 0x55cbd0dcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.119+0000 7f96f9d9b640 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0dcd550 0x55cbd0dcf970 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0f1a3722cb18a6 server_cookie=ffb48696cb120513 in_seq=0 out_seq=0 2024-04-24T23:36:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.119+0000 7f96e1ffb640 10 client.29936 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.119+0000 7f96e1ffb640 1 -- 172.21.15.33:0/1610535185 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbd0caaf00 con 0x55cbd0dcd550 2024-04-24T23:36:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.119+0000 7f96e1ffb640 10 client.29936 handle_command_reply: tid=0 2024-04-24T23:36:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.119+0000 7f97039f1740 1 client.29936 shutdown 2024-04-24T23:36:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.120+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 msgr2=0x55cbd0d5f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.120+0000 7f97039f1740 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5f500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96ec026700 tx=0x7f96ec01e040 comp rx=0 tx=0).stop 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 shutdown_connections 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f97039f1740 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55cbd0dcd550 0x55cbd0dcf970 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f97039f1740 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cbd0d44d90 0x55cbd0d60970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f97039f1740 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbd0d3fc20 0x55cbd0d5fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f97039f1740 1 --2- 172.21.15.33:0/1610535185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbd0d3f230 0x55cbd0d5f500 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 >> 172.21.15.33:0/1610535185 conn(0x55cbd0d3a8a0 msgr2=0x55cbd0d392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.121+0000 7f96e1ffb640 0 client.29936 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.124+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 shutdown_connections 2024-04-24T23:36:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.124+0000 7f97039f1740 1 -- 172.21.15.33:0/1610535185 wait complete. 2024-04-24T23:36:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.124+0000 7f97039f1740 20 client.29936 trim_cache size 0 max 16384 2024-04-24T23:36:33.125 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:36:33.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.131+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e40a2b90 msgr2=0x7f96e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.131+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e40a2b90 0x7f96e40a5050 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55cbd0d40020 tx=0x7f96ec005f90 comp rx=0 tx=0).stop 2024-04-24T23:36:33.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.131+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc107480 msgr2=0x7f96fc19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:33.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.131+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc107480 0x7f96fc19ee60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009b30 tx=0x7f96f4030890 comp rx=0 tx=0).stop 2024-04-24T23:36:33.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.133+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 shutdown_connections 2024-04-24T23:36:33.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.133+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e40a2b90 0x7f96e40a5050 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.133+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96fc107480 0x7f96fc19ee60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.133+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96fc104b50 0x7f96fc197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.133+0000 7f9700ce7640 1 --2- 172.21.15.33:0/1310966079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96fc102220 0x7f96fc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:33.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.133+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 >> 172.21.15.33:0/1310966079 conn(0x7f96fc0fbc30 msgr2=0x7f96fc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:33.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.136+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 shutdown_connections 2024-04-24T23:36:33.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:33.137+0000 7f9700ce7640 1 -- 172.21.15.33:0/1310966079 wait complete. 2024-04-24T23:36:33.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 264486 21.86 MB/sec execute 518 sec latency 170.685 ms 2024-04-24T23:36:33.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:33 smithi038 ceph-mon[32450]: pgmap v1087: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 338 op/s 2024-04-24T23:36:33.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:33 smithi117 ceph-mon[32529]: pgmap v1087: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 338 op/s 2024-04-24T23:36:33.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:33 smithi033 ceph-mon[29201]: pgmap v1087: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 338 op/s 2024-04-24T23:36:34.160 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:36:34.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:36:34.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:36:34.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 -- 172.21.15.33:0/310809924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf81006a0 msgr2=0x7facf8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 --2- 172.21.15.33:0/310809924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf81006a0 0x7facf8100ac0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7face8009900 tx=0x7face802a400 comp rx=0 tx=0).stop 2024-04-24T23:36:34.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 -- 172.21.15.33:0/310809924 shutdown_connections 2024-04-24T23:36:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 --2- 172.21.15.33:0/310809924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 --2- 172.21.15.33:0/310809924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facf8101890 0x7facf8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 --2- 172.21.15.33:0/310809924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf81006a0 0x7facf8100ac0 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7face8009900 tx=0x7face802a400 comp rx=0 tx=0).stop 2024-04-24T23:36:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.346+0000 7facfea41640 1 -- 172.21.15.33:0/310809924 >> 172.21.15.33:0/310809924 conn(0x7facf80fbc30 msgr2=0x7facf80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.347+0000 7facfea41640 1 -- 172.21.15.33:0/310809924 shutdown_connections 2024-04-24T23:36:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.348+0000 7facfea41640 1 -- 172.21.15.33:0/310809924 wait complete. 2024-04-24T23:36:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.349+0000 7facfea41640 1 Processor -- start 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.349+0000 7facfea41640 1 -- start start 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.349+0000 7facfea41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf8101890 0x7facf8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.349+0000 7facfea41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.349+0000 7facfea41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facf819d270 0x7facf819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.349+0000 7facfea41640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7facf806a5e0 con 0x7facf8102250 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.350+0000 7facfea41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7facf806a460 con 0x7facf8101890 2024-04-24T23:36:34.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.350+0000 7facfea41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facf806a760 con 0x7facf819d270 2024-04-24T23:36:34.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.352+0000 7facf77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.352+0000 7facf77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43374/0 (socket says 172.21.15.33:43374) 2024-04-24T23:36:34.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.352+0000 7facf77fe640 1 -- 172.21.15.33:0/497790712 learned_addr learned my addr 172.21.15.33:0/497790712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.353+0000 7facf7fff640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf8101890 0x7facf8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.353+0000 7facf77fe640 1 -- 172.21.15.33:0/497790712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facf819d270 msgr2=0x7facf819f660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:36:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.353+0000 7facf77fe640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facf819d270 0x7facf819f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.353+0000 7facf77fe640 1 -- 172.21.15.33:0/497790712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf8101890 msgr2=0x7facf8197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.353+0000 7facf77fe640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf8101890 0x7facf8197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.353+0000 7facf77fe640 1 -- 172.21.15.33:0/497790712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf806b290 con 0x7facf8102250 2024-04-24T23:36:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.354+0000 7facf77fe640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf81982a0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7facec00c890 tx=0x7facec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.356+0000 7facf57fa640 1 -- 172.21.15.33:0/497790712 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec004220 con 0x7facf8102250 2024-04-24T23:36:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.356+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf810b7c0 con 0x7facf8102250 2024-04-24T23:36:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.356+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf810bce0 con 0x7facf8102250 2024-04-24T23:36:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.356+0000 7facf57fa640 1 -- 172.21.15.33:0/497790712 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec0043c0 con 0x7facf8102250 2024-04-24T23:36:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.356+0000 7facf57fa640 1 -- 172.21.15.33:0/497790712 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec0049c0 con 0x7facf8102250 2024-04-24T23:36:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.360+0000 7facf57fa640 1 -- 172.21.15.33:0/497790712 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7facec020020 con 0x7facf8102250 2024-04-24T23:36:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.361+0000 7fad01820740 1 Processor -- start 2024-04-24T23:36:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.362+0000 7fad01820740 1 -- start start 2024-04-24T23:36:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.362+0000 7fad01820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f2661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.362+0000 7fad01820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f266780 0x56282f26b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.362+0000 7facf57fa640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7face00a2d60 0x7face00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.362+0000 7fad01820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f26b8f0 0x56282f26dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.362+0000 7facfcfde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f2661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facfcfde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f2661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43378/0 (socket says 172.21.15.33:43378) 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facf7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f266780 0x56282f26b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facf7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f266780 0x56282f26b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:39800/0 (socket says 172.21.15.33:39800) 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facfcfde640 1 -- 172.21.15.33:0/1578756833 learned_addr learned my addr 172.21.15.33:0/1578756833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facf57fa640 1 -- 172.21.15.33:0/497790712 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7facec0d3f50 con 0x7facf8102250 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56282f1c92c0 con 0x56282f265d90 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56282f171910 con 0x56282f26b8f0 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56282f1c5280 con 0x56282f266780 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facf77fe640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7face00a2d60 0x7face00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.363+0000 7facfcfde640 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f26b8f0 0x56282f26dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facfcfde640 1 -- 172.21.15.33:0/1578756833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f266780 msgr2=0x56282f26b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facfcfde640 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f266780 0x56282f26b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facfcfde640 1 -- 172.21.15.33:0/1578756833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f26b8f0 msgr2=0x56282f26dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facfcfde640 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f26b8f0 0x56282f26dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facfcfde640 1 -- 172.21.15.33:0/1578756833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282f182fb0 con 0x56282f265d90 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facf77fe640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7face00a2d60 0x7face00a5220 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7facec09cd00 tx=0x7facec013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facfcfde640 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f2661b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7facf0009f80 tx=0x7facf0009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be08d20d7230990 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facde7fc640 1 -- 172.21.15.33:0/1578756833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0018070 con 0x56282f265d90 2024-04-24T23:36:34.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facde7fc640 1 -- 172.21.15.33:0/1578756833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0010870 con 0x56282f265d90 2024-04-24T23:36:34.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.364+0000 7facde7fc640 1 -- 172.21.15.33:0/1578756833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00054c0 con 0x56282f265d90 2024-04-24T23:36:34.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.365+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 msgr2=0x56282f2661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.365+0000 7fad01820740 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f2661b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7facf0009f80 tx=0x7facf0009a40 comp rx=0 tx=0).stop 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 shutdown_connections 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f26b8f0 0x56282f26dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f266780 0x56282f26b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 --2- 172.21.15.33:0/1578756833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f2661b0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 >> 172.21.15.33:0/1578756833 conn(0x56282f22d950 msgr2=0x56282f261810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 shutdown_connections 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 -- 172.21.15.33:0/1578756833 wait complete. 2024-04-24T23:36:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.366+0000 7fad01820740 1 -- start start 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 Processor -- start 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f265d90 0x56282f278850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 0x56282f278d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56282f26e8a0 con 0x56282f266780 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56282f184e90 con 0x56282f265d90 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56282f26ea20 con 0x56282f26b8f0 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.367+0000 7fad01820740 10 client.0 _ll_register_callbacks cb 0x56282f1c0970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 0x56282f278d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facfcfde640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 0x56282f278d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43382/0 (socket says 172.21.15.33:43382) 2024-04-24T23:36:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 -- 172.21.15.33:0/284531786 learned_addr learned my addr 172.21.15.33:0/284531786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facfcfde640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f265d90 0x56282f278850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 -- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 msgr2=0x56282f27fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 -- 172.21.15.33:0/284531786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f265d90 msgr2=0x56282f278850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f265d90 0x56282f278850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 -- 172.21.15.33:0/284531786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282efa57e0 con 0x56282f266780 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facfcfde640 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.368+0000 7facf7fff640 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 0x56282f278d90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7facf81016f0 tx=0x7face8032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ee5c391a182ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.369+0000 7facd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.369+0000 7facd77fe640 1 -- 172.21.15.33:0/284531786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8035040 con 0x56282f266780 2024-04-24T23:36:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.369+0000 7facd77fe640 1 -- 172.21.15.33:0/284531786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7face80040c0 con 0x56282f266780 2024-04-24T23:36:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.369+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf0010870 con 0x56282f266780 2024-04-24T23:36:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.370+0000 7facd77fe640 1 -- 172.21.15.33:0/284531786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7face804f050 con 0x56282f266780 2024-04-24T23:36:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.370+0000 7fad01820740 10 client.20493 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.370+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56282f005f70 con 0x56282f266780 2024-04-24T23:36:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.370+0000 7facd77fe640 1 -- 172.21.15.33:0/284531786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0010870 con 0x56282f266780 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.371+0000 7facd77fe640 1 -- 172.21.15.33:0/284531786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7face8004a40 con 0x56282f266780 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.371+0000 7fad01820740 10 client.20493 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.371+0000 7fad01820740 10 client.20493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.371+0000 7fad01820740 20 client.20493 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.371+0000 7fad01820740 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f284310 0x56282f286730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.372+0000 7fad01820740 4 client.20493 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:34.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.372+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56282f19d5a0 con 0x56282f284310 2024-04-24T23:36:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.375+0000 7facfcfde640 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f284310 0x56282f286730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.383+0000 7facfcfde640 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f284310 0x56282f286730 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7158c2d12f5affec server_cookie=758d13002b828078 in_seq=0 out_seq=0 2024-04-24T23:36:34.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.383+0000 7facd77fe640 10 client.20493 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.383+0000 7facd77fe640 1 -- 172.21.15.33:0/284531786 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56282f19d5a0 con 0x56282f284310 2024-04-24T23:36:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.383+0000 7facd77fe640 10 client.20493 handle_command_reply: tid=0 2024-04-24T23:36:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.384+0000 7fad01820740 1 client.20493 shutdown 2024-04-24T23:36:34.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.386+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 msgr2=0x56282f278d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.386+0000 7fad01820740 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 0x56282f278d90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7facf81016f0 tx=0x7face8032e60 comp rx=0 tx=0).stop 2024-04-24T23:36:34.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.386+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 shutdown_connections 2024-04-24T23:36:34.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.387+0000 7fad01820740 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f284310 0x56282f286730 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.387+0000 7fad01820740 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.387+0000 7fad01820740 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f266780 0x56282f278d90 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.387+0000 7fad01820740 1 --2- 172.21.15.33:0/284531786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f265d90 0x56282f278850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.387+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 >> 172.21.15.33:0/284531786 conn(0x56282f22d950 msgr2=0x56282f260950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.388+0000 7facd77fe640 0 client.20493 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.388+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 shutdown_connections 2024-04-24T23:36:34.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.388+0000 7fad01820740 1 -- 172.21.15.33:0/284531786 wait complete. 2024-04-24T23:36:34.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.388+0000 7fad01820740 20 client.20493 trim_cache size 0 max 16384 2024-04-24T23:36:34.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.421+0000 7fad01820740 1 Processor -- start 2024-04-24T23:36:34.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 -- start start 2024-04-24T23:36:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f261290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f2617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7face40123f0 con 0x56282f265d90 2024-04-24T23:36:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7face400fc90 con 0x56282f266780 2024-04-24T23:36:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.422+0000 7fad01820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7face400db90 con 0x56282f26b8f0 2024-04-24T23:36:34.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f261290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f261290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43388/0 (socket says 172.21.15.33:43388) 2024-04-24T23:36:34.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 -- 172.21.15.33:0/2747071878 learned_addr learned my addr 172.21.15.33:0/2747071878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 -- 172.21.15.33:0/2747071878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 msgr2=0x56282f2617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f2617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 -- 172.21.15.33:0/2747071878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 msgr2=0x56282f27fd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 -- 172.21.15.33:0/2747071878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282ee9d170 con 0x56282f265d90 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.424+0000 7facfcfde640 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f261290 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7facf0009f80 tx=0x7facf00098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60eb1de2547d2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.425+0000 7facddffb640 1 -- 172.21.15.33:0/2747071878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0018040 con 0x56282f265d90 2024-04-24T23:36:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.425+0000 7facddffb640 1 -- 172.21.15.33:0/2747071878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0010b80 con 0x56282f265d90 2024-04-24T23:36:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.425+0000 7facddffb640 1 -- 172.21.15.33:0/2747071878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0025040 con 0x56282f265d90 2024-04-24T23:36:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 -- 172.21.15.33:0/2747071878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 msgr2=0x56282f261290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f261290 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7facf0009f80 tx=0x7facf00098e0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 -- 172.21.15.33:0/2747071878 shutdown_connections 2024-04-24T23:36:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f2617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 --2- 172.21.15.33:0/2747071878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f261290 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.426+0000 7fad01820740 1 -- 172.21.15.33:0/2747071878 >> 172.21.15.33:0/2747071878 conn(0x56282f22d950 msgr2=0x56282f267a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.427+0000 7fad01820740 1 -- 172.21.15.33:0/2747071878 shutdown_connections 2024-04-24T23:36:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.427+0000 7fad01820740 1 -- 172.21.15.33:0/2747071878 wait complete. 2024-04-24T23:36:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.427+0000 7fad01820740 1 -- start start 2024-04-24T23:36:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 Processor -- start 2024-04-24T23:36:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f27d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7face400e150 con 0x56282f265d90 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7face400f5c0 con 0x56282f266780 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7face400e820 con 0x56282f26b8f0 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7fad01820740 10 client.0 _ll_register_callbacks cb 0x7face8031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7facf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7facf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35962/0 (socket says 172.21.15.33:35962) 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.428+0000 7facf7fff640 1 -- 172.21.15.33:0/338113716 learned_addr learned my addr 172.21.15.33:0/338113716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facfcfde640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facfcfde640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f27d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facf7fff640 1 -- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 msgr2=0x56282f27e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facf7fff640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facf7fff640 1 -- 172.21.15.33:0/338113716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 msgr2=0x56282f27d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facf7fff640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f27d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facf7fff640 1 -- 172.21.15.33:0/338113716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282f27ebb0 con 0x56282f266780 2024-04-24T23:36:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facf7fff640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27d5b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face8003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1f6275fa7537a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facfcfde640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.429+0000 7facd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.430+0000 7facd77fe640 1 -- 172.21.15.33:0/338113716 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8035030 con 0x56282f266780 2024-04-24T23:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.430+0000 7facd77fe640 1 -- 172.21.15.33:0/338113716 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7face80046c0 con 0x56282f266780 2024-04-24T23:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.432+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf0010b80 con 0x56282f266780 2024-04-24T23:36:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.432+0000 7facd77fe640 1 -- 172.21.15.33:0/338113716 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7face80319b0 con 0x56282f266780 2024-04-24T23:36:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.435+0000 7fad01820740 10 client.29951 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.435+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56282f27ff20 con 0x56282f266780 2024-04-24T23:36:34.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.436+0000 7facd77fe640 1 -- 172.21.15.33:0/338113716 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0010b80 con 0x56282f266780 2024-04-24T23:36:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7facd77fe640 1 -- 172.21.15.33:0/338113716 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7face8046470 con 0x56282f266780 2024-04-24T23:36:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7fad01820740 10 client.29951 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7fad01820740 10 client.29951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7fad01820740 20 client.29951 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7fad01820740 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f26f420 0x56282f271840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7fad01820740 4 client.29951 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:36:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56282f1d6800 con 0x56282f26f420 2024-04-24T23:36:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.437+0000 7facfcfde640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f26f420 0x56282f271840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.438+0000 7facfcfde640 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f26f420 0x56282f271840 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22c18e1b17ded58 server_cookie=3d529e3a80da6e9c in_seq=0 out_seq=0 2024-04-24T23:36:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.438+0000 7facd77fe640 10 client.29951 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.439+0000 7facd77fe640 1 -- 172.21.15.33:0/338113716 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56282f1d6800 con 0x56282f26f420 2024-04-24T23:36:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7facd77fe640 10 client.29951 handle_command_reply: tid=0 2024-04-24T23:36:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 client.29951 shutdown 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 msgr2=0x56282f27d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27d5b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face8003800 comp rx=0 tx=0).stop 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 shutdown_connections 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56282f26f420 0x56282f271840 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56282f26b8f0 0x56282f27e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56282f266780 0x56282f27d5b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 --2- 172.21.15.33:0/338113716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56282f265d90 0x56282f27d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7facd77fe640 0 client.29951 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 >> 172.21.15.33:0/338113716 conn(0x56282f22d950 msgr2=0x56282f261290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 shutdown_connections 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.440+0000 7fad01820740 1 -- 172.21.15.33:0/338113716 wait complete. 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7fad01820740 20 client.29951 trim_cache size 0 max 16384 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "6c751a6a-8df3-43e3-acca-f6c2a3d532a4", 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:36:34.441 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:36:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7face00a2d60 msgr2=0x7face00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7face00a2d60 0x7face00a5220 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7facec09cd00 tx=0x7facec013040 comp rx=0 tx=0).stop 2024-04-24T23:36:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 msgr2=0x7facf81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf81982a0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7facec00c890 tx=0x7facec00cd50 comp rx=0 tx=0).stop 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 shutdown_connections 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7face00a2d60 0x7face00a5220 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facf819d270 0x7facf819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf8102250 0x7facf81982a0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 --2- 172.21.15.33:0/497790712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facf8101890 0x7facf8197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.441+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 >> 172.21.15.33:0/497790712 conn(0x7facf80fbc30 msgr2=0x7facf8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.442+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 shutdown_connections 2024-04-24T23:36:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.442+0000 7facfea41640 1 -- 172.21.15.33:0/497790712 wait complete. 2024-04-24T23:36:34.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:36:34.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.576+0000 7fb909d61640 1 -- 172.21.15.33:0/2719047416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904101870 msgr2=0x7fb904101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.576+0000 7fb909d61640 1 --2- 172.21.15.33:0/2719047416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904101870 0x7fb904101cf0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009900 tx=0x7fb8f402a410 comp rx=0 tx=0).stop 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 -- 172.21.15.33:0/2719047416 shutdown_connections 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 --2- 172.21.15.33:0/2719047416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb904102230 0x7fb904108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 --2- 172.21.15.33:0/2719047416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904101870 0x7fb904101cf0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 --2- 172.21.15.33:0/2719047416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904100680 0x7fb904100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 -- 172.21.15.33:0/2719047416 >> 172.21.15.33:0/2719047416 conn(0x7fb9040fbbf0 msgr2=0x7fb9040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 -- 172.21.15.33:0/2719047416 shutdown_connections 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 -- 172.21.15.33:0/2719047416 wait complete. 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 Processor -- start 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.577+0000 7fb909d61640 1 -- start start 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb904100680 0x7fb904197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904101870 0x7fb904198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 0x7fb90419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb90410c300 con 0x7fb904102230 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb90410c180 con 0x7fb904101870 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb90410c480 con 0x7fb904100680 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 0x7fb90419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb904100680 0x7fb904197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 0x7fb90419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43422/0 (socket says 172.21.15.33:43422) 2024-04-24T23:36:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 -- 172.21.15.33:0/500477116 learned_addr learned my addr 172.21.15.33:0/500477116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb902ffd640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904101870 0x7fb904198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 -- 172.21.15.33:0/500477116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb904100680 msgr2=0x7fb904197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb904100680 0x7fb904197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 -- 172.21.15.33:0/500477116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904101870 msgr2=0x7fb904198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904101870 0x7fb904198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 -- 172.21.15.33:0/500477116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90419f720 con 0x7fb904102230 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb902ffd640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904101870 0x7fb904198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb903fff640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 0x7fb90419f0b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00cad0 tx=0x7fb8fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb900ff9640 1 -- 172.21.15.33:0/500477116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00ae20 con 0x7fb904102230 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb900ff9640 1 -- 172.21.15.33:0/500477116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8fc005020 con 0x7fb904102230 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90419fa10 con 0x7fb904102230 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb900ff9640 1 -- 172.21.15.33:0/500477116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00f6f0 con 0x7fb904102230 2024-04-24T23:36:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.578+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90419ff30 con 0x7fb904102230 2024-04-24T23:36:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb900ff9640 1 -- 172.21.15.33:0/500477116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb8fc0051d0 con 0x7fb904102230 2024-04-24T23:36:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 Processor -- start 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 -- start start 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398ea4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 0x557398ea9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557398e046b0 con 0x557398ea5040 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557398dac880 con 0x557398ea4650 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557398e00670 con 0x557398eaa1b0 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 0x557398ea9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 0x557398ea9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43434/0 (socket says 172.21.15.33:43434) 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb9037fe640 1 -- 172.21.15.33:0/216158286 learned_addr learned my addr 172.21.15.33:0/216158286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.581+0000 7fb902ffd640 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398ea4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb902ffd640 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb900ff9640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8ec0a2b90 0x7fb8ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb9037fe640 1 -- 172.21.15.33:0/216158286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 msgr2=0x557398eac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb9037fe640 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb9037fe640 1 -- 172.21.15.33:0/216158286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 msgr2=0x557398ea4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb9037fe640 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398ea4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb9037fe640 1 -- 172.21.15.33:0/216158286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557398be27d0 con 0x557398ea5040 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb903fff640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8ec0a2b90 0x7fb8ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb9037fe640 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 0x557398ea9b40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009ca0 tx=0x7fb8f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8558996ee25cb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb900ff9640 1 -- 172.21.15.33:0/500477116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8fc0cf680 con 0x7fb904102230 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb8e9ffb640 1 -- 172.21.15.33:0/216158286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8009e50 con 0x557398ea5040 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb8e9ffb640 1 -- 172.21.15.33:0/216158286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f80035f0 con 0x557398ea5040 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.582+0000 7fb8e9ffb640 1 -- 172.21.15.33:0/216158286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8019670 con 0x557398ea5040 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 -- 172.21.15.33:0/216158286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 msgr2=0x557398ea9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 0x557398ea9b40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009ca0 tx=0x7fb8f8011810 comp rx=0 tx=0).stop 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb903fff640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8ec0a2b90 0x7fb8ec0a5050 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0a4290 tx=0x7fb8fc096540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 -- 172.21.15.33:0/216158286 shutdown_connections 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea5040 0x557398ea9b40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/216158286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398ea4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 -- 172.21.15.33:0/216158286 >> 172.21.15.33:0/216158286 conn(0x557398e9b8b0 msgr2=0x557398e9bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 -- 172.21.15.33:0/216158286 shutdown_connections 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.583+0000 7fb90ca6d740 1 -- 172.21.15.33:0/216158286 wait complete. 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 -- start start 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 Processor -- start 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea4650 0x557398eaea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 0x557398eaefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eb5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557398ead160 con 0x557398ea4650 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557398dbff50 con 0x557398ea5040 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557398ead2e0 con 0x557398eaa1b0 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.584+0000 7fb90ca6d740 10 client.0 _ll_register_callbacks cb 0x557398dfb000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 0x557398eaefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea4650 0x557398eaea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 0x557398eaefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:35992/0 (socket says 172.21.15.33:35992) 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 -- 172.21.15.33:0/3302203314 learned_addr learned my addr 172.21.15.33:0/3302203314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb902ffd640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea4650 0x557398eaea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43448/0 (socket says 172.21.15.33:43448) 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb902ffd640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eb5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 -- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 msgr2=0x557398eb5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eb5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 -- 172.21.15.33:0/3302203314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea4650 msgr2=0x557398eaea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea4650 0x557398eaea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 -- 172.21.15.33:0/3302203314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557398b183f0 con 0x557398ea5040 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb9037fe640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 0x557398eaefb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009c70 tx=0x7fb8f80039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248cee7caec74e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb8e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/3302203314 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8028020 con 0x557398ea5040 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f80035f0 con 0x557398ea5040 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.585+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/3302203314 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8f8016040 con 0x557398ea5040 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/3302203314 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8f8033050 con 0x557398ea5040 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 10 client.29963 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557398ad80f0 con 0x557398ea5040 2024-04-24T23:36:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/3302203314 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f80035f0 con 0x557398ea5040 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/3302203314 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb8f8004800 con 0x557398ea5040 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 10 client.29963 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 10 client.29963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 20 client.29963 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398eba6d0 0x557398ebcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 4 client.29963 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557398e11bf0 con 0x557398eba6d0 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.586+0000 7fb902ffd640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398eba6d0 0x557398ebcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.587+0000 7fb8e2ffd640 10 client.29963 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.587+0000 7fb902ffd640 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398eba6d0 0x557398ebcb50 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240307d1f5e160f1 server_cookie=ab20184c63b8e856 in_seq=0 out_seq=0 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/3302203314 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557398e11bf0 con 0x557398eba6d0 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb8e2ffd640 10 client.29963 handle_command_reply: tid=0 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 client.29963 shutdown 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 msgr2=0x557398eaefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 0x557398eaefb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009c70 tx=0x7fb8f80039d0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 shutdown_connections 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398eba6d0 0x557398ebcb50 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398eaa1b0 0x557398eb5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea5040 0x557398eaefb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/3302203314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398ea4650 0x557398eaea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 >> 172.21.15.33:0/3302203314 conn(0x557398e9b8b0 msgr2=0x557398e9aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb8e2ffd640 0 client.29963 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 shutdown_connections 2024-04-24T23:36:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 1 -- 172.21.15.33:0/3302203314 wait complete. 2024-04-24T23:36:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.588+0000 7fb90ca6d740 20 client.29963 trim_cache size 0 max 16384 2024-04-24T23:36:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 Processor -- start 2024-04-24T23:36:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 -- start start 2024-04-24T23:36:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398e9a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398e9a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041aab20 con 0x557398eaa1b0 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041a83c0 con 0x557398ea4650 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041a6210 con 0x557398ea5040 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398e9a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398e9a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43456/0 (socket says 172.21.15.33:43456) 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 -- 172.21.15.33:0/966371873 learned_addr learned my addr 172.21.15.33:0/966371873 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb9037fe640 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 -- 172.21.15.33:0/966371873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 msgr2=0x557398eb5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 -- 172.21.15.33:0/966371873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 msgr2=0x557398e9a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398e9a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.610+0000 7fb902ffd640 1 -- 172.21.15.33:0/966371873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557398e9c620 con 0x557398eaa1b0 2024-04-24T23:36:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.611+0000 7fb902ffd640 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398e9a7a0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4031610 tx=0x7fb8f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dc623dcffb286e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.611+0000 7fb8e97fa640 1 -- 172.21.15.33:0/966371873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4053070 con 0x557398eaa1b0 2024-04-24T23:36:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.611+0000 7fb8e97fa640 1 -- 172.21.15.33:0/966371873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f4004c40 con 0x557398eaa1b0 2024-04-24T23:36:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.611+0000 7fb8e97fa640 1 -- 172.21.15.33:0/966371873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4035040 con 0x557398eaa1b0 2024-04-24T23:36:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 -- 172.21.15.33:0/966371873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 msgr2=0x557398e9a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398e9a7a0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4031610 tx=0x7fb8f4005dd0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 -- 172.21.15.33:0/966371873 shutdown_connections 2024-04-24T23:36:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398e9a7a0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/966371873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398e9a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 -- 172.21.15.33:0/966371873 >> 172.21.15.33:0/966371873 conn(0x557398e9b8b0 msgr2=0x557398eaea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 -- 172.21.15.33:0/966371873 shutdown_connections 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 -- 172.21.15.33:0/966371873 wait complete. 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 -- start start 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.612+0000 7fb90ca6d740 1 Processor -- start 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398eb37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398eb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041a6880 con 0x557398eaa1b0 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041a7cf0 con 0x557398ea4650 2024-04-24T23:36:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041a6f50 con 0x557398ea5040 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 10 client.0 _ll_register_callbacks cb 0x7fb8f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398eb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398eb4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43460/0 (socket says 172.21.15.33:43460) 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb902ffd640 1 -- 172.21.15.33:0/1412904304 learned_addr learned my addr 172.21.15.33:0/1412904304 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb902ffd640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398eb37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 -- 172.21.15.33:0/1412904304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 msgr2=0x557398eb37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398eb37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 -- 172.21.15.33:0/1412904304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 msgr2=0x557398eb4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398eb4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 -- 172.21.15.33:0/1412904304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557398eb5290 con 0x557398ea5040 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb902ffd640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398eb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:36:34.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb9037fe640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb3d00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8004c20 tx=0x7fb8f8005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a0169e17dac00c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb8e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/1412904304 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8028070 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.613+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f4004c40 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/1412904304 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb8f8005250 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/1412904304 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8f80035f0 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 10 client.29911 fetch_fsmap learned FSMap version 35 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557398f20d50 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/1412904304 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f4004c40 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/1412904304 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb8f8005470 con 0x557398ea5040 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 10 client.29911 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 10 client.29911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 20 client.29911 populate_metadata read hostname 'smithi033' 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398f216e0 0x557398f23b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:36:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 4 client.29911 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557398e0bf40 con 0x557398f216e0 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.614+0000 7fb902ffd640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398f216e0 0x557398f23b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb8e2ffd640 10 client.29911 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb902ffd640 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398f216e0 0x557398f23b00 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dbf90ffc0935a94 server_cookie=7163d36e5c0e65ff in_seq=0 out_seq=0 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb8e2ffd640 1 -- 172.21.15.33:0/1412904304 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557398e0bf40 con 0x557398f216e0 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb8e2ffd640 10 client.29911 handle_command_reply: tid=0 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 client.29911 shutdown 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 msgr2=0x557398eb3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb3d00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8004c20 tx=0x7fb8f8005d30 comp rx=0 tx=0).stop 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 shutdown_connections 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x557398f216e0 0x557398f23b00 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557398eaa1b0 0x557398eb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557398ea5040 0x557398eb3d00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb8e2ffd640 0 client.29911 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:36:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 --2- 172.21.15.33:0/1412904304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557398ea4650 0x557398eb37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.615+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 >> 172.21.15.33:0/1412904304 conn(0x557398e9b8b0 msgr2=0x557398e9e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 shutdown_connections 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb90ca6d740 1 -- 172.21.15.33:0/1412904304 wait complete. 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb90ca6d740 20 client.29911 trim_cache size 0 max 16384 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stdout: "6c751a6a-8df3-43e3-acca-f6c2a3d532a4": { 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:36:34.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "6c751a6a-8df3-43e3-acca-f6c2a3d532a4", 2024-04-24T23:36:34.622 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:36:34.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:36:34.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:36:34.622 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8ec0a2b90 msgr2=0x7fb8ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8ec0a2b90 0x7fb8ec0a5050 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0a4290 tx=0x7fb8fc096540 comp rx=0 tx=0).stop 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 msgr2=0x7fb90419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 0x7fb90419f0b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00cad0 tx=0x7fb8fc00cf90 comp rx=0 tx=0).stop 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 shutdown_connections 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb8ec0a2b90 0x7fb8ec0a5050 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904102230 0x7fb90419f0b0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb904101870 0x7fb904198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 --2- 172.21.15.33:0/500477116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb904100680 0x7fb904197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 >> 172.21.15.33:0/500477116 conn(0x7fb9040fbbf0 msgr2=0x7fb9040688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.616+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 shutdown_connections 2024-04-24T23:36:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:36:34.617+0000 7fb909d61640 1 -- 172.21.15.33:0/500477116 wait complete. 2024-04-24T23:36:34.627 INFO:tasks.cephfs.filesystem:scrub status for tag:6c751a6a-8df3-43e3-acca-f6c2a3d532a4 - {'path': '/', 'tag': '6c751a6a-8df3-43e3-acca-f6c2a3d532a4', 'options': 'recursive,force'} 2024-04-24T23:36:34.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 264762 21.85 MB/sec execute 519 sec latency 134.552 ms 2024-04-24T23:36:34.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:34 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:36:34.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:34 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:34.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:34 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:34.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:34 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:36:34.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:34 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:34.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:34 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:34.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:34 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:36:34.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:34 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:34.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:34 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:35.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 265089 21.84 MB/sec execute 520 sec latency 78.902 ms 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: pgmap v1088: 129 pgs: 129 active+clean; 638 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 397 op/s 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:35 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: pgmap v1088: 129 pgs: 129 active+clean; 638 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 397 op/s 2024-04-24T23:36:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:35.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:36:35.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:35 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: pgmap v1088: 129 pgs: 129 active+clean; 638 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 397 op/s 2024-04-24T23:36:35.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:35.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:36:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:35.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:35 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:36:36.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 265440 21.82 MB/sec execute 521 sec latency 115.029 ms 2024-04-24T23:36:37.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:37 smithi033 ceph-mon[29201]: pgmap v1089: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 425 op/s 2024-04-24T23:36:37.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 265639 21.79 MB/sec execute 522 sec latency 355.630 ms 2024-04-24T23:36:37.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:37 smithi038 ceph-mon[32450]: pgmap v1089: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 425 op/s 2024-04-24T23:36:37.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:37 smithi117 ceph-mon[32529]: pgmap v1089: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 425 op/s 2024-04-24T23:36:38.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 266029 21.79 MB/sec execute 523 sec latency 157.544 ms 2024-04-24T23:36:39.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 266442 21.81 MB/sec execute 524 sec latency 116.788 ms 2024-04-24T23:36:39.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:39 smithi038 ceph-mon[32450]: pgmap v1090: 129 pgs: 129 active+clean; 646 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.2 MiB/s wr, 451 op/s 2024-04-24T23:36:39.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:39 smithi117 ceph-mon[32529]: pgmap v1090: 129 pgs: 129 active+clean; 646 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.2 MiB/s wr, 451 op/s 2024-04-24T23:36:39.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:39 smithi033 ceph-mon[29201]: pgmap v1090: 129 pgs: 129 active+clean; 646 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.2 MiB/s wr, 451 op/s 2024-04-24T23:36:40.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 267057 21.81 MB/sec execute 525 sec latency 92.955 ms 2024-04-24T23:36:41.907 INFO:tasks.workunit.client.0.smithi033.stdout: 10 267575 21.83 MB/sec execute 526 sec latency 44.911 ms 2024-04-24T23:36:41.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:41 smithi038 ceph-mon[32450]: pgmap v1091: 129 pgs: 129 active+clean; 650 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 428 op/s 2024-04-24T23:36:41.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:41 smithi117 ceph-mon[32529]: pgmap v1091: 129 pgs: 129 active+clean; 650 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 428 op/s 2024-04-24T23:36:41.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:41 smithi033 ceph-mon[29201]: pgmap v1091: 129 pgs: 129 active+clean; 650 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 428 op/s 2024-04-24T23:36:42.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 267906 21.83 MB/sec execute 527 sec latency 309.304 ms 2024-04-24T23:36:43.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 268210 21.82 MB/sec execute 528 sec latency 464.586 ms 2024-04-24T23:36:43.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:43 smithi038 ceph-mon[32450]: pgmap v1092: 129 pgs: 129 active+clean; 650 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 415 op/s 2024-04-24T23:36:43.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:43 smithi117 ceph-mon[32529]: pgmap v1092: 129 pgs: 129 active+clean; 650 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 415 op/s 2024-04-24T23:36:43.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:43 smithi033 ceph-mon[29201]: pgmap v1092: 129 pgs: 129 active+clean; 650 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 415 op/s 2024-04-24T23:36:44.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 268509 21.82 MB/sec execute 529 sec latency 155.138 ms 2024-04-24T23:36:45.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 268806 21.81 MB/sec execute 530 sec latency 78.708 ms 2024-04-24T23:36:45.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:45 smithi038 ceph-mon[32450]: pgmap v1093: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 449 op/s 2024-04-24T23:36:45.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:45 smithi117 ceph-mon[32529]: pgmap v1093: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 449 op/s 2024-04-24T23:36:45.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:45 smithi033 ceph-mon[29201]: pgmap v1093: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 449 op/s 2024-04-24T23:36:46.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 269136 21.79 MB/sec execute 531 sec latency 101.187 ms 2024-04-24T23:36:47.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 269534 21.79 MB/sec execute 532 sec latency 207.047 ms 2024-04-24T23:36:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:47 smithi038 ceph-mon[32450]: pgmap v1094: 129 pgs: 129 active+clean; 640 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-04-24T23:36:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:47 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:36:47.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:47 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:36:47.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:47 smithi117 ceph-mon[32529]: pgmap v1094: 129 pgs: 129 active+clean; 640 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-04-24T23:36:47.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:47 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:36:47.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:47 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:36:47.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:47 smithi033 ceph-mon[29201]: pgmap v1094: 129 pgs: 129 active+clean; 640 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-04-24T23:36:47.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:47 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:36:47.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:47 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:36:48.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 269908 21.77 MB/sec execute 533 sec latency 375.131 ms 2024-04-24T23:36:48.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:48 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:36:48.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:48 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:36:48.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:48 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:36:49.825 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:49 smithi117 ceph-mon[32529]: pgmap v1095: 129 pgs: 129 active+clean; 633 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 410 op/s 2024-04-24T23:36:49.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 270364 21.78 MB/sec execute 534 sec latency 136.546 ms 2024-04-24T23:36:49.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:49 smithi038 ceph-mon[32450]: pgmap v1095: 129 pgs: 129 active+clean; 633 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 410 op/s 2024-04-24T23:36:49.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:49 smithi033 ceph-mon[29201]: pgmap v1095: 129 pgs: 129 active+clean; 633 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 410 op/s 2024-04-24T23:36:50.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 270802 21.80 MB/sec execute 535 sec latency 67.470 ms 2024-04-24T23:36:51.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 271329 21.80 MB/sec execute 536 sec latency 87.376 ms 2024-04-24T23:36:51.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:51 smithi038 ceph-mon[32450]: pgmap v1096: 129 pgs: 129 active+clean; 633 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 356 op/s 2024-04-24T23:36:51.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:51 smithi117 ceph-mon[32529]: pgmap v1096: 129 pgs: 129 active+clean; 633 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 356 op/s 2024-04-24T23:36:51.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:51 smithi033 ceph-mon[29201]: pgmap v1096: 129 pgs: 129 active+clean; 633 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 356 op/s 2024-04-24T23:36:52.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 271691 21.79 MB/sec execute 537 sec latency 97.471 ms 2024-04-24T23:36:53.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 272042 21.80 MB/sec execute 538 sec latency 367.579 ms 2024-04-24T23:36:53.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:53 smithi038 ceph-mon[32450]: pgmap v1097: 129 pgs: 129 active+clean; 634 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-04-24T23:36:53.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:53 smithi117 ceph-mon[32529]: pgmap v1097: 129 pgs: 129 active+clean; 634 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-04-24T23:36:53.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:53 smithi033 ceph-mon[29201]: pgmap v1097: 129 pgs: 129 active+clean; 634 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-04-24T23:36:54.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 272361 21.79 MB/sec execute 539 sec latency 136.692 ms 2024-04-24T23:36:55.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 272840 21.81 MB/sec execute 540 sec latency 63.728 ms 2024-04-24T23:36:55.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:55 smithi038 ceph-mon[32450]: pgmap v1098: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 409 op/s 2024-04-24T23:36:55.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:55 smithi117 ceph-mon[32529]: pgmap v1098: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 409 op/s 2024-04-24T23:36:55.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:55 smithi033 ceph-mon[29201]: pgmap v1098: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 409 op/s 2024-04-24T23:36:56.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 273275 21.80 MB/sec execute 541 sec latency 93.842 ms 2024-04-24T23:36:57.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 273675 21.80 MB/sec execute 542 sec latency 73.887 ms 2024-04-24T23:36:57.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:57 smithi038 ceph-mon[32450]: pgmap v1099: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 367 op/s 2024-04-24T23:36:57.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:57 smithi117 ceph-mon[32529]: pgmap v1099: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 367 op/s 2024-04-24T23:36:57.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:57 smithi033 ceph-mon[29201]: pgmap v1099: 129 pgs: 129 active+clean; 643 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 367 op/s 2024-04-24T23:36:58.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 273920 21.77 MB/sec execute 543 sec latency 339.228 ms 2024-04-24T23:36:59.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 274263 21.77 MB/sec execute 544 sec latency 107.506 ms 2024-04-24T23:36:59.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:36:59 smithi038 ceph-mon[32450]: pgmap v1100: 129 pgs: 129 active+clean; 646 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-04-24T23:36:59.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:36:59 smithi117 ceph-mon[32529]: pgmap v1100: 129 pgs: 129 active+clean; 646 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-04-24T23:36:59.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:36:59 smithi033 ceph-mon[29201]: pgmap v1100: 129 pgs: 129 active+clean; 646 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-04-24T23:37:00.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 274675 21.78 MB/sec execute 545 sec latency 57.712 ms 2024-04-24T23:37:01.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 275084 21.77 MB/sec execute 546 sec latency 120.068 ms 2024-04-24T23:37:01.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:01 smithi038 ceph-mon[32450]: pgmap v1101: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-04-24T23:37:01.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:01 smithi117 ceph-mon[32529]: pgmap v1101: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-04-24T23:37:01.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:01 smithi033 ceph-mon[29201]: pgmap v1101: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-04-24T23:37:02.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 275658 21.78 MB/sec execute 547 sec latency 83.452 ms 2024-04-24T23:37:03.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 276126 21.79 MB/sec execute 548 sec latency 247.910 ms 2024-04-24T23:37:03.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:03 smithi038 ceph-mon[32450]: pgmap v1102: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 391 op/s 2024-04-24T23:37:03.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:03 smithi117 ceph-mon[32529]: pgmap v1102: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 391 op/s 2024-04-24T23:37:03.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:03 smithi033 ceph-mon[29201]: pgmap v1102: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 391 op/s 2024-04-24T23:37:04.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:37:04.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:37:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.816+0000 7fb9d4429640 1 -- 172.21.15.33:0/142624547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 msgr2=0x7fb9cc108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.816+0000 7fb9d4429640 1 --2- 172.21.15.33:0/142624547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 0x7fb9cc108ac0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009ca0 tx=0x7fb9c402a8d0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 -- 172.21.15.33:0/142624547 shutdown_connections 2024-04-24T23:37:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 --2- 172.21.15.33:0/142624547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 0x7fb9cc108ac0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 --2- 172.21.15.33:0/142624547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc101870 0x7fb9cc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 --2- 172.21.15.33:0/142624547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc100680 0x7fb9cc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 -- 172.21.15.33:0/142624547 >> 172.21.15.33:0/142624547 conn(0x7fb9cc0fbbf0 msgr2=0x7fb9cc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 -- 172.21.15.33:0/142624547 shutdown_connections 2024-04-24T23:37:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.817+0000 7fb9d4429640 1 -- 172.21.15.33:0/142624547 wait complete. 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 Processor -- start 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 -- start start 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc100680 0x7fb9cc19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc101870 0x7fb9cc19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 0x7fb9cc1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10b1c0 con 0x7fb9cc101870 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10b040 con 0x7fb9cc102230 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.818+0000 7fb9d4429640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10b340 con 0x7fb9cc100680 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d19c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc101870 0x7fb9cc19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d19c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc101870 0x7fb9cc19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40390/0 (socket says 172.21.15.33:40390) 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2122589503 learned_addr learned my addr 172.21.15.33:0/2122589503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc100680 0x7fb9cc19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 0x7fb9cc1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2122589503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 msgr2=0x7fb9cc1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 0x7fb9cc1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc101870 msgr2=0x7fb9cc19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.819+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc101870 0x7fb9cc19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.820+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2122589503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc10ff30 con 0x7fb9cc100680 2024-04-24T23:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.820+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc100680 0x7fb9cc19e060 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800af00 tx=0x7fb9c800cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.820+0000 7fb9c37fe640 1 -- 172.21.15.33:0/2122589503 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8004220 con 0x7fb9cc100680 2024-04-24T23:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.821+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc110220 con 0x7fb9cc100680 2024-04-24T23:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.821+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1107c0 con 0x7fb9cc100680 2024-04-24T23:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.821+0000 7fb9c37fe640 1 -- 172.21.15.33:0/2122589503 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c80043c0 con 0x7fb9cc100680 2024-04-24T23:37:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.822+0000 7fb9c37fe640 1 -- 172.21.15.33:0/2122589503 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8031460 con 0x7fb9cc100680 2024-04-24T23:37:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 Processor -- start 2024-04-24T23:37:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 -- start start 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8cc30 0x564e67d91730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d91da0 0x564e67d94230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564e67cf06b0 con 0x564e67d91da0 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e67c98880 con 0x564e67d8c240 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d720b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e67cec670 con 0x564e67d8cc30 2024-04-24T23:37:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d29c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d19c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d91da0 0x564e67d94230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d29c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d8c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46282/0 (socket says 172.21.15.33:46282) 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.825+0000 7fb9d29c6640 1 -- 172.21.15.33:0/2622259715 learned_addr learned my addr 172.21.15.33:0/2622259715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.826+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2622259715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8cc30 msgr2=0x564e67d91730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.826+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8cc30 0x564e67d91730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.826+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2622259715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 msgr2=0x564e67d8c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.826+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d8c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.826+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2622259715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e67ace7d0 con 0x564e67d91da0 2024-04-24T23:37:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.826+0000 7fb9c37fe640 1 -- 172.21.15.33:0/2122589503 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb9c80079b0 con 0x7fb9cc100680 2024-04-24T23:37:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.827+0000 7fb9c37fe640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9b80a2b30 0x7fb9b80a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.828+0000 7fb9c37fe640 1 -- 172.21.15.33:0/2122589503 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9c80f0e10 con 0x7fb9cc100680 2024-04-24T23:37:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.828+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d91da0 0x564e67d94230 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0058c0 tx=0x7fb9bc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30228bcd97b4271 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.828+0000 7fb9c0bfe640 1 -- 172.21.15.33:0/2622259715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc010630 con 0x564e67d91da0 2024-04-24T23:37:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.828+0000 7fb9c0bfe640 1 -- 172.21.15.33:0/2622259715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc010cb0 con 0x564e67d91da0 2024-04-24T23:37:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.829+0000 7fb9c0bfe640 1 -- 172.21.15.33:0/2622259715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc019c10 con 0x564e67d91da0 2024-04-24T23:37:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.830+0000 7fb9d720b740 1 -- 172.21.15.33:0/2622259715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d91da0 msgr2=0x564e67d94230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.830+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d91da0 0x564e67d94230 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0058c0 tx=0x7fb9bc00e720 comp rx=0 tx=0).stop 2024-04-24T23:37:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.831+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9b80a2b30 0x7fb9b80a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.832+0000 7fb9d720b740 1 -- 172.21.15.33:0/2622259715 shutdown_connections 2024-04-24T23:37:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.832+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d91da0 0x564e67d94230 secure :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0058c0 tx=0x7fb9bc00e720 comp rx=0 tx=0).stop 2024-04-24T23:37:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.832+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8cc30 0x564e67d91730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.832+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2622259715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d8c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.832+0000 7fb9d720b740 1 -- 172.21.15.33:0/2622259715 >> 172.21.15.33:0/2622259715 conn(0x564e67d878b0 msgr2=0x564e67d87cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.833+0000 7fb9d720b740 1 -- 172.21.15.33:0/2622259715 shutdown_connections 2024-04-24T23:37:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.833+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9b80a2b30 0x7fb9b80a4ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x564e67d8d490 tx=0x7fb9c402add0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.834+0000 7fb9d720b740 1 -- 172.21.15.33:0/2622259715 wait complete. 2024-04-24T23:37:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.834+0000 7fb9d720b740 1 -- start start 2024-04-24T23:37:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.834+0000 7fb9d720b740 1 Processor -- start 2024-04-24T23:37:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.835+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.835+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.835+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d9e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.835+0000 7fb9d19c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.835+0000 7fb9d21c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.835+0000 7fb9d19c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40408/0 (socket says 172.21.15.33:40408) 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d720b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564e67d94d50 con 0x564e67d8cc30 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d21c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46284/0 (socket says 172.21.15.33:46284) 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2787431787 learned_addr learned my addr 172.21.15.33:0/2787431787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e67cabf50 con 0x564e67d8c240 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e67d94ed0 con 0x564e67d9bed0 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d720b740 10 client.0 _ll_register_callbacks cb 0x564e67ce7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d9e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 msgr2=0x564e67d9e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d9e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2787431787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 msgr2=0x564e67d96960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d19c4640 1 -- 172.21.15.33:0/2787431787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e67a043f0 con 0x564e67d8cc30 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.836+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.837+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d9e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:37:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.837+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc012000 tx=0x7fb9bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bdfbb3acb95b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.837+0000 7fb9b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.837+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/2787431787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc029060 con 0x564e67d8cc30 2024-04-24T23:37:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.837+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/2787431787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9bc005250 con 0x564e67d8cc30 2024-04-24T23:37:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.837+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9bc010cb0 con 0x564e67d8cc30 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/2787431787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc010e60 con 0x564e67d8cc30 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9d720b740 10 client.20526 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e679c40f0 con 0x564e67d8cc30 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/2787431787 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc010cb0 con 0x564e67d8cc30 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/2787431787 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb9bc0046c0 con 0x564e67d8cc30 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9d720b740 10 client.20526 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9d720b740 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.838+0000 7fb9d720b740 20 client.20526 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.839+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67da2800 0x564e67da4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.839+0000 7fb9d720b740 4 client.20526 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.839+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e67cfdbf0 con 0x564e67da2800 2024-04-24T23:37:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.840+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67da2800 0x564e67da4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.841+0000 7fb9b5ffb640 10 client.20526 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.841+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67da2800 0x564e67da4bf0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c130f2b6e45ce2e8 server_cookie=69e13979db85c01e in_seq=0 out_seq=0 2024-04-24T23:37:04.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.845+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/2787431787 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e67cfdbf0 con 0x564e67da2800 2024-04-24T23:37:04.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.845+0000 7fb9b5ffb640 10 client.20526 handle_command_reply: tid=0 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.845+0000 7fb9d720b740 1 client.20526 shutdown 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.845+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 msgr2=0x564e67d96ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.845+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc012000 tx=0x7fb9bc00c940 comp rx=0 tx=0).stop 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 shutdown_connections 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67da2800 0x564e67da4bf0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d9e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2787431787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 >> 172.21.15.33:0/2787431787 conn(0x564e67d878b0 msgr2=0x564e67d86af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9b5ffb640 0 client.20526 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 shutdown_connections 2024-04-24T23:37:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 1 -- 172.21.15.33:0/2787431787 wait complete. 2024-04-24T23:37:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.846+0000 7fb9d720b740 20 client.20526 trim_cache size 0 max 16384 2024-04-24T23:37:04.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.881+0000 7fb9d720b740 1 Processor -- start 2024-04-24T23:37:04.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.881+0000 7fb9d720b740 1 -- start start 2024-04-24T23:37:04.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d21c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d21c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46296/0 (socket says 172.21.15.33:46296) 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2822141256 learned_addr learned my addr 172.21.15.33:0/2822141256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d8e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b00124c0 con 0x564e67d8cc30 2024-04-24T23:37:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000fd60 con 0x564e67d8c240 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.882+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000dbb0 con 0x564e67d9bed0 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d8e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2822141256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 msgr2=0x564e67d8e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d8e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2822141256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 msgr2=0x564e67d96ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d21c5640 1 -- 172.21.15.33:0/2822141256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e67d91570 con 0x564e67d8c240 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d29c6640 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d8e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80039a0 tx=0x7fb9c8004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a35b1440d48fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9b7fff640 1 -- 172.21.15.33:0/2822141256 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c803f040 con 0x564e67d8c240 2024-04-24T23:37:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9b7fff640 1 -- 172.21.15.33:0/2822141256 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c80c5ef0 con 0x564e67d8c240 2024-04-24T23:37:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.883+0000 7fb9b7fff640 1 -- 172.21.15.33:0/2822141256 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80dd0d0 con 0x564e67d8c240 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.884+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 msgr2=0x564e67d96960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.884+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80039a0 tx=0x7fb9c8004570 comp rx=0 tx=0).stop 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 shutdown_connections 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d9bed0 0x564e67d8e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d8cc30 0x564e67d96ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 --2- 172.21.15.33:0/2822141256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8c240 0x564e67d96960 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 >> 172.21.15.33:0/2822141256 conn(0x564e67d878b0 msgr2=0x564e67d95c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 shutdown_connections 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 -- 172.21.15.33:0/2822141256 wait complete. 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 -- start start 2024-04-24T23:37:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.885+0000 7fb9d720b740 1 Processor -- start 2024-04-24T23:37:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8c240 0x564e67db3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8cc30 0x564e67db3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d9bed0 0x564e67db4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000e220 con 0x564e67d9bed0 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000f690 con 0x564e67d8cc30 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000e8f0 con 0x564e67d8c240 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d720b740 10 client.0 _ll_register_callbacks cb 0x7fb9bc014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d21c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8c240 0x564e67db3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d21c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8c240 0x564e67db3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42596/0 (socket says 172.21.15.33:42596) 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d21c5640 1 -- 172.21.15.33:0/3620004428 learned_addr learned my addr 172.21.15.33:0/3620004428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:04.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.886+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8cc30 0x564e67db3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.889+0000 7fb9d19c4640 1 -- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8c240 msgr2=0x564e67db3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.889+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8c240 0x564e67db3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.889+0000 7fb9d19c4640 1 -- 172.21.15.33:0/3620004428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d9bed0 msgr2=0x564e67db4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.889+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d9bed0 0x564e67db4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.889+0000 7fb9d19c4640 1 -- 172.21.15.33:0/3620004428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e67db51d0 con 0x564e67d8cc30 2024-04-24T23:37:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.892+0000 7fb9d19c4640 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8cc30 0x564e67db3bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0058c0 tx=0x7fb9bc00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c50d5246c71a586 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.892+0000 7fb9b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:37:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.893+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/3620004428 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc029050 con 0x564e67d8cc30 2024-04-24T23:37:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.893+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/3620004428 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9bc0195d0 con 0x564e67d8cc30 2024-04-24T23:37:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.893+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c80c5ef0 con 0x564e67d8cc30 2024-04-24T23:37:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.893+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/3620004428 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc033080 con 0x564e67d8cc30 2024-04-24T23:37:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 10 client.29984 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e67e087f0 con 0x564e67d8cc30 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/3620004428 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc00ec20 con 0x564e67d8cc30 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/3620004428 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb9bc010880 con 0x564e67d8cc30 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 10 client.29984 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 10 client.29984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 20 client.29984 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67e09130 0x564e67e0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 4 client.29984 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:37:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.894+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb9c80c5ef0 con 0x564e67e09130 2024-04-24T23:37:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.895+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67e09130 0x564e67e0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:04.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.895+0000 7fb9b5ffb640 10 client.29984 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.895+0000 7fb9d21c5640 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67e09130 0x564e67e0b550 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4f49e1e1bbb8ac server_cookie=6a89ba9dc6503f8d in_seq=0 out_seq=0 2024-04-24T23:37:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9b5ffb640 1 -- 172.21.15.33:0/3620004428 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fb9c80c5ef0 con 0x564e67e09130 2024-04-24T23:37:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9b5ffb640 10 client.29984 handle_command_reply: tid=0 2024-04-24T23:37:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9d720b740 1 client.29984 shutdown 2024-04-24T23:37:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8cc30 msgr2=0x564e67db3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9d720b740 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8cc30 0x564e67db3bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0058c0 tx=0x7fb9bc00cad0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 shutdown_connections 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.896+0000 7fb9d720b740 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x564e67e09130 0x564e67e0b550 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9d720b740 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564e67d9bed0 0x564e67db4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9d720b740 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e67d8cc30 0x564e67db3bd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9b5ffb640 0 client.29984 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9d720b740 1 --2- 172.21.15.33:0/3620004428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e67d8c240 0x564e67db3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 >> 172.21.15.33:0/3620004428 conn(0x564e67d878b0 msgr2=0x564e67d96900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 shutdown_connections 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.897+0000 7fb9d720b740 1 -- 172.21.15.33:0/3620004428 wait complete. 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.898+0000 7fb9d720b740 20 client.29984 trim_cache size 0 max 16384 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:37:04.899 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9b80a2b30 msgr2=0x7fb9b80a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9b80a2b30 0x7fb9b80a4ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x564e67d8d490 tx=0x7fb9c402add0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc100680 msgr2=0x7fb9cc19e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc100680 0x7fb9cc19e060 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800af00 tx=0x7fb9c800cc90 comp rx=0 tx=0).stop 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 shutdown_connections 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb9b80a2b30 0x7fb9b80a4ff0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9cc102230 0x7fb9cc1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9cc101870 0x7fb9cc19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.900+0000 7fb9d4429640 1 --2- 172.21.15.33:0/2122589503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc100680 0x7fb9cc19e060 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.901+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 >> 172.21.15.33:0/2122589503 conn(0x7fb9cc0fbbf0 msgr2=0x7fb9cc10dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.901+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 shutdown_connections 2024-04-24T23:37:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:04.901+0000 7fb9d4429640 1 -- 172.21.15.33:0/2122589503 wait complete. 2024-04-24T23:37:04.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 276542 21.80 MB/sec execute 549 sec latency 109.233 ms 2024-04-24T23:37:04.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:37:04.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:37:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.072+0000 7f3a3e942640 1 -- 172.21.15.33:0/697001589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 msgr2=0x7f3a3810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.072+0000 7f3a3e942640 1 --2- 172.21.15.33:0/697001589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a3810df30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009900 tx=0x7f3a2802a440 comp rx=0 tx=0).stop 2024-04-24T23:37:05.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.073+0000 7f3a3e942640 1 -- 172.21.15.33:0/697001589 shutdown_connections 2024-04-24T23:37:05.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.073+0000 7f3a3e942640 1 --2- 172.21.15.33:0/697001589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a3810df30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.073+0000 7f3a3e942640 1 --2- 172.21.15.33:0/697001589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a3810b0b0 0x7f3a3810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.073+0000 7f3a3e942640 1 --2- 172.21.15.33:0/697001589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 0x7f3a3810ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.073+0000 7f3a3e942640 1 -- 172.21.15.33:0/697001589 >> 172.21.15.33:0/697001589 conn(0x7f3a3806e5e0 msgr2=0x7f3a38070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.077+0000 7f3a3e942640 1 -- 172.21.15.33:0/697001589 shutdown_connections 2024-04-24T23:37:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.081+0000 7f3a3e942640 1 -- 172.21.15.33:0/697001589 wait complete. 2024-04-24T23:37:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.081+0000 7f3a3e942640 1 Processor -- start 2024-04-24T23:37:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.081+0000 7f3a3e942640 1 -- start start 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3e942640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 0x7f3a381a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3e942640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a3810b0b0 0x7f3a381a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3e942640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a381ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3e942640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3811c290 con 0x7f3a3810b0b0 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3e942640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3811c110 con 0x7f3a38108740 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3e942640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3811c410 con 0x7f3a3810da80 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3cedf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a381ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3cedf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a381ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42622/0 (socket says 172.21.15.33:42622) 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.082+0000 7f3a3cedf640 1 -- 172.21.15.33:0/3014908011 learned_addr learned my addr 172.21.15.33:0/3014908011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a37fff640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 0x7f3a381a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a377fe640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a3810b0b0 0x7f3a381a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a3cedf640 1 -- 172.21.15.33:0/3014908011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 msgr2=0x7f3a381a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a3cedf640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 0x7f3a381a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a3cedf640 1 -- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a3810b0b0 msgr2=0x7f3a381a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a3cedf640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a3810b0b0 0x7f3a381a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a3cedf640 1 -- 172.21.15.33:0/3014908011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a381ae500 con 0x7f3a3810da80 2024-04-24T23:37:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a37fff640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 0x7f3a381a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.083+0000 7f3a3cedf640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a381ade90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009850 tx=0x7f3a28005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.084+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2803f070 con 0x7f3a3810da80 2024-04-24T23:37:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.084+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a381ae790 con 0x7f3a3810da80 2024-04-24T23:37:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.084+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381aec80 con 0x7f3a3810da80 2024-04-24T23:37:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.084+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a28031b20 con 0x7f3a3810da80 2024-04-24T23:37:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.084+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28035030 con 0x7f3a3810da80 2024-04-24T23:37:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.086+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a240052c0 con 0x7f3a3810da80 2024-04-24T23:37:05.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.087+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a28042070 con 0x7f3a3810da80 2024-04-24T23:37:05.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.088+0000 7f3a357fa640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2d90 0x7f3a200a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.088+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a280efd00 con 0x7f3a3810da80 2024-04-24T23:37:05.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.089+0000 7f3a37fff640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2d90 0x7f3a200a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.090+0000 7f3a37fff640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2d90 0x7f3a200a5250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0058c0 tx=0x7f3a2c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.098+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a280bc870 con 0x7f3a3810da80 2024-04-24T23:37:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.383+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a240055b0 con 0x7f3a3810da80 2024-04-24T23:37:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.384+0000 7f3a357fa640 1 -- 172.21.15.33:0/3014908011 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11861 (secure 0 0 0) 0x7f3a280bc170 con 0x7f3a3810da80 2024-04-24T23:37:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:37:05.385 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:35:37.265197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:37:05.385 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2d90 msgr2=0x7f3a200a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2d90 0x7f3a200a5250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0058c0 tx=0x7f3a2c005850 comp rx=0 tx=0).stop 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 msgr2=0x7f3a381ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a381ade90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009850 tx=0x7f3a28005230 comp rx=0 tx=0).stop 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 shutdown_connections 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f3a200a2d90 0x7f3a200a5250 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3810da80 0x7f3a381ade90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a3810b0b0 0x7f3a381a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 --2- 172.21.15.33:0/3014908011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38108740 0x7f3a381a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.386+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 >> 172.21.15.33:0/3014908011 conn(0x7f3a3806e5e0 msgr2=0x7f3a3810c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.387+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 shutdown_connections 2024-04-24T23:37:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.387+0000 7f3a3e942640 1 -- 172.21.15.33:0/3014908011 wait complete. 2024-04-24T23:37:05.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:37:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 -- 172.21.15.33:0/2752267705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74102980 msgr2=0x7fba74102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 --2- 172.21.15.33:0/2752267705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74102980 0x7fba74102e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fba68009860 tx=0x7fba6802a340 comp rx=0 tx=0).stop 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 -- 172.21.15.33:0/2752267705 shutdown_connections 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 --2- 172.21.15.33:0/2752267705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba74103340 0x7fba74109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 --2- 172.21.15.33:0/2752267705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74102980 0x7fba74102e00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 --2- 172.21.15.33:0/2752267705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74101790 0x7fba74101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 -- 172.21.15.33:0/2752267705 >> 172.21.15.33:0/2752267705 conn(0x7fba740fcd20 msgr2=0x7fba740ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 -- 172.21.15.33:0/2752267705 shutdown_connections 2024-04-24T23:37:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.517+0000 7fba78b4f640 1 -- 172.21.15.33:0/2752267705 wait complete. 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 Processor -- start 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 -- start start 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74101790 0x7fba74197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 0x7fba74198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba74103340 0x7fba7419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410d410 con 0x7fba74102980 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410d290 con 0x7fba74103340 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.518+0000 7fba78b4f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410d590 con 0x7fba74101790 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 0x7fba74198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 0x7fba74198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40468/0 (socket says 172.21.15.33:40468) 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba7259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74101790 0x7fba74197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 -- 172.21.15.33:0/1533888324 learned_addr learned my addr 172.21.15.33:0/1533888324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba72d9d640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba74103340 0x7fba7419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 -- 172.21.15.33:0/1533888324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74101790 msgr2=0x7fba74197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74101790 0x7fba74197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 -- 172.21.15.33:0/1533888324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba74103340 msgr2=0x7fba7419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba74103340 0x7fba7419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 -- 172.21.15.33:0/1533888324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7419f6e0 con 0x7fba74102980 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba7259c640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74101790 0x7fba74197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba71d9b640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 0x7fba74198000 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fba68005940 tx=0x7fba68033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.519+0000 7fba637fe640 1 -- 172.21.15.33:0/1533888324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6803f020 con 0x7fba74102980 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.520+0000 7fba637fe640 1 -- 172.21.15.33:0/1533888324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba68030850 con 0x7fba74102980 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.520+0000 7fba637fe640 1 -- 172.21.15.33:0/1533888324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68037040 con 0x7fba74102980 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.520+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7419f970 con 0x7fba74102980 2024-04-24T23:37:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.520+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7419fe10 con 0x7fba74102980 2024-04-24T23:37:05.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.522+0000 7fba7b859740 1 Processor -- start 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 -- start start 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 0x555b00c21650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c26d90 0x555b00c29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b00b85670 con 0x555b00c21c20 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b00b2d970 con 0x555b00c21230 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7b859740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b00b81630 con 0x555b00c26d90 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba637fe640 1 -- 172.21.15.33:0/1533888324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fba68042020 con 0x7fba74102980 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba72d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 0x555b00c21650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba72d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 0x555b00c21650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:46358/0 (socket says 172.21.15.33:46358) 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba72d9d640 1 -- 172.21.15.33:0/289460497 learned_addr learned my addr 172.21.15.33:0/289460497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7259c640 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba71d9b640 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c26d90 0x555b00c29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7259c640 1 -- 172.21.15.33:0/289460497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c26d90 msgr2=0x555b00c29220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7259c640 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c26d90 0x555b00c29220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7259c640 1 -- 172.21.15.33:0/289460497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 msgr2=0x555b00c21650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7259c640 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 0x555b00c21650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba7259c640 1 -- 172.21.15.33:0/289460497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b00b51400 con 0x555b00c21c20 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.523+0000 7fba72d9d640 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 0x555b00c21650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba7259c640 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c26720 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fba64005fd0 tx=0x7fba640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafb60d4af3ad027 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba637fe640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba5c0a2b90 0x7fba5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba60ff9640 1 -- 172.21.15.33:0/289460497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64005020 con 0x555b00c21c20 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba60ff9640 1 -- 172.21.15.33:0/289460497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba640047a0 con 0x555b00c21c20 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba60ff9640 1 -- 172.21.15.33:0/289460497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6400f870 con 0x555b00c21c20 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba637fe640 1 -- 172.21.15.33:0/1533888324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fba680ef6c0 con 0x7fba74102980 2024-04-24T23:37:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.524+0000 7fba72d9d640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba5c0a2b90 0x7fba5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 -- 172.21.15.33:0/289460497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 msgr2=0x555b00c26720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba72d9d640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba5c0a2b90 0x7fba5c0a5050 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x555b00c22480 tx=0x7fba6c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c26720 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fba64005fd0 tx=0x7fba640118d0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 -- 172.21.15.33:0/289460497 shutdown_connections 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c26d90 0x555b00c29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c26720 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 --2- 172.21.15.33:0/289460497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c21230 0x555b00c21650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 -- 172.21.15.33:0/289460497 >> 172.21.15.33:0/289460497 conn(0x555b00c1c8a0 msgr2=0x555b00c1cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 -- 172.21.15.33:0/289460497 shutdown_connections 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 -- 172.21.15.33:0/289460497 wait complete. 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 -- start start 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.525+0000 7fba7b859740 1 Processor -- start 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b00c29d40 con 0x555b00c21c20 2024-04-24T23:37:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b00b41050 con 0x555b00c26d90 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b00c29ec0 con 0x555b00c21230 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 10 client.0 _ll_register_callbacks cb 0x555b00898be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c2b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:42654/0 (socket says 172.21.15.33:42654) 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 -- 172.21.15.33:0/2426742583 learned_addr learned my addr 172.21.15.33:0/2426742583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba72d9d640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba71d9b640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 -- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 msgr2=0x555b00c328e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c328e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 -- 172.21.15.33:0/2426742583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 msgr2=0x555b00c2b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 -- 172.21.15.33:0/2426742583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b00b12dc0 con 0x555b00c21230 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7259c640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c2b360 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba64005890 tx=0x7fba640052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a845b16c4e4c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba59ffb640 1 -- 172.21.15.33:0/2426742583 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64028070 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba640047a0 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.526+0000 7fba59ffb640 1 -- 172.21.15.33:0/2426742583 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fba640054a0 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba59ffb640 1 -- 172.21.15.33:0/2426742583 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba64033050 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 10 client.29938 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b008993f0 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba59ffb640 1 -- 172.21.15.33:0/2426742583 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba640047a0 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba59ffb640 1 -- 172.21.15.33:0/2426742583 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fba64024840 con 0x555b00c21230 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 10 client.29938 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 10 client.29938 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 20 client.29938 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00c36f60 0x555b00c393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 4 client.29938 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b00b92bb0 con 0x555b00c36f60 2024-04-24T23:37:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.527+0000 7fba71d9b640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00c36f60 0x555b00c393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.528+0000 7fba71d9b640 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00c36f60 0x555b00c393e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=846abb28e4a433d server_cookie=3b01f76010286193 in_seq=0 out_seq=0 2024-04-24T23:37:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.528+0000 7fba59ffb640 10 client.29938 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba59ffb640 1 -- 172.21.15.33:0/2426742583 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b00b92bb0 con 0x555b00c36f60 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba59ffb640 10 client.29938 handle_command_reply: tid=0 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 client.29938 shutdown 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 msgr2=0x555b00c2b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c2b360 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba64005890 tx=0x7fba640052b0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 shutdown_connections 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00c36f60 0x555b00c393e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 --2- 172.21.15.33:0/2426742583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c2b360 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba59ffb640 0 client.29938 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 >> 172.21.15.33:0/2426742583 conn(0x555b00c1c8a0 msgr2=0x555b00c1bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.532+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 shutdown_connections 2024-04-24T23:37:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.533+0000 7fba7b859740 1 -- 172.21.15.33:0/2426742583 wait complete. 2024-04-24T23:37:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.533+0000 7fba7b859740 20 client.29938 trim_cache size 0 max 16384 2024-04-24T23:37:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 Processor -- start 2024-04-24T23:37:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 -- start start 2024-04-24T23:37:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c1b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba741acd00 con 0x555b00c21c20 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fba741aa5a0 con 0x555b00c26d90 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7b859740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba741a83f0 con 0x555b00c21230 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba71d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba71d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40510/0 (socket says 172.21.15.33:40510) 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba71d9b640 1 -- 172.21.15.33:0/573978553 learned_addr learned my addr 172.21.15.33:0/573978553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba7259c640 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c1b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.546+0000 7fba72d9d640 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba71d9b640 1 -- 172.21.15.33:0/573978553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 msgr2=0x555b00c1b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba71d9b640 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c1b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba71d9b640 1 -- 172.21.15.33:0/573978553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 msgr2=0x555b00c2aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba71d9b640 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c2aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba71d9b640 1 -- 172.21.15.33:0/573978553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b009637d0 con 0x555b00c21c20 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba72d9d640 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.547+0000 7fba71d9b640 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2a980 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fba68009980 tx=0x7fba6802ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816e78d871c7c6be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.549+0000 7fba5bfff640 1 -- 172.21.15.33:0/573978553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680bb6d0 con 0x555b00c21c20 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.549+0000 7fba5bfff640 1 -- 172.21.15.33:0/573978553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba68030a00 con 0x555b00c21c20 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.549+0000 7fba5bfff640 1 -- 172.21.15.33:0/573978553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680bb4f0 con 0x555b00c21c20 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 -- 172.21.15.33:0/573978553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 msgr2=0x555b00c2a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2a980 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fba68009980 tx=0x7fba6802ab40 comp rx=0 tx=0).stop 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 -- 172.21.15.33:0/573978553 shutdown_connections 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c2a980 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 --2- 172.21.15.33:0/573978553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c1b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 -- 172.21.15.33:0/573978553 >> 172.21.15.33:0/573978553 conn(0x555b00c1c8a0 msgr2=0x555b00c20010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 -- 172.21.15.33:0/573978553 shutdown_connections 2024-04-24T23:37:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 -- 172.21.15.33:0/573978553 wait complete. 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 -- start start 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.550+0000 7fba7b859740 1 Processor -- start 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c98b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c9fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba741a8a60 con 0x555b00c21c20 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fba741a9ed0 con 0x555b00c26d90 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba741a9130 con 0x555b00c21230 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 10 client.0 _ll_register_callbacks cb 0x7fba6c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c98b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba72d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c9fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c98b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40516/0 (socket says 172.21.15.33:40516) 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 -- 172.21.15.33:0/192565434 learned_addr learned my addr 172.21.15.33:0/192565434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 -- 172.21.15.33:0/192565434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 msgr2=0x555b00c98610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c98610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 -- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 msgr2=0x555b00c9fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c9fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 -- 172.21.15.33:0/192565434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b00ca0180 con 0x555b00c21c20 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba72d9d640 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c9fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba71d9b640 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c98b50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fba680ece80 tx=0x7fba680bb570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b56f4cf46a4ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba59ffb640 1 -- 172.21.15.33:0/192565434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680bff40 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba59ffb640 1 -- 172.21.15.33:0/192565434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fba680b8200 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.551+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba68030a00 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba59ffb640 1 -- 172.21.15.33:0/192565434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba680b9ca0 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 10 client.20550 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b00ca1480 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba59ffb640 1 -- 172.21.15.33:0/192565434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba68030a00 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba59ffb640 1 -- 172.21.15.33:0/192565434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fba680ee070 con 0x555b00c21c20 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 10 client.20550 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 10 client.20550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 20 client.20550 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00ca1e30 0x555b00ca4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 4 client.20550 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b00b8cf00 con 0x555b00ca1e30 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.552+0000 7fba7259c640 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00ca1e30 0x555b00ca4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.553+0000 7fba7259c640 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00ca1e30 0x555b00ca4250 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=801ca0002e288ef4 server_cookie=b9cd033a9decebd5 in_seq=0 out_seq=0 2024-04-24T23:37:05.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.553+0000 7fba59ffb640 10 client.20550 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba59ffb640 1 -- 172.21.15.33:0/192565434 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b00b8cf00 con 0x555b00ca1e30 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba59ffb640 10 client.20550 handle_command_reply: tid=0 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 client.20550 shutdown 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 msgr2=0x555b00c98b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c98b50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fba680ece80 tx=0x7fba680bb570 comp rx=0 tx=0).stop 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 shutdown_connections 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x555b00ca1e30 0x555b00ca4250 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b00c26d90 0x555b00c9fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b00c21c20 0x555b00c98b50 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 --2- 172.21.15.33:0/192565434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b00c21230 0x555b00c98610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 >> 172.21.15.33:0/192565434 conn(0x555b00c1c8a0 msgr2=0x555b00c224b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba59ffb640 0 client.20550 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 shutdown_connections 2024-04-24T23:37:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 1 -- 172.21.15.33:0/192565434 wait complete. 2024-04-24T23:37:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.556+0000 7fba7b859740 20 client.20550 trim_cache size 0 max 16384 2024-04-24T23:37:05.561 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba5c0a2b90 msgr2=0x7fba5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba5c0a2b90 0x7fba5c0a5050 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x555b00c22480 tx=0x7fba6c009500 comp rx=0 tx=0).stop 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 msgr2=0x7fba74198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 0x7fba74198000 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fba68005940 tx=0x7fba68033d90 comp rx=0 tx=0).stop 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 shutdown_connections 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fba5c0a2b90 0x7fba5c0a5050 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba74103340 0x7fba7419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba74102980 0x7fba74198000 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 --2- 172.21.15.33:0/1533888324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba74101790 0x7fba74197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 >> 172.21.15.33:0/1533888324 conn(0x7fba740fcd20 msgr2=0x7fba74105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 shutdown_connections 2024-04-24T23:37:05.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.557+0000 7fba78b4f640 1 -- 172.21.15.33:0/1533888324 wait complete. 2024-04-24T23:37:05.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:37:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 -- 172.21.15.33:0/3514966310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 msgr2=0x7f4198102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 --2- 172.21.15.33:0/3514966310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198102e10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f418c009900 tx=0x7f418c02a410 comp rx=0 tx=0).stop 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 -- 172.21.15.33:0/3514966310 shutdown_connections 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 --2- 172.21.15.33:0/3514966310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4198103350 0x7f4198109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 --2- 172.21.15.33:0/3514966310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198102e10 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 --2- 172.21.15.33:0/3514966310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 -- 172.21.15.33:0/3514966310 >> 172.21.15.33:0/3514966310 conn(0x7f41980fcd10 msgr2=0x7f41980ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 -- 172.21.15.33:0/3514966310 shutdown_connections 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.682+0000 7f419dc38640 1 -- 172.21.15.33:0/3514966310 wait complete. 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 Processor -- start 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 -- start start 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4198103350 0x7f419819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f419810d400 con 0x7f4198102990 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f419810d280 con 0x7f41981017a0 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f419dc38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f419810d580 con 0x7f4198103350 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f41977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53916/0 (socket says 172.21.15.33:53916) 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f41977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48110/0 (socket says 172.21.15.33:48110) 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f4196ffd640 1 -- 172.21.15.33:0/2469210539 learned_addr learned my addr 172.21.15.33:0/2469210539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.683+0000 7f4197fff640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4198103350 0x7f419819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4196ffd640 1 -- 172.21.15.33:0/2469210539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4198103350 msgr2=0x7f419819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4196ffd640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4198103350 0x7f419819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4196ffd640 1 -- 172.21.15.33:0/2469210539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 msgr2=0x7f4198197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4196ffd640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4196ffd640 1 -- 172.21.15.33:0/2469210539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419819f6d0 con 0x7f4198102990 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f41977fe640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4196ffd640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198197ff0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f418c009230 tx=0x7f418c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4194ff9640 1 -- 172.21.15.33:0/2469210539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c03f030 con 0x7f4198102990 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4194ff9640 1 -- 172.21.15.33:0/2469210539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418c003ad0 con 0x7f4198102990 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419819f960 con 0x7f4198102990 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f4194ff9640 1 -- 172.21.15.33:0/2469210539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c035070 con 0x7f4198102990 2024-04-24T23:37:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.684+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419819fe50 con 0x7f4198102990 2024-04-24T23:37:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.686+0000 7f4194ff9640 1 -- 172.21.15.33:0/2469210539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f418c042030 con 0x7f4198102990 2024-04-24T23:37:05.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 Processor -- start 2024-04-24T23:37:05.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 -- start start 2024-04-24T23:37:05.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f4194ff9640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41800a2b90 0x7f41800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8682650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8687720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d868a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f4197fff640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41800a2b90 0x7f41800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a9d85e6670 con 0x55a9d8687d90 2024-04-24T23:37:05.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a9d858e970 con 0x55a9d8682230 2024-04-24T23:37:05.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41a0a29740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a9d85e2630 con 0x55a9d8682c20 2024-04-24T23:37:05.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f4194ff9640 1 -- 172.21.15.33:0/2469210539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f418c0ef9e0 con 0x7f4198102990 2024-04-24T23:37:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8682650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8682650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48124/0 (socket says 172.21.15.33:48124) 2024-04-24T23:37:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 -- 172.21.15.33:0/1208433396 learned_addr learned my addr 172.21.15.33:0/1208433396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f4197fff640 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d868a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f4196ffd640 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8687720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 -- 172.21.15.33:0/1208433396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 msgr2=0x55a9d8687720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8687720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 -- 172.21.15.33:0/1208433396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 msgr2=0x55a9d868a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d868a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f41977fe640 1 -- 172.21.15.33:0/1208433396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d85b2400 con 0x55a9d8682230 2024-04-24T23:37:05.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.687+0000 7f4196ffd640 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8687720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:05.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.688+0000 7f4197fff640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41800a2b90 0x7f41800a5050 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4190009370 tx=0x7f4190009e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.688+0000 7f41977fe640 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8682650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f41880114f0 tx=0x7f41880119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d276fecbbacb7358 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.688+0000 7f417dffb640 1 -- 172.21.15.33:0/1208433396 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418801e070 con 0x55a9d8682230 2024-04-24T23:37:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.688+0000 7f417dffb640 1 -- 172.21.15.33:0/1208433396 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418800fa20 con 0x55a9d8682230 2024-04-24T23:37:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.688+0000 7f417dffb640 1 -- 172.21.15.33:0/1208433396 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188019ad0 con 0x55a9d8682230 2024-04-24T23:37:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- 172.21.15.33:0/1208433396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 msgr2=0x55a9d8682650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8682650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f41880114f0 tx=0x7f41880119b0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- 172.21.15.33:0/1208433396 shutdown_connections 2024-04-24T23:37:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d868a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8687720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1208433396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8682650 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- 172.21.15.33:0/1208433396 >> 172.21.15.33:0/1208433396 conn(0x55a9d867d8a0 msgr2=0x55a9d867dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- 172.21.15.33:0/1208433396 shutdown_connections 2024-04-24T23:37:05.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- 172.21.15.33:0/1208433396 wait complete. 2024-04-24T23:37:05.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- start start 2024-04-24T23:37:05.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 Processor -- start 2024-04-24T23:37:05.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8690710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8690c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d8697c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.689+0000 7f41a0a29740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a9d868ad40 con 0x55a9d8687d90 2024-04-24T23:37:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41a0a29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a9d85a2050 con 0x55a9d8682230 2024-04-24T23:37:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41a0a29740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a9d868aec0 con 0x55a9d8682c20 2024-04-24T23:37:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41a0a29740 10 client.0 _ll_register_callbacks cb 0x55a9d82f9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8690710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f4197fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d8697c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8690c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8690710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48132/0 (socket says 172.21.15.33:48132) 2024-04-24T23:37:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 -- 172.21.15.33:0/2104029233 learned_addr learned my addr 172.21.15.33:0/2104029233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 -- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 msgr2=0x55a9d8690c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8690c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 -- 172.21.15.33:0/2104029233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 msgr2=0x55a9d8697c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d8697c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 -- 172.21.15.33:0/2104029233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d8573dc0 con 0x55a9d8682230 2024-04-24T23:37:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f4196ffd640 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8690c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41977fe640 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8690710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4188019530 tx=0x7f418800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4f05a4f6bb3e255 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f4176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:37:05.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f4176ffd640 1 -- 172.21.15.33:0/2104029233 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418801e040 con 0x55a9d8682230 2024-04-24T23:37:05.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418800fa20 con 0x55a9d8682230 2024-04-24T23:37:05.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.690+0000 7f4176ffd640 1 -- 172.21.15.33:0/2104029233 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4188005070 con 0x55a9d8682230 2024-04-24T23:37:05.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f4176ffd640 1 -- 172.21.15.33:0/2104029233 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4188030050 con 0x55a9d8682230 2024-04-24T23:37:05.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 10 client.30020 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:05.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a9d82fa3f0 con 0x55a9d8682230 2024-04-24T23:37:05.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f4176ffd640 1 -- 172.21.15.33:0/2104029233 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418800fa20 con 0x55a9d8682230 2024-04-24T23:37:05.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f4176ffd640 1 -- 172.21.15.33:0/2104029233 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f41880035f0 con 0x55a9d8682230 2024-04-24T23:37:05.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 10 client.30020 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:05.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 10 client.30020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:37:05.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 20 client.30020 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:05.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d869c290 0x55a9d869e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 4 client.30020 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:05.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d85f3bb0 con 0x55a9d869c290 2024-04-24T23:37:05.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.691+0000 7f4196ffd640 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d869c290 0x55a9d869e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.692+0000 7f4176ffd640 10 client.30020 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:05.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.692+0000 7f4196ffd640 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d869c290 0x55a9d869e700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9f200ae16500838 server_cookie=7505df15d4dc7ea4 in_seq=0 out_seq=0 2024-04-24T23:37:05.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.692+0000 7f4176ffd640 1 -- 172.21.15.33:0/2104029233 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9d85f3bb0 con 0x55a9d869c290 2024-04-24T23:37:05.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.692+0000 7f4176ffd640 10 client.30020 handle_command_reply: tid=0 2024-04-24T23:37:05.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.692+0000 7f41a0a29740 1 client.30020 shutdown 2024-04-24T23:37:05.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 msgr2=0x55a9d8690710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8690710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4188019530 tx=0x7f418800f7f0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 shutdown_connections 2024-04-24T23:37:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d869c290 0x55a9d869e700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8687d90 0x55a9d8697c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8690c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f4176ffd640 0 client.30020 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 --2- 172.21.15.33:0/2104029233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682230 0x55a9d8690710 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 >> 172.21.15.33:0/2104029233 conn(0x55a9d867d8a0 msgr2=0x55a9d867cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 shutdown_connections 2024-04-24T23:37:05.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 1 -- 172.21.15.33:0/2104029233 wait complete. 2024-04-24T23:37:05.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.693+0000 7f41a0a29740 20 client.30020 trim_cache size 0 max 16384 2024-04-24T23:37:05.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:05 smithi033 ceph-mon[29201]: pgmap v1103: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2024-04-24T23:37:05.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:05 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3014908011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:37:05.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 Processor -- start 2024-04-24T23:37:05.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 -- start start 2024-04-24T23:37:05.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d867cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682c20 0x55a9d8697c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8687d90 0x55a9d8683480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41981aaa70 con 0x55a9d8682230 2024-04-24T23:37:05.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41981a8310 con 0x55a9d8682c20 2024-04-24T23:37:05.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41a0a29740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41981a6160 con 0x55a9d8687d90 2024-04-24T23:37:05.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d867cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d867cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53942/0 (socket says 172.21.15.33:53942) 2024-04-24T23:37:05.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f41977fe640 1 -- 172.21.15.33:0/4145199913 learned_addr learned my addr 172.21.15.33:0/4145199913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f4197fff640 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8687d90 0x55a9d8683480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.708+0000 7f4196ffd640 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682c20 0x55a9d8697c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f41977fe640 1 -- 172.21.15.33:0/4145199913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8687d90 msgr2=0x55a9d8683480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f41977fe640 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8687d90 0x55a9d8683480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f41977fe640 1 -- 172.21.15.33:0/4145199913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682c20 msgr2=0x55a9d8697c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f41977fe640 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682c20 0x55a9d8697c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f41977fe640 1 -- 172.21.15.33:0/4145199913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d83c47d0 con 0x55a9d8682230 2024-04-24T23:37:05.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f41977fe640 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d867cb40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4188003c00 tx=0x7f4188005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247c7569bc0f95b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f417d7fa640 1 -- 172.21.15.33:0/4145199913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418801e040 con 0x55a9d8682230 2024-04-24T23:37:05.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f417d7fa640 1 -- 172.21.15.33:0/4145199913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4188016030 con 0x55a9d8682230 2024-04-24T23:37:05.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.709+0000 7f417d7fa640 1 -- 172.21.15.33:0/4145199913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880196f0 con 0x55a9d8682230 2024-04-24T23:37:05.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 -- 172.21.15.33:0/4145199913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 msgr2=0x55a9d867cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d867cb40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4188003c00 tx=0x7f4188005fd0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 -- 172.21.15.33:0/4145199913 shutdown_connections 2024-04-24T23:37:05.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8687d90 0x55a9d8683480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8682c20 0x55a9d8697c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 --2- 172.21.15.33:0/4145199913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d867cb40 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 -- 172.21.15.33:0/4145199913 >> 172.21.15.33:0/4145199913 conn(0x55a9d867d8a0 msgr2=0x55a9d8690a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 -- 172.21.15.33:0/4145199913 shutdown_connections 2024-04-24T23:37:05.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 -- 172.21.15.33:0/4145199913 wait complete. 2024-04-24T23:37:05.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.710+0000 7f41a0a29740 1 -- start start 2024-04-24T23:37:05.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 Processor -- start 2024-04-24T23:37:05.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d8694ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8695000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8687d90 0x55a9d8700830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41981a67d0 con 0x55a9d8682230 2024-04-24T23:37:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41981a7c40 con 0x55a9d8687d90 2024-04-24T23:37:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41981a6ea0 con 0x55a9d8682c20 2024-04-24T23:37:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41a0a29740 10 client.0 _ll_register_callbacks cb 0x7f418801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:05.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8695000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d8694ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8695000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46776/0 (socket says 172.21.15.33:46776) 2024-04-24T23:37:05.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4196ffd640 1 -- 172.21.15.33:0/1723929895 learned_addr learned my addr 172.21.15.33:0/1723929895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4197fff640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8687d90 0x55a9d8700830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 -- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 msgr2=0x55a9d8695000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8695000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 -- 172.21.15.33:0/1723929895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8687d90 msgr2=0x55a9d8700830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8687d90 0x55a9d8700830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 -- 172.21.15.33:0/1723929895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d8700d70 con 0x55a9d8682230 2024-04-24T23:37:05.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4196ffd640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8695000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4197fff640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8687d90 0x55a9d8700830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f41977fe640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d8694ac0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f4188010010 tx=0x7f4188009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb49a128d0db792 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.711+0000 7f4176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:05.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f4176ffd640 1 -- 172.21.15.33:0/1723929895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188038020 con 0x55a9d8682230 2024-04-24T23:37:05.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f4176ffd640 1 -- 172.21.15.33:0/1723929895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41880052e0 con 0x55a9d8682230 2024-04-24T23:37:05.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4188016030 con 0x55a9d8682230 2024-04-24T23:37:05.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f4176ffd640 1 -- 172.21.15.33:0/1723929895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f418801e040 con 0x55a9d8682230 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 10 client.20568 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a9d8701f30 con 0x55a9d8682230 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f4176ffd640 1 -- 172.21.15.33:0/1723929895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4188016030 con 0x55a9d8682230 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f4176ffd640 1 -- 172.21.15.33:0/1723929895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4188019c60 con 0x55a9d8682230 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 10 client.20568 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 10 client.20568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:37:05.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 20 client.20568 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d8702930 0x55a9d8704d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 4 client.20568 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:37:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.712+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9d8687b00 con 0x55a9d8702930 2024-04-24T23:37:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f4196ffd640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d8702930 0x55a9d8704d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f4176ffd640 10 client.20568 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f4196ffd640 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d8702930 0x55a9d8704d50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40b591369dcc263a server_cookie=66ce657fd59b470d in_seq=0 out_seq=0 2024-04-24T23:37:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f4176ffd640 1 -- 172.21.15.33:0/1723929895 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9d8687b00 con 0x55a9d8702930 2024-04-24T23:37:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f4176ffd640 10 client.20568 handle_command_reply: tid=0 2024-04-24T23:37:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f41a0a29740 1 client.20568 shutdown 2024-04-24T23:37:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 msgr2=0x55a9d8694ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d8694ac0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f4188010010 tx=0x7f4188009370 comp rx=0 tx=0).stop 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 shutdown_connections 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.713+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55a9d8702930 0x55a9d8704d50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a9d8687d90 0x55a9d8700830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9d8682c20 0x55a9d8695000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 1 --2- 172.21.15.33:0/1723929895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9d8682230 0x55a9d8694ac0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f4176ffd640 0 client.20568 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:05.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 >> 172.21.15.33:0/1723929895 conn(0x55a9d867d8a0 msgr2=0x55a9d867cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 shutdown_connections 2024-04-24T23:37:05.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 1 -- 172.21.15.33:0/1723929895 wait complete. 2024-04-24T23:37:05.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f41a0a29740 20 client.20568 trim_cache size 0 max 16384 2024-04-24T23:37:05.742 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:37:05.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41800a2b90 msgr2=0x7f41800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41800a2b90 0x7f41800a5050 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4190009370 tx=0x7f4190009e20 comp rx=0 tx=0).stop 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 msgr2=0x7f4198197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198197ff0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f418c009230 tx=0x7f418c031b60 comp rx=0 tx=0).stop 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 shutdown_connections 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f41800a2b90 0x7f41800a5050 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4198103350 0x7f419819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198102990 0x7f4198197ff0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 --2- 172.21.15.33:0/2469210539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41981017a0 0x7f4198197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 >> 172.21.15.33:0/2469210539 conn(0x7f41980fcd10 msgr2=0x7f4198105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 shutdown_connections 2024-04-24T23:37:05.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.714+0000 7f419dc38640 1 -- 172.21.15.33:0/2469210539 wait complete. 2024-04-24T23:37:05.746 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:37:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 -- 172.21.15.33:0/3637679159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228102230 msgr2=0x7f0228108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 --2- 172.21.15.33:0/3637679159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228102230 0x7f0228108ac0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f022002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 -- 172.21.15.33:0/3637679159 shutdown_connections 2024-04-24T23:37:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 --2- 172.21.15.33:0/3637679159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228102230 0x7f0228108ac0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 --2- 172.21.15.33:0/3637679159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0228101870 0x7f0228101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 --2- 172.21.15.33:0/3637679159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 -- 172.21.15.33:0/3637679159 >> 172.21.15.33:0/3637679159 conn(0x7f02280fbbf0 msgr2=0x7f02280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 -- 172.21.15.33:0/3637679159 shutdown_connections 2024-04-24T23:37:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.856+0000 7f022ca41640 1 -- 172.21.15.33:0/3637679159 wait complete. 2024-04-24T23:37:05.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 Processor -- start 2024-04-24T23:37:05.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 -- start start 2024-04-24T23:37:05.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228101870 0x7f0228197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0228102230 0x7f022819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f022810c300 con 0x7f0228101870 2024-04-24T23:37:05.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f022810c180 con 0x7f0228100680 2024-04-24T23:37:05.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.857+0000 7f022ca41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f022810c480 con 0x7f0228102230 2024-04-24T23:37:05.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f022659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228101870 0x7f0228197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f022659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48160/0 (socket says 172.21.15.33:48160) 2024-04-24T23:37:05.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f022659c640 1 -- 172.21.15.33:0/1868558108 learned_addr learned my addr 172.21.15.33:0/1868558108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0226d9d640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0228102230 0x7f022819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 -- 172.21.15.33:0/1868558108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0228102230 msgr2=0x7f022819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0228102230 0x7f022819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 -- 172.21.15.33:0/1868558108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 msgr2=0x7f0228197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 -- 172.21.15.33:0/1868558108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022819f6b0 con 0x7f0228101870 2024-04-24T23:37:05.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f022659c640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f0225d9b640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228101870 0x7f0228197fd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f021c00e9f0 tx=0x7f021c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f02177fe640 1 -- 172.21.15.33:0/1868558108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c00ccc0 con 0x7f0228101870 2024-04-24T23:37:05.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f02177fe640 1 -- 172.21.15.33:0/1868558108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f021c005020 con 0x7f0228101870 2024-04-24T23:37:05.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022819f9a0 con 0x7f0228101870 2024-04-24T23:37:05.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.858+0000 7f02177fe640 1 -- 172.21.15.33:0/1868558108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c0049e0 con 0x7f0228101870 2024-04-24T23:37:05.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.859+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022819fea0 con 0x7f0228101870 2024-04-24T23:37:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.861+0000 7f02177fe640 1 -- 172.21.15.33:0/1868558108 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f021c004be0 con 0x7f0228101870 2024-04-24T23:37:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.861+0000 7f022f820740 1 Processor -- start 2024-04-24T23:37:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.861+0000 7f022f820740 1 -- start start 2024-04-24T23:37:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022f820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022f820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 0x556582fd2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022f820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fd35a0 0x556582fd5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022f820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556582f29670 con 0x556582fceaa0 2024-04-24T23:37:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022f820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556582ed1970 con 0x556582fd35a0 2024-04-24T23:37:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022f820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556582f25630 con 0x556582fce0b0 2024-04-24T23:37:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 0x556582fd2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 0x556582fd2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53984/0 (socket says 172.21.15.33:53984) 2024-04-24T23:37:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 -- 172.21.15.33:0/117668817 learned_addr learned my addr 172.21.15.33:0/117668817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f02177fe640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02100a2b90 0x7f02100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f0226d9d640 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f0225d9b640 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fd35a0 0x556582fd5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f02177fe640 1 -- 172.21.15.33:0/1868558108 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f021c0cf350 con 0x7f0228101870 2024-04-24T23:37:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 -- 172.21.15.33:0/117668817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 msgr2=0x556582fce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f0226d9d640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02100a2b90 0x7f02100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 -- 172.21.15.33:0/117668817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fd35a0 msgr2=0x556582fd5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fd35a0 0x556582fd5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 -- 172.21.15.33:0/117668817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556582ef5400 con 0x556582fceaa0 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.862+0000 7f022659c640 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 0x556582fd2f30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f02180115f0 tx=0x7f0218011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5724832e5fb016ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f0214ff9640 1 -- 172.21.15.33:0/117668817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021800f6f0 con 0x556582fceaa0 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f0214ff9640 1 -- 172.21.15.33:0/117668817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f021800fd70 con 0x556582fceaa0 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f0226d9d640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02100a2b90 0x7f02100a5050 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x556582fc43e0 tx=0x7f022002a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f0214ff9640 1 -- 172.21.15.33:0/117668817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180183b0 con 0x556582fceaa0 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 -- 172.21.15.33:0/117668817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 msgr2=0x556582fd2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 0x556582fd2f30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f02180115f0 tx=0x7f0218011ab0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 -- 172.21.15.33:0/117668817 shutdown_connections 2024-04-24T23:37:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fd35a0 0x556582fd5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fceaa0 0x556582fd2f30 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 --2- 172.21.15.33:0/117668817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.863+0000 7f022f820740 1 -- 172.21.15.33:0/117668817 >> 172.21.15.33:0/117668817 conn(0x556582fc0820 msgr2=0x556582fc0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 -- 172.21.15.33:0/117668817 shutdown_connections 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 -- 172.21.15.33:0/117668817 wait complete. 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 -- start start 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 Processor -- start 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fd7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fd8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fdf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556582fd6550 con 0x556582fd35a0 2024-04-24T23:37:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556582ee5050 con 0x556582fceaa0 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556582fd66d0 con 0x556582fce0b0 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022f820740 10 client.0 _ll_register_callbacks cb 0x556582c3cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f0226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fdf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f0226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fdf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53998/0 (socket says 172.21.15.33:53998) 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f0226d9d640 1 -- 172.21.15.33:0/1772834042 learned_addr learned my addr 172.21.15.33:0/1772834042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f0225d9b640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fd8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.864+0000 7f022659c640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fd7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0226d9d640 1 -- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 msgr2=0x556582fd7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0226d9d640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fd7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0226d9d640 1 -- 172.21.15.33:0/1772834042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 msgr2=0x556582fd8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0226d9d640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fd8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0226d9d640 1 -- 172.21.15.33:0/1772834042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556582eb6dc0 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0225d9b640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fd8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f0226d9d640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fdf1b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f02200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2fe580a64bd988 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f020dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f020dffb640 1 -- 172.21.15.33:0/1772834042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022004e030 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f020dffb640 1 -- 172.21.15.33:0/1772834042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f022002fc70 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f021800fd70 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f020dffb640 1 -- 172.21.15.33:0/1772834042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f022005b050 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f022f820740 10 client.20580 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556582c3d3f0 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.865+0000 7f020dffb640 1 -- 172.21.15.33:0/1772834042 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f021800fd70 con 0x556582fd35a0 2024-04-24T23:37:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f020dffb640 1 -- 172.21.15.33:0/1772834042 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f02200051d0 con 0x556582fd35a0 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022f820740 10 client.20580 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022f820740 10 client.20580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022f820740 20 client.20580 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022f820740 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x556582fe6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022f820740 4 client.20580 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556582f36bb0 con 0x556582fe3bb0 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022659c640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x556582fe6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f022659c640 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x556582fe6010 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2075723e7b696483 server_cookie=5102de5c629ef5ef in_seq=0 out_seq=0 2024-04-24T23:37:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.866+0000 7f020dffb640 10 client.20580 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f020dffb640 1 -- 172.21.15.33:0/1772834042 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556582f36bb0 con 0x556582fe3bb0 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f020dffb640 10 client.20580 handle_command_reply: tid=0 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 client.20580 shutdown 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 msgr2=0x556582fdf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fdf1b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f02200095c0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 shutdown_connections 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x556582fe6010 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fdf1b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f020dffb640 0 client.20580 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fd8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 --2- 172.21.15.33:0/1772834042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fce0b0 0x556582fd7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 >> 172.21.15.33:0/1772834042 conn(0x556582fc0820 msgr2=0x556582fbfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 shutdown_connections 2024-04-24T23:37:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 1 -- 172.21.15.33:0/1772834042 wait complete. 2024-04-24T23:37:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.867+0000 7f022f820740 20 client.20580 trim_cache size 0 max 16384 2024-04-24T23:37:05.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 Processor -- start 2024-04-24T23:37:05.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 -- start start 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fce0b0 0x556582fd7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fceaa0 0x556582fdf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fbf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02281aaa60 con 0x556582fd35a0 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02281a8300 con 0x556582fce0b0 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022f820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02281a6150 con 0x556582fceaa0 2024-04-24T23:37:05.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f0226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fbf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f0226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fbf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54002/0 (socket says 172.21.15.33:54002) 2024-04-24T23:37:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f0226d9d640 1 -- 172.21.15.33:0/2656259259 learned_addr learned my addr 172.21.15.33:0/2656259259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.881+0000 7f022659c640 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fce0b0 0x556582fd7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0225d9b640 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fceaa0 0x556582fdf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0226d9d640 1 -- 172.21.15.33:0/2656259259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fceaa0 msgr2=0x556582fdf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0226d9d640 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fceaa0 0x556582fdf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0226d9d640 1 -- 172.21.15.33:0/2656259259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fce0b0 msgr2=0x556582fd7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0226d9d640 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fce0b0 0x556582fd7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0226d9d640 1 -- 172.21.15.33:0/2656259259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556582d077d0 con 0x556582fd35a0 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f0226d9d640 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fbf2b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f02200055f0 tx=0x7f02200059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec55005fcfc7004e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f020ffff640 1 -- 172.21.15.33:0/2656259259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022004e070 con 0x556582fd35a0 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f020ffff640 1 -- 172.21.15.33:0/2656259259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0220036040 con 0x556582fd35a0 2024-04-24T23:37:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.882+0000 7f020ffff640 1 -- 172.21.15.33:0/2656259259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220030c60 con 0x556582fd35a0 2024-04-24T23:37:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- 172.21.15.33:0/2656259259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 msgr2=0x556582fbf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fbf2b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f02200055f0 tx=0x7f02200059f0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- 172.21.15.33:0/2656259259 shutdown_connections 2024-04-24T23:37:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fd35a0 0x556582fbf2b0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fceaa0 0x556582fdf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- 172.21.15.33:0/2656259259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fce0b0 0x556582fd7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- 172.21.15.33:0/2656259259 >> 172.21.15.33:0/2656259259 conn(0x556582fc0820 msgr2=0x556582fd6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- 172.21.15.33:0/2656259259 shutdown_connections 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- 172.21.15.33:0/2656259259 wait complete. 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- start start 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 Processor -- start 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 0x556582fdef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fdb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fd35a0 0x556582fdbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02281a67c0 con 0x556582fce0b0 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02281a7c30 con 0x556582fceaa0 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02281a6e90 con 0x556582fd35a0 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.883+0000 7f022f820740 10 client.0 _ll_register_callbacks cb 0x7f0220009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 0x556582fdef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 0x556582fdef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54014/0 (socket says 172.21.15.33:54014) 2024-04-24T23:37:05.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 -- 172.21.15.33:0/3595278452 learned_addr learned my addr 172.21.15.33:0/3595278452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:05.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f0225d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fdb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f0226d9d640 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fd35a0 0x556582fdbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 -- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fd35a0 msgr2=0x556582fdbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fd35a0 0x556582fdbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 -- 172.21.15.33:0/3595278452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 msgr2=0x556582fdb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fdb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 -- 172.21.15.33:0/3595278452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556583048180 con 0x556582fce0b0 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022659c640 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 0x556582fdef20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f021801b040 tx=0x7f0218005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900b325f5e14ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f020dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f020dffb640 1 -- 172.21.15.33:0/3595278452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218005390 con 0x556582fce0b0 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f020dffb640 1 -- 172.21.15.33:0/3595278452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0218005530 con 0x556582fce0b0 2024-04-24T23:37:05.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0220036040 con 0x556582fce0b0 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f020dffb640 1 -- 172.21.15.33:0/3595278452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0218004020 con 0x556582fce0b0 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022f820740 10 client.20586 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556583049340 con 0x556582fce0b0 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.884+0000 7f020dffb640 1 -- 172.21.15.33:0/3595278452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0220036040 con 0x556582fce0b0 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f020dffb640 1 -- 172.21.15.33:0/3595278452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f0218004680 con 0x556582fce0b0 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 10 client.20586 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 10 client.20586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 20 client.20586 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x55658304bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 4 client.20586 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556582f30f00 con 0x556582fe3bb0 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f0225d9b640 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x55658304bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f0225d9b640 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x55658304bc20 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f86361c8a89d0a server_cookie=446c31aaa4b1e791 in_seq=0 out_seq=0 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f020dffb640 10 client.20586 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f020dffb640 1 -- 172.21.15.33:0/3595278452 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556582f30f00 con 0x556582fe3bb0 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f020dffb640 10 client.20586 handle_command_reply: tid=0 2024-04-24T23:37:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.885+0000 7f022f820740 1 client.20586 shutdown 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 msgr2=0x556582fdef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 0x556582fdef20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f021801b040 tx=0x7f0218005c50 comp rx=0 tx=0).stop 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 shutdown_connections 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x556582fe3bb0 0x55658304bc20 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556582fd35a0 0x556582fdbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556582fceaa0 0x556582fdb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 --2- 172.21.15.33:0/3595278452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556582fce0b0 0x556582fdef20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 >> 172.21.15.33:0/3595278452 conn(0x556582fc0820 msgr2=0x556582fbf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f020dffb640 0 client.20586 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 shutdown_connections 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 1 -- 172.21.15.33:0/3595278452 wait complete. 2024-04-24T23:37:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022f820740 20 client.20586 trim_cache size 0 max 16384 2024-04-24T23:37:05.904 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02100a2b90 msgr2=0x7f02100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02100a2b90 0x7f02100a5050 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x556582fc43e0 tx=0x7f022002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228101870 msgr2=0x7f0228197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228101870 0x7f0228197fd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f021c00e9f0 tx=0x7f021c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 shutdown_connections 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f02100a2b90 0x7f02100a5050 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0228102230 0x7f022819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0228101870 0x7f0228197fd0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 --2- 172.21.15.33:0/1868558108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0228100680 0x7f0228197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 >> 172.21.15.33:0/1868558108 conn(0x7f02280fbbf0 msgr2=0x7f02280fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 shutdown_connections 2024-04-24T23:37:05.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:05.886+0000 7f022ca41640 1 -- 172.21.15.33:0/1868558108 wait complete. 2024-04-24T23:37:05.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 276912 21.81 MB/sec execute 550 sec latency 126.335 ms 2024-04-24T23:37:05.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:05 smithi038 ceph-mon[32450]: pgmap v1103: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2024-04-24T23:37:05.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:05 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3014908011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:37:05.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:05 smithi117 ceph-mon[32529]: pgmap v1103: 129 pgs: 129 active+clean; 651 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 397 op/s 2024-04-24T23:37:05.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:05 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3014908011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:37:06.908 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:37:06.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:37:06.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:37:06.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 277189 21.80 MB/sec execute 551 sec latency 163.222 ms 2024-04-24T23:37:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 -- 172.21.15.33:0/1544683604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c100050 msgr2=0x7f781c100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 --2- 172.21.15.33:0/1544683604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c100050 0x7f781c100470 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f781802a410 comp rx=0 tx=0).stop 2024-04-24T23:37:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 -- 172.21.15.33:0/1544683604 shutdown_connections 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 --2- 172.21.15.33:0/1544683604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f781c101350 0x7f781c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 --2- 172.21.15.33:0/1544683604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c1009b0 0x7f781c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 --2- 172.21.15.33:0/1544683604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c100050 0x7f781c100470 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 -- 172.21.15.33:0/1544683604 >> 172.21.15.33:0/1544683604 conn(0x7f781c0fbbf0 msgr2=0x7f781c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 -- 172.21.15.33:0/1544683604 shutdown_connections 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.051+0000 7f7823577640 1 -- 172.21.15.33:0/1544683604 wait complete. 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 Processor -- start 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 -- start start 2024-04-24T23:37:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f781c100050 0x7f781c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c1009b0 0x7f781c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 0x7f781c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f610 con 0x7f781c101350 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.052+0000 7f7823577640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f490 con 0x7f781c1009b0 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7823577640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f790 con 0x7f781c100050 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 0x7f781c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f781c100050 0x7f781c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 0x7f781c1a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54026/0 (socket says 172.21.15.33:54026) 2024-04-24T23:37:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 -- 172.21.15.33:0/1672018911 learned_addr learned my addr 172.21.15.33:0/1672018911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7820b12640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c1009b0 0x7f781c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 -- 172.21.15.33:0/1672018911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f781c100050 msgr2=0x7f781c199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f781c100050 0x7f781c199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 -- 172.21.15.33:0/1672018911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c1009b0 msgr2=0x7f781c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c1009b0 0x7f781c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 -- 172.21.15.33:0/1672018911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c1a1910 con 0x7f781c101350 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7821b14640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 0x7f781c1a1270 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7814009fd0 tx=0x7f781400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f78127fc640 1 -- 172.21.15.33:0/1672018911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814017070 con 0x7f781c101350 2024-04-24T23:37:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f78127fc640 1 -- 172.21.15.33:0/1672018911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7814003e50 con 0x7f781c101350 2024-04-24T23:37:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f78127fc640 1 -- 172.21.15.33:0/1672018911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140106c0 con 0x7f781c101350 2024-04-24T23:37:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.053+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c1a1c00 con 0x7f781c101350 2024-04-24T23:37:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.054+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1a2120 con 0x7f781c101350 2024-04-24T23:37:07.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 Processor -- start 2024-04-24T23:37:07.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f78127fc640 1 -- 172.21.15.33:0/1672018911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7814003970 con 0x7f781c101350 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 -- start start 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c13d90 0x559ec8c141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 0x559ec8c19280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c1bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ec8b772c0 con 0x559ec8c14780 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559ec8b1f910 con 0x559ec8c13d90 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.056+0000 7f7826367740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ec8b73280 con 0x559ec8c198f0 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7820b12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c13d90 0x559ec8c141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 0x559ec8c19280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 0x559ec8c19280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54028/0 (socket says 172.21.15.33:54028) 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 -- 172.21.15.33:0/2323829242 learned_addr learned my addr 172.21.15.33:0/2323829242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7820b12640 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c1bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 -- 172.21.15.33:0/2323829242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 msgr2=0x559ec8c1bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c1bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 -- 172.21.15.33:0/2323829242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c13d90 msgr2=0x559ec8c141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c13d90 0x559ec8c141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 -- 172.21.15.33:0/2323829242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ec8b30fb0 con 0x559ec8c14780 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f78127fc640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78080a2c10 0x7f78080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7821313640 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 0x559ec8c19280 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f78180314e0 tx=0x7f7818004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d927619bec633f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f7820b12640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78080a2c10 0x7f78080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f78127fc640 1 -- 172.21.15.33:0/1672018911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f78140cf0e0 con 0x7f781c101350 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f78077fe640 1 -- 172.21.15.33:0/2323829242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818035040 con 0x559ec8c14780 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f78077fe640 1 -- 172.21.15.33:0/2323829242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7818005520 con 0x559ec8c14780 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.057+0000 7f78077fe640 1 -- 172.21.15.33:0/2323829242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78180316b0 con 0x559ec8c14780 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 -- 172.21.15.33:0/2323829242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 msgr2=0x559ec8c19280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7820b12640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78080a2c10 0x7f78080a50d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x559ec8c14fe0 tx=0x7f780c0096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 0x559ec8c19280 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f78180314e0 tx=0x7f7818004170 comp rx=0 tx=0).stop 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 -- 172.21.15.33:0/2323829242 shutdown_connections 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c1bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c14780 0x559ec8c19280 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 --2- 172.21.15.33:0/2323829242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c13d90 0x559ec8c141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 -- 172.21.15.33:0/2323829242 >> 172.21.15.33:0/2323829242 conn(0x559ec8bdb950 msgr2=0x559ec8c0f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 -- 172.21.15.33:0/2323829242 shutdown_connections 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 -- 172.21.15.33:0/2323829242 wait complete. 2024-04-24T23:37:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.058+0000 7f7826367740 1 -- start start 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 Processor -- start 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c14780 0x559ec8c1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c198f0 0x559ec8c256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ec8c1c840 con 0x559ec8c13d90 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559ec8b32e90 con 0x559ec8c198f0 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ec8c1c9c0 con 0x559ec8c14780 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7821313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7821313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c1e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54030/0 (socket says 172.21.15.33:54030) 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7821313640 1 -- 172.21.15.33:0/3326828698 learned_addr learned my addr 172.21.15.33:0/3326828698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7820b12640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c14780 0x559ec8c1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7821b14640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c198f0 0x559ec8c256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7826367740 10 client.0 _ll_register_callbacks cb 0x559ec8b6e970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7820b12640 1 -- 172.21.15.33:0/3326828698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c198f0 msgr2=0x559ec8c256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7820b12640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c198f0 0x559ec8c256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7820b12640 1 -- 172.21.15.33:0/3326828698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 msgr2=0x559ec8c1e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.059+0000 7f7820b12640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c1e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7820b12640 1 -- 172.21.15.33:0/3326828698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ec89537e0 con 0x559ec8c14780 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7821313640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7820b12640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c14780 0x559ec8c1e6a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f780c009fd0 tx=0x7f780c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cbaac86b44007ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7804ff9640 1 -- 172.21.15.33:0/3326828698 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c021a90 con 0x559ec8c14780 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7818005520 con 0x559ec8c14780 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7804ff9640 1 -- 172.21.15.33:0/3326828698 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f780c009040 con 0x559ec8c14780 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7804ff9640 1 -- 172.21.15.33:0/3326828698 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f780c034050 con 0x559ec8c14780 2024-04-24T23:37:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7826367740 10 client.29992 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ec89b3f70 con 0x559ec8c14780 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7804ff9640 1 -- 172.21.15.33:0/3326828698 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7818005520 con 0x559ec8c14780 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7804ff9640 1 -- 172.21.15.33:0/3326828698 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f780c01d070 con 0x559ec8c14780 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7826367740 10 client.29992 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.060+0000 7f7826367740 10 client.29992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.061+0000 7f7826367740 20 client.29992 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.061+0000 7f7826367740 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c29d60 0x559ec8c2c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.061+0000 7f7826367740 4 client.29992 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.061+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ec8b4b5a0 con 0x559ec8c29d60 2024-04-24T23:37:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.061+0000 7f7821313640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c29d60 0x559ec8c2c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.061+0000 7f7804ff9640 10 client.29992 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.062+0000 7f7821313640 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c29d60 0x559ec8c2c200 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991c99046493f166 server_cookie=1718d58de275be64 in_seq=0 out_seq=0 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.062+0000 7f7804ff9640 1 -- 172.21.15.33:0/3326828698 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ec8b4b5a0 con 0x559ec8c29d60 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.062+0000 7f7804ff9640 10 client.29992 handle_command_reply: tid=0 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.062+0000 7f7826367740 1 client.29992 shutdown 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c14780 msgr2=0x559ec8c1e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c14780 0x559ec8c1e6a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f780c009fd0 tx=0x7f780c004730 comp rx=0 tx=0).stop 2024-04-24T23:37:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 shutdown_connections 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c29d60 0x559ec8c2c200 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c198f0 0x559ec8c256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c14780 0x559ec8c1e6a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7804ff9640 0 client.29992 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 --2- 172.21.15.33:0/3326828698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 >> 172.21.15.33:0/3326828698 conn(0x559ec8bdb950 msgr2=0x559ec8c0e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 shutdown_connections 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 1 -- 172.21.15.33:0/3326828698 wait complete. 2024-04-24T23:37:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.063+0000 7f7826367740 20 client.29992 trim_cache size 0 max 16384 2024-04-24T23:37:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 Processor -- start 2024-04-24T23:37:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 -- start start 2024-04-24T23:37:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c15b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aee70 con 0x559ec8c13d90 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1ac710 con 0x559ec8c14780 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7826367740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aa560 con 0x559ec8c198f0 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7821313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7821313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c15b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54036/0 (socket says 172.21.15.33:54036) 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7821313640 1 -- 172.21.15.33:0/4141023542 learned_addr learned my addr 172.21.15.33:0/4141023542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7821b14640 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.081+0000 7f7820b12640 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821313640 1 -- 172.21.15.33:0/4141023542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 msgr2=0x559ec8c256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821313640 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821313640 1 -- 172.21.15.33:0/4141023542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 msgr2=0x559ec8c160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821313640 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821313640 1 -- 172.21.15.33:0/4141023542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ec884b170 con 0x559ec8c13d90 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821b14640 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7821313640 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c15b80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f78180315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05ed2be4143cccb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7806ffd640 1 -- 172.21.15.33:0/4141023542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818059070 con 0x559ec8c13d90 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7806ffd640 1 -- 172.21.15.33:0/4141023542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78180538f0 con 0x559ec8c13d90 2024-04-24T23:37:07.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.082+0000 7f7806ffd640 1 -- 172.21.15.33:0/4141023542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818053070 con 0x559ec8c13d90 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 -- 172.21.15.33:0/4141023542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 msgr2=0x559ec8c15b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c15b80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f78180315b0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 -- 172.21.15.33:0/4141023542 shutdown_connections 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c198f0 0x559ec8c256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 --2- 172.21.15.33:0/4141023542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c13d90 0x559ec8c15b80 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 -- 172.21.15.33:0/4141023542 >> 172.21.15.33:0/4141023542 conn(0x559ec8bdb950 msgr2=0x559ec8c1e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 -- 172.21.15.33:0/4141023542 shutdown_connections 2024-04-24T23:37:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 -- 172.21.15.33:0/4141023542 wait complete. 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 -- start start 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.083+0000 7f7826367740 1 Processor -- start 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c13d90 0x559ec8c22b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c23060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 0x559ec8c24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aabd0 con 0x559ec8c198f0 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1ac040 con 0x559ec8c14780 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1ab2a0 con 0x559ec8c13d90 2024-04-24T23:37:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7826367740 10 client.0 _ll_register_callbacks cb 0x7f780c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 0x559ec8c24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7820b12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c23060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c13d90 0x559ec8c22b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 0x559ec8c24000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54044/0 (socket says 172.21.15.33:54044) 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 -- 172.21.15.33:0/769806681 learned_addr learned my addr 172.21.15.33:0/769806681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 -- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c13d90 msgr2=0x559ec8c22b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c13d90 0x559ec8c22b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 -- 172.21.15.33:0/769806681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 msgr2=0x559ec8c23060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c23060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 -- 172.21.15.33:0/769806681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ec8c245f0 con 0x559ec8c198f0 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821313640 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c13d90 0x559ec8c22b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7821b14640 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 0x559ec8c24000 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7814097a30 tx=0x7f781409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a70fcf8d7cdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.084+0000 7f7804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7804ff9640 1 -- 172.21.15.33:0/769806681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140b3070 con 0x559ec8c198f0 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7804ff9640 1 -- 172.21.15.33:0/769806681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7814084b60 con 0x559ec8c198f0 2024-04-24T23:37:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78180538f0 con 0x559ec8c198f0 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7804ff9640 1 -- 172.21.15.33:0/769806681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7814020040 con 0x559ec8c198f0 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 10 client.20604 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ec8c25a00 con 0x559ec8c198f0 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7804ff9640 1 -- 172.21.15.33:0/769806681 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78180538f0 con 0x559ec8c198f0 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7804ff9640 1 -- 172.21.15.33:0/769806681 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f781409aab0 con 0x559ec8c198f0 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 10 client.20604 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 10 client.20604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 20 client.20604 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c90b40 0x559ec8c92f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 4 client.20604 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ec8b84800 con 0x559ec8c90b40 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.085+0000 7f7821313640 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c90b40 0x559ec8c92f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.086+0000 7f7804ff9640 10 client.20604 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.086+0000 7f7821313640 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c90b40 0x559ec8c92f60 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c40a6da4025e6a server_cookie=6dda08a3d3742455 in_seq=0 out_seq=0 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7804ff9640 1 -- 172.21.15.33:0/769806681 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ec8b84800 con 0x559ec8c90b40 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7804ff9640 10 client.20604 handle_command_reply: tid=0 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 client.20604 shutdown 2024-04-24T23:37:07.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 msgr2=0x559ec8c24000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 0x559ec8c24000 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7814097a30 tx=0x7f781409b2d0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 shutdown_connections 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559ec8c90b40 0x559ec8c92f60 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ec8c198f0 0x559ec8c24000 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ec8c14780 0x559ec8c23060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 --2- 172.21.15.33:0/769806681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ec8c13d90 0x559ec8c22b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7804ff9640 0 client.20604 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 >> 172.21.15.33:0/769806681 conn(0x559ec8bdb950 msgr2=0x559ec8c1e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 shutdown_connections 2024-04-24T23:37:07.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.087+0000 7f7826367740 1 -- 172.21.15.33:0/769806681 wait complete. 2024-04-24T23:37:07.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7826367740 20 client.20604 trim_cache size 0 max 16384 2024-04-24T23:37:07.100 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:37:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:37:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "2b60eb3c-0a2d-4a28-8d00-552a81288fa6", 2024-04-24T23:37:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:37:07.101 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78080a2c10 msgr2=0x7f78080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78080a2c10 0x7f78080a50d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x559ec8c14fe0 tx=0x7f780c0096c0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 msgr2=0x7f781c1a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 0x7f781c1a1270 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7814009fd0 tx=0x7f781400b260 comp rx=0 tx=0).stop 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 shutdown_connections 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f78080a2c10 0x7f78080a50d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f781c101350 0x7f781c1a1270 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f781c1009b0 0x7f781c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 --2- 172.21.15.33:0/1672018911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f781c100050 0x7f781c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 >> 172.21.15.33:0/1672018911 conn(0x7f781c0fbbf0 msgr2=0x7f781c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 shutdown_connections 2024-04-24T23:37:07.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.088+0000 7f7823577640 1 -- 172.21.15.33:0/1672018911 wait complete. 2024-04-24T23:37:07.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:37:07.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:37:07.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.229+0000 7ffa27476640 1 -- 172.21.15.33:0/1773951277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20102220 msgr2=0x7ffa20104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.229+0000 7ffa27476640 1 --2- 172.21.15.33:0/1773951277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20102220 0x7ffa20104610 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009900 tx=0x7ffa1c02a410 comp rx=0 tx=0).stop 2024-04-24T23:37:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- 172.21.15.33:0/1773951277 shutdown_connections 2024-04-24T23:37:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 --2- 172.21.15.33:0/1773951277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa20107480 0x7ffa201098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 --2- 172.21.15.33:0/1773951277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa20104b50 0x7ffa20106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 --2- 172.21.15.33:0/1773951277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20102220 0x7ffa20104610 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- 172.21.15.33:0/1773951277 >> 172.21.15.33:0/1773951277 conn(0x7ffa200fbc10 msgr2=0x7ffa200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- 172.21.15.33:0/1773951277 shutdown_connections 2024-04-24T23:37:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- 172.21.15.33:0/1773951277 wait complete. 2024-04-24T23:37:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 Processor -- start 2024-04-24T23:37:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- start start 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa20102220 0x7ffa20197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 0x7ffa20198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa20107480 0x7ffa2019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010d4a0 con 0x7ffa20104b50 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010d320 con 0x7ffa20107480 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.230+0000 7ffa27476640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010d620 con 0x7ffa20102220 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 0x7ffa20198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 0x7ffa20198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54064/0 (socket says 172.21.15.33:54064) 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 -- 172.21.15.33:0/3915737563 learned_addr learned my addr 172.21.15.33:0/3915737563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa25212640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa20102220 0x7ffa20197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa25a13640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa20107480 0x7ffa2019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 -- 172.21.15.33:0/3915737563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa20102220 msgr2=0x7ffa20197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa20102220 0x7ffa20197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 -- 172.21.15.33:0/3915737563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa20107480 msgr2=0x7ffa2019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa20107480 0x7ffa2019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 -- 172.21.15.33:0/3915737563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2019f740 con 0x7ffa20104b50 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa24a11640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 0x7ffa20198060 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9f0 tx=0x7ffa1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa167fc640 1 -- 172.21.15.33:0/3915737563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000ccc0 con 0x7ffa20104b50 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2019fa30 con 0x7ffa20104b50 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa167fc640 1 -- 172.21.15.33:0/3915737563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa10005500 con 0x7ffa20104b50 2024-04-24T23:37:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa167fc640 1 -- 172.21.15.33:0/3915737563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10010690 con 0x7ffa20104b50 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.231+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2019ff80 con 0x7ffa20104b50 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.235+0000 7ffa2a184740 1 Processor -- start 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 -- start start 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a21660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a26730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a26da0 0x55b692a29230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6929856b0 con 0x55b692a26da0 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b69292d880 con 0x55b692a21c30 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.237+0000 7ffa2a184740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b692981670 con 0x55b692a21240 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa167fc640 1 -- 172.21.15.33:0/3915737563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffa100046c0 con 0x7ffa20104b50 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a21660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a21660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46910/0 (socket says 172.21.15.33:46910) 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 -- 172.21.15.33:0/1856089614 learned_addr learned my addr 172.21.15.33:0/1856089614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 -- 172.21.15.33:0/1856089614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 msgr2=0x55b692a26730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a26730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 -- 172.21.15.33:0/1856089614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a26da0 msgr2=0x55b692a29230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a26da0 0x55b692a29230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.238+0000 7ffa25a13640 1 -- 172.21.15.33:0/1856089614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6927637d0 con 0x55b692a21240 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.239+0000 7ffa25a13640 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a21660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009fd0 tx=0x7ffa180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e66f58d716a971a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.239+0000 7ffa0b7fe640 1 -- 172.21.15.33:0/1856089614 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004d20 con 0x55b692a21240 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.239+0000 7ffa167fc640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa0c0a2c10 0x7ffa0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.239+0000 7ffa0b7fe640 1 -- 172.21.15.33:0/1856089614 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18004050 con 0x55b692a21240 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.239+0000 7ffa0b7fe640 1 -- 172.21.15.33:0/1856089614 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18005280 con 0x55b692a21240 2024-04-24T23:37:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.239+0000 7ffa25212640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa0c0a2c10 0x7ffa0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa167fc640 1 -- 172.21.15.33:0/3915737563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa100cf7a0 con 0x7ffa20104b50 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 -- 172.21.15.33:0/1856089614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 msgr2=0x55b692a21660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a21660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009fd0 tx=0x7ffa180118d0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 -- 172.21.15.33:0/1856089614 shutdown_connections 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a26da0 0x55b692a29230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a26730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1856089614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a21660 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa2a184740 1 -- 172.21.15.33:0/1856089614 >> 172.21.15.33:0/1856089614 conn(0x55b692a1c8b0 msgr2=0x55b692a1ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.240+0000 7ffa25212640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa0c0a2c10 0x7ffa0c0a50d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c031440 tx=0x7ffa1c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.246+0000 7ffa2a184740 1 -- 172.21.15.33:0/1856089614 shutdown_connections 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.247+0000 7ffa2a184740 1 -- 172.21.15.33:0/1856089614 wait complete. 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.247+0000 7ffa2a184740 1 -- start start 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.247+0000 7ffa2a184740 1 Processor -- start 2024-04-24T23:37:07.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a23d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 0x55b692a242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a252e0 0x55b692a25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b692a2e160 con 0x55b692a21c30 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b692940f50 con 0x55b692a252e0 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b692a2e2e0 con 0x55b692a21240 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa2a184740 10 client.0 _ll_register_callbacks cb 0x55b69297c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa24a11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 0x55b692a242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa24a11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 0x55b692a242d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54080/0 (socket says 172.21.15.33:54080) 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa24a11640 1 -- 172.21.15.33:0/1068730274 learned_addr learned my addr 172.21.15.33:0/1068730274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.248+0000 7ffa25212640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a23d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.249+0000 7ffa25a13640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a252e0 0x55b692a25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.249+0000 7ffa24a11640 1 -- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 msgr2=0x55b692a23d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.249+0000 7ffa24a11640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a23d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.249+0000 7ffa24a11640 1 -- 172.21.15.33:0/1068730274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a252e0 msgr2=0x55b692a25760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.249+0000 7ffa24a11640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a252e0 0x55b692a25760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.249+0000 7ffa24a11640 1 -- 172.21.15.33:0/1068730274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6926993f0 con 0x55b692a21c30 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa24a11640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 0x55b692a242d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9c0 tx=0x7ffa1009b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3024c33cbab992 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa08ff9640 1 -- 172.21.15.33:0/1068730274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1009bc10 con 0x55b692a21c30 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa18004050 con 0x55b692a21c30 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa08ff9640 1 -- 172.21.15.33:0/1068730274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa100a43f0 con 0x55b692a21c30 2024-04-24T23:37:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa08ff9640 1 -- 172.21.15.33:0/1068730274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa1009ffc0 con 0x55b692a21c30 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa2a184740 10 client.20613 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.250+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b6926590f0 con 0x55b692a21c30 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.251+0000 7ffa08ff9640 1 -- 172.21.15.33:0/1068730274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18004050 con 0x55b692a21c30 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.251+0000 7ffa08ff9640 1 -- 172.21.15.33:0/1068730274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ffa1009aad0 con 0x55b692a21c30 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa2a184740 10 client.20613 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa2a184740 10 client.20613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa2a184740 20 client.20613 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a40120 0x55b692a425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa2a184740 4 client.20613 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b692992bf0 con 0x55b692a40120 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.252+0000 7ffa25212640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a40120 0x55b692a425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.254+0000 7ffa25212640 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a40120 0x55b692a425a0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c76ba371b9db066 server_cookie=da7dbaed7f05d3f in_seq=0 out_seq=0 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.255+0000 7ffa08ff9640 10 client.20613 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.255+0000 7ffa08ff9640 1 -- 172.21.15.33:0/1068730274 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b692992bf0 con 0x55b692a40120 2024-04-24T23:37:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.255+0000 7ffa08ff9640 10 client.20613 handle_command_reply: tid=0 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.255+0000 7ffa2a184740 1 client.20613 shutdown 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.258+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 msgr2=0x55b692a242d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.258+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 0x55b692a242d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9c0 tx=0x7ffa1009b630 comp rx=0 tx=0).stop 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 shutdown_connections 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a40120 0x55b692a425a0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a252e0 0x55b692a25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21c30 0x55b692a242d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 --2- 172.21.15.33:0/1068730274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a23d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa08ff9640 0 client.20613 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 >> 172.21.15.33:0/1068730274 conn(0x55b692a1c8b0 msgr2=0x55b692a1baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 shutdown_connections 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.259+0000 7ffa2a184740 1 -- 172.21.15.33:0/1068730274 wait complete. 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.260+0000 7ffa2a184740 20 client.20613 trim_cache size 0 max 16384 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.295+0000 7ffa2a184740 1 Processor -- start 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 -- start start 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21240 0x55b692a299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a252e0 0x55b692a22a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201aabb0 con 0x55b692a21240 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201a8450 con 0x55b692a21c30 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.296+0000 7ffa2a184740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201a62a0 con 0x55b692a252e0 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.297+0000 7ffa25212640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21240 0x55b692a299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.297+0000 7ffa25212640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21240 0x55b692a299a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54082/0 (socket says 172.21.15.33:54082) 2024-04-24T23:37:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.297+0000 7ffa25212640 1 -- 172.21.15.33:0/2630470435 learned_addr learned my addr 172.21.15.33:0/2630470435 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 -- 172.21.15.33:0/2630470435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a252e0 msgr2=0x55b692a22a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a252e0 0x55b692a22a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 -- 172.21.15.33:0/2630470435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21240 msgr2=0x55b692a299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21240 0x55b692a299a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 -- 172.21.15.33:0/2630470435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b692a182a0 con 0x55b692a21c30 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.298+0000 7ffa24a11640 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a224c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa1009bba0 tx=0x7ffa1009b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00f0d1e38137a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.302+0000 7ffa0affd640 1 -- 172.21.15.33:0/2630470435 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1009bc10 con 0x55b692a21c30 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.302+0000 7ffa0affd640 1 -- 172.21.15.33:0/2630470435 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa100a4880 con 0x55b692a21c30 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.302+0000 7ffa0affd640 1 -- 172.21.15.33:0/2630470435 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1009b060 con 0x55b692a21c30 2024-04-24T23:37:07.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.304+0000 7ffa2a184740 1 -- 172.21.15.33:0/2630470435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 msgr2=0x55b692a224c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.304+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a224c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa1009bba0 tx=0x7ffa1009b6a0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.305+0000 7ffa2a184740 1 -- 172.21.15.33:0/2630470435 shutdown_connections 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.305+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a252e0 0x55b692a22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.305+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a224c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.305+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2630470435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a21240 0x55b692a299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.305+0000 7ffa2a184740 1 -- 172.21.15.33:0/2630470435 >> 172.21.15.33:0/2630470435 conn(0x55b692a1c8b0 msgr2=0x55b692a2a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.309+0000 7ffa2a184740 1 -- 172.21.15.33:0/2630470435 shutdown_connections 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.309+0000 7ffa2a184740 1 -- 172.21.15.33:0/2630470435 wait complete. 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.309+0000 7ffa2a184740 1 -- start start 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.310+0000 7ffa2a184740 1 Processor -- start 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a252e0 0x55b692aa46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201a6910 con 0x55b692a252e0 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201a7d80 con 0x55b692a21c30 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201a6fe0 con 0x55b692a21240 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.313+0000 7ffa2a184740 10 client.0 _ll_register_callbacks cb 0x7ffa1009ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa24a11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa24a11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a39d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:48302/0 (socket says 172.21.15.33:48302) 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa24a11640 1 -- 172.21.15.33:0/2638407818 learned_addr learned my addr 172.21.15.33:0/2638407818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25212640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25a13640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a252e0 0x55b692aa46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25a13640 1 -- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 msgr2=0x55b692a397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25a13640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25a13640 1 -- 172.21.15.33:0/2638407818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 msgr2=0x55b692a39d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25a13640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a39d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.314+0000 7ffa25a13640 1 -- 172.21.15.33:0/2638407818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b692a3a3a0 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.315+0000 7ffa25a13640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a252e0 0x55b692aa46d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7ffa18006010 tx=0x7ffa18005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe48e3c2cda27656 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.315+0000 7ffa08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.316+0000 7ffa08ff9640 1 -- 172.21.15.33:0/2638407818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180055c0 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.316+0000 7ffa08ff9640 1 -- 172.21.15.33:0/2638407818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa18005760 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.316+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa100a4880 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.317+0000 7ffa08ff9640 1 -- 172.21.15.33:0/2638407818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa180059a0 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.317+0000 7ffa2a184740 10 client.20619 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.317+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b692a2f800 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.318+0000 7ffa08ff9640 1 -- 172.21.15.33:0/2638407818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa100a4880 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa08ff9640 1 -- 172.21.15.33:0/2638407818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ffa1800f4e0 con 0x55b692a252e0 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa2a184740 10 client.20619 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa2a184740 10 client.20619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:07.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa2a184740 20 client.20619 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a2ff90 0x55b692a323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa2a184740 4 client.20619 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.319+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b69298cf40 con 0x55b692a2ff90 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.321+0000 7ffa25212640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a2ff90 0x55b692a323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa25212640 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a2ff90 0x55b692a323b0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11a4f1d681a5e01 server_cookie=3fa91ef271b9ce32 in_seq=0 out_seq=0 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa08ff9640 10 client.20619 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa08ff9640 1 -- 172.21.15.33:0/2638407818 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b69298cf40 con 0x55b692a2ff90 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa08ff9640 10 client.20619 handle_command_reply: tid=0 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 client.20619 shutdown 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a252e0 msgr2=0x55b692aa46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a252e0 0x55b692aa46d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7ffa18006010 tx=0x7ffa18005040 comp rx=0 tx=0).stop 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 shutdown_connections 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b692a2ff90 0x55b692a323b0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b692a252e0 0x55b692aa46d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b692a21c30 0x55b692a39d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 --2- 172.21.15.33:0/2638407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b692a21240 0x55b692a397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 >> 172.21.15.33:0/2638407818 conn(0x55b692a1c8b0 msgr2=0x55b692a224c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.322+0000 7ffa08ff9640 0 client.20619 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.323+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 shutdown_connections 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.323+0000 7ffa2a184740 1 -- 172.21.15.33:0/2638407818 wait complete. 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.323+0000 7ffa2a184740 20 client.20619 trim_cache size 0 max 16384 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "2b60eb3c-0a2d-4a28-8d00-552a81288fa6": { 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "2b60eb3c-0a2d-4a28-8d00-552a81288fa6", 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:37:07.360 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:37:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa0c0a2c10 msgr2=0x7ffa0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa0c0a2c10 0x7ffa0c0a50d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c031440 tx=0x7ffa1c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 msgr2=0x7ffa20198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 0x7ffa20198060 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9f0 tx=0x7ffa1000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 shutdown_connections 2024-04-24T23:37:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ffa0c0a2c10 0x7ffa0c0a50d0 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c031440 tx=0x7ffa1c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa20107480 0x7ffa2019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa20104b50 0x7ffa20198060 secure :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9f0 tx=0x7ffa1000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 --2- 172.21.15.33:0/3915737563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa20102220 0x7ffa20197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:07.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.324+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 >> 172.21.15.33:0/3915737563 conn(0x7ffa200fbc10 msgr2=0x7ffa20102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:07.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.325+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 shutdown_connections 2024-04-24T23:37:07.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:07.325+0000 7ffa27476640 1 -- 172.21.15.33:0/3915737563 wait complete. 2024-04-24T23:37:07.363 INFO:tasks.cephfs.filesystem:scrub status for tag:2b60eb3c-0a2d-4a28-8d00-552a81288fa6 - {'path': '/', 'tag': '2b60eb3c-0a2d-4a28-8d00-552a81288fa6', 'options': 'recursive,force'} 2024-04-24T23:37:07.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:07 smithi033 ceph-mon[29201]: pgmap v1104: 129 pgs: 129 active+clean; 658 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.0 MiB/s wr, 394 op/s 2024-04-24T23:37:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:07 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:37:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:07 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:07 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:07 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:07.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:07 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:07.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 277498 21.78 MB/sec execute 552 sec latency 122.877 ms 2024-04-24T23:37:07.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:07 smithi038 ceph-mon[32450]: pgmap v1104: 129 pgs: 129 active+clean; 658 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.0 MiB/s wr, 394 op/s 2024-04-24T23:37:07.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:07 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:37:07.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:07 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:07 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:07 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:07.920 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:07 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:07.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:07 smithi117 ceph-mon[32529]: pgmap v1104: 129 pgs: 129 active+clean; 658 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.0 MiB/s wr, 394 op/s 2024-04-24T23:37:07.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:07 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:37:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:07 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:07 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:07 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:07.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:07 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:08.908 INFO:tasks.workunit.client.0.smithi033.stdout: 10 277712 21.75 MB/sec execute 553 sec latency 343.760 ms 2024-04-24T23:37:08.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:08 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:08.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:08 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:08.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:08 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:08.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:08 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:08.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:08 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:08.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:08 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:08.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:08 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:08.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:08 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:08.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:08 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:08.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:08 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:08.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:08 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:08.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:08 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:09.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 278041 21.75 MB/sec execute 554 sec latency 103.353 ms 2024-04-24T23:37:09.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:09 smithi038 ceph-mon[32450]: pgmap v1105: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 366 op/s 2024-04-24T23:37:09.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:09 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:09.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:09 smithi117 ceph-mon[32529]: pgmap v1105: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 366 op/s 2024-04-24T23:37:09.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:09 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:09.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:09 smithi033 ceph-mon[29201]: pgmap v1105: 129 pgs: 129 active+clean; 659 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 366 op/s 2024-04-24T23:37:09.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:09 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:10.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 278525 21.75 MB/sec execute 555 sec latency 122.973 ms 2024-04-24T23:37:11.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 279106 21.77 MB/sec execute 556 sec latency 55.485 ms 2024-04-24T23:37:11.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:11 smithi038 ceph-mon[32450]: pgmap v1106: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 402 op/s 2024-04-24T23:37:11.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:11 smithi117 ceph-mon[32529]: pgmap v1106: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 402 op/s 2024-04-24T23:37:11.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:11 smithi033 ceph-mon[29201]: pgmap v1106: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 402 op/s 2024-04-24T23:37:12.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 279547 21.76 MB/sec execute 557 sec latency 101.807 ms 2024-04-24T23:37:13.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 279896 21.76 MB/sec execute 558 sec latency 368.482 ms 2024-04-24T23:37:13.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:13 smithi038 ceph-mon[32450]: pgmap v1107: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 373 op/s 2024-04-24T23:37:13.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:13 smithi117 ceph-mon[32529]: pgmap v1107: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 373 op/s 2024-04-24T23:37:13.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:13 smithi033 ceph-mon[29201]: pgmap v1107: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 373 op/s 2024-04-24T23:37:14.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 280145 21.75 MB/sec execute 559 sec latency 136.302 ms 2024-04-24T23:37:15.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 280529 21.77 MB/sec execute 560 sec latency 96.210 ms 2024-04-24T23:37:15.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:15 smithi038 ceph-mon[32450]: pgmap v1108: 129 pgs: 129 active+clean; 649 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 447 op/s 2024-04-24T23:37:15.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:15 smithi117 ceph-mon[32529]: pgmap v1108: 129 pgs: 129 active+clean; 649 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 447 op/s 2024-04-24T23:37:15.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:15 smithi033 ceph-mon[29201]: pgmap v1108: 129 pgs: 129 active+clean; 649 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 447 op/s 2024-04-24T23:37:16.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 281024 21.78 MB/sec execute 561 sec latency 71.886 ms 2024-04-24T23:37:17.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 281489 21.77 MB/sec execute 562 sec latency 126.967 ms 2024-04-24T23:37:17.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:17 smithi038 ceph-mon[32450]: pgmap v1109: 129 pgs: 129 active+clean; 648 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-04-24T23:37:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:17 smithi117 ceph-mon[32529]: pgmap v1109: 129 pgs: 129 active+clean; 648 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-04-24T23:37:17.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:17 smithi033 ceph-mon[29201]: pgmap v1109: 129 pgs: 129 active+clean; 648 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-04-24T23:37:18.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 281870 21.77 MB/sec execute 563 sec latency 299.514 ms 2024-04-24T23:37:19.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 282133 21.76 MB/sec execute 564 sec latency 134.637 ms 2024-04-24T23:37:19.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:19 smithi038 ceph-mon[32450]: pgmap v1110: 129 pgs: 129 active+clean; 648 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 366 op/s 2024-04-24T23:37:19.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:19 smithi117 ceph-mon[32529]: pgmap v1110: 129 pgs: 129 active+clean; 648 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 366 op/s 2024-04-24T23:37:19.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:19 smithi033 ceph-mon[29201]: pgmap v1110: 129 pgs: 129 active+clean; 648 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 366 op/s 2024-04-24T23:37:20.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 282456 21.75 MB/sec execute 565 sec latency 81.462 ms 2024-04-24T23:37:21.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 282848 21.74 MB/sec execute 566 sec latency 157.090 ms 2024-04-24T23:37:21.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:21 smithi038 ceph-mon[32450]: pgmap v1111: 129 pgs: 129 active+clean; 654 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-04-24T23:37:21.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:21 smithi117 ceph-mon[32529]: pgmap v1111: 129 pgs: 129 active+clean; 654 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-04-24T23:37:21.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:21 smithi033 ceph-mon[29201]: pgmap v1111: 129 pgs: 129 active+clean; 654 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-04-24T23:37:22.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 283400 21.75 MB/sec execute 567 sec latency 88.937 ms 2024-04-24T23:37:22.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:22 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:37:22.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:22 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:37:22.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:22 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:37:22.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:22 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:37:22.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:22 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:37:22.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:22 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:37:23.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 283767 21.75 MB/sec execute 568 sec latency 340.793 ms 2024-04-24T23:37:23.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:23 smithi038 ceph-mon[32450]: pgmap v1112: 129 pgs: 129 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 396 op/s 2024-04-24T23:37:23.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:23 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:37:23.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:23 smithi117 ceph-mon[32529]: pgmap v1112: 129 pgs: 129 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 396 op/s 2024-04-24T23:37:23.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:23 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:37:23.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:23 smithi033 ceph-mon[29201]: pgmap v1112: 129 pgs: 129 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 396 op/s 2024-04-24T23:37:23.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:23 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:37:24.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 284002 21.73 MB/sec execute 569 sec latency 441.313 ms 2024-04-24T23:37:25.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 284487 21.74 MB/sec execute 570 sec latency 74.788 ms 2024-04-24T23:37:25.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:25 smithi038 ceph-mon[32450]: pgmap v1113: 129 pgs: 129 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 413 op/s 2024-04-24T23:37:25.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:25 smithi117 ceph-mon[32529]: pgmap v1113: 129 pgs: 129 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 413 op/s 2024-04-24T23:37:25.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:25 smithi033 ceph-mon[29201]: pgmap v1113: 129 pgs: 129 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 413 op/s 2024-04-24T23:37:26.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 285000 21.77 MB/sec execute 571 sec latency 82.178 ms 2024-04-24T23:37:27.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 285444 21.77 MB/sec execute 572 sec latency 73.413 ms 2024-04-24T23:37:27.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:27 smithi038 ceph-mon[32450]: pgmap v1114: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 347 op/s 2024-04-24T23:37:27.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:27 smithi117 ceph-mon[32529]: pgmap v1114: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 347 op/s 2024-04-24T23:37:27.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:27 smithi033 ceph-mon[29201]: pgmap v1114: 129 pgs: 129 active+clean; 656 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 347 op/s 2024-04-24T23:37:28.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 285766 21.76 MB/sec execute 573 sec latency 245.832 ms 2024-04-24T23:37:28.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:37:28.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:37:28.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:37:29.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 285924 21.74 MB/sec execute 574 sec latency 396.308 ms 2024-04-24T23:37:29.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:29 smithi038 ceph-mon[32450]: pgmap v1115: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 354 op/s 2024-04-24T23:37:29.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:29 smithi117 ceph-mon[32529]: pgmap v1115: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 354 op/s 2024-04-24T23:37:29.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:29 smithi033 ceph-mon[29201]: pgmap v1115: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 354 op/s 2024-04-24T23:37:30.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 286185 21.73 MB/sec execute 575 sec latency 91.108 ms 2024-04-24T23:37:30.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:37:30.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:37:30.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:37:30.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:37:30.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:37:30.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:37:30.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:37:30.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:37:30.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:37:31.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 286558 21.72 MB/sec execute 576 sec latency 84.492 ms 2024-04-24T23:37:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:31 smithi038 ceph-mon[32450]: pgmap v1116: 129 pgs: 129 active+clean; 663 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 420 op/s 2024-04-24T23:37:31.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:31 smithi117 ceph-mon[32529]: pgmap v1116: 129 pgs: 129 active+clean; 663 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 420 op/s 2024-04-24T23:37:31.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:31 smithi033 ceph-mon[29201]: pgmap v1116: 129 pgs: 129 active+clean; 663 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 420 op/s 2024-04-24T23:37:32.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 286990 21.71 MB/sec execute 577 sec latency 88.703 ms 2024-04-24T23:37:33.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 287688 21.74 MB/sec execute 578 sec latency 141.360 ms 2024-04-24T23:37:33.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:33 smithi038 ceph-mon[32450]: pgmap v1117: 129 pgs: 129 active+clean; 664 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-04-24T23:37:33.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:33 smithi117 ceph-mon[32529]: pgmap v1117: 129 pgs: 129 active+clean; 664 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-04-24T23:37:33.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:33 smithi033 ceph-mon[29201]: pgmap v1117: 129 pgs: 129 active+clean; 664 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-04-24T23:37:34.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 287878 21.71 MB/sec execute 579 sec latency 411.436 ms 2024-04-24T23:37:35.909 INFO:tasks.workunit.client.0.smithi033.stdout: 10 288261 21.72 MB/sec execute 580 sec latency 103.923 ms 2024-04-24T23:37:35.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:35 smithi117 ceph-mon[32529]: pgmap v1118: 129 pgs: 129 active+clean; 671 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 411 op/s 2024-04-24T23:37:35.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:35 smithi033 ceph-mon[29201]: pgmap v1118: 129 pgs: 129 active+clean; 671 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 411 op/s 2024-04-24T23:37:36.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:35 smithi038 ceph-mon[32450]: pgmap v1118: 129 pgs: 129 active+clean; 671 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 411 op/s 2024-04-24T23:37:36.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 288749 21.74 MB/sec execute 581 sec latency 89.717 ms 2024-04-24T23:37:37.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:37:37.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:37:37.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.529+0000 7ff91f59e640 1 -- 172.21.15.33:0/3421780980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 msgr2=0x7ff9201022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.529+0000 7ff91f59e640 1 --2- 172.21.15.33:0/3421780980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff9201022f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402a420 comp rx=0 tx=0).stop 2024-04-24T23:37:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 -- 172.21.15.33:0/3421780980 shutdown_connections 2024-04-24T23:37:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 --2- 172.21.15.33:0/3421780980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9201051e0 0x7ff920105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 --2- 172.21.15.33:0/3421780980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff920102830 0x7ff920104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 --2- 172.21.15.33:0/3421780980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff9201022f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 -- 172.21.15.33:0/3421780980 >> 172.21.15.33:0/3421780980 conn(0x7ff9200fbae0 msgr2=0x7ff9200fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 -- 172.21.15.33:0/3421780980 shutdown_connections 2024-04-24T23:37:37.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.530+0000 7ff91f59e640 1 -- 172.21.15.33:0/3421780980 wait complete. 2024-04-24T23:37:37.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.531+0000 7ff91f59e640 1 Processor -- start 2024-04-24T23:37:37.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.531+0000 7ff91f59e640 1 -- start start 2024-04-24T23:37:37.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff92019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff920102830 0x7ff92019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9201051e0 0x7ff9201a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91f59e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff920113970 con 0x7ff920102830 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91f59e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201137f0 con 0x7ff9201051e0 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91f59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff920113af0 con 0x7ff9200ffec0 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff920102830 0x7ff92019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff920102830 0x7ff92019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35032/0 (socket says 172.21.15.33:35032) 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff92019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff92019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51516/0 (socket says 172.21.15.33:51516) 2024-04-24T23:37:37.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 -- 172.21.15.33:0/1114556234 learned_addr learned my addr 172.21.15.33:0/1114556234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:37.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 -- 172.21.15.33:0/1114556234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9201051e0 msgr2=0x7ff9201a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9201051e0 0x7ff9201a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 -- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff920102830 msgr2=0x7ff92019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff920102830 0x7ff92019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.532+0000 7ff91e59c640 1 -- 172.21.15.33:0/1114556234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92006b290 con 0x7ff9200ffec0 2024-04-24T23:37:37.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.533+0000 7ff91e59c640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff92019df10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff91402aa10 tx=0x7ff914004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.533+0000 7ff90f7fe640 1 -- 172.21.15.33:0/1114556234 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91403f030 con 0x7ff9200ffec0 2024-04-24T23:37:37.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.533+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92006b520 con 0x7ff9200ffec0 2024-04-24T23:37:37.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.533+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92006ba60 con 0x7ff9200ffec0 2024-04-24T23:37:37.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.534+0000 7ff90f7fe640 1 -- 172.21.15.33:0/1114556234 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9140316e0 con 0x7ff9200ffec0 2024-04-24T23:37:37.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.535+0000 7ff90f7fe640 1 -- 172.21.15.33:0/1114556234 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914035070 con 0x7ff9200ffec0 2024-04-24T23:37:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.537+0000 7ff9277fd740 1 Processor -- start 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 -- start start 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18430 0x55b360c18850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c1d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c1df90 0x55b360c20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b360b7c6b0 con 0x55b360c18430 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b360b24880 con 0x55b360c1df90 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.538+0000 7ff9277fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b360b78670 con 0x55b360c18e20 2024-04-24T23:37:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18430 0x55b360c18850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18430 0x55b360c18850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35042/0 (socket says 172.21.15.33:35042) 2024-04-24T23:37:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91ed9d640 1 -- 172.21.15.33:0/1934386371 learned_addr learned my addr 172.21.15.33:0/1934386371 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c1df90 0x55b360c20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff90f7fe640 1 -- 172.21.15.33:0/1114556234 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff914042030 con 0x7ff9200ffec0 2024-04-24T23:37:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91dd9b640 1 -- 172.21.15.33:0/1934386371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 msgr2=0x55b360c1d920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:37:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c1d920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91dd9b640 1 -- 172.21.15.33:0/1934386371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18430 msgr2=0x55b360c18850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18430 0x55b360c18850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.539+0000 7ff91dd9b640 1 -- 172.21.15.33:0/1934386371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36095a7d0 con 0x55b360c1df90 2024-04-24T23:37:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.540+0000 7ff90f7fe640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9080a2c50 0x7ff9080a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.540+0000 7ff90f7fe640 1 -- 172.21.15.33:0/1114556234 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9140efc50 con 0x7ff9200ffec0 2024-04-24T23:37:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.541+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c1df90 0x55b360c20420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff91000bbc0 tx=0x7ff9100107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf95069de3a3c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.542+0000 7ff90cff9640 1 -- 172.21.15.33:0/1934386371 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910004030 con 0x55b360c1df90 2024-04-24T23:37:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.542+0000 7ff90cff9640 1 -- 172.21.15.33:0/1934386371 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9100041d0 con 0x55b360c1df90 2024-04-24T23:37:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.542+0000 7ff90cff9640 1 -- 172.21.15.33:0/1934386371 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100116a0 con 0x55b360c1df90 2024-04-24T23:37:37.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.543+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9080a2c50 0x7ff9080a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.543+0000 7ff9277fd740 1 -- 172.21.15.33:0/1934386371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c1df90 msgr2=0x55b360c20420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.543+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c1df90 0x55b360c20420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff91000bbc0 tx=0x7ff9100107a0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.544+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9080a2c50 0x7ff9080a5110 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55b360c19680 tx=0x7ff91800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 -- 172.21.15.33:0/1934386371 shutdown_connections 2024-04-24T23:37:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c1df90 0x55b360c20420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c1d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1934386371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18430 0x55b360c18850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 -- 172.21.15.33:0/1934386371 >> 172.21.15.33:0/1934386371 conn(0x55b360c13aa0 msgr2=0x55b360c13ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 -- 172.21.15.33:0/1934386371 shutdown_connections 2024-04-24T23:37:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.545+0000 7ff9277fd740 1 -- 172.21.15.33:0/1934386371 wait complete. 2024-04-24T23:37:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.546+0000 7ff9277fd740 1 -- start start 2024-04-24T23:37:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.546+0000 7ff9277fd740 1 Processor -- start 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c22940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c22e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c1df90 0x55b360c29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b360c20f40 con 0x55b360c1df90 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b360b37f50 con 0x55b360c18430 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b360c210c0 con 0x55b360c18e20 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff9277fd740 10 client.0 _ll_register_callbacks cb 0x55b360b73000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c22940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c22940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39082/0 (socket says 172.21.15.33:39082) 2024-04-24T23:37:37.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 -- 172.21.15.33:0/3335220045 learned_addr learned my addr 172.21.15.33:0/3335220045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 -- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 msgr2=0x55b360c22e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:37:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c22e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 -- 172.21.15.33:0/3335220045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c1df90 msgr2=0x55b360c29ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c1df90 0x55b360c29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.547+0000 7ff91e59c640 1 -- 172.21.15.33:0/3335220045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3608903f0 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff91e59c640 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c22940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9140b8a00 tx=0x7ff9140ba2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947588458eb7586a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff905ffb640 1 -- 172.21.15.33:0/3335220045 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914049030 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff905ffb640 1 -- 172.21.15.33:0/3335220045 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9140c0120 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9100041d0 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff905ffb640 1 -- 172.21.15.33:0/3335220045 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff9140d50c0 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff9277fd740 10 client.30086 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b3608500f0 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.548+0000 7ff905ffb640 1 -- 172.21.15.33:0/3335220045 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9100041d0 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff905ffb640 1 -- 172.21.15.33:0/3335220045 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ff91403bce0 con 0x55b360c18430 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff9277fd740 10 client.30086 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff9277fd740 10 client.30086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff9277fd740 20 client.30086 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c2e740 0x55b360c30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff9277fd740 4 client.30086 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.549+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b360b89bf0 con 0x55b360c2e740 2024-04-24T23:37:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.550+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c2e740 0x55b360c30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.551+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c2e740 0x55b360c30c00 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad77f706882bb63 server_cookie=b751875a35d738e4 in_seq=0 out_seq=0 2024-04-24T23:37:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.551+0000 7ff905ffb640 10 client.30086 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff905ffb640 1 -- 172.21.15.33:0/3335220045 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b360b89bf0 con 0x55b360c2e740 2024-04-24T23:37:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff905ffb640 10 client.30086 handle_command_reply: tid=0 2024-04-24T23:37:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 client.30086 shutdown 2024-04-24T23:37:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 msgr2=0x55b360c22940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c22940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9140b8a00 tx=0x7ff9140ba2c0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 shutdown_connections 2024-04-24T23:37:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c2e740 0x55b360c30c00 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c1df90 0x55b360c29ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c22e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3335220045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c22940 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 >> 172.21.15.33:0/3335220045 conn(0x55b360c13aa0 msgr2=0x55b360c12d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff905ffb640 0 client.30086 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 shutdown_connections 2024-04-24T23:37:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.552+0000 7ff9277fd740 1 -- 172.21.15.33:0/3335220045 wait complete. 2024-04-24T23:37:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.553+0000 7ff9277fd740 20 client.30086 trim_cache size 0 max 16384 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 Processor -- start 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 -- start start 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c13aa0 0x55b360c22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c16b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c17070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c13aa0 0x55b360c22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff900012540 con 0x55b360c13aa0 2024-04-24T23:37:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000fde0 con 0x55b360c18430 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c13aa0 0x55b360c22c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35072/0 (socket says 172.21.15.33:35072) 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff9277fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000dc30 con 0x55b360c18e20 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff91e59c640 1 -- 172.21.15.33:0/3540589884 learned_addr learned my addr 172.21.15.33:0/3540589884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.589+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c17070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c16b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91ed9d640 1 -- 172.21.15.33:0/3540589884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 msgr2=0x55b360c16b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c16b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91ed9d640 1 -- 172.21.15.33:0/3540589884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c13aa0 msgr2=0x55b360c22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c13aa0 0x55b360c22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91ed9d640 1 -- 172.21.15.33:0/3540589884 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b360c14810 con 0x55b360c18e20 2024-04-24T23:37:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.590+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c17070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9a0 tx=0x7ff918004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3d72d95b8a470e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.591+0000 7ff907fff640 1 -- 172.21.15.33:0/3540589884 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91801b040 con 0x55b360c18e20 2024-04-24T23:37:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.591+0000 7ff907fff640 1 -- 172.21.15.33:0/3540589884 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff918024070 con 0x55b360c18e20 2024-04-24T23:37:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.591+0000 7ff907fff640 1 -- 172.21.15.33:0/3540589884 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918013880 con 0x55b360c18e20 2024-04-24T23:37:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.592+0000 7ff9277fd740 1 -- 172.21.15.33:0/3540589884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 msgr2=0x55b360c17070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.592+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c17070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9a0 tx=0x7ff918004b50 comp rx=0 tx=0).stop 2024-04-24T23:37:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.593+0000 7ff9277fd740 1 -- 172.21.15.33:0/3540589884 shutdown_connections 2024-04-24T23:37:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.593+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18e20 0x55b360c17070 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.593+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c18430 0x55b360c16b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.593+0000 7ff9277fd740 1 --2- 172.21.15.33:0/3540589884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c13aa0 0x55b360c22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.593+0000 7ff9277fd740 1 -- 172.21.15.33:0/3540589884 >> 172.21.15.33:0/3540589884 conn(0x55b360c0ef00 msgr2=0x55b360c22550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.594+0000 7ff9277fd740 1 -- 172.21.15.33:0/3540589884 shutdown_connections 2024-04-24T23:37:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.594+0000 7ff9277fd740 1 -- 172.21.15.33:0/3540589884 wait complete. 2024-04-24T23:37:37.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.594+0000 7ff9277fd740 1 -- start start 2024-04-24T23:37:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.594+0000 7ff9277fd740 1 Processor -- start 2024-04-24T23:37:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c13aa0 0x55b360c26e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18430 0x55b360c27340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18e20 0x55b360c28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000e2a0 con 0x55b360c18e20 2024-04-24T23:37:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000f710 con 0x55b360c13aa0 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000e970 con 0x55b360c18430 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff9277fd740 10 client.0 _ll_register_callbacks cb 0x7ff914005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c13aa0 0x55b360c26e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff91e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c13aa0 0x55b360c26e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39094/0 (socket says 172.21.15.33:39094) 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff91e59c640 1 -- 172.21.15.33:0/1131001023 learned_addr learned my addr 172.21.15.33:0/1131001023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff91dd9b640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18430 0x55b360c27340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.595+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18e20 0x55b360c28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff91ed9d640 1 -- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18430 msgr2=0x55b360c27340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18430 0x55b360c27340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff91ed9d640 1 -- 172.21.15.33:0/1131001023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c13aa0 msgr2=0x55b360c26e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c13aa0 0x55b360c26e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff91ed9d640 1 -- 172.21.15.33:0/1131001023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b360c28940 con 0x55b360c18e20 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff91ed9d640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18e20 0x55b360c28350 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff918015c50 tx=0x7ff91801fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69254b137a34623c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.596+0000 7ff905ffb640 1 -- 172.21.15.33:0/1131001023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918013800 con 0x55b360c18e20 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.597+0000 7ff905ffb640 1 -- 172.21.15.33:0/1131001023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9180139a0 con 0x55b360c18e20 2024-04-24T23:37:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.597+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff918024070 con 0x55b360c18e20 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.597+0000 7ff905ffb640 1 -- 172.21.15.33:0/1131001023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff918037050 con 0x55b360c18e20 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.597+0000 7ff9277fd740 10 client.20631 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.597+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b360c95080 con 0x55b360c18e20 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.597+0000 7ff905ffb640 1 -- 172.21.15.33:0/1131001023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff918024070 con 0x55b360c18e20 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff905ffb640 1 -- 172.21.15.33:0/1131001023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ff91801e030 con 0x55b360c18e20 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff9277fd740 10 client.20631 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff9277fd740 10 client.20631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff9277fd740 20 client.20631 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c1df90 0x55b360c97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff9277fd740 4 client.20631 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:37:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.598+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b360b83f40 con 0x55b360c1df90 2024-04-24T23:37:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.601+0000 7ff91e59c640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c1df90 0x55b360c97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.601+0000 7ff91e59c640 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c1df90 0x55b360c97a10 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827d66d863e293d4 server_cookie=43ec4f6c0ef4857d in_seq=0 out_seq=0 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.602+0000 7ff905ffb640 10 client.20631 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff905ffb640 1 -- 172.21.15.33:0/1131001023 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b360b83f40 con 0x55b360c1df90 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff905ffb640 10 client.20631 handle_command_reply: tid=0 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 client.20631 shutdown 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18e20 msgr2=0x55b360c28350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18e20 0x55b360c28350 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff918015c50 tx=0x7ff91801fb00 comp rx=0 tx=0).stop 2024-04-24T23:37:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 shutdown_connections 2024-04-24T23:37:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b360c1df90 0x55b360c97a10 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b360c18e20 0x55b360c28350 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b360c18430 0x55b360c27340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 --2- 172.21.15.33:0/1131001023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b360c13aa0 0x55b360c26e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 >> 172.21.15.33:0/1131001023 conn(0x55b360c0ef00 msgr2=0x55b360c214f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.604+0000 7ff905ffb640 0 client.20631 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.605+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 shutdown_connections 2024-04-24T23:37:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.605+0000 7ff9277fd740 1 -- 172.21.15.33:0/1131001023 wait complete. 2024-04-24T23:37:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.605+0000 7ff9277fd740 20 client.20631 trim_cache size 0 max 16384 2024-04-24T23:37:37.608 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:37:37.608 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:37:37.608 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:37:37.608 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:37:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.606+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9080a2c50 msgr2=0x7ff9080a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.606+0000 7ff91f59e640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9080a2c50 0x7ff9080a5110 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55b360c19680 tx=0x7ff91800b040 comp rx=0 tx=0).stop 2024-04-24T23:37:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.606+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 msgr2=0x7ff92019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.606+0000 7ff91f59e640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff92019df10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff91402aa10 tx=0x7ff914004610 comp rx=0 tx=0).stop 2024-04-24T23:37:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.609+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 shutdown_connections 2024-04-24T23:37:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.609+0000 7ff91f59e640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7ff9080a2c50 0x7ff9080a5110 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.609+0000 7ff91f59e640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9201051e0 0x7ff9201a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.609+0000 7ff91f59e640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff920102830 0x7ff92019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.609+0000 7ff91f59e640 1 --2- 172.21.15.33:0/1114556234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9200ffec0 0x7ff92019df10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.609+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 >> 172.21.15.33:0/1114556234 conn(0x7ff9200fbae0 msgr2=0x7ff920103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.610+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 shutdown_connections 2024-04-24T23:37:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.610+0000 7ff91f59e640 1 -- 172.21.15.33:0/1114556234 wait complete. 2024-04-24T23:37:37.628 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:37:37.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:37:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 -- 172.21.15.33:0/113246387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c102250 msgr2=0x7f327c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 --2- 172.21.15.33:0/113246387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c102250 0x7f327c108ae0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f3274009ca0 tx=0x7f327402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 -- 172.21.15.33:0/113246387 shutdown_connections 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 --2- 172.21.15.33:0/113246387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c102250 0x7f327c108ae0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 --2- 172.21.15.33:0/113246387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f327c101890 0x7f327c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 --2- 172.21.15.33:0/113246387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f327c1006a0 0x7f327c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 -- 172.21.15.33:0/113246387 >> 172.21.15.33:0/113246387 conn(0x7f327c0fbc30 msgr2=0x7f327c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 -- 172.21.15.33:0/113246387 shutdown_connections 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 -- 172.21.15.33:0/113246387 wait complete. 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 Processor -- start 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.771+0000 7f328264d640 1 -- start start 2024-04-24T23:37:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f328264d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f327c1006a0 0x7f327c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f328264d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 0x7f327c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f328264d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f327c102250 0x7f327c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f328264d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10c320 con 0x7f327c101890 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f328264d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10c1a0 con 0x7f327c1006a0 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f328264d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10c4a0 con 0x7f327c102250 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 0x7f327c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 0x7f327c19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35098/0 (socket says 172.21.15.33:35098) 2024-04-24T23:37:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 -- 172.21.15.33:0/2583264970 learned_addr learned my addr 172.21.15.33:0/2583264970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f3280bea640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f327c102250 0x7f327c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327bfff640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f327c1006a0 0x7f327c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 -- 172.21.15.33:0/2583264970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f327c102250 msgr2=0x7f327c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f327c102250 0x7f327c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 -- 172.21.15.33:0/2583264970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f327c1006a0 msgr2=0x7f327c06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f327c1006a0 0x7f327c06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 -- 172.21.15.33:0/2583264970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c19f690 con 0x7f327c101890 2024-04-24T23:37:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f327b7fe640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 0x7f327c19c790 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f327000e9f0 tx=0x7f327000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.772+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327000cd80 con 0x7f327c101890 2024-04-24T23:37:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.773+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c19f980 con 0x7f327c101890 2024-04-24T23:37:37.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.773+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c19fed0 con 0x7f327c101890 2024-04-24T23:37:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.773+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3270005500 con 0x7f327c101890 2024-04-24T23:37:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.773+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270010690 con 0x7f327c101890 2024-04-24T23:37:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.774+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c107a40 con 0x7f327c101890 2024-04-24T23:37:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.775+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f32700046c0 con 0x7f327c101890 2024-04-24T23:37:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.775+0000 7f32797fa640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32640a2b90 0x7f32640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.776+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f32700cf750 con 0x7f327c101890 2024-04-24T23:37:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.776+0000 7f327bfff640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32640a2b90 0x7f32640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.776+0000 7f327bfff640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32640a2b90 0x7f32640a5050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f327c1016f0 tx=0x7f326c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:37.781+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f327009c320 con 0x7f327c101890 2024-04-24T23:37:37.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 289171 21.74 MB/sec execute 582 sec latency 79.732 ms 2024-04-24T23:37:37.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:37 smithi033 ceph-mon[29201]: pgmap v1119: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.2 MiB/s wr, 387 op/s 2024-04-24T23:37:38.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.055+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f327c062340 con 0x7f327c101890 2024-04-24T23:37:38.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.056+0000 7f32797fa640 1 -- 172.21.15.33:0/2583264970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11861 (secure 0 0 0) 0x7f3270086020 con 0x7f327c101890 2024-04-24T23:37:38.055 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:37:38.055 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:35:37.265197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:37:38.055 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32640a2b90 msgr2=0x7f32640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32640a2b90 0x7f32640a5050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f327c1016f0 tx=0x7f326c005f90 comp rx=0 tx=0).stop 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 msgr2=0x7f327c19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 0x7f327c19c790 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f327000e9f0 tx=0x7f327000edf0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 shutdown_connections 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f32640a2b90 0x7f32640a5050 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f327c102250 0x7f327c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f327c101890 0x7f327c19c790 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 --2- 172.21.15.33:0/2583264970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f327c1006a0 0x7f327c06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 >> 172.21.15.33:0/2583264970 conn(0x7f327c0fbc30 msgr2=0x7f327c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 shutdown_connections 2024-04-24T23:37:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.058+0000 7f328264d640 1 -- 172.21.15.33:0/2583264970 wait complete. 2024-04-24T23:37:38.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:37:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:37 smithi038 ceph-mon[32450]: pgmap v1119: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.2 MiB/s wr, 387 op/s 2024-04-24T23:37:38.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:37 smithi117 ceph-mon[32529]: pgmap v1119: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.2 MiB/s wr, 387 op/s 2024-04-24T23:37:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 -- 172.21.15.33:0/106910009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0104b30 msgr2=0x7f86b0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 --2- 172.21.15.33:0/106910009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0104b30 0x7f86b0106f20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009900 tx=0x7f86a002a410 comp rx=0 tx=0).stop 2024-04-24T23:37:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 -- 172.21.15.33:0/106910009 shutdown_connections 2024-04-24T23:37:38.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 --2- 172.21.15.33:0/106910009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86b0107460 0x7f86b01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 --2- 172.21.15.33:0/106910009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0104b30 0x7f86b0106f20 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 --2- 172.21.15.33:0/106910009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0102200 0x7f86b01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 -- 172.21.15.33:0/106910009 >> 172.21.15.33:0/106910009 conn(0x7f86b00fbbf0 msgr2=0x7f86b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 -- 172.21.15.33:0/106910009 shutdown_connections 2024-04-24T23:37:38.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.193+0000 7f86b7b09640 1 -- 172.21.15.33:0/106910009 wait complete. 2024-04-24T23:37:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b7b09640 1 Processor -- start 2024-04-24T23:37:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b7b09640 1 -- start start 2024-04-24T23:37:38.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b7b09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 0x7f86b0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b7b09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0104b30 0x7f86b0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b7b09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86b0107460 0x7f86b019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b7b09640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010d440 con 0x7f86b0102200 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.194+0000 7f86b58a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 0x7f86b0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 0x7f86b0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35110/0 (socket says 172.21.15.33:35110) 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 -- 172.21.15.33:0/2186766050 learned_addr learned my addr 172.21.15.33:0/2186766050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010d2c0 con 0x7f86b0107460 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010d5c0 con 0x7f86b0104b30 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86b0107460 0x7f86b019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 -- 172.21.15.33:0/2186766050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0104b30 msgr2=0x7f86b0198050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0104b30 0x7f86b0198050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 -- 172.21.15.33:0/2186766050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86b0107460 msgr2=0x7f86b019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86b0107460 0x7f86b019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.195+0000 7f86b58a5640 1 -- 172.21.15.33:0/2186766050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b01000f0 con 0x7f86b0102200 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.196+0000 7f86b58a5640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 0x7f86b0197b10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009fd0 tx=0x7f86ac00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.197+0000 7f86a6ffd640 1 -- 172.21.15.33:0/2186766050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00cd80 con 0x7f86b0102200 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.197+0000 7f86a6ffd640 1 -- 172.21.15.33:0/2186766050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac018900 con 0x7f86b0102200 2024-04-24T23:37:38.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.197+0000 7f86a6ffd640 1 -- 172.21.15.33:0/2186766050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac010a40 con 0x7f86b0102200 2024-04-24T23:37:38.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.198+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b01003e0 con 0x7f86b0102200 2024-04-24T23:37:38.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.198+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01008e0 con 0x7f86b0102200 2024-04-24T23:37:38.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.201+0000 7f86ba80f740 1 Processor -- start 2024-04-24T23:37:38.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86a6ffd640 1 -- 172.21.15.33:0/2186766050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f86ac018ab0 con 0x7f86b0102200 2024-04-24T23:37:38.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 -- start start 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 0x5623a708a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a708f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708fd90 0x5623a7092220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5623a6fee670 con 0x5623a708a230 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5623a6f96970 con 0x5623a708fd90 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.202+0000 7f86ba80f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5623a6fea630 con 0x5623a708ac20 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 0x5623a708a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 0x5623a708a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35120/0 (socket says 172.21.15.33:35120) 2024-04-24T23:37:38.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b58a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a708f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 -- 172.21.15.33:0/2660083489 learned_addr learned my addr 172.21.15.33:0/2660083489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b50a4640 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708fd90 0x5623a7092220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 -- 172.21.15.33:0/2660083489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 msgr2=0x5623a708f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a708f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 -- 172.21.15.33:0/2660083489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708fd90 msgr2=0x5623a7092220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708fd90 0x5623a7092220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 -- 172.21.15.33:0/2660083489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a6fba400 con 0x5623a708a230 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86a6ffd640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f869c0a2d90 0x7f869c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.203+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 0x5623a708a650 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f86a8010370 tx=0x7f86a8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6714a3815bcd41b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.204+0000 7f86a6ffd640 1 -- 172.21.15.33:0/2186766050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f86ac0e0840 con 0x7f86b0102200 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.204+0000 7f86b58a5640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f869c0a2d90 0x7f869c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.204+0000 7f869bfff640 1 -- 172.21.15.33:0/2660083489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8016070 con 0x5623a708a230 2024-04-24T23:37:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.204+0000 7f869bfff640 1 -- 172.21.15.33:0/2660083489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a8004d20 con 0x5623a708a230 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.204+0000 7f869bfff640 1 -- 172.21.15.33:0/2660083489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8005480 con 0x5623a708a230 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 -- 172.21.15.33:0/2660083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 msgr2=0x5623a708a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 0x5623a708a650 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f86a8010370 tx=0x7f86a8010830 comp rx=0 tx=0).stop 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86b58a5640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f869c0a2d90 0x7f869c0a5250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0acc40 tx=0x7f86ac0e6530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 -- 172.21.15.33:0/2660083489 shutdown_connections 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708fd90 0x5623a7092220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a708f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2660083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708a230 0x5623a708a650 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 -- 172.21.15.33:0/2660083489 >> 172.21.15.33:0/2660083489 conn(0x5623a70858a0 msgr2=0x5623a7085ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 -- 172.21.15.33:0/2660083489 shutdown_connections 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 -- 172.21.15.33:0/2660083489 wait complete. 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.205+0000 7f86ba80f740 1 -- start start 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 Processor -- start 2024-04-24T23:37:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a7094350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a7094890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a709b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5623a7092d90 con 0x5623a708fd90 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5623a6faa050 con 0x5623a708a230 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5623a7092f10 con 0x5623a708ac20 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86ba80f740 10 client.0 _ll_register_callbacks cb 0x5623a6d01be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a7094350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a7094350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39168/0 (socket says 172.21.15.33:39168) 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 -- 172.21.15.33:0/1083871847 learned_addr learned my addr 172.21.15.33:0/1083871847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a709b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b50a4640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a7094890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 -- 172.21.15.33:0/1083871847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 msgr2=0x5623a7094890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a7094890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 -- 172.21.15.33:0/1083871847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 msgr2=0x5623a709b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a709b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.206+0000 7f86b60a6640 1 -- 172.21.15.33:0/1083871847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a6f7bdc0 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a7094350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86a80055f0 tx=0x7f86a8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d14d92d932bbdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86997fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86997fa640 1 -- 172.21.15.33:0/1083871847 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8016070 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86997fa640 1 -- 172.21.15.33:0/1083871847 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f86a8005850 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a8004d20 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86997fa640 1 -- 172.21.15.33:0/1083871847 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86a8023040 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 10 client.30110 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5623a6d023f0 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86997fa640 1 -- 172.21.15.33:0/1083871847 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a8004d20 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86997fa640 1 -- 172.21.15.33:0/1083871847 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f86a800f040 con 0x5623a708a230 2024-04-24T23:37:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 10 client.30110 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 10 client.30110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:37:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 20 client.30110 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a709fee0 0x5623a70a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 4 client.30110 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.207+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623a6ffbbb0 con 0x5623a709fee0 2024-04-24T23:37:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.208+0000 7f86b50a4640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a709fee0 0x5623a70a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.208+0000 7f86997fa640 10 client.30110 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.208+0000 7f86b50a4640 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a709fee0 0x5623a70a22f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29347a3143b312ba server_cookie=bb6374d30f8a8b65 in_seq=0 out_seq=0 2024-04-24T23:37:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86997fa640 1 -- 172.21.15.33:0/1083871847 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623a6ffbbb0 con 0x5623a709fee0 2024-04-24T23:37:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86997fa640 10 client.30110 handle_command_reply: tid=0 2024-04-24T23:37:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 client.30110 shutdown 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 msgr2=0x5623a7094350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a7094350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86a80055f0 tx=0x7f86a8009840 comp rx=0 tx=0).stop 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 shutdown_connections 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a709fee0 0x5623a70a22f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a709b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708ac20 0x5623a7094890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1083871847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a7094350 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 >> 172.21.15.33:0/1083871847 conn(0x5623a70858a0 msgr2=0x5623a7084b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86997fa640 0 client.30110 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 shutdown_connections 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 1 -- 172.21.15.33:0/1083871847 wait complete. 2024-04-24T23:37:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.210+0000 7f86ba80f740 20 client.30110 trim_cache size 0 max 16384 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 Processor -- start 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 -- start start 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708a230 0x5623a70842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 0x5623a709b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a70847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01aabc0 con 0x5623a708fd90 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01a8460 con 0x5623a708ac20 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.224+0000 7f86ba80f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01a62b0 con 0x5623a708a230 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708a230 0x5623a70842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b50a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 0x5623a709b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a70847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b50a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 0x5623a709b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39172/0 (socket says 172.21.15.33:39172) 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b50a4640 1 -- 172.21.15.33:0/1415943593 learned_addr learned my addr 172.21.15.33:0/1415943593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 -- 172.21.15.33:0/1415943593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708a230 msgr2=0x5623a70842b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708a230 0x5623a70842b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 -- 172.21.15.33:0/1415943593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 msgr2=0x5623a709b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 0x5623a709b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 -- 172.21.15.33:0/1415943593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a6dcc7d0 con 0x5623a708fd90 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b50a4640 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 0x5623a709b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f86b60a6640 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a70847f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f86a802a4d0 tx=0x7f86a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5550f6191759dce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f869b7fe640 1 -- 172.21.15.33:0/1415943593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8016070 con 0x5623a708fd90 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f869b7fe640 1 -- 172.21.15.33:0/1415943593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a8005b00 con 0x5623a708fd90 2024-04-24T23:37:38.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.225+0000 7f869b7fe640 1 -- 172.21.15.33:0/1415943593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8030040 con 0x5623a708fd90 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 -- 172.21.15.33:0/1415943593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 msgr2=0x5623a70847f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a70847f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f86a802a4d0 tx=0x7f86a8005fd0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 -- 172.21.15.33:0/1415943593 shutdown_connections 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708fd90 0x5623a70847f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708ac20 0x5623a709b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 --2- 172.21.15.33:0/1415943593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708a230 0x5623a70842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 -- 172.21.15.33:0/1415943593 >> 172.21.15.33:0/1415943593 conn(0x5623a70858a0 msgr2=0x5623a708bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 -- 172.21.15.33:0/1415943593 shutdown_connections 2024-04-24T23:37:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 -- 172.21.15.33:0/1415943593 wait complete. 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.226+0000 7f86ba80f740 1 -- start start 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 Processor -- start 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a709b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 0x5623a7097ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708fd90 0x5623a7098530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01a6920 con 0x5623a708ac20 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01a7d90 con 0x5623a708a230 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01a6ff0 con 0x5623a708fd90 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86ba80f740 10 client.0 _ll_register_callbacks cb 0x7f86ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 0x5623a7097ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 0x5623a7097ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35156/0 (socket says 172.21.15.33:35156) 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 -- 172.21.15.33:0/2716837543 learned_addr learned my addr 172.21.15.33:0/2716837543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708fd90 0x5623a7098530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a709b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 -- 172.21.15.33:0/2716837543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708fd90 msgr2=0x5623a7098530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708fd90 0x5623a7098530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 -- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 msgr2=0x5623a709b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a709b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b50a4640 1 -- 172.21.15.33:0/2716837543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a7098b20 con 0x5623a708ac20 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.227+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708fd90 0x5623a7098530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86b50a4640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 0x5623a7097ff0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009890 tx=0x7f86a0004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cf9a9742204558 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86997fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86997fa640 1 -- 172.21.15.33:0/2716837543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0035040 con 0x5623a708ac20 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86997fa640 1 -- 172.21.15.33:0/2716837543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f86a0045dc0 con 0x5623a708ac20 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a8005b00 con 0x5623a708ac20 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86997fa640 1 -- 172.21.15.33:0/2716837543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86a0053050 con 0x5623a708ac20 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 10 client.20655 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5623a7105570 con 0x5623a708ac20 2024-04-24T23:37:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86997fa640 1 -- 172.21.15.33:0/2716837543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a8005b00 con 0x5623a708ac20 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86997fa640 1 -- 172.21.15.33:0/2716837543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f86a00046c0 con 0x5623a708ac20 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 10 client.20655 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 10 client.20655 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 20 client.20655 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a7105fd0 0x5623a71083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 4 client.20655 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.228+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623a708fb00 con 0x5623a7105fd0 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.229+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a7105fd0 0x5623a71083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.229+0000 7f86b60a6640 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a7105fd0 0x5623a71083f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e855469acbd1603 server_cookie=9d16fb18ab24cce8 in_seq=0 out_seq=0 2024-04-24T23:37:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.229+0000 7f86997fa640 10 client.20655 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:38.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.232+0000 7f86997fa640 1 -- 172.21.15.33:0/2716837543 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623a708fb00 con 0x5623a7105fd0 2024-04-24T23:37:38.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.232+0000 7f86997fa640 10 client.20655 handle_command_reply: tid=0 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.232+0000 7f86ba80f740 1 client.20655 shutdown 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.232+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 msgr2=0x5623a7097ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.232+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 0x5623a7097ff0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009890 tx=0x7f86a0004110 comp rx=0 tx=0).stop 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 shutdown_connections 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5623a7105fd0 0x5623a71083f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623a708fd90 0x5623a7098530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623a708ac20 0x5623a7097ff0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 --2- 172.21.15.33:0/2716837543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623a708a230 0x5623a709b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86997fa640 0 client.20655 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 >> 172.21.15.33:0/2716837543 conn(0x5623a70858a0 msgr2=0x5623a7088810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 shutdown_connections 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 1 -- 172.21.15.33:0/2716837543 wait complete. 2024-04-24T23:37:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86ba80f740 20 client.20655 trim_cache size 0 max 16384 2024-04-24T23:37:38.236 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:37:38.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f869c0a2d90 msgr2=0x7f869c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f869c0a2d90 0x7f869c0a5250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0acc40 tx=0x7f86ac0e6530 comp rx=0 tx=0).stop 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 msgr2=0x7f86b0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 0x7f86b0197b10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009fd0 tx=0x7f86ac00edd0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 shutdown_connections 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f869c0a2d90 0x7f869c0a5250 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86b0107460 0x7f86b019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0104b30 0x7f86b0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 --2- 172.21.15.33:0/2186766050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b0102200 0x7f86b0197b10 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 >> 172.21.15.33:0/2186766050 conn(0x7f86b00fbbf0 msgr2=0x7f86b0102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 shutdown_connections 2024-04-24T23:37:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.233+0000 7f86b7b09640 1 -- 172.21.15.33:0/2186766050 wait complete. 2024-04-24T23:37:38.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:37:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.366+0000 7f476f7fe640 1 -- 172.21.15.33:0/3906882182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600319b0 con 0x7f47701006a0 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 -- 172.21.15.33:0/3906882182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47701006a0 msgr2=0x7f4770100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 --2- 172.21.15.33:0/3906882182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47701006a0 0x7f4770100ac0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4760009900 tx=0x7f476002a410 comp rx=0 tx=0).stop 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 -- 172.21.15.33:0/3906882182 shutdown_connections 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 --2- 172.21.15.33:0/3906882182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4770102250 0x7f4770108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 --2- 172.21.15.33:0/3906882182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4770101890 0x7f4770101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 --2- 172.21.15.33:0/3906882182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47701006a0 0x7f4770100ac0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 -- 172.21.15.33:0/3906882182 >> 172.21.15.33:0/3906882182 conn(0x7f47700fbc30 msgr2=0x7f47700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 -- 172.21.15.33:0/3906882182 shutdown_connections 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 -- 172.21.15.33:0/3906882182 wait complete. 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.367+0000 7f4776b63640 1 Processor -- start 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 -- start start 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47701006a0 0x7f477006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4770101890 0x7f477019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 0x7f477019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f477010c310 con 0x7f4770102250 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f477010c190 con 0x7f4770101890 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4776b63640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f477010c490 con 0x7f47701006a0 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 0x7f477019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 0x7f477019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35182/0 (socket says 172.21.15.33:35182) 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 -- 172.21.15.33:0/1280943725 learned_addr learned my addr 172.21.15.33:0/1280943725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f476ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4770101890 0x7f477019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f47748ff640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47701006a0 0x7f477006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 -- 172.21.15.33:0/1280943725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47701006a0 msgr2=0x7f477006a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47701006a0 0x7f477006a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 -- 172.21.15.33:0/1280943725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4770101890 msgr2=0x7f477019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4770101890 0x7f477019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 -- 172.21.15.33:0/1280943725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477019f700 con 0x7f4770102250 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.368+0000 7f4775100640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 0x7f477019ee40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f476800eb10 tx=0x7f476800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.369+0000 7f476dffb640 1 -- 172.21.15.33:0/1280943725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768004b20 con 0x7f4770102250 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.369+0000 7f476dffb640 1 -- 172.21.15.33:0/1280943725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768004cc0 con 0x7f4770102250 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.369+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477019f9f0 con 0x7f4770102250 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.369+0000 7f476dffb640 1 -- 172.21.15.33:0/1280943725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768010760 con 0x7f4770102250 2024-04-24T23:37:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.369+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477019fef0 con 0x7f4770102250 2024-04-24T23:37:38.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.371+0000 7f476dffb640 1 -- 172.21.15.33:0/1280943725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4768005020 con 0x7f4770102250 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 Processor -- start 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f476dffb640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f475c0a2b90 0x7f475c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 -- start start 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f476dffb640 1 -- 172.21.15.33:0/1280943725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47680cf980 con 0x7f4770102250 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f476ffff640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f475c0a2b90 0x7f475c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f695650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f69a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f69d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56532f5f9670 con 0x56532f69ad90 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56532f5a1970 con 0x56532f695230 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f477987c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56532f5f5630 con 0x56532f695c20 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f47748ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f695650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f476ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f69a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f69d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f47748ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f695650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39212/0 (socket says 172.21.15.33:39212) 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f476ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f69a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51678/0 (socket says 172.21.15.33:51678) 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.372+0000 7f47748ff640 1 -- 172.21.15.33:0/2664698609 learned_addr learned my addr 172.21.15.33:0/2664698609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4775100640 1 -- 172.21.15.33:0/2664698609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 msgr2=0x56532f69a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4775100640 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f69a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4775100640 1 -- 172.21.15.33:0/2664698609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 msgr2=0x56532f695650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4775100640 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f695650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f476ffff640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f475c0a2b90 0x7f475c0a5050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f477006a680 tx=0x7f4758011920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4775100640 1 -- 172.21.15.33:0/2664698609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532f5c5400 con 0x56532f69ad90 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f47748ff640 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f695650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4775100640 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f69d220 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f47680cd140 tx=0x7f476809bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6221994d8f2aa5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4766ffd640 1 -- 172.21.15.33:0/2664698609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476809be50 con 0x56532f69ad90 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4766ffd640 1 -- 172.21.15.33:0/2664698609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476809e070 con 0x56532f69ad90 2024-04-24T23:37:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.373+0000 7f4766ffd640 1 -- 172.21.15.33:0/2664698609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680a4890 con 0x56532f69ad90 2024-04-24T23:37:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 -- 172.21.15.33:0/2664698609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 msgr2=0x56532f69d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f69d220 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f47680cd140 tx=0x7f476809bce0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 -- 172.21.15.33:0/2664698609 shutdown_connections 2024-04-24T23:37:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f69d220 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f69a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 --2- 172.21.15.33:0/2664698609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f695650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 -- 172.21.15.33:0/2664698609 >> 172.21.15.33:0/2664698609 conn(0x56532f6908a0 msgr2=0x56532f690ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 -- 172.21.15.33:0/2664698609 shutdown_connections 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 -- 172.21.15.33:0/2664698609 wait complete. 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 -- start start 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.374+0000 7f477987c740 1 Processor -- start 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695230 0x56532f69f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695c20 0x56532f69f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f6a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56532f69dd40 con 0x56532f69ad90 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56532f5b5050 con 0x56532f695c20 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56532f69dec0 con 0x56532f695230 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 10 client.0 _ll_register_callbacks cb 0x56532f30cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f6a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f476ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695c20 0x56532f69f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f6a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35202/0 (socket says 172.21.15.33:35202) 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 -- 172.21.15.33:0/1062770814 learned_addr learned my addr 172.21.15.33:0/1062770814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f47748ff640 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695230 0x56532f69f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 -- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695230 msgr2=0x56532f69f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695230 0x56532f69f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 -- 172.21.15.33:0/1062770814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695c20 msgr2=0x56532f69f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695c20 0x56532f69f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 -- 172.21.15.33:0/1062770814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532f586dc0 con 0x56532f69ad90 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f4775100640 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f6a6880 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f476809bb50 tx=0x7f47680a49a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e81824bb734556 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f474ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f474ffff640 1 -- 172.21.15.33:0/1062770814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680b8070 con 0x56532f69ad90 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f474ffff640 1 -- 172.21.15.33:0/1062770814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f476809b2a0 con 0x56532f69ad90 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.375+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476809e070 con 0x56532f69ad90 2024-04-24T23:37:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f474ffff640 1 -- 172.21.15.33:0/1062770814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f476809a150 con 0x56532f69ad90 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 10 client.20667 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56532f30d3f0 con 0x56532f69ad90 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f474ffff640 1 -- 172.21.15.33:0/1062770814 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476809e070 con 0x56532f69ad90 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f474ffff640 1 -- 172.21.15.33:0/1062770814 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4768082030 con 0x56532f69ad90 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 10 client.20667 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 10 client.20667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 20 client.20667 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f6aadc0 0x56532f6ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 4 client.20667 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532f606bb0 con 0x56532f6aadc0 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.376+0000 7f47748ff640 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f6aadc0 0x56532f6ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f474ffff640 10 client.20667 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f47748ff640 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f6aadc0 0x56532f6ad1f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7c1d2baecdff872 server_cookie=6ec3b539aa5088fb in_seq=0 out_seq=0 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f474ffff640 1 -- 172.21.15.33:0/1062770814 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56532f606bb0 con 0x56532f6aadc0 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f474ffff640 10 client.20667 handle_command_reply: tid=0 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 client.20667 shutdown 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 msgr2=0x56532f6a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f6a6880 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f476809bb50 tx=0x7f47680a49a0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 shutdown_connections 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f6aadc0 0x56532f6ad1f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f6a6880 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695c20 0x56532f69f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f474ffff640 0 client.20667 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 --2- 172.21.15.33:0/1062770814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695230 0x56532f69f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.377+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 >> 172.21.15.33:0/1062770814 conn(0x56532f6908a0 msgr2=0x56532f68fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.378+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 shutdown_connections 2024-04-24T23:37:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.378+0000 7f477987c740 1 -- 172.21.15.33:0/1062770814 wait complete. 2024-04-24T23:37:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.378+0000 7f477987c740 20 client.20667 trim_cache size 0 max 16384 2024-04-24T23:37:38.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.398+0000 7f477987c740 1 Processor -- start 2024-04-24T23:37:38.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.398+0000 7f477987c740 1 -- start start 2024-04-24T23:37:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.398+0000 7f477987c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f69a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.398+0000 7f477987c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.398+0000 7f477987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f693810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f477987c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47701aeef0 con 0x56532f69ad90 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f477987c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47701ac790 con 0x56532f695230 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f477987c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47701aa5e0 con 0x56532f695c20 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f693810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f693810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35204/0 (socket says 172.21.15.33:35204) 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 -- 172.21.15.33:0/3861853781 learned_addr learned my addr 172.21.15.33:0/3861853781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f47748ff640 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f69a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 -- 172.21.15.33:0/3861853781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 msgr2=0x56532f6a6840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 -- 172.21.15.33:0/3861853781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 msgr2=0x56532f69a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f69a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.399+0000 7f4775100640 1 -- 172.21.15.33:0/3861853781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532f3d77d0 con 0x56532f69ad90 2024-04-24T23:37:38.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.400+0000 7f4775100640 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f693810 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f476809a100 tx=0x7f47680057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f54a4febf1d1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.400+0000 7f47667fc640 1 -- 172.21.15.33:0/3861853781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680b8070 con 0x56532f69ad90 2024-04-24T23:37:38.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.400+0000 7f47667fc640 1 -- 172.21.15.33:0/3861853781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476809e070 con 0x56532f69ad90 2024-04-24T23:37:38.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.400+0000 7f47667fc640 1 -- 172.21.15.33:0/3861853781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680a4ac0 con 0x56532f69ad90 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 -- 172.21.15.33:0/3861853781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 msgr2=0x56532f693810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f693810 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f476809a100 tx=0x7f47680057d0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 -- 172.21.15.33:0/3861853781 shutdown_connections 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f69ad90 0x56532f693810 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 --2- 172.21.15.33:0/3861853781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f695230 0x56532f69a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 -- 172.21.15.33:0/3861853781 >> 172.21.15.33:0/3861853781 conn(0x56532f6908a0 msgr2=0x56532f68fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 -- 172.21.15.33:0/3861853781 shutdown_connections 2024-04-24T23:37:38.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.401+0000 7f477987c740 1 -- 172.21.15.33:0/3861853781 wait complete. 2024-04-24T23:37:38.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.402+0000 7f477987c740 1 -- start start 2024-04-24T23:37:38.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.402+0000 7f477987c740 1 Processor -- start 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.402+0000 7f477987c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 0x56532f6a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f477987c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f47748ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 0x56532f6a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f47748ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 0x56532f6a40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35210/0 (socket says 172.21.15.33:35210) 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f47748ff640 1 -- 172.21.15.33:0/800854926 learned_addr learned my addr 172.21.15.33:0/800854926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f476ffff640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f69ad90 0x56532f6a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47701aac50 con 0x56532f695230 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47701ac0c0 con 0x56532f69ad90 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f4775100640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f69ad90 0x56532f6a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47701ab320 con 0x56532f695c20 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.403+0000 7f477987c740 10 client.0 _ll_register_callbacks cb 0x7f4758005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f47748ff640 1 -- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 msgr2=0x56532f6a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f47748ff640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f47748ff640 1 -- 172.21.15.33:0/800854926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f69ad90 msgr2=0x56532f6a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f47748ff640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f69ad90 0x56532f6a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f47748ff640 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532f6a5bf0 con 0x56532f695230 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f4775100640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f69ad90 0x56532f6a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f47748ff640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 0x56532f6a40b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4760005ab0 tx=0x7f47600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fb459e0d43be03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f474ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f474ffff640 1 -- 172.21.15.33:0/800854926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760035040 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f474ffff640 1 -- 172.21.15.33:0/800854926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47600040c0 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476809e070 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f474ffff640 1 -- 172.21.15.33:0/800854926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4760054050 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f477987c740 10 client.20673 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.404+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56532f711540 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f474ffff640 1 -- 172.21.15.33:0/800854926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476809e070 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f474ffff640 1 -- 172.21.15.33:0/800854926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4760046620 con 0x56532f695230 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f477987c740 10 client.20673 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f477987c740 10 client.20673 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f477987c740 20 client.20673 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f711ec0 0x56532f7122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f477987c740 4 client.20673 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56532f600f00 con 0x56532f711ec0 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.405+0000 7f476ffff640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f711ec0 0x56532f7122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f476ffff640 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f711ec0 0x56532f7122d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8418b1d4d251938 server_cookie=c33ea92300ad25e1 in_seq=0 out_seq=0 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f474ffff640 10 client.20673 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f474ffff640 1 -- 172.21.15.33:0/800854926 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56532f600f00 con 0x56532f711ec0 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f474ffff640 10 client.20673 handle_command_reply: tid=0 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 client.20673 shutdown 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 msgr2=0x56532f6a40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 0x56532f6a40b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4760005ab0 tx=0x7f47600046c0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 shutdown_connections 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x56532f711ec0 0x56532f7122d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56532f69ad90 0x56532f6a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56532f695c20 0x56532f6a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f474ffff640 0 client.20673 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 --2- 172.21.15.33:0/800854926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56532f695230 0x56532f6a40b0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 >> 172.21.15.33:0/800854926 conn(0x56532f6908a0 msgr2=0x56532f6966c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 shutdown_connections 2024-04-24T23:37:38.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 1 -- 172.21.15.33:0/800854926 wait complete. 2024-04-24T23:37:38.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.406+0000 7f477987c740 20 client.20673 trim_cache size 0 max 16384 2024-04-24T23:37:38.410 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f475c0a2b90 msgr2=0x7f475c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f475c0a2b90 0x7f475c0a5050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f477006a680 tx=0x7f4758011920 comp rx=0 tx=0).stop 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 msgr2=0x7f477019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 0x7f477019ee40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f476800eb10 tx=0x7f476800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 shutdown_connections 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f475c0a2b90 0x7f475c0a5050 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4770102250 0x7f477019ee40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4770101890 0x7f477019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 --2- 172.21.15.33:0/1280943725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47701006a0 0x7f477006a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 >> 172.21.15.33:0/1280943725 conn(0x7f47700fbc30 msgr2=0x7f4770104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 shutdown_connections 2024-04-24T23:37:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.407+0000 7f4776b63640 1 -- 172.21.15.33:0/1280943725 wait complete. 2024-04-24T23:37:38.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:37:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 -- 172.21.15.33:0/269444713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0108740 msgr2=0x7f69c010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 --2- 172.21.15.33:0/269444713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0108740 0x7f69c010ab70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b402a410 comp rx=0 tx=0).stop 2024-04-24T23:37:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 -- 172.21.15.33:0/269444713 shutdown_connections 2024-04-24T23:37:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 --2- 172.21.15.33:0/269444713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010da80 0x7f69c010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 --2- 172.21.15.33:0/269444713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c010b0b0 0x7f69c010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 --2- 172.21.15.33:0/269444713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0108740 0x7f69c010ab70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.553+0000 7f69c791b640 1 -- 172.21.15.33:0/269444713 >> 172.21.15.33:0/269444713 conn(0x7f69c006e5d0 msgr2=0x7f69c0070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.554+0000 7f69c791b640 1 -- 172.21.15.33:0/269444713 shutdown_connections 2024-04-24T23:37:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.554+0000 7f69c791b640 1 -- 172.21.15.33:0/269444713 wait complete. 2024-04-24T23:37:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 Processor -- start 2024-04-24T23:37:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 -- start start 2024-04-24T23:37:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c0108740 0x7f69c01a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 0x7f69c01a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c010da80 0x7f69c01ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69c011c230 con 0x7f69c010b0b0 2024-04-24T23:37:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f69c011c0b0 con 0x7f69c0108740 2024-04-24T23:37:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c791b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69c011c3b0 con 0x7f69c010da80 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c4eb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 0x7f69c01a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.555+0000 7f69c4eb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 0x7f69c01a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35226/0 (socket says 172.21.15.33:35226) 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c4eb6640 1 -- 172.21.15.33:0/1679381792 learned_addr learned my addr 172.21.15.33:0/1679381792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c56b7640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c0108740 0x7f69c01a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c4eb6640 1 -- 172.21.15.33:0/1679381792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c010da80 msgr2=0x7f69c01ade60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c010da80 0x7f69c01ade60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c4eb6640 1 -- 172.21.15.33:0/1679381792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c0108740 msgr2=0x7f69c01a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c0108740 0x7f69c01a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c4eb6640 1 -- 172.21.15.33:0/1679381792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01ae4d0 con 0x7f69c010b0b0 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.556+0000 7f69c56b7640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c0108740 0x7f69c01a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.557+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 0x7f69c01a6df0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f69b800e9c0 tx=0x7f69b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.557+0000 7f69be7fc640 1 -- 172.21.15.33:0/1679381792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b800cd80 con 0x7f69c010b0b0 2024-04-24T23:37:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.557+0000 7f69be7fc640 1 -- 172.21.15.33:0/1679381792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b80035f0 con 0x7f69c010b0b0 2024-04-24T23:37:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.557+0000 7f69be7fc640 1 -- 172.21.15.33:0/1679381792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8010690 con 0x7f69c010b0b0 2024-04-24T23:37:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.557+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01ae7c0 con 0x7f69c010b0b0 2024-04-24T23:37:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.557+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c01aed10 con 0x7f69c010b0b0 2024-04-24T23:37:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.562+0000 7f69be7fc640 1 -- 172.21.15.33:0/1679381792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f69b8010830 con 0x7f69c010b0b0 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.562+0000 7f69ca62e740 1 Processor -- start 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.562+0000 7f69ca62e740 1 -- start start 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbcf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 0x56529fbd4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbd7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69ca62e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56529fb33670 con 0x56529fbcf230 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69ca62e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56529fadb970 con 0x56529fbcfc20 2024-04-24T23:37:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69ca62e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56529fb2f630 con 0x56529fbd4d90 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69be7fc640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69a80a2d90 0x7f69a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c5eb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbcf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 0x56529fbd4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 0x56529fbd4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39268/0 (socket says 172.21.15.33:39268) 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 -- 172.21.15.33:0/2575416439 learned_addr learned my addr 172.21.15.33:0/2575416439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbd7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69a80a2d90 0x7f69a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 -- 172.21.15.33:0/2575416439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 msgr2=0x56529fbd7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbd7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 -- 172.21.15.33:0/2575416439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 msgr2=0x56529fbcf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbcf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c56b7640 1 -- 172.21.15.33:0/2575416439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56529faff400 con 0x56529fbcfc20 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.563+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbcf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.564+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 0x56529fbd4720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f69b40314e0 tx=0x7f69b40050f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f4d612aa52ab4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.564+0000 7f69be7fc640 1 -- 172.21.15.33:0/1679381792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f69b80cf820 con 0x7f69c010b0b0 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.564+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69a80a2d90 0x7f69a80a5250 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x56529fbd0480 tx=0x7f69b0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.564+0000 7f69a77fe640 1 -- 172.21.15.33:0/2575416439 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4035040 con 0x56529fbcfc20 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.564+0000 7f69a77fe640 1 -- 172.21.15.33:0/2575416439 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b4005460 con 0x56529fbcfc20 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.564+0000 7f69a77fe640 1 -- 172.21.15.33:0/2575416439 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031c50 con 0x56529fbcfc20 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 -- 172.21.15.33:0/2575416439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 msgr2=0x56529fbd4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 0x56529fbd4720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f69b40314e0 tx=0x7f69b40050f0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 -- 172.21.15.33:0/2575416439 shutdown_connections 2024-04-24T23:37:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbd7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcfc20 0x56529fbd4720 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2575416439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbcf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 -- 172.21.15.33:0/2575416439 >> 172.21.15.33:0/2575416439 conn(0x56529fbca8a0 msgr2=0x56529fbcaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 -- 172.21.15.33:0/2575416439 shutdown_connections 2024-04-24T23:37:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 -- 172.21.15.33:0/2575416439 wait complete. 2024-04-24T23:37:38.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 -- start start 2024-04-24T23:37:38.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.565+0000 7f69ca62e740 1 Processor -- start 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 0x56529fbd9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbd9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbe08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 0x56529fbd9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56529fbd7ce0 con 0x56529fbcfc20 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56529faef050 con 0x56529fbcf230 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 0x56529fbd9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39276/0 (socket says 172.21.15.33:39276) 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 -- 172.21.15.33:0/1285281047 learned_addr learned my addr 172.21.15.33:0/1285281047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56529fbd7e60 con 0x56529fbd4d90 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbd9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbe08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69ca62e740 10 client.0 _ll_register_callbacks cb 0x56529f846be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 -- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 msgr2=0x56529fbe08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbe08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 -- 172.21.15.33:0/1285281047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 msgr2=0x56529fbd9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbd9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c56b7640 1 -- 172.21.15.33:0/1285281047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529fac0dc0 con 0x56529fbcf230 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbd9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.566+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbe08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69c56b7640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 0x56529fbd9320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f69b402aa00 tx=0x7f69b4031c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d84139b4636579fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69a4ff9640 1 -- 172.21.15.33:0/1285281047 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4052060 con 0x56529fbcf230 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69a4ff9640 1 -- 172.21.15.33:0/1285281047 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f69b4005260 con 0x56529fbcf230 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b4005460 con 0x56529fbcf230 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69a4ff9640 1 -- 172.21.15.33:0/1285281047 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f69b4035050 con 0x56529fbcf230 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69ca62e740 10 client.30146 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56529f8473f0 con 0x56529fbcf230 2024-04-24T23:37:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.567+0000 7f69a4ff9640 1 -- 172.21.15.33:0/1285281047 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b4005610 con 0x56529fbcf230 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69a4ff9640 1 -- 172.21.15.33:0/1285281047 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f69b40039d0 con 0x56529fbcf230 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69ca62e740 10 client.30146 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69ca62e740 10 client.30146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69ca62e740 20 client.30146 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69ca62e740 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fbe51a0 0x56529fbe7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69ca62e740 4 client.30146 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f69b4005460 con 0x56529fbe51a0 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.568+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fbe51a0 0x56529fbe7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.569+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fbe51a0 0x56529fbe7620 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56231f0c5f0bbe11 server_cookie=13a1ef628a4eefc3 in_seq=0 out_seq=0 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.569+0000 7f69a4ff9640 10 client.30146 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.569+0000 7f69a4ff9640 1 -- 172.21.15.33:0/1285281047 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f69b4005460 con 0x56529fbe51a0 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.569+0000 7f69a4ff9640 10 client.30146 handle_command_reply: tid=0 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.569+0000 7f69ca62e740 1 client.30146 shutdown 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 msgr2=0x56529fbd9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 0x56529fbd9320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f69b402aa00 tx=0x7f69b4031c20 comp rx=0 tx=0).stop 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 shutdown_connections 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fbe51a0 0x56529fbe7620 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbd4d90 0x56529fbe08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbd9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 --2- 172.21.15.33:0/1285281047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbcf230 0x56529fbd9320 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69a4ff9640 0 client.30146 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.570+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 >> 172.21.15.33:0/1285281047 conn(0x56529fbca8a0 msgr2=0x56529fbc9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.571+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 shutdown_connections 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.571+0000 7f69ca62e740 1 -- 172.21.15.33:0/1285281047 wait complete. 2024-04-24T23:37:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.571+0000 7f69ca62e740 20 client.30146 trim_cache size 0 max 16384 2024-04-24T23:37:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 Processor -- start 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 -- start start 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 0x56529fbc92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbe0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fbd8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 0x56529fbc92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 0x56529fbc92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51748/0 (socket says 172.21.15.33:51748) 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 -- 172.21.15.33:0/2524674533 learned_addr learned my addr 172.21.15.33:0/2524674533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01bbae0 con 0x56529fbcfc20 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b9380 con 0x56529fbd4d90 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b71d0 con 0x56529fbcf230 2024-04-24T23:37:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbe0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fbd8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 -- 172.21.15.33:0/2524674533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 msgr2=0x56529fbd8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fbd8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 -- 172.21.15.33:0/2524674533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 msgr2=0x56529fbe0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbe0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.591+0000 7f69c56b7640 1 -- 172.21.15.33:0/2524674533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56529f9117d0 con 0x56529fbcf230 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.592+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 0x56529fbc92b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69b4031610 tx=0x7f69b4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27d13e27958616e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.592+0000 7f69a6ffd640 1 -- 172.21.15.33:0/2524674533 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4052040 con 0x56529fbcf230 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.592+0000 7f69a6ffd640 1 -- 172.21.15.33:0/2524674533 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b40035f0 con 0x56529fbcf230 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.592+0000 7f69a6ffd640 1 -- 172.21.15.33:0/2524674533 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4004a30 con 0x56529fbcf230 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 msgr2=0x56529fbc92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 0x56529fbc92b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69b4031610 tx=0x7f69b4005fd0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 shutdown_connections 2024-04-24T23:37:38.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fbd8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcfc20 0x56529fbe0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2524674533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcf230 0x56529fbc92b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 >> 172.21.15.33:0/2524674533 conn(0x56529fbca8a0 msgr2=0x56529fbd0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 shutdown_connections 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 -- 172.21.15.33:0/2524674533 wait complete. 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 -- start start 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 Processor -- start 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.593+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbdd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcfc20 0x56529fbddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69ca62e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fc49790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69ca62e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b7840 con 0x56529fbcf230 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69ca62e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b8cb0 con 0x56529fbd4d90 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69ca62e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b7f10 con 0x56529fbcfc20 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbdd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c5eb8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fc49790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbdd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35254/0 (socket says 172.21.15.33:35254) 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 -- 172.21.15.33:0/2149686601 learned_addr learned my addr 172.21.15.33:0/2149686601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcfc20 0x56529fbddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69ca62e740 10 client.0 _ll_register_callbacks cb 0x7f69b80987f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 -- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcfc20 msgr2=0x56529fbddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcfc20 0x56529fbddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 -- 172.21.15.33:0/2149686601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 msgr2=0x56529fc49790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fc49790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 -- 172.21.15.33:0/2149686601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529fbe0ad0 con 0x56529fbcf230 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcfc20 0x56529fbddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c5eb8640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fc49790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69c56b7640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbdd580 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b4031c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db29a933abbf251 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.594+0000 7f69a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:38.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69a4ff9640 1 -- 172.21.15.33:0/2149686601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4052070 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b40035f0 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69a4ff9640 1 -- 172.21.15.33:0/2149686601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f69b4004770 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69a4ff9640 1 -- 172.21.15.33:0/2149686601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f69b4004990 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 10 client.20688 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56529fc4ac90 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69a4ff9640 1 -- 172.21.15.33:0/2149686601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b40035f0 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69a4ff9640 1 -- 172.21.15.33:0/2149686601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f69b405c030 con 0x56529fbcf230 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 10 client.20688 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 10 client.20688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 20 client.20688 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fc4b4c0 0x56529fc4d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 4 client.20688 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56529fb40bb0 con 0x56529fc4b4c0 2024-04-24T23:37:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.595+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fc4b4c0 0x56529fc4d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.596+0000 7f69c4eb6640 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fc4b4c0 0x56529fc4d8e0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d212b3324b1e0c12 server_cookie=22c3fc94d6e7a762 in_seq=0 out_seq=0 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.596+0000 7f69a4ff9640 10 client.20688 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.596+0000 7f69a4ff9640 1 -- 172.21.15.33:0/2149686601 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56529fb40bb0 con 0x56529fc4b4c0 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.596+0000 7f69a4ff9640 10 client.20688 handle_command_reply: tid=0 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.596+0000 7f69ca62e740 1 client.20688 shutdown 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 msgr2=0x56529fbdd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbdd580 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b4031c20 comp rx=0 tx=0).stop 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 shutdown_connections 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56529fc4b4c0 0x56529fc4d8e0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56529fbd4d90 0x56529fc49790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69a4ff9640 0 client.20688 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56529fbcfc20 0x56529fbddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 --2- 172.21.15.33:0/2149686601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56529fbcf230 0x56529fbdd580 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 >> 172.21.15.33:0/2149686601 conn(0x56529fbca8a0 msgr2=0x56529fbcd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 shutdown_connections 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 1 -- 172.21.15.33:0/2149686601 wait complete. 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69ca62e740 20 client.20688 trim_cache size 0 max 16384 2024-04-24T23:37:38.601 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69a80a2d90 msgr2=0x7f69a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69c791b640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69a80a2d90 0x7f69a80a5250 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x56529fbd0480 tx=0x7f69b0004a40 comp rx=0 tx=0).stop 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 msgr2=0x7f69c01a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.597+0000 7f69c791b640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 0x7f69c01a6df0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f69b800e9c0 tx=0x7f69b800ee80 comp rx=0 tx=0).stop 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 shutdown_connections 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f69a80a2d90 0x7f69a80a5250 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c010da80 0x7f69c01ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69c010b0b0 0x7f69c01a6df0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 --2- 172.21.15.33:0/1679381792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c0108740 0x7f69c01a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 >> 172.21.15.33:0/1679381792 conn(0x7f69c006e5d0 msgr2=0x7f69c010c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 shutdown_connections 2024-04-24T23:37:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:38.598+0000 7f69c791b640 1 -- 172.21.15.33:0/1679381792 wait complete. 2024-04-24T23:37:38.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 289613 21.74 MB/sec execute 583 sec latency 78.283 ms 2024-04-24T23:37:38.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:38 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2583264970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:37:39.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:38 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2583264970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:37:39.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:38 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2583264970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:37:39.610 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:37:39.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:37:39.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:37:39.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.785+0000 7f2bf324d640 1 -- 172.21.15.33:0/2334476282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec100010 msgr2=0x7f2bec102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.785+0000 7f2bf324d640 1 --2- 172.21.15.33:0/2334476282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec100010 0x7f2bec102440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009be0 tx=0x7f2be402a6f0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.786+0000 7f2bf324d640 1 -- 172.21.15.33:0/2334476282 shutdown_connections 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.786+0000 7f2bf324d640 1 --2- 172.21.15.33:0/2334476282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec105330 0x7f2bec1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.786+0000 7f2bf324d640 1 --2- 172.21.15.33:0/2334476282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec102980 0x7f2bec104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.786+0000 7f2bf324d640 1 --2- 172.21.15.33:0/2334476282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec100010 0x7f2bec102440 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.786+0000 7f2bf324d640 1 -- 172.21.15.33:0/2334476282 >> 172.21.15.33:0/2334476282 conn(0x7f2bec0fbc30 msgr2=0x7f2bec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.787+0000 7f2bf324d640 1 -- 172.21.15.33:0/2334476282 shutdown_connections 2024-04-24T23:37:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.787+0000 7f2bf324d640 1 -- 172.21.15.33:0/2334476282 wait complete. 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 Processor -- start 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 -- start start 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 0x7f2bec195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec102980 0x7f2bec195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec105330 0x7f2bec19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec109270 con 0x7f2bec100010 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1090f0 con 0x7f2bec102980 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.788+0000 7f2bf324d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1093f0 con 0x7f2bec105330 2024-04-24T23:37:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 0x7f2bec195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 0x7f2bec195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35264/0 (socket says 172.21.15.33:35264) 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1896643417 learned_addr learned my addr 172.21.15.33:0/1896643417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf17ea640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec105330 0x7f2bec19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bebfff640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec102980 0x7f2bec195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1896643417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec105330 msgr2=0x7f2bec19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec105330 0x7f2bec19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1896643417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec102980 msgr2=0x7f2bec195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec102980 0x7f2bec195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.789+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1896643417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec19d570 con 0x7f2bec100010 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.790+0000 7f2bf17ea640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec105330 0x7f2bec19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.790+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 0x7f2bec195950 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2be4034040 tx=0x7f2be4030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.790+0000 7f2be9ffb640 1 -- 172.21.15.33:0/1896643417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4041020 con 0x7f2bec100010 2024-04-24T23:37:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.790+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec19d800 con 0x7f2bec100010 2024-04-24T23:37:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.791+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec19dd40 con 0x7f2bec100010 2024-04-24T23:37:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.791+0000 7f2be9ffb640 1 -- 172.21.15.33:0/1896643417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be4031800 con 0x7f2bec100010 2024-04-24T23:37:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.792+0000 7f2be9ffb640 1 -- 172.21.15.33:0/1896643417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4036040 con 0x7f2bec100010 2024-04-24T23:37:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.796+0000 7f2be9ffb640 1 -- 172.21.15.33:0/1896643417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2be4044020 con 0x7f2bec100010 2024-04-24T23:37:39.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.797+0000 7f2be9ffb640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd40a2b30 0x7f2bd40a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.797+0000 7f2be9ffb640 1 -- 172.21.15.33:0/1896643417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2be40f0140 con 0x7f2bec100010 2024-04-24T23:37:39.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.798+0000 7f2bebfff640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd40a2b30 0x7f2bd40a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.799+0000 7f2bebfff640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd40a2b30 0x7f2bd40a4ff0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2bec196e60 tx=0x7f2be0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.801+0000 7f2bf6036740 1 Processor -- start 2024-04-24T23:37:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 -- start start 2024-04-24T23:37:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f34280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8a9e922c0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d8a9e3a910 con 0x55d8a9f2f780 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf6036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8a9e8e280 con 0x55d8a9f348f0 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f34280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f34280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39324/0 (socket says 172.21.15.33:39324) 2024-04-24T23:37:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bebfff640 1 -- 172.21.15.33:0/174342836 learned_addr learned my addr 172.21.15.33:0/174342836 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.802+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.803+0000 7f2bebfff640 1 -- 172.21.15.33:0/174342836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 msgr2=0x55d8a9f36d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.803+0000 7f2bebfff640 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.803+0000 7f2bebfff640 1 -- 172.21.15.33:0/174342836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 msgr2=0x55d8a9f2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.803+0000 7f2bebfff640 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.803+0000 7f2bebfff640 1 -- 172.21.15.33:0/174342836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a9e4bfb0 con 0x55d8a9f2f780 2024-04-24T23:37:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.804+0000 7f2bebfff640 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f34280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2be0011f90 tx=0x7f2be0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d11963ec714714 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.804+0000 7f2bd2ffd640 1 -- 172.21.15.33:0/174342836 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004220 con 0x55d8a9f2f780 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.804+0000 7f2bd2ffd640 1 -- 172.21.15.33:0/174342836 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be00043c0 con 0x55d8a9f2f780 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.804+0000 7f2bd2ffd640 1 -- 172.21.15.33:0/174342836 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be001f980 con 0x55d8a9f2f780 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 -- 172.21.15.33:0/174342836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 msgr2=0x55d8a9f34280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f34280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2be0011f90 tx=0x7f2be0014000 comp rx=0 tx=0).stop 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 -- 172.21.15.33:0/174342836 shutdown_connections 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f36d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f34280 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 --2- 172.21.15.33:0/174342836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 -- 172.21.15.33:0/174342836 >> 172.21.15.33:0/174342836 conn(0x55d8a9ef6950 msgr2=0x55d8a9f2a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 -- 172.21.15.33:0/174342836 shutdown_connections 2024-04-24T23:37:39.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.805+0000 7f2bf6036740 1 -- 172.21.15.33:0/174342836 wait complete. 2024-04-24T23:37:39.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.806+0000 7f2bf6036740 1 -- start start 2024-04-24T23:37:39.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.806+0000 7f2bf6036740 1 Processor -- start 2024-04-24T23:37:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f2f780 0x55d8a9f39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f348f0 0x55d8a9f405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8a9f378a0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d8a9e4de90 con 0x55d8a9f348f0 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8a9f37a20 con 0x55d8a9f2f780 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35282/0 (socket says 172.21.15.33:35282) 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf6036740 10 client.0 _ll_register_callbacks cb 0x55d8a9e89970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/4225663576 learned_addr learned my addr 172.21.15.33:0/4225663576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.807+0000 7f2bebfff640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f2f780 0x55d8a9f39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.808+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f2f780 msgr2=0x55d8a9f39570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.808+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f2f780 0x55d8a9f39570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.808+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/4225663576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f348f0 msgr2=0x55d8a9f405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.808+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f348f0 0x55d8a9f405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.808+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/4225663576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a9c6e7e0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.808+0000 7f2bebfff640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f2f780 0x55d8a9f39570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:37:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.809+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39030 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f2be4005ee0 tx=0x7f2be402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94294276f0f27eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.809+0000 7f2bcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.809+0000 7f2bcbfff640 1 -- 172.21.15.33:0/4225663576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40d4070 con 0x55d8a9f2ed90 2024-04-24T23:37:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.809+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be00043c0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.809+0000 7f2bcbfff640 1 -- 172.21.15.33:0/4225663576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2be40bc810 con 0x55d8a9f2ed90 2024-04-24T23:37:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.810+0000 7f2bcbfff640 1 -- 172.21.15.33:0/4225663576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2be40bccb0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.810+0000 7f2bf6036740 10 client.20700 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.810+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d8a9ccef70 con 0x55d8a9f2ed90 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.810+0000 7f2bcbfff640 1 -- 172.21.15.33:0/4225663576 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be00043c0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.810+0000 7f2bcbfff640 1 -- 172.21.15.33:0/4225663576 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2be404e060 con 0x55d8a9f2ed90 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bf6036740 10 client.20700 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bf6036740 10 client.20700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bf6036740 20 client.20700 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bf6036740 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9f46fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bf6036740 4 client.20700 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8a9e665a0 con 0x55d8a9f44b30 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.811+0000 7f2bebfff640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9f46fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.812+0000 7f2bebfff640 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9f46fa0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd856e9c1064fa3 server_cookie=9ef8ed4a556425a5 in_seq=0 out_seq=0 2024-04-24T23:37:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.812+0000 7f2bcbfff640 10 client.20700 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.813+0000 7f2bcbfff640 1 -- 172.21.15.33:0/4225663576 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8a9e665a0 con 0x55d8a9f44b30 2024-04-24T23:37:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.813+0000 7f2bcbfff640 10 client.20700 handle_command_reply: tid=0 2024-04-24T23:37:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.815+0000 7f2bf6036740 1 client.20700 shutdown 2024-04-24T23:37:39.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.815+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 msgr2=0x55d8a9f39030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.815+0000 7f2bf6036740 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39030 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f2be4005ee0 tx=0x7f2be402f000 comp rx=0 tx=0).stop 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 shutdown_connections 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bf6036740 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9f46fa0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bcbfff640 0 client.20700 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bf6036740 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f348f0 0x55d8a9f405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bf6036740 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f2f780 0x55d8a9f39570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bf6036740 1 --2- 172.21.15.33:0/4225663576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39030 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.816+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 >> 172.21.15.33:0/4225663576 conn(0x55d8a9ef6950 msgr2=0x55d8a9f29950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.817+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 shutdown_connections 2024-04-24T23:37:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.817+0000 7f2bf6036740 1 -- 172.21.15.33:0/4225663576 wait complete. 2024-04-24T23:37:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.818+0000 7f2bf6036740 20 client.20700 trim_cache size 0 max 16384 2024-04-24T23:37:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 Processor -- start 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 -- start start 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f2d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1aaa50 con 0x55d8a9f2ed90 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1a82f0 con 0x55d8a9f2f780 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf6036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1a6140 con 0x55d8a9f348f0 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.848+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35288/0 (socket says 172.21.15.33:35288) 2024-04-24T23:37:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/3311546545 learned_addr learned my addr 172.21.15.33:0/3311546545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf17ea640 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/3311546545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 msgr2=0x55d8a9f2dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f2dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/3311546545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 msgr2=0x55d8a9f2d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f2d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/3311546545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a9b66170 con 0x55d8a9f2ed90 2024-04-24T23:37:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf17ea640 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:37:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39350 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f2be40b8f80 tx=0x7f2be403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782836684f78f734 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bd27fc640 1 -- 172.21.15.33:0/3311546545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40d4070 con 0x55d8a9f2ed90 2024-04-24T23:37:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bd27fc640 1 -- 172.21.15.33:0/3311546545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be40c0960 con 0x55d8a9f2ed90 2024-04-24T23:37:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.849+0000 7f2bd27fc640 1 -- 172.21.15.33:0/3311546545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40bc810 con 0x55d8a9f2ed90 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 -- 172.21.15.33:0/3311546545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 msgr2=0x55d8a9f39350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39350 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f2be40b8f80 tx=0x7f2be403d000 comp rx=0 tx=0).stop 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 -- 172.21.15.33:0/3311546545 shutdown_connections 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f2d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 --2- 172.21.15.33:0/3311546545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f39350 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 -- 172.21.15.33:0/3311546545 >> 172.21.15.33:0/3311546545 conn(0x55d8a9ef6950 msgr2=0x55d8a9f38c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.851+0000 7f2bf6036740 1 -- 172.21.15.33:0/3311546545 shutdown_connections 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.852+0000 7f2bf6036740 1 -- 172.21.15.33:0/3311546545 wait complete. 2024-04-24T23:37:39.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.852+0000 7f2bf6036740 1 -- start start 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.852+0000 7f2bf6036740 1 Processor -- start 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.852+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f3d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f3dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf6036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f3ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf6036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1a67b0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f3d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf0fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f3d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35292/0 (socket says 172.21.15.33:35292) 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1641738942 learned_addr learned my addr 172.21.15.33:0/1641738942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bebfff640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f3dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1a7c20 con 0x55d8a9f2f780 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1a6e80 con 0x55d8a9f348f0 2024-04-24T23:37:39.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf17ea640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f3ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.853+0000 7f2bf6036740 10 client.0 _ll_register_callbacks cb 0x7f2be40a70b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 msgr2=0x55d8a9f3ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f3ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1641738942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 msgr2=0x55d8a9f3dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f3dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf0fe9640 1 -- 172.21.15.33:0/1641738942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a9f3f320 con 0x55d8a9f2ed90 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf0fe9640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f3d850 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f2be40c4cf0 tx=0x7f2be4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1318c70cb305445a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be40c0960 con 0x55d8a9f2ed90 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bcbfff640 1 -- 172.21.15.33:0/1641738942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40d4070 con 0x55d8a9f2ed90 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bcbfff640 1 -- 172.21.15.33:0/1641738942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2be4004b40 con 0x55d8a9f2ed90 2024-04-24T23:37:39.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.854+0000 7f2bcbfff640 1 -- 172.21.15.33:0/1641738942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2be40bc0f0 con 0x55d8a9f2ed90 2024-04-24T23:37:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 10 client.20706 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=35}) -- 0x55d8a9f3f320 con 0x55d8a9f2ed90 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bcbfff640 1 -- 172.21.15.33:0/1641738942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2be40bba00 con 0x55d8a9f2ed90 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 10 client.20706 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 10 client.20706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 20 client.20706 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9fada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 4 client.20706 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:37:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.855+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8a9e9f800 con 0x55d8a9f44b30 2024-04-24T23:37:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.856+0000 7f2bebfff640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9fada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.856+0000 7f2bebfff640 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9fada30 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c8899fc8671083 server_cookie=9b525fa40cc2fc45 in_seq=0 out_seq=0 2024-04-24T23:37:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.856+0000 7f2bcbfff640 10 client.20706 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.859+0000 7f2bcbfff640 1 -- 172.21.15.33:0/1641738942 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8a9e9f800 con 0x55d8a9f44b30 2024-04-24T23:37:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.859+0000 7f2bcbfff640 10 client.20706 handle_command_reply: tid=0 2024-04-24T23:37:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.859+0000 7f2bf6036740 1 client.20706 shutdown 2024-04-24T23:37:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 msgr2=0x55d8a9f3d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f3d850 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f2be40c4cf0 tx=0x7f2be4000f40 comp rx=0 tx=0).stop 2024-04-24T23:37:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 shutdown_connections 2024-04-24T23:37:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55d8a9f44b30 0x55d8a9fada30 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a9f348f0 0x55d8a9f3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8a9f2f780 0x55d8a9f3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.862+0000 7f2bf6036740 1 --2- 172.21.15.33:0/1641738942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8a9f2ed90 0x55d8a9f3d850 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.863+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 >> 172.21.15.33:0/1641738942 conn(0x55d8a9ef6950 msgr2=0x55d8a9f38bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:39.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.863+0000 7f2bcbfff640 0 client.20706 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:39.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.863+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 shutdown_connections 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.863+0000 7f2bf6036740 1 -- 172.21.15.33:0/1641738942 wait complete. 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.864+0000 7f2bf6036740 20 client.20706 trim_cache size 0 max 16384 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "b8368b2c-f998-46df-92d0-59894c849650", 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:37:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:37:39.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.865+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd40a2b30 msgr2=0x7f2bd40a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.865+0000 7f2bf324d640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd40a2b30 0x7f2bd40a4ff0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2bec196e60 tx=0x7f2be0005f90 comp rx=0 tx=0).stop 2024-04-24T23:37:39.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.865+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 msgr2=0x7f2bec195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.865+0000 7f2bf324d640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 0x7f2bec195950 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2be4034040 tx=0x7f2be4030fe0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 shutdown_connections 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2bd40a2b30 0x7f2bd40a4ff0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bec105330 0x7f2bec19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bec102980 0x7f2bec195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 --2- 172.21.15.33:0/1896643417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec100010 0x7f2bec195950 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 >> 172.21.15.33:0/1896643417 conn(0x7f2bec0fbc30 msgr2=0x7f2bec0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 shutdown_connections 2024-04-24T23:37:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:39.866+0000 7f2bf324d640 1 -- 172.21.15.33:0/1896643417 wait complete. 2024-04-24T23:37:39.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:37:39.881 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:37:39.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 289847 21.73 MB/sec execute 584 sec latency 237.045 ms 2024-04-24T23:37:39.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:39 smithi033 ceph-mon[29201]: pgmap v1120: 129 pgs: 129 active+clean; 674 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 362 op/s 2024-04-24T23:37:40.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.064+0000 7fe1b591c640 1 -- 172.21.15.33:0/2111490930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a8008c10 msgr2=0x7fe1a8007060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.064+0000 7fe1b591c640 1 --2- 172.21.15.33:0/2111490930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a8008c10 0x7fe1a8007060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0067440 tx=0x7fe1b00960a0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.067+0000 7fe1b591c640 1 -- 172.21.15.33:0/2111490930 shutdown_connections 2024-04-24T23:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.067+0000 7fe1b591c640 1 --2- 172.21.15.33:0/2111490930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1a80a2360 0x7fe1a80a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.067+0000 7fe1b591c640 1 --2- 172.21.15.33:0/2111490930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8007630 0x7fe1a8007ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.067+0000 7fe1b591c640 1 --2- 172.21.15.33:0/2111490930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a8008c10 0x7fe1a8007060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.067+0000 7fe1b591c640 1 -- 172.21.15.33:0/2111490930 >> 172.21.15.33:0/2111490930 conn(0x7fe1a80912e0 msgr2=0x7fe1a8093740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.069+0000 7fe1b591c640 1 -- 172.21.15.33:0/2111490930 shutdown_connections 2024-04-24T23:37:40.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.070+0000 7fe1b591c640 1 -- 172.21.15.33:0/2111490930 wait complete. 2024-04-24T23:37:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.071+0000 7fe1b591c640 1 Processor -- start 2024-04-24T23:37:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 -- start start 2024-04-24T23:37:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1a8007630 0x7fe1a8132600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8008c10 0x7fe1a8132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 0x7fe1a8139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80a7fd0 con 0x7fe1a8007630 2024-04-24T23:37:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80a7e50 con 0x7fe1a80a2360 2024-04-24T23:37:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.072+0000 7fe1b591c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80a8150 con 0x7fe1a8008c10 2024-04-24T23:37:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.074+0000 7fe1b511b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 0x7fe1a8139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.074+0000 7fe1b511b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 0x7fe1a8139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39392/0 (socket says 172.21.15.33:39392) 2024-04-24T23:37:40.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.074+0000 7fe1b511b640 1 -- 172.21.15.33:0/859741989 learned_addr learned my addr 172.21.15.33:0/859741989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:40.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.076+0000 7fe1b511b640 1 -- 172.21.15.33:0/859741989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8008c10 msgr2=0x7fe1a8132b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.076+0000 7fe1b511b640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8008c10 0x7fe1a8132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.076+0000 7fe1b511b640 1 -- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1a8007630 msgr2=0x7fe1a8132600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:37:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.076+0000 7fe1b511b640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1a8007630 0x7fe1a8132600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.076+0000 7fe1b511b640 1 -- 172.21.15.33:0/859741989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a813a220 con 0x7fe1a80a2360 2024-04-24T23:37:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.077+0000 7fe1b511b640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 0x7fe1a8139bb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400b3f0 tx=0x7fe1a400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:40.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.078+0000 7fe1adffb640 1 -- 172.21.15.33:0/859741989 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4015020 con 0x7fe1a80a2360 2024-04-24T23:37:40.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.078+0000 7fe1adffb640 1 -- 172.21.15.33:0/859741989 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a4011d10 con 0x7fe1a80a2360 2024-04-24T23:37:40.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.078+0000 7fe1adffb640 1 -- 172.21.15.33:0/859741989 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a400f2e0 con 0x7fe1a80a2360 2024-04-24T23:37:40.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.080+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a813a510 con 0x7fe1a80a2360 2024-04-24T23:37:40.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.080+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a813a9e0 con 0x7fe1a80a2360 2024-04-24T23:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.085+0000 7fe1adffb640 1 -- 172.21.15.33:0/859741989 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe1a4018020 con 0x7fe1a80a2360 2024-04-24T23:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.085+0000 7fe1b996d740 1 Processor -- start 2024-04-24T23:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1adffb640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 -- start start 2024-04-24T23:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b774830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b774e00 0x563e0b779900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 0x563e0b77c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e0b6d86b0 con 0x563e0b774e00 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e0b680880 con 0x563e0b774410 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1adffb640 1 -- 172.21.15.33:0/859741989 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1a40ceba0 con 0x7fe1a80a2360 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.087+0000 7fe1b511b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 0x563e0b77c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.087+0000 7fe1affff640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.087+0000 7fe1b511b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 0x563e0b77c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51818/0 (socket says 172.21.15.33:51818) 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.087+0000 7fe1b511b640 1 -- 172.21.15.33:0/1591599336 learned_addr learned my addr 172.21.15.33:0/1591599336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.087+0000 7fe1affff640 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b774e00 0x563e0b779900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.086+0000 7fe1b996d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e0b6d4670 con 0x563e0b779f70 2024-04-24T23:37:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.091+0000 7fe1b491a640 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b774830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.092+0000 7fe1affff640 1 -- 172.21.15.33:0/1591599336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 msgr2=0x563e0b77c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.092+0000 7fe1affff640 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 0x563e0b77c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.092+0000 7fe1affff640 1 -- 172.21.15.33:0/1591599336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 msgr2=0x563e0b774830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.092+0000 7fe1affff640 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b774830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.092+0000 7fe1affff640 1 -- 172.21.15.33:0/1591599336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0b4b67d0 con 0x563e0b774e00 2024-04-24T23:37:40.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.096+0000 7fe1b511b640 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 0x563e0b77c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.098+0000 7fe1affff640 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b774e00 0x563e0b779900 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005d00 tx=0x7fe19c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39e71b84001b264 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.098+0000 7fe1affff640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00afd0 tx=0x7fe19c00a9b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:40.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.099+0000 7fe19affd640 1 -- 172.21.15.33:0/1591599336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c020070 con 0x563e0b774e00 2024-04-24T23:37:40.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.099+0000 7fe19affd640 1 -- 172.21.15.33:0/1591599336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19c00fe10 con 0x563e0b774e00 2024-04-24T23:37:40.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.099+0000 7fe19affd640 1 -- 172.21.15.33:0/1591599336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c004ce0 con 0x563e0b774e00 2024-04-24T23:37:40.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.101+0000 7fe1b996d740 1 -- 172.21.15.33:0/1591599336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b774e00 msgr2=0x563e0b779900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.101+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b774e00 0x563e0b779900 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005d00 tx=0x7fe19c019000 comp rx=0 tx=0).stop 2024-04-24T23:37:40.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.103+0000 7fe1b996d740 1 -- 172.21.15.33:0/1591599336 shutdown_connections 2024-04-24T23:37:40.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.103+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b779f70 0x563e0b77c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.103+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b774e00 0x563e0b779900 secure :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005d00 tx=0x7fe19c019000 comp rx=0 tx=0).stop 2024-04-24T23:37:40.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.103+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1591599336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b774830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.103+0000 7fe1b996d740 1 -- 172.21.15.33:0/1591599336 >> 172.21.15.33:0/1591599336 conn(0x563e0b76fa80 msgr2=0x563e0b76fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.108+0000 7fe1b996d740 1 -- 172.21.15.33:0/1591599336 shutdown_connections 2024-04-24T23:37:40.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.109+0000 7fe1b996d740 1 -- 172.21.15.33:0/1591599336 wait complete. 2024-04-24T23:37:40.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.109+0000 7fe1b996d740 1 -- start start 2024-04-24T23:37:40.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.109+0000 7fe1b996d740 1 Processor -- start 2024-04-24T23:37:40.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b78fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b790020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b797440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e0b78ded0 con 0x563e0b779f70 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e0b693f50 con 0x563e0b774410 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e0b78e050 con 0x563e0b795050 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.110+0000 7fe1b996d740 10 client.0 _ll_register_callbacks cb 0x563e0b6cf000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.111+0000 7fe1b511b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b797440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.111+0000 7fe1b511b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b797440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:51834/0 (socket says 172.21.15.33:51834) 2024-04-24T23:37:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.111+0000 7fe1b511b640 1 -- 172.21.15.33:0/3232406222 learned_addr learned my addr 172.21.15.33:0/3232406222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.112+0000 7fe1affff640 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b790020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.113+0000 7fe1b511b640 1 -- 172.21.15.33:0/3232406222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 msgr2=0x563e0b78fae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:37:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.113+0000 7fe1b511b640 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b78fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.113+0000 7fe1b511b640 1 -- 172.21.15.33:0/3232406222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 msgr2=0x563e0b790020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:40.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.113+0000 7fe1b511b640 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b790020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.113+0000 7fe1b511b640 1 -- 172.21.15.33:0/3232406222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0b3ec3f0 con 0x563e0b795050 2024-04-24T23:37:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.114+0000 7fe1b511b640 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b797440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4097470 tx=0x7fe1a409b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a50e1d5fef433b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.114+0000 7fe18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:37:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.114+0000 7fe18ffff640 1 -- 172.21.15.33:0/3232406222 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40b3070 con 0x563e0b795050 2024-04-24T23:37:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.114+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe19c00fe10 con 0x563e0b795050 2024-04-24T23:37:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.114+0000 7fe18ffff640 1 -- 172.21.15.33:0/3232406222 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1a40bb0b0 con 0x563e0b795050 2024-04-24T23:37:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.116+0000 7fe18ffff640 1 -- 172.21.15.33:0/3232406222 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1a40a3c80 con 0x563e0b795050 2024-04-24T23:37:40.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.119+0000 7fe1b996d740 10 client.30097 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:40.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.119+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563e0b3ac0f0 con 0x563e0b795050 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe18ffff640 1 -- 172.21.15.33:0/3232406222 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19c00fe10 con 0x563e0b795050 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe18ffff640 1 -- 172.21.15.33:0/3232406222 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe1a4022060 con 0x563e0b795050 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe1b996d740 10 client.30097 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe1b996d740 10 client.30097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe1b996d740 20 client.30097 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe1b996d740 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b7ad0f0 0x563e0b7af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe1b996d740 4 client.30097 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:37:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.122+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e0b6e5bf0 con 0x563e0b7ad0f0 2024-04-24T23:37:40.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.123+0000 7fe1b491a640 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b7ad0f0 0x563e0b7af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.125+0000 7fe18ffff640 10 client.30097 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.125+0000 7fe1b491a640 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b7ad0f0 0x563e0b7af530 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bad0f088eb91697 server_cookie=a730c837ca90ff3a in_seq=0 out_seq=0 2024-04-24T23:37:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.126+0000 7fe18ffff640 1 -- 172.21.15.33:0/3232406222 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e0b6e5bf0 con 0x563e0b7ad0f0 2024-04-24T23:37:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.126+0000 7fe18ffff640 10 client.30097 handle_command_reply: tid=0 2024-04-24T23:37:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.126+0000 7fe1b996d740 1 client.30097 shutdown 2024-04-24T23:37:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 msgr2=0x563e0b797440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b797440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4097470 tx=0x7fe1a409b850 comp rx=0 tx=0).stop 2024-04-24T23:37:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 shutdown_connections 2024-04-24T23:37:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b7ad0f0 0x563e0b7af530 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b797440 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b790020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 --2- 172.21.15.33:0/3232406222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b78fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 >> 172.21.15.33:0/3232406222 conn(0x563e0b76fa80 msgr2=0x563e0b76eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.127+0000 7fe18ffff640 0 client.30097 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.128+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 shutdown_connections 2024-04-24T23:37:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.128+0000 7fe1b996d740 1 -- 172.21.15.33:0/3232406222 wait complete. 2024-04-24T23:37:40.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.129+0000 7fe1b996d740 20 client.30097 trim_cache size 0 max 16384 2024-04-24T23:37:40.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.162+0000 7fe1b996d740 1 Processor -- start 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.162+0000 7fe1b996d740 1 -- start start 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.162+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b775900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.162+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b491a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b775900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b491a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b775900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39410/0 (socket says 172.21.15.33:39410) 2024-04-24T23:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b491a640 1 -- 172.21.15.33:0/1085467673 learned_addr learned my addr 172.21.15.33:0/1085467673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a813f130 con 0x563e0b779f70 2024-04-24T23:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a813c9d0 con 0x563e0b774410 2024-04-24T23:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80a1e80 con 0x563e0b795050 2024-04-24T23:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b511b640 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b511b640 1 -- 172.21.15.33:0/1085467673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 msgr2=0x563e0b775900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b511b640 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b775900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b511b640 1 -- 172.21.15.33:0/1085467673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 msgr2=0x563e0b7764b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b511b640 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7764b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.163+0000 7fe1b511b640 1 -- 172.21.15.33:0/1085467673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0b76b470 con 0x563e0b795050 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.164+0000 7fe1b511b640 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7769f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe1a409afa0 tx=0x7fe1a400b360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26669de01c73ed70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.164+0000 7fe19a7fc640 1 -- 172.21.15.33:0/1085467673 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40b3070 con 0x563e0b795050 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.164+0000 7fe19a7fc640 1 -- 172.21.15.33:0/1085467673 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a40105a0 con 0x563e0b795050 2024-04-24T23:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.164+0000 7fe19a7fc640 1 -- 172.21.15.33:0/1085467673 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4097770 con 0x563e0b795050 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.165+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 msgr2=0x563e0b7769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.165+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7769f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe1a409afa0 tx=0x7fe1a400b360 comp rx=0 tx=0).stop 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.165+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 shutdown_connections 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.166+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7769f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.166+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.166+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1085467673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b775900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.166+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 >> 172.21.15.33:0/1085467673 conn(0x563e0b76fa80 msgr2=0x563e0b78fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:40.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.166+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 shutdown_connections 2024-04-24T23:37:40.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.166+0000 7fe1b996d740 1 -- 172.21.15.33:0/1085467673 wait complete. 2024-04-24T23:37:40.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.167+0000 7fe1b996d740 1 -- start start 2024-04-24T23:37:40.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.167+0000 7fe1b996d740 1 Processor -- start 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.168+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b7bda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.168+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.168+0000 7fe1b996d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.168+0000 7fe1b996d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a813ae90 con 0x563e0b779f70 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.168+0000 7fe1b491a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b7bda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.169+0000 7fe1b491a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b7bda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:39424/0 (socket says 172.21.15.33:39424) 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.169+0000 7fe1b491a640 1 -- 172.21.15.33:0/1113366728 learned_addr learned my addr 172.21.15.33:0/1113366728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.169+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a813c300 con 0x563e0b774410 2024-04-24T23:37:40.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.169+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a813b560 con 0x563e0b795050 2024-04-24T23:37:40.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:39 smithi038 ceph-mon[32450]: pgmap v1120: 129 pgs: 129 active+clean; 674 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 362 op/s 2024-04-24T23:37:40.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.169+0000 7fe1b996d740 10 client.0 _ll_register_callbacks cb 0x7fe1a4010f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.169+0000 7fe1b511b640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1affff640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1b491a640 1 -- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 msgr2=0x563e0b7bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1b491a640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7bef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1b491a640 1 -- 172.21.15.33:0/1113366728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 msgr2=0x563e0b7bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1b491a640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1b491a640 1 -- 172.21.15.33:0/1113366728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0b7bf520 con 0x563e0b774410 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.170+0000 7fe1b491a640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b7bda80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0064c60 tx=0x7fe1b009d4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104bcfbe8138daef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.171+0000 7fe18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.171+0000 7fe18ffff640 1 -- 172.21.15.33:0/1113366728 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00ac030 con 0x563e0b774410 2024-04-24T23:37:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.171+0000 7fe18ffff640 1 -- 172.21.15.33:0/1113366728 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1b009d760 con 0x563e0b774410 2024-04-24T23:37:40.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.173+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a40105a0 con 0x563e0b774410 2024-04-24T23:37:40.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.174+0000 7fe18ffff640 1 -- 172.21.15.33:0/1113366728 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1b009d960 con 0x563e0b774410 2024-04-24T23:37:40.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.174+0000 7fe1b996d740 10 client.30179 fetch_fsmap learned FSMap version 35 2024-04-24T23:37:40.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.174+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563e0b7c08a0 con 0x563e0b774410 2024-04-24T23:37:40.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:39 smithi117 ceph-mon[32529]: pgmap v1120: 129 pgs: 129 active+clean; 674 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 362 op/s 2024-04-24T23:37:40.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.175+0000 7fe18ffff640 1 -- 172.21.15.33:0/1113366728 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a40105a0 con 0x563e0b774410 2024-04-24T23:37:40.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.175+0000 7fe18ffff640 1 -- 172.21.15.33:0/1113366728 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe1b0069130 con 0x563e0b774410 2024-04-24T23:37:40.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.175+0000 7fe1b996d740 10 client.30179 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:37:40.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.176+0000 7fe1b996d740 10 client.30179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:37:40.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.176+0000 7fe1b996d740 20 client.30179 populate_metadata read hostname 'smithi033' 2024-04-24T23:37:40.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.176+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b813480 0x563e0b8158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:37:40.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.176+0000 7fe1b996d740 4 client.30179 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:37:40.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.176+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e0b6dff40 con 0x563e0b813480 2024-04-24T23:37:40.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.181+0000 7fe1affff640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b813480 0x563e0b8158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:37:40.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.181+0000 7fe18ffff640 10 client.30179 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:40.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.181+0000 7fe1affff640 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b813480 0x563e0b8158a0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136b390efb3180e8 server_cookie=afbeb41ae34ca244 in_seq=0 out_seq=0 2024-04-24T23:37:40.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.184+0000 7fe18ffff640 1 -- 172.21.15.33:0/1113366728 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e0b6dff40 con 0x563e0b813480 2024-04-24T23:37:40.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.184+0000 7fe18ffff640 10 client.30179 handle_command_reply: tid=0 2024-04-24T23:37:40.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.184+0000 7fe1b996d740 1 client.30179 shutdown 2024-04-24T23:37:40.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.184+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 msgr2=0x563e0b7bda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.184+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b7bda80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0064c60 tx=0x7fe1b009d4e0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 shutdown_connections 2024-04-24T23:37:40.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x563e0b813480 0x563e0b8158a0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0b795050 0x563e0b7bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e0b779f70 0x563e0b7bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 --2- 172.21.15.33:0/1113366728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e0b774410 0x563e0b7bda80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 >> 172.21.15.33:0/1113366728 conn(0x563e0b76fa80 msgr2=0x563e0b78fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe18ffff640 0 client.30179 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:37:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 shutdown_connections 2024-04-24T23:37:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 1 -- 172.21.15.33:0/1113366728 wait complete. 2024-04-24T23:37:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.185+0000 7fe1b996d740 20 client.30179 trim_cache size 0 max 16384 2024-04-24T23:37:40.186 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: "b8368b2c-f998-46df-92d0-59894c849650": { 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "b8368b2c-f998-46df-92d0-59894c849650", 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:37:40.187 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:37:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.186+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe1a00a2cd0 msgr2=0x7fe1a00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.186+0000 7fe1b591c640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00afd0 tx=0x7fe19c00a9b0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.186+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 msgr2=0x7fe1a8139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:37:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.186+0000 7fe1b591c640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 0x7fe1a8139bb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400b3f0 tx=0x7fe1a400b8b0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 shutdown_connections 2024-04-24T23:37:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a80a2360 0x7fe1a8139bb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8008c10 0x7fe1a8132b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 --2- 172.21.15.33:0/859741989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1a8007630 0x7fe1a8132600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:37:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 >> 172.21.15.33:0/859741989 conn(0x7fe1a80912e0 msgr2=0x7fe1a8095700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:37:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 shutdown_connections 2024-04-24T23:37:40.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:37:40.187+0000 7fe1b591c640 1 -- 172.21.15.33:0/859741989 wait complete. 2024-04-24T23:37:40.200 INFO:tasks.cephfs.filesystem:scrub status for tag:b8368b2c-f998-46df-92d0-59894c849650 - {'path': '/', 'tag': 'b8368b2c-f998-46df-92d0-59894c849650', 'options': 'recursive,force'} 2024-04-24T23:37:40.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 290202 21.72 MB/sec execute 585 sec latency 211.048 ms 2024-04-24T23:37:40.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:37:40.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:40 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:37:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:41.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:37:41.171 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:40 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:37:41.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:40 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:41.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 290598 21.72 MB/sec execute 586 sec latency 67.516 ms 2024-04-24T23:37:41.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:41 smithi033 ceph-mon[29201]: pgmap v1121: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 423 op/s 2024-04-24T23:37:41.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:41 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:42.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:41 smithi038 ceph-mon[32450]: pgmap v1121: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 423 op/s 2024-04-24T23:37:42.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:41 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:42.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:41 smithi117 ceph-mon[32529]: pgmap v1121: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 423 op/s 2024-04-24T23:37:42.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:41 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:37:42.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 291005 21.72 MB/sec execute 587 sec latency 102.172 ms 2024-04-24T23:37:43.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 291549 21.72 MB/sec execute 588 sec latency 73.951 ms 2024-04-24T23:37:43.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:43 smithi033 ceph-mon[29201]: pgmap v1122: 129 pgs: 129 active+clean; 657 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 391 op/s 2024-04-24T23:37:44.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:43 smithi038 ceph-mon[32450]: pgmap v1122: 129 pgs: 129 active+clean; 657 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 391 op/s 2024-04-24T23:37:44.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:43 smithi117 ceph-mon[32529]: pgmap v1122: 129 pgs: 129 active+clean; 657 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 391 op/s 2024-04-24T23:37:44.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 291680 21.69 MB/sec execute 589 sec latency 525.448 ms 2024-04-24T23:37:45.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 291944 21.68 MB/sec execute 590 sec latency 232.013 ms 2024-04-24T23:37:45.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:45 smithi033 ceph-mon[29201]: pgmap v1123: 129 pgs: 129 active+clean; 657 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-04-24T23:37:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:45 smithi038 ceph-mon[32450]: pgmap v1123: 129 pgs: 129 active+clean; 657 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-04-24T23:37:46.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:45 smithi117 ceph-mon[32529]: pgmap v1123: 129 pgs: 129 active+clean; 657 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-04-24T23:37:46.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 292382 21.69 MB/sec execute 591 sec latency 100.754 ms 2024-04-24T23:37:47.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 292954 21.71 MB/sec execute 592 sec latency 85.188 ms 2024-04-24T23:37:48.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:47 smithi038 ceph-mon[32450]: pgmap v1124: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-04-24T23:37:48.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:47 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:37:48.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:47 smithi117 ceph-mon[32529]: pgmap v1124: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-04-24T23:37:48.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:47 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:37:48.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:47 smithi033 ceph-mon[29201]: pgmap v1124: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-04-24T23:37:48.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:47 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:37:48.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 293549 21.73 MB/sec execute 593 sec latency 78.500 ms 2024-04-24T23:37:49.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:48 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:37:49.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:48 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:37:49.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:48 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:37:49.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 293734 21.71 MB/sec execute 594 sec latency 395.171 ms 2024-04-24T23:37:50.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:49 smithi038 ceph-mon[32450]: pgmap v1125: 129 pgs: 129 active+clean; 656 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 380 op/s 2024-04-24T23:37:50.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:49 smithi117 ceph-mon[32529]: pgmap v1125: 129 pgs: 129 active+clean; 656 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 380 op/s 2024-04-24T23:37:50.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:49 smithi033 ceph-mon[29201]: pgmap v1125: 129 pgs: 129 active+clean; 656 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 380 op/s 2024-04-24T23:37:50.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 294005 21.71 MB/sec execute 595 sec latency 153.016 ms 2024-04-24T23:37:51.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 294257 21.69 MB/sec execute 596 sec latency 127.853 ms 2024-04-24T23:37:52.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:51 smithi038 ceph-mon[32450]: pgmap v1126: 129 pgs: 129 active+clean; 646 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-04-24T23:37:52.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:51 smithi117 ceph-mon[32529]: pgmap v1126: 129 pgs: 129 active+clean; 646 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-04-24T23:37:52.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:51 smithi033 ceph-mon[29201]: pgmap v1126: 129 pgs: 129 active+clean; 646 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-04-24T23:37:52.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 294583 21.69 MB/sec execute 597 sec latency 133.079 ms 2024-04-24T23:37:53.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:37:53.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:37:53.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:52 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:37:53.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:37:53.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:37:53.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:52 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:37:53.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:37:53.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:37:53.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:52 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:37:53.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 294983 21.68 MB/sec execute 598 sec latency 117.547 ms 2024-04-24T23:37:54.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:53 smithi038 ceph-mon[32450]: pgmap v1127: 129 pgs: 129 active+clean; 648 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 385 op/s 2024-04-24T23:37:54.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:53 smithi117 ceph-mon[32529]: pgmap v1127: 129 pgs: 129 active+clean; 648 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 385 op/s 2024-04-24T23:37:54.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:53 smithi033 ceph-mon[29201]: pgmap v1127: 129 pgs: 129 active+clean; 648 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 385 op/s 2024-04-24T23:37:54.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 295255 21.65 MB/sec execute 599 sec latency 352.582 ms 2024-04-24T23:37:55.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 600 sec 2024-04-24T23:37:56.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:55 smithi038 ceph-mon[32450]: pgmap v1128: 129 pgs: 129 active+clean; 654 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 427 op/s 2024-04-24T23:37:56.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:55 smithi117 ceph-mon[32529]: pgmap v1128: 129 pgs: 129 active+clean; 654 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 427 op/s 2024-04-24T23:37:56.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:55 smithi033 ceph-mon[29201]: pgmap v1128: 129 pgs: 129 active+clean; 654 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 427 op/s 2024-04-24T23:37:56.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 601 sec 2024-04-24T23:37:57.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 602 sec 2024-04-24T23:37:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:57 smithi038 ceph-mon[32450]: pgmap v1129: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-04-24T23:37:58.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:57 smithi117 ceph-mon[32529]: pgmap v1129: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-04-24T23:37:58.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:57 smithi033 ceph-mon[29201]: pgmap v1129: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-04-24T23:37:58.910 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 603 sec 2024-04-24T23:37:59.911 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 604 sec 2024-04-24T23:38:00.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:37:59 smithi038 ceph-mon[32450]: pgmap v1130: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-04-24T23:38:00.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:37:59 smithi117 ceph-mon[32529]: pgmap v1130: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-04-24T23:38:00.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:37:59 smithi033 ceph-mon[29201]: pgmap v1130: 129 pgs: 129 active+clean; 650 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-04-24T23:38:00.911 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 605 sec 2024-04-24T23:38:01.911 INFO:tasks.workunit.client.0.smithi033.stdout: 9 cleanup 606 sec 2024-04-24T23:38:02.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:01 smithi038 ceph-mon[32450]: pgmap v1131: 129 pgs: 129 active+clean; 621 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 392 op/s 2024-04-24T23:38:02.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:01 smithi117 ceph-mon[32529]: pgmap v1131: 129 pgs: 129 active+clean; 621 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 392 op/s 2024-04-24T23:38:02.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:01 smithi033 ceph-mon[29201]: pgmap v1131: 129 pgs: 129 active+clean; 621 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 392 op/s 2024-04-24T23:38:02.911 INFO:tasks.workunit.client.0.smithi033.stdout: 9 cleanup 607 sec 2024-04-24T23:38:03.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:02 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:03.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:02 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:03.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:02 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:03.911 INFO:tasks.workunit.client.0.smithi033.stdout: 9 cleanup 608 sec 2024-04-24T23:38:04.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:03 smithi038 ceph-mon[32450]: pgmap v1132: 129 pgs: 129 active+clean; 603 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 313 op/s 2024-04-24T23:38:04.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:03 smithi117 ceph-mon[32529]: pgmap v1132: 129 pgs: 129 active+clean; 603 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 313 op/s 2024-04-24T23:38:04.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:03 smithi033 ceph-mon[29201]: pgmap v1132: 129 pgs: 129 active+clean; 603 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 313 op/s 2024-04-24T23:38:04.911 INFO:tasks.workunit.client.0.smithi033.stdout: 9 cleanup 609 sec 2024-04-24T23:38:05.911 INFO:tasks.workunit.client.0.smithi033.stdout: 9 cleanup 610 sec 2024-04-24T23:38:06.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:05 smithi038 ceph-mon[32450]: pgmap v1133: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-04-24T23:38:06.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:05 smithi117 ceph-mon[32529]: pgmap v1133: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-04-24T23:38:06.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:05 smithi033 ceph-mon[29201]: pgmap v1133: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-04-24T23:38:06.911 INFO:tasks.workunit.client.0.smithi033.stdout: 8 cleanup 611 sec 2024-04-24T23:38:07.911 INFO:tasks.workunit.client.0.smithi033.stdout: 5 cleanup 612 sec 2024-04-24T23:38:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:07 smithi038 ceph-mon[32450]: pgmap v1134: 129 pgs: 129 active+clean; 588 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-04-24T23:38:08.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:07 smithi117 ceph-mon[32529]: pgmap v1134: 129 pgs: 129 active+clean; 588 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-04-24T23:38:08.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:07 smithi033 ceph-mon[29201]: pgmap v1134: 129 pgs: 129 active+clean; 588 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-04-24T23:38:08.911 INFO:tasks.workunit.client.0.smithi033.stdout: 5 cleanup 613 sec 2024-04-24T23:38:09.911 INFO:tasks.workunit.client.0.smithi033.stdout: 5 cleanup 614 sec 2024-04-24T23:38:10.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:09 smithi038 ceph-mon[32450]: pgmap v1135: 129 pgs: 129 active+clean; 586 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-04-24T23:38:10.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:09 smithi117 ceph-mon[32529]: pgmap v1135: 129 pgs: 129 active+clean; 586 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-04-24T23:38:10.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:38:10.202 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:38:10.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:09 smithi033 ceph-mon[29201]: pgmap v1135: 129 pgs: 129 active+clean; 586 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-04-24T23:38:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.307+0000 7f039fa09640 1 -- 172.21.15.33:0/1545579719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398104bf0 msgr2=0x7f0398106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- 172.21.15.33:0/1545579719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398104bf0 0x7f0398106fe0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0388009900 tx=0x7f038802a410 comp rx=0 tx=0).stop 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- 172.21.15.33:0/1545579719 shutdown_connections 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- 172.21.15.33:0/1545579719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0398107520 0x7f0398109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- 172.21.15.33:0/1545579719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398104bf0 0x7f0398106fe0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- 172.21.15.33:0/1545579719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03981022c0 0x7f03981046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- 172.21.15.33:0/1545579719 >> 172.21.15.33:0/1545579719 conn(0x7f03980fbbf0 msgr2=0x7f03980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- 172.21.15.33:0/1545579719 shutdown_connections 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- 172.21.15.33:0/1545579719 wait complete. 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 Processor -- start 2024-04-24T23:38:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- start start 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03981022c0 0x7f0398101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0398104bf0 0x7f0398100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 0x7f03981007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f039810d460 con 0x7f0398107520 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f039810d2e0 con 0x7f0398104bf0 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.308+0000 7f039fa09640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f039810d5e0 con 0x7f03981022c0 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 0x7f03981007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 0x7f03981007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52638/0 (socket says 172.21.15.33:52638) 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 -- 172.21.15.33:0/1712615197 learned_addr learned my addr 172.21.15.33:0/1712615197 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039d7a5640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03981022c0 0x7f0398101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039cfa4640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0398104bf0 0x7f0398100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 -- 172.21.15.33:0/1712615197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03981022c0 msgr2=0x7f0398101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03981022c0 0x7f0398101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 -- 172.21.15.33:0/1712615197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0398104bf0 msgr2=0x7f0398100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0398104bf0 0x7f0398100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 -- 172.21.15.33:0/1712615197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0398101040 con 0x7f0398107520 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039dfa6640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 0x7f03981007b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f039400d750 tx=0x7f039400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f038e7fc640 1 -- 172.21.15.33:0/1712615197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394017070 con 0x7f0398107520 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03981a1ba0 con 0x7f0398107520 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f038e7fc640 1 -- 172.21.15.33:0/1712615197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0394005500 con 0x7f0398107520 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f038e7fc640 1 -- 172.21.15.33:0/1712615197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394004d90 con 0x7f0398107520 2024-04-24T23:38:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.309+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a20c0 con 0x7f0398107520 2024-04-24T23:38:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.311+0000 7f038e7fc640 1 -- 172.21.15.33:0/1712615197 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0394004f30 con 0x7f0398107520 2024-04-24T23:38:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 Processor -- start 2024-04-24T23:38:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 -- start start 2024-04-24T23:38:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649e240 0x55bc3649e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bc364026b0 con 0x55bc3649ec30 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bc363aa880 con 0x55bc3649e240 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f03a27f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc363fe670 con 0x55bc364a3da0 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f038e7fc640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03840a2b90 0x7f03840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52652/0 (socket says 172.21.15.33:52652) 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 -- 172.21.15.33:0/2356634705 learned_addr learned my addr 172.21.15.33:0/2356634705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039dfa6640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03840a2b90 0x7f03840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649e240 0x55bc3649e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f038e7fc640 1 -- 172.21.15.33:0/1712615197 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03940ceda0 con 0x7f0398107520 2024-04-24T23:38:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 -- 172.21.15.33:0/2356634705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 msgr2=0x55bc364a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 -- 172.21.15.33:0/2356634705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649e240 msgr2=0x55bc3649e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649e240 0x55bc3649e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039d7a5640 1 -- 172.21.15.33:0/2356634705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc361e07d0 con 0x55bc3649ec30 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.312+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.313+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a3730 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0390007f90 tx=0x7f039000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b575685789c732 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.313+0000 7f039dfa6640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03840a2b90 0x7f03840a5050 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f03940cc560 tx=0x7f039406a1f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.313+0000 7f03837fe640 1 -- 172.21.15.33:0/2356634705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390004d20 con 0x55bc3649ec30 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.313+0000 7f03837fe640 1 -- 172.21.15.33:0/2356634705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0390010be0 con 0x55bc3649ec30 2024-04-24T23:38:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.313+0000 7f03837fe640 1 -- 172.21.15.33:0/2356634705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900057a0 con 0x55bc3649ec30 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- 172.21.15.33:0/2356634705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 msgr2=0x55bc364a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a3730 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0390007f90 tx=0x7f039000f850 comp rx=0 tx=0).stop 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- 172.21.15.33:0/2356634705 shutdown_connections 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a3730 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2356634705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649e240 0x55bc3649e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- 172.21.15.33:0/2356634705 >> 172.21.15.33:0/2356634705 conn(0x55bc364998b0 msgr2=0x55bc36499cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- 172.21.15.33:0/2356634705 shutdown_connections 2024-04-24T23:38:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- 172.21.15.33:0/2356634705 wait complete. 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- start start 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 Processor -- start 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc364a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc364a3da0 0x55bc364afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bc364a6d50 con 0x55bc3649ec30 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bc363bdf50 con 0x55bc364a3da0 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc364a6ed0 con 0x55bc3649e240 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.314+0000 7f03a27f0740 10 client.0 _ll_register_callbacks cb 0x55bc363f9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039d7a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc364a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039d7a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52658/0 (socket says 172.21.15.33:52658) 2024-04-24T23:38:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039d7a5640 1 -- 172.21.15.33:0/2172112838 learned_addr learned my addr 172.21.15.33:0/2172112838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc364a3da0 0x55bc364afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 -- 172.21.15.33:0/2172112838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc364a3da0 msgr2=0x55bc364afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc364a3da0 0x55bc364afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 -- 172.21.15.33:0/2172112838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 msgr2=0x55bc364a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 -- 172.21.15.33:0/2172112838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc361163f0 con 0x55bc3649e240 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc364a8610 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0388009230 tx=0x7f038802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e94119af3cf9bafb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f0380ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f0380ff9640 1 -- 172.21.15.33:0/2172112838 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388035040 con 0x55bc3649e240 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f0380ff9640 1 -- 172.21.15.33:0/2172112838 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0388004230 con 0x55bc3649e240 2024-04-24T23:38:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0390010be0 con 0x55bc3649e240 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f0380ff9640 1 -- 172.21.15.33:0/2172112838 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f038804f050 con 0x55bc3649e240 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f03a27f0740 10 client.30112 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.315+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc360d60f0 con 0x55bc3649e240 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f0380ff9640 1 -- 172.21.15.33:0/2172112838 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0390010be0 con 0x55bc3649e240 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f0380ff9640 1 -- 172.21.15.33:0/2172112838 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f0388047020 con 0x55bc3649e240 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f03a27f0740 10 client.30112 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f03a27f0740 10 client.30112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:10.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f03a27f0740 20 client.30112 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc364b4430 0x55bc364b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f03a27f0740 4 client.30112 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc3640fbf0 con 0x55bc364b4430 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc364b4430 0x55bc364b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc364b4430 0x55bc364b68b0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d951398261ebd6a0 server_cookie=9fb956ea0318cb in_seq=0 out_seq=0 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.316+0000 7f0380ff9640 10 client.30112 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f0380ff9640 1 -- 172.21.15.33:0/2172112838 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc3640fbf0 con 0x55bc364b4430 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f0380ff9640 10 client.30112 handle_command_reply: tid=0 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 client.30112 shutdown 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 msgr2=0x55bc364a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc364a8610 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0388009230 tx=0x7f038802a410 comp rx=0 tx=0).stop 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 shutdown_connections 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc364b4430 0x55bc364b68b0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc364a3da0 0x55bc364afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649ec30 0x55bc364a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2172112838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc364a8610 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f0380ff9640 0 client.30112 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 >> 172.21.15.33:0/2172112838 conn(0x55bc364998b0 msgr2=0x55bc36498af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 shutdown_connections 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 1 -- 172.21.15.33:0/2172112838 wait complete. 2024-04-24T23:38:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.317+0000 7f03a27f0740 20 client.30112 trim_cache size 0 max 16384 2024-04-24T23:38:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 Processor -- start 2024-04-24T23:38:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 -- start start 2024-04-24T23:38:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc36499c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 0x55bc364a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03981aed70 con 0x55bc364a3da0 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03981ac610 con 0x55bc3649ec30 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f03a27f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03981aa460 con 0x55bc3649e240 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 0x55bc364a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 0x55bc364a8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52664/0 (socket says 172.21.15.33:52664) 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 -- 172.21.15.33:0/622227737 learned_addr learned my addr 172.21.15.33:0/622227737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc36499c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039d7a5640 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 -- 172.21.15.33:0/622227737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 msgr2=0x55bc36499c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc36499c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 -- 172.21.15.33:0/622227737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 msgr2=0x55bc364afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.335+0000 7f039cfa4640 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f039cfa4640 1 -- 172.21.15.33:0/622227737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc364a0670 con 0x55bc364a3da0 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f039cfa4640 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 0x55bc364a8930 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f0388031a30 tx=0x7f0388005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84e77836ddaa759 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f0382ffd640 1 -- 172.21.15.33:0/622227737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388035070 con 0x55bc364a3da0 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f0382ffd640 1 -- 172.21.15.33:0/622227737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038803be20 con 0x55bc364a3da0 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f0382ffd640 1 -- 172.21.15.33:0/622227737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880039f0 con 0x55bc364a3da0 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 -- 172.21.15.33:0/622227737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 msgr2=0x55bc364a8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 0x55bc364a8930 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f0388031a30 tx=0x7f0388005fd0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 -- 172.21.15.33:0/622227737 shutdown_connections 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc364a3da0 0x55bc364a8930 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 --2- 172.21.15.33:0/622227737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc3649e240 0x55bc36499c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.336+0000 7f03a27f0740 1 -- 172.21.15.33:0/622227737 >> 172.21.15.33:0/622227737 conn(0x55bc364998b0 msgr2=0x55bc364a8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 -- 172.21.15.33:0/622227737 shutdown_connections 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 -- 172.21.15.33:0/622227737 wait complete. 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 -- start start 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 Processor -- start 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 0x55bc364acc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03981aaad0 con 0x55bc3649e240 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03981abf40 con 0x55bc3649ec30 2024-04-24T23:38:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03981ab1a0 con 0x55bc364a3da0 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f03a27f0740 10 client.0 _ll_register_callbacks cb 0x7f0388009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f039cfa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 0x55bc364acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f039cfa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 0x55bc364acc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52672/0 (socket says 172.21.15.33:52672) 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.337+0000 7f039cfa4640 1 -- 172.21.15.33:0/2128336647 learned_addr learned my addr 172.21.15.33:0/2128336647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 -- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 msgr2=0x55bc364ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 -- 172.21.15.33:0/2128336647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 msgr2=0x55bc364ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 -- 172.21.15.33:0/2128336647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc364ae710 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f039cfa4640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 0x55bc364acc20 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f038803b650 tx=0x7f0388031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63903098908dc3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f0380ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f0380ff9640 1 -- 172.21.15.33:0/2128336647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388035070 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038803be20 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f0380ff9640 1 -- 172.21.15.33:0/2128336647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0388003770 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f0380ff9640 1 -- 172.21.15.33:0/2128336647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0388003970 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f03a27f0740 10 client.20733 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc364afae0 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.338+0000 7f0380ff9640 1 -- 172.21.15.33:0/2128336647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038803be20 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f0380ff9640 1 -- 172.21.15.33:0/2128336647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f0388047070 con 0x55bc3649e240 2024-04-24T23:38:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f03a27f0740 10 client.20733 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f03a27f0740 10 client.20733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f03a27f0740 20 client.20733 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc3651ace0 0x55bc3651d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f03a27f0740 4 client.20733 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc36409f40 con 0x55bc3651ace0 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc3651ace0 0x55bc3651d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f0380ff9640 10 client.20733 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.339+0000 7f039d7a5640 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc3651ace0 0x55bc3651d100 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc92109180b6b14c server_cookie=b5943eed59108099 in_seq=0 out_seq=0 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f0380ff9640 1 -- 172.21.15.33:0/2128336647 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc36409f40 con 0x55bc3651ace0 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f0380ff9640 10 client.20733 handle_command_reply: tid=0 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 client.20733 shutdown 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 msgr2=0x55bc364acc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 0x55bc364acc20 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f038803b650 tx=0x7f0388031400 comp rx=0 tx=0).stop 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 shutdown_connections 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55bc3651ace0 0x55bc3651d100 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc364a3da0 0x55bc364ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc3649ec30 0x55bc364ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f0380ff9640 0 client.20733 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 --2- 172.21.15.33:0/2128336647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc3649e240 0x55bc364acc20 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 >> 172.21.15.33:0/2128336647 conn(0x55bc364998b0 msgr2=0x55bc3649c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 shutdown_connections 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 1 -- 172.21.15.33:0/2128336647 wait complete. 2024-04-24T23:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f03a27f0740 20 client.20733 trim_cache size 0 max 16384 2024-04-24T23:38:10.350 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:10.350 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:38:10.350 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:38:10.350 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03840a2b90 msgr2=0x7f03840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f039fa09640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03840a2b90 0x7f03840a5050 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f03940cc560 tx=0x7f039406a1f0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 msgr2=0x7f03981007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.340+0000 7f039fa09640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 0x7f03981007b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f039400d750 tx=0x7f039400dc10 comp rx=0 tx=0).stop 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 shutdown_connections 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f03840a2b90 0x7f03840a5050 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398107520 0x7f03981007b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0398104bf0 0x7f0398100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 --2- 172.21.15.33:0/1712615197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03981022c0 0x7f0398101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 >> 172.21.15.33:0/1712615197 conn(0x7f03980fbbf0 msgr2=0x7f0398102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 shutdown_connections 2024-04-24T23:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.341+0000 7f039fa09640 1 -- 172.21.15.33:0/1712615197 wait complete. 2024-04-24T23:38:10.352 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:38:10.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 -- 172.21.15.33:0/2316346750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac100680 msgr2=0x7f2dac100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 --2- 172.21.15.33:0/2316346750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac100680 0x7f2dac100aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009900 tx=0x7f2d9402a410 comp rx=0 tx=0).stop 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 -- 172.21.15.33:0/2316346750 shutdown_connections 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 --2- 172.21.15.33:0/2316346750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac102230 0x7f2dac108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 --2- 172.21.15.33:0/2316346750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2dac101870 0x7f2dac101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 --2- 172.21.15.33:0/2316346750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac100680 0x7f2dac100aa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 -- 172.21.15.33:0/2316346750 >> 172.21.15.33:0/2316346750 conn(0x7f2dac0fbbf0 msgr2=0x7f2dac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 -- 172.21.15.33:0/2316346750 shutdown_connections 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 -- 172.21.15.33:0/2316346750 wait complete. 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 Processor -- start 2024-04-24T23:38:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.454+0000 7f2db182a640 1 -- start start 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2db182a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 0x7f2dac197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2db182a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac101870 0x7f2dac198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2db182a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2dac102230 0x7f2dac19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2db182a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac10d480 con 0x7f2dac100680 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2db182a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac10d300 con 0x7f2dac102230 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2db182a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac10d600 con 0x7f2dac101870 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 0x7f2dac197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 0x7f2dac197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52682/0 (socket says 172.21.15.33:52682) 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 -- 172.21.15.33:0/1799499900 learned_addr learned my addr 172.21.15.33:0/1799499900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2dab7fe640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2dac102230 0x7f2dac19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daa7fc640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac101870 0x7f2dac198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 -- 172.21.15.33:0/1799499900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac101870 msgr2=0x7f2dac198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac101870 0x7f2dac198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 -- 172.21.15.33:0/1799499900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2dac102230 msgr2=0x7f2dac19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2dac102230 0x7f2dac19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 -- 172.21.15.33:0/1799499900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac19f700 con 0x7f2dac100680 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.455+0000 7f2daaffd640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 0x7f2dac197ae0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f2d94031440 tx=0x7f2d94032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.456+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9403f030 con 0x7f2dac100680 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.456+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac19f990 con 0x7f2dac100680 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.456+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d94003ce0 con 0x7f2dac100680 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.456+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94035070 con 0x7f2dac100680 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.456+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac19fe80 con 0x7f2dac100680 2024-04-24T23:38:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.457+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac1079a0 con 0x7f2dac100680 2024-04-24T23:38:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.458+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2d94042030 con 0x7f2dac100680 2024-04-24T23:38:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.459+0000 7f2db0828640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2d980a2b90 0x7f2d980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.459+0000 7f2daa7fc640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2d980a2b90 0x7f2d980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.459+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2d940efc30 con 0x7f2dac100680 2024-04-24T23:38:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.460+0000 7f2daa7fc640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2d980a2b90 0x7f2d980a5050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2dac198ff0 tx=0x7f2d9c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.464+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d940bc1b0 con 0x7f2dac100680 2024-04-24T23:38:10.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.873+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dac02d170 con 0x7f2dac100680 2024-04-24T23:38:10.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.874+0000 7f2db0828640 1 -- 172.21.15.33:0/1799499900 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11860 (secure 0 0 0) 0x7f2d940c04b0 con 0x7f2dac100680 2024-04-24T23:38:10.875 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:38:10.875 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:38:01.352642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24481,24508,24479]},"id":1}]} 2024-04-24T23:38:10.875 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 38 2024-04-24T23:38:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.877+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2d980a2b90 msgr2=0x7f2d980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.877+0000 7f2db182a640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2d980a2b90 0x7f2d980a5050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2dac198ff0 tx=0x7f2d9c005f90 comp rx=0 tx=0).stop 2024-04-24T23:38:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.877+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 msgr2=0x7f2dac197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.877+0000 7f2db182a640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 0x7f2dac197ae0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f2d94031440 tx=0x7f2d94032ce0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 shutdown_connections 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2d980a2b90 0x7f2d980a5050 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2dac102230 0x7f2dac19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dac101870 0x7f2dac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 --2- 172.21.15.33:0/1799499900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dac100680 0x7f2dac197ae0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 >> 172.21.15.33:0/1799499900 conn(0x7f2dac0fbbf0 msgr2=0x7f2dac104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 shutdown_connections 2024-04-24T23:38:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:10.878+0000 7f2db182a640 1 -- 172.21.15.33:0/1799499900 wait complete. 2024-04-24T23:38:10.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:38:10.911 INFO:tasks.workunit.client.0.smithi033.stdout: 5 cleanup 615 sec 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 -- 172.21.15.33:0/1678674403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68101870 msgr2=0x7fed68101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 --2- 172.21.15.33:0/1678674403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68101870 0x7fed68101cf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed64009860 tx=0x7fed6402a340 comp rx=0 tx=0).stop 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 -- 172.21.15.33:0/1678674403 shutdown_connections 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 --2- 172.21.15.33:0/1678674403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68102230 0x7fed68108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 --2- 172.21.15.33:0/1678674403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68101870 0x7fed68101cf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 --2- 172.21.15.33:0/1678674403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed68100680 0x7fed68100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 -- 172.21.15.33:0/1678674403 >> 172.21.15.33:0/1678674403 conn(0x7fed680fbbf0 msgr2=0x7fed680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.047+0000 7fed70053640 1 -- 172.21.15.33:0/1678674403 shutdown_connections 2024-04-24T23:38:11.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.048+0000 7fed70053640 1 -- 172.21.15.33:0/1678674403 wait complete. 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.048+0000 7fed70053640 1 Processor -- start 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.048+0000 7fed70053640 1 -- start start 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed70053640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68100680 0x7fed68196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed70053640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68101870 0x7fed68196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed70053640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed68102230 0x7fed6819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6ddef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68100680 0x7fed68196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6d5ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68101870 0x7fed68196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6ddef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68100680 0x7fed68196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53664/0 (socket says 172.21.15.33:53664) 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6ddef640 1 -- 172.21.15.33:0/3924980579 learned_addr learned my addr 172.21.15.33:0/3924980579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed6810c2e0 con 0x7fed68102230 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed6810c160 con 0x7fed68101870 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed6810c460 con 0x7fed68100680 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed68102230 0x7fed6819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6d5ee640 1 -- 172.21.15.33:0/3924980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68100680 msgr2=0x7fed68196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68100680 0x7fed68196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6d5ee640 1 -- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed68102230 msgr2=0x7fed6819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed68102230 0x7fed6819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.049+0000 7fed6d5ee640 1 -- 172.21.15.33:0/3924980579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6819e420 con 0x7fed68101870 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.050+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68101870 0x7fed68196d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fed6402a860 tx=0x7fed64033e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.050+0000 7fed5effd640 1 -- 172.21.15.33:0/3924980579 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64037040 con 0x7fed68101870 2024-04-24T23:38:11.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.050+0000 7fed5effd640 1 -- 172.21.15.33:0/3924980579 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed64004050 con 0x7fed68101870 2024-04-24T23:38:11.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.050+0000 7fed5effd640 1 -- 172.21.15.33:0/3924980579 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed640046c0 con 0x7fed68101870 2024-04-24T23:38:11.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.050+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed680688d0 con 0x7fed68101870 2024-04-24T23:38:11.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.051+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed68068c80 con 0x7fed68101870 2024-04-24T23:38:11.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.053+0000 7fed5effd640 1 -- 172.21.15.33:0/3924980579 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fed64042020 con 0x7fed68101870 2024-04-24T23:38:11.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.053+0000 7fed5effd640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed540a2c10 0x7fed540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.053+0000 7fed5effd640 1 -- 172.21.15.33:0/3924980579 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed640eef00 con 0x7fed68101870 2024-04-24T23:38:11.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed6ddef640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed540a2c10 0x7fed540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 Processor -- start 2024-04-24T23:38:11.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 -- start start 2024-04-24T23:38:11.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 0x562634ba7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634baaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634bab0a0 0x562634bbcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed6ddef640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed540a2c10 0x7fed540a50d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fed580096c0 tx=0x7fed58010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562634b10670 con 0x562634ba7550 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562634ab8970 con 0x562634bab0a0 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed72e33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562634b0c630 con 0x562634ba7f40 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed6ddef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 0x562634ba7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed6ddef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 0x562634ba7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52704/0 (socket says 172.21.15.33:52704) 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed6ddef640 1 -- 172.21.15.33:0/3584665938 learned_addr learned my addr 172.21.15.33:0/3584665938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.054+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634bab0a0 0x562634bbcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634baaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6e5f0640 1 -- 172.21.15.33:0/3584665938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 msgr2=0x562634baaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634baaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6e5f0640 1 -- 172.21.15.33:0/3584665938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 msgr2=0x562634ba7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 0x562634ba7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6e5f0640 1 -- 172.21.15.33:0/3584665938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562634adc400 con 0x562634bab0a0 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6ddef640 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 0x562634ba7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634bab0a0 0x562634bbcb50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fed60009f00 tx=0x7fed60011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee1d1403b5e55ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed53fff640 1 -- 172.21.15.33:0/3584665938 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60004d20 con 0x562634bab0a0 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed53fff640 1 -- 172.21.15.33:0/3584665938 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60004030 con 0x562634bab0a0 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.055+0000 7fed53fff640 1 -- 172.21.15.33:0/3584665938 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60005260 con 0x562634bab0a0 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 -- 172.21.15.33:0/3584665938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634bab0a0 msgr2=0x562634bbcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634bab0a0 0x562634bbcb50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fed60009f00 tx=0x7fed60011810 comp rx=0 tx=0).stop 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 -- 172.21.15.33:0/3584665938 shutdown_connections 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634bab0a0 0x562634bbcb50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634baaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 --2- 172.21.15.33:0/3584665938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7550 0x562634ba7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 -- 172.21.15.33:0/3584665938 >> 172.21.15.33:0/3584665938 conn(0x562634babcb0 msgr2=0x562634bac0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 -- 172.21.15.33:0/3584665938 shutdown_connections 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.056+0000 7fed72e33740 1 -- 172.21.15.33:0/3584665938 wait complete. 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 -- start start 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 Processor -- start 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bbeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 0x562634bbf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634bab0a0 0x562634bc6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562634bbd550 con 0x562634ba7f40 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562634acc050 con 0x562634ba7550 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562634bbd6d0 con 0x562634bab0a0 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed72e33740 10 client.0 _ll_register_callbacks cb 0x562634823be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed6d5ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 0x562634bbf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.057+0000 7fed6ddef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bbeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6d5ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 0x562634bbf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52706/0 (socket says 172.21.15.33:52706) 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bbeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60762/0 (socket says 172.21.15.33:60762) 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6d5ee640 1 -- 172.21.15.33:0/1669099539 learned_addr learned my addr 172.21.15.33:0/1669099539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634bab0a0 0x562634bc6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 -- 172.21.15.33:0/1669099539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634bab0a0 msgr2=0x562634bc6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634bab0a0 0x562634bc6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 -- 172.21.15.33:0/1669099539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 msgr2=0x562634bbf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 0x562634bbf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 -- 172.21.15.33:0/1669099539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562634a9ddc0 con 0x562634ba7550 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634bab0a0 0x562634bc6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 0x562634bbf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bbeae0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fed580058c0 tx=0x7fed58004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17d8bfc4cf013dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed517fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed517fa640 1 -- 172.21.15.33:0/1669099539 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5802b060 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed517fa640 1 -- 172.21.15.33:0/1669099539 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed580189d0 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.058+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed60004030 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed517fa640 1 -- 172.21.15.33:0/1669099539 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fed58032050 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 10 client.30212 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5626348243f0 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed517fa640 1 -- 172.21.15.33:0/1669099539 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60004030 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed517fa640 1 -- 172.21.15.33:0/1669099539 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fed580253e0 con 0x562634ba7550 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 10 client.30212 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 10 client.30212 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 20 client.30212 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634bca9b0 0x562634bcce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 4 client.30212 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562634b1dbb0 con 0x562634bca9b0 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.059+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634bca9b0 0x562634bcce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.060+0000 7fed517fa640 10 client.30212 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.060+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634bca9b0 0x562634bcce50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a368315d8d6727d server_cookie=81442876be6aa5e4 in_seq=0 out_seq=0 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.060+0000 7fed517fa640 1 -- 172.21.15.33:0/1669099539 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562634b1dbb0 con 0x562634bca9b0 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.060+0000 7fed517fa640 10 client.30212 handle_command_reply: tid=0 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 client.30212 shutdown 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 msgr2=0x562634bbeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bbeae0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fed580058c0 tx=0x7fed58004730 comp rx=0 tx=0).stop 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 shutdown_connections 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634bca9b0 0x562634bcce50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634bab0a0 0x562634bc6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634ba7f40 0x562634bbf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 --2- 172.21.15.33:0/1669099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bbeae0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed517fa640 0 client.30212 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 >> 172.21.15.33:0/1669099539 conn(0x562634babcb0 msgr2=0x562634ba6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 shutdown_connections 2024-04-24T23:38:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 1 -- 172.21.15.33:0/1669099539 wait complete. 2024-04-24T23:38:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.061+0000 7fed72e33740 20 client.30212 trim_cache size 0 max 16384 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.075+0000 7fed72e33740 1 Processor -- start 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.075+0000 7fed72e33740 1 -- start start 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed72e33740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bb9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed72e33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634bc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed72e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634baebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed72e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed681aa920 con 0x562634bab0a0 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed72e33740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a81c0 con 0x562634ba7550 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed72e33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a6010 con 0x562634ba7f40 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634baebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634baebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52718/0 (socket says 172.21.15.33:52718) 2024-04-24T23:38:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 -- 172.21.15.33:0/828119535 learned_addr learned my addr 172.21.15.33:0/828119535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634bc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6ddef640 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bb9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 -- 172.21.15.33:0/828119535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 msgr2=0x562634bc6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634bc6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 -- 172.21.15.33:0/828119535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 msgr2=0x562634bb9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bb9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 -- 172.21.15.33:0/828119535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626348ee7d0 con 0x562634bab0a0 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634bc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634baebc0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x562634bacb80 tx=0x7fed60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9da89412d93508 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed537fe640 1 -- 172.21.15.33:0/828119535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60027070 con 0x562634bab0a0 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed537fe640 1 -- 172.21.15.33:0/828119535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60004e50 con 0x562634bab0a0 2024-04-24T23:38:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.076+0000 7fed537fe640 1 -- 172.21.15.33:0/828119535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600047b0 con 0x562634bab0a0 2024-04-24T23:38:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 -- 172.21.15.33:0/828119535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 msgr2=0x562634baebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634baebc0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x562634bacb80 tx=0x7fed60009b50 comp rx=0 tx=0).stop 2024-04-24T23:38:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 -- 172.21.15.33:0/828119535 shutdown_connections 2024-04-24T23:38:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634baebc0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7f40 0x562634bc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 --2- 172.21.15.33:0/828119535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7550 0x562634bb9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 -- 172.21.15.33:0/828119535 >> 172.21.15.33:0/828119535 conn(0x562634babcb0 msgr2=0x562634bb8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 -- 172.21.15.33:0/828119535 shutdown_connections 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 -- 172.21.15.33:0/828119535 wait complete. 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.077+0000 7fed72e33740 1 -- start start 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 Processor -- start 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7550 0x562634bc2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7f40 0x562634bc3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634c2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a6680 con 0x562634bab0a0 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a7af0 con 0x562634ba7f40 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a6d50 con 0x562634ba7550 2024-04-24T23:38:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed72e33740 10 client.0 _ll_register_callbacks cb 0x7fed58009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634c2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634c2ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52722/0 (socket says 172.21.15.33:52722) 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 -- 172.21.15.33:0/1654504824 learned_addr learned my addr 172.21.15.33:0/1654504824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6d5ee640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7f40 0x562634bc3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7550 0x562634bc2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 -- 172.21.15.33:0/1654504824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7550 msgr2=0x562634bc2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7550 0x562634bc2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 -- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7f40 msgr2=0x562634bc3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7f40 0x562634bc3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6e5f0640 1 -- 172.21.15.33:0/1654504824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562634bc62d0 con 0x562634bab0a0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.078+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7550 0x562634bc2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed6e5f0640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634c2ef80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fed60006010 tx=0x7fed60005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc5d83c2edafa59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed517fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed517fa640 1 -- 172.21.15.33:0/1654504824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60027040 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed517fa640 1 -- 172.21.15.33:0/1654504824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed60004aa0 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed60004e50 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed517fa640 1 -- 172.21.15.33:0/1654504824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fed6003b050 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 10 client.20757 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562634c30440 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed517fa640 1 -- 172.21.15.33:0/1654504824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60004e50 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed517fa640 1 -- 172.21.15.33:0/1654504824 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fed60018070 con 0x562634bab0a0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 10 client.20757 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 10 client.20757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 20 client.20757 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634c30c10 0x562634c33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 4 client.20757 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.079+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562634b17f00 con 0x562634c30c10 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634c30c10 0x562634c33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed6ddef640 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634c30c10 0x562634c33030 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=333656bf1c19880a server_cookie=eedae3bf8dd49aff in_seq=0 out_seq=0 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed517fa640 10 client.20757 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed517fa640 1 -- 172.21.15.33:0/1654504824 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562634b17f00 con 0x562634c30c10 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed517fa640 10 client.20757 handle_command_reply: tid=0 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 client.20757 shutdown 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 msgr2=0x562634c2ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634c2ef80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fed60006010 tx=0x7fed60005d00 comp rx=0 tx=0).stop 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 shutdown_connections 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x562634c30c10 0x562634c33030 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562634bab0a0 0x562634c2ef80 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562634ba7f40 0x562634bc3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 --2- 172.21.15.33:0/1654504824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562634ba7550 0x562634bc2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed517fa640 0 client.20757 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 >> 172.21.15.33:0/1654504824 conn(0x562634babcb0 msgr2=0x562634bb93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 shutdown_connections 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 1 -- 172.21.15.33:0/1654504824 wait complete. 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.080+0000 7fed72e33740 20 client.20757 trim_cache size 0 max 16384 2024-04-24T23:38:11.089 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:38:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed540a2c10 msgr2=0x7fed540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed540a2c10 0x7fed540a50d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fed580096c0 tx=0x7fed58010040 comp rx=0 tx=0).stop 2024-04-24T23:38:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68101870 msgr2=0x7fed68196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68101870 0x7fed68196d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fed6402a860 tx=0x7fed64033e00 comp rx=0 tx=0).stop 2024-04-24T23:38:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 shutdown_connections 2024-04-24T23:38:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fed540a2c10 0x7fed540a50d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed68102230 0x7fed6819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68101870 0x7fed68196d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 --2- 172.21.15.33:0/3924980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed68100680 0x7fed68196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 >> 172.21.15.33:0/3924980579 conn(0x7fed680fbbf0 msgr2=0x7fed68104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 shutdown_connections 2024-04-24T23:38:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.081+0000 7fed70053640 1 -- 172.21.15.33:0/3924980579 wait complete. 2024-04-24T23:38:11.092 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 -- 172.21.15.33:0/2608551941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8102250 msgr2=0x7fa5f8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 --2- 172.21.15.33:0/2608551941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8102250 0x7fa5f8108ae0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009ca0 tx=0x7fa5f002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 -- 172.21.15.33:0/2608551941 shutdown_connections 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 --2- 172.21.15.33:0/2608551941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8102250 0x7fa5f8108ae0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 --2- 172.21.15.33:0/2608551941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f8101890 0x7fa5f8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 --2- 172.21.15.33:0/2608551941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f81006a0 0x7fa5f8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 -- 172.21.15.33:0/2608551941 >> 172.21.15.33:0/2608551941 conn(0x7fa5f80fbc30 msgr2=0x7fa5f80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 -- 172.21.15.33:0/2608551941 shutdown_connections 2024-04-24T23:38:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.237+0000 7fa6004e1640 1 -- 172.21.15.33:0/2608551941 wait complete. 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 Processor -- start 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 -- start start 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81006a0 0x7fa5f806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 0x7fa5f819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f8102250 0x7fa5f819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f810c300 con 0x7fa5f8101890 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f810c180 con 0x7fa5f8102250 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa6004e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f810c480 con 0x7fa5f81006a0 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 0x7fa5f819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 0x7fa5f819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52730/0 (socket says 172.21.15.33:52730) 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 -- 172.21.15.33:0/3762604289 learned_addr learned my addr 172.21.15.33:0/3762604289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f8102250 0x7fa5f819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81006a0 0x7fa5f806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 -- 172.21.15.33:0/3762604289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81006a0 msgr2=0x7fa5f806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81006a0 0x7fa5f806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 -- 172.21.15.33:0/3762604289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f8102250 msgr2=0x7fa5f819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f8102250 0x7fa5f819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.238+0000 7fa5fda7c640 1 -- 172.21.15.33:0/3762604289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f819f650 con 0x7fa5f8101890 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.239+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 0x7fa5f819c750 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800c890 tx=0x7fa5e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.239+0000 7fa5ef7fe640 1 -- 172.21.15.33:0/3762604289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8004220 con 0x7fa5f8101890 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.239+0000 7fa5ef7fe640 1 -- 172.21.15.33:0/3762604289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e80043c0 con 0x7fa5f8101890 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.239+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f819f940 con 0x7fa5f8101890 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.239+0000 7fa5ef7fe640 1 -- 172.21.15.33:0/3762604289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80049e0 con 0x7fa5f8101890 2024-04-24T23:38:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.239+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f819fe90 con 0x7fa5f8101890 2024-04-24T23:38:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.241+0000 7fa5ef7fe640 1 -- 172.21.15.33:0/3762604289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa5e80079b0 con 0x7fa5f8101890 2024-04-24T23:38:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 Processor -- start 2024-04-24T23:38:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 -- start start 2024-04-24T23:38:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca5e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5ec20 0x556d1ca63720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca63d90 0x556d1ca66220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d1c9c2670 con 0x556d1ca63d90 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d1c96a970 con 0x556d1ca5e230 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa6031f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d1c9be630 con 0x556d1ca5ec20 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5ef7fe640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5e40a2b90 0x7fa5e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca63d90 0x556d1ca66220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5fe27d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5ec20 0x556d1ca63720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca63d90 0x556d1ca66220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52734/0 (socket says 172.21.15.33:52734) 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5fda7c640 1 -- 172.21.15.33:0/923551357 learned_addr learned my addr 172.21.15.33:0/923551357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca5e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5e40a2b90 0x7fa5e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.242+0000 7fa5ef7fe640 1 -- 172.21.15.33:0/3762604289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5e80cf430 con 0x7fa5f8101890 2024-04-24T23:38:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fe27d640 1 -- 172.21.15.33:0/923551357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 msgr2=0x556d1ca5e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca5e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fe27d640 1 -- 172.21.15.33:0/923551357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca63d90 msgr2=0x556d1ca66220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca63d90 0x556d1ca66220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fe27d640 1 -- 172.21.15.33:0/923551357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1c98e400 con 0x556d1ca5ec20 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5ec20 0x556d1ca63720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400bbb0 tx=0x7fa5f400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968c21b639c48839 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5e40a2b90 0x7fa5e40a5050 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009430 tx=0x7fa5f0005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5ecbfe640 1 -- 172.21.15.33:0/923551357 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4015070 con 0x556d1ca5ec20 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5ecbfe640 1 -- 172.21.15.33:0/923551357 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4005020 con 0x556d1ca5ec20 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.243+0000 7fa5ecbfe640 1 -- 172.21.15.33:0/923551357 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4004a90 con 0x556d1ca5ec20 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 -- 172.21.15.33:0/923551357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5ec20 msgr2=0x556d1ca63720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5ec20 0x556d1ca63720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400bbb0 tx=0x7fa5f400f730 comp rx=0 tx=0).stop 2024-04-24T23:38:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 -- 172.21.15.33:0/923551357 shutdown_connections 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca63d90 0x556d1ca66220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5ec20 0x556d1ca63720 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 --2- 172.21.15.33:0/923551357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca5e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 -- 172.21.15.33:0/923551357 >> 172.21.15.33:0/923551357 conn(0x556d1ca598a0 msgr2=0x556d1ca59ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 -- 172.21.15.33:0/923551357 shutdown_connections 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 -- 172.21.15.33:0/923551357 wait complete. 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 -- start start 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.244+0000 7fa6031f2740 1 Processor -- start 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca68330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca68870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca6f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d1ca66d40 con 0x556d1ca5ec20 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d1c97e050 con 0x556d1ca5e230 2024-04-24T23:38:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d1ca66ec0 con 0x556d1ca63d90 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 10 client.0 _ll_register_callbacks cb 0x556d1c6d5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca68870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca68870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52736/0 (socket says 172.21.15.33:52736) 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fda7c640 1 -- 172.21.15.33:0/720621054 learned_addr learned my addr 172.21.15.33:0/720621054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca68330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca6f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 -- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 msgr2=0x556d1ca6f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca6f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 -- 172.21.15.33:0/720621054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 msgr2=0x556d1ca68870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca68870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 -- 172.21.15.33:0/720621054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1c94fdc0 con 0x556d1ca5e230 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca68870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca68330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x556d1ca5a770 tx=0x7fa5f400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35857492c33ba5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/720621054 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4015040 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/720621054 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5f40053e0 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.245+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f4005020 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/720621054 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa5f40055e0 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 10 client.30230 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556d1c6d63f0 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/720621054 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4005020 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/720621054 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fa5f4011ab0 con 0x556d1ca5e230 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 10 client.30230 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 10 client.30230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 20 client.30230 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1ca73ef0 0x556d1ca76340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 4 client.30230 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:11.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d1c9cfbb0 con 0x556d1ca73ef0 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.246+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1ca73ef0 0x556d1ca76340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa5e1ffb640 10 client.30230 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1ca73ef0 0x556d1ca76340 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcc53d4b7f652fbd server_cookie=18e36d039286a8b1 in_seq=0 out_seq=0 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/720621054 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d1c9cfbb0 con 0x556d1ca73ef0 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa5e1ffb640 10 client.30230 handle_command_reply: tid=0 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 client.30230 shutdown 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 msgr2=0x556d1ca68330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca68330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x556d1ca5a770 tx=0x7fa5f400be30 comp rx=0 tx=0).stop 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 shutdown_connections 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1ca73ef0 0x556d1ca76340 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca6f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa5e1ffb640 0 client.30230 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca68870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 --2- 172.21.15.33:0/720621054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca68330 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 >> 172.21.15.33:0/720621054 conn(0x556d1ca598a0 msgr2=0x556d1ca58b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 shutdown_connections 2024-04-24T23:38:11.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.247+0000 7fa6031f2740 1 -- 172.21.15.33:0/720621054 wait complete. 2024-04-24T23:38:11.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.248+0000 7fa6031f2740 20 client.30230 trim_cache size 0 max 16384 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 Processor -- start 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 -- start start 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca7abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81accd0 con 0x556d1ca5ec20 2024-04-24T23:38:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81aa570 con 0x556d1ca5e230 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa6031f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81a83c0 con 0x556d1ca63d90 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52744/0 (socket says 172.21.15.33:52744) 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2956088324 learned_addr learned my addr 172.21.15.33:0/2956088324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca7abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.261+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2956088324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 msgr2=0x556d1ca7abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca7abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2956088324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 msgr2=0x556d1ca582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2956088324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1c7a07d0 con 0x556d1ca5ec20 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca7abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6f870 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa5e809b600 tx=0x7fa5e805ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38d8ff65799861f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5e3fff640 1 -- 172.21.15.33:0/2956088324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e801d040 con 0x556d1ca5ec20 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5e3fff640 1 -- 172.21.15.33:0/2956088324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e809c000 con 0x556d1ca5ec20 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa5e3fff640 1 -- 172.21.15.33:0/2956088324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8007ca0 con 0x556d1ca5ec20 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 -- 172.21.15.33:0/2956088324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 msgr2=0x556d1ca6f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6f870 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa5e809b600 tx=0x7fa5e805ba20 comp rx=0 tx=0).stop 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 -- 172.21.15.33:0/2956088324 shutdown_connections 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca63d90 0x556d1ca7abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6f870 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2956088324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca5e230 0x556d1ca582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.262+0000 7fa6031f2740 1 -- 172.21.15.33:0/2956088324 >> 172.21.15.33:0/2956088324 conn(0x556d1ca598a0 msgr2=0x556d1ca5ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 -- 172.21.15.33:0/2956088324 shutdown_connections 2024-04-24T23:38:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 -- 172.21.15.33:0/2956088324 wait complete. 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 -- start start 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 Processor -- start 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5e230 0x556d1ca6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca63d90 0x556d1cad8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81a8a30 con 0x556d1ca5ec20 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81a9ea0 con 0x556d1ca63d90 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81a9100 con 0x556d1ca5e230 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa6031f2740 10 client.0 _ll_register_callbacks cb 0x7fa5e8097480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52748/0 (socket says 172.21.15.33:52748) 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2633201099 learned_addr learned my addr 172.21.15.33:0/2633201099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5e230 0x556d1ca6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5e230 msgr2=0x556d1ca6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5e230 0x556d1ca6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fea7e640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca63d90 0x556d1cad8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2633201099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca63d90 msgr2=0x556d1cad8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca63d90 0x556d1cad8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.263+0000 7fa5fda7c640 1 -- 172.21.15.33:0/2633201099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1cad8980 con 0x556d1ca5ec20 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5fda7c640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6cbb0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa5e809b890 tx=0x7fa5e8099d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1e79cca529a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/2633201099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8064070 con 0x556d1ca5ec20 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/2633201099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5e80a40e0 con 0x556d1ca5ec20 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e809c000 con 0x556d1ca5ec20 2024-04-24T23:38:11.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/2633201099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa5e801d040 con 0x556d1ca5ec20 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 10 client.20775 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556d1cad9b20 con 0x556d1ca5ec20 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/2633201099 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e809c000 con 0x556d1ca5ec20 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/2633201099 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fa5e809ad50 con 0x556d1ca5ec20 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 10 client.20775 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 10 client.20775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 20 client.20775 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1cada520 0x556d1cadc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 4 client.20775 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.264+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d1c9c9f00 con 0x556d1cada520 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1cada520 0x556d1cadc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa5e1ffb640 10 client.20775 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa5fe27d640 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1cada520 0x556d1cadc940 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cb92ad9ae756723 server_cookie=66b060e57093246b in_seq=0 out_seq=0 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa5e1ffb640 1 -- 172.21.15.33:0/2633201099 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d1c9c9f00 con 0x556d1cada520 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa5e1ffb640 10 client.20775 handle_command_reply: tid=0 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 client.20775 shutdown 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 msgr2=0x556d1ca6cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6cbb0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa5e809b890 tx=0x7fa5e8099d00 comp rx=0 tx=0).stop 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 shutdown_connections 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x556d1cada520 0x556d1cadc940 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d1ca63d90 0x556d1cad8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa5e1ffb640 0 client.20775 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d1ca5ec20 0x556d1ca6cbb0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 --2- 172.21.15.33:0/2633201099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d1ca5e230 0x556d1ca6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 >> 172.21.15.33:0/2633201099 conn(0x556d1ca598a0 msgr2=0x556d1ca5f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 shutdown_connections 2024-04-24T23:38:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 1 -- 172.21.15.33:0/2633201099 wait complete. 2024-04-24T23:38:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.265+0000 7fa6031f2740 20 client.20775 trim_cache size 0 max 16384 2024-04-24T23:38:11.275 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5e40a2b90 msgr2=0x7fa5e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5e40a2b90 0x7fa5e40a5050 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009430 tx=0x7fa5f0005d80 comp rx=0 tx=0).stop 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 msgr2=0x7fa5f819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 0x7fa5f819c750 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800c890 tx=0x7fa5e800cd50 comp rx=0 tx=0).stop 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 shutdown_connections 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fa5e40a2b90 0x7fa5e40a5050 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5f8102250 0x7fa5f819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f8101890 0x7fa5f819c750 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 --2- 172.21.15.33:0/3762604289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81006a0 0x7fa5f806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 >> 172.21.15.33:0/3762604289 conn(0x7fa5f80fbc30 msgr2=0x7fa5f8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 shutdown_connections 2024-04-24T23:38:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.266+0000 7fa6004e1640 1 -- 172.21.15.33:0/3762604289 wait complete. 2024-04-24T23:38:11.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:38:11.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 -- 172.21.15.33:0/2384317294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4101860 msgr2=0x7f85c4101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 --2- 172.21.15.33:0/2384317294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4101860 0x7f85c4101ce0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009900 tx=0x7f85b402a410 comp rx=0 tx=0).stop 2024-04-24T23:38:11.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 -- 172.21.15.33:0/2384317294 shutdown_connections 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 --2- 172.21.15.33:0/2384317294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85c4102220 0x7f85c4108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 --2- 172.21.15.33:0/2384317294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4101860 0x7f85c4101ce0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 --2- 172.21.15.33:0/2384317294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c4100670 0x7f85c4100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 -- 172.21.15.33:0/2384317294 >> 172.21.15.33:0/2384317294 conn(0x7f85c40fbbe0 msgr2=0x7f85c40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 -- 172.21.15.33:0/2384317294 shutdown_connections 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.382+0000 7f85cc334640 1 -- 172.21.15.33:0/2384317294 wait complete. 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 Processor -- start 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 -- start start 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 0x7f85c410aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85c4101860 0x7f85c4109630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c4102220 0x7f85c4109b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410d470 con 0x7f85c4100670 2024-04-24T23:38:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410d2f0 con 0x7f85c4102220 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85cc334640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410d5f0 con 0x7f85c4101860 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 0x7f85c410aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 0x7f85c410aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52772/0 (socket says 172.21.15.33:52772) 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3991790957 learned_addr learned my addr 172.21.15.33:0/3991790957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85c98cf640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85c4101860 0x7f85c4109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c4102220 0x7f85c4109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3991790957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85c4101860 msgr2=0x7f85c4109630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85c4101860 0x7f85c4109630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3991790957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c4102220 msgr2=0x7f85c4109b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c4102220 0x7f85c4109b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3991790957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c410a400 con 0x7f85c4100670 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.383+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 0x7f85c410aff0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d770 tx=0x7f85c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.384+0000 7f85bb7fe640 1 -- 172.21.15.33:0/3991790957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0017070 con 0x7f85c4100670 2024-04-24T23:38:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.384+0000 7f85bb7fe640 1 -- 172.21.15.33:0/3991790957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c0009c30 con 0x7f85c4100670 2024-04-24T23:38:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.384+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c41a1ba0 con 0x7f85c4100670 2024-04-24T23:38:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.384+0000 7f85bb7fe640 1 -- 172.21.15.33:0/3991790957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0003dd0 con 0x7f85c4100670 2024-04-24T23:38:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.384+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c41a20c0 con 0x7f85c4100670 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.386+0000 7f85bb7fe640 1 -- 172.21.15.33:0/3991790957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85c0003fb0 con 0x7f85c4100670 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.386+0000 7f85cf041740 1 Processor -- start 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.386+0000 7f85cf041740 1 -- start start 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85cf041740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec023650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85cf041740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec028720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85cf041740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec02b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85cf041740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588ebf87670 con 0x5588ec023c20 2024-04-24T23:38:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85cf041740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588ebf2f970 con 0x5588ec028d90 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85cf041740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588ebf83630 con 0x5588ec023230 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec028720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec028720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52776/0 (socket says 172.21.15.33:52776) 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3286240477 learned_addr learned my addr 172.21.15.33:0/3286240477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85bb7fe640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f85b00a2b90 0x7f85b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec023650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85c98cf640 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec02b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f85b00a2b90 0x7f85b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85bb7fe640 1 -- 172.21.15.33:0/3991790957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f85c00cedc0 con 0x7f85c4100670 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3286240477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 msgr2=0x5588ec023650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec023650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3286240477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 msgr2=0x5588ec02b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec02b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca0d0640 1 -- 172.21.15.33:0/3286240477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ebf53400 con 0x5588ec023c20 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.387+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec023650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.388+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f85b00a2b90 0x7f85b00a5050 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f85bc005c40 tx=0x7f85bc009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.388+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec028720 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f85c009af30 tx=0x7f85c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaec2de243c8cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.388+0000 7f85b8bfe640 1 -- 172.21.15.33:0/3286240477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0020070 con 0x5588ec023c20 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.388+0000 7f85b8bfe640 1 -- 172.21.15.33:0/3286240477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c009b930 con 0x5588ec023c20 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.388+0000 7f85b8bfe640 1 -- 172.21.15.33:0/3286240477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c00042a0 con 0x5588ec023c20 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- 172.21.15.33:0/3286240477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 msgr2=0x5588ec028720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec028720 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f85c009af30 tx=0x7f85c0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- 172.21.15.33:0/3286240477 shutdown_connections 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec02b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec028720 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- 172.21.15.33:0/3286240477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec023650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- 172.21.15.33:0/3286240477 >> 172.21.15.33:0/3286240477 conn(0x5588ec01e8a0 msgr2=0x5588ec01eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- 172.21.15.33:0/3286240477 shutdown_connections 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- 172.21.15.33:0/3286240477 wait complete. 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- start start 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 Processor -- start 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec02d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec02d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec0348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588ec02bce0 con 0x5588ec023c20 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588ebf43050 con 0x5588ec028d90 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588ec02be60 con 0x5588ec023230 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85cf041740 10 client.0 _ll_register_callbacks cb 0x5588ebc9abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85c98cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec02d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85c98cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec02d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52782/0 (socket says 172.21.15.33:52782) 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.389+0000 7f85c98cf640 1 -- 172.21.15.33:0/883739761 learned_addr learned my addr 172.21.15.33:0/883739761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec02d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec0348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85c98cf640 1 -- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 msgr2=0x5588ec02d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85c98cf640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec02d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85c98cf640 1 -- 172.21.15.33:0/883739761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 msgr2=0x5588ec0348c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85c98cf640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec0348c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85c98cf640 1 -- 172.21.15.33:0/883739761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ebf14dc0 con 0x5588ec023c20 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec0348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85c98cf640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec02d8f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f85b40314e0 tx=0x7f85b40044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0cd8d5e2ad65c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85adffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85adffb640 1 -- 172.21.15.33:0/883739761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4035040 con 0x5588ec023c20 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85adffb640 1 -- 172.21.15.33:0/883739761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f85b40054d0 con 0x5588ec023c20 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c009b930 con 0x5588ec023c20 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85adffb640 1 -- 172.21.15.33:0/883739761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f85b404f050 con 0x5588ec023c20 2024-04-24T23:38:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85cf041740 10 client.20787 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5588ebc9b3f0 con 0x5588ec023c20 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.390+0000 7f85adffb640 1 -- 172.21.15.33:0/883739761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c009b930 con 0x5588ec023c20 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85adffb640 1 -- 172.21.15.33:0/883739761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f85b4031b60 con 0x5588ec023c20 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85cf041740 10 client.20787 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85cf041740 10 client.20787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85cf041740 20 client.20787 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85cf041740 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec039240 0x5588ec03b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85cf041740 4 client.20787 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ebf94bb0 con 0x5588ec039240 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec039240 0x5588ec03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec039240 0x5588ec03b690 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa35f0eacdf38d9 server_cookie=5cc9db0d9d72e1d3 in_seq=0 out_seq=0 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.391+0000 7f85adffb640 10 client.20787 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85adffb640 1 -- 172.21.15.33:0/883739761 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588ebf94bb0 con 0x5588ec039240 2024-04-24T23:38:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85adffb640 10 client.20787 handle_command_reply: tid=0 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 client.20787 shutdown 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 msgr2=0x5588ec02d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec02d8f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f85b40314e0 tx=0x7f85b40044f0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 shutdown_connections 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec039240 0x5588ec03b690 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec0348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec02d8f0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85adffb640 0 client.20787 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 --2- 172.21.15.33:0/883739761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec02d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 >> 172.21.15.33:0/883739761 conn(0x5588ec01e8a0 msgr2=0x5588ec01db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 shutdown_connections 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 1 -- 172.21.15.33:0/883739761 wait complete. 2024-04-24T23:38:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.392+0000 7f85cf041740 20 client.20787 trim_cache size 0 max 16384 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.405+0000 7f85cf041740 1 Processor -- start 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 -- start start 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec023230 0x5588ec021580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023c20 0x5588ec034880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 0x5588ec021ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41aeeb0 con 0x5588ec028d90 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ac750 con 0x5588ec023230 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85cf041740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41aa5a0 con 0x5588ec023c20 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 0x5588ec021ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 0x5588ec021ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52792/0 (socket says 172.21.15.33:52792) 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca0d0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec023230 0x5588ec021580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 -- 172.21.15.33:0/2247381925 learned_addr learned my addr 172.21.15.33:0/2247381925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85c98cf640 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023c20 0x5588ec034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 -- 172.21.15.33:0/2247381925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023c20 msgr2=0x5588ec034880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023c20 0x5588ec034880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 -- 172.21.15.33:0/2247381925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec023230 msgr2=0x5588ec021580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec023230 0x5588ec021580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 -- 172.21.15.33:0/2247381925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ebd657d0 con 0x5588ec028d90 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec023230 0x5588ec021580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 0x5588ec021ac0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f85bc012480 tx=0x7f85bc009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918b64ad66d2437d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.406+0000 7f85affff640 1 -- 172.21.15.33:0/2247381925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc003e00 con 0x5588ec028d90 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85affff640 1 -- 172.21.15.33:0/2247381925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc025070 con 0x5588ec028d90 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85affff640 1 -- 172.21.15.33:0/2247381925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc005020 con 0x5588ec028d90 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 -- 172.21.15.33:0/2247381925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 msgr2=0x5588ec021ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 0x5588ec021ac0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f85bc012480 tx=0x7f85bc009a00 comp rx=0 tx=0).stop 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 -- 172.21.15.33:0/2247381925 shutdown_connections 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec028d90 0x5588ec021ac0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023c20 0x5588ec034880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 --2- 172.21.15.33:0/2247381925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec023230 0x5588ec021580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 -- 172.21.15.33:0/2247381925 >> 172.21.15.33:0/2247381925 conn(0x5588ec01e8a0 msgr2=0x5588ec01db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 -- 172.21.15.33:0/2247381925 shutdown_connections 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 -- 172.21.15.33:0/2247381925 wait complete. 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.407+0000 7f85cf041740 1 -- start start 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 Processor -- start 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec031e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec032370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec033380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41aac10 con 0x5588ec023c20 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ac080 con 0x5588ec028d90 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ab2e0 con 0x5588ec023230 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85ca0d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec031e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec032370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52806/0 (socket says 172.21.15.33:52806) 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85cf041740 10 client.0 _ll_register_callbacks cb 0x7f85b4004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 -- 172.21.15.33:0/50913314 learned_addr learned my addr 172.21.15.33:0/50913314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec033380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 -- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 msgr2=0x5588ec031e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec031e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 -- 172.21.15.33:0/50913314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 msgr2=0x5588ec033380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec033380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 -- 172.21.15.33:0/50913314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ec033970 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85ca8d1640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec033380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85c98cf640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec032370 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f85b40036e0 tx=0x7f85b4003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed7c87275f64fba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.408+0000 7f85adffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85adffb640 1 -- 172.21.15.33:0/50913314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b404c040 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85adffb640 1 -- 172.21.15.33:0/50913314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f85b4003bc0 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85bc025070 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85adffb640 1 -- 172.21.15.33:0/50913314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f85b4061050 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 10 client.20793 fetch_fsmap learned FSMap version 38 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5588ec09f7d0 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85adffb640 1 -- 172.21.15.33:0/50913314 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc025070 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85adffb640 1 -- 172.21.15.33:0/50913314 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f85b4004800 con 0x5588ec023c20 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 10 client.20793 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 10 client.20793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 20 client.20793 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec0a0010 0x5588ec0a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 4 client.20793 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588ec02d1a0 con 0x5588ec0a0010 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.409+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec0a0010 0x5588ec0a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85adffb640 10 client.20793 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85ca0d0640 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec0a0010 0x5588ec0a2430 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4192fa642a9ed6c4 server_cookie=24d5fabf252ae14b in_seq=0 out_seq=0 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85adffb640 1 -- 172.21.15.33:0/50913314 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588ec02d1a0 con 0x5588ec0a0010 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85adffb640 10 client.20793 handle_command_reply: tid=0 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 client.20793 shutdown 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 msgr2=0x5588ec032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec032370 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f85b40036e0 tx=0x7f85b4003990 comp rx=0 tx=0).stop 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 shutdown_connections 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5588ec0a0010 0x5588ec0a2430 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588ec028d90 0x5588ec033380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588ec023c20 0x5588ec032370 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 --2- 172.21.15.33:0/50913314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588ec023230 0x5588ec031e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 >> 172.21.15.33:0/50913314 conn(0x5588ec01e8a0 msgr2=0x5588ec0246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85adffb640 0 client.20793 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 shutdown_connections 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 1 -- 172.21.15.33:0/50913314 wait complete. 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.410+0000 7f85cf041740 20 client.20793 trim_cache size 0 max 16384 2024-04-24T23:38:11.417 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:38:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f85b00a2b90 msgr2=0x7f85b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f85b00a2b90 0x7f85b00a5050 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f85bc005c40 tx=0x7f85bc009500 comp rx=0 tx=0).stop 2024-04-24T23:38:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 msgr2=0x7f85c410aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 0x7f85c410aff0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d770 tx=0x7f85c000dc30 comp rx=0 tx=0).stop 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 shutdown_connections 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f85b00a2b90 0x7f85b00a5050 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c4102220 0x7f85c4109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85c4101860 0x7f85c4109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 --2- 172.21.15.33:0/3991790957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85c4100670 0x7f85c410aff0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 >> 172.21.15.33:0/3991790957 conn(0x7f85c40fbbe0 msgr2=0x7f85c4104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 shutdown_connections 2024-04-24T23:38:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:11.411+0000 7f85cc334640 1 -- 172.21.15.33:0/3991790957 wait complete. 2024-04-24T23:38:11.912 INFO:tasks.workunit.client.0.smithi033.stdout: 5 cleanup 616 sec 2024-04-24T23:38:12.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:11 smithi038 ceph-mon[32450]: pgmap v1136: 129 pgs: 129 active+clean; 560 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 3.6 MiB/s wr, 214 op/s 2024-04-24T23:38:12.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:11 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1799499900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:38:12.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:11 smithi117 ceph-mon[32529]: pgmap v1136: 129 pgs: 129 active+clean; 560 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 3.6 MiB/s wr, 214 op/s 2024-04-24T23:38:12.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:11 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1799499900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:38:12.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:11 smithi033 ceph-mon[29201]: pgmap v1136: 129 pgs: 129 active+clean; 560 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 3.6 MiB/s wr, 214 op/s 2024-04-24T23:38:12.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:11 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1799499900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: 0 cleanup 616 sec 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: Operation Count AvgLat MaxLat 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: ---------------------------------------- 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: NTCreateX 417208 5.946 4043.690 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: Close 306298 0.003 0.267 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: Rename 17655 23.688 531.093 2024-04-24T23:38:12.355 INFO:tasks.workunit.client.0.smithi033.stdout: Unlink 84414 8.888 972.962 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Deltree 2 282.938 409.263 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Mkdir 1 0.015 0.015 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Qpathinfo 378328 4.446 503.028 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Qfileinfo 65937 0.001 0.057 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Qfsinfo 69270 0.336 265.791 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Sfileinfo 34023 0.193 119.086 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Find 146104 0.564 442.575 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: WriteX 206309 0.055 380.919 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: ReadX 653061 0.052 363.816 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: LockX 1354 7.673 174.246 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: UnlockX 1354 6.733 154.214 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: Flush 29259 16.585 497.888 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-04-24T23:38:12.356 INFO:tasks.workunit.client.0.smithi033.stdout:Throughput 21.6541 MB/sec 10 clients 10 procs max_latency=4043.695 ms 2024-04-24T23:38:12.357 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T23:38:12.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T23:38:12.422 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:38:12.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:38:12.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:38:12.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 -- 172.21.15.33:0/218888315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948102230 msgr2=0x7f7948108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 --2- 172.21.15.33:0/218888315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948102230 0x7f7948108ac0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7940009ca0 tx=0x7f794002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 -- 172.21.15.33:0/218888315 shutdown_connections 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 --2- 172.21.15.33:0/218888315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948102230 0x7f7948108ac0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 --2- 172.21.15.33:0/218888315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948101870 0x7f7948101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 --2- 172.21.15.33:0/218888315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7948100680 0x7f7948100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 -- 172.21.15.33:0/218888315 >> 172.21.15.33:0/218888315 conn(0x7f79480fbbf0 msgr2=0x7f79480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 -- 172.21.15.33:0/218888315 shutdown_connections 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.542+0000 7f794e2a5640 1 -- 172.21.15.33:0/218888315 wait complete. 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 Processor -- start 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 -- start start 2024-04-24T23:38:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7948100680 0x7f7948197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 0x7f7948197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948102230 0x7f794819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f794810c300 con 0x7f7948101870 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f794810c180 con 0x7f7948102230 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794e2a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f794810c480 con 0x7f7948100680 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 0x7f7948197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 0x7f7948197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52822/0 (socket says 172.21.15.33:52822) 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 -- 172.21.15.33:0/4203556941 learned_addr learned my addr 172.21.15.33:0/4203556941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794c842640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948102230 0x7f794819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f7947fff640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7948100680 0x7f7948197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 -- 172.21.15.33:0/4203556941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7948100680 msgr2=0x7f7948197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7948100680 0x7f7948197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 -- 172.21.15.33:0/4203556941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948102230 msgr2=0x7f794819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948102230 0x7f794819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f79477fe640 1 -- 172.21.15.33:0/4203556941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794819f680 con 0x7f7948101870 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.543+0000 7f794c842640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948102230 0x7f794819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.544+0000 7f79477fe640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 0x7f7948197fa0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e9f0 tx=0x7f793c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.544+0000 7f79457fa640 1 -- 172.21.15.33:0/4203556941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00cd80 con 0x7f7948101870 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.544+0000 7f79457fa640 1 -- 172.21.15.33:0/4203556941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c010910 con 0x7f7948101870 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.544+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794819f970 con 0x7f7948101870 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.544+0000 7f79457fa640 1 -- 172.21.15.33:0/4203556941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c018990 con 0x7f7948101870 2024-04-24T23:38:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.544+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794819ff10 con 0x7f7948101870 2024-04-24T23:38:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.546+0000 7f79457fa640 1 -- 172.21.15.33:0/4203556941 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f793c010ac0 con 0x7f7948101870 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.546+0000 7f7950fb3740 1 Processor -- start 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.546+0000 7f7950fb3740 1 -- start start 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x5654839565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 0x56548395b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x56548395e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7950fb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5654838b52c0 con 0x56548395bd00 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7950fb3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56548385d910 con 0x565483956b90 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7950fb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654838b1280 con 0x5654839561a0 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79457fa640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f79300a2c10 0x7f79300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x56548395e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f794c842640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x5654839565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7947fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 0x56548395b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f794c842640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x5654839565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53822/0 (socket says 172.21.15.33:53822) 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f794c842640 1 -- 172.21.15.33:0/582059523 learned_addr learned my addr 172.21.15.33:0/582059523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7947fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 0x56548395b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60930/0 (socket says 172.21.15.33:60930) 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f794c842640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f79300a2c10 0x7f79300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79457fa640 1 -- 172.21.15.33:0/4203556941 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f793c0cf6a0 con 0x7f7948101870 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 -- 172.21.15.33:0/582059523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 msgr2=0x5654839565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x5654839565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 -- 172.21.15.33:0/582059523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 msgr2=0x56548395b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 0x56548395b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 -- 172.21.15.33:0/582059523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548386efb0 con 0x56548395bd00 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f7947fff640 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 0x56548395b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f794c842640 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x5654839565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.547+0000 7f79477fe640 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x56548395e190 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c6a0 tx=0x7f793c06ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f056940fc6e7a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f794c842640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f79300a2c10 0x7f79300a50d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5654839573f0 tx=0x7f7940005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f792e7fc640 1 -- 172.21.15.33:0/582059523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0a4780 con 0x56548395bd00 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f792e7fc640 1 -- 172.21.15.33:0/582059523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c0ccec0 con 0x56548395bd00 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f792e7fc640 1 -- 172.21.15.33:0/582059523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0a4550 con 0x56548395bd00 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 -- 172.21.15.33:0/582059523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 msgr2=0x56548395e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x56548395e190 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c6a0 tx=0x7f793c06ed40 comp rx=0 tx=0).stop 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 -- 172.21.15.33:0/582059523 shutdown_connections 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x56548395e190 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483956b90 0x56548395b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 --2- 172.21.15.33:0/582059523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x5654839565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 -- 172.21.15.33:0/582059523 >> 172.21.15.33:0/582059523 conn(0x565483919950 msgr2=0x56548394d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 -- 172.21.15.33:0/582059523 shutdown_connections 2024-04-24T23:38:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.548+0000 7f7950fb3740 1 -- 172.21.15.33:0/582059523 wait complete. 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 -- start start 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 Processor -- start 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x565483960710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x565483960c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56548395bd00 0x565483967c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56548395ec50 con 0x565483956b90 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x565483870e90 con 0x5654839561a0 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56548395edd0 con 0x56548395bd00 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7950fb3740 10 client.0 _ll_register_callbacks cb 0x5654838ac970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x565483960c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x565483960c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52842/0 (socket says 172.21.15.33:52842) 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f79477fe640 1 -- 172.21.15.33:0/3318694900 learned_addr learned my addr 172.21.15.33:0/3318694900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.549+0000 7f7947fff640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x565483960710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f794c842640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56548395bd00 0x565483967c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79477fe640 1 -- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56548395bd00 msgr2=0x565483967c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79477fe640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56548395bd00 0x565483967c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79477fe640 1 -- 172.21.15.33:0/3318694900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 msgr2=0x565483960710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79477fe640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x565483960710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79477fe640 1 -- 172.21.15.33:0/3318694900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654836917e0 con 0x565483956b90 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f794c842640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56548395bd00 0x565483967c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79477fe640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x565483960c50 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f793c0a4250 tx=0x7f793c0a4500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ae3cc8c379daf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79277fe640 1 -- 172.21.15.33:0/3318694900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0a4570 con 0x565483956b90 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79277fe640 1 -- 172.21.15.33:0/3318694900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f793c018b30 con 0x565483956b90 2024-04-24T23:38:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793c0ccec0 con 0x565483956b90 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79277fe640 1 -- 172.21.15.33:0/3318694900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f793c09be20 con 0x565483956b90 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f7950fb3740 10 client.20805 fetch_fsmap learned FSMap version 39 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5654836f1f70 con 0x565483956b90 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.550+0000 7f79277fe640 1 -- 172.21.15.33:0/3318694900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c0ccec0 con 0x565483956b90 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f79277fe640 1 -- 172.21.15.33:0/3318694900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f793c09b010 con 0x565483956b90 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7950fb3740 10 client.20805 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7950fb3740 10 client.20805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7950fb3740 20 client.20805 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7950fb3740 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56548396bdf0 0x56548396e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7950fb3740 4 client.20805 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654838895a0 con 0x56548396bdf0 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7947fff640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56548396bdf0 0x56548396e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f79277fe640 10 client.20805 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.551+0000 7f7947fff640 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56548396bdf0 0x56548396e1e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a22420315d843c server_cookie=18efe479f742d47f in_seq=0 out_seq=0 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f79277fe640 1 -- 172.21.15.33:0/3318694900 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654838895a0 con 0x56548396bdf0 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f79277fe640 10 client.20805 handle_command_reply: tid=0 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 client.20805 shutdown 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 msgr2=0x565483960c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x565483960c50 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f793c0a4250 tx=0x7f793c0a4500 comp rx=0 tx=0).stop 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 shutdown_connections 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x56548396bdf0 0x56548396e1e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56548395bd00 0x565483967c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x565483960c50 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f79277fe640 0 client.20805 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 --2- 172.21.15.33:0/3318694900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x565483960710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 >> 172.21.15.33:0/3318694900 conn(0x565483919950 msgr2=0x56548394c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 shutdown_connections 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 1 -- 172.21.15.33:0/3318694900 wait complete. 2024-04-24T23:38:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.552+0000 7f7950fb3740 20 client.20805 trim_cache size 0 max 16384 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 Processor -- start 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 -- start start 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x56548394c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x56548394cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56548395bd00 0x565483967c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79481aaab0 con 0x565483956b90 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79481a8320 con 0x56548395bd00 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7950fb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79481a6170 con 0x5654839561a0 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x56548394cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f79477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x56548394cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52852/0 (socket says 172.21.15.33:52852) 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f79477fe640 1 -- 172.21.15.33:0/1295016546 learned_addr learned my addr 172.21.15.33:0/1295016546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f7947fff640 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x56548394c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.570+0000 7f794c842640 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56548395bd00 0x565483967c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f79477fe640 1 -- 172.21.15.33:0/1295016546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 msgr2=0x56548394c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f79477fe640 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x56548394c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f79477fe640 1 -- 172.21.15.33:0/1295016546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56548395bd00 msgr2=0x565483967c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f79477fe640 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56548395bd00 0x565483967c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f79477fe640 1 -- 172.21.15.33:0/1295016546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565483589170 con 0x565483956b90 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f79477fe640 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x56548394cd10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f793c0ccdb0 tx=0x7f793c09bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb1de1da4176610 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f792dffb640 1 -- 172.21.15.33:0/1295016546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c09bfc0 con 0x565483956b90 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f792dffb640 1 -- 172.21.15.33:0/1295016546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c097b60 con 0x565483956b90 2024-04-24T23:38:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.571+0000 7f792dffb640 1 -- 172.21.15.33:0/1295016546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0ba500 con 0x565483956b90 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- 172.21.15.33:0/1295016546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 msgr2=0x56548394cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x56548394cd10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f793c0ccdb0 tx=0x7f793c09bbc0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- 172.21.15.33:0/1295016546 shutdown_connections 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56548395bd00 0x565483967c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565483956b90 0x56548394cd10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1295016546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654839561a0 0x56548394c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- 172.21.15.33:0/1295016546 >> 172.21.15.33:0/1295016546 conn(0x565483919950 msgr2=0x56548394c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- 172.21.15.33:0/1295016546 shutdown_connections 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- 172.21.15.33:0/1295016546 wait complete. 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- start start 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 Processor -- start 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x5654839675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565483956b90 0x565483967b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x565483965c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79481a67e0 con 0x56548395bd00 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79481a7c50 con 0x5654839561a0 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79481a6eb0 con 0x565483956b90 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f7950fb3740 10 client.0 _ll_register_callbacks cb 0x7f7938009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.572+0000 7f794c842640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x565483965c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x565483965c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52860/0 (socket says 172.21.15.33:52860) 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f7947fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x5654839675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f79477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565483956b90 0x565483967b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 -- 172.21.15.33:0/1953683050 learned_addr learned my addr 172.21.15.33:0/1953683050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 -- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565483956b90 msgr2=0x565483967b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565483956b90 0x565483967b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 -- 172.21.15.33:0/1953683050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 msgr2=0x5654839675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x5654839675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 -- 172.21.15.33:0/1953683050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565483966450 con 0x56548395bd00 2024-04-24T23:38:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f794c842640 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x565483965c50 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f7940004af0 tx=0x7f79400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a829b06ff1590a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f79277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f79277fe640 1 -- 172.21.15.33:0/1953683050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940058060 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f79277fe640 1 -- 172.21.15.33:0/1953683050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7940030a10 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793c097b60 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f79277fe640 1 -- 172.21.15.33:0/1953683050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7940030c10 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f7950fb3740 10 client.20811 fetch_fsmap learned FSMap version 39 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.573+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5654839d2d40 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f79277fe640 1 -- 172.21.15.33:0/1953683050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c097b60 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f79277fe640 1 -- 172.21.15.33:0/1953683050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7940052070 con 0x56548395bd00 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7950fb3740 10 client.20811 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7950fb3740 10 client.20811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7950fb3740 20 client.20811 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654839d3720 0x5654839d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7950fb3740 4 client.20811 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5654838c2800 con 0x5654839d3720 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7947fff640 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654839d3720 0x5654839d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f79277fe640 10 client.20811 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.574+0000 7f7947fff640 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654839d3720 0x5654839d5b40 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60a172ca8ca3676 server_cookie=f8e138a4311a5e7a in_seq=0 out_seq=0 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.575+0000 7f79277fe640 1 -- 172.21.15.33:0/1953683050 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5654838c2800 con 0x5654839d3720 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.575+0000 7f79277fe640 10 client.20811 handle_command_reply: tid=0 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.575+0000 7f7950fb3740 1 client.20811 shutdown 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.575+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 msgr2=0x565483965c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.575+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x565483965c50 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f7940004af0 tx=0x7f79400095c0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 shutdown_connections 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x5654839d3720 0x5654839d5b40 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f79277fe640 0 client.20811 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56548395bd00 0x565483965c50 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565483956b90 0x565483967b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 --2- 172.21.15.33:0/1953683050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654839561a0 0x5654839675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 >> 172.21.15.33:0/1953683050 conn(0x565483919950 msgr2=0x56548394c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 shutdown_connections 2024-04-24T23:38:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 1 -- 172.21.15.33:0/1953683050 wait complete. 2024-04-24T23:38:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f7950fb3740 20 client.20811 trim_cache size 0 max 16384 2024-04-24T23:38:12.584 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-24T23:38:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "57756a3f-8b4b-41a6-9d84-392b32cb9416", 2024-04-24T23:38:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-24T23:38:12.585 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:12.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f79300a2c10 msgr2=0x7f79300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f79300a2c10 0x7f79300a50d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5654839573f0 tx=0x7f7940005d80 comp rx=0 tx=0).stop 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 msgr2=0x7f7948197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 0x7f7948197fa0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e9f0 tx=0x7f793c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 shutdown_connections 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f79300a2c10 0x7f79300a50d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7948102230 0x7f794819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7948101870 0x7f7948197fa0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 --2- 172.21.15.33:0/4203556941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7948100680 0x7f7948197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 >> 172.21.15.33:0/4203556941 conn(0x7f79480fbbf0 msgr2=0x7f79480fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 shutdown_connections 2024-04-24T23:38:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.576+0000 7f794e2a5640 1 -- 172.21.15.33:0/4203556941 wait complete. 2024-04-24T23:38:12.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:38:12.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.694+0000 7fd07ef34640 1 -- 172.21.15.33:0/672083222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078101870 msgr2=0x7fd078101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.694+0000 7fd07ef34640 1 --2- 172.21.15.33:0/672083222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078101870 0x7fd078101cf0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd064009900 tx=0x7fd06402a410 comp rx=0 tx=0).stop 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- 172.21.15.33:0/672083222 shutdown_connections 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 --2- 172.21.15.33:0/672083222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102230 0x7fd078108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 --2- 172.21.15.33:0/672083222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078101870 0x7fd078101cf0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 --2- 172.21.15.33:0/672083222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078100680 0x7fd078100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- 172.21.15.33:0/672083222 >> 172.21.15.33:0/672083222 conn(0x7fd0780fbbf0 msgr2=0x7fd0780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- 172.21.15.33:0/672083222 shutdown_connections 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- 172.21.15.33:0/672083222 wait complete. 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 Processor -- start 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- start start 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 0x7fd0780694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101870 0x7fd07819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102230 0x7fd07819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810c2e0 con 0x7fd078100680 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810c160 con 0x7fd078102230 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ef34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd07810c460 con 0x7fd078101870 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ccd0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 0x7fd0780694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ccd0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 0x7fd0780694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52888/0 (socket says 172.21.15.33:52888) 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd07ccd0640 1 -- 172.21.15.33:0/2077806633 learned_addr learned my addr 172.21.15.33:0/2077806633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.695+0000 7fd077fff640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101870 0x7fd07819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102230 0x7fd07819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ccd0640 1 -- 172.21.15.33:0/2077806633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101870 msgr2=0x7fd07819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101870 0x7fd07819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ccd0640 1 -- 172.21.15.33:0/2077806633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102230 msgr2=0x7fd07819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102230 0x7fd07819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ccd0640 1 -- 172.21.15.33:0/2077806633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07819f620 con 0x7fd078100680 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 0x7fd0780694a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c970 tx=0x7fd06c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd075ffb640 1 -- 172.21.15.33:0/2077806633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c007c10 con 0x7fd078100680 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07819f910 con 0x7fd078100680 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd075ffb640 1 -- 172.21.15.33:0/2077806633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06c007db0 con 0x7fd078100680 2024-04-24T23:38:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd075ffb640 1 -- 172.21.15.33:0/2077806633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00f8a0 con 0x7fd078100680 2024-04-24T23:38:12.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.696+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07819fe80 con 0x7fd078100680 2024-04-24T23:38:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd075ffb640 1 -- 172.21.15.33:0/2077806633 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd06c00fa60 con 0x7fd078100680 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 Processor -- start 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 -- start start 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876466660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b87646b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b87646e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b8763ca6b0 con 0x55b87646bda0 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b876372880 con 0x55b876466240 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.699+0000 7fd081c43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b8763c6670 con 0x55b876466c30 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876466660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd075ffb640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0680a2b90 0x7fd0680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07ccd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b87646b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b87646e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876466660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.33:60974/0 (socket says 172.21.15.33:60974) 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 -- 172.21.15.33:0/1516342182 learned_addr learned my addr 172.21.15.33:0/1516342182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0680a2b90 0x7fd0680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 -- 172.21.15.33:0/1516342182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 msgr2=0x55b87646b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b87646b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 -- 172.21.15.33:0/1516342182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 msgr2=0x55b87646e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b87646e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 -- 172.21.15.33:0/1516342182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8761a87d0 con 0x55b876466240 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd077fff640 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b87646e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd075ffb640 1 -- 172.21.15.33:0/2077806633 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd06c0cf6d0 con 0x7fd078100680 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876466660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd070005f40 tx=0x7fd070014f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9ebac9897b746 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd062ffd640 1 -- 172.21.15.33:0/1516342182 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070012d90 con 0x55b876466240 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0680a2b90 0x7fd0680a5050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd070022000 tx=0x7fd070020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd062ffd640 1 -- 172.21.15.33:0/1516342182 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070023070 con 0x55b876466240 2024-04-24T23:38:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.700+0000 7fd062ffd640 1 -- 172.21.15.33:0/1516342182 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070005750 con 0x55b876466240 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 -- 172.21.15.33:0/1516342182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 msgr2=0x55b876466660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876466660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd070005f40 tx=0x7fd070014f70 comp rx=0 tx=0).stop 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 -- 172.21.15.33:0/1516342182 shutdown_connections 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b87646e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b87646b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 --2- 172.21.15.33:0/1516342182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876466660 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 -- 172.21.15.33:0/1516342182 >> 172.21.15.33:0/1516342182 conn(0x55b8764618b0 msgr2=0x55b876461cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 -- 172.21.15.33:0/1516342182 shutdown_connections 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 -- 172.21.15.33:0/1516342182 wait complete. 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 -- start start 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.701+0000 7fd081c43740 1 Processor -- start 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876470660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876470ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876477be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b87646ed50 con 0x55b87646bda0 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b876385f50 con 0x55b876466c30 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b87646eed0 con 0x55b876466240 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd081c43740 10 client.0 _ll_register_callbacks cb 0x55b8763c1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876477be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07ccd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876470660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876477be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52908/0 (socket says 172.21.15.33:52908) 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd077fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876470ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 -- 172.21.15.33:0/3075146104 learned_addr learned my addr 172.21.15.33:0/3075146104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 -- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 msgr2=0x55b876470660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876470660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 -- 172.21.15.33:0/3075146104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 msgr2=0x55b876470ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876470ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 -- 172.21.15.33:0/3075146104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8760de3f0 con 0x55b87646bda0 2024-04-24T23:38:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876477be0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd070012820 tx=0x7fd070005110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea77f5fda991966b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.702+0000 7fd057fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd057fff640 1 -- 172.21.15.33:0/3075146104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070014590 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd057fff640 1 -- 172.21.15.33:0/3075146104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd070004c50 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd070023070 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd057fff640 1 -- 172.21.15.33:0/3075146104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd070004e50 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 10 client.20823 fetch_fsmap learned FSMap version 39 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b87609e0f0 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd057fff640 1 -- 172.21.15.33:0/3075146104 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070023070 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd057fff640 1 -- 172.21.15.33:0/3075146104 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd07000f040 con 0x55b87646bda0 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 10 client.20823 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 10 client.20823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 20 client.20823 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b87647c360 0x55b87647e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 4 client.20823 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8763d7bf0 con 0x55b87647c360 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.703+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b87647c360 0x55b87647e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.704+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b87647c360 0x55b87647e790 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc333dd79f7836ad server_cookie=c68f4ca839a97cb9 in_seq=0 out_seq=0 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.704+0000 7fd057fff640 10 client.20823 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.704+0000 7fd057fff640 1 -- 172.21.15.33:0/3075146104 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8763d7bf0 con 0x55b87647c360 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.704+0000 7fd057fff640 10 client.20823 handle_command_reply: tid=0 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.704+0000 7fd081c43740 1 client.20823 shutdown 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 msgr2=0x55b876477be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876477be0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd070012820 tx=0x7fd070005110 comp rx=0 tx=0).stop 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 shutdown_connections 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b87647c360 0x55b87647e790 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876477be0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876470ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd057fff640 0 client.20823 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 --2- 172.21.15.33:0/3075146104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876470660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 >> 172.21.15.33:0/3075146104 conn(0x55b8764618b0 msgr2=0x55b876460af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 shutdown_connections 2024-04-24T23:38:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 1 -- 172.21.15.33:0/3075146104 wait complete. 2024-04-24T23:38:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.705+0000 7fd081c43740 20 client.20823 trim_cache size 0 max 16384 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 Processor -- start 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 -- start start 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876461c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b876477ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876464630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781acc70 con 0x55b87646bda0 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781aa4e0 con 0x55b876466240 2024-04-24T23:38:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd081c43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a8330 con 0x55b876466c30 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876464630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876464630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52924/0 (socket says 172.21.15.33:52924) 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 -- 172.21.15.33:0/378148734 learned_addr learned my addr 172.21.15.33:0/378148734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd077fff640 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b876477ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876461c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 -- 172.21.15.33:0/378148734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 msgr2=0x55b876477ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b876477ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 -- 172.21.15.33:0/378148734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 msgr2=0x55b876461c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876461c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 -- 172.21.15.33:0/378148734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b876453090 con 0x55b87646bda0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876461c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876464630 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097d0 tx=0x7fd0700128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09f8c841309e2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.723+0000 7fd0627fc640 1 -- 172.21.15.33:0/378148734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070004850 con 0x55b87646bda0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd0627fc640 1 -- 172.21.15.33:0/378148734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07002b020 con 0x55b87646bda0 2024-04-24T23:38:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd0627fc640 1 -- 172.21.15.33:0/378148734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070027680 con 0x55b87646bda0 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 -- 172.21.15.33:0/378148734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 msgr2=0x55b876464630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876464630 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097d0 tx=0x7fd0700128d0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 -- 172.21.15.33:0/378148734 shutdown_connections 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876464630 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466c30 0x55b876477ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 --2- 172.21.15.33:0/378148734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466240 0x55b876461c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 -- 172.21.15.33:0/378148734 >> 172.21.15.33:0/378148734 conn(0x55b8764618b0 msgr2=0x55b876470660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 -- 172.21.15.33:0/378148734 shutdown_connections 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 -- 172.21.15.33:0/378148734 wait complete. 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.724+0000 7fd081c43740 1 -- start start 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 Processor -- start 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876474c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876475170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876476180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a89a0 con 0x55b87646bda0 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a9e10 con 0x55b876466c30 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781a9070 con 0x55b876466240 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd081c43740 10 client.0 _ll_register_callbacks cb 0x7fd064009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876476180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07ccd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876474c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876476180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52926/0 (socket says 172.21.15.33:52926) 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 -- 172.21.15.33:0/983723574 learned_addr learned my addr 172.21.15.33:0/983723574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07ccd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876474c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:53904/0 (socket says 172.21.15.33:53904) 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd077fff640 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876475170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 -- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 msgr2=0x55b876474c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876474c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 -- 172.21.15.33:0/983723574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 msgr2=0x55b876475170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:12.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876475170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.725+0000 7fd07d4d1640 1 -- 172.21.15.33:0/983723574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b876476770 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd07d4d1640 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876476180 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd070006010 tx=0x7fd070009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bad0928ebbe6b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd057fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd057fff640 1 -- 172.21.15.33:0/983723574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070003b80 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd057fff640 1 -- 172.21.15.33:0/983723574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd070003d20 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07002b020 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd057fff640 1 -- 172.21.15.33:0/983723574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd070052050 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 10 client.20829 fetch_fsmap learned FSMap version 39 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8764779f0 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd057fff640 1 -- 172.21.15.33:0/983723574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07002b020 con 0x55b87646bda0 2024-04-24T23:38:12.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd057fff640 1 -- 172.21.15.33:0/983723574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd070047020 con 0x55b87646bda0 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 10 client.20829 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 10 client.20829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 20 client.20829 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b8764e2a20 0x55b8764e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 4 client.20829 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.726+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8763d1f40 con 0x55b8764e2a20 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.727+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b8764e2a20 0x55b8764e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.727+0000 7fd057fff640 10 client.20829 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.727+0000 7fd07ccd0640 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b8764e2a20 0x55b8764e4e40 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9712f577df19c332 server_cookie=dcafa121aa3c6cb0 in_seq=0 out_seq=0 2024-04-24T23:38:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd057fff640 1 -- 172.21.15.33:0/983723574 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b8763d1f40 con 0x55b8764e2a20 2024-04-24T23:38:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd057fff640 10 client.20829 handle_command_reply: tid=0 2024-04-24T23:38:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 client.20829 shutdown 2024-04-24T23:38:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 msgr2=0x55b876476180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876476180 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd070006010 tx=0x7fd070009a70 comp rx=0 tx=0).stop 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 shutdown_connections 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55b8764e2a20 0x55b8764e4e40 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b87646bda0 0x55b876476180 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b876466c30 0x55b876475170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 --2- 172.21.15.33:0/983723574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b876466240 0x55b876474c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 >> 172.21.15.33:0/983723574 conn(0x55b8764618b0 msgr2=0x55b876460260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd057fff640 0 client.20829 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 shutdown_connections 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 1 -- 172.21.15.33:0/983723574 wait complete. 2024-04-24T23:38:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.728+0000 7fd081c43740 20 client.20829 trim_cache size 0 max 16384 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "57756a3f-8b4b-41a6-9d84-392b32cb9416": { 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "57756a3f-8b4b-41a6-9d84-392b32cb9416", 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:12.735 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0680a2b90 msgr2=0x7fd0680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0680a2b90 0x7fd0680a5050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd070022000 tx=0x7fd070020000 comp rx=0 tx=0).stop 2024-04-24T23:38:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 msgr2=0x7fd0780694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 0x7fd0780694a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c970 tx=0x7fd06c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 shutdown_connections 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fd0680a2b90 0x7fd0680a5050 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd078102230 0x7fd07819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101870 0x7fd07819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 --2- 172.21.15.33:0/2077806633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078100680 0x7fd0780694a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 >> 172.21.15.33:0/2077806633 conn(0x7fd0780fbbf0 msgr2=0x7fd078104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 shutdown_connections 2024-04-24T23:38:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:12.729+0000 7fd07ef34640 1 -- 172.21.15.33:0/2077806633 wait complete. 2024-04-24T23:38:12.740 INFO:tasks.cephfs.filesystem:scrub status for tag:57756a3f-8b4b-41a6-9d84-392b32cb9416 - {'path': '/', 'tag': '57756a3f-8b4b-41a6-9d84-392b32cb9416', 'options': 'recursive,force'} 2024-04-24T23:38:13.060 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:12 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:13.061 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-24T23:38:13.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T23:38:13.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:12 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:13.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:12 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:13.448 DEBUG:teuthology.parallel:result is None 2024-04-24T23:38:13.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:38:13.489 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:38:13.489 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-24T23:38:13.500 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-24T23:38:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: pgmap v1137: 129 pgs: 129 active+clean; 526 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 3.7 MiB/s wr, 185 op/s 2024-04-24T23:38:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub queued for path: / 2024-04-24T23:38:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:38:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:38:14.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:38:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: active paths [/] 2024-04-24T23:38:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:13 smithi038 ceph-mon[32450]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: pgmap v1137: 129 pgs: 129 active+clean; 526 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 3.7 MiB/s wr, 185 op/s 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub queued for path: / 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: active paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:13 smithi117 ceph-mon[32529]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: pgmap v1137: 129 pgs: 129 active+clean; 526 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 3.7 MiB/s wr, 185 op/s 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub queued for path: / 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: active paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:14.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:13 smithi033 ceph-mon[29201]: scrub summary: idle+waiting paths [/] 2024-04-24T23:38:16.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:15 smithi038 ceph-mon[32450]: pgmap v1138: 129 pgs: 129 active+clean; 515 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 3.1 MiB/s wr, 174 op/s 2024-04-24T23:38:16.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:15 smithi117 ceph-mon[32529]: pgmap v1138: 129 pgs: 129 active+clean; 515 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 3.1 MiB/s wr, 174 op/s 2024-04-24T23:38:16.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:15 smithi033 ceph-mon[29201]: pgmap v1138: 129 pgs: 129 active+clean; 515 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 3.1 MiB/s wr, 174 op/s 2024-04-24T23:38:18.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:17 smithi038 ceph-mon[32450]: pgmap v1139: 129 pgs: 129 active+clean; 497 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-04-24T23:38:18.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:17 smithi117 ceph-mon[32529]: pgmap v1139: 129 pgs: 129 active+clean; 497 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-04-24T23:38:18.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:17 smithi033 ceph-mon[29201]: pgmap v1139: 129 pgs: 129 active+clean; 497 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-04-24T23:38:19.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:18 smithi038 ceph-mon[32450]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:19.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:18 smithi117 ceph-mon[32529]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:18 smithi033 ceph-mon[29201]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:38:20.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:19 smithi038 ceph-mon[32450]: pgmap v1140: 129 pgs: 129 active+clean; 494 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 3.0 MiB/s wr, 186 op/s 2024-04-24T23:38:20.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:19 smithi117 ceph-mon[32529]: pgmap v1140: 129 pgs: 129 active+clean; 494 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 3.0 MiB/s wr, 186 op/s 2024-04-24T23:38:20.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:19 smithi033 ceph-mon[29201]: pgmap v1140: 129 pgs: 129 active+clean; 494 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 3.0 MiB/s wr, 186 op/s 2024-04-24T23:38:22.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:21 smithi038 ceph-mon[32450]: pgmap v1141: 129 pgs: 129 active+clean; 490 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-04-24T23:38:22.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:21 smithi117 ceph-mon[32529]: pgmap v1141: 129 pgs: 129 active+clean; 490 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-04-24T23:38:22.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:21 smithi033 ceph-mon[29201]: pgmap v1141: 129 pgs: 129 active+clean; 490 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-04-24T23:38:24.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:23 smithi038 ceph-mon[32450]: pgmap v1142: 129 pgs: 129 active+clean; 483 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.8 MiB/s wr, 184 op/s 2024-04-24T23:38:24.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:23 smithi117 ceph-mon[32529]: pgmap v1142: 129 pgs: 129 active+clean; 483 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.8 MiB/s wr, 184 op/s 2024-04-24T23:38:24.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:23 smithi033 ceph-mon[29201]: pgmap v1142: 129 pgs: 129 active+clean; 483 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.8 MiB/s wr, 184 op/s 2024-04-24T23:38:26.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:25 smithi038 ceph-mon[32450]: pgmap v1143: 129 pgs: 129 active+clean; 473 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.3 MiB/s wr, 160 op/s 2024-04-24T23:38:26.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:25 smithi117 ceph-mon[32529]: pgmap v1143: 129 pgs: 129 active+clean; 473 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.3 MiB/s wr, 160 op/s 2024-04-24T23:38:26.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:25 smithi033 ceph-mon[29201]: pgmap v1143: 129 pgs: 129 active+clean; 473 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.3 MiB/s wr, 160 op/s 2024-04-24T23:38:28.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:27 smithi038 ceph-mon[32450]: pgmap v1144: 129 pgs: 129 active+clean; 445 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 4.0 MiB/s wr, 174 op/s 2024-04-24T23:38:28.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:27 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:38:28.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:27 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:38:28.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:27 smithi038 ceph-mon[32450]: scrub summary: idle 2024-04-24T23:38:28.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:27 smithi117 ceph-mon[32529]: pgmap v1144: 129 pgs: 129 active+clean; 445 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 4.0 MiB/s wr, 174 op/s 2024-04-24T23:38:28.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:27 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:38:28.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:27 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:38:28.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:27 smithi117 ceph-mon[32529]: scrub summary: idle 2024-04-24T23:38:28.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:27 smithi033 ceph-mon[29201]: pgmap v1144: 129 pgs: 129 active+clean; 445 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 4.0 MiB/s wr, 174 op/s 2024-04-24T23:38:28.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:27 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:38:28.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:27 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:38:28.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:27 smithi033 ceph-mon[29201]: scrub summary: idle 2024-04-24T23:38:30.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:29 smithi038 ceph-mon[32450]: pgmap v1145: 129 pgs: 129 active+clean; 437 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-04-24T23:38:30.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:29 smithi117 ceph-mon[32529]: pgmap v1145: 129 pgs: 129 active+clean; 437 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-04-24T23:38:30.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:29 smithi033 ceph-mon[29201]: pgmap v1145: 129 pgs: 129 active+clean; 437 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-04-24T23:38:31.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:38:31.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:38:31.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:38:31.170 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:30 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:38:31.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:38:31.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:38:31.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:38:31.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:30 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:38:31.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:38:31.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:38:31.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:38:31.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:30 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' 2024-04-24T23:38:32.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:31 smithi038 ceph-mon[32450]: pgmap v1146: 129 pgs: 129 active+clean; 422 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.6 MiB/s wr, 144 op/s 2024-04-24T23:38:32.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:31 smithi117 ceph-mon[32529]: pgmap v1146: 129 pgs: 129 active+clean; 422 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.6 MiB/s wr, 144 op/s 2024-04-24T23:38:32.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:31 smithi033 ceph-mon[29201]: pgmap v1146: 129 pgs: 129 active+clean; 422 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.6 MiB/s wr, 144 op/s 2024-04-24T23:38:34.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:33 smithi038 ceph-mon[32450]: pgmap v1147: 129 pgs: 129 active+clean; 417 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.8 MiB/s wr, 123 op/s 2024-04-24T23:38:34.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:33 smithi117 ceph-mon[32529]: pgmap v1147: 129 pgs: 129 active+clean; 417 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.8 MiB/s wr, 123 op/s 2024-04-24T23:38:34.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:33 smithi033 ceph-mon[29201]: pgmap v1147: 129 pgs: 129 active+clean; 417 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.8 MiB/s wr, 123 op/s 2024-04-24T23:38:36.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:35 smithi038 ceph-mon[32450]: pgmap v1148: 129 pgs: 129 active+clean; 413 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-04-24T23:38:36.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:35 smithi117 ceph-mon[32529]: pgmap v1148: 129 pgs: 129 active+clean; 413 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-04-24T23:38:36.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:35 smithi033 ceph-mon[29201]: pgmap v1148: 129 pgs: 129 active+clean; 413 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-04-24T23:38:38.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:37 smithi038 ceph-mon[32450]: pgmap v1149: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-04-24T23:38:38.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:37 smithi117 ceph-mon[32529]: pgmap v1149: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-04-24T23:38:38.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:37 smithi033 ceph-mon[29201]: pgmap v1149: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-04-24T23:38:40.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:39 smithi038 ceph-mon[32450]: pgmap v1150: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 746 KiB/s wr, 59 op/s 2024-04-24T23:38:40.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:39 smithi117 ceph-mon[32529]: pgmap v1150: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 746 KiB/s wr, 59 op/s 2024-04-24T23:38:40.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:39 smithi033 ceph-mon[29201]: pgmap v1150: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 746 KiB/s wr, 59 op/s 2024-04-24T23:38:42.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:41 smithi038 ceph-mon[32450]: pgmap v1151: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 465 KiB/s wr, 45 op/s 2024-04-24T23:38:42.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:41 smithi117 ceph-mon[32529]: pgmap v1151: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 465 KiB/s wr, 45 op/s 2024-04-24T23:38:42.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:41 smithi033 ceph-mon[29201]: pgmap v1151: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 465 KiB/s wr, 45 op/s 2024-04-24T23:38:42.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:38:42.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:38:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/1990726235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 msgr2=0x7fdd68102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/1990726235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68102e10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009900 tx=0x7fdd5402a410 comp rx=0 tx=0).stop 2024-04-24T23:38:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/1990726235 shutdown_connections 2024-04-24T23:38:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/1990726235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd68103350 0x7fdd68109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/1990726235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68102e10 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/1990726235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd681017a0 0x7fdd68101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/1990726235 >> 172.21.15.33:0/1990726235 conn(0x7fdd680fcd10 msgr2=0x7fdd680ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/1990726235 shutdown_connections 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.859+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/1990726235 wait complete. 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 Processor -- start 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 -- start start 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681017a0 0x7fdd68197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd68103350 0x7fdd6819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6810d400 con 0x7fdd68102990 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6810d280 con 0x7fdd681017a0 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ecf5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6810d580 con 0x7fdd68103350 2024-04-24T23:38:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45384/0 (socket says 172.21.15.33:45384) 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd67fff640 1 -- 172.21.15.33:0/2619523383 learned_addr learned my addr 172.21.15.33:0/2619523383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681017a0 0x7fdd68197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.860+0000 7fdd6d292640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd68103350 0x7fdd6819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd67fff640 1 -- 172.21.15.33:0/2619523383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd68103350 msgr2=0x7fdd6819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd67fff640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd68103350 0x7fdd6819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd67fff640 1 -- 172.21.15.33:0/2619523383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681017a0 msgr2=0x7fdd68197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd67fff640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681017a0 0x7fdd68197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd67fff640 1 -- 172.21.15.33:0/2619523383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6819f6a0 con 0x7fdd68102990 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd67fff640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68197fc0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009230 tx=0x7fdd54004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd65ffb640 1 -- 172.21.15.33:0/2619523383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5403f030 con 0x7fdd68102990 2024-04-24T23:38:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd65ffb640 1 -- 172.21.15.33:0/2619523383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54003ce0 con 0x7fdd68102990 2024-04-24T23:38:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6819f930 con 0x7fdd68102990 2024-04-24T23:38:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd65ffb640 1 -- 172.21.15.33:0/2619523383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54035070 con 0x7fdd68102990 2024-04-24T23:38:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.861+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6819fe20 con 0x7fdd68102990 2024-04-24T23:38:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.863+0000 7fdd65ffb640 1 -- 172.21.15.33:0/2619523383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdd54042030 con 0x7fdd68102990 2024-04-24T23:38:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 Processor -- start 2024-04-24T23:38:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 -- start start 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 0x5596142f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd65ffb640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd580a2b90 0x7fdd580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x5596142fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142feda0 0x559614301230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55961425d6b0 con 0x5596142f9240 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559614205880 con 0x5596142f9c30 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd71a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559614259670 con 0x5596142feda0 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd6d292640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 0x5596142f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd580a2b90 0x7fdd580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd6d292640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 0x5596142f9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45392/0 (socket says 172.21.15.33:45392) 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd6d292640 1 -- 172.21.15.33:0/3689997343 learned_addr learned my addr 172.21.15.33:0/3689997343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.864+0000 7fdd65ffb640 1 -- 172.21.15.33:0/2619523383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd540efe20 con 0x7fdd68102990 2024-04-24T23:38:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd67fff640 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x5596142fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142feda0 0x559614301230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 -- 172.21.15.33:0/3689997343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142feda0 msgr2=0x559614301230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142feda0 0x559614301230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 -- 172.21.15.33:0/3689997343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 msgr2=0x5596142fe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x5596142fe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 -- 172.21.15.33:0/3689997343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961403b7d0 con 0x5596142f9240 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd580a2b90 0x7fdd580a5050 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd681027f0 tx=0x7fdd5c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 0x5596142f9660 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009fd0 tx=0x7fdd600098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4361b1dc787fd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd52ffd640 1 -- 172.21.15.33:0/3689997343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6001a070 con 0x5596142f9240 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd52ffd640 1 -- 172.21.15.33:0/3689997343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd60012910 con 0x5596142f9240 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.865+0000 7fdd52ffd640 1 -- 172.21.15.33:0/3689997343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60005560 con 0x5596142f9240 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 -- 172.21.15.33:0/3689997343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 msgr2=0x5596142f9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 0x5596142f9660 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009fd0 tx=0x7fdd600098f0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 -- 172.21.15.33:0/3689997343 shutdown_connections 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142feda0 0x559614301230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x5596142fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3689997343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9240 0x5596142f9660 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 -- 172.21.15.33:0/3689997343 >> 172.21.15.33:0/3689997343 conn(0x5596142f48b0 msgr2=0x5596142f4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 -- 172.21.15.33:0/3689997343 shutdown_connections 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 -- 172.21.15.33:0/3689997343 wait complete. 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 -- start start 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.866+0000 7fdd71a04740 1 Processor -- start 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x559614303610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x559614303b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x55961430ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559614301d50 con 0x5596142feda0 2024-04-24T23:38:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559614218f50 con 0x5596142f9c30 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559614301ed0 con 0x5596142f9240 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 10 client.0 _ll_register_callbacks cb 0x559614254000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x55961430ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x55961430ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45398/0 (socket says 172.21.15.33:45398) 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 -- 172.21.15.33:0/3747618593 learned_addr learned my addr 172.21.15.33:0/3747618593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd67fff640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x559614303b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x559614303610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 -- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 msgr2=0x559614303610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x559614303610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 -- 172.21.15.33:0/3747618593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 msgr2=0x559614303b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x559614303b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 -- 172.21.15.33:0/3747618593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559613f713f0 con 0x5596142feda0 2024-04-24T23:38:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd67fff640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x559614303b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd6d292640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x55961430ab90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdd600145d0 tx=0x7fdd600096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57af589c93091347 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd47fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd47fff640 1 -- 172.21.15.33:0/3747618593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6002c060 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd47fff640 1 -- 172.21.15.33:0/3747618593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd600049e0 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.867+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd60012910 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd47fff640 1 -- 172.21.15.33:0/3747618593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdd60033050 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 10 client.20841 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559613f310f0 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd47fff640 1 -- 172.21.15.33:0/3747618593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd60012910 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd47fff640 1 -- 172.21.15.33:0/3747618593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fdd6000f040 con 0x5596142feda0 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 10 client.20841 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 10 client.20841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 20 client.20841 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55961430f190 0x559614311580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 4 client.20841 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55961426abf0 con 0x55961430f190 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.868+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55961430f190 0x559614311580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.869+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55961430f190 0x559614311580 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9191417568ba6f1 server_cookie=4ac734058d352d64 in_seq=0 out_seq=0 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.869+0000 7fdd47fff640 10 client.20841 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd47fff640 1 -- 172.21.15.33:0/3747618593 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55961426abf0 con 0x55961430f190 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd47fff640 10 client.20841 handle_command_reply: tid=0 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 client.20841 shutdown 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 msgr2=0x55961430ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x55961430ab90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdd600145d0 tx=0x7fdd600096e0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 shutdown_connections 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55961430f190 0x559614311580 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x55961430ab90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9c30 0x559614303b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd47fff640 0 client.20841 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 --2- 172.21.15.33:0/3747618593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x559614303610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 >> 172.21.15.33:0/3747618593 conn(0x5596142f48b0 msgr2=0x5596142f3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 shutdown_connections 2024-04-24T23:38:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 1 -- 172.21.15.33:0/3747618593 wait complete. 2024-04-24T23:38:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.870+0000 7fdd71a04740 20 client.20841 trim_cache size 0 max 16384 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 Processor -- start 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 -- start start 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9240 0x559614302c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9c30 0x55961430ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x5596142f7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681accd0 con 0x5596142feda0 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681aa540 con 0x5596142f9240 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd71a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681a8390 con 0x5596142f9c30 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd6d292640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x5596142f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd6d292640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x5596142f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45402/0 (socket says 172.21.15.33:45402) 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd6d292640 1 -- 172.21.15.33:0/726229461 learned_addr learned my addr 172.21.15.33:0/726229461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.888+0000 7fdd67fff640 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9c30 0x55961430ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9240 0x559614302c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6d292640 1 -- 172.21.15.33:0/726229461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9c30 msgr2=0x55961430ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6d292640 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9c30 0x55961430ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6d292640 1 -- 172.21.15.33:0/726229461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9240 msgr2=0x559614302c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6d292640 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9240 0x559614302c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6d292640 1 -- 172.21.15.33:0/726229461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559614311bb0 con 0x5596142feda0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd6d292640 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x5596142f7860 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000a010 tx=0x7fdd60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5e424a2d2d65fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd527fc640 1 -- 172.21.15.33:0/726229461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6002c040 con 0x5596142feda0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd527fc640 1 -- 172.21.15.33:0/726229461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd600049e0 con 0x5596142feda0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.889+0000 7fdd527fc640 1 -- 172.21.15.33:0/726229461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6001a040 con 0x5596142feda0 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- 172.21.15.33:0/726229461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 msgr2=0x5596142f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x5596142f7860 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000a010 tx=0x7fdd60009bb0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- 172.21.15.33:0/726229461 shutdown_connections 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142feda0 0x5596142f7860 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9c30 0x55961430ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- 172.21.15.33:0/726229461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142f9240 0x559614302c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- 172.21.15.33:0/726229461 >> 172.21.15.33:0/726229461 conn(0x5596142f48b0 msgr2=0x559614303610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- 172.21.15.33:0/726229461 shutdown_connections 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- 172.21.15.33:0/726229461 wait complete. 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- start start 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 Processor -- start 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x5596143081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 0x5596143086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142feda0 0x5596143096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681a8a00 con 0x5596142f9c30 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681a9e70 con 0x5596142feda0 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681a90d0 con 0x5596142f9240 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.890+0000 7fdd71a04740 10 client.0 _ll_register_callbacks cb 0x7fdd600146b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 0x5596143086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 0x5596143086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45418/0 (socket says 172.21.15.33:45418) 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 -- 172.21.15.33:0/441431088 learned_addr learned my addr 172.21.15.33:0/441431088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd6d292640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142feda0 0x5596143096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x5596143081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 -- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 msgr2=0x5596143081a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x5596143081a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 -- 172.21.15.33:0/441431088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142feda0 msgr2=0x5596143096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142feda0 0x5596143096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 -- 172.21.15.33:0/441431088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559614309ce0 con 0x5596142f9c30 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x5596143081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd67fff640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 0x5596143086e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fdd540b8c60 tx=0x7fdd540bbcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39db4e3cf26e9347 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd47fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd47fff640 1 -- 172.21.15.33:0/441431088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54049030 con 0x5596142f9c30 2024-04-24T23:38:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd47fff640 1 -- 172.21.15.33:0/441431088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd540ba590 con 0x5596142f9c30 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd600049e0 con 0x5596142f9c30 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd47fff640 1 -- 172.21.15.33:0/441431088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdd540ceba0 con 0x5596142f9c30 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd71a04740 10 client.20847 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.891+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596143756b0 con 0x5596142f9c30 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd47fff640 1 -- 172.21.15.33:0/441431088 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd600049e0 con 0x5596142f9c30 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd47fff640 1 -- 172.21.15.33:0/441431088 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fdd540b8900 con 0x5596142f9c30 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd71a04740 10 client.20847 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd71a04740 10 client.20847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd71a04740 20 client.20847 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd71a04740 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559614375fe0 0x559614378400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd71a04740 4 client.20847 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559614303400 con 0x559614375fe0 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559614375fe0 0x559614378400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.892+0000 7fdd6ca91640 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559614375fe0 0x559614378400 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb8c9df1cbb8e79 server_cookie=33ad592724bb7294 in_seq=0 out_seq=0 2024-04-24T23:38:42.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd47fff640 10 client.20847 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd47fff640 1 -- 172.21.15.33:0/441431088 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559614303400 con 0x559614375fe0 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd47fff640 10 client.20847 handle_command_reply: tid=0 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 client.20847 shutdown 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 msgr2=0x5596143086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 0x5596143086e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fdd540b8c60 tx=0x7fdd540bbcb0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 shutdown_connections 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x559614375fe0 0x559614378400 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596142feda0 0x5596143096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596142f9c30 0x5596143086e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd47fff640 0 client.20847 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 --2- 172.21.15.33:0/441431088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596142f9240 0x5596143081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 >> 172.21.15.33:0/441431088 conn(0x5596142f48b0 msgr2=0x5596142f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 shutdown_connections 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 1 -- 172.21.15.33:0/441431088 wait complete. 2024-04-24T23:38:42.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.893+0000 7fdd71a04740 20 client.20847 trim_cache size 0 max 16384 2024-04-24T23:38:42.899 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:42.899 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-24T23:38:42.899 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-24T23:38:42.899 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd580a2b90 msgr2=0x7fdd580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd580a2b90 0x7fdd580a5050 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd681027f0 tx=0x7fdd5c00b390 comp rx=0 tx=0).stop 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 msgr2=0x7fdd68197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68197fc0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009230 tx=0x7fdd54004360 comp rx=0 tx=0).stop 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 shutdown_connections 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdd580a2b90 0x7fdd580a5050 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd68103350 0x7fdd6819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd68102990 0x7fdd68197fc0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 --2- 172.21.15.33:0/2619523383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd681017a0 0x7fdd68197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:42.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 >> 172.21.15.33:0/2619523383 conn(0x7fdd680fcd10 msgr2=0x7fdd68105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:42.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 shutdown_connections 2024-04-24T23:38:42.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:42.894+0000 7fdd6ecf5640 1 -- 172.21.15.33:0/2619523383 wait complete. 2024-04-24T23:38:42.904 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:38:42.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:38:43.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 -- 172.21.15.33:0/2170360737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c102250 msgr2=0x7f277c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 --2- 172.21.15.33:0/2170360737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c102250 0x7f277c108ae0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2774009ca0 tx=0x7f277402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 -- 172.21.15.33:0/2170360737 shutdown_connections 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 --2- 172.21.15.33:0/2170360737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c102250 0x7f277c108ae0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 --2- 172.21.15.33:0/2170360737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277c101890 0x7f277c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 --2- 172.21.15.33:0/2170360737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1006a0 0x7f277c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 -- 172.21.15.33:0/2170360737 >> 172.21.15.33:0/2170360737 conn(0x7f277c0fbc30 msgr2=0x7f277c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 -- 172.21.15.33:0/2170360737 shutdown_connections 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.008+0000 7f2780c3f640 1 -- 172.21.15.33:0/2170360737 wait complete. 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 Processor -- start 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 -- start start 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277c1006a0 0x7f277c1045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 0x7f277c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c102250 0x7f277c1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10c320 con 0x7f277c101890 2024-04-24T23:38:43.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10c1a0 con 0x7f277c1006a0 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2780c3f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f277c10c4a0 con 0x7f277c102250 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 0x7f277c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 0x7f277c102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45436/0 (socket says 172.21.15.33:45436) 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 -- 172.21.15.33:0/665631671 learned_addr learned my addr 172.21.15.33:0/665631671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f277a59c640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277c1006a0 0x7f277c1045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 -- 172.21.15.33:0/665631671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c102250 msgr2=0x7f277c1031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c102250 0x7f277c1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 -- 172.21.15.33:0/665631671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277c1006a0 msgr2=0x7f277c1045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277c1006a0 0x7f277c1045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.009+0000 7f2773fff640 1 -- 172.21.15.33:0/665631671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c103a70 con 0x7f277c101890 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.010+0000 7f2773fff640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 0x7f277c102c70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f276400e9f0 tx=0x7f276400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.010+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276400cd80 con 0x7f277c101890 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.010+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c1a1bb0 con 0x7f277c101890 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.010+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2764010910 con 0x7f277c101890 2024-04-24T23:38:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.010+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27640188f0 con 0x7f277c101890 2024-04-24T23:38:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.010+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1a2120 con 0x7f277c101890 2024-04-24T23:38:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.011+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277c1a22f0 con 0x7f277c101890 2024-04-24T23:38:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.012+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2764010ac0 con 0x7f277c101890 2024-04-24T23:38:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.012+0000 7f27737fe640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f27680a2b90 0x7f27680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.013+0000 7f277a59c640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f27680a2b90 0x7f27680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.013+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f27640cf810 con 0x7f277c101890 2024-04-24T23:38:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.013+0000 7f277a59c640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f27680a2b90 0x7f27680a5050 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f277c1016f0 tx=0x7f276c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.017+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f276409c3e0 con 0x7f277c101890 2024-04-24T23:38:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.239+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f277c1a2680 con 0x7f277c101890 2024-04-24T23:38:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.240+0000 7f27737fe640 1 -- 172.21.15.33:0/665631671 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7f276409bce0 con 0x7f277c101890 2024-04-24T23:38:43.239 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:38:43.239 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:38:17.286450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:38:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 40 2024-04-24T23:38:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.241+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f27680a2b90 msgr2=0x7f27680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.241+0000 7f2780c3f640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f27680a2b90 0x7f27680a5050 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f277c1016f0 tx=0x7f276c005f90 comp rx=0 tx=0).stop 2024-04-24T23:38:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.241+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 msgr2=0x7f277c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.241+0000 7f2780c3f640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 0x7f277c102c70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f276400e9f0 tx=0x7f276400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 shutdown_connections 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f27680a2b90 0x7f27680a5050 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c102250 0x7f277c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c101890 0x7f277c102c70 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 --2- 172.21.15.33:0/665631671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f277c1006a0 0x7f277c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 >> 172.21.15.33:0/665631671 conn(0x7f277c0fbc30 msgr2=0x7f277c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 shutdown_connections 2024-04-24T23:38:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.242+0000 7f2780c3f640 1 -- 172.21.15.33:0/665631671 wait complete. 2024-04-24T23:38:43.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1528904481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38101870 msgr2=0x7f2f38101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1528904481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38101870 0x7f2f38101cf0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009900 tx=0x7f2f2002a410 comp rx=0 tx=0).stop 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1528904481 shutdown_connections 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1528904481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f38102230 0x7f2f38108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1528904481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38101870 0x7f2f38101cf0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1528904481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f38100680 0x7f2f38100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1528904481 >> 172.21.15.33:0/1528904481 conn(0x7f2f380fbbf0 msgr2=0x7f2f380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1528904481 shutdown_connections 2024-04-24T23:38:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.357+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1528904481 wait complete. 2024-04-24T23:38:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 Processor -- start 2024-04-24T23:38:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 -- start start 2024-04-24T23:38:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f38100680 0x7f2f38197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f38101870 0x7f2f38198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 0x7f2f3819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3810c350 con 0x7f2f38102230 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3810c1d0 con 0x7f2f38101870 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3cc2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3810c4d0 con 0x7f2f38100680 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f36d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 0x7f2f3819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f36d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 0x7f2f3819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45444/0 (socket says 172.21.15.33:45444) 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f36d9d640 1 -- 172.21.15.33:0/1513157981 learned_addr learned my addr 172.21.15.33:0/1513157981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f3659c640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f38100680 0x7f2f38197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.358+0000 7f2f36d9d640 1 -- 172.21.15.33:0/1513157981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f38100680 msgr2=0x7f2f38197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f36d9d640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f38100680 0x7f2f38197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f36d9d640 1 -- 172.21.15.33:0/1513157981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f38101870 msgr2=0x7f2f38198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f36d9d640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f38101870 0x7f2f38198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f36d9d640 1 -- 172.21.15.33:0/1513157981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3819f700 con 0x7f2f38102230 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f36d9d640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 0x7f2f3819f090 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800cce0 tx=0x7f2f2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f2f7fe640 1 -- 172.21.15.33:0/1513157981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800f450 con 0x7f2f38102230 2024-04-24T23:38:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3819f9f0 con 0x7f2f38102230 2024-04-24T23:38:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f2f7fe640 1 -- 172.21.15.33:0/1513157981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f2800fad0 con 0x7f2f38102230 2024-04-24T23:38:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f2f7fe640 1 -- 172.21.15.33:0/1513157981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28018c10 con 0x7f2f38102230 2024-04-24T23:38:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.359+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3819ff40 con 0x7f2f38102230 2024-04-24T23:38:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.361+0000 7f2f2f7fe640 1 -- 172.21.15.33:0/1513157981 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2f2800f5f0 con 0x7f2f38102230 2024-04-24T23:38:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.361+0000 7f2f3fa12740 1 Processor -- start 2024-04-24T23:38:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 -- start start 2024-04-24T23:38:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac44a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac49b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac4a1a0 0x5607aac4c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607aaba4670 con 0x5607aac44640 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607aab4c970 con 0x5607aac45030 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607aaba0630 con 0x5607aac4a1a0 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac44a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac44a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45454/0 (socket says 172.21.15.33:45454) 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3328507622 learned_addr learned my addr 172.21.15.33:0/3328507622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f2f7fe640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2f240a2b90 0x7f2f240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3659c640 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac49b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f36d9d640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2f240a2b90 0x7f2f240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3328507622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac4a1a0 msgr2=0x5607aac4c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac4a1a0 0x5607aac4c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3328507622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 msgr2=0x5607aac49b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f2f7fe640 1 -- 172.21.15.33:0/1513157981 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2f280cfb60 con 0x7f2f38102230 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac49b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3328507622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607aab70400 con 0x5607aac44640 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.362+0000 7f2f3659c640 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac49b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac44a60 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2f200314e0 tx=0x7f2f20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bab9f45522361e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f36d9d640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2f240a2b90 0x7f2f240a5050 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2f280cd640 tx=0x7f2f28091510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f2cff9640 1 -- 172.21.15.33:0/3328507622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20040020 con 0x5607aac44640 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f2cff9640 1 -- 172.21.15.33:0/3328507622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f20004470 con 0x5607aac44640 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f2cff9640 1 -- 172.21.15.33:0/3328507622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20035040 con 0x5607aac44640 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3328507622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 msgr2=0x5607aac44a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac44a60 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2f200314e0 tx=0x7f2f20004040 comp rx=0 tx=0).stop 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3328507622 shutdown_connections 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac4a1a0 0x5607aac4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac49b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3328507622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac44a60 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.363+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3328507622 >> 172.21.15.33:0/3328507622 conn(0x5607aac3b8a0 msgr2=0x5607aac3bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3328507622 shutdown_connections 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3328507622 wait complete. 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 -- start start 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 Processor -- start 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 0x5607aac55cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607aac4d0f0 con 0x5607aac4a1a0 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607aab60050 con 0x5607aac45030 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607aac4d270 con 0x5607aac44640 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3fa12740 10 client.0 _ll_register_callbacks cb 0x5607aa8b7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 0x5607aac55cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 0x5607aac55cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45456/0 (socket says 172.21.15.33:45456) 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f35d9b640 1 -- 172.21.15.33:0/1275348988 learned_addr learned my addr 172.21.15.33:0/1275348988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f3659c640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.364+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f35d9b640 1 -- 172.21.15.33:0/1275348988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 msgr2=0x5607aac4e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac4e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f35d9b640 1 -- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 msgr2=0x5607aac4ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac4ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f35d9b640 1 -- 172.21.15.33:0/1275348988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607aab31dc0 con 0x5607aac4a1a0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 0x5607aac55cc0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2f20031440 tx=0x7f2f20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b774c8ddcaf401 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f1dffb640 1 -- 172.21.15.33:0/1275348988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20035030 con 0x5607aac4a1a0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f1dffb640 1 -- 172.21.15.33:0/1275348988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2f20045450 con 0x5607aac4a1a0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f20004470 con 0x5607aac4a1a0 2024-04-24T23:38:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f1dffb640 1 -- 172.21.15.33:0/1275348988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2f20040070 con 0x5607aac4a1a0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f3fa12740 10 client.20865 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5607aa8b83f0 con 0x5607aac4a1a0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f1dffb640 1 -- 172.21.15.33:0/1275348988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f20004470 con 0x5607aac4a1a0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f1dffb640 1 -- 172.21.15.33:0/1275348988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f2f200476e0 con 0x5607aac4a1a0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f3fa12740 10 client.20865 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.365+0000 7f2f3fa12740 10 client.20865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f3fa12740 20 client.20865 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aac5a1b0 0x5607aac5c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f3fa12740 4 client.20865 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607aabb1bb0 con 0x5607aac5a1b0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f3659c640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aac5a1b0 0x5607aac5c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f1dffb640 10 client.20865 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.366+0000 7f2f3659c640 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aac5a1b0 0x5607aac5c5a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d82fee4b04e5420 server_cookie=2ee1584d5d53e24a in_seq=0 out_seq=0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f1dffb640 1 -- 172.21.15.33:0/1275348988 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607aabb1bb0 con 0x5607aac5a1b0 2024-04-24T23:38:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f1dffb640 10 client.20865 handle_command_reply: tid=0 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 client.20865 shutdown 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 msgr2=0x5607aac55cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 0x5607aac55cc0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2f20031440 tx=0x7f2f20004270 comp rx=0 tx=0).stop 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 shutdown_connections 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aac5a1b0 0x5607aac5c5a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac4a1a0 0x5607aac55cc0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac45030 0x5607aac4ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f1dffb640 0 client.20865 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/1275348988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac4e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 >> 172.21.15.33:0/1275348988 conn(0x5607aac3b8a0 msgr2=0x5607aac3ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 shutdown_connections 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 1 -- 172.21.15.33:0/1275348988 wait complete. 2024-04-24T23:38:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.367+0000 7f2f3fa12740 20 client.20865 trim_cache size 0 max 16384 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 Processor -- start 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 -- start start 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac49740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac45030 0x5607aac55be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381aa970 con 0x5607aac44640 2024-04-24T23:38:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381a81e0 con 0x5607aac4a1a0 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381a6030 con 0x5607aac45030 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac49740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac49740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45458/0 (socket says 172.21.15.33:45458) 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3466950071 learned_addr learned my addr 172.21.15.33:0/3466950071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f3659c640 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac45030 0x5607aac55be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3466950071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac45030 msgr2=0x5607aac55be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac45030 0x5607aac55be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3466950071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 msgr2=0x5607aac3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.381+0000 7f2f35d9b640 1 -- 172.21.15.33:0/3466950071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607aa9827d0 con 0x5607aac44640 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac49740 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2f20031610 tx=0x7f2f20003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740911f72ed255aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f1ffff640 1 -- 172.21.15.33:0/3466950071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20035040 con 0x5607aac44640 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f1ffff640 1 -- 172.21.15.33:0/3466950071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f2004f070 con 0x5607aac44640 2024-04-24T23:38:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f1ffff640 1 -- 172.21.15.33:0/3466950071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20005020 con 0x5607aac44640 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3466950071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 msgr2=0x5607aac49740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac49740 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2f20031610 tx=0x7f2f20003ff0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3466950071 shutdown_connections 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac45030 0x5607aac55be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/3466950071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac44640 0x5607aac49740 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3466950071 >> 172.21.15.33:0/3466950071 conn(0x5607aac3b8a0 msgr2=0x5607aac46360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3466950071 shutdown_connections 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.382+0000 7f2f3fa12740 1 -- 172.21.15.33:0/3466950071 wait complete. 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 -- start start 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 Processor -- start 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac53630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 0x5607aac53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac54ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381a66a0 con 0x5607aac45030 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381a7b10 con 0x5607aac4a1a0 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f381a6d70 con 0x5607aac44640 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3fa12740 10 client.0 _ll_register_callbacks cb 0x7f2f20009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 0x5607aac53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 0x5607aac53b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45466/0 (socket says 172.21.15.33:45466) 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 -- 172.21.15.33:0/306684433 learned_addr learned my addr 172.21.15.33:0/306684433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac53630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac54ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 -- 172.21.15.33:0/306684433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 msgr2=0x5607aac53630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac53630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 -- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 msgr2=0x5607aac54ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac54ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.383+0000 7f2f3659c640 1 -- 172.21.15.33:0/306684433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607aac550d0 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3659c640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 0x5607aac53b70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x5607aac45430 tx=0x7f2f3001b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc5f87aff8b635 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f1dffb640 1 -- 172.21.15.33:0/306684433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3001bdc0 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f1dffb640 1 -- 172.21.15.33:0/306684433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2f30024470 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2004f070 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f1dffb640 1 -- 172.21.15.33:0/306684433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2f3002d050 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 10 client.20871 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5607aacc0870 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f1dffb640 1 -- 172.21.15.33:0/306684433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f2004f070 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f1dffb640 1 -- 172.21.15.33:0/306684433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f2f3000f040 con 0x5607aac45030 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 10 client.20871 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 10 client.20871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 20 client.20871 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aacc1230 0x5607aacc3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 4 client.20871 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 --> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607aac49f10 con 0x5607aacc1230 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.384+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aacc1230 0x5607aacc3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f35d9b640 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aacc1230 0x5607aacc3650 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e696d463564bcc47 server_cookie=d7e260393eb0ab0e in_seq=0 out_seq=0 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f1dffb640 10 client.20871 ms_handle_connect on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f1dffb640 1 -- 172.21.15.33:0/306684433 <== mds.2 v2:172.21.15.38:6838/920905379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607aac49f10 con 0x5607aacc1230 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f1dffb640 10 client.20871 handle_command_reply: tid=0 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 client.20871 shutdown 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 msgr2=0x5607aac53b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 0x5607aac53b70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x5607aac45430 tx=0x7f2f3001b600 comp rx=0 tx=0).stop 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 shutdown_connections 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:6838/920905379,v1:172.21.15.38:6839/920905379] conn(0x5607aacc1230 0x5607aacc3650 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607aac4a1a0 0x5607aac54ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607aac45030 0x5607aac53b70 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f1dffb640 0 client.20871 ms_handle_reset on v2:172.21.15.38:6838/920905379 2024-04-24T23:38:43.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 --2- 172.21.15.33:0/306684433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607aac44640 0x5607aac53630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 >> 172.21.15.33:0/306684433 conn(0x5607aac3b8a0 msgr2=0x5607aac3f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 shutdown_connections 2024-04-24T23:38:43.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 1 -- 172.21.15.33:0/306684433 wait complete. 2024-04-24T23:38:43.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.385+0000 7f2f3fa12740 20 client.20871 trim_cache size 0 max 16384 2024-04-24T23:38:43.392 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2f240a2b90 msgr2=0x7f2f240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2f240a2b90 0x7f2f240a5050 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2f280cd640 tx=0x7f2f28091510 comp rx=0 tx=0).stop 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 msgr2=0x7f2f3819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 0x7f2f3819f090 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800cce0 tx=0x7f2f2800a620 comp rx=0 tx=0).stop 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 shutdown_connections 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f2f240a2b90 0x7f2f240a5050 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f38102230 0x7f2f3819f090 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f38101870 0x7f2f38198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 --2- 172.21.15.33:0/1513157981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f38100680 0x7f2f38197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 >> 172.21.15.33:0/1513157981 conn(0x7f2f380fbbf0 msgr2=0x7f2f380fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 shutdown_connections 2024-04-24T23:38:43.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.386+0000 7f2f3cc2a640 1 -- 172.21.15.33:0/1513157981 wait complete. 2024-04-24T23:38:43.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:38:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 -- 172.21.15.33:0/3425403594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 msgr2=0x7efe9c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 --2- 172.21.15.33:0/3425403594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c109be0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7efe94009ca0 tx=0x7efe9402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 -- 172.21.15.33:0/3425403594 shutdown_connections 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 --2- 172.21.15.33:0/3425403594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c109be0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 --2- 172.21.15.33:0/3425403594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe9c102990 0x7efe9c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 --2- 172.21.15.33:0/3425403594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c1017a0 0x7efe9c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 -- 172.21.15.33:0/3425403594 >> 172.21.15.33:0/3425403594 conn(0x7efe9c0fcd10 msgr2=0x7efe9c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 -- 172.21.15.33:0/3425403594 shutdown_connections 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 -- 172.21.15.33:0/3425403594 wait complete. 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 Processor -- start 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.503+0000 7efea2ae1640 1 -- start start 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe9c1017a0 0x7efe9c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c102990 0x7efe9c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c10d420 con 0x7efe9c103350 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c10d2a0 con 0x7efe9c102990 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c10d5a0 con 0x7efe9c1017a0 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45490/0 (socket says 172.21.15.33:45490) 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 -- 172.21.15.33:0/6780753 learned_addr learned my addr 172.21.15.33:0/6780753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efe9bfff640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c102990 0x7efe9c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea087d640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe9c1017a0 0x7efe9c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 -- 172.21.15.33:0/6780753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe9c1017a0 msgr2=0x7efe9c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe9c1017a0 0x7efe9c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 -- 172.21.15.33:0/6780753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c102990 msgr2=0x7efe9c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c102990 0x7efe9c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 -- 172.21.15.33:0/6780753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c19f5f0 con 0x7efe9c103350 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efe9bfff640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c102990 0x7efe9c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea107e640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c19ef80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efe940098d0 tx=0x7efe9402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efe99ffb640 1 -- 172.21.15.33:0/6780753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9402fc30 con 0x7efe9c103350 2024-04-24T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9c19f880 con 0x7efe9c103350 2024-04-24T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efe99ffb640 1 -- 172.21.15.33:0/6780753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe94003e50 con 0x7efe9c103350 2024-04-24T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.504+0000 7efe99ffb640 1 -- 172.21.15.33:0/6780753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe940305c0 con 0x7efe9c103350 2024-04-24T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.505+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9c19fdd0 con 0x7efe9c103350 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.507+0000 7efea57f4740 1 Processor -- start 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe99ffb640 1 -- 172.21.15.33:0/6780753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efe94003970 con 0x7efe9c103350 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 -- start start 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3757650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757c20 0x55d1b375c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b375f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d1b36bb670 con 0x55d1b3757230 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3663970 con 0x55d1b3757c20 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea57f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1b36b7630 con 0x55d1b375cd90 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3757650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3757650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45506/0 (socket says 172.21.15.33:45506) 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 -- 172.21.15.33:0/2529579400 learned_addr learned my addr 172.21.15.33:0/2529579400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea087d640 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757c20 0x55d1b375c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 -- 172.21.15.33:0/2529579400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 msgr2=0x55d1b375f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b375f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 -- 172.21.15.33:0/2529579400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757c20 msgr2=0x55d1b375c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757c20 0x55d1b375c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 -- 172.21.15.33:0/2529579400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b3687400 con 0x55d1b3757230 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe99ffb640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe8c0a2b90 0x7efe8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efea087d640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe8c0a2b90 0x7efe8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3757650 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efe8800cce0 tx=0x7efe8800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595cd93057fbfbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.508+0000 7efe99ffb640 1 -- 172.21.15.33:0/6780753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe940f0900 con 0x7efe9c103350 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efe86ffd640 1 -- 172.21.15.33:0/2529579400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88004220 con 0x55d1b3757230 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efe86ffd640 1 -- 172.21.15.33:0/2529579400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe880043c0 con 0x55d1b3757230 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efe86ffd640 1 -- 172.21.15.33:0/2529579400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe880035f0 con 0x55d1b3757230 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 -- 172.21.15.33:0/2529579400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 msgr2=0x55d1b3757650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea087d640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe8c0a2b90 0x7efe8c0a5050 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55d1b3758020 tx=0x7efe9000b3f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3757650 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efe8800cce0 tx=0x7efe8800e720 comp rx=0 tx=0).stop 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 -- 172.21.15.33:0/2529579400 shutdown_connections 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b375f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757c20 0x55d1b375c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 --2- 172.21.15.33:0/2529579400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3757650 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 -- 172.21.15.33:0/2529579400 >> 172.21.15.33:0/2529579400 conn(0x55d1b37528a0 msgr2=0x55d1b3752ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 -- 172.21.15.33:0/2529579400 shutdown_connections 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.509+0000 7efea57f4740 1 -- 172.21.15.33:0/2529579400 wait complete. 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 -- start start 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 Processor -- start 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757230 0x55d1b3761450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3761990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b3768960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d1b375fd40 con 0x55d1b3757c20 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d1b3677050 con 0x55d1b3757230 2024-04-24T23:38:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1b375fec0 con 0x55d1b375cd90 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea57f4740 10 client.0 _ll_register_callbacks cb 0x55d1b33cebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3761990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3761990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45516/0 (socket says 172.21.15.33:45516) 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efe9bfff640 1 -- 172.21.15.33:0/2241346280 learned_addr learned my addr 172.21.15.33:0/2241346280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea087d640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757230 0x55d1b3761450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.510+0000 7efea107e640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b3768960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe9bfff640 1 -- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 msgr2=0x55d1b3768960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b3768960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe9bfff640 1 -- 172.21.15.33:0/2241346280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757230 msgr2=0x55d1b3761450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757230 0x55d1b3761450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe9bfff640 1 -- 172.21.15.33:0/2241346280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b3648dc0 con 0x55d1b3757c20 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3761990 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7efe88005d70 tx=0x7efe8800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23ec9086708de1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe7bfff640 1 -- 172.21.15.33:0/2241346280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8802a060 con 0x55d1b3757c20 2024-04-24T23:38:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe880043c0 con 0x55d1b3757c20 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe7bfff640 1 -- 172.21.15.33:0/2241346280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe88005230 con 0x55d1b3757c20 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe7bfff640 1 -- 172.21.15.33:0/2241346280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efe88031050 con 0x55d1b3757c20 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efea57f4740 10 client.20883 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d1b33cf3f0 con 0x55d1b3757c20 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe7bfff640 1 -- 172.21.15.33:0/2241346280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe880043c0 con 0x55d1b3757c20 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efe7bfff640 1 -- 172.21.15.33:0/2241346280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7efe88010430 con 0x55d1b3757c20 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efea57f4740 10 client.20883 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.511+0000 7efea57f4740 10 client.20883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efea57f4740 20 client.20883 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efea57f4740 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b376cf60 0x55d1b376f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efea57f4740 4 client.20883 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b36c8bb0 con 0x55d1b376cf60 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efea087d640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b376cf60 0x55d1b376f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efea087d640 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b376cf60 0x55d1b376f350 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5fc45b6b29c9a5b server_cookie=e00da825410141a7 in_seq=0 out_seq=0 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.512+0000 7efe7bfff640 10 client.20883 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efe7bfff640 1 -- 172.21.15.33:0/2241346280 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1b36c8bb0 con 0x55d1b376cf60 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efe7bfff640 10 client.20883 handle_command_reply: tid=0 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 client.20883 shutdown 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 msgr2=0x55d1b3761990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3761990 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7efe88005d70 tx=0x7efe8800c9e0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 shutdown_connections 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b376cf60 0x55d1b376f350 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b375cd90 0x55d1b3768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efe7bfff640 0 client.20883 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3761990 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 --2- 172.21.15.33:0/2241346280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b3757230 0x55d1b3761450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 >> 172.21.15.33:0/2241346280 conn(0x55d1b37528a0 msgr2=0x55d1b3751b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 shutdown_connections 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.513+0000 7efea57f4740 1 -- 172.21.15.33:0/2241346280 wait complete. 2024-04-24T23:38:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.514+0000 7efea57f4740 20 client.20883 trim_cache size 0 max 16384 2024-04-24T23:38:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 Processor -- start 2024-04-24T23:38:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 -- start start 2024-04-24T23:38:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3755f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757c20 0x55d1b3756450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b3768920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1acbc0 con 0x55d1b3757230 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1aa430 con 0x55d1b375cd90 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea57f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1a8280 con 0x55d1b3757c20 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3755f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3755f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45528/0 (socket says 172.21.15.33:45528) 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 -- 172.21.15.33:0/3733143956 learned_addr learned my addr 172.21.15.33:0/3733143956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efe9bfff640 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757c20 0x55d1b3756450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea107e640 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b3768920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 -- 172.21.15.33:0/3733143956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757c20 msgr2=0x55d1b3756450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757c20 0x55d1b3756450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 -- 172.21.15.33:0/3733143956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 msgr2=0x55d1b3768920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b3768920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 -- 172.21.15.33:0/3733143956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b34997d0 con 0x55d1b3757230 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efe9bfff640 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757c20 0x55d1b3756450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.527+0000 7efea087d640 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3755f10 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x55d1b3758690 tx=0x7efe90013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c05bc342936041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efe867fc640 1 -- 172.21.15.33:0/3733143956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9001e2d0 con 0x55d1b3757230 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efe867fc640 1 -- 172.21.15.33:0/3733143956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9002e070 con 0x55d1b3757230 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efe867fc640 1 -- 172.21.15.33:0/3733143956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9001c520 con 0x55d1b3757230 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 -- 172.21.15.33:0/3733143956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 msgr2=0x55d1b3755f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3755f10 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x55d1b3758690 tx=0x7efe90013000 comp rx=0 tx=0).stop 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 -- 172.21.15.33:0/3733143956 shutdown_connections 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b3768920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757c20 0x55d1b3756450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 --2- 172.21.15.33:0/3733143956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757230 0x55d1b3755f10 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 -- 172.21.15.33:0/3733143956 >> 172.21.15.33:0/3733143956 conn(0x55d1b37528a0 msgr2=0x55d1b3755800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 -- 172.21.15.33:0/3733143956 shutdown_connections 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 -- 172.21.15.33:0/3733143956 wait complete. 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.528+0000 7efea57f4740 1 -- start start 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 Processor -- start 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757230 0x55d1b37662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3766810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b37677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1a88f0 con 0x55d1b3757c20 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1a9d60 con 0x55d1b375cd90 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1a8fc0 con 0x55d1b3757230 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea57f4740 10 client.0 _ll_register_callbacks cb 0x7efe88014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3766810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3766810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45544/0 (socket says 172.21.15.33:45544) 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 -- 172.21.15.33:0/2877072142 learned_addr learned my addr 172.21.15.33:0/2877072142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea087d640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757230 0x55d1b37662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efea107e640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b37677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 -- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757230 msgr2=0x55d1b37662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757230 0x55d1b37662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 -- 172.21.15.33:0/2877072142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 msgr2=0x55d1b37677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b37677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.529+0000 7efe9bfff640 1 -- 172.21.15.33:0/2877072142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b3767dc0 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe9bfff640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3766810 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7efe88005d70 tx=0x7efe880104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69c8fcd26db2688 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe7bfff640 1 -- 172.21.15.33:0/2877072142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8803a060 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe7bfff640 1 -- 172.21.15.33:0/2877072142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efe88010c00 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9002e070 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe7bfff640 1 -- 172.21.15.33:0/2877072142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efe88041050 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 10 client.20889 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d1b37d3670 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe7bfff640 1 -- 172.21.15.33:0/2877072142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9002e070 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efe7bfff640 1 -- 172.21.15.33:0/2877072142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7efe8801a560 con 0x55d1b3757c20 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 10 client.20889 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 10 client.20889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 20 client.20889 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b37d4070 0x55d1b37d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 4 client.20889 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.530+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 --> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1b36c2f00 con 0x55d1b37d4070 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea087d640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b37d4070 0x55d1b37d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea087d640 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b37d4070 0x55d1b37d6490 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d2e29e0aec6ae09 server_cookie=a83d5a3706e8f2a9 in_seq=0 out_seq=0 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efe7bfff640 10 client.20889 ms_handle_connect on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efe7bfff640 1 -- 172.21.15.33:0/2877072142 <== mds.1 v2:172.21.15.117:6836/878889242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1b36c2f00 con 0x55d1b37d4070 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efe7bfff640 10 client.20889 handle_command_reply: tid=0 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 client.20889 shutdown 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 msgr2=0x55d1b3766810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3766810 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7efe88005d70 tx=0x7efe880104b0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 shutdown_connections 2024-04-24T23:38:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:6836/878889242,v1:172.21.15.117:6837/878889242] conn(0x55d1b37d4070 0x55d1b37d6490 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1b375cd90 0x55d1b37677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1b3757c20 0x55d1b3766810 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efe7bfff640 0 client.20889 ms_handle_reset on v2:172.21.15.117:6836/878889242 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 --2- 172.21.15.33:0/2877072142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1b3757230 0x55d1b37662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 >> 172.21.15.33:0/2877072142 conn(0x55d1b37528a0 msgr2=0x55d1b3756090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 shutdown_connections 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.531+0000 7efea57f4740 1 -- 172.21.15.33:0/2877072142 wait complete. 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea57f4740 20 client.20889 trim_cache size 0 max 16384 2024-04-24T23:38:43.537 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe8c0a2b90 msgr2=0x7efe8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe8c0a2b90 0x7efe8c0a5050 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55d1b3758020 tx=0x7efe9000b3f0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 msgr2=0x7efe9c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c19ef80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efe940098d0 tx=0x7efe9402f840 comp rx=0 tx=0).stop 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 shutdown_connections 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7efe8c0a2b90 0x7efe8c0a5050 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c103350 0x7efe9c19ef80 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe9c102990 0x7efe9c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 --2- 172.21.15.33:0/6780753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe9c1017a0 0x7efe9c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 >> 172.21.15.33:0/6780753 conn(0x7efe9c0fcd10 msgr2=0x7efe9c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 shutdown_connections 2024-04-24T23:38:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.532+0000 7efea2ae1640 1 -- 172.21.15.33:0/6780753 wait complete. 2024-04-24T23:38:43.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 -- 172.21.15.33:0/66255547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1006a0 msgr2=0x7f474c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 --2- 172.21.15.33:0/66255547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1006a0 0x7f474c100ac0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f473c009900 tx=0x7f473c02a410 comp rx=0 tx=0).stop 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 -- 172.21.15.33:0/66255547 shutdown_connections 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 --2- 172.21.15.33:0/66255547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f474c102250 0x7f474c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 --2- 172.21.15.33:0/66255547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f474c101890 0x7f474c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 --2- 172.21.15.33:0/66255547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1006a0 0x7f474c100ac0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 -- 172.21.15.33:0/66255547 >> 172.21.15.33:0/66255547 conn(0x7f474c0fbc30 msgr2=0x7f474c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 -- 172.21.15.33:0/66255547 shutdown_connections 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.646+0000 7f4752c8c640 1 -- 172.21.15.33:0/66255547 wait complete. 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 Processor -- start 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 -- start start 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f474c1006a0 0x7f474c10b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f474c101890 0x7f474c109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 0x7f474c109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10d440 con 0x7f474c102250 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10d2c0 con 0x7f474c1006a0 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4752c8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10d5c0 con 0x7f474c101890 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4751229640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 0x7f474c109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4751229640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 0x7f474c109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45568/0 (socket says 172.21.15.33:45568) 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4751229640 1 -- 172.21.15.33:0/772382568 learned_addr learned my addr 172.21.15.33:0/772382568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f4750a28640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f474c1006a0 0x7f474c10b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.647+0000 7f474bfff640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f474c101890 0x7f474c109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4751229640 1 -- 172.21.15.33:0/772382568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f474c101890 msgr2=0x7f474c109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4751229640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f474c101890 0x7f474c109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4751229640 1 -- 172.21.15.33:0/772382568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f474c1006a0 msgr2=0x7f474c10b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4751229640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f474c1006a0 0x7f474c10b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4751229640 1 -- 172.21.15.33:0/772382568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c10a430 con 0x7f474c102250 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4751229640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 0x7f474c109ba0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f474400eaa0 tx=0x7f474400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4749ffb640 1 -- 172.21.15.33:0/772382568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400cdb0 con 0x7f474c102250 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4749ffb640 1 -- 172.21.15.33:0/772382568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4744005020 con 0x7f474c102250 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c1a1bf0 con 0x7f474c102250 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4749ffb640 1 -- 172.21.15.33:0/772382568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440116b0 con 0x7f474c102250 2024-04-24T23:38:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.648+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1a2190 con 0x7f474c102250 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.650+0000 7f4749ffb640 1 -- 172.21.15.33:0/772382568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4744005560 con 0x7f474c102250 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 Processor -- start 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 -- start start 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c242018230 0x55c242018650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 0x55c24201d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c24201dd90 0x55c242020220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c241f7c670 con 0x55c242018c20 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c241f24970 con 0x55c24201dd90 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f475599a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c241f78630 con 0x55c242018230 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f4749ffb640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47340a2c10 0x7f47340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f4750a28640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 0x55c24201d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f4750a28640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 0x55c24201d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45578/0 (socket says 172.21.15.33:45578) 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f4750a28640 1 -- 172.21.15.33:0/3795842139 learned_addr learned my addr 172.21.15.33:0/3795842139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f4751229640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47340a2c10 0x7f47340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f474bfff640 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c24201dd90 0x55c242020220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f474bfff640 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c242018230 0x55c242018650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.651+0000 7f4749ffb640 1 -- 172.21.15.33:0/772382568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47440cf630 con 0x7f474c102250 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4750a28640 1 -- 172.21.15.33:0/3795842139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c242018230 msgr2=0x55c242018650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4750a28640 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c242018230 0x55c242018650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4750a28640 1 -- 172.21.15.33:0/3795842139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c24201dd90 msgr2=0x55c242020220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4750a28640 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c24201dd90 0x55c242020220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4750a28640 1 -- 172.21.15.33:0/3795842139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c241f48400 con 0x55c242018c20 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f474bfff640 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c24201dd90 0x55c242020220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4751229640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47340a2c10 0x7f47340a50d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4744000f80 tx=0x7f474406b950 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4750a28640 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 0x55c24201d720 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f473c031610 tx=0x7f473c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8789d6b136b6f8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4732ffd640 1 -- 172.21.15.33:0/3795842139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c040070 con 0x55c242018c20 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4732ffd640 1 -- 172.21.15.33:0/3795842139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c0318b0 con 0x55c242018c20 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.652+0000 7f4732ffd640 1 -- 172.21.15.33:0/3795842139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c035040 con 0x55c242018c20 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- 172.21.15.33:0/3795842139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 msgr2=0x55c24201d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 0x55c24201d720 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f473c031610 tx=0x7f473c005c00 comp rx=0 tx=0).stop 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- 172.21.15.33:0/3795842139 shutdown_connections 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c24201dd90 0x55c242020220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018c20 0x55c24201d720 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- 172.21.15.33:0/3795842139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c242018230 0x55c242018650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- 172.21.15.33:0/3795842139 >> 172.21.15.33:0/3795842139 conn(0x55c2420138a0 msgr2=0x55c242013ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- 172.21.15.33:0/3795842139 shutdown_connections 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- 172.21.15.33:0/3795842139 wait complete. 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- start start 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 Processor -- start 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c242022330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242022870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c2420298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c242020ce0 con 0x55c242018230 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c241f38050 con 0x55c242018c20 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.653+0000 7f475599a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c242020e60 con 0x55c24201dd90 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f475599a740 10 client.0 _ll_register_callbacks cb 0x55c241c8fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c242022330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c242022330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45588/0 (socket says 172.21.15.33:45588) 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 -- 172.21.15.33:0/4115996643 learned_addr learned my addr 172.21.15.33:0/4115996643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f4750a28640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242022870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c2420298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 -- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 msgr2=0x55c2420298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c2420298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 -- 172.21.15.33:0/4115996643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 msgr2=0x55c242022870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242022870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 -- 172.21.15.33:0/4115996643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c241f09dc0 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f4750a28640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242022870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f474bfff640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c242022330 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f474000ee90 tx=0x7f4740004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c4d699bcb9ec1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f472bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f472bfff640 1 -- 172.21.15.33:0/4115996643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474001a070 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f472bfff640 1 -- 172.21.15.33:0/4115996643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4740014970 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473c0318b0 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f472bfff640 1 -- 172.21.15.33:0/4115996643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f47400142d0 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f475599a740 10 client.20901 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.654+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c241c903f0 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f472bfff640 1 -- 172.21.15.33:0/4115996643 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c0318b0 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f472bfff640 1 -- 172.21.15.33:0/4115996643 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f474001d020 con 0x55c242018230 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f475599a740 10 client.20901 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f475599a740 10 client.20901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f475599a740 20 client.20901 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f475599a740 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c24202e1b0 0x55c242030670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f475599a740 4 client.20901 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c241f89bb0 con 0x55c24202e1b0 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f4750a28640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c24202e1b0 0x55c242030670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f472bfff640 10 client.20901 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.655+0000 7f4750a28640 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c24202e1b0 0x55c242030670 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d189a41a00c624f0 server_cookie=34f3081eab68aded in_seq=0 out_seq=0 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f472bfff640 1 -- 172.21.15.33:0/4115996643 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c241f89bb0 con 0x55c24202e1b0 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f472bfff640 10 client.20901 handle_command_reply: tid=0 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 client.20901 shutdown 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 msgr2=0x55c242022330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c242022330 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f474000ee90 tx=0x7f4740004160 comp rx=0 tx=0).stop 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 shutdown_connections 2024-04-24T23:38:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c24202e1b0 0x55c242030670 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c2420298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242022870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 --2- 172.21.15.33:0/4115996643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c242022330 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f472bfff640 0 client.20901 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 >> 172.21.15.33:0/4115996643 conn(0x55c2420138a0 msgr2=0x55c242012b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 shutdown_connections 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.656+0000 7f475599a740 1 -- 172.21.15.33:0/4115996643 wait complete. 2024-04-24T23:38:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.657+0000 7f475599a740 20 client.20901 trim_cache size 0 max 16384 2024-04-24T23:38:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 Processor -- start 2024-04-24T23:38:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 -- start start 2024-04-24T23:38:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c24201a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242029870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242022630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1aefa0 con 0x55c242018230 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1ac810 con 0x55c242018c20 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f475599a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1aa660 con 0x55c24201dd90 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f474bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c24201a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f474bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c24201a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45592/0 (socket says 172.21.15.33:45592) 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f474bfff640 1 -- 172.21.15.33:0/2692107673 learned_addr learned my addr 172.21.15.33:0/2692107673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f474bfff640 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242022630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.670+0000 7f4750a28640 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242029870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f474bfff640 1 -- 172.21.15.33:0/2692107673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 msgr2=0x55c242022630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f474bfff640 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242022630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f474bfff640 1 -- 172.21.15.33:0/2692107673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 msgr2=0x55c242029870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f474bfff640 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242029870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f474bfff640 1 -- 172.21.15.33:0/2692107673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c241d5a7d0 con 0x55c242018230 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f474bfff640 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c24201a190 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f474000f010 tx=0x7f4740005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8af142c99f1f8b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f47327fc640 1 -- 172.21.15.33:0/2692107673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740031070 con 0x55c242018230 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f47327fc640 1 -- 172.21.15.33:0/2692107673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740014d20 con 0x55c242018230 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.671+0000 7f47327fc640 1 -- 172.21.15.33:0/2692107673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474001a040 con 0x55c242018230 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- 172.21.15.33:0/2692107673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 msgr2=0x55c24201a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c24201a190 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f474000f010 tx=0x7f4740005fb0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- 172.21.15.33:0/2692107673 shutdown_connections 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242022630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242029870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- 172.21.15.33:0/2692107673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c24201a190 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- 172.21.15.33:0/2692107673 >> 172.21.15.33:0/2692107673 conn(0x55c2420138a0 msgr2=0x55c242017040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- 172.21.15.33:0/2692107673 shutdown_connections 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- 172.21.15.33:0/2692107673 wait complete. 2024-04-24T23:38:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- start start 2024-04-24T23:38:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 Processor -- start 2024-04-24T23:38:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c2420276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242027c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242028bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1aacd0 con 0x55c242018230 2024-04-24T23:38:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1ac140 con 0x55c242018c20 2024-04-24T23:38:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1ab3a0 con 0x55c24201dd90 2024-04-24T23:38:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f475599a740 10 client.0 _ll_register_callbacks cb 0x7f4740004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:38:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.672+0000 7f474bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c2420276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c2420276f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45602/0 (socket says 172.21.15.33:45602) 2024-04-24T23:38:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 -- 172.21.15.33:0/1069823799 learned_addr learned my addr 172.21.15.33:0/1069823799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f4750a28640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242027c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242028bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 -- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 msgr2=0x55c242028bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242028bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 -- 172.21.15.33:0/1069823799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 msgr2=0x55c242027c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242027c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 -- 172.21.15.33:0/1069823799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2420291e0 con 0x55c242018230 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f474bfff640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c2420276f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4740012da0 tx=0x7f474000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef0011281ed8ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f472bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f472bfff640 1 -- 172.21.15.33:0/1069823799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474002d070 con 0x55c242018230 2024-04-24T23:38:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f472bfff640 1 -- 172.21.15.33:0/1069823799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4740014b00 con 0x55c242018230 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4740014d20 con 0x55c242018230 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f472bfff640 1 -- 172.21.15.33:0/1069823799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4740031070 con 0x55c242018230 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f475599a740 10 client.20907 fetch_fsmap learned FSMap version 40 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c242094e70 con 0x55c242018230 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.673+0000 7f472bfff640 1 -- 172.21.15.33:0/1069823799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740014d20 con 0x55c242018230 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f472bfff640 1 -- 172.21.15.33:0/1069823799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f474001d030 con 0x55c242018230 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f475599a740 10 client.20907 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T23:38:43.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f475599a740 10 client.20907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f475599a740 20 client.20907 populate_metadata read hostname 'smithi033' 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f475599a740 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2420956b0 0x55c242097ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f475599a740 4 client.20907 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 --> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c241f83f00 con 0x55c2420956b0 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f4750a28640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2420956b0 0x55c242097ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f4750a28640 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2420956b0 0x55c242097ad0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e5f2b3ebaca748 server_cookie=f1fdcdb9d3a77020 in_seq=0 out_seq=0 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.674+0000 7f472bfff640 10 client.20907 ms_handle_connect on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f472bfff640 1 -- 172.21.15.33:0/1069823799 <== mds.0 v2:172.21.15.117:6838/851112851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c241f83f00 con 0x55c2420956b0 2024-04-24T23:38:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f472bfff640 10 client.20907 handle_command_reply: tid=0 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 client.20907 shutdown 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 msgr2=0x55c2420276f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c2420276f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4740012da0 tx=0x7f474000eb00 comp rx=0 tx=0).stop 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 shutdown_connections 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:6838/851112851,v1:172.21.15.117:6839/851112851] conn(0x55c2420956b0 0x55c242097ad0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c24201dd90 0x55c242028bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c242018c20 0x55c242027c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f472bfff640 0 client.20907 ms_handle_reset on v2:172.21.15.117:6838/851112851 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 --2- 172.21.15.33:0/1069823799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c242018230 0x55c2420276f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 >> 172.21.15.33:0/1069823799 conn(0x55c2420138a0 msgr2=0x55c242021620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 shutdown_connections 2024-04-24T23:38:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 1 -- 172.21.15.33:0/1069823799 wait complete. 2024-04-24T23:38:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f475599a740 20 client.20907 trim_cache size 0 max 16384 2024-04-24T23:38:43.689 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47340a2c10 msgr2=0x7f47340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f4752c8c640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47340a2c10 0x7f47340a50d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4744000f80 tx=0x7f474406b950 comp rx=0 tx=0).stop 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 msgr2=0x7f474c109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.675+0000 7f4752c8c640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 0x7f474c109ba0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f474400eaa0 tx=0x7f474400ef60 comp rx=0 tx=0).stop 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 shutdown_connections 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f47340a2c10 0x7f47340a50d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c102250 0x7f474c109ba0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f474c101890 0x7f474c109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 --2- 172.21.15.33:0/772382568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f474c1006a0 0x7f474c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 >> 172.21.15.33:0/772382568 conn(0x7f474c0fbc30 msgr2=0x7f474c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 shutdown_connections 2024-04-24T23:38:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:43.676+0000 7f4752c8c640 1 -- 172.21.15.33:0/772382568 wait complete. 2024-04-24T23:38:44.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:43 smithi038 ceph-mon[32450]: pgmap v1152: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 178 KiB/s wr, 25 op/s 2024-04-24T23:38:44.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:43 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/665631671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:38:44.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:43 smithi117 ceph-mon[32529]: pgmap v1152: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 178 KiB/s wr, 25 op/s 2024-04-24T23:38:44.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:43 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/665631671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:38:44.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:43 smithi033 ceph-mon[29201]: pgmap v1152: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 178 KiB/s wr, 25 op/s 2024-04-24T23:38:44.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:43 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/665631671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:38:44.694 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-24T23:38:44.694 INFO:tasks.fwd_scrub:done joining 2024-04-24T23:38:44.695 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T23:38:44.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.820+0000 7fe8f71a2640 1 -- 172.21.15.33:0/483023490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f01006a0 msgr2=0x7fe8f0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.820+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/483023490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f01006a0 0x7fe8f0100ac0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009900 tx=0x7fe8e002a410 comp rx=0 tx=0).stop 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- 172.21.15.33:0/483023490 shutdown_connections 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/483023490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f0102250 0x7fe8f0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/483023490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f0101890 0x7fe8f0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/483023490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f01006a0 0x7fe8f0100ac0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:44.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- 172.21.15.33:0/483023490 >> 172.21.15.33:0/483023490 conn(0x7fe8f00fbc30 msgr2=0x7fe8f00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- 172.21.15.33:0/483023490 shutdown_connections 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- 172.21.15.33:0/483023490 wait complete. 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 Processor -- start 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- start start 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f01006a0 0x7fe8f0104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f0101890 0x7fe8f0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 0x7fe8f01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010c320 con 0x7fe8f0102250 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010c1a0 con 0x7fe8f01006a0 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.821+0000 7fe8f71a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010c4a0 con 0x7fe8f0101890 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 0x7fe8f01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 0x7fe8f01031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45616/0 (socket says 172.21.15.33:45616) 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 -- 172.21.15.33:0/2600054878 learned_addr learned my addr 172.21.15.33:0/2600054878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f4f3e640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f01006a0 0x7fe8f0104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8effff640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f0101890 0x7fe8f0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 -- 172.21.15.33:0/2600054878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f0101890 msgr2=0x7fe8f0102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f0101890 0x7fe8f0102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 -- 172.21.15.33:0/2600054878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f01006a0 msgr2=0x7fe8f0104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f01006a0 0x7fe8f0104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 -- 172.21.15.33:0/2600054878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f0103a40 con 0x7fe8f0102250 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f573f640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 0x7fe8f01031b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800eaa0 tx=0x7fe8e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800cdb0 con 0x7fe8f0102250 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8005020 con 0x7fe8f0102250 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80116d0 con 0x7fe8f0102250 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.822+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f01a1bb0 con 0x7fe8f0102250 2024-04-24T23:38:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.823+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a20b0 con 0x7fe8f0102250 2024-04-24T23:38:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.824+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f01a22e0 con 0x7fe8f0102250 2024-04-24T23:38:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.825+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe8e80054d0 con 0x7fe8f0102250 2024-04-24T23:38:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.825+0000 7fe8edffb640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe8d80a2b90 0x7fe8d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.826+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe8e80cf5d0 con 0x7fe8f0102250 2024-04-24T23:38:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.826+0000 7fe8f4f3e640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe8d80a2b90 0x7fe8d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.826+0000 7fe8f4f3e640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe8d80a2b90 0x7fe8d80a5050 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005ab0 tx=0x7fe8e0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:44.830+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe8e8016030 con 0x7fe8f0102250 2024-04-24T23:38:45.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.020+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8f01a2640 con 0x7fe8f0102250 2024-04-24T23:38:45.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.021+0000 7fe8edffb640 1 -- 172.21.15.33:0/2600054878 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7fe8e809bb50 con 0x7fe8f0102250 2024-04-24T23:38:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:38:45.021 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe8d80a2b90 msgr2=0x7fe8d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe8d80a2b90 0x7fe8d80a5050 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005ab0 tx=0x7fe8e0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 msgr2=0x7fe8f01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 0x7fe8f01031b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800eaa0 tx=0x7fe8e800ef60 comp rx=0 tx=0).stop 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 shutdown_connections 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe8d80a2b90 0x7fe8d80a5050 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f0102250 0x7fe8f01031b0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f0101890 0x7fe8f0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 --2- 172.21.15.33:0/2600054878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8f01006a0 0x7fe8f0104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 >> 172.21.15.33:0/2600054878 conn(0x7fe8f00fbc30 msgr2=0x7fe8f00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 shutdown_connections 2024-04-24T23:38:45.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.023+0000 7fe8f71a2640 1 -- 172.21.15.33:0/2600054878 wait complete. 2024-04-24T23:38:45.034 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:38:45.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 -- 172.21.15.33:0/2115874480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c101310 msgr2=0x7f341c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 --2- 172.21.15.33:0/2115874480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c101310 0x7f341c101790 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f340c009ca0 tx=0x7f340c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 -- 172.21.15.33:0/2115874480 shutdown_connections 2024-04-24T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 --2- 172.21.15.33:0/2115874480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c101310 0x7f341c101790 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 --2- 172.21.15.33:0/2115874480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 0x7f341c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 --2- 172.21.15.33:0/2115874480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c100010 0x7f341c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 -- 172.21.15.33:0/2115874480 >> 172.21.15.33:0/2115874480 conn(0x7f341c0fbc30 msgr2=0x7f341c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 -- 172.21.15.33:0/2115874480 shutdown_connections 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.140+0000 7f3423b70640 1 -- 172.21.15.33:0/2115874480 wait complete. 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 Processor -- start 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 -- start start 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 0x7f341c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 0x7f341c19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c101310 0x7f341c1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f341c10f610 con 0x7f341c100010 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f341c10f490 con 0x7f341c101310 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f3423b70640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f341c10f790 con 0x7f341c100970 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f342190c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 0x7f341c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f342190c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 0x7f341c199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45634/0 (socket says 172.21.15.33:45634) 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.141+0000 7f342190c640 1 -- 172.21.15.33:0/1765679063 learned_addr learned my addr 172.21.15.33:0/1765679063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342210d640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c101310 0x7f341c1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342110b640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 0x7f341c19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342190c640 1 -- 172.21.15.33:0/1765679063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 msgr2=0x7f341c19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342190c640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 0x7f341c19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342190c640 1 -- 172.21.15.33:0/1765679063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c101310 msgr2=0x7f341c1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342190c640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c101310 0x7f341c1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342190c640 1 -- 172.21.15.33:0/1765679063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c1a1860 con 0x7f341c100010 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342110b640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 0x7f341c19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342210d640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c101310 0x7f341c1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f342190c640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 0x7f341c199c50 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f341400d950 tx=0x7f341400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414017070 con 0x7f341c100010 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c1a1b50 con 0x7f341c100010 2024-04-24T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414009c30 con 0x7f341c100010 2024-04-24T23:38:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400f6b0 con 0x7f341c100010 2024-04-24T23:38:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.142+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c1a20f0 con 0x7f341c100010 2024-04-24T23:38:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.143+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341c10ad60 con 0x7f341c100010 2024-04-24T23:38:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.144+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f34140109d0 con 0x7f341c100010 2024-04-24T23:38:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.145+0000 7f341affd640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f34040a2b90 0x7f34040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.145+0000 7f342110b640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f34040a2b90 0x7f34040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.145+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f34140cf190 con 0x7f341c100010 2024-04-24T23:38:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.145+0000 7f342110b640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f34040a2b90 0x7f34040a5050 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3410009b50 tx=0x7f3410005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.148+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3414009de0 con 0x7f341c100010 2024-04-24T23:38:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.334+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f341c02d170 con 0x7f341c100010 2024-04-24T23:38:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.334+0000 7f341affd640 1 -- 172.21.15.33:0/1765679063 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f341409bd00 con 0x7f341c100010 2024-04-24T23:38:45.333 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:38:45.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f34040a2b90 msgr2=0x7f34040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f34040a2b90 0x7f34040a5050 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3410009b50 tx=0x7f3410005f90 comp rx=0 tx=0).stop 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 msgr2=0x7f341c199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 0x7f341c199c50 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f341400d950 tx=0x7f341400de10 comp rx=0 tx=0).stop 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 shutdown_connections 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f34040a2b90 0x7f34040a5050 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c101310 0x7f341c1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c100970 0x7f341c19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 --2- 172.21.15.33:0/1765679063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f341c100010 0x7f341c199c50 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 >> 172.21.15.33:0/1765679063 conn(0x7f341c0fbc30 msgr2=0x7f341c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 shutdown_connections 2024-04-24T23:38:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.336+0000 7f3423b70640 1 -- 172.21.15.33:0/1765679063 wait complete. 2024-04-24T23:38:45.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 -- 172.21.15.33:0/4192502775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834102200 msgr2=0x7f38341045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 --2- 172.21.15.33:0/4192502775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834102200 0x7f38341045f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f3824009900 tx=0x7f382402a410 comp rx=0 tx=0).stop 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 -- 172.21.15.33:0/4192502775 shutdown_connections 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 --2- 172.21.15.33:0/4192502775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834107460 0x7f38341098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 --2- 172.21.15.33:0/4192502775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3834104b30 0x7f3834106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 --2- 172.21.15.33:0/4192502775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834102200 0x7f38341045f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 -- 172.21.15.33:0/4192502775 >> 172.21.15.33:0/4192502775 conn(0x7f38340fbbf0 msgr2=0x7f38340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.452+0000 7f383adee640 1 -- 172.21.15.33:0/4192502775 shutdown_connections 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 -- 172.21.15.33:0/4192502775 wait complete. 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 Processor -- start 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 -- start start 2024-04-24T23:38:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3834102200 0x7f3834197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834104b30 0x7f3834198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 0x7f383419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f383410d430 con 0x7f3834107460 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f383410d2b0 con 0x7f3834104b30 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383adee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f383410d5b0 con 0x7f3834102200 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383938b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 0x7f383419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383938b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 0x7f383419f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45666/0 (socket says 172.21.15.33:45666) 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f383938b640 1 -- 172.21.15.33:0/1660790977 learned_addr learned my addr 172.21.15.33:0/1660790977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f3838b8a640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3834102200 0x7f3834197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.453+0000 7f3833fff640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834104b30 0x7f3834198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383938b640 1 -- 172.21.15.33:0/1660790977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3834102200 msgr2=0x7f3834197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383938b640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3834102200 0x7f3834197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383938b640 1 -- 172.21.15.33:0/1660790977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834104b30 msgr2=0x7f3834198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383938b640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834104b30 0x7f3834198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383938b640 1 -- 172.21.15.33:0/1660790977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383419f760 con 0x7f3834107460 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f3833fff640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834104b30 0x7f3834198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383938b640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 0x7f383419f0f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f382c00eaa0 tx=0x7f382c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c00cdb0 con 0x7f3834107460 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383419fa50 con 0x7f3834107460 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c005020 con 0x7f3834107460 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c0116b0 con 0x7f3834107460 2024-04-24T23:38:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.454+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383419ff50 con 0x7f3834107460 2024-04-24T23:38:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.455+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38340fd6e0 con 0x7f3834107460 2024-04-24T23:38:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.456+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f382c005560 con 0x7f3834107460 2024-04-24T23:38:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.457+0000 7f3831ffb640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f381c0a2b90 0x7f381c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.457+0000 7f3838b8a640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f381c0a2b90 0x7f381c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.457+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f382c0cf620 con 0x7f3834107460 2024-04-24T23:38:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.458+0000 7f3838b8a640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f381c0a2b90 0x7f381c0a5050 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f38240314e0 tx=0x7f3824000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.462+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f382c016030 con 0x7f3834107460 2024-04-24T23:38:45.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.647+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3834062340 con 0x7f3834107460 2024-04-24T23:38:45.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.647+0000 7f3831ffb640 1 -- 172.21.15.33:0/1660790977 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f382c09c1f0 con 0x7f3834107460 2024-04-24T23:38:45.647 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f381c0a2b90 msgr2=0x7f381c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f381c0a2b90 0x7f381c0a5050 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f38240314e0 tx=0x7f3824000bc0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 msgr2=0x7f383419f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 0x7f383419f0f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f382c00eaa0 tx=0x7f382c00ef60 comp rx=0 tx=0).stop 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 shutdown_connections 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f381c0a2b90 0x7f381c0a5050 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3834107460 0x7f383419f0f0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3834104b30 0x7f3834198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 --2- 172.21.15.33:0/1660790977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3834102200 0x7f3834197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 >> 172.21.15.33:0/1660790977 conn(0x7f38340fbbf0 msgr2=0x7f3834102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.649+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 shutdown_connections 2024-04-24T23:38:45.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.650+0000 7f383adee640 1 -- 172.21.15.33:0/1660790977 wait complete. 2024-04-24T23:38:45.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:38:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 -- 172.21.15.33:0/9701820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 msgr2=0x7f706c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 --2- 172.21.15.33:0/9701820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c100c40 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f705c009900 tx=0x7f705c02a410 comp rx=0 tx=0).stop 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 -- 172.21.15.33:0/9701820 shutdown_connections 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 --2- 172.21.15.33:0/9701820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 0x7f706c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 --2- 172.21.15.33:0/9701820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c100c40 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 --2- 172.21.15.33:0/9701820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f706c0ffa60 0x7f706c0ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 -- 172.21.15.33:0/9701820 >> 172.21.15.33:0/9701820 conn(0x7f706c0fb370 msgr2=0x7f706c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 -- 172.21.15.33:0/9701820 shutdown_connections 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 -- 172.21.15.33:0/9701820 wait complete. 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 Processor -- start 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 -- start start 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f706c0ffa60 0x7f706c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.766+0000 7f70741c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f70741c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 0x7f706c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f70741c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f706c10f620 con 0x7f706c1007c0 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f70741c8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f706c10f4a0 con 0x7f706c0ffa60 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f70741c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f706c10f7a0 con 0x7f706c101180 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52504/0 (socket says 172.21.15.33:52504) 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 -- 172.21.15.33:0/2300755786 learned_addr learned my addr 172.21.15.33:0/2300755786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071f64640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f706c0ffa60 0x7f706c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7072765640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 0x7f706c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 -- 172.21.15.33:0/2300755786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 msgr2=0x7f706c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 0x7f706c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 -- 172.21.15.33:0/2300755786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f706c0ffa60 msgr2=0x7f706c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f706c0ffa60 0x7f706c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 -- 172.21.15.33:0/2300755786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c1a1930 con 0x7f706c1007c0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7072765640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 0x7f706c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7071763640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c19a260 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f705c031610 tx=0x7f705c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c03f030 con 0x7f706c1007c0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705c003ce0 con 0x7f706c1007c0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c1a1bc0 con 0x7f706c1007c0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c035070 con 0x7f706c1007c0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.767+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c1a2100 con 0x7f706c1007c0 2024-04-24T23:38:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.768+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706c10b570 con 0x7f706c1007c0 2024-04-24T23:38:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.769+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f705c042030 con 0x7f706c1007c0 2024-04-24T23:38:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.770+0000 7f7062ffd640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f70580a2c10 0x7f70580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.770+0000 7f7071f64640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f70580a2c10 0x7f70580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.770+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f705c0ef9b0 con 0x7f706c1007c0 2024-04-24T23:38:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.771+0000 7f7071f64640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f70580a2c10 0x7f70580a50d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f706c19b1f0 tx=0x7f7068009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.775+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f705c0bc580 con 0x7f706c1007c0 2024-04-24T23:38:45.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:45 smithi033 ceph-mon[29201]: pgmap v1153: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 89 KiB/s wr, 18 op/s 2024-04-24T23:38:45.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:45 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2600054878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:38:45.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:45 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1765679063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:38:45.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:45 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1660790977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:38:45.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.960+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f706c02d170 con 0x7f706c1007c0 2024-04-24T23:38:45.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.960+0000 7f7062ffd640 1 -- 172.21.15.33:0/2300755786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f706c02d170 con 0x7f706c1007c0 2024-04-24T23:38:45.960 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f70580a2c10 msgr2=0x7f70580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f70580a2c10 0x7f70580a50d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f706c19b1f0 tx=0x7f7068009680 comp rx=0 tx=0).stop 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 msgr2=0x7f706c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c19a260 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f705c031610 tx=0x7f705c0043b0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 shutdown_connections 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f70580a2c10 0x7f70580a50d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706c101180 0x7f706c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706c1007c0 0x7f706c19a260 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 --2- 172.21.15.33:0/2300755786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f706c0ffa60 0x7f706c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 >> 172.21.15.33:0/2300755786 conn(0x7f706c0fb370 msgr2=0x7f706c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 shutdown_connections 2024-04-24T23:38:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:45.962+0000 7f70741c8640 1 -- 172.21.15.33:0/2300755786 wait complete. 2024-04-24T23:38:45.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/4137525261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 msgr2=0x7f9aa0102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/4137525261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0102e10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009900 tx=0x7f9a9002a410 comp rx=0 tx=0).stop 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/4137525261 shutdown_connections 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/4137525261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0103350 0x7f9aa0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/4137525261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0102e10 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/4137525261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 0x7f9aa0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/4137525261 >> 172.21.15.33:0/4137525261 conn(0x7f9aa00fcd10 msgr2=0x7f9aa00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/4137525261 shutdown_connections 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.078+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/4137525261 wait complete. 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 Processor -- start 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 -- start start 2024-04-24T23:38:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 0x7f9aa0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0103350 0x7f9aa019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010d420 con 0x7f9aa0102990 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010d2a0 con 0x7f9aa01017a0 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa7d4f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010d5a0 con 0x7f9aa0103350 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52518/0 (socket says 172.21.15.33:52518) 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 -- 172.21.15.33:0/1275420853 learned_addr learned my addr 172.21.15.33:0/1275420853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa62ec640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0103350 0x7f9aa019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa5aeb640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 0x7f9aa0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 -- 172.21.15.33:0/1275420853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0103350 msgr2=0x7f9aa019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0103350 0x7f9aa019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 -- 172.21.15.33:0/1275420853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 msgr2=0x7f9aa0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 0x7f9aa0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa52ea640 1 -- 172.21.15.33:0/1275420853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa019f6c0 con 0x7f9aa0102990 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.079+0000 7f9aa5aeb640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 0x7f9aa0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.080+0000 7f9aa52ea640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0197fe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009230 tx=0x7f9a90031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.080+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9003f030 con 0x7f9aa0102990 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.080+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a90003ad0 con 0x7f9aa0102990 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.080+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa019f950 con 0x7f9aa0102990 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.080+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90035070 con 0x7f9aa0102990 2024-04-24T23:38:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.080+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa019fe40 con 0x7f9aa0102990 2024-04-24T23:38:46.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.081+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa0108ae0 con 0x7f9aa0102990 2024-04-24T23:38:46.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.082+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9a90042030 con 0x7f9aa0102990 2024-04-24T23:38:46.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.083+0000 7f9a96ffd640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9a8c0a2b90 0x7f9a8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.083+0000 7f9aa5aeb640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9a8c0a2b90 0x7f9a8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.083+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a900efa30 con 0x7f9aa0102990 2024-04-24T23:38:46.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.083+0000 7f9aa5aeb640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9a8c0a2b90 0x7f9a8c0a5050 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9aa01027f0 tx=0x7f9a9c005e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:46.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.087+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9a900bc600 con 0x7f9aa0102990 2024-04-24T23:38:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:45 smithi038 ceph-mon[32450]: pgmap v1153: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 89 KiB/s wr, 18 op/s 2024-04-24T23:38:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:45 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2600054878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:38:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:45 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1765679063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:38:46.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:45 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1660790977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:38:46.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:45 smithi117 ceph-mon[32529]: pgmap v1153: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 89 KiB/s wr, 18 op/s 2024-04-24T23:38:46.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:45 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2600054878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:38:46.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:45 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1765679063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:38:46.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:45 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1660790977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:38:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.270+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9aa0062340 con 0x7f9aa0102990 2024-04-24T23:38:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.271+0000 7f9a96ffd640 1 -- 172.21.15.33:0/1275420853 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f9a900bbf00 con 0x7f9aa0102990 2024-04-24T23:38:46.270 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9a8c0a2b90 msgr2=0x7f9a8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9a8c0a2b90 0x7f9a8c0a5050 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9aa01027f0 tx=0x7f9a9c005e70 comp rx=0 tx=0).stop 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 msgr2=0x7f9aa0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0197fe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009230 tx=0x7f9a90031b60 comp rx=0 tx=0).stop 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 shutdown_connections 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9a8c0a2b90 0x7f9a8c0a5050 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0103350 0x7f9aa019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa0102990 0x7f9aa0197fe0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.272+0000 7f9aa7d4f640 1 --2- 172.21.15.33:0/1275420853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9aa01017a0 0x7f9aa0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.273+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 >> 172.21.15.33:0/1275420853 conn(0x7f9aa00fcd10 msgr2=0x7f9aa0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:46.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.273+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 shutdown_connections 2024-04-24T23:38:46.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.273+0000 7f9aa7d4f640 1 -- 172.21.15.33:0/1275420853 wait complete. 2024-04-24T23:38:46.283 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 -- 172.21.15.33:0/788877645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1029a0 msgr2=0x7fe62c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 --2- 172.21.15.33:0/788877645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1029a0 0x7fe62c102e20 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe614009900 tx=0x7fe61402a410 comp rx=0 tx=0).stop 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 -- 172.21.15.33:0/788877645 shutdown_connections 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 --2- 172.21.15.33:0/788877645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe62c103360 0x7fe62c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 --2- 172.21.15.33:0/788877645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1029a0 0x7fe62c102e20 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 --2- 172.21.15.33:0/788877645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c1017b0 0x7fe62c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 -- 172.21.15.33:0/788877645 >> 172.21.15.33:0/788877645 conn(0x7fe62c0fcd20 msgr2=0x7fe62c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 -- 172.21.15.33:0/788877645 shutdown_connections 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.385+0000 7fe630b7e640 1 -- 172.21.15.33:0/788877645 wait complete. 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 Processor -- start 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 -- start start 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 0x7fe62c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe62c1029a0 0x7fe62c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c103360 0x7fe62c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10d410 con 0x7fe62c1017b0 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10d290 con 0x7fe62c1029a0 2024-04-24T23:38:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe630b7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10d590 con 0x7fe62c103360 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe62a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 0x7fe62c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe62a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 0x7fe62c0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52532/0 (socket says 172.21.15.33:52532) 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe62a59c640 1 -- 172.21.15.33:0/3558152483 learned_addr learned my addr 172.21.15.33:0/3558152483 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe62ad9d640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c103360 0x7fe62c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.386+0000 7fe623fff640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe62c1029a0 0x7fe62c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62a59c640 1 -- 172.21.15.33:0/3558152483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c103360 msgr2=0x7fe62c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62a59c640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c103360 0x7fe62c074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62a59c640 1 -- 172.21.15.33:0/3558152483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe62c1029a0 msgr2=0x7fe62c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62a59c640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe62c1029a0 0x7fe62c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62a59c640 1 -- 172.21.15.33:0/3558152483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c074900 con 0x7fe62c1017b0 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62ad9d640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c103360 0x7fe62c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe62a59c640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 0x7fe62c0754f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d770 tx=0x7fe61c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c017070 con 0x7fe62c1017b0 2024-04-24T23:38:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c009c30 con 0x7fe62c1017b0 2024-04-24T23:38:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c1a1be0 con 0x7fe62c1017b0 2024-04-24T23:38:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c003dd0 con 0x7fe62c1017b0 2024-04-24T23:38:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.387+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1a2100 con 0x7fe62c1017b0 2024-04-24T23:38:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.388+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c108ad0 con 0x7fe62c1017b0 2024-04-24T23:38:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.390+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe61c003f90 con 0x7fe62c1017b0 2024-04-24T23:38:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.390+0000 7fe6237fe640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6180a2b90 0x7fe6180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:38:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.391+0000 7fe623fff640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6180a2b90 0x7fe6180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:38:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.391+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe61c0cf280 con 0x7fe62c1017b0 2024-04-24T23:38:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.391+0000 7fe623fff640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6180a2b90 0x7fe6180a5050 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe6140315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:38:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.394+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe61c009de0 con 0x7fe62c1017b0 2024-04-24T23:38:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.628+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe62c02d420 con 0x7fe62c1017b0 2024-04-24T23:38:46.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.636+0000 7fe6237fe640 1 -- 172.21.15.33:0/3558152483 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740371 (secure 0 0 0) 0x7fe61c09bdf0 con 0x7fe62c1017b0 2024-04-24T23:38:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 20, 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 14148, 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3494091302 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3494091302 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/3494091302", 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-04-24T23:03:05.692638+0000", 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 14214, 2024-04-24T23:38:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-24T23:38:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:38:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-24T23:38:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-24T23:38:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:38:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:38:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:38:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-24T23:38:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:38:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:38:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-24T23:38:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-24T23:38:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-24T23:38:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-24T23:38:46.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.814 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-24T23:38:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-24T23:38:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-24T23:38:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:38:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-24T23:38:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-24T23:38:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-24T23:38:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-24T23:38:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-24T23:38:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-24T23:38:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-24T23:38:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:38:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-24T23:38:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-24T23:38:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.979 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.980 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:46.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-24T23:38:46.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-24T23:38:46.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-24T23:38:46.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-24T23:38:46.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-24T23:38:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-24T23:38:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-24T23:38:46.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-24T23:38:46.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-24T23:38:46.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-24T23:38:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:38:46.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:38:46.998 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:46.999 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.002 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.012 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-24T23:38:47.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.015 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.021 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-24T23:38:47.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.041 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.059 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-24T23:38:47.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24116, 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.071 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-24T23:38:47.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-24T23:38:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-24T23:38:47.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.082 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:38:47.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:38:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.091 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-24T23:38:47.097 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-24T23:38:47.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-24T23:38:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-24T23:38:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:38:47.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:38:47.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-24T23:38:47.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.121 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-24T23:38:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-24T23:38:47.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:38:47.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:38:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-24T23:38:47.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:38:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:38:47.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.141 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.146 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-24T23:38:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:38:47.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:38:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.164 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.166 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:38:47.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:38:47.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:38:47.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:38:47.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:38:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:38:47.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:38:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:38:47.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.208 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-24T23:38:47.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-24T23:38:47.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.217 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.223 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-24T23:38:47.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-24T23:38:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.227 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.232 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:38:47.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:38:47.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.240 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-24T23:38:47.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-24T23:38:47.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-24T23:38:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.248 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-24T23:38:47.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:38:47.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:38:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.265 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:38:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-24T23:38:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-24T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:38:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-24T23:38:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-24T23:38:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-24T23:38:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-24T23:38:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:47.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.377 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-24T23:38:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-24T23:38:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-24T23:38:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-24T23:38:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-24T23:38:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-24T23:38:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-24T23:38:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-24T23:38:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-24T23:38:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-24T23:38:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-24T23:38:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "restful" 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-24T23:38:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.483 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.483 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-24T23:38:47.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-24T23:38:47.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-24T23:38:47.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-24T23:38:47.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:38:47.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:38:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.503 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-24T23:38:47.509 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-24T23:38:47.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-24T23:38:47.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:38:47.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.524 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-24T23:38:47.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.526 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-24T23:38:47.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-24T23:38:47.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-24T23:38:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-24T23:38:47.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-24T23:38:47.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-24T23:38:47.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-24T23:38:47.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-24T23:38:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-24T23:38:47.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-24T23:38:47.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:38:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-24T23:38:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:38:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:38:47.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-24T23:38:47.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-24T23:38:47.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-24T23:38:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-24T23:38:47.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-24T23:38:47.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-24T23:38:47.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-24T23:38:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:38:47.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:38:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:38:47.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.600 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:38:47.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:38:47.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:38:47.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:38:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.628 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-24T23:38:47.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.633 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-24T23:38:47.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.646 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.652 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.654 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.655 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.656 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.658 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-24T23:38:47.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-24T23:38:47.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-24T23:38:47.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-24T23:38:47.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-24T23:38:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.682 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.684 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.686 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.689 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.689 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.691 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-24T23:38:47.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-24T23:38:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.706 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:38:47.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:38:47.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.721 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.726 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.728 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.730 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-24T23:38:47.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.737 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.738 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.739 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.747 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:38:47.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.759 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-24T23:38:47.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-24T23:38:47.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.768 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-24T23:38:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.778 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.779 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.804 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-24T23:38:47.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-24T23:38:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-24T23:38:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-24T23:38:47.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.844 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:38:47.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.876 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-24T23:38:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:38:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-24T23:38:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.888 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-24T23:38:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-24T23:38:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-24T23:38:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-24T23:38:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-24T23:38:47.898 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-24T23:38:47.899 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-24T23:38:47.900 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 3, 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-24T23:38:47.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4249192916 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-24T23:38:47.902 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1685989953 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2578669762 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-24T23:38:47.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3132001874 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-24T23:38:47.904 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:47.905 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:46 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2300755786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:38:47.905 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:46 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1275420853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:38:47.905 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:46 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3558152483' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:38:47.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:46 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2300755786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:38:47.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:46 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1275420853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:38:47.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:46 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3558152483' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:38:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6180a2b90 msgr2=0x7fe6180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6180a2b90 0x7fe6180a5050 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe6140315d0 comp rx=0 tx=0).stop 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 msgr2=0x7fe62c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 0x7fe62c0754f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d770 tx=0x7fe61c00dc30 comp rx=0 tx=0).stop 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 shutdown_connections 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fe6180a2b90 0x7fe6180a5050 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c103360 0x7fe62c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe62c1029a0 0x7fe62c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 --2- 172.21.15.33:0/3558152483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c1017b0 0x7fe62c0754f0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 >> 172.21.15.33:0/3558152483 conn(0x7fe62c0fcd20 msgr2=0x7fe62c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 shutdown_connections 2024-04-24T23:38:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:38:46.641+0000 7fe630b7e640 1 -- 172.21.15.33:0/3558152483 wait complete. 2024-04-24T23:38:47.913 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T23:38:47.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T23:38:47.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:46 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2300755786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:38:47.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:46 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1275420853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:38:47.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:46 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3558152483' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:38:48.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:47 smithi038 ceph-mon[32450]: pgmap v1154: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 68 KiB/s wr, 13 op/s 2024-04-24T23:38:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:38:48.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:47 smithi117 ceph-mon[32529]: pgmap v1154: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 68 KiB/s wr, 13 op/s 2024-04-24T23:38:48.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:47 smithi033 ceph-mon[29201]: pgmap v1154: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 68 KiB/s wr, 13 op/s 2024-04-24T23:38:49.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:49 smithi033 ceph-mon[29201]: pgmap v1155: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-04-24T23:38:50.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:49 smithi038 ceph-mon[32450]: pgmap v1155: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-04-24T23:38:50.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:49 smithi117 ceph-mon[32529]: pgmap v1155: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 463, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 458, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 823648, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 283312, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.063494419, 2024-04-24T23:38:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.024229628, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.027947696, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.009224080, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 457, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.017177283, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000037587 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 823648, 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 283312 2024-04-24T23:38:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 6, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 352, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3344, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 217632, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.022440894, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.017540537, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002028685, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 351, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.013923728, 2024-04-24T23:38:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000039668 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 217632 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-04-24T23:38:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000363877, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000266352, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-24T23:38:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:38:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:38:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148704, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 29, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-24T23:38:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 465, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 758186, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 465, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 465, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 758186, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:50.318 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:51.105 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-24T23:38:51.105 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-24T23:38:51.105 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T23:38:51.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T23:38:51.361 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:38:52.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:51 smithi038 ceph-mon[32450]: pgmap v1156: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-24T23:38:52.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:51 smithi117 ceph-mon[32529]: pgmap v1156: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-24T23:38:52.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:51 smithi033 ceph-mon[29201]: pgmap v1156: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-24T23:38:53.189 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 469, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 464, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 825280, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 287152, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.059448692, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.024036388, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.024297150, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.008897933, 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 463, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.016941898, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000036591 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 825280, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 287152 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 5, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 356, 2024-04-24T23:38:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3184, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 140368, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 3, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.023153750, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.018276671, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001964109, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 355, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.015434350, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000043477 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 140368 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-24T23:38:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000478249, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000381515, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:38:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:38:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:38:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148967, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 29, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-24T23:38:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 470, 2024-04-24T23:38:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 758939, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 470, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 470, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 758939, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:53.200 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:53.905 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T23:38:53.905 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T23:38:53.905 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T23:38:53.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T23:38:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:38:54.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:53 smithi038 ceph-mon[32450]: pgmap v1157: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-24T23:38:54.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:53 smithi117 ceph-mon[32529]: pgmap v1157: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-24T23:38:54.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:53 smithi033 ceph-mon[29201]: pgmap v1157: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 5299, 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 5301, 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 9186131, 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 1685089, 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-04-24T23:38:55.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.389927051, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.105715709, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.108824701, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.147452392, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5299, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.163362658, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000030828 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 354, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 876, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 786234, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-24T23:38:55.890 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.023152279, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.017490953, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002755550, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000102904, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 352, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.014349823, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000040766 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 785536 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 495, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 472, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 913084, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 286894, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.060144713, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.020864823, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.028990663, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.009699263, 2024-04-24T23:38:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 472, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.019438100, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000041182 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 912816, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 286576 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-24T23:38:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 8940, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.203556013, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000022769 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-04-24T23:38:55.894 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-04-24T23:38:55.895 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 1060, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 1060, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 1060, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-04-24T23:38:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-04-24T23:38:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:38:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-04-24T23:38:55.899 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:38:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 6212, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-04-24T23:38:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 8, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 2, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 11896698, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 11895560, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 6212 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 10, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 1075, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 1065, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 8, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 21, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 13, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 1060, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 1082, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 22, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-04-24T23:38:55.903 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 53688, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 215420 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-04-24T23:38:55.904 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.905 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.906 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-04-24T23:38:55.908 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.911 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-04-24T23:38:55.913 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-04-24T23:38:55.915 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 2, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 2, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 3906, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:38:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:38:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-24T23:38:55.918 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 183118, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 49, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:38:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 8888, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 170, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 779160, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 4335, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-04-24T23:38:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 5302, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 5302, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5302, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 6.341901940, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001196133 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5302, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 5303 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.921 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 5302, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 5302, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 3266, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 2034, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-04-24T23:38:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:38:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-04-24T23:38:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 94, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 4, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 4, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 6, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 2, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-04-24T23:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-04-24T23:38:55.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 5794, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 9550420, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 5794, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-24T23:38:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5794, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 9550420, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-24T23:38:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 5302, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 7702399, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5302, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 7702399, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-04-24T23:38:55.929 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 5302, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 5302, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5302, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 5302, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.931 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x559db33e46e0": { 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-04-24T23:38:55.932 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:55.933 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:55.933 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:55.933 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:55.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:55.933 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:56.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:55 smithi038 ceph-mon[32450]: pgmap v1158: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:38:56.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:55 smithi117 ceph-mon[32529]: pgmap v1158: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:38:56.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:55 smithi033 ceph-mon[29201]: pgmap v1158: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:38:56.644 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-24T23:38:56.645 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-24T23:38:56.645 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T23:38:56.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T23:38:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:38:58.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:57 smithi038 ceph-mon[32450]: pgmap v1159: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:38:58.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:57 smithi117 ceph-mon[32529]: pgmap v1159: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:38:58.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:57 smithi033 ceph-mon[29201]: pgmap v1159: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:38:58.532 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-24T23:38:58.532 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:38:58.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 471, 2024-04-24T23:38:58.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 466, 2024-04-24T23:38:58.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 825824, 2024-04-24T23:38:58.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 288432, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.061080274, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.024435805, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.025667827, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.008894065, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 465, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.016906002, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000036356 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 825824, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 288432 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 356, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 218240, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-24T23:38:58.534 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.022427966, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.018159743, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001364685, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 354, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.016089587, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000045450 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:58.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 218240 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3024, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000775405, 2024-04-24T23:38:58.536 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000040363, 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000703546, 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000794152, 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000264717 2024-04-24T23:38:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-24T23:38:58.538 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-24T23:38:58.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.539 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-24T23:38:58.539 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:38:58.540 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:38:58.541 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148721, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-24T23:38:58.542 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-24T23:38:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 473, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 759226, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 473, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-24T23:38:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 473, 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 759226, 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-24T23:38:58.545 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-24T23:38:59.253 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T23:38:59.253 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T23:38:59.253 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T23:38:59.253 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T23:38:59.574 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:39:00.023 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:38:59 smithi038 ceph-mon[32450]: pgmap v1160: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:00.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:38:59 smithi117 ceph-mon[32529]: pgmap v1160: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:00.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:38:59 smithi033 ceph-mon[29201]: pgmap v1160: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 343677, 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 472832, 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 78454380, 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 359644927, 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 100, 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 7, 2024-04-24T23:39:01.699 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 27.230806727, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 10.818121996, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 11.869828561, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 4.789679757, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 472836, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 39.967382913, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000084526 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 570914, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.093161271, 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000163 2024-04-24T23:39:01.700 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 940064, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 333216 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 256533, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 348402, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 57493085, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 558692903, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 14, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 21.206143164, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 6.944349627, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 7.314306523, 2024-04-24T23:39:01.701 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 6.170042388, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 348400, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 35.879733907, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000102984 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 289292, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.053373479, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000184 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 701232 2024-04-24T23:39:01.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1289521, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1938148, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 447421520, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 2289266346, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 12, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 7, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 95.243529310, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 42.242854461, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 57.425386152, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 8.332982941, 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:01.703 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1938142, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 169.682138135, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000087548 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2613584, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.466384005, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000178 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 383882016, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 1575704880 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-24T23:39:01.704 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 282618, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 585.054474629, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002070124 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 229426, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 79.772490621, 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000347704 2024-04-24T23:39:01.705 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 810731, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 804092, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 804092, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 6097.541170439, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.007583138 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 6639, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 706, 2024-04-24T23:39:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 51, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 60, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 7542, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 4349, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 2518, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 675, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 1328, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 278, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 1, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 1, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 3, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 1255724, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 711994, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 6633, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 7, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-24T23:39:01.707 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 11353, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 10, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 153, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 10, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 201, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 16971, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 309, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 104054838, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 16971, 2024-04-24T23:39:01.708 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 2313, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 2313, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 14658, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 12, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 33946, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 494716, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 215416, 2024-04-24T23:39:01.709 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 101535, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 46363, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 520410 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 19, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 6, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 4, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 10, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 47, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 15, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 134871, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 134856, 2024-04-24T23:39:01.710 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 11, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 111, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-24T23:39:01.711 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 628611, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 528471, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 528471, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 100140, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 12290, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 771, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 643, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 643, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 128, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 9, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 17, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 1100986264, 2024-04-24T23:39:01.712 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 1303547116, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 166132, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1420.065699760, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.008547815 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 35, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 134938, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 134903, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 87, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 594, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 507, 2024-04-24T23:39:01.713 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 7542, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 271634, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 264092, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 1, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 145289, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 145288, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 109896, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 223612 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 1066901, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 4569, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 810731, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 17152, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 26087, 2024-04-24T23:39:01.714 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 134701, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 945.652340327, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.007020380 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 11889, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 160.784599044, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.013523811 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-24T23:39:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 283735, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1457.332392751, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005136244 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-24T23:39:01.716 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000115512, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000115512 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.717 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.718 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 86, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.591009747, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.006872206 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.719 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 9, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.005947657, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000660850 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 207547, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 2347.917972414, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.011312704 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-24T23:39:01.720 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 27291, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 190.018081246, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.006962664 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.721 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 91, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.489121556, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005374962 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.722 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 114, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 4.245898710, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.037244725 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3848, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 21.549748438, 2024-04-24T23:39:01.723 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005600246 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.724 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 134780, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 968.953943037, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.007189152 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.725 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 42209, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 42209, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 1578, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:01.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:01.728 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 291282, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 132, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:01.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8888, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 170, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 5927337, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 84414, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-24T23:39:01.730 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 464615, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 1335447752, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 28, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 464615, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 464615, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3871.870526631, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.008333503 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 464615, 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 470142 2024-04-24T23:39:01.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 464615, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 20098, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 444517, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 135, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 1054, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 3127, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 169259, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 1396, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 50, 2024-04-24T23:39:01.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 270589, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 16971, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 1637, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-24T23:39:01.733 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-24T23:39:01.734 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 94, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 12, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 12, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 4301, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 1623 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:39:01.735 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 16, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 269636, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 134856, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.736 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 1889727, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 361661240, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 1889727, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1889727, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 361661240, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.737 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 464615, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1350146448, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 464615, 2024-04-24T23:39:01.738 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1350146448, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:01.739 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 464615, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 464615, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 464615, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 464615, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 134856, 2024-04-24T23:39:01.740 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 14698696, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 134856, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3127, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 14698696, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x557988026500": { 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 628611, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1299352812, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 628611, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 166132, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1299352812, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:01.742 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-24T23:39:02.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:01 smithi038 ceph-mon[32450]: pgmap v1161: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:39:02.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:01 smithi117 ceph-mon[32529]: pgmap v1161: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:39:02.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:01 smithi033 ceph-mon[29201]: pgmap v1161: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:39:02.518 INFO:tasks.check_counter:Daemon mds.b mds.exported=10 2024-04-24T23:39:02.519 INFO:tasks.check_counter:Daemon mds.b mds.imported=10 2024-04-24T23:39:02.519 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T23:39:02.519 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T23:39:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:39:04.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:03 smithi038 ceph-mon[32450]: pgmap v1162: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:04.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:03 smithi117 ceph-mon[32529]: pgmap v1162: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:04.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:03 smithi033 ceph-mon[29201]: pgmap v1162: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:04.850 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-24T23:39:04.851 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:04.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-04-24T23:39:04.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-04-24T23:39:04.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:39:04.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-04-24T23:39:04.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-24T23:39:04.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-04-24T23:39:04.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.001310694, 2024-04-24T23:39:04.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:39:04.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000245928, 2024-04-24T23:39:04.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:39:04.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:04.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 471, 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 466, 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 825824, 2024-04-24T23:39:04.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 288432, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.056234184, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.020752214, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.024743287, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.009943601, 2024-04-24T23:39:04.855 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:04.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 465, 2024-04-24T23:39:04.856 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.018556143, 2024-04-24T23:39:04.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000039905 2024-04-24T23:39:04.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.856 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 825824, 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 288432 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 6, 2024-04-24T23:39:04.857 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 358, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 3344, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 218688, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.015235765, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.011060165, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001378439, 2024-04-24T23:39:04.858 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:39:04.859 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:04.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 357, 2024-04-24T23:39:04.859 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.018011194, 2024-04-24T23:39:04.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000050451 2024-04-24T23:39:04.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.859 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:04.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:04.860 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:04.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:04.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.860 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 218688 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:04.861 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:04.862 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:04.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:04.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:04.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:04.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:04.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:04.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:04.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-24T23:39:04.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148704, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 29, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:04.866 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:04.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-24T23:39:04.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-24T23:39:04.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:04.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:04.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:04.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:04.868 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-04-24T23:39:04.868 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-04-24T23:39:04.868 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:04.868 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:04.868 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:04.869 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 473, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 759226, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 473, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:04.870 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 473, 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 759226, 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:04.871 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-24T23:39:05.808 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-24T23:39:05.808 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-24T23:39:05.809 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T23:39:05.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T23:39:06.063 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:39:06.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:05 smithi038 ceph-mon[32450]: pgmap v1163: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:06.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:05 smithi117 ceph-mon[32529]: pgmap v1163: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:06.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:05 smithi033 ceph-mon[29201]: pgmap v1163: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:07.758 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1774, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1774, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 364481015, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 563372, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 8, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.322358487, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.038583421, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.229949103, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.041455712, 2024-04-24T23:39:07.759 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1773, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.064843820, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000036572 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2101, 2024-04-24T23:39:07.760 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2454, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 503858424, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 1369677, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.483731491, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.050218722, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.360500532, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.056966937, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2452, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.079250429, 2024-04-24T23:39:07.761 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000032320 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 701872 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 3879, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3858, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 433921967, 2024-04-24T23:39:07.762 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 1365440, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.567524222, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.089873534, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.359478582, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.096119488, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3857, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.120767626, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000031311 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.763 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 911152, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 289328 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 11752, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.552151879, 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000046983 2024-04-24T23:39:07.764 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-24T23:39:07.765 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 134995, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 78927, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 56068, 2024-04-24T23:39:07.766 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 22242, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-24T23:39:07.767 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:39:07.768 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-24T23:39:07.769 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:39:07.770 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-24T23:39:07.771 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 628611, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 128, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 53, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 1100986264, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 1303547116, 2024-04-24T23:39:07.772 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 1303546517, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 628611 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 28, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 134944, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 134916, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 79, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 592, 2024-04-24T23:39:07.773 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 513, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 134995, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 136184, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 1189, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 166944, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215420 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-04-24T23:39:07.774 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.776 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-24T23:39:07.777 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.778 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-24T23:39:07.779 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-24T23:39:07.780 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-24T23:39:07.781 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.784 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-24T23:39:07.785 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-04-24T23:39:07.786 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 2729, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:07.787 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:07.788 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 239986, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 93, 2024-04-24T23:39:07.789 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8888, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 170, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-24T23:39:07.790 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 95211668, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 542124, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 7254, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 7254, 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-24T23:39:07.791 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 7254, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 17.375696712, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002395326 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 7254, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 7255 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 7254, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 7254, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 3963, 2024-04-24T23:39:07.792 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 3289, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:39:07.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-24T23:39:07.794 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 94, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 11, 2024-04-24T23:39:07.795 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 11, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-24T23:39:07.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 7750, 2024-04-24T23:39:07.797 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1301535478, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 7750, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 7750, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1301535478, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.798 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 7254, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1299360274, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 7254, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1299360274, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.799 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 7254, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 7254, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 7254, 2024-04-24T23:39:07.800 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 7254, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:07.801 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55dae4c7c6e0": { 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-24T23:39:07.802 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:07.803 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-24T23:39:08.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:07 smithi038 ceph-mon[32450]: pgmap v1164: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-04-24T23:39:08.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:07 smithi117 ceph-mon[32529]: pgmap v1164: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-04-24T23:39:08.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:07 smithi033 ceph-mon[29201]: pgmap v1164: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-04-24T23:39:08.471 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-24T23:39:08.471 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-24T23:39:08.471 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T23:39:08.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T23:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:39:10.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:09 smithi038 ceph-mon[32450]: pgmap v1165: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:10.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:09 smithi117 ceph-mon[32529]: pgmap v1165: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:10.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:09 smithi033 ceph-mon[29201]: pgmap v1165: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:39:10.613 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-24T23:39:10.613 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.001486893, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000392999, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:10.614 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 359, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 320, 2024-04-24T23:39:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 218768, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.015507522, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.011327174, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001054387, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 357, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.014032946, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000039307 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:10.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 218768 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 479, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 472, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 829936, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 291440, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.058668961, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.020869218, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.027838902, 2024-04-24T23:39:10.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.010025759, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 472, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.017510870, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000037099 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 829936, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 291440 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.618 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:10.619 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:10.620 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148704, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 29, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:10.621 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:10.622 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 477, 2024-04-24T23:39:10.623 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 759746, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 477, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 477, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 759746, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-24T23:39:10.624 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-24T23:39:11.353 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-24T23:39:11.354 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-24T23:39:11.354 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T23:39:11.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T23:39:11.658 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:39:12.114 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:11 smithi117 ceph-mon[32529]: pgmap v1166: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:39:12.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:11 smithi033 ceph-mon[29201]: pgmap v1166: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:39:12.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:11 smithi038 ceph-mon[32450]: pgmap v1166: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 214622, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 114323, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 58680126, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 28392016, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 60, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 9.622521516, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 3.529781914, 2024-04-24T23:39:13.632 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 8.341488416, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.145013945, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 114322, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.197989339, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000019226 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 321689, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.044219666, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000137 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 932688, 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 314720 2024-04-24T23:39:13.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 943, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1298, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 234659, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1629176, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 60, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.085973295, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.026068208, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.029354792, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.024283978, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1296, 2024-04-24T23:39:13.634 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.090977440, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000070198 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 61, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000006090, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000099 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 798256 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 46976, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 51431, 2024-04-24T23:39:13.635 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 15146045, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 16934379, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 12, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 3.374218937, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 1.298489887, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 2.764300087, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.065154849, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 51430, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.435555306, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000027912 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:13.636 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 92154, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.012670549, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000137 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 14438784, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 15885472 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-24T23:39:13.637 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4538, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.463587075, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000542879 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3275, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.155371641, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000047441 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-24T23:39:13.638 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 41460, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 5822, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5822, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 44.025061400, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007561844 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 35638, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-04-24T23:39:13.639 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 16, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 2, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 14, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 13, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 76, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 4, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 50309, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 7659, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 35636, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 23, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:39:13.640 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 350, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 1, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 1, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 3, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 140, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 1, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 309, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 104054838, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 140, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-04-24T23:39:13.641 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 140, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 105, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 4033, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 2076, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 633, 2024-04-24T23:39:13.642 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 346, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 5939 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 48, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 34, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 41, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 1053, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 1053, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-04-24T23:39:13.643 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 2, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.644 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 6212, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 6212, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 8, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 8, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-04-24T23:39:13.645 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 11896698, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1985, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 13.330032105, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006715381 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 21, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 1112, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 1091, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 20, 2024-04-24T23:39:13.646 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 59, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 39, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 16, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 1232, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 1216, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 1127, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 1127, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 61096, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 223612 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 47571, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 5, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 41460, 2024-04-24T23:39:13.647 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 374, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 2518, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1050, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 17.458820190, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.016627447 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001346715, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001346715 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1355, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 12.903740453, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.009523055 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-04-24T23:39:13.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.651 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-04-24T23:39:13.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.653 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1733, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.198767349, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000691729 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 350, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 5.412859099, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.015465311 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.656 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 280, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.190416733, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000680059 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.657 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1053, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 6.859110861, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006513875 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:39:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 594, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 594, 2024-04-24T23:39:13.659 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 1588, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:13.660 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:13.661 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 300503, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 118, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:13.662 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8888, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 170, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 84429, 2024-04-24T23:39:13.663 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 375, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 5762, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 7996887, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 29, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 5762, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5762, 2024-04-24T23:39:13.664 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 50.560048672, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008774739 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5762, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 6330 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 5762, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 557, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 5205, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-04-24T23:39:13.665 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 417, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 2402, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 269, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 2119, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 140, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:39:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-04-24T23:39:13.667 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 94, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 4, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 17, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 5, 2024-04-24T23:39:13.668 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 697, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 256 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.669 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 8, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 2106, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 1053, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 262537, 2024-04-24T23:39:13.670 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 48275101, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 262537, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 262537, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 48275101, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:13.671 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 5762, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 8111664, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 5762, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 8111664, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 5762, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 5762, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 5762, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 5762, 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:13.673 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1053, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 114777, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1053, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:13.674 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 417, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 114777, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x5644248b0500": { 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 6212, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 7702394, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:13.675 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 6212, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1985, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 7702394, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:13.676 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-24T23:39:14.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:13 smithi117 ceph-mon[32529]: pgmap v1167: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:14.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:13 smithi033 ceph-mon[29201]: pgmap v1167: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:14.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:13 smithi038 ceph-mon[32450]: pgmap v1167: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:14.578 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-04-24T23:39:14.578 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-04-24T23:39:14.578 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T23:39:14.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T23:39:14.825 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:39:16.190 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:15 smithi117 ceph-mon[32529]: pgmap v1168: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:39:16.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:15 smithi033 ceph-mon[29201]: pgmap v1168: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:39:16.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:15 smithi038 ceph-mon[32450]: pgmap v1168: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:39:17.007 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-24T23:39:17.007 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:17.007 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3799, 2024-04-24T23:39:17.007 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 3778, 2024-04-24T23:39:17.007 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 81588742, 2024-04-24T23:39:17.007 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1341134, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 7, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.334249212, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.092970959, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.129246941, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.091668021, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3777, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.127053272, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000033638 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.008 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 912400, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 292048 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 979, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1335, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 115370532, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1103808, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 10, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.139737074, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.031294186, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.077305874, 2024-04-24T23:39:17.009 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.023009505, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1333, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.048731417, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000036557 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 793056 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.010 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1748, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1748, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 92713114, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 555047, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.169713113, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.041414692, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.071712530, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.045756289, 2024-04-24T23:39:17.011 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1747, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.056549010, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000032369 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.012 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10113, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.227566462, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000022502 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-24T23:39:17.013 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-04-24T23:39:17.014 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 25918, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 4414, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 21504, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 19611, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 2, 2024-04-24T23:39:17.015 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-04-24T23:39:17.016 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:39:17.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-04-24T23:39:17.018 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:39:17.019 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 140703, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-04-24T23:39:17.020 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 132, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 13, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 80862051, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 291281454, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 291280855, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 140703 2024-04-24T23:39:17.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 29, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 25960, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 25931, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 24, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 104, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 80, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 25918, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 26363, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 445, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-04-24T23:39:17.022 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 75960, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215420 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-04-24T23:39:17.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-04-24T23:39:17.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-04-24T23:39:17.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.027 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.028 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-04-24T23:39:17.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:39:17.033 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 4, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 3, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 3874, 2024-04-24T23:39:17.034 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:17.035 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:17.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 240437, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 86, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:17.037 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8888, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 170, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 18342279, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 104568, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:17.038 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 6020, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 26, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 4, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 6020, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6020, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 7.128864899, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001184196 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-04-24T23:39:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6020, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 6021 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 6020, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 6020, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 3427, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 2590, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-04-24T23:39:17.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 2, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:39:17.041 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-04-24T23:39:17.042 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 94, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 4, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 11, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 13, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 2, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-04-24T23:39:17.043 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-04-24T23:39:17.044 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 6522, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 289056683, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 6522, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 6522, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 289056683, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.045 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 6020, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 287087160, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 6020, 2024-04-24T23:39:17.046 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 287087160, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:17.047 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 6020, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 6020, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 6020, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 6020, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:17.048 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55f77da966e0": { 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:17.049 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:17.051 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-24T23:39:17.825 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T23:39:17.825 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T23:39:17.825 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T23:39:17.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T23:39:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:39:18.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:17 smithi117 ceph-mon[32529]: pgmap v1169: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:39:18.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:17 smithi033 ceph-mon[29201]: pgmap v1169: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:39:18.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:17 smithi038 ceph-mon[32450]: pgmap v1169: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:39:19.930 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 53920, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 62584, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 12924381, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 166072842, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 374, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 5.157802978, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 1.886632569, 2024-04-24T23:39:19.931 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 1.356556935, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 1.659244375, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 62585, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 3.968201677, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000063404 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 29357, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.005401102, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000183 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 42560, 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 1456 2024-04-24T23:39:19.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 341754, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 464801, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 116075758, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 432597350, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 32, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 23.982326568, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 10.096741475, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 15.264490082, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 1.641346259, 2024-04-24T23:39:19.933 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 464800, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 22.186056340, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000047732 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 688433, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.096418655, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000140 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 104203280, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 328008608 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.934 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 386684, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 257479, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 101527397, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 114397943, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 13, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 19.289739121, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 7.002068110, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 14.663135914, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 1.337361465, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 257475, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 9.288079956, 2024-04-24T23:39:19.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000036073 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 574423, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.079544502, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000138 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 954064, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 363152 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.936 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 100938, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 101.551901496, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001006081 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 48023, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4.592305618, 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000095627 2024-04-24T23:39:19.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 226842, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 167894, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 167894, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 729.134462460, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.004342826 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 58948, 2024-04-24T23:39:19.938 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 5, 2024-04-24T23:39:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 2, 2024-04-24T23:39:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 83, 2024-04-24T23:39:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 5, 2024-04-24T23:39:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 5, 2024-04-24T23:39:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 607, 2024-04-24T23:39:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 408, 2024-04-24T23:39:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 183, 2024-04-24T23:39:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 16, 2024-04-24T23:39:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 24, 2024-04-24T23:39:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 279, 2024-04-24T23:39:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 2, 2024-04-24T23:39:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 2, 2024-04-24T23:39:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 4, 2024-04-24T23:39:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 318975, 2024-04-24T23:39:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 154430, 2024-04-24T23:39:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 58948, 2024-04-24T23:39:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 10, 2024-04-24T23:39:19.941 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:39:19.942 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-04-24T23:39:19.942 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 1724, 2024-04-24T23:39:19.942 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-04-24T23:39:19.942 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-04-24T23:39:19.942 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 9, 2024-04-24T23:39:19.942 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 201, 2024-04-24T23:39:19.943 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 9, 2024-04-24T23:39:19.943 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 150, 2024-04-24T23:39:19.943 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-04-24T23:39:19.943 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 3725, 2024-04-24T23:39:19.943 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-04-24T23:39:19.944 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-04-24T23:39:19.944 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-04-24T23:39:19.944 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-04-24T23:39:19.944 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 3725, 2024-04-24T23:39:19.944 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-04-24T23:39:19.944 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:39:19.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:39:19.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 901, 2024-04-24T23:39:19.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 48, 2024-04-24T23:39:19.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 853, 2024-04-24T23:39:19.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 2824, 2024-04-24T23:39:19.946 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 16, 2024-04-24T23:39:19.946 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 13260, 2024-04-24T23:39:19.946 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 103101, 2024-04-24T23:39:19.946 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:39:19.946 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 60311, 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 25535, 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 10524, 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 101701 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.947 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 4, 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 21, 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 14, 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 17, 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 21, 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 6, 2024-04-24T23:39:19.948 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-04-24T23:39:19.949 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:39:19.949 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 25800, 2024-04-24T23:39:19.949 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 25794, 2024-04-24T23:39:19.949 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-04-24T23:39:19.949 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-04-24T23:39:19.949 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-04-24T23:39:19.950 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:39:19.950 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:39:19.950 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-04-24T23:39:19.950 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-04-24T23:39:19.950 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:39:19.950 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 48, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 9, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 10, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.951 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 140703, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 40456, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 36871, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 103832, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 3585, 2024-04-24T23:39:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 181, 2024-04-24T23:39:19.953 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 53, 2024-04-24T23:39:19.953 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 49, 2024-04-24T23:39:19.953 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 132, 2024-04-24T23:39:19.953 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 9, 2024-04-24T23:39:19.953 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-04-24T23:39:19.953 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 4, 2024-04-24T23:39:19.954 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 80862051, 2024-04-24T23:39:19.954 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 291281454, 2024-04-24T23:39:19.954 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-04-24T23:39:19.954 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-04-24T23:39:19.954 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 52327, 2024-04-24T23:39:19.954 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 331.317875491, 2024-04-24T23:39:19.955 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006331681 2024-04-24T23:39:19.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.955 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-04-24T23:39:19.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.955 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 32, 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 26034, 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 26002, 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 28, 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 127, 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 99, 2024-04-24T23:39:19.956 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 607, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 56061, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 55454, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 2, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 28922, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 28920, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 88728, 2024-04-24T23:39:19.957 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 231804 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 281039, 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 16937, 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 226842, 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 4730, 2024-04-24T23:39:19.958 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 35939, 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 25864, 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 82.704566920, 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003197671 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-04-24T23:39:19.959 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11076, 2024-04-24T23:39:19.960 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 273.069645024, 2024-04-24T23:39:19.960 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.024654175 2024-04-24T23:39:19.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.960 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-04-24T23:39:19.960 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.960 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.961 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.961 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-04-24T23:39:19.961 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.961 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.961 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 58068, 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 46.148420958, 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000794730 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.962 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000145326, 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000145326 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.963 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-04-24T23:39:19.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 33, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.109033334, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003304040 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.966 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000278146, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000278146 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 39835, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 218.815198539, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005493038 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-04-24T23:39:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6108, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 23.813298458, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003898706 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.968 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 33, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 15.142439294, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.458861796 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 32, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.312112655, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.009753520 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1056, 2024-04-24T23:39:19.970 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.033397549, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000978596 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 9, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.022551696, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002505744 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-04-24T23:39:19.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 25778, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 67.963374561, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002636487 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.972 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 4, 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 3, 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-04-24T23:39:19.973 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 14986, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 14986, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 1602, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:19.974 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:19.975 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:19.976 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 297381, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 117, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:19.977 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8888, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 170, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 572095, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 7696, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:19.978 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 111589, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 293224033, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 35, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 111589, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 111589, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1174.386867535, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.010524217 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.979 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 111589, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 113248 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 111589, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 4656, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 106933, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 231, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 876, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 53155, 2024-04-24T23:39:19.980 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 505, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 5, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 52210, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 3780, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 372, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-04-24T23:39:19.981 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-04-24T23:39:19.982 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 94, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 9, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-04-24T23:39:19.983 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 1589, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 14, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 511 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-04-24T23:39:19.984 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 13, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 51572, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 25794, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:19.985 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 782354, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 145789472, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 782354, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 782354, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 145789472, 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:19.986 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 111589, 2024-04-24T23:39:19.987 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 296035363, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 111589, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 296035363, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:19.988 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 111589, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 111589, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 111589, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 111589, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:39:19.989 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 25794, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 2811418, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 25794, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 828, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2811418, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.990 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55c96dae2500": { 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 140703, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 287087150, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 140703, 2024-04-24T23:39:19.991 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 52327, 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 287087150, 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:19.992 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-24T23:39:20.174 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:19 smithi117 ceph-mon[32529]: pgmap v1170: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:20.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:19 smithi033 ceph-mon[29201]: pgmap v1170: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:20.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:19 smithi038 ceph-mon[32450]: pgmap v1170: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:20.808 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-04-24T23:39:20.808 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2024-04-24T23:39:20.808 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T23:39:20.809 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T23:39:21.049 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:39:22.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:21 smithi117 ceph-mon[32529]: pgmap v1171: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:39:22.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:21 smithi033 ceph-mon[29201]: pgmap v1171: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:39:22.221 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:21 smithi038 ceph-mon[32450]: pgmap v1171: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 479, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 474, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 828000, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 293552, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.058560044, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.022077760, 2024-04-24T23:39:22.831 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.025147063, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.009765814, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 473, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.021268827, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000044965 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 828000, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 293552 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.832 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 6, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 364, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3344, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 219744, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.016330174, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.012222608, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001098658, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 363, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.020613930, 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000056787 2024-04-24T23:39:22.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 219744 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-04-24T23:39:22.834 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.000273413, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000166997, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:22.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:39:22.836 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:39:22.837 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 148704, 2024-04-24T23:39:22.838 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 29, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-24T23:39:22.839 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 928, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 481, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 760266, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 481, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-24T23:39:22.840 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 481, 2024-04-24T23:39:22.841 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 760266, 2024-04-24T23:39:22.841 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-24T23:39:22.841 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-24T23:39:22.841 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-24T23:39:22.841 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-24T23:39:22.842 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:22.842 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-24T23:39:22.842 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-24T23:39:23.571 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-24T23:39:23.571 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-24T23:39:23.571 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T23:39:23.582 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T23:39:23.582 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:39:23.582 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:39:23.616 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-24T23:39:23.616 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:39:23.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:39:23.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 -- 172.21.15.33:0/4191640002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae01006a0 msgr2=0x7fcae0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:23.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 --2- 172.21.15.33:0/4191640002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae01006a0 0x7fcae0100ac0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009900 tx=0x7fcad002a410 comp rx=0 tx=0).stop 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 -- 172.21.15.33:0/4191640002 shutdown_connections 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 --2- 172.21.15.33:0/4191640002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 0x7fcae0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 --2- 172.21.15.33:0/4191640002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0101890 0x7fcae0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 --2- 172.21.15.33:0/4191640002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae01006a0 0x7fcae0100ac0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 -- 172.21.15.33:0/4191640002 >> 172.21.15.33:0/4191640002 conn(0x7fcae00fbc30 msgr2=0x7fcae00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 -- 172.21.15.33:0/4191640002 shutdown_connections 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 -- 172.21.15.33:0/4191640002 wait complete. 2024-04-24T23:39:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.833+0000 7fcae5d90640 1 Processor -- start 2024-04-24T23:39:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 -- start start 2024-04-24T23:39:23.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae01006a0 0x7fcae006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 0x7fcae019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 0x7fcae019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010c2f0 con 0x7fcae0101890 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010c170 con 0x7fcae0102250 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcae5d90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010c470 con 0x7fcae01006a0 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 0x7fcae019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 0x7fcae019c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38526/0 (socket says 172.21.15.33:38526) 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 -- 172.21.15.33:0/412224840 learned_addr learned my addr 172.21.15.33:0/412224840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadf7fe640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae01006a0 0x7fcae006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadffff640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 0x7fcae019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 -- 172.21.15.33:0/412224840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae01006a0 msgr2=0x7fcae006a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae01006a0 0x7fcae006a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 -- 172.21.15.33:0/412224840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 msgr2=0x7fcae019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 0x7fcae019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadeffd640 1 -- 172.21.15.33:0/412224840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae019f770 con 0x7fcae0101890 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.834+0000 7fcadffff640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 0x7fcae019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.835+0000 7fcadeffd640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 0x7fcae019c840 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcad400d950 tx=0x7fcad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.835+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4017030 con 0x7fcae0101890 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.835+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae019fa00 con 0x7fcae0101890 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.835+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad4004020 con 0x7fcae0101890 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.835+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad40047a0 con 0x7fcae0101890 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.835+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae019ff50 con 0x7fcae0101890 2024-04-24T23:39:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.836+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae0107a40 con 0x7fcae0101890 2024-04-24T23:39:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.837+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcad40041d0 con 0x7fcae0101890 2024-04-24T23:39:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.838+0000 7fcadcff9640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcac80a2b90 0x7fcac80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.838+0000 7fcadf7fe640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcac80a2b90 0x7fcac80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.838+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcad40ceea0 con 0x7fcae0101890 2024-04-24T23:39:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.838+0000 7fcadf7fe640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcac80a2b90 0x7fcac80a5050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcad0005ab0 tx=0x7fcad0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:23.842+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcad409ba10 con 0x7fcae0101890 2024-04-24T23:39:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.033+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcae0062340 con 0x7fcae0101890 2024-04-24T23:39:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.034+0000 7fcadcff9640 1 -- 172.21.15.33:0/412224840 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7fcad409b310 con 0x7fcae0101890 2024-04-24T23:39:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:39:24.035 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcac80a2b90 msgr2=0x7fcac80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcac80a2b90 0x7fcac80a5050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcad0005ab0 tx=0x7fcad0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 msgr2=0x7fcae019c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 0x7fcae019c840 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcad400d950 tx=0x7fcad400de10 comp rx=0 tx=0).stop 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 shutdown_connections 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fcac80a2b90 0x7fcac80a5050 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcae0102250 0x7fcae019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae0101890 0x7fcae019c840 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 --2- 172.21.15.33:0/412224840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae01006a0 0x7fcae006a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 >> 172.21.15.33:0/412224840 conn(0x7fcae00fbc30 msgr2=0x7fcae0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 shutdown_connections 2024-04-24T23:39:24.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.037+0000 7fcae5d90640 1 -- 172.21.15.33:0/412224840 wait complete. 2024-04-24T23:39:24.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:23 smithi033 ceph-mon[29201]: pgmap v1172: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:24.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:05:32.235031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:10.244538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:11.097778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:10:14.786012+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:39:24.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:39:24.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 -- 172.21.15.33:0/1372759974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56741006a0 msgr2=0x7f5674100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 --2- 172.21.15.33:0/1372759974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56741006a0 0x7f5674100ac0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-04-24T23:39:24.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 -- 172.21.15.33:0/1372759974 shutdown_connections 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 --2- 172.21.15.33:0/1372759974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674102250 0x7f5674108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 --2- 172.21.15.33:0/1372759974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5674101890 0x7f5674101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 --2- 172.21.15.33:0/1372759974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56741006a0 0x7f5674100ac0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 -- 172.21.15.33:0/1372759974 >> 172.21.15.33:0/1372759974 conn(0x7f56740fbc30 msgr2=0x7f56740fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 -- 172.21.15.33:0/1372759974 shutdown_connections 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 -- 172.21.15.33:0/1372759974 wait complete. 2024-04-24T23:39:24.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.155+0000 7f567a129640 1 Processor -- start 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 -- start start 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56741006a0 0x7f5674199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674101890 0x7f567419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 0x7f56741a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f567410c2f0 con 0x7f5674102250 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f567410c170 con 0x7f56741006a0 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f567a129640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f567410c470 con 0x7f5674101890 2024-04-24T23:39:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 0x7f56741a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 0x7f56741a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38544/0 (socket says 172.21.15.33:38544) 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 -- 172.21.15.33:0/3495773208 learned_addr learned my addr 172.21.15.33:0/3495773208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f56737fe640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56741006a0 0x7f5674199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5672ffd640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674101890 0x7f567419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 -- 172.21.15.33:0/3495773208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674101890 msgr2=0x7f567419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674101890 0x7f567419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 -- 172.21.15.33:0/3495773208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56741006a0 msgr2=0x7f5674199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56741006a0 0x7f5674199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 -- 172.21.15.33:0/3495773208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a1940 con 0x7f5674102250 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5672ffd640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674101890 0x7f567419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.156+0000 7f5673fff640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 0x7f56741a12a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f566c009fd0 tx=0x7f566c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.157+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c017070 con 0x7f5674102250 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.157+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a1c30 con 0x7f5674102250 2024-04-24T23:39:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.157+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566c003e50 con 0x7f5674102250 2024-04-24T23:39:24.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.157+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0106c0 con 0x7f5674102250 2024-04-24T23:39:24.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.157+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a2150 con 0x7f5674102250 2024-04-24T23:39:24.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.158+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56741a2320 con 0x7f5674102250 2024-04-24T23:39:24.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.159+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f566c003970 con 0x7f5674102250 2024-04-24T23:39:24.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.159+0000 7f5670ff9640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f565c0a2c10 0x7f565c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.159+0000 7f56737fe640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f565c0a2c10 0x7f565c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.159+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f566c0cf190 con 0x7f5674102250 2024-04-24T23:39:24.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.160+0000 7f56737fe640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f565c0a2c10 0x7f565c0a50d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5664005ab0 tx=0x7f5664000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.164+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f566c09bd60 con 0x7f5674102250 2024-04-24T23:39:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.350+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56741a2590 con 0x7f5674102250 2024-04-24T23:39:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.350+0000 7f5670ff9640 1 -- 172.21.15.33:0/3495773208 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f566c09b660 con 0x7f5674102250 2024-04-24T23:39:24.351 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f565c0a2c10 msgr2=0x7f565c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f565c0a2c10 0x7f565c0a50d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5664005ab0 tx=0x7f5664000bc0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 msgr2=0x7f56741a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 0x7f56741a12a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f566c009fd0 tx=0x7f566c00b260 comp rx=0 tx=0).stop 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 shutdown_connections 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f565c0a2c10 0x7f565c0a50d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5674102250 0x7f56741a12a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5674101890 0x7f567419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 --2- 172.21.15.33:0/3495773208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56741006a0 0x7f5674199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 >> 172.21.15.33:0/3495773208 conn(0x7f56740fbc30 msgr2=0x7f56740fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 shutdown_connections 2024-04-24T23:39:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.352+0000 7f567a129640 1 -- 172.21.15.33:0/3495773208 wait complete. 2024-04-24T23:39:24.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:39:24.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:23 smithi038 ceph-mon[32450]: pgmap v1172: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:24.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:23 smithi117 ceph-mon[32529]: pgmap v1172: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:39:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 -- 172.21.15.33:0/491336879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc1017a0 msgr2=0x7f01bc101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/491336879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc1017a0 0x7f01bc101bc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009900 tx=0x7f01ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 -- 172.21.15.33:0/491336879 shutdown_connections 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/491336879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc103350 0x7f01bc109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/491336879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01bc102990 0x7f01bc102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/491336879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc1017a0 0x7f01bc101bc0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 -- 172.21.15.33:0/491336879 >> 172.21.15.33:0/491336879 conn(0x7f01bc0fcd10 msgr2=0x7f01bc0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 -- 172.21.15.33:0/491336879 shutdown_connections 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.470+0000 7f01c1bb0640 1 -- 172.21.15.33:0/491336879 wait complete. 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 Processor -- start 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 -- start start 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01bc1017a0 0x7f01bc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 0x7f01bc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc103350 0x7f01bc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc10d3f0 con 0x7f01bc102990 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc10d270 con 0x7f01bc1017a0 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01c1bb0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc10d570 con 0x7f01bc103350 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 0x7f01bc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 0x7f01bc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38562/0 (socket says 172.21.15.33:38562) 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01baffd640 1 -- 172.21.15.33:0/3590675772 learned_addr learned my addr 172.21.15.33:0/3590675772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.471+0000 7f01bb7fe640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01bc1017a0 0x7f01bc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01bbfff640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc103350 0x7f01bc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01baffd640 1 -- 172.21.15.33:0/3590675772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc103350 msgr2=0x7f01bc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01baffd640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc103350 0x7f01bc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01baffd640 1 -- 172.21.15.33:0/3590675772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01bc1017a0 msgr2=0x7f01bc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01baffd640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01bc1017a0 0x7f01bc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01baffd640 1 -- 172.21.15.33:0/3590675772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc19f660 con 0x7f01bc102990 2024-04-24T23:39:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01baffd640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 0x7f01bc197f80 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d950 tx=0x7f01b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0017030 con 0x7f01bc102990 2024-04-24T23:39:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b0004020 con 0x7f01bc102990 2024-04-24T23:39:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc19f950 con 0x7f01bc102990 2024-04-24T23:39:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b00047a0 con 0x7f01bc102990 2024-04-24T23:39:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.472+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc19fea0 con 0x7f01bc102990 2024-04-24T23:39:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.473+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc108ac0 con 0x7f01bc102990 2024-04-24T23:39:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.474+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f01b00041d0 con 0x7f01bc102990 2024-04-24T23:39:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.475+0000 7f01b8ff9640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01a40a2b90 0x7f01a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.475+0000 7f01bb7fe640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01a40a2b90 0x7f01a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.476+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f01b00cf4f0 con 0x7f01bc102990 2024-04-24T23:39:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.476+0000 7f01bb7fe640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01a40a2b90 0x7f01a40a5050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005ab0 tx=0x7f01ac000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.479+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f01b009c0a0 con 0x7f01bc102990 2024-04-24T23:39:24.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.663+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01bc02d380 con 0x7f01bc102990 2024-04-24T23:39:24.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.663+0000 7f01b8ff9640 1 -- 172.21.15.33:0/3590675772 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f01b009b9a0 con 0x7f01bc102990 2024-04-24T23:39:24.664 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:39:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01a40a2b90 msgr2=0x7f01a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01a40a2b90 0x7f01a40a5050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005ab0 tx=0x7f01ac000bc0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 msgr2=0x7f01bc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 0x7f01bc197f80 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d950 tx=0x7f01b000de10 comp rx=0 tx=0).stop 2024-04-24T23:39:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 shutdown_connections 2024-04-24T23:39:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f01a40a2b90 0x7f01a40a5050 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc103350 0x7f01bc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01bc102990 0x7f01bc197f80 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 --2- 172.21.15.33:0/3590675772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01bc1017a0 0x7f01bc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.665+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 >> 172.21.15.33:0/3590675772 conn(0x7f01bc0fcd10 msgr2=0x7f01bc105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.666+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 shutdown_connections 2024-04-24T23:39:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.666+0000 7f01c1bb0640 1 -- 172.21.15.33:0/3590675772 wait complete. 2024-04-24T23:39:24.677 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 -- 172.21.15.33:0/3961649981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 msgr2=0x7f6180108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 --2- 172.21.15.33:0/3961649981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f6180108ac0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f6178009cc0 tx=0x7f617802cad0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 -- 172.21.15.33:0/3961649981 shutdown_connections 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 --2- 172.21.15.33:0/3961649981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f6180108ac0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 --2- 172.21.15.33:0/3961649981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6180101870 0x7f6180101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 --2- 172.21.15.33:0/3961649981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6180100680 0x7f6180100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.781+0000 7f61868df640 1 -- 172.21.15.33:0/3961649981 >> 172.21.15.33:0/3961649981 conn(0x7f61800fbbf0 msgr2=0x7f61800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 -- 172.21.15.33:0/3961649981 shutdown_connections 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 -- 172.21.15.33:0/3961649981 wait complete. 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 Processor -- start 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 -- start start 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6180100680 0x7f6180195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6180101870 0x7f6180195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f618019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f618010b1b0 con 0x7f6180102230 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f618010b030 con 0x7f6180101870 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.782+0000 7f61868df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f618010b330 con 0x7f6180100680 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f618019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f618019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38578/0 (socket says 172.21.15.33:38578) 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 -- 172.21.15.33:0/3341300579 learned_addr learned my addr 172.21.15.33:0/3341300579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f617ffff640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6180100680 0x7f6180195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f617f7fe640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6180101870 0x7f6180195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 -- 172.21.15.33:0/3341300579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6180100680 msgr2=0x7f6180195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6180100680 0x7f6180195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 -- 172.21.15.33:0/3341300579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6180101870 msgr2=0x7f6180195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6180101870 0x7f6180195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 -- 172.21.15.33:0/3341300579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618019d450 con 0x7f6180102230 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f6184e7c640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f618019cde0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6178038f20 tx=0x7f6178038b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178046070 con 0x7f6180102230 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618019d6e0 con 0x7f6180102230 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6178004730 con 0x7f6180102230 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178041410 con 0x7f6180102230 2024-04-24T23:39:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.783+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618019dc20 con 0x7f6180102230 2024-04-24T23:39:24.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.784+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6180107a70 con 0x7f6180102230 2024-04-24T23:39:24.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.786+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f617802e630 con 0x7f6180102230 2024-04-24T23:39:24.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.786+0000 7f617d7fa640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61680a2b90 0x7f61680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:24.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.786+0000 7f617ffff640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61680a2b90 0x7f61680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:24.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.786+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f61780f1e30 con 0x7f6180102230 2024-04-24T23:39:24.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.787+0000 7f617ffff640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61680a2b90 0x7f61680a5050 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f61801016d0 tx=0x7f617000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.790+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6178034030 con 0x7f6180102230 2024-04-24T23:39:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.975+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6180062340 con 0x7f6180102230 2024-04-24T23:39:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.976+0000 7f617d7fa640 1 -- 172.21.15.33:0/3341300579 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f617802e920 con 0x7f6180102230 2024-04-24T23:39:24.976 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61680a2b90 msgr2=0x7f61680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61680a2b90 0x7f61680a5050 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f61801016d0 tx=0x7f617000b390 comp rx=0 tx=0).stop 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 msgr2=0x7f618019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f618019cde0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6178038f20 tx=0x7f6178038b30 comp rx=0 tx=0).stop 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 shutdown_connections 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f61680a2b90 0x7f61680a5050 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6180102230 0x7f618019cde0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6180101870 0x7f6180195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 --2- 172.21.15.33:0/3341300579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6180100680 0x7f6180195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 >> 172.21.15.33:0/3341300579 conn(0x7f61800fbbf0 msgr2=0x7f6180104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 shutdown_connections 2024-04-24T23:39:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:24.978+0000 7f61868df640 1 -- 172.21.15.33:0/3341300579 wait complete. 2024-04-24T23:39:24.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:39:25.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 -- 172.21.15.33:0/2863620793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 msgr2=0x7f33c0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 --2- 172.21.15.33:0/2863620793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c0101d10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009900 tx=0x7f33a802a410 comp rx=0 tx=0).stop 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 -- 172.21.15.33:0/2863620793 shutdown_connections 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 --2- 172.21.15.33:0/2863620793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c0102250 0x7f33c0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 --2- 172.21.15.33:0/2863620793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c0101d10 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 --2- 172.21.15.33:0/2863620793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c01006a0 0x7f33c0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 -- 172.21.15.33:0/2863620793 >> 172.21.15.33:0/2863620793 conn(0x7f33c00fbc10 msgr2=0x7f33c00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 -- 172.21.15.33:0/2863620793 shutdown_connections 2024-04-24T23:39:25.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.094+0000 7f33c494c640 1 -- 172.21.15.33:0/2863620793 wait complete. 2024-04-24T23:39:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 Processor -- start 2024-04-24T23:39:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 -- start start 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c01006a0 0x7f33c006a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c0102250 0x7f33c019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33c010c380 con 0x7f33c0101890 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33c010c200 con 0x7f33c0102250 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33c494c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33c010c500 con 0x7f33c01006a0 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38602/0 (socket says 172.21.15.33:38602) 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 -- 172.21.15.33:0/1443851148 learned_addr learned my addr 172.21.15.33:0/1443851148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33be59c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c01006a0 0x7f33c006a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 -- 172.21.15.33:0/1443851148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c01006a0 msgr2=0x7f33c006a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c01006a0 0x7f33c006a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 -- 172.21.15.33:0/1443851148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c0102250 msgr2=0x7f33c019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c0102250 0x7f33c019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.095+0000 7f33b7fff640 1 -- 172.21.15.33:0/1443851148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c019f7b0 con 0x7f33c0101890 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.096+0000 7f33b7fff640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c019c880 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009230 tx=0x7f33a8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.096+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a803f030 con 0x7f33c0101890 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.096+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a8003ce0 con 0x7f33c0101890 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.096+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c019f9e0 con 0x7f33c0101890 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.096+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8035070 con 0x7f33c0101890 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.096+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c019ff50 con 0x7f33c0101890 2024-04-24T23:39:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.097+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c01079c0 con 0x7f33c0101890 2024-04-24T23:39:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.098+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f33a8042030 con 0x7f33c0101890 2024-04-24T23:39:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.099+0000 7f33b77fe640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33ac0a2b90 0x7f33ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.099+0000 7f33be59c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33ac0a2b90 0x7f33ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.099+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33a80eff90 con 0x7f33c0101890 2024-04-24T23:39:25.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.099+0000 7f33be59c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33ac0a2b90 0x7f33ac0a5050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f33c01016f0 tx=0x7f33b000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:25.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.102+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f33a80bcb00 con 0x7f33c0101890 2024-04-24T23:39:25.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:24 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/412224840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:39:25.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:24 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3495773208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:39:25.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:24 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3590675772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:39:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.287+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f33c002d300 con 0x7f33c0101890 2024-04-24T23:39:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.288+0000 7f33b77fe640 1 -- 172.21.15.33:0/1443851148 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f33a80bc400 con 0x7f33c0101890 2024-04-24T23:39:25.288 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-24T23:39:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33ac0a2b90 msgr2=0x7f33ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33ac0a2b90 0x7f33ac0a5050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f33c01016f0 tx=0x7f33b000a2d0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 msgr2=0x7f33c019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c019c880 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009230 tx=0x7f33a8004430 comp rx=0 tx=0).stop 2024-04-24T23:39:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 shutdown_connections 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f33ac0a2b90 0x7f33ac0a5050 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33c0102250 0x7f33c019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c0101890 0x7f33c019c880 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 --2- 172.21.15.33:0/1443851148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c01006a0 0x7f33c006a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 >> 172.21.15.33:0/1443851148 conn(0x7f33c00fbc10 msgr2=0x7f33c0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 shutdown_connections 2024-04-24T23:39:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.290+0000 7f33c494c640 1 -- 172.21.15.33:0/1443851148 wait complete. 2024-04-24T23:39:25.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 -- 172.21.15.33:0/4062902040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a0102250 msgr2=0x7f74a0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 --2- 172.21.15.33:0/4062902040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a0102250 0x7f74a0108ae0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7498009ca0 tx=0x7f749802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 -- 172.21.15.33:0/4062902040 shutdown_connections 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 --2- 172.21.15.33:0/4062902040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a0102250 0x7f74a0108ae0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 --2- 172.21.15.33:0/4062902040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0101890 0x7f74a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 --2- 172.21.15.33:0/4062902040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74a01006a0 0x7f74a0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 -- 172.21.15.33:0/4062902040 >> 172.21.15.33:0/4062902040 conn(0x7f74a00fbc30 msgr2=0x7f74a00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 -- 172.21.15.33:0/4062902040 shutdown_connections 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.404+0000 7f74a489e640 1 -- 172.21.15.33:0/4062902040 wait complete. 2024-04-24T23:39:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 Processor -- start 2024-04-24T23:39:25.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 -- start start 2024-04-24T23:39:25.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 0x7f74a010b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74a0101890 0x7f74a0109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0102250 0x7f74a0109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010d440 con 0x7f74a01006a0 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010d2c0 con 0x7f74a0101890 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f74a489e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010d5c0 con 0x7f74a0102250 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f749e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 0x7f74a010b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f749e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 0x7f74a010b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38614/0 (socket says 172.21.15.33:38614) 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f749e59c640 1 -- 172.21.15.33:0/1533822612 learned_addr learned my addr 172.21.15.33:0/1533822612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f749ed9d640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0102250 0x7f74a0109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.405+0000 7f7495d9b640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74a0101890 0x7f74a0109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749e59c640 1 -- 172.21.15.33:0/1533822612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0102250 msgr2=0x7f74a0109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749e59c640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0102250 0x7f74a0109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749e59c640 1 -- 172.21.15.33:0/1533822612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74a0101890 msgr2=0x7f74a0109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749e59c640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74a0101890 0x7f74a0109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749e59c640 1 -- 172.21.15.33:0/1533822612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a010a430 con 0x7f74a01006a0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749ed9d640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0102250 0x7f74a0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f749e59c640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 0x7f74a010b020 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f749000c970 tx=0x7f749000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490004220 con 0x7f74a01006a0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74900043c0 con 0x7f74a01006a0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a1b60 con 0x7f74a01006a0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749000f450 con 0x7f74a01006a0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.406+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a20b0 con 0x7f74a01006a0 2024-04-24T23:39:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.407+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a01079c0 con 0x7f74a01006a0 2024-04-24T23:39:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.408+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f74900079b0 con 0x7f74a01006a0 2024-04-24T23:39:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.409+0000 7f7497fff640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f748c0a2b90 0x7f748c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.409+0000 7f7495d9b640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f748c0a2b90 0x7f748c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.409+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f74900cf2a0 con 0x7f74a01006a0 2024-04-24T23:39:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.409+0000 7f7495d9b640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f748c0a2b90 0x7f748c0a5050 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7488005fd0 tx=0x7f7488005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.413+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7490014030 con 0x7f74a01006a0 2024-04-24T23:39:25.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:24 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/412224840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:39:25.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:24 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3495773208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:39:25.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:24 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3590675772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:39:25.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:24 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/412224840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:39:25.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:24 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3495773208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:39:25.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:24 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3590675772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:39:25.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.610+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f74a002d170 con 0x7f74a01006a0 2024-04-24T23:39:25.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.610+0000 7f7497fff640 1 -- 172.21.15.33:0/1533822612 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+590 (secure 0 0 0) 0x7f74a002d170 con 0x7f74a01006a0 2024-04-24T23:39:25.610 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/946624938 2024-04-25T23:03:05.692528+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2813273186 2024-04-25T23:03:05.692528+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2813273186 2024-04-25T23:03:05.692528+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1214692934 2024-04-25T23:02:38.216643+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1172519445 2024-04-25T23:02:38.216643+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1299073362 2024-04-25T23:02:38.216643+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1299073362 2024-04-25T23:02:38.216643+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1574498973 2024-04-25T23:03:05.692528+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/288675277 2024-04-25T23:02:38.216643+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2398589700 2024-04-25T23:03:05.692528+0000 2024-04-24T23:39:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-04-24T23:39:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f748c0a2b90 msgr2=0x7f748c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f748c0a2b90 0x7f748c0a5050 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7488005fd0 tx=0x7f7488005850 comp rx=0 tx=0).stop 2024-04-24T23:39:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 msgr2=0x7f74a010b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 0x7f74a010b020 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f749000c970 tx=0x7f749000ce30 comp rx=0 tx=0).stop 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 shutdown_connections 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f748c0a2b90 0x7f748c0a5050 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a0102250 0x7f74a0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74a0101890 0x7f74a0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 --2- 172.21.15.33:0/1533822612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a01006a0 0x7f74a010b020 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 >> 172.21.15.33:0/1533822612 conn(0x7f74a00fbc30 msgr2=0x7f74a0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 shutdown_connections 2024-04-24T23:39:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:25.612+0000 7f74a489e640 1 -- 172.21.15.33:0/1533822612 wait complete. 2024-04-24T23:39:25.623 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:39:25.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:25.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:39:25.798 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-24T23:39:25.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:25.799 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:39:25.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:25.835 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-24T23:39:25.917 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:39:25.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:25.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:39:26.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:39:26.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-24T23:39:26.083 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T23:39:26.094 INFO:tasks.cephadm:Teardown begin 2024-04-24T23:39:26.095 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:39:26.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:39:26.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 -- 172.21.15.33:0/45806258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc102230 msgr2=0x7f64bc108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 --2- 172.21.15.33:0/45806258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc102230 0x7f64bc108ac0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009ca0 tx=0x7f64b402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 -- 172.21.15.33:0/45806258 shutdown_connections 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 --2- 172.21.15.33:0/45806258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc102230 0x7f64bc108ac0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 --2- 172.21.15.33:0/45806258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc101870 0x7f64bc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 --2- 172.21.15.33:0/45806258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc100680 0x7f64bc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 -- 172.21.15.33:0/45806258 >> 172.21.15.33:0/45806258 conn(0x7f64bc0fbbf0 msgr2=0x7f64bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 -- 172.21.15.33:0/45806258 shutdown_connections 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.199+0000 7f64c19f9640 1 -- 172.21.15.33:0/45806258 wait complete. 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 Processor -- start 2024-04-24T23:39:26.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 -- start start 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 0x7f64bc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc101870 0x7f64bc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc102230 0x7f64bc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d480 con 0x7f64bc100680 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d300 con 0x7f64bc101870 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64c19f9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d600 con 0x7f64bc102230 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 0x7f64bc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 0x7f64bc197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50054/0 (socket says 172.21.15.33:50054) 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 -- 172.21.15.33:0/3915782201 learned_addr learned my addr 172.21.15.33:0/3915782201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64b27fc640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc101870 0x7f64bc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64bb7fe640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc102230 0x7f64bc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 -- 172.21.15.33:0/3915782201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc102230 msgr2=0x7f64bc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc102230 0x7f64bc19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 -- 172.21.15.33:0/3915782201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc101870 msgr2=0x7f64bc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.200+0000 7f64baffd640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc101870 0x7f64bc197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64baffd640 1 -- 172.21.15.33:0/3915782201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc19f640 con 0x7f64bc100680 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64bb7fe640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc102230 0x7f64bc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64baffd640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 0x7f64bc197a20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00e9c0 tx=0x7f64ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac00cd80 con 0x7f64bc100680 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ac0035f0 con 0x7f64bc100680 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc19f930 con 0x7f64bc100680 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac010690 con 0x7f64bc100680 2024-04-24T23:39:26.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.201+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc19fe30 con 0x7f64bc100680 2024-04-24T23:39:26.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.202+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc1079a0 con 0x7f64bc100680 2024-04-24T23:39:26.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.203+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f64ac0037a0 con 0x7f64bc100680 2024-04-24T23:39:26.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:25 smithi033 ceph-mon[29201]: pgmap v1173: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:39:26.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3341300579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:39:26.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1443851148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:39:26.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:25 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1533822612' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:39:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.204+0000 7f64c09f7640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64a80a2c10 0x7f64a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.204+0000 7f64b27fc640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64a80a2c10 0x7f64a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.204+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64ac0cf920 con 0x7f64bc100680 2024-04-24T23:39:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.205+0000 7f64b27fc640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64a80a2c10 0x7f64a80a50d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f64bc198f30 tx=0x7f64a400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:26.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.208+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f64ac09c490 con 0x7f64bc100680 2024-04-24T23:39:26.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:25 smithi038 ceph-mon[32450]: pgmap v1173: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:39:26.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3341300579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:39:26.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1443851148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:39:26.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:25 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1533822612' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:39:26.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:25 smithi117 ceph-mon[32529]: pgmap v1173: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T23:39:26.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3341300579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:39:26.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1443851148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:39:26.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:25 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1533822612' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:39:26.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.433+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64bc02d420 con 0x7f64bc100680 2024-04-24T23:39:26.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.434+0000 7f64c09f7640 1 -- 172.21.15.33:0/3915782201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7f64ac09bd90 con 0x7f64bc100680 2024-04-24T23:39:26.434 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:39:26.434 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:38:17.286450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:39:26.434 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 40 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64a80a2c10 msgr2=0x7f64a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64a80a2c10 0x7f64a80a50d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f64bc198f30 tx=0x7f64a400b390 comp rx=0 tx=0).stop 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 msgr2=0x7f64bc197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 0x7f64bc197a20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00e9c0 tx=0x7f64ac00ee80 comp rx=0 tx=0).stop 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 shutdown_connections 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f64a80a2c10 0x7f64a80a50d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64bc102230 0x7f64bc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc101870 0x7f64bc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 --2- 172.21.15.33:0/3915782201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc100680 0x7f64bc197a20 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 >> 172.21.15.33:0/3915782201 conn(0x7f64bc0fbbf0 msgr2=0x7f64bc104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:26.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 shutdown_connections 2024-04-24T23:39:26.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.436+0000 7f64c19f9640 1 -- 172.21.15.33:0/3915782201 wait complete. 2024-04-24T23:39:26.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:39:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 -- 172.21.15.33:0/552299561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c1006a0 msgr2=0x7f7d9c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/552299561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c1006a0 0x7f7d9c100ac0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009900 tx=0x7f7d8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:39:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 -- 172.21.15.33:0/552299561 shutdown_connections 2024-04-24T23:39:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/552299561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d9c102250 0x7f7d9c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/552299561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c101890 0x7f7d9c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/552299561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c1006a0 0x7f7d9c100ac0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 -- 172.21.15.33:0/552299561 >> 172.21.15.33:0/552299561 conn(0x7f7d9c0fbc30 msgr2=0x7f7d9c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:26.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 -- 172.21.15.33:0/552299561 shutdown_connections 2024-04-24T23:39:26.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.550+0000 7f7da1fb7640 1 -- 172.21.15.33:0/552299561 wait complete. 2024-04-24T23:39:26.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 Processor -- start 2024-04-24T23:39:26.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 -- start start 2024-04-24T23:39:26.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d9c1006a0 0x7f7d9c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 0x7f7d9c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c102250 0x7f7d9c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c340 con 0x7f7d9c101890 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c1c0 con 0x7f7d9c1006a0 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7da1fb7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c4c0 con 0x7f7d9c102250 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7d9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 0x7f7d9c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7d9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 0x7f7d9c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50066/0 (socket says 172.21.15.33:50066) 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.551+0000 7f7d9affd640 1 -- 172.21.15.33:0/3313607440 learned_addr learned my addr 172.21.15.33:0/3313607440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9bfff640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c102250 0x7f7d9c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9b7fe640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d9c1006a0 0x7f7d9c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9affd640 1 -- 172.21.15.33:0/3313607440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c102250 msgr2=0x7f7d9c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9affd640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c102250 0x7f7d9c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9affd640 1 -- 172.21.15.33:0/3313607440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d9c1006a0 msgr2=0x7f7d9c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9affd640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d9c1006a0 0x7f7d9c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9affd640 1 -- 172.21.15.33:0/3313607440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c1a1810 con 0x7f7d9c101890 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9bfff640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c102250 0x7f7d9c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d9affd640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 0x7f7d9c19a180 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c920 tx=0x7f7d9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90007c70 con 0x7f7d9c101890 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d90007e10 con 0x7f7d9c101890 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c1a1b00 con 0x7f7d9c101890 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90018890 con 0x7f7d9c101890 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.552+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a20a0 con 0x7f7d9c101890 2024-04-24T23:39:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.553+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c1a22d0 con 0x7f7d9c101890 2024-04-24T23:39:26.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.554+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d90018a70 con 0x7f7d9c101890 2024-04-24T23:39:26.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.555+0000 7f7d98ff9640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d840a2c10 0x7f7d840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.555+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7d900cf590 con 0x7f7d9c101890 2024-04-24T23:39:26.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.555+0000 7f7d9b7fe640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d840a2c10 0x7f7d840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.555+0000 7f7d9b7fe640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d840a2c10 0x7f7d840a50d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c02aa00 tx=0x7f7d8c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:26.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.559+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d90014030 con 0x7f7d9c101890 2024-04-24T23:39:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.784+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d9c1a2540 con 0x7f7d9c101890 2024-04-24T23:39:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.784+0000 7f7d98ff9640 1 -- 172.21.15.33:0/3313607440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7f7d9009c100 con 0x7f7d9c101890 2024-04-24T23:39:26.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:39:26.785 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1264288095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1264288095},{"type":"v1","addr":"172.21.15.33:6835","nonce":1264288095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1577112159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1577112159},{"type":"v1","addr":"172.21.15.33:6837","nonce":1577112159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1277151249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1277151249},{"type":"v1","addr":"172.21.15.33:6841","nonce":1277151249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/2605742679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2605742679},{"type":"v1","addr":"172.21.15.38:6833","nonce":2605742679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1005158826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1005158826},{"type":"v1","addr":"172.21.15.38:6835","nonce":1005158826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2542827674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2542827674},{"type":"v1","addr":"172.21.15.117:6833","nonce":2542827674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:10:11.664486+0000","modified":"2024-04-24T23:38:17.286450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24508,"mds_1":24481,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/287276222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":287276222},{"type":"v1","addr":"172.21.15.38:6837","nonce":287276222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"g","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/2446242312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2446242312},{"type":"v1","addr":"172.21.15.33:6839","nonce":2446242312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/221661334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":221661334},{"type":"v1","addr":"172.21.15.117:6835","nonce":221661334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6839/920905379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":920905379},{"type":"v1","addr":"172.21.15.38:6839","nonce":920905379}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/878889242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":878889242},{"type":"v1","addr":"172.21.15.117:6837","nonce":878889242}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/851112851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":851112851},{"type":"v1","addr":"172.21.15.117:6839","nonce":851112851}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24479,24481]},"id":1}]} 2024-04-24T23:39:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 40 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d840a2c10 msgr2=0x7f7d840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d840a2c10 0x7f7d840a50d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c02aa00 tx=0x7f7d8c031400 comp rx=0 tx=0).stop 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 msgr2=0x7f7d9c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 0x7f7d9c19a180 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c920 tx=0x7f7d9000cde0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 shutdown_connections 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7d840a2c10 0x7f7d840a50d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9c102250 0x7f7d9c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d9c101890 0x7f7d9c19a180 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 --2- 172.21.15.33:0/3313607440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d9c1006a0 0x7f7d9c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 >> 172.21.15.33:0/3313607440 conn(0x7f7d9c0fbc30 msgr2=0x7f7d9c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 shutdown_connections 2024-04-24T23:39:26.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.786+0000 7f7da1fb7640 1 -- 172.21.15.33:0/3313607440 wait complete. 2024-04-24T23:39:26.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 -- 172.21.15.33:0/2034422837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8102200 msgr2=0x7f96f81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 --2- 172.21.15.33:0/2034422837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8102200 0x7f96f81045f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009900 tx=0x7f96e802a410 comp rx=0 tx=0).stop 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 -- 172.21.15.33:0/2034422837 shutdown_connections 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 --2- 172.21.15.33:0/2034422837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8107460 0x7f96f81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 --2- 172.21.15.33:0/2034422837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96f8104b30 0x7f96f8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 --2- 172.21.15.33:0/2034422837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8102200 0x7f96f81045f0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.899+0000 7f96fdff9640 1 -- 172.21.15.33:0/2034422837 >> 172.21.15.33:0/2034422837 conn(0x7f96f80fbbf0 msgr2=0x7f96f80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 -- 172.21.15.33:0/2034422837 shutdown_connections 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 -- 172.21.15.33:0/2034422837 wait complete. 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 Processor -- start 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 -- start start 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8102200 0x7f96f8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96f8104b30 0x7f96f8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 0x7f96f819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96f810d430 con 0x7f96f8107460 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f96f810d2b0 con 0x7f96f8102200 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.900+0000 7f96fdff9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96f810d5b0 con 0x7f96f8104b30 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 0x7f96f819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 0x7f96f819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50078/0 (socket says 172.21.15.33:50078) 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 -- 172.21.15.33:0/1404119422 learned_addr learned my addr 172.21.15.33:0/1404119422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f6ffd640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96f8104b30 0x7f96f8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f77fe640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8102200 0x7f96f8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 -- 172.21.15.33:0/1404119422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96f8104b30 msgr2=0x7f96f8197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96f8104b30 0x7f96f8197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 -- 172.21.15.33:0/1404119422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8102200 msgr2=0x7f96f8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8102200 0x7f96f8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 -- 172.21.15.33:0/1404119422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f819f6a0 con 0x7f96f8107460 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f77fe640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8102200 0x7f96f8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f7fff640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 0x7f96f819f030 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f96f000cce0 tx=0x7f96f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000f450 con 0x7f96f8107460 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f819f990 con 0x7f96f8107460 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f000fad0 con 0x7f96f8107460 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0018c10 con 0x7f96f8107460 2024-04-24T23:39:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.901+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f819fe90 con 0x7f96f8107460 2024-04-24T23:39:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.902+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f80fd6e0 con 0x7f96f8107460 2024-04-24T23:39:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.903+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f96f000f5f0 con 0x7f96f8107460 2024-04-24T23:39:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.904+0000 7f96f4ff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e00a2b90 0x7f96e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.904+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f96f00cfb60 con 0x7f96f8107460 2024-04-24T23:39:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.904+0000 7f96f77fe640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e00a2b90 0x7f96e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:26.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.904+0000 7f96f77fe640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e00a2b90 0x7f96e00a5050 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f96e8000c00 tx=0x7f96e80095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:26.908+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f96f0014030 con 0x7f96f8107460 2024-04-24T23:39:27.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.094+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f8062340 con 0x7f96f8107460 2024-04-24T23:39:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.095+0000 7f96f4ff9640 1 -- 172.21.15.33:0/1404119422 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24714 (secure 0 0 0) 0x7f96f009c730 con 0x7f96f8107460 2024-04-24T23:39:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-24T23:39:27.096 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"8917d4f4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:09.732561+0000","modified":"2024-04-24T23:12:07.840931+0000","last_up_change":"2024-04-24T23:09:21.300094+0000","last_in_change":"2024-04-24T23:09:01.926936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:05:32.235031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:10.244538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:11.097778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:10:14.786012+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d93333af-c06d-4be4-91c8-bd832217bd8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6803","nonce":2906488193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6805","nonce":2906488193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6809","nonce":2906488193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2906488193},{"type":"v1","addr":"172.21.15.33:6807","nonce":2906488193}]},"public_addr":"172.21.15.33:6803/2906488193","cluster_addr":"172.21.15.33:6805/2906488193","heartbeat_back_addr":"172.21.15.33:6809/2906488193","heartbeat_front_addr":"172.21.15.33:6807/2906488193","state":["exists","up"]},{"osd":1,"uuid":"0c497d09-e263-481e-a4dc-5352cb3ae2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6801","nonce":2106722793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6803","nonce":2106722793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6807","nonce":2106722793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2106722793},{"type":"v1","addr":"172.21.15.38:6805","nonce":2106722793}]},"public_addr":"172.21.15.38:6801/2106722793","cluster_addr":"172.21.15.38:6803/2106722793","heartbeat_back_addr":"172.21.15.38:6807/2106722793","heartbeat_front_addr":"172.21.15.38:6805/2106722793","state":["exists","up"]},{"osd":2,"uuid":"956cb2b5-6d5f-4817-b066-74c8227c7960","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6801","nonce":1483584233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6803","nonce":1483584233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6807","nonce":1483584233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1483584233},{"type":"v1","addr":"172.21.15.117:6805","nonce":1483584233}]},"public_addr":"172.21.15.117:6801/1483584233","cluster_addr":"172.21.15.117:6803/1483584233","heartbeat_back_addr":"172.21.15.117:6807/1483584233","heartbeat_front_addr":"172.21.15.117:6805/1483584233","state":["exists","up"]},{"osd":3,"uuid":"8ec098a5-9343-4caf-931d-f4c79d4fbdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6811","nonce":1011423676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6813","nonce":1011423676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6817","nonce":1011423676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1011423676},{"type":"v1","addr":"172.21.15.33:6815","nonce":1011423676}]},"public_addr":"172.21.15.33:6811/1011423676","cluster_addr":"172.21.15.33:6813/1011423676","heartbeat_back_addr":"172.21.15.33:6817/1011423676","heartbeat_front_addr":"172.21.15.33:6815/1011423676","state":["exists","up"]},{"osd":4,"uuid":"34a597a4-f838-4edf-9198-7d674b4aa9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6809","nonce":1943632582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6811","nonce":1943632582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6815","nonce":1943632582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1943632582},{"type":"v1","addr":"172.21.15.38:6813","nonce":1943632582}]},"public_addr":"172.21.15.38:6809/1943632582","cluster_addr":"172.21.15.38:6811/1943632582","heartbeat_back_addr":"172.21.15.38:6815/1943632582","heartbeat_front_addr":"172.21.15.38:6813/1943632582","state":["exists","up"]},{"osd":5,"uuid":"f4fef747-c797-4d33-b6bd-c1f7d55e052b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6809","nonce":2567468541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6811","nonce":2567468541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6815","nonce":2567468541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2567468541},{"type":"v1","addr":"172.21.15.117:6813","nonce":2567468541}]},"public_addr":"172.21.15.117:6809/2567468541","cluster_addr":"172.21.15.117:6811/2567468541","heartbeat_back_addr":"172.21.15.117:6815/2567468541","heartbeat_front_addr":"172.21.15.117:6813/2567468541","state":["exists","up"]},{"osd":6,"uuid":"aeb1c83d-c2cd-4d7b-95dc-dff2b00400db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6819","nonce":713285361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6821","nonce":713285361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6825","nonce":713285361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":713285361},{"type":"v1","addr":"172.21.15.33:6823","nonce":713285361}]},"public_addr":"172.21.15.33:6819/713285361","cluster_addr":"172.21.15.33:6821/713285361","heartbeat_back_addr":"172.21.15.33:6825/713285361","heartbeat_front_addr":"172.21.15.33:6823/713285361","state":["exists","up"]},{"osd":7,"uuid":"1e464d3b-4514-43db-9449-25f51b362bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6817","nonce":3279589891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6819","nonce":3279589891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6823","nonce":3279589891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":3279589891},{"type":"v1","addr":"172.21.15.38:6821","nonce":3279589891}]},"public_addr":"172.21.15.38:6817/3279589891","cluster_addr":"172.21.15.38:6819/3279589891","heartbeat_back_addr":"172.21.15.38:6823/3279589891","heartbeat_front_addr":"172.21.15.38:6821/3279589891","state":["exists","up"]},{"osd":8,"uuid":"69890093-ead1-4825-8a46-26a2b2c1b401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6817","nonce":1247270534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6819","nonce":1247270534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6823","nonce":1247270534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1247270534},{"type":"v1","addr":"172.21.15.117:6821","nonce":1247270534}]},"public_addr":"172.21.15.117:6817/1247270534","cluster_addr":"172.21.15.117:6819/1247270534","heartbeat_back_addr":"172.21.15.117:6823/1247270534","heartbeat_front_addr":"172.21.15.117:6821/1247270534","state":["exists","up"]},{"osd":9,"uuid":"8293c811-4410-4f09-9ea7-4de698fc76c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6827","nonce":498632081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6829","nonce":498632081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6833","nonce":498632081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":498632081},{"type":"v1","addr":"172.21.15.33:6831","nonce":498632081}]},"public_addr":"172.21.15.33:6827/498632081","cluster_addr":"172.21.15.33:6829/498632081","heartbeat_back_addr":"172.21.15.33:6833/498632081","heartbeat_front_addr":"172.21.15.33:6831/498632081","state":["exists","up"]},{"osd":10,"uuid":"57c8391d-387d-400c-80da-ac841c5d4608","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6825","nonce":1322822430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6827","nonce":1322822430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6831","nonce":1322822430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1322822430},{"type":"v1","addr":"172.21.15.38:6829","nonce":1322822430}]},"public_addr":"172.21.15.38:6825/1322822430","cluster_addr":"172.21.15.38:6827/1322822430","heartbeat_back_addr":"172.21.15.38:6831/1322822430","heartbeat_front_addr":"172.21.15.38:6829/1322822430","state":["exists","up"]},{"osd":11,"uuid":"c8842cc2-f8eb-4b78-9439-9aea72e0ae4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6825","nonce":2336105808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6827","nonce":2336105808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6831","nonce":2336105808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2336105808},{"type":"v1","addr":"172.21.15.117:6829","nonce":2336105808}]},"public_addr":"172.21.15.117:6825/2336105808","cluster_addr":"172.21.15.117:6827/2336105808","heartbeat_back_addr":"172.21.15.117:6831/2336105808","heartbeat_front_addr":"172.21.15.117:6829/2336105808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:04:40.272634+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:04.485524+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:27.827900+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:53.547913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:18.746957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:43.579209+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:09.035291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:35.187101+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:00.638019+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:27.576860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:54.278337+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:20.016053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/946624938":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6801/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:6800/2813273186":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/1214692934":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1172519445":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6800/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:6801/1299073362":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/1574498973":"2024-04-25T23:03:05.692528+0000","172.21.15.33:0/288675277":"2024-04-25T23:02:38.216643+0000","172.21.15.33:0/2398589700":"2024-04-25T23:03:05.692528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e00a2b90 msgr2=0x7f96e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e00a2b90 0x7f96e00a5050 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f96e8000c00 tx=0x7f96e80095f0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 msgr2=0x7f96f819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 0x7f96f819f030 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f96f000cce0 tx=0x7f96f000a620 comp rx=0 tx=0).stop 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 shutdown_connections 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f96e00a2b90 0x7f96e00a5050 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8107460 0x7f96f819f030 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96f8104b30 0x7f96f8197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 --2- 172.21.15.33:0/1404119422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96f8102200 0x7f96f8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.097+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 >> 172.21.15.33:0/1404119422 conn(0x7f96f80fbbf0 msgr2=0x7f96f8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.098+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 shutdown_connections 2024-04-24T23:39:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.098+0000 7f96fdff9640 1 -- 172.21.15.33:0/1404119422 wait complete. 2024-04-24T23:39:27.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:26 smithi033 ceph-mon[29201]: pgmap v1174: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T23:39:27.108 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3915782201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:39:27.108 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:26 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3313607440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:39:27.109 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 -- 172.21.15.33:0/118235199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 msgr2=0x7fdfdc100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 --2- 172.21.15.33:0/118235199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc100ac0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009900 tx=0x7fdfcc02a410 comp rx=0 tx=0).stop 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 -- 172.21.15.33:0/118235199 shutdown_connections 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 --2- 172.21.15.33:0/118235199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc102250 0x7fdfdc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 --2- 172.21.15.33:0/118235199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfdc101890 0x7fdfdc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 --2- 172.21.15.33:0/118235199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc100ac0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 -- 172.21.15.33:0/118235199 >> 172.21.15.33:0/118235199 conn(0x7fdfdc0fbc30 msgr2=0x7fdfdc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 -- 172.21.15.33:0/118235199 shutdown_connections 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.212+0000 7fdfe1076640 1 -- 172.21.15.33:0/118235199 wait complete. 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 Processor -- start 2024-04-24T23:39:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 -- start start 2024-04-24T23:39:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc101890 0x7fdfdc102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfdc102250 0x7fdfdc1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10c320 con 0x7fdfdc1006a0 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10c1a0 con 0x7fdfdc101890 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfe1076640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10c4a0 con 0x7fdfdc102250 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfdad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfdad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50090/0 (socket says 172.21.15.33:50090) 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfdad9d640 1 -- 172.21.15.33:0/4036374940 learned_addr learned my addr 172.21.15.33:0/4036374940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfdb59e640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfdc102250 0x7fdfdc1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.213+0000 7fdfda59c640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc101890 0x7fdfdc102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfdad9d640 1 -- 172.21.15.33:0/4036374940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfdc102250 msgr2=0x7fdfdc1031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfdad9d640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfdc102250 0x7fdfdc1031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfdad9d640 1 -- 172.21.15.33:0/4036374940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc101890 msgr2=0x7fdfdc102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfdad9d640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc101890 0x7fdfdc102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfdad9d640 1 -- 172.21.15.33:0/4036374940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc103a40 con 0x7fdfdc1006a0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfda59c640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc101890 0x7fdfdc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfdad9d640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc104630 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc031610 tx=0x7fdfcc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc03f030 con 0x7fdfdc1006a0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc1a1be0 con 0x7fdfdc1006a0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfcc003ce0 con 0x7fdfdc1006a0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc035070 con 0x7fdfdc1006a0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.214+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1a2130 con 0x7fdfdc1006a0 2024-04-24T23:39:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.215+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfdc1a2300 con 0x7fdfdc1006a0 2024-04-24T23:39:27.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.216+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdfcc042030 con 0x7fdfdc1006a0 2024-04-24T23:39:27.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.217+0000 7fdfc3fff640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdfc40a2b90 0x7fdfc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.217+0000 7fdfda59c640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdfc40a2b90 0x7fdfc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.217+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfcc0ef9d0 con 0x7fdfdc1006a0 2024-04-24T23:39:27.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.218+0000 7fdfda59c640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdfc40a2b90 0x7fdfc40a5050 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00058c0 tx=0x7fdfd0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:27.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.222+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfcc0bc540 con 0x7fdfdc1006a0 2024-04-24T23:39:27.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:26 smithi038 ceph-mon[32450]: pgmap v1174: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T23:39:27.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3915782201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:39:27.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:26 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3313607440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:39:27.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:26 smithi117 ceph-mon[32529]: pgmap v1174: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T23:39:27.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3915782201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:39:27.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:26 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3313607440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:39:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.445+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdfdc1a24d0 con 0x7fdfdc1006a0 2024-04-24T23:39:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.447+0000 7fdfc3fff640 1 -- 172.21.15.33:0/4036374940 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7fdfcc0bbe40 con 0x7fdfdc1006a0 2024-04-24T23:39:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdfc40a2b90 msgr2=0x7fdfc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdfc40a2b90 0x7fdfc40a5050 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00058c0 tx=0x7fdfd0005850 comp rx=0 tx=0).stop 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 msgr2=0x7fdfdc104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc104630 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc031610 tx=0x7fdfcc0043b0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 shutdown_connections 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fdfc40a2b90 0x7fdfc40a5050 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfdc102250 0x7fdfdc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfdc101890 0x7fdfdc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 --2- 172.21.15.33:0/4036374940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfdc1006a0 0x7fdfdc104630 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.451+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 >> 172.21.15.33:0/4036374940 conn(0x7fdfdc0fbc30 msgr2=0x7fdfdc0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.452+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 shutdown_connections 2024-04-24T23:39:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.452+0000 7fdfe1076640 1 -- 172.21.15.33:0/4036374940 wait complete. 2024-04-24T23:39:27.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:39:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/1492043934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b28102170 msgr2=0x7f9b28104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/1492043934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b28102170 0x7f9b28104560 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009900 tx=0x7f9b1802a410 comp rx=0 tx=0).stop 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/1492043934 shutdown_connections 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/1492043934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b281073d0 0x7f9b281097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/1492043934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28104aa0 0x7f9b28106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/1492043934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b28102170 0x7f9b28104560 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/1492043934 >> 172.21.15.33:0/1492043934 conn(0x7f9b280fbae0 msgr2=0x7f9b280fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/1492043934 shutdown_connections 2024-04-24T23:39:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.572+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/1492043934 wait complete. 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 Processor -- start 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 -- start start 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 0x7f9b28197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b28104aa0 0x7f9b28197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 0x7f9b2819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2810d2e0 con 0x7f9b281073d0 2024-04-24T23:39:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2810d160 con 0x7f9b28104aa0 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b2cd9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2810d460 con 0x7f9b28102170 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 0x7f9b2819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 0x7f9b28197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 0x7f9b2819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50136/0 (socket says 172.21.15.33:50136) 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 0x7f9b28197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46570/0 (socket says 172.21.15.33:46570) 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b27fff640 1 -- 172.21.15.33:0/2078773306 learned_addr learned my addr 172.21.15.33:0/2078773306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b26ffd640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b28104aa0 0x7f9b28197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.573+0000 7f9b27fff640 1 -- 172.21.15.33:0/2078773306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 msgr2=0x7f9b28197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b27fff640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 0x7f9b28197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b27fff640 1 -- 172.21.15.33:0/2078773306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b28104aa0 msgr2=0x7f9b28197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b27fff640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b28104aa0 0x7f9b28197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b27fff640 1 -- 172.21.15.33:0/2078773306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2819f540 con 0x7f9b281073d0 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b277fe640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 0x7f9b28197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b26ffd640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b28104aa0 0x7f9b28197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b27fff640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 0x7f9b2819eed0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000cce0 tx=0x7f9b2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000f450 con 0x7f9b281073d0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2000fad0 con 0x7f9b281073d0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20018c10 con 0x7f9b281073d0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2819f830 con 0x7f9b281073d0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.574+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2819fd30 con 0x7f9b281073d0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.575+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b280fd5d0 con 0x7f9b281073d0 2024-04-24T23:39:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.576+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9b2000f5f0 con 0x7f9b281073d0 2024-04-24T23:39:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.577+0000 7f9b24ff9640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b100a2b90 0x7f9b100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.577+0000 7f9b277fe640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b100a2b90 0x7f9b100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.577+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b200cfc00 con 0x7f9b281073d0 2024-04-24T23:39:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.577+0000 7f9b277fe640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b100a2b90 0x7f9b100a5050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9b180314e0 tx=0x7f9b18000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.581+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9b20014030 con 0x7f9b281073d0 2024-04-24T23:39:27.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a[29197]: 2024-04-24T23:39:27.446+0000 7f55ddfb0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:39:27.704 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g[70876]: ignoring --setuser ceph since I am not root 2024-04-24T23:39:27.704 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g[70876]: ignoring --setgroup ceph since I am not root 2024-04-24T23:39:27.704 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mds[70880]: -- 172.21.15.33:0/3743358511 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d644005a40 con 0x55d643fd8800 2024-04-24T23:39:27.704 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g[70876]: starting mds.g at 2024-04-24T23:39:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:27.805+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9b28062340 con 0x7f9b281073d0 2024-04-24T23:39:27.919 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h[62534]: ignoring --setuser ceph since I am not root 2024-04-24T23:39:27.919 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h[62534]: ignoring --setgroup ceph since I am not root 2024-04-24T23:39:27.919 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mds[62538]: -- 172.21.15.38:0/2861350416 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fb0483a40 con 0x557fb0456c00 2024-04-24T23:39:27.919 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h[62534]: starting mds.h at 2024-04-24T23:39:27.919 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b[62848]: ignoring --setuser ceph since I am not root 2024-04-24T23:39:27.920 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b[62848]: ignoring --setgroup ceph since I am not root 2024-04-24T23:39:27.920 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mds[62852]: -- 172.21.15.38:0/2990514769 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca25afba40 con 0x55ca25acec00 2024-04-24T23:39:27.920 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b[62848]: starting mds.b at 2024-04-24T23:39:27.924 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: ignoring --setuser ceph since I am not root 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: ignoring --setgroup ceph since I am not root 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mds[62778]: -- 172.21.15.117:0/1155061144 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631bef2ba40 con 0x5631beefec00 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: starting mds.f at 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: 2024-04-24T23:39:27.507+0000 7f90e124c700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: ignoring --setuser ceph since I am not root 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: ignoring --setgroup ceph since I am not root 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mds[63398]: -- 172.21.15.117:0/4042531575 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555be429ba40 con 0x555be426ec00 2024-04-24T23:39:27.925 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: starting mds.i at 2024-04-24T23:39:27.926 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c[63084]: ignoring --setuser ceph since I am not root 2024-04-24T23:39:27.926 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c[63084]: ignoring --setgroup ceph since I am not root 2024-04-24T23:39:27.926 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mds[63088]: -- 172.21.15.117:0/1922452699 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9dd1fda40 con 0x55d9dd1d1000 2024-04-24T23:39:27.926 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c[63084]: starting mds.c at 2024-04-24T23:39:28.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/1404119422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:39:28.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4036374940' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:39:28.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:39:28.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:39:28.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/4036374940' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:39:28.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:39:28.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:39:28.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:27 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2078773306' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:39:28.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/1404119422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:39:28.424 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4036374940' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:39:28.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:39:28.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:39:28.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/4036374940' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:39:28.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:39:28.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:39:28.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:27 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2078773306' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:39:28.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.449+0000 7f9b24ff9640 1 -- 172.21.15.33:0/2078773306 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f9b2009c690 con 0x7f9b281073d0 2024-04-24T23:39:28.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b100a2b90 msgr2=0x7f9b100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:28.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b100a2b90 0x7f9b100a5050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9b180314e0 tx=0x7f9b18000bc0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 msgr2=0x7f9b2819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 0x7f9b2819eed0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000cce0 tx=0x7f9b2000a620 comp rx=0 tx=0).stop 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 shutdown_connections 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f9b100a2b90 0x7f9b100a5050 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b281073d0 0x7f9b2819eed0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b28104aa0 0x7f9b28197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 --2- 172.21.15.33:0/2078773306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b28102170 0x7f9b28197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 >> 172.21.15.33:0/2078773306 conn(0x7f9b280fbae0 msgr2=0x7f9b28102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 shutdown_connections 2024-04-24T23:39:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.453+0000 7f9b2cd9e640 1 -- 172.21.15.33:0/2078773306 wait complete. 2024-04-24T23:39:28.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/1404119422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:39:28.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4036374940' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:39:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:39:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:39:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/4036374940' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:39:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:39:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:39:28.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:27 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2078773306' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:39:28.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 -- 172.21.15.33:0/157179595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4102250 msgr2=0x7f29d4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 --2- 172.21.15.33:0/157179595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4102250 0x7f29d4108ae0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009ca0 tx=0x7f29cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 -- 172.21.15.33:0/157179595 shutdown_connections 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 --2- 172.21.15.33:0/157179595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4102250 0x7f29d4108ae0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 --2- 172.21.15.33:0/157179595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4101890 0x7f29d4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 --2- 172.21.15.33:0/157179595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29d41006a0 0x7f29d4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 -- 172.21.15.33:0/157179595 >> 172.21.15.33:0/157179595 conn(0x7f29d40fbc30 msgr2=0x7f29d40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 -- 172.21.15.33:0/157179595 shutdown_connections 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.582+0000 7f29daca5640 1 -- 172.21.15.33:0/157179595 wait complete. 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 Processor -- start 2024-04-24T23:39:28.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 -- start start 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29d41006a0 0x7f29d4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 0x7f29d4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4102250 0x7f29d419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29d410c370 con 0x7f29d4101890 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29d410c1f0 con 0x7f29d41006a0 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29daca5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29d410c4f0 con 0x7f29d4102250 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 0x7f29d4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 0x7f29d4197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50148/0 (socket says 172.21.15.33:50148) 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29d3fff640 1 -- 172.21.15.33:0/3746141835 learned_addr learned my addr 172.21.15.33:0/3746141835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.583+0000 7f29d9242640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4102250 0x7f29d419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d8a41640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29d41006a0 0x7f29d4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d3fff640 1 -- 172.21.15.33:0/3746141835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4102250 msgr2=0x7f29d419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:28.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d3fff640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4102250 0x7f29d419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d3fff640 1 -- 172.21.15.33:0/3746141835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29d41006a0 msgr2=0x7f29d4197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d3fff640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29d41006a0 0x7f29d4197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d3fff640 1 -- 172.21.15.33:0/3746141835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d419f670 con 0x7f29d4101890 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d9242640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4102250 0x7f29d419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d3fff640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 0x7f29d4197f90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f29c000d770 tx=0x7f29c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0017070 con 0x7f29d4101890 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c0005500 con 0x7f29d4101890 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d419f960 con 0x7f29d4101890 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0004e00 con 0x7f29d4101890 2024-04-24T23:39:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.584+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d419feb0 con 0x7f29d4101890 2024-04-24T23:39:28.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.585+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d41a00e0 con 0x7f29d4101890 2024-04-24T23:39:28.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.586+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f29c0004fa0 con 0x7f29d4101890 2024-04-24T23:39:28.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.587+0000 7f29d1ffb640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c40a2c10 0x7f29c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:28.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.587+0000 7f29d8a41640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c40a2c10 0x7f29c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:28.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.587+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f29c00cef70 con 0x7f29d4101890 2024-04-24T23:39:28.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.587+0000 7f29d8a41640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c40a2c10 0x7f29c40a50d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005fd0 tx=0x7f29c8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:28.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.591+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f29c009ba00 con 0x7f29d4101890 2024-04-24T23:39:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:28.779+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f29d41a0780 con 0x7f29d4101890 2024-04-24T23:39:29.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: pgmap v1176: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-24T23:39:29.419 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: Cluster is now healthy 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2078773306' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6834/1407497093,v1:172.21.15.117:6835/1407497093] up:boot 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.33:6838/3759883651,v1:172.21.15.33:6839/3759883651] up:boot 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6836/3292100675,v1:172.21.15.117:6838/3292100675] up:boot 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.117:6837/3219022802,v1:172.21.15.117:6839/3219022802] up:boot 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6837/3185495033,v1:172.21.15.38:6839/3185495033] up:boot 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: mds.? [v2:172.21.15.38:6836/2727332390,v1:172.21.15.38:6838/2727332390] up:boot 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: fsmap 12 up:standby 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:39:29.420 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:39:29.421 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:28 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3746141835' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: pgmap v1176: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: Cluster is now healthy 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2078773306' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6834/1407497093,v1:172.21.15.117:6835/1407497093] up:boot 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.33:6838/3759883651,v1:172.21.15.33:6839/3759883651] up:boot 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6836/3292100675,v1:172.21.15.117:6838/3292100675] up:boot 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.117:6837/3219022802,v1:172.21.15.117:6839/3219022802] up:boot 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6837/3185495033,v1:172.21.15.38:6839/3185495033] up:boot 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: mds.? [v2:172.21.15.38:6836/2727332390,v1:172.21.15.38:6838/2727332390] up:boot 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: fsmap 12 up:standby 2024-04-24T23:39:29.425 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:39:29.426 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:28 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3746141835' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:29.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.451+0000 7f29d1ffb640 1 -- 172.21.15.33:0/3746141835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f29c009b300 con 0x7f29d4101890 2024-04-24T23:39:29.451 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: pgmap v1176: 129 pgs: 129 active+clean; 410 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: Cluster is now healthy 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2078773306' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6834/1407497093,v1:172.21.15.117:6835/1407497093] up:boot 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.33:6838/3759883651,v1:172.21.15.33:6839/3759883651] up:boot 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6836/3292100675,v1:172.21.15.117:6838/3292100675] up:boot 2024-04-24T23:39:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.117:6837/3219022802,v1:172.21.15.117:6839/3219022802] up:boot 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6837/3185495033,v1:172.21.15.38:6839/3185495033] up:boot 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: mds.? [v2:172.21.15.38:6836/2727332390,v1:172.21.15.38:6838/2727332390] up:boot 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: fsmap 12 up:standby 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:39:29.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:28 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3746141835' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.455+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c40a2c10 msgr2=0x7f29c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.455+0000 7f29daca5640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c40a2c10 0x7f29c40a50d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005fd0 tx=0x7f29c8005850 comp rx=0 tx=0).stop 2024-04-24T23:39:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.455+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 msgr2=0x7f29d4197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.455+0000 7f29daca5640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 0x7f29d4197f90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f29c000d770 tx=0x7f29c000dc30 comp rx=0 tx=0).stop 2024-04-24T23:39:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 shutdown_connections 2024-04-24T23:39:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f29c40a2c10 0x7f29c40a50d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d4102250 0x7f29d419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d4101890 0x7f29d4197f90 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 --2- 172.21.15.33:0/3746141835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29d41006a0 0x7f29d4197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 >> 172.21.15.33:0/3746141835 conn(0x7f29d40fbc30 msgr2=0x7f29d40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 shutdown_connections 2024-04-24T23:39:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.456+0000 7f29daca5640 1 -- 172.21.15.33:0/3746141835 wait complete. 2024-04-24T23:39:29.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:39:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 -- 172.21.15.33:0/2150102207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c0102990 msgr2=0x7fb6c0102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:29.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/2150102207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c0102990 0x7fb6c0102e10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009900 tx=0x7fb6b402a410 comp rx=0 tx=0).stop 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 -- 172.21.15.33:0/2150102207 shutdown_connections 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/2150102207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c0103350 0x7fb6c0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/2150102207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c0102990 0x7fb6c0102e10 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/2150102207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c01017a0 0x7fb6c0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 -- 172.21.15.33:0/2150102207 >> 172.21.15.33:0/2150102207 conn(0x7fb6c00fcd10 msgr2=0x7fb6c00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 -- 172.21.15.33:0/2150102207 shutdown_connections 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.591+0000 7fb6c6b42640 1 -- 172.21.15.33:0/2150102207 wait complete. 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 Processor -- start 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 -- start start 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 0x7fb6c00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c0102990 0x7fb6c0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c0103350 0x7fb6c0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c010d420 con 0x7fb6c01017a0 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c010d2a0 con 0x7fb6c0102990 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c6b42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c010d5a0 con 0x7fb6c0103350 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 0x7fb6c00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 0x7fb6c00754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50172/0 (socket says 172.21.15.33:50172) 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 -- 172.21.15.33:0/631387082 learned_addr learned my addr 172.21.15.33:0/631387082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6bffff640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c0102990 0x7fb6c0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c50df640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c0103350 0x7fb6c0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 -- 172.21.15.33:0/631387082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c0103350 msgr2=0x7fb6c0074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c0103350 0x7fb6c0074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 -- 172.21.15.33:0/631387082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c0102990 msgr2=0x7fb6c0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c0102990 0x7fb6c0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6c48de640 1 -- 172.21.15.33:0/631387082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c0074900 con 0x7fb6c01017a0 2024-04-24T23:39:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.592+0000 7fb6bffff640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c0102990 0x7fb6c0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:39:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.593+0000 7fb6c48de640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 0x7fb6c00754f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e9c0 tx=0x7fb6b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.593+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000cd80 con 0x7fb6c01017a0 2024-04-24T23:39:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.593+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c01a1bc0 con 0x7fb6c01017a0 2024-04-24T23:39:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.593+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b00035f0 con 0x7fb6c01017a0 2024-04-24T23:39:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.593+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0010690 con 0x7fb6c01017a0 2024-04-24T23:39:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.593+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c01a20e0 con 0x7fb6c01017a0 2024-04-24T23:39:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.594+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c00689b0 con 0x7fb6c01017a0 2024-04-24T23:39:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.595+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb6b00037a0 con 0x7fb6c01017a0 2024-04-24T23:39:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.596+0000 7fb6bdffb640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6a80a2b90 0x7fb6a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.596+0000 7fb6bffff640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6a80a2b90 0x7fb6a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.596+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8854+0+0 (secure 0 0 0) 0x7fb6b00cf720 con 0x7fb6c01017a0 2024-04-24T23:39:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.596+0000 7fb6bffff640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6a80a2b90 0x7fb6a80a5050 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40314e0 tx=0x7fb6b4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:29.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.600+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb6b009c3b0 con 0x7fb6c01017a0 2024-04-24T23:39:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:29.784+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6c0108ae0 con 0x7fb6c01017a0 2024-04-24T23:39:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.457+0000 7fb6bdffb640 1 -- 172.21.15.33:0/631387082 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7fb6c0108ae0 con 0x7fb6c01017a0 2024-04-24T23:39:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-04-24T23:39:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.460+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6a80a2b90 msgr2=0x7fb6a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.460+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6a80a2b90 0x7fb6a80a5050 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40314e0 tx=0x7fb6b4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.460+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 msgr2=0x7fb6c00754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:30.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.460+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 0x7fb6c00754f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e9c0 tx=0x7fb6b000ee80 comp rx=0 tx=0).stop 2024-04-24T23:39:30.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.461+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 shutdown_connections 2024-04-24T23:39:30.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.461+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7fb6a80a2b90 0x7fb6a80a5050 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.461+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c0103350 0x7fb6c0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.461+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c0102990 0x7fb6c0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.461+0000 7fb6c6b42640 1 --2- 172.21.15.33:0/631387082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c01017a0 0x7fb6c00754f0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.461+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 >> 172.21.15.33:0/631387082 conn(0x7fb6c00fcd10 msgr2=0x7fb6c0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:30.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.462+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 shutdown_connections 2024-04-24T23:39:30.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.462+0000 7fb6c6b42640 1 -- 172.21.15.33:0/631387082 wait complete. 2024-04-24T23:39:30.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-24T23:39:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 -- 172.21.15.33:0/1988117198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 msgr2=0x7f7d08108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 --2- 172.21.15.33:0/1988117198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 0x7f7d08108ad0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009ca0 tx=0x7f7d0002a8b0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 -- 172.21.15.33:0/1988117198 shutdown_connections 2024-04-24T23:39:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 --2- 172.21.15.33:0/1988117198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 0x7f7d08108ad0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 --2- 172.21.15.33:0/1988117198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08101880 0x7f7d08101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 --2- 172.21.15.33:0/1988117198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d08100690 0x7f7d08100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 -- 172.21.15.33:0/1988117198 >> 172.21.15.33:0/1988117198 conn(0x7f7d080fbc00 msgr2=0x7f7d080fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.641+0000 7f7d0f970640 1 -- 172.21.15.33:0/1988117198 shutdown_connections 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 -- 172.21.15.33:0/1988117198 wait complete. 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 Processor -- start 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 -- start start 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 0x7f7d08199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d08101880 0x7f7d0819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 0x7f7d081a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810f630 con 0x7f7d08102240 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810f4b0 con 0x7f7d08101880 2024-04-24T23:39:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.642+0000 7f7d0f970640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810f7b0 con 0x7f7d08100690 2024-04-24T23:39:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 0x7f7d08199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 0x7f7d08199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.33:46618/0 (socket says 172.21.15.33:46618) 2024-04-24T23:39:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 -- 172.21.15.33:0/2662916696 learned_addr learned my addr 172.21.15.33:0/2662916696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0df0d640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 0x7f7d081a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 -- 172.21.15.33:0/2662916696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d08101880 msgr2=0x7f7d0819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d08101880 0x7f7d0819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 -- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 msgr2=0x7f7d081a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 0x7f7d081a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 -- 172.21.15.33:0/2662916696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081a1840 con 0x7f7d08100690 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.643+0000 7f7d0d70c640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 0x7f7d08199c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009960 tx=0x7f7d04009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.644+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0401b070 con 0x7f7d08100690 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.644+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d04016410 con 0x7f7d08100690 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.644+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d081a1b30 con 0x7f7d08100690 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.644+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04015610 con 0x7f7d08100690 2024-04-24T23:39:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.644+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a20d0 con 0x7f7d08100690 2024-04-24T23:39:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.645+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d08107950 con 0x7f7d08100690 2024-04-24T23:39:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.646+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d0401e070 con 0x7f7d08100690 2024-04-24T23:39:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.647+0000 7f7cfe7fc640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7cf40a2c50 0x7f7cf40a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:39:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.647+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8324+0+0 (secure 0 0 0) 0x7f7d040ce670 con 0x7f7d08100690 2024-04-24T23:39:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.647+0000 7f7d0cf0b640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7cf40a2c50 0x7f7cf40a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:39:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.648+0000 7f7d0cf0b640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7cf40a2c50 0x7f7cf40a5110 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005fd0 tx=0x7f7cf8005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:39:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.655+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d0409b570 con 0x7f7d08100690 2024-04-24T23:39:30.852 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:30 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/3746141835' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:39:30.852 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:30 smithi117 ceph-mon[32529]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:39:30.852 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:30 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/631387082' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:30.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:30.861+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d08062340 con 0x7f7d08100690 2024-04-24T23:39:30.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:30 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/3746141835' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:39:30.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:30 smithi038 ceph-mon[32450]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:39:30.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:30 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/631387082' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:30.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:30 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/3746141835' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:39:30.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:30 smithi033 ceph-mon[29201]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:39:30.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:30 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/631387082' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.462+0000 7f7cfe7fc640 1 -- 172.21.15.33:0/2662916696 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7f7d0409ae70 con 0x7f7d08100690 2024-04-24T23:39:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' removed 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.465+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7cf40a2c50 msgr2=0x7f7cf40a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.465+0000 7f7d0f970640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7cf40a2c50 0x7f7cf40a5110 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005fd0 tx=0x7f7cf8005c70 comp rx=0 tx=0).stop 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.465+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 msgr2=0x7f7d08199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.465+0000 7f7d0f970640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 0x7f7d08199c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009960 tx=0x7f7d04009e20 comp rx=0 tx=0).stop 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.468+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 shutdown_connections 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.468+0000 7f7d0f970640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:6800/3494091302,v1:172.21.15.33:6801/3494091302] conn(0x7f7cf40a2c50 0x7f7cf40a5110 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.468+0000 7f7d0f970640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08102240 0x7f7d081a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.468+0000 7f7d0f970640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d08101880 0x7f7d0819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.468+0000 7f7d0f970640 1 --2- 172.21.15.33:0/2662916696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08100690 0x7f7d08199c30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.469+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 >> 172.21.15.33:0/2662916696 conn(0x7f7d080fbc00 msgr2=0x7f7d08104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.469+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 shutdown_connections 2024-04-24T23:39:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-24T23:39:31.469+0000 7f7d0f970640 1 -- 172.21.15.33:0/2662916696 wait complete. 2024-04-24T23:39:31.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:39:31.518 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:39:31.571 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:39:31.597 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T23:39:31.598 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:39:31.613 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:39:31.629 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:39:31.652 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T23:39:31.652 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T23:39:31.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:39:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:31 smithi038 ceph-mon[32450]: pgmap v1178: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:39:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:31 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/631387082' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:39:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:31 smithi038 ceph-mon[32450]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:39:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:31 smithi038 ceph-mon[32450]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:39:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:31 smithi038 ceph-mon[32450]: from='client.? 172.21.15.33:0/2662916696' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.919 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:31 smithi038 ceph-mon[32450]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:31 smithi117 ceph-mon[32529]: pgmap v1178: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:39:31.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:31 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/631387082' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:39:31.924 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:31 smithi117 ceph-mon[32529]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:39:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:31 smithi117 ceph-mon[32529]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:39:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:31 smithi117 ceph-mon[32529]: from='client.? 172.21.15.33:0/2662916696' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:31 smithi117 ceph-mon[32529]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 ceph-mon[29201]: pgmap v1178: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:39:31.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/631387082' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:39:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 ceph-mon[29201]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:39:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 ceph-mon[29201]: from='mgr.14148 172.21.15.33:0/3262714391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:39:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 ceph-mon[29201]: from='client.? 172.21.15.33:0/2662916696' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:39:31.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:31 smithi033 systemd[1]: Stopping Ceph mon.a for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:32.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a[29197]: 2024-04-24T23:39:32.020+0000 7f55e37bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:32.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a[29197]: 2024-04-24T23:39:32.020+0000 7f55e37bb700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T23:39:32.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:32 smithi033 podman[112218]: 2024-04-24 23:39:32.043243848 +0000 UTC m=+0.161853715 container died f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T23:39:32.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:32 smithi033 podman[112218]: 2024-04-24 23:39:32.388052694 +0000 UTC m=+0.506662560 container cleanup f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T23:39:32.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:32 smithi033 bash[112218]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:39:33.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 24 23:39:32 smithi033 podman[112232]: 2024-04-24 23:39:32.848328588 +0000 UTC m=+0.796869139 container remove f59ab6d57b368162dfa21f676c590f05f69611f0751519654b07902fd4807381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:39:34.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T23:39:34.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:34.377 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T23:39:34.377 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T23:39:34.377 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.b 2024-04-24T23:39:34.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:34 smithi038 systemd[1]: Stopping Ceph mon.b for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:35.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b[32445]: 2024-04-24T23:39:34.851+0000 7f37ce574700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:35.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b[32445]: 2024-04-24T23:39:34.851+0000 7f37ce574700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:39:35.169 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:34 smithi038 podman[71265]: 2024-04-24 23:39:34.883140892 +0000 UTC m=+0.214189503 container died dfde4ac5135b9cc8f7f56ce4aafc4624cc7a1b9ce213fa45eb809f9672f79cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:39:35.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:35 smithi038 podman[71265]: 2024-04-24 23:39:35.370329358 +0000 UTC m=+0.701377970 container cleanup dfde4ac5135b9cc8f7f56ce4aafc4624cc7a1b9ce213fa45eb809f9672f79cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T23:39:35.669 INFO:journalctl@ceph.mon.b.smithi038.stdout:Apr 24 23:39:35 smithi038 bash[71265]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-b 2024-04-24T23:39:36.603 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T23:39:36.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:36.657 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T23:39:36.657 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T23:39:36.657 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.c 2024-04-24T23:39:36.996 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:36 smithi117 systemd[1]: Stopping Ceph mon.c for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:37.363 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c[32512]: 2024-04-24T23:39:36.995+0000 7f4acce52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:37.364 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c[32512]: 2024-04-24T23:39:36.995+0000 7f4acce52700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:39:37.364 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:37 smithi117 podman[73649]: 2024-04-24 23:39:37.05740425 +0000 UTC m=+0.173494390 container died 266d12945107cf0b688b0f80da17a8bed1535bb40976ec93c3b99951ebdda814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:39:37.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:37 smithi117 podman[73649]: 2024-04-24 23:39:37.480342692 +0000 UTC m=+0.596432831 container cleanup 266d12945107cf0b688b0f80da17a8bed1535bb40976ec93c3b99951ebdda814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:39:37.674 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 24 23:39:37 smithi117 bash[73649]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mon-c 2024-04-24T23:39:37.909 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T23:39:37.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:37.950 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T23:39:37.950 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T23:39:37.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:39:38.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:39:37 smithi033 systemd[1]: Stopping Ceph mgr.x for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:38.555 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:39:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x[29436]: 2024-04-24T23:39:38.270+0000 7f7ba66c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:38.555 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:39:38 smithi033 podman[114717]: 2024-04-24 23:39:38.301599359 +0000 UTC m=+0.149475520 container died a72910c86a2e39aa381be45306e68979d371d2abea5a82739611ef3e7cd512e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T23:39:38.954 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:39:38 smithi033 podman[114717]: 2024-04-24 23:39:38.654732126 +0000 UTC m=+0.502608271 container cleanup a72910c86a2e39aa381be45306e68979d371d2abea5a82739611ef3e7cd512e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T23:39:38.954 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 24 23:39:38 smithi033 bash[114717]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-x 2024-04-24T23:39:39.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T23:39:39.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:39.191 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T23:39:39.192 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T23:39:39.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.y 2024-04-24T23:39:39.543 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:39 smithi038 systemd[1]: Stopping Ceph mgr.y for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:39.850 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:39 smithi038 podman[73813]: 2024-04-24 23:39:39.543949121 +0000 UTC m=+0.145939233 container died aec5763f7f1498cd97870cd0a7cd24058f698b7127a2f44b7e87b287cad68416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T23:39:40.169 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:39 smithi038 podman[73813]: 2024-04-24 23:39:39.957522856 +0000 UTC m=+0.559512968 container cleanup aec5763f7f1498cd97870cd0a7cd24058f698b7127a2f44b7e87b287cad68416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T23:39:40.169 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:39 smithi038 bash[73813]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y 2024-04-24T23:39:40.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T23:39:40.460 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:40 smithi038 podman[73826]: 2024-04-24 23:39:40.193233661 +0000 UTC m=+0.645561382 container remove aec5763f7f1498cd97870cd0a7cd24058f698b7127a2f44b7e87b287cad68416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T23:39:40.460 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:40 smithi038 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-24T23:39:40.461 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:40 smithi038 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-24T23:39:40.461 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:40 smithi038 systemd[1]: Stopped Ceph mgr.y for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:39:40.461 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Apr 24 23:39:40 smithi038 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.y.service: Consumed 17.186s CPU time. 2024-04-24T23:39:40.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:40.473 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T23:39:40.473 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T23:39:40.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.z 2024-04-24T23:39:40.813 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:39:40 smithi117 systemd[1]: Stopping Ceph mgr.z for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:41.078 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:39:40 smithi117 podman[73771]: 2024-04-24 23:39:40.814017493 +0000 UTC m=+0.149475728 container died 56296d78a223bc382881535ea8a6c4fd9bd33125f7038564fa678f9ae18ccb8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:39:41.424 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:39:41 smithi117 podman[73771]: 2024-04-24 23:39:41.177393647 +0000 UTC m=+0.512851879 container cleanup 56296d78a223bc382881535ea8a6c4fd9bd33125f7038564fa678f9ae18ccb8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:39:41.424 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 24 23:39:41 smithi117 bash[73771]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mgr-z 2024-04-24T23:39:41.608 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T23:39:41.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:41.640 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T23:39:41.640 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T23:39:41.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.0 2024-04-24T23:39:41.954 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:41 smithi033 systemd[1]: Stopping Ceph osd.0 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:42.453 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0[44302]: 2024-04-24T23:39:41.968+0000 7f95346b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:42.453 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0[44302]: 2024-04-24T23:39:41.968+0000 7f95346b5700 -1 osd.0 98 *** Got signal Terminated *** 2024-04-24T23:39:42.454 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0[44302]: 2024-04-24T23:39:41.968+0000 7f95346b5700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:39:47.749 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:47 smithi033 podman[114850]: 2024-04-24 23:39:47.468532042 +0000 UTC m=+5.620237934 container died 745e9e8a1f57a241b60298477f3ce5051d63be56cbf27a5784cdbf484e12ce79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:39:48.178 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:47 smithi033 podman[114850]: 2024-04-24 23:39:47.852668647 +0000 UTC m=+6.004374585 container cleanup 745e9e8a1f57a241b60298477f3ce5051d63be56cbf27a5784cdbf484e12ce79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:39:48.178 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:47 smithi033 bash[114850]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:39:48.453 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:48 smithi033 podman[114867]: 2024-04-24 23:39:48.179395454 +0000 UTC m=+0.710408903 container remove 745e9e8a1f57a241b60298477f3ce5051d63be56cbf27a5784cdbf484e12ce79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:39:48.766 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:48 smithi033 podman[114957]: 2024-04-24 23:39:48.425177445 +0000 UTC m=+0.018560187 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:39:48.766 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:48 smithi033 podman[114957]: 2024-04-24 23:39:48.53876064 +0000 UTC m=+0.132143366 container create 3228857263bfc848a7ad261a657e6a1230199b8d2a12c8190120e6f962e27c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T23:39:48.767 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:48 smithi033 podman[114957]: 2024-04-24 23:39:48.767188327 +0000 UTC m=+0.360571066 container init 3228857263bfc848a7ad261a657e6a1230199b8d2a12c8190120e6f962e27c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:39:49.045 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:48 smithi033 podman[114957]: 2024-04-24 23:39:48.770695034 +0000 UTC m=+0.364077760 container start 3228857263bfc848a7ad261a657e6a1230199b8d2a12c8190120e6f962e27c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T23:39:49.045 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:48 smithi033 podman[114957]: 2024-04-24 23:39:48.858675515 +0000 UTC m=+0.452058240 container attach 3228857263bfc848a7ad261a657e6a1230199b8d2a12c8190120e6f962e27c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-24T23:39:49.336 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 24 23:39:49 smithi033 podman[114957]: 2024-04-24 23:39:49.084506037 +0000 UTC m=+0.677888763 container died 3228857263bfc848a7ad261a657e6a1230199b8d2a12c8190120e6f962e27c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T23:39:49.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T23:39:49.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:49.729 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T23:39:49.730 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T23:39:49.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.1 2024-04-24T23:39:50.076 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:49 smithi038 systemd[1]: Stopping Ceph osd.1 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:50.419 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1[37044]: 2024-04-24T23:39:50.075+0000 7f0fccc0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:50.419 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1[37044]: 2024-04-24T23:39:50.075+0000 7f0fccc0c700 -1 osd.1 98 *** Got signal Terminated *** 2024-04-24T23:39:50.419 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1[37044]: 2024-04-24T23:39:50.075+0000 7f0fccc0c700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:39:55.814 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:55 smithi038 podman[73939]: 2024-04-24 23:39:55.519591041 +0000 UTC m=+5.589480541 container died 5d7c154479e8b819e9f91f0639cf88db283ec0d33301807a389041e0d2931714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T23:39:56.169 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:55 smithi038 podman[73939]: 2024-04-24 23:39:55.931170219 +0000 UTC m=+6.001059724 container cleanup 5d7c154479e8b819e9f91f0639cf88db283ec0d33301807a389041e0d2931714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9) 2024-04-24T23:39:56.169 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:55 smithi038 bash[73939]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:39:56.517 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:56 smithi038 podman[73955]: 2024-04-24 23:39:56.200140559 +0000 UTC m=+0.670848326 container remove 5d7c154479e8b819e9f91f0639cf88db283ec0d33301807a389041e0d2931714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:39:56.837 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:56 smithi038 podman[74045]: 2024-04-24 23:39:56.421342898 +0000 UTC m=+0.018272148 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:39:56.837 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:56 smithi038 podman[74045]: 2024-04-24 23:39:56.535105636 +0000 UTC m=+0.132034865 container create 6a69ea9ef9ac599a6a87ef783874288a50935008eda9ee7d81623412cc6fc994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T23:39:57.152 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:56 smithi038 podman[74045]: 2024-04-24 23:39:56.837988425 +0000 UTC m=+0.434917654 container init 6a69ea9ef9ac599a6a87ef783874288a50935008eda9ee7d81623412cc6fc994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:39:57.152 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:56 smithi038 podman[74045]: 2024-04-24 23:39:56.841358906 +0000 UTC m=+0.438288148 container start 6a69ea9ef9ac599a6a87ef783874288a50935008eda9ee7d81623412cc6fc994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-24T23:39:57.152 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:56 smithi038 podman[74045]: 2024-04-24 23:39:56.947257573 +0000 UTC m=+0.544186812 container attach 6a69ea9ef9ac599a6a87ef783874288a50935008eda9ee7d81623412cc6fc994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:39:57.419 INFO:journalctl@ceph.osd.1.smithi038.stdout:Apr 24 23:39:57 smithi038 podman[74045]: 2024-04-24 23:39:57.153405213 +0000 UTC m=+0.750334454 container died 6a69ea9ef9ac599a6a87ef783874288a50935008eda9ee7d81623412cc6fc994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T23:39:57.960 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T23:39:57.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:39:57.999 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T23:39:57.999 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T23:39:57.999 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.2 2024-04-24T23:39:58.321 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:39:58 smithi117 systemd[1]: Stopping Ceph osd.2 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:39:58.580 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:39:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2[37657]: 2024-04-24T23:39:58.321+0000 7f784fe7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:39:58.580 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:39:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2[37657]: 2024-04-24T23:39:58.321+0000 7f784fe7b700 -1 osd.2 98 *** Got signal Terminated *** 2024-04-24T23:39:58.580 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:39:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2[37657]: 2024-04-24T23:39:58.321+0000 7f784fe7b700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:04.114 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:03 smithi117 podman[73895]: 2024-04-24 23:40:03.813705357 +0000 UTC m=+5.622230327 container died 487caf15c0a828d675d519fa17a5b0c98057177078fc65ce22f7448985421570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T23:40:04.424 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:04 smithi117 podman[73895]: 2024-04-24 23:40:04.205245026 +0000 UTC m=+6.013769985 container cleanup 487caf15c0a828d675d519fa17a5b0c98057177078fc65ce22f7448985421570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T23:40:04.425 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:04 smithi117 bash[73895]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:40:04.829 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:04 smithi117 podman[73925]: 2024-04-24 23:40:04.515234549 +0000 UTC m=+0.697997401 container remove 487caf15c0a828d675d519fa17a5b0c98057177078fc65ce22f7448985421570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:40:05.143 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:04 smithi117 podman[74018]: 2024-04-24 23:40:04.732611009 +0000 UTC m=+0.018099192 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:05.144 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:04 smithi117 podman[74018]: 2024-04-24 23:40:04.866391075 +0000 UTC m=+0.151879301 container create d2a358bcd6e7fed6af46c4001a02c7cadda4a3d50950561202baf8d8c3ac147b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T23:40:05.425 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:05 smithi117 podman[74018]: 2024-04-24 23:40:05.14453954 +0000 UTC m=+0.430027733 container init d2a358bcd6e7fed6af46c4001a02c7cadda4a3d50950561202baf8d8c3ac147b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T23:40:05.425 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:05 smithi117 podman[74018]: 2024-04-24 23:40:05.1479632 +0000 UTC m=+0.433451384 container start d2a358bcd6e7fed6af46c4001a02c7cadda4a3d50950561202baf8d8c3ac147b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:40:05.425 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:05 smithi117 podman[74018]: 2024-04-24 23:40:05.227419288 +0000 UTC m=+0.512907471 container attach d2a358bcd6e7fed6af46c4001a02c7cadda4a3d50950561202baf8d8c3ac147b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:40:05.788 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:05 smithi117 podman[74018]: 2024-04-24 23:40:05.437737684 +0000 UTC m=+0.723225868 container died d2a358bcd6e7fed6af46c4001a02c7cadda4a3d50950561202baf8d8c3ac147b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:40:06.023 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T23:40:06.040 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:06 smithi117 podman[74044]: 2024-04-24 23:40:06.006621549 +0000 UTC m=+0.559480905 container remove d2a358bcd6e7fed6af46c4001a02c7cadda4a3d50950561202baf8d8c3ac147b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T23:40:06.040 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:06 smithi117 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-24T23:40:06.041 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:06 smithi117 systemd[1]: Stopped Ceph osd.2 for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:40:06.041 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 24 23:40:06 smithi117 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.2.service: Consumed 10min 49.962s CPU time. 2024-04-24T23:40:06.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:06.062 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T23:40:06.062 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T23:40:06.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.3 2024-04-24T23:40:06.381 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:06 smithi033 systemd[1]: Stopping Ceph osd.3 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:06.703 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:06 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3[51096]: 2024-04-24T23:40:06.380+0000 7ff1b1f3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:06.704 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:06 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3[51096]: 2024-04-24T23:40:06.380+0000 7ff1b1f3a700 -1 osd.3 98 *** Got signal Terminated *** 2024-04-24T23:40:06.704 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:06 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3[51096]: 2024-04-24T23:40:06.380+0000 7ff1b1f3a700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:08.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:08 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:08.596+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:09.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:09 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:09.580+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:09.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:09.512+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:10.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:10 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:10.588+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:10.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:10.557+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:11.203 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:11 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3[51096]: 2024-04-24T23:40:11.049+0000 7ff1ae52e700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:46.268303+0000 front 2024-04-24T23:39:46.268332+0000 (oldest deadline 2024-04-24T23:40:10.368526+0000) 2024-04-24T23:40:11.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:10 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:10.853+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:11.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:11 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:11.557+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:11.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:11.538+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:12.182 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:11 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:11.864+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:12.183 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:11 smithi033 podman[115064]: 2024-04-24 23:40:11.928657887 +0000 UTC m=+5.668888364 container died ad189a7f55cd061fdadd23b3393a4ab53ebd00f38bff2bb14a4560bea62d8d94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:40:12.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:12.098+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:12.453 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:12 smithi033 podman[115064]: 2024-04-24 23:40:12.273854829 +0000 UTC m=+6.014085304 container cleanup ad189a7f55cd061fdadd23b3393a4ab53ebd00f38bff2bb14a4560bea62d8d94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T23:40:12.453 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:12 smithi033 bash[115064]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:40:12.839 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:12 smithi033 podman[115092]: 2024-04-24 23:40:12.517358964 +0000 UTC m=+0.589144529 container remove ad189a7f55cd061fdadd23b3393a4ab53ebd00f38bff2bb14a4560bea62d8d94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T23:40:12.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:12 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:12.582+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:12.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:12.523+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:13.090 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:12 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:12.839+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:13.090 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:12 smithi033 podman[115183]: 2024-04-24 23:40:12.849204392 +0000 UTC m=+0.018585211 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:13.090 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:12 smithi033 podman[115183]: 2024-04-24 23:40:12.98499314 +0000 UTC m=+0.154373984 container create fbfd72376ee5e3b68d04f5c0d2a44e12b8af3e0415b827485250de027ab7a9a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T23:40:13.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:13.097+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:13.454 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:13 smithi033 podman[115183]: 2024-04-24 23:40:13.221201954 +0000 UTC m=+0.390582755 container init fbfd72376ee5e3b68d04f5c0d2a44e12b8af3e0415b827485250de027ab7a9a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T23:40:13.454 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:13 smithi033 podman[115183]: 2024-04-24 23:40:13.224303573 +0000 UTC m=+0.393684382 container start fbfd72376ee5e3b68d04f5c0d2a44e12b8af3e0415b827485250de027ab7a9a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:40:13.454 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:13 smithi033 podman[115183]: 2024-04-24 23:40:13.296320969 +0000 UTC m=+0.465701788 container attach fbfd72376ee5e3b68d04f5c0d2a44e12b8af3e0415b827485250de027ab7a9a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:40:13.790 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 24 23:40:13 smithi033 podman[115183]: 2024-04-24 23:40:13.53569604 +0000 UTC m=+0.705076842 container died fbfd72376ee5e3b68d04f5c0d2a44e12b8af3e0415b827485250de027ab7a9a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T23:40:13.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:13 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:13.622+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:13.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:13.507+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:14.108 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:13 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:13.878+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:14.131 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T23:40:14.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:14.173 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T23:40:14.173 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T23:40:14.173 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.4 2024-04-24T23:40:14.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:14.126+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:14.496 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:14 smithi038 systemd[1]: Stopping Ceph osd.4 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:14.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:14 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:14.597+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:14.919 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:14 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-24T23:40:14.496+0000 7f8f252cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:14.919 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:14 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-24T23:40:14.496+0000 7f8f252cc700 -1 osd.4 98 *** Got signal Terminated *** 2024-04-24T23:40:14.919 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:14 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-24T23:40:14.496+0000 7f8f252cc700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:14.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:14.481+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:15.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:14 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:14.868+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:15.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:15.157+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:15.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:15 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:15.554+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:15.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:15.504+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:16.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:15 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:15.917+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:16.418 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:16.207+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:16.674 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:16.552+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:16.675 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:16 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:16.418+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:16.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:16 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:16.231+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:16.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:16 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:16.541+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:17.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:16 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:16.881+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:17.522 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:17 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:17.250+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:17.522 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:17 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:17.454+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:17.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:17 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:17.276+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:17.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:17 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:17.568+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:17.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:17 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:17.568+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:17.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:17 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:17.522+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:18.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:17 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:17.885+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:18.618 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:18 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:18.316+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:18.674 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:18.243+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:18.675 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:18.424+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:18.675 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:18 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:18.474+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:18.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:18 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:18.300+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:18.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:18 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:18.618+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:18.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:18 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:18.618+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:19.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:18 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:18.906+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:19.503 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:19 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:19.236+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:19.503 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:19 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:19.465+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:19.642 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:19.288+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:19.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:19.272+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:19.919 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:19 smithi038 podman[74164]: 2024-04-24 23:40:19.881184067 +0000 UTC m=+5.505933958 container died 3e599fb22567dd2380e91e921105cdd20e17a76108c03d5ae9855c0047eb9721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:40:19.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:19.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:19.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:19 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:19.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:19.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:19 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:19.503+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:20.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:19.930+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:20.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:19.930+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:20.534 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:20.463+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:20.534 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:20.237+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:20.534 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:20.237+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:20.555 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:20 smithi038 podman[74164]: 2024-04-24 23:40:20.279541269 +0000 UTC m=+5.904291124 container cleanup 3e599fb22567dd2380e91e921105cdd20e17a76108c03d5ae9855c0047eb9721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:40:20.555 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:20 smithi038 bash[74164]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:40:20.555 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:20 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:20.305+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:20.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:20 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:20.315+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:20.876 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:20 smithi038 podman[74183]: 2024-04-24 23:40:20.556041092 +0000 UTC m=+0.677368049 container remove 3e599fb22567dd2380e91e921105cdd20e17a76108c03d5ae9855c0047eb9721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True) 2024-04-24T23:40:20.876 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:20 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:20.688+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:20.877 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:20 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:20.688+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:20.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:20 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:20.533+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:21.166 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:20 smithi038 podman[74269]: 2024-04-24 23:40:20.780777412 +0000 UTC m=+0.018769633 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:21.166 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:21 smithi038 podman[74269]: 2024-04-24 23:40:21.040193693 +0000 UTC m=+0.278185894 container create 90149289758ad1fbfdc4e1b54f8a0de2527e5ab09795c48e1daceef277250b6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T23:40:21.364 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:20 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:20.970+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:21.364 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:20 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:20.970+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:21.418 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:21.289+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:21.419 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:21 smithi038 podman[74269]: 2024-04-24 23:40:21.301498372 +0000 UTC m=+0.539490573 container init 90149289758ad1fbfdc4e1b54f8a0de2527e5ab09795c48e1daceef277250b6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T23:40:21.419 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:21 smithi038 podman[74269]: 2024-04-24 23:40:21.305064719 +0000 UTC m=+0.543056928 container start 90149289758ad1fbfdc4e1b54f8a0de2527e5ab09795c48e1daceef277250b6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:40:21.419 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:21 smithi038 podman[74269]: 2024-04-24 23:40:21.409775362 +0000 UTC m=+0.647767563 container attach 90149289758ad1fbfdc4e1b54f8a0de2527e5ab09795c48e1daceef277250b6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T23:40:21.584 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:21.463+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:21.584 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:21.252+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:21.584 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:21.252+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:21.701 INFO:journalctl@ceph.osd.4.smithi038.stdout:Apr 24 23:40:21 smithi038 podman[74269]: 2024-04-24 23:40:21.60665906 +0000 UTC m=+0.844651261 container died 90149289758ad1fbfdc4e1b54f8a0de2527e5ab09795c48e1daceef277250b6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T23:40:21.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:21 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:21.364+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:21.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:21 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:21.584+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:21.962 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:21.700+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:21.962 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:21 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:21.700+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:22.299 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T23:40:22.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:22.342 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T23:40:22.342 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T23:40:22.343 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.5 2024-04-24T23:40:22.414 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:21 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:21.983+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:22.415 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:21 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:21.983+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:22.534 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:22.441+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:22.535 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:22.533+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:22.535 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:22.203+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:22.535 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:22.203+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:22.535 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:22 smithi117 systemd[1]: Stopping Ceph osd.5 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:22.567 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:22 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:22.300+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:22.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:22 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:22.414+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:22.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:22 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:22.653+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:22.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:22 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:22.653+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:22.924 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:22.673+0000 7fe270acf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:22.924 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:22.673+0000 7fe270acf700 -1 osd.5 98 *** Got signal Terminated *** 2024-04-24T23:40:22.925 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:22 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:22.673+0000 7fe270acf700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:23.368 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:23 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:23.010+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:23.368 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:23 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:23.010+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:23.489 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:23 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:23.185+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:23.489 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:23 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:23.185+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:23.540 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:23.284+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:23.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:23 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:23.368+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:23.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:23.653+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:23.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:23 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:23.653+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:23.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:23 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:23.489+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:23.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:23 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:23.560+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:24.388 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:24 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:24.040+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:24.388 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:24 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:24.040+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:24.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:24 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:24.137+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:24.425 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:24 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:24.137+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:24.425 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:24 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:24.137+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.295195+0000 front 2024-04-24T23:39:59.295215+0000 (oldest deadline 2024-04-24T23:40:23.995492+0000) 2024-04-24T23:40:24.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:24.316+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:24.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:24 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:24.388+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:24.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:24 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:24.582+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:24.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:24 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:24.474+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:25.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:24.698+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:25.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:24 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:24.698+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:25.387 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:25 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:25.013+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:25.387 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:25 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:25.013+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:25.435 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:25 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:25.175+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:25.435 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:25 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:25.175+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:25.435 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:25 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:25.175+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.295195+0000 front 2024-04-24T23:39:59.295215+0000 (oldest deadline 2024-04-24T23:40:23.995492+0000) 2024-04-24T23:40:25.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:25 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:25.330+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:25.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:25 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:25.387+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:25.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:25 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:25.387+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:25.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:25 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:25.565+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:25.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:25 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:25.435+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:25.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:25 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:25.435+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:26.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:25 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:25.698+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:26.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:25 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:25.698+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:26.400 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:26 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:26.033+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:26.401 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:26 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:26.033+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:26.401 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:26 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:26.033+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:26.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:26 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:26.151+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:26.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:26 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:26.151+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:26.425 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:26 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:26.151+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.295195+0000 front 2024-04-24T23:39:59.295215+0000 (oldest deadline 2024-04-24T23:40:23.995492+0000) 2024-04-24T23:40:26.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:26 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:26.299+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:26.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:26 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:26.299+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:26.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:26 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:26.400+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:26.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:26 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:26.400+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:26.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:26 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:26.526+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:26.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:26 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:26.446+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:26.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:26 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:26.446+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:27.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:26 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:26.698+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:27.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:26 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:26.698+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:27.386 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:27.061+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:27.386 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:27.061+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:27.386 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:27.061+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:27.424 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:27.174+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:42.892990+0000 front 2024-04-24T23:39:42.892856+0000 (oldest deadline 2024-04-24T23:40:11.093611+0000) 2024-04-24T23:40:27.425 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:27.174+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.594854+0000 front 2024-04-24T23:39:54.594914+0000 (oldest deadline 2024-04-24T23:40:19.294836+0000) 2024-04-24T23:40:27.425 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5[44614]: 2024-04-24T23:40:27.174+0000 7fe26d0c3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.295195+0000 front 2024-04-24T23:39:59.295215+0000 (oldest deadline 2024-04-24T23:40:23.995492+0000) 2024-04-24T23:40:27.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:27.267+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:27.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:27.267+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:27.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:27.384+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:27.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:27 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:27.384+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:27.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:27.566+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:27.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:27.475+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:27.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:27 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:27.475+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:28.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:27.692+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:28.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:27.692+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:28.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:27 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:27.692+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:28.372 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:28.069+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:28.372 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:28.069+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:28.372 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:28.069+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:28.539 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:28 smithi117 podman[74124]: 2024-04-24 23:40:28.252464072 +0000 UTC m=+5.710627915 container died 695b6e98e29f5331457d8126452665ad3003df6d0dbb722d9dbb806db3a31451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:40:28.540 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:28 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:28.497+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:28.540 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:28 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:28.497+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:28.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:28 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:28.274+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:28.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:28 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:28.274+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:28.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:28.372+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:28.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:28.372+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:28.891 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:28 smithi117 podman[74124]: 2024-04-24 23:40:28.649837478 +0000 UTC m=+6.108001373 container cleanup 695b6e98e29f5331457d8126452665ad3003df6d0dbb722d9dbb806db3a31451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T23:40:28.892 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:28 smithi117 bash[74124]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:40:28.892 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:28 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:28.539+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:28.892 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:28 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:28.539+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:29.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:28 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:28.710+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:29.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:28 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:28.710+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:29.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:28 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:28.710+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:29.174 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:28 smithi117 podman[74141]: 2024-04-24 23:40:28.892630408 +0000 UTC m=+0.639587435 container remove 695b6e98e29f5331457d8126452665ad3003df6d0dbb722d9dbb806db3a31451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:40:29.410 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:29 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:29.115+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:29.410 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:29 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:29.115+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:29.410 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:29 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:29.115+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:29.470 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:29 smithi117 podman[74230]: 2024-04-24 23:40:29.116155058 +0000 UTC m=+0.017961266 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:29.470 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:29 smithi117 podman[74230]: 2024-04-24 23:40:29.227028684 +0000 UTC m=+0.128834917 container create d048ead4a3fd0a45be4dfbc5a3f7464b2c0270bd01e8d87dfff9109de9df2ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T23:40:29.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:29 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:29.247+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:29.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:29 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:29.247+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:29.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:29 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:29.410+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:29.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:29 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:29.410+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:29.761 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:29 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:29.493+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:29.762 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:29 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:29.493+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:29.762 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:29 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:29.583+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:29.762 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:29 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:29.583+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:29.762 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:29 smithi117 podman[74230]: 2024-04-24 23:40:29.471011968 +0000 UTC m=+0.372818159 container init d048ead4a3fd0a45be4dfbc5a3f7464b2c0270bd01e8d87dfff9109de9df2ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-24T23:40:29.762 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:29 smithi117 podman[74230]: 2024-04-24 23:40:29.474109127 +0000 UTC m=+0.375915329 container start d048ead4a3fd0a45be4dfbc5a3f7464b2c0270bd01e8d87dfff9109de9df2ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9) 2024-04-24T23:40:29.762 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:29 smithi117 podman[74230]: 2024-04-24 23:40:29.579567392 +0000 UTC m=+0.481373585 container attach d048ead4a3fd0a45be4dfbc5a3f7464b2c0270bd01e8d87dfff9109de9df2ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T23:40:30.057 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 24 23:40:29 smithi117 podman[74230]: 2024-04-24 23:40:29.763183823 +0000 UTC m=+0.664990017 container died d048ead4a3fd0a45be4dfbc5a3f7464b2c0270bd01e8d87dfff9109de9df2ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:40:30.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:29 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:29.730+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:30.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:29 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:29.730+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:30.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:29 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:29.730+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:30.335 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T23:40:30.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:30.374 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T23:40:30.374 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T23:40:30.375 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.6 2024-04-24T23:40:30.402 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:30.133+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:30.402 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:30.133+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:30.403 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:30.133+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:30.560 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:30 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:30.484+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:30.560 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:30 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:30.484+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:30.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:30 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:30.209+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:30.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:30 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:30.209+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:30.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:30.423+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:30.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:30.423+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:30.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:30 smithi033 systemd[1]: Stopping Ceph osd.6 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:30.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:30.702+0000 7f62c9aea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:30.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:30.702+0000 7f62c9aea700 -1 osd.6 98 *** Got signal Terminated *** 2024-04-24T23:40:30.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:30 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:30.702+0000 7f62c9aea700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:30.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:30 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:30.560+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:30.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:30 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:30.560+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:31.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:30 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:30.778+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:31.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:30 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:30.778+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:31.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:30 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:30.778+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:31.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:31.159+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:31.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:31.159+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:31.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:31.159+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:31.454 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:31.388+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:31.454 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:31 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:31.388+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:31.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:31 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:31.238+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:31.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:31 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:31.238+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:31.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:31 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:31.515+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:31.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:31 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:31.515+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:31.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:31 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:31.585+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:31.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:31 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:31.585+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:32.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:31 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:31.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:32.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:31 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:31.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:32.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:31 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:31.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:32.453 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:32.378+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:32.453 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:32.378+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:32.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:32.194+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:32.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:32.194+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:32.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:32.194+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:32.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:32 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:32.243+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:32.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:32 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:32.243+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:32.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:32 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:32.560+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:32.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:32 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:32.560+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:32.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:32 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:32.545+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:32.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:32 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:32.546+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:33.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:32 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:32.815+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:33.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:32 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:32.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:33.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:32 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:32.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:33.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:32 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:32.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:33.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:33.260+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:33.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:33.260+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:33.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:33.260+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:33.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:33.359+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:33.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:33.359+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:33.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:33.226+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:33.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:33.226+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:33.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:33 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:33.226+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:33.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:33 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:33.598+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:33.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:33 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:33.598+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:33.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:33 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:33.578+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:33.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:33 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:33.578+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:34.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:33.801+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:34.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:33.801+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:34.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:33.801+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:34.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:33 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:33.801+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:34.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:34.278+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:34.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:34.278+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:34.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:34.278+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:34.703 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:34.363+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:34.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:34.363+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:34.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:34.363+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.610788+0000 front 2024-04-24T23:40:10.610761+0000 (oldest deadline 2024-04-24T23:40:34.110876+0000) 2024-04-24T23:40:34.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:34.249+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:34.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:34.249+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:34.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:34 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:34.249+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:34.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:34 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:34.637+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:34.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:34 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:34.637+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:34.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:34 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:34.538+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:34.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:34 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:34.538+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:34.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:34 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:34.538+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:35.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:34.804+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:35.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:34.804+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:35.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:34.804+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:35.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:34 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:34.804+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:35.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:35.287+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:35.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:35.287+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:35.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:35.287+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:35.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:35.323+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.607056+0000 front 2024-04-24T23:39:52.607252+0000 (oldest deadline 2024-04-24T23:40:16.107123+0000) 2024-04-24T23:40:35.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:35.323+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:03.009658+0000 front 2024-04-24T23:40:03.009644+0000 (oldest deadline 2024-04-24T23:40:25.309721+0000) 2024-04-24T23:40:35.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6[57787]: 2024-04-24T23:40:35.323+0000 7f62c60de700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.610788+0000 front 2024-04-24T23:40:10.610761+0000 (oldest deadline 2024-04-24T23:40:34.110876+0000) 2024-04-24T23:40:35.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:35.239+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:35.705 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:35.239+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:35.705 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:35.239+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:35.705 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:35 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:35.239+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:35.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:35 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:35.595+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:35.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:35 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:35.595+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:35.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:35 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:35.565+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:35.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:35 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:35.565+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:35.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:35 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:35.565+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:36.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:35.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:36.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:35.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:36.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:35.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:36.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:35 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:35.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:36.512 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:36 smithi033 podman[115291]: 2024-04-24 23:40:36.26064947 +0000 UTC m=+5.681859671 container died 9128638536a34f2a0538ccd32da2290c677c456dc68294dcea947dece2e82297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T23:40:36.512 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:36.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:36.512 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:36.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:36.512 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:36.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:36.512 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:36 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:36.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:36.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:36.331+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:36.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:36.331+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:36.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:36.331+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:36.795 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:36 smithi033 podman[115291]: 2024-04-24 23:40:36.603221311 +0000 UTC m=+6.024431511 container cleanup 9128638536a34f2a0538ccd32da2290c677c456dc68294dcea947dece2e82297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:40:36.795 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:36 smithi033 bash[115291]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6 2024-04-24T23:40:36.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:36.587+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:36.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:36.587+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:36.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:36.582+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:36.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:36.582+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:36.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:36 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:36.582+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:37.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:36.802+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:37.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:36.802+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:37.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:36.802+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:37.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:36.802+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:37.201 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:36 smithi033 podman[115309]: 2024-04-24 23:40:36.796198141 +0000 UTC m=+0.536177124 container remove 9128638536a34f2a0538ccd32da2290c677c456dc68294dcea947dece2e82297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T23:40:37.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:37.276+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:37.276+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:37.276+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:37 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:37.276+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:37 smithi033 podman[115402]: 2024-04-24 23:40:37.105190589 +0000 UTC m=+0.019017710 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:37 smithi033 podman[115402]: 2024-04-24 23:40:37.21368644 +0000 UTC m=+0.127513563 container create f9786fe61ec41422d910fd0b37ed31545fff206f0827fc4e043e752c6ac90da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:37 smithi033 podman[115402]: 2024-04-24 23:40:37.42451326 +0000 UTC m=+0.338340383 container init f9786fe61ec41422d910fd0b37ed31545fff206f0827fc4e043e752c6ac90da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T23:40:37.454 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:37 smithi033 podman[115402]: 2024-04-24 23:40:37.427893109 +0000 UTC m=+0.341720234 container start f9786fe61ec41422d910fd0b37ed31545fff206f0827fc4e043e752c6ac90da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T23:40:37.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:37.368+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:37.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:37.368+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:37.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:37.368+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:37.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:37.580+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:37.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:37.580+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:37.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:37.580+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:37.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:37.543+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:37.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:37.543+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:37.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:37 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:37.543+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:37.954 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:37 smithi033 podman[115402]: 2024-04-24 23:40:37.508064235 +0000 UTC m=+0.421891361 container attach f9786fe61ec41422d910fd0b37ed31545fff206f0827fc4e043e752c6ac90da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-24T23:40:37.954 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 24 23:40:37 smithi033 podman[115402]: 2024-04-24 23:40:37.728409895 +0000 UTC m=+0.642237016 container died f9786fe61ec41422d910fd0b37ed31545fff206f0827fc4e043e752c6ac90da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T23:40:38.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:37.780+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:38.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:37.780+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:38.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:37.780+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:38.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:37 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:37.780+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:38.378 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:38.233+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:38.378 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:38.233+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:38.378 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:38.233+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:38.378 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:38 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:38.233+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:38.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T23:40:38.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:38.483 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T23:40:38.484 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T23:40:38.484 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.7 2024-04-24T23:40:38.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:38.349+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:38.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:38.349+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:38.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:38.349+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:38.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 systemd[1]: Stopping Ceph osd.7 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:38.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:38 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:38.549+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:38.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:38 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:38.549+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:38.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:38 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:38.549+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:38.926 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:38 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:38.583+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:38.926 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:38 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:38.583+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:38.926 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:38 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:38.583+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:39.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:38.808+0000 7fb1ff0b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:39.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:38.808+0000 7fb1ff0b3700 -1 osd.7 98 *** Got signal Terminated *** 2024-04-24T23:40:39.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:38.808+0000 7fb1ff0b3700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:39.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:38.741+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:39.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:38.741+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:39.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:38.741+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:39.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:38 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:38.741+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:39.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:39.303+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:39.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:39.303+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:39.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:39.303+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:39.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:39 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:39.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:39.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:39 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:39.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:39.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:39 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:39.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:39.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:39 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:39.253+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:39.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:39 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:39.501+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:39.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:39 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:39.501+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:39.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:39 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:39.501+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:39.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:39 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:39.621+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:39.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:39 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:39.621+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:39.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:39 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:39.621+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:40.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:39.723+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:40.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:39.723+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:40.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:39.723+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:40.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:39 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:39.723+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:40.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:40.285+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:40.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:40.285+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:40.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:40.285+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:40.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:40 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:40.243+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:40.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:40 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:40.243+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:40.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:40 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:40.243+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:40.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:40 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:40.243+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:40.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:40 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:40.509+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:40.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:40 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:40.509+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:40.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:40 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:40.509+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:40.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:40 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:40.660+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:40.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:40 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:40.660+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:40.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:40 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:40.660+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:41.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:40.689+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:41.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:40.689+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:41.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:40.689+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:41.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:40.689+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:41.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:40.689+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:41.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:41.200+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:41.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:41.200+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:41.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:41.200+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:41.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:41 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:41.200+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:41.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:41.305+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:41.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:41.305+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:41.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:41.305+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:41.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:41.305+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:19.146323+0000 front 2024-04-24T23:40:19.146359+0000 (oldest deadline 2024-04-24T23:40:40.846510+0000) 2024-04-24T23:40:41.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:41 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:41.696+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:41.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:41 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:41.696+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:41.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:41 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:41.696+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:41.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:41 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:41.533+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:41.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:41 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:41.533+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:41.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:41 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:41.533+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:42.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:41.674+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:42.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:41.674+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:42.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:41.674+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:42.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:41.674+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:42.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:41 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:41.674+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:42.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:42 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:42.156+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:42.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:42 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:42.156+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:42.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:42 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:42.156+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:42.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:42 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:42.156+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:42.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:42 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:42.156+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:42.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:42.343+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:42.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:42.343+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:42.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:42.343+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:42.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:42.343+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:19.146323+0000 front 2024-04-24T23:40:19.146359+0000 (oldest deadline 2024-04-24T23:40:40.846510+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:42.572+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:42.572+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:42.572+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:42.572+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:42.692+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:42.692+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:42.692+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:42.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:42 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:42.692+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:43.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:42.694+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:43.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:42.694+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:43.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:42.694+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:43.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:42.694+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:43.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:42 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:42.694+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:43.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:43.145+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:43.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:43.145+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:43.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:43.145+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:43.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:43.145+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:43.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:43 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:43.145+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:43.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:43.329+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.442501+0000 front 2024-04-24T23:39:53.442533+0000 (oldest deadline 2024-04-24T23:40:18.142625+0000) 2024-04-24T23:40:43.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:43.329+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.243783+0000 front 2024-04-24T23:40:02.243714+0000 (oldest deadline 2024-04-24T23:40:25.743784+0000) 2024-04-24T23:40:43.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:43.329+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:08.545680+0000 front 2024-04-24T23:40:08.545665+0000 (oldest deadline 2024-04-24T23:40:33.245438+0000) 2024-04-24T23:40:43.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7[50550]: 2024-04-24T23:40:43.329+0000 7fb1f769f700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:19.146323+0000 front 2024-04-24T23:40:19.146359+0000 (oldest deadline 2024-04-24T23:40:40.846510+0000) 2024-04-24T23:40:43.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:43.622+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:43.622+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:43.622+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:43.622+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:43.687+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:43.687+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:43.687+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:43.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:43 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:43.687+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:44.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:43.736+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:44.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:43.736+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:44.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:43.736+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:44.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:43.736+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:44.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:43.736+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:44.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:44 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:44.155+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:44.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:44 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:44.155+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:44.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:44 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:44.155+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:44.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:44 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:44.155+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:44.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:44 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:44.155+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:44.634 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:44 smithi038 podman[74385]: 2024-04-24 23:40:44.32916316 +0000 UTC m=+5.640945332 container died 8933c2386c60e3aa5a9a1e298d24098a16dd29e7773884e909582c9e6b4691d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:40:44.919 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:44 smithi038 podman[74385]: 2024-04-24 23:40:44.734282936 +0000 UTC m=+6.046065060 container cleanup 8933c2386c60e3aa5a9a1e298d24098a16dd29e7773884e909582c9e6b4691d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T23:40:44.919 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:44 smithi038 bash[74385]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7 2024-04-24T23:40:44.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:44 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:44.707+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:44.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:44 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:44.707+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:44.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:44 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:44.707+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:44.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:44 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:44.707+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:44.920 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:44 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:44.707+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:44.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:44.646+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:44.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:44.646+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:44.924 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:44.646+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:44.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:44.646+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:44.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:44.731+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:44.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:44.731+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:44.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:44.731+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:44.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:44 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:44.731+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:45.341 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74402]: 2024-04-24 23:40:45.035716345 +0000 UTC m=+0.699168279 container remove 8933c2386c60e3aa5a9a1e298d24098a16dd29e7773884e909582c9e6b4691d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:40:45.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:45 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:45.193+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:45.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:45 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:45.193+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:45.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:45 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:45.193+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:45.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:45 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:45.193+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:45.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:45 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:45.193+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:45.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74493]: 2024-04-24 23:40:45.245603129 +0000 UTC m=+0.018888940 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:45.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74493]: 2024-04-24 23:40:45.344790482 +0000 UTC m=+0.118076336 container create c2dc2c5870b4fca53e29801b2025963fc1d7479d281bd22511571dc43b2e067d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:40:45.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74493]: 2024-04-24 23:40:45.580637677 +0000 UTC m=+0.353923486 container init c2dc2c5870b4fca53e29801b2025963fc1d7479d281bd22511571dc43b2e067d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:40:45.669 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74493]: 2024-04-24 23:40:45.583764209 +0000 UTC m=+0.357050023 container start c2dc2c5870b4fca53e29801b2025963fc1d7479d281bd22511571dc43b2e067d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:45.665+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:45.665+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:45.665+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:45.665+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:45.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:45.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:45.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:45.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:45.926 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:45 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:45.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:46.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74493]: 2024-04-24 23:40:45.689023796 +0000 UTC m=+0.462309622 container attach c2dc2c5870b4fca53e29801b2025963fc1d7479d281bd22511571dc43b2e067d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:40:46.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Apr 24 23:40:45 smithi038 podman[74493]: 2024-04-24 23:40:45.886404611 +0000 UTC m=+0.659690421 container died c2dc2c5870b4fca53e29801b2025963fc1d7479d281bd22511571dc43b2e067d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T23:40:46.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:45 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:45.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:46.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:45 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:45.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:46.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:45 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:45.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:46.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:45 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:45.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:46.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:45 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:45.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:46.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:46 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:46.163+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:46.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:46 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:46.163+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:46.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:46 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:46.163+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:46.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:46 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:46.163+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:46.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:46 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:46.163+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:46.620 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T23:40:46.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:46.663 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T23:40:46.663 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T23:40:46.663 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.8 2024-04-24T23:40:46.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:46 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:46.663+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:46.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:46 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:46.663+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:46.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:46 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:46.663+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:46.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:46 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:46.663+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:46.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:46 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:46.663+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:46.700+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:46.700+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:46.700+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:46.700+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.696+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.696+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.696+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.696+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:46.959 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 systemd[1]: Stopping Ceph osd.8 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:46.960 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.959+0000 7fe7fc882700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:46.960 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.959+0000 7fe7fc882700 -1 osd.8 98 *** Got signal Terminated *** 2024-04-24T23:40:46.960 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:46 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:46.959+0000 7fe7fc882700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:47.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:47 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:47.197+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:47.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:47 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:47.197+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:47.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:47 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:47.197+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:47.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:47 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:47.197+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:47.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:47 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:47.197+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:47.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:47 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:47.639+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:47.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:47 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:47.639+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:47.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:47 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:47.639+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:47.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:47 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:47.639+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:47.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:47 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:47.639+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:48.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:47.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:47.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:47.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:47.717+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:47.691+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:47.691+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:47.691+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:48.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:47 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:47.691+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:48.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:48 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:48.234+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:48.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:48 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:48.234+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:48.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:48 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:48.234+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:48.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:48 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:48.234+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:48.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:48 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:48.234+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:48.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:48.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:48.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:48.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:48.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:48.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:48.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:48.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:48.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:48.642+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:49.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:48.727+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:48.727+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:48.727+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:48.727+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:48.700+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:48.700+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:48.700+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:49.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:48 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:48.700+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:49.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:49 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:49.270+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:49.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:49 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:49.270+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:49.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:49 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:49.270+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:49.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:49 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:49.270+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:49.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:49 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:49.270+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:49.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:49 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:49.668+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:49.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:49 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:49.668+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:49.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:49 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:49.668+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:49.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:49 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:49.668+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:49.919 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:49 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:49.668+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:49.739+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:49.739+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:49.739+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:49.739+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:49.748+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:49.748+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:49.748+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:49.748+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:50.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:49 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:49.748+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:50.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:50 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:50.310+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:50.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:50 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:50.310+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:50.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:50 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:50.310+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:50.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:50 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:50.310+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:50.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:50 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:50.310+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:50.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:50 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:50.310+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:51.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:50.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:51.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:50.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:51.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:50.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:51.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:50.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:51.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:50.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:51.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:50 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:50.703+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:51.174 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:50.703+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:51.174 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:50.703+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:50.703+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:50.703+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:50.792+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:50.792+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:50.792+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:50.792+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:51.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:50 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:50.792+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:51.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:51 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:51.282+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:51.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:51 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:51.282+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:51.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:51 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:51.282+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:51.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:51 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:51.282+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:51.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:51 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:51.282+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:51.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:51 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:51.282+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:52.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:51 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:51.746+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:52.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:51 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:51.746+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:52.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:51 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:51.746+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:52.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:51 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:51.746+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:52.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:51 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:51.746+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:52.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:51 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:51.746+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:52.174 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:51.683+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:54.783726+0000 front 2024-04-24T23:39:54.783808+0000 (oldest deadline 2024-04-24T23:40:15.883744+0000) 2024-04-24T23:40:52.174 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:51.683+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.285750+0000 front 2024-04-24T23:40:02.285781+0000 (oldest deadline 2024-04-24T23:40:24.486473+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:51.683+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:11.289411+0000 front 2024-04-24T23:40:11.289503+0000 (oldest deadline 2024-04-24T23:40:36.589466+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8[51153]: 2024-04-24T23:40:51.683+0000 7fe7f8e76700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:16.590166+0000 front 2024-04-24T23:40:16.589948+0000 (oldest deadline 2024-04-24T23:40:41.889924+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:51.782+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:51.782+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:51.782+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:51.782+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:52.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:51 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:51.782+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:52.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:52 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:52.259+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:52.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:52 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:52.259+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:52.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:52 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:52.259+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:52.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:52 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:52.259+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:52.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:52 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:52.259+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:52.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:52 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:52.259+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:52.858 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:52 smithi117 podman[74347]: 2024-04-24 23:40:52.54890781 +0000 UTC m=+5.687009578 container died 3e5b24f5ac9be52316e55b3afad97f15092572df9fc6cd23f5ad35f15c4dfac5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:40:52.859 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:52 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:52.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:52.859 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:52 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:52.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:52.859 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:52 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:52.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:52.859 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:52 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:52.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:52.859 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:52 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:52.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:53.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:52 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:52.764+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:53.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:52 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:52.764+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:53.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:52 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:52.764+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:53.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:52 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:52.764+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:53.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:52 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:52.764+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:53.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:52 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:52.764+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:53.174 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:52 smithi117 podman[74347]: 2024-04-24 23:40:52.967206932 +0000 UTC m=+6.105308700 container cleanup 3e5b24f5ac9be52316e55b3afad97f15092572df9fc6cd23f5ad35f15c4dfac5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:40:53.175 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:52 smithi117 bash[74347]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8 2024-04-24T23:40:53.626 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:53 smithi117 podman[74364]: 2024-04-24 23:40:53.193677448 +0000 UTC m=+0.641142139 container remove 3e5b24f5ac9be52316e55b3afad97f15092572df9fc6cd23f5ad35f15c4dfac5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T23:40:53.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:53 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:53.219+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:53.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:53 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:53.219+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:53.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:53 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:53.219+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:53.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:53 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:53.219+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:53.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:53 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:53.219+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:53.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:53 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:53.219+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:53.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:53.780+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:53.780+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:53.780+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:53.780+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:53.780+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:53 smithi117 podman[74453]: 2024-04-24 23:40:53.627862701 +0000 UTC m=+0.109208587 container create ac27434ef000652a669fd2be33d82d39994e5dd8d7de5e700954d09d40f62ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:53 smithi117 podman[74453]: 2024-04-24 23:40:53.536043086 +0000 UTC m=+0.017388987 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:53 smithi117 podman[74453]: 2024-04-24 23:40:53.86396993 +0000 UTC m=+0.345315816 container init ac27434ef000652a669fd2be33d82d39994e5dd8d7de5e700954d09d40f62ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:40:53.925 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:53 smithi117 podman[74453]: 2024-04-24 23:40:53.866924909 +0000 UTC m=+0.348270798 container start ac27434ef000652a669fd2be33d82d39994e5dd8d7de5e700954d09d40f62ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-24T23:40:54.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:53 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:53.743+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:54.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:53 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:53.743+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:54.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:53 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:53.743+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:54.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:53 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:53.743+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:54.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:53 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:53.743+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:54.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:53 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:53.743+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:54.184 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:53 smithi117 podman[74453]: 2024-04-24 23:40:53.964102384 +0000 UTC m=+0.445448284 container attach ac27434ef000652a669fd2be33d82d39994e5dd8d7de5e700954d09d40f62ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:40:54.184 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 24 23:40:54 smithi117 podman[74453]: 2024-04-24 23:40:54.160343094 +0000 UTC m=+0.641688972 container died ac27434ef000652a669fd2be33d82d39994e5dd8d7de5e700954d09d40f62ce9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T23:40:54.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:54.209+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:54.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:54.209+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:54.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:54.209+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:54.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:54.209+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:54.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:54.209+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:54.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:54.209+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:54.881 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T23:40:54.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:40:54.918 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T23:40:54.918 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T23:40:54.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.9 2024-04-24T23:40:55.070 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:54.759+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:55.070 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:54.759+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:55.071 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:54.759+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:55.071 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:54.759+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:55.071 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:54.759+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:55.071 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:54 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:54.759+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:55.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:54 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:54.762+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:55.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:54 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:54.762+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:55.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:54 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:54.762+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:55.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:54 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:54.762+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:55.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:54 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:54.762+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:54 smithi033 systemd[1]: Stopping Ceph osd.9 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.175+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.175+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.175+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.175+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.175+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:55.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.175+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:55.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.264+0000 7f9e3503c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:40:55.703 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.264+0000 7f9e3503c700 -1 osd.9 98 *** Got signal Terminated *** 2024-04-24T23:40:55.704 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:55 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:55.264+0000 7f9e3503c700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:40:56.086 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:55 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:55.798+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:56.086 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:55 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:55.798+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:56.086 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:55 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:55.798+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:56.086 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:55 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:55.798+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:56.087 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:55 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:55.798+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:56.087 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:55 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:55.798+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:56.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:55 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:55.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:56.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:55 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:55.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:56.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:55 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:55.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:56.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:55 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:55.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:56.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:55 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:55.759+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:56.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:56 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:56.164+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:56.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:56 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:56.164+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:56.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:56 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:56.164+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:56.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:56 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:56.164+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:56.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:56 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:56.164+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:56.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:56 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:56.164+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:57.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:56 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:56.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:57.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:56 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:56.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:57.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:56 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:56.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:57.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:56 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:56.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:57.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:56 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:56.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:57.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:56 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:56.816+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:57.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:56.770+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:57.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:56.770+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:57.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:56.770+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:57.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:56.770+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:57.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:56.770+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:57.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:57 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:57.118+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:57.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:57 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:57.118+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:57.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:57 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:57.118+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:57.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:57 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:57.118+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:57.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:57 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:57.118+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:57.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:57 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:57.118+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:58.076 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:57 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:57.787+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:58.076 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:57 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:57.787+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:58.076 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:57 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:57.787+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:58.076 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:57 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:57.788+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:58.076 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:57 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:57.788+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:58.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:57 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:57.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:58.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:57 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:57.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:58.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:57 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:57.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:58.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:57 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:57.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:58.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:57 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:57.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:58.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:57 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:57.813+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:58.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:58.135+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:58.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:58.135+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:58.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:58.135+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:58.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:58.135+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:58.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:58.135+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:58.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:58 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:58.135+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:40:59.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:58 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:58.857+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:40:59.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:58 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:58.857+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:40:59.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:58 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:58.857+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:40:59.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:58 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:58.857+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:40:59.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:58 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:58.857+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:40:59.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:58 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:58.857+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:40:59.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:58.801+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:40:59.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:58.801+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:40:59.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:58.801+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:40:59.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:58.801+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:40:59.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:58 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:58.801+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:40:59.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:59.149+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:40:59.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:59.149+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:40:59.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:59.149+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:40:59.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:59.149+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:40:59.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:59.149+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:40:59.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:40:59 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:40:59.149+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:00.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:40:59 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:40:59.819+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:00.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:59 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:59.848+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:00.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:59 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:59.848+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:00.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:59 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:59.848+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:00.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:59 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:59.848+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:00.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:40:59 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:40:59.848+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:00.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:41:00.157+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.096038+0000 front 2024-04-24T23:39:44.096274+0000 (oldest deadline 2024-04-24T23:40:09.996217+0000) 2024-04-24T23:41:00.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:41:00.157+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:53.298732+0000 front 2024-04-24T23:39:53.298557+0000 (oldest deadline 2024-04-24T23:40:19.198763+0000) 2024-04-24T23:41:00.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:41:00.157+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:39:59.199395+0000 front 2024-04-24T23:39:59.199376+0000 (oldest deadline 2024-04-24T23:40:25.099275+0000) 2024-04-24T23:41:00.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:41:00.157+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.400121+0000 front 2024-04-24T23:40:10.400151+0000 (oldest deadline 2024-04-24T23:40:34.500295+0000) 2024-04-24T23:41:00.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:41:00.157+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:18.502049+0000 front 2024-04-24T23:40:18.502195+0000 (oldest deadline 2024-04-24T23:40:41.402035+0000) 2024-04-24T23:41:00.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9[64839]: 2024-04-24T23:41:00.157+0000 7f9e2d628700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.703447+0000 front 2024-04-24T23:40:26.703315+0000 (oldest deadline 2024-04-24T23:40:49.603129+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:01.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:00 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:00.797+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:01.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:00.895+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:01.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:00.895+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:01.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:00.895+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:01.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:00.895+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:01.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:00.895+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:01.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:00.895+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:01.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:00 smithi033 podman[115519]: 2024-04-24 23:41:00.955799959 +0000 UTC m=+5.823745386 container died b954d274acda16b158712484cdd2edb2679d6f72767b2d9c468522a20263a703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:41:01.581 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:01 smithi033 podman[115519]: 2024-04-24 23:41:01.306199174 +0000 UTC m=+6.174144599 container cleanup b954d274acda16b158712484cdd2edb2679d6f72767b2d9c468522a20263a703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:41:01.581 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:01 smithi033 bash[115519]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9 2024-04-24T23:41:01.927 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:01 smithi033 podman[115536]: 2024-04-24 23:41:01.582422653 +0000 UTC m=+0.629987147 container remove b954d274acda16b158712484cdd2edb2679d6f72767b2d9c468522a20263a703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:02.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:01 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:01.795+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:02.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:01 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:01.904+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:02.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:01 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:01.904+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:02.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:01 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:01.904+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:02.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:01 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:01.904+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:02.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:01 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:01.904+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:02.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:01 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:01.904+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:02.181 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:01 smithi033 podman[115628]: 2024-04-24 23:41:01.831312038 +0000 UTC m=+0.018306143 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:41:02.181 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:01 smithi033 podman[115628]: 2024-04-24 23:41:01.933391058 +0000 UTC m=+0.120385179 container create 1388ed01e813503884974cf8a36cc62be77aa2e15f82816727841925ec5c05d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:41:02.181 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:02 smithi033 podman[115628]: 2024-04-24 23:41:02.178390144 +0000 UTC m=+0.365384228 container init 1388ed01e813503884974cf8a36cc62be77aa2e15f82816727841925ec5c05d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T23:41:02.181 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:02 smithi033 podman[115628]: 2024-04-24 23:41:02.181913675 +0000 UTC m=+0.368907755 container start 1388ed01e813503884974cf8a36cc62be77aa2e15f82816727841925ec5c05d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:41:02.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:02 smithi033 podman[115628]: 2024-04-24 23:41:02.286180967 +0000 UTC m=+0.473175046 container attach 1388ed01e813503884974cf8a36cc62be77aa2e15f82816727841925ec5c05d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T23:41:02.780 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 24 23:41:02 smithi033 podman[115628]: 2024-04-24 23:41:02.489352068 +0000 UTC m=+0.676346162 container died 1388ed01e813503884974cf8a36cc62be77aa2e15f82816727841925ec5c05d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T23:41:03.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T23:41:03.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:03.113 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T23:41:03.113 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T23:41:03.113 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.10 2024-04-24T23:41:03.141 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:03.141 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:03.141 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:03.141 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:03.141 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:03.141 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:03.142 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:02 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:02.788+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:03.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:02 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:02.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:03.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:02 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:02.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:03.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:02 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:02.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:03.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:02 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:02.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:03.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:02 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:02.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:03.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:02 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:02.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:03.418 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 systemd[1]: Stopping Ceph osd.10 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:03.803 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.449+0000 7fa4be685700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:03.803 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.449+0000 7fa4be685700 -1 osd.10 98 *** Got signal Terminated *** 2024-04-24T23:41:03.804 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.449+0000 7fa4be685700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:41:04.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:04.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:04.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:04.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:04.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:04.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:04.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:03 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:03.803+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:04.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:03 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:03.889+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:04.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:03 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:03.889+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:04.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:03 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:03.889+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:04.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:03 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:03.889+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:04.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:03 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:03.889+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:04.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:03 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:03.889+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:05.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:04 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:04.791+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:05.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:04.874+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:05.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:04.874+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:05.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:04.874+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:05.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:04.874+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:05.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:04.874+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:05.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:04.874+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:06.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:05 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:05.800+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:06.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:06.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:06.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:06.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:06.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:06.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:06.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:05 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:05.841+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:07.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:07.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:07.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:07.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:07.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:07.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:07.170 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:06 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:06.834+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:07.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:07.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:07.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:07.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:07.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:07.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:07.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:06 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:06.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:08.129 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:08.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:08.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:08.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:08.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:08.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:08.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:07 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:07.837+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:43.238518+0000 front 2024-04-24T23:39:43.238379+0000 (oldest deadline 2024-04-24T23:40:08.538557+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-04-24T23:39:52.639808+0000 front 2024-04-24T23:39:52.639837+0000 (oldest deadline 2024-04-24T23:40:16.739952+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.641439+0000 front 2024-04-24T23:40:02.641254+0000 (oldest deadline 2024-04-24T23:40:26.741186+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:10.643605+0000 front 2024-04-24T23:40:10.643708+0000 (oldest deadline 2024-04-24T23:40:32.343735+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:15.844769+0000 front 2024-04-24T23:40:15.844740+0000 (oldest deadline 2024-04-24T23:40:40.544852+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:27.446879+0000 front 2024-04-24T23:40:27.446724+0000 (oldest deadline 2024-04-24T23:40:49.746673+0000) 2024-04-24T23:41:08.169 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:07 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10[57596]: 2024-04-24T23:41:07.872+0000 7fa4bac79700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.948272+0000 front 2024-04-24T23:40:34.948221+0000 (oldest deadline 2024-04-24T23:40:59.048163+0000) 2024-04-24T23:41:09.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:09.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:09.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:09.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:09.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:09.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:09.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:08 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:08.828+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:09.212 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:08 smithi038 podman[74611]: 2024-04-24 23:41:08.938784737 +0000 UTC m=+5.614935449 container died 2a55c46f3c43ee550fe3ce6019527c989c760502695ae82c9323291c62c6f472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T23:41:09.570 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:09 smithi038 podman[74611]: 2024-04-24 23:41:09.312232654 +0000 UTC m=+5.988383374 container cleanup 2a55c46f3c43ee550fe3ce6019527c989c760502695ae82c9323291c62c6f472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-24T23:41:09.571 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:09 smithi038 bash[74611]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10 2024-04-24T23:41:09.878 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:09 smithi038 podman[74628]: 2024-04-24 23:41:09.571462514 +0000 UTC m=+0.627192991 container remove 2a55c46f3c43ee550fe3ce6019527c989c760502695ae82c9323291c62c6f472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T23:41:10.145 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:09 smithi038 podman[74717]: 2024-04-24 23:41:09.781894334 +0000 UTC m=+0.018335867 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:41:10.145 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:10 smithi038 podman[74717]: 2024-04-24 23:41:10.042548353 +0000 UTC m=+0.278989925 container create cd4967eb736014e3eea4adc77969a5112a39594fc899d2247faa915fd4623a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T23:41:10.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:10.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:10.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:10.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:10.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:10.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:10.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:09 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:09.808+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:10.419 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:10 smithi038 podman[74717]: 2024-04-24 23:41:10.224961445 +0000 UTC m=+0.461402985 container init cd4967eb736014e3eea4adc77969a5112a39594fc899d2247faa915fd4623a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T23:41:10.419 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:10 smithi038 podman[74717]: 2024-04-24 23:41:10.22839661 +0000 UTC m=+0.464838146 container start cd4967eb736014e3eea4adc77969a5112a39594fc899d2247faa915fd4623a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:41:10.419 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:10 smithi038 podman[74717]: 2024-04-24 23:41:10.299407277 +0000 UTC m=+0.535848808 container attach cd4967eb736014e3eea4adc77969a5112a39594fc899d2247faa915fd4623a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:41:10.818 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:10 smithi038 podman[74717]: 2024-04-24 23:41:10.534316847 +0000 UTC m=+0.770758377 container died cd4967eb736014e3eea4adc77969a5112a39594fc899d2247faa915fd4623a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:41:11.073 INFO:journalctl@ceph.osd.10.smithi038.stdout:Apr 24 23:41:11 smithi038 podman[74743]: 2024-04-24 23:41:11.060831334 +0000 UTC m=+0.520958007 container remove cd4967eb736014e3eea4adc77969a5112a39594fc899d2247faa915fd4623a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:41:11.078 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T23:41:11.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:11.158 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T23:41:11.158 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T23:41:11.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.11 2024-04-24T23:41:11.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:11.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:11.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:11.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:11.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:11.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:11.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:10 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:10.835+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:11.460 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 systemd[1]: Stopping Ceph osd.11 for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:11.871 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.460+0000 7f89278fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:11.871 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.460+0000 7f89278fa700 -1 osd.11 98 *** Got signal Terminated *** 2024-04-24T23:41:11.871 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.460+0000 7f89278fa700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:41:12.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:12.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:12.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:12.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:12.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:12.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:12.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:11 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:11.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:13.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:13.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:13.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:13.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:13.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:13.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:13.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:12 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:12.824+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:14.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:14.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:14.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:14.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:14.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:14.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:14.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:13 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:13.854+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:15.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:15.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:14 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:14.872+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6820 osd.8 since back 2024-04-24T23:40:48.016201+0000 front 2024-04-24T23:40:48.016167+0000 (oldest deadline 2024-04-24T23:41:13.916239+0000) 2024-04-24T23:41:16.174 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-24T23:39:44.098256+0000 front 2024-04-24T23:39:44.098485+0000 (oldest deadline 2024-04-24T23:40:08.798455+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-24T23:40:02.701826+0000 front 2024-04-24T23:40:02.701798+0000 (oldest deadline 2024-04-24T23:40:28.002037+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-24T23:40:09.703229+0000 front 2024-04-24T23:40:09.703141+0000 (oldest deadline 2024-04-24T23:40:34.403232+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-04-24T23:40:17.904344+0000 front 2024-04-24T23:40:17.904414+0000 (oldest deadline 2024-04-24T23:40:42.004412+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-24T23:40:26.506484+0000 front 2024-04-24T23:40:26.506543+0000 (oldest deadline 2024-04-24T23:40:49.413045+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-24T23:40:34.014336+0000 front 2024-04-24T23:40:34.014398+0000 (oldest deadline 2024-04-24T23:40:59.914270+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-04-24T23:40:39.914937+0000 front 2024-04-24T23:40:39.914838+0000 (oldest deadline 2024-04-24T23:41:05.814761+0000) 2024-04-24T23:41:16.175 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:15 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11[57873]: 2024-04-24T23:41:15.906+0000 7f8923eee700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.117:6820 osd.8 since back 2024-04-24T23:40:48.016201+0000 front 2024-04-24T23:40:48.016167+0000 (oldest deadline 2024-04-24T23:41:13.916239+0000) 2024-04-24T23:41:17.319 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:17 smithi117 podman[74570]: 2024-04-24 23:41:17.035745215 +0000 UTC m=+5.688743114 container died d0e42bacd60a9bada7430375f9ad17f7beb0ecc9063fc3ee484b373cf0e324e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:41:17.651 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:17 smithi117 podman[74570]: 2024-04-24 23:41:17.40166292 +0000 UTC m=+6.054660882 container cleanup d0e42bacd60a9bada7430375f9ad17f7beb0ecc9063fc3ee484b373cf0e324e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T23:41:17.651 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:17 smithi117 bash[74570]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11 2024-04-24T23:41:17.924 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:17 smithi117 podman[74587]: 2024-04-24 23:41:17.652520612 +0000 UTC m=+0.613507493 container remove d0e42bacd60a9bada7430375f9ad17f7beb0ecc9063fc3ee484b373cf0e324e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T23:41:18.263 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:17 smithi117 podman[74676]: 2024-04-24 23:41:17.961817628 +0000 UTC m=+0.102244838 container create baa1ec618853664d84283a4aaed97713bdeacfdb26643e518524720f32384c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T23:41:18.263 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:17 smithi117 podman[74676]: 2024-04-24 23:41:17.877208877 +0000 UTC m=+0.017636089 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:41:18.264 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:18 smithi117 podman[74676]: 2024-04-24 23:41:18.15590543 +0000 UTC m=+0.296332641 container init baa1ec618853664d84283a4aaed97713bdeacfdb26643e518524720f32384c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True) 2024-04-24T23:41:18.264 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:18 smithi117 podman[74676]: 2024-04-24 23:41:18.158960527 +0000 UTC m=+0.299387741 container start baa1ec618853664d84283a4aaed97713bdeacfdb26643e518524720f32384c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T23:41:18.264 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:18 smithi117 podman[74676]: 2024-04-24 23:41:18.264164403 +0000 UTC m=+0.404591615 container attach baa1ec618853664d84283a4aaed97713bdeacfdb26643e518524720f32384c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T23:41:18.675 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 24 23:41:18 smithi117 podman[74676]: 2024-04-24 23:41:18.448788804 +0000 UTC m=+0.589216006 container died baa1ec618853664d84283a4aaed97713bdeacfdb26643e518524720f32384c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:41:18.980 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T23:41:19.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:19.057 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T23:41:19.058 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T23:41:19.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.a 2024-04-24T23:41:19.342 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:41:19 smithi033 systemd[1]: Stopping Ceph mds.a for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:19.703 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:41:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a[71237]: 2024-04-24T23:41:19.341+0000 7ff57c51d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:19.703 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:41:19 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a[71237]: 2024-04-24T23:41:19.341+0000 7ff57c51d700 -1 mds.a *** got signal Terminated *** 2024-04-24T23:41:23.308 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:41:23 smithi033 podman[115745]: 2024-04-24 23:41:23.028285708 +0000 UTC m=+3.773177576 container died 5c917eb96288d642ab3969f6eb9ad9413b2268e0f11f8b047a3b36ea51520268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:41:23.683 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:41:23 smithi033 podman[115745]: 2024-04-24 23:41:23.407854088 +0000 UTC m=+4.152745950 container cleanup 5c917eb96288d642ab3969f6eb9ad9413b2268e0f11f8b047a3b36ea51520268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:41:23.683 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 24 23:41:23 smithi033 bash[115745]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-a 2024-04-24T23:41:23.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T23:41:23.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:23.898 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T23:41:23.899 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T23:41:23.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.d 2024-04-24T23:41:24.337 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:41:23 smithi033 systemd[1]: Stopping Ceph mds.d for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:24.703 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:41:24 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d[70274]: 2024-04-24T23:41:24.337+0000 7f1920407700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:24.703 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:41:24 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d[70274]: 2024-04-24T23:41:24.337+0000 7f1920407700 -1 mds.d *** got signal Terminated *** 2024-04-24T23:41:27.203 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:41:26 smithi033 podman[115866]: 2024-04-24 23:41:26.9023688 +0000 UTC m=+2.779994145 container died b1510d0314f41aede758349e95e6768ee54906f1d61025df5fc0e1d94397403c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:41:27.529 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:41:27 smithi033 podman[115866]: 2024-04-24 23:41:27.28710057 +0000 UTC m=+3.164725923 container cleanup b1510d0314f41aede758349e95e6768ee54906f1d61025df5fc0e1d94397403c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:41:27.529 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 24 23:41:27 smithi033 bash[115866]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-d 2024-04-24T23:41:27.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T23:41:27.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:27.718 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T23:41:27.718 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T23:41:27.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.g 2024-04-24T23:41:28.040 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:41:27 smithi033 systemd[1]: Stopping Ceph mds.g for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:28.454 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:41:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g[70876]: 2024-04-24T23:41:28.040+0000 7f9990058700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:28.454 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:41:28 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g[70876]: 2024-04-24T23:41:28.040+0000 7f9990058700 -1 mds.g *** got signal Terminated *** 2024-04-24T23:41:31.790 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:41:31 smithi033 podman[115987]: 2024-04-24 23:41:31.522952613 +0000 UTC m=+3.582585971 container died 1428cf86b9b5893b08ff9fae969c473dc16cb65982c3fd455bf5b0e658a81601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:41:32.157 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:41:31 smithi033 podman[115987]: 2024-04-24 23:41:31.881993905 +0000 UTC m=+3.941627257 container cleanup 1428cf86b9b5893b08ff9fae969c473dc16cb65982c3fd455bf5b0e658a81601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:41:32.157 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 24 23:41:31 smithi033 bash[115987]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-g 2024-04-24T23:41:32.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T23:41:32.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:32.353 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T23:41:32.353 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T23:41:32.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.j 2024-04-24T23:41:32.703 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:32 smithi033 systemd[1]: Stopping Ceph mds.j for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:32.704 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j[69880]: 2024-04-24T23:41:32.660+0000 7fc306982700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:32.704 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:32 smithi033 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j[69880]: 2024-04-24T23:41:32.660+0000 7fc306982700 -1 mds.j *** got signal Terminated *** 2024-04-24T23:41:36.160 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:35 smithi033 podman[116108]: 2024-04-24 23:41:35.863201115 +0000 UTC m=+3.287185000 container died 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:41:36.160 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:36 smithi033 podman[116108]: 2024-04-24 23:41:36.161100653 +0000 UTC m=+3.585084551 container cleanup 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T23:41:36.453 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:36 smithi033 bash[116108]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j 2024-04-24T23:41:36.453 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 24 23:41:36 smithi033 podman[116122]: 2024-04-24 23:41:36.370451502 +0000 UTC m=+0.502259517 container remove 930d7a69a0d248a500ac2c84d9446713ff10514e95c66179ccdcab94bbb705ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:41:36.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T23:41:36.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:36.618 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T23:41:36.618 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T23:41:36.618 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.b 2024-04-24T23:41:36.919 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:41:36 smithi038 systemd[1]: Stopping Ceph mds.b for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:37.419 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:41:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b[62848]: 2024-04-24T23:41:36.928+0000 7fa5cc643700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:37.419 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:41:36 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b[62848]: 2024-04-24T23:41:36.928+0000 7fa5cc643700 -1 mds.b *** got signal Terminated *** 2024-04-24T23:41:39.828 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:41:39 smithi038 podman[74835]: 2024-04-24 23:41:39.567908548 +0000 UTC m=+2.748186862 container died 94a82dbe1869f24bf46909907ea7e2346f9ca3f16e7215b67a3b751ed2dbb8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T23:41:40.169 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:41:39 smithi038 podman[74835]: 2024-04-24 23:41:39.92819242 +0000 UTC m=+3.108470748 container cleanup 94a82dbe1869f24bf46909907ea7e2346f9ca3f16e7215b67a3b751ed2dbb8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T23:41:40.169 INFO:journalctl@ceph.mds.b.smithi038.stdout:Apr 24 23:41:39 smithi038 bash[74835]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-b 2024-04-24T23:41:40.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T23:41:40.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:40.411 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T23:41:40.411 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T23:41:40.411 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.e 2024-04-24T23:41:40.731 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:41:40 smithi038 systemd[1]: Stopping Ceph mds.e for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:41.168 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:41:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e[62226]: 2024-04-24T23:41:40.731+0000 7f631cc9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:41.169 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:41:40 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e[62226]: 2024-04-24T23:41:40.731+0000 7f631cc9a700 -1 mds.e *** got signal Terminated *** 2024-04-24T23:41:42.246 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:41:41 smithi038 podman[74954]: 2024-04-24 23:41:41.920317286 +0000 UTC m=+1.284585895 container died e93cd3d3bad9bcaec577a81253ebed4716ff8617e2f1fd80b7228d84de2da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-24T23:41:42.665 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:41:42 smithi038 podman[74954]: 2024-04-24 23:41:42.398297667 +0000 UTC m=+1.762566276 container cleanup e93cd3d3bad9bcaec577a81253ebed4716ff8617e2f1fd80b7228d84de2da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T23:41:42.665 INFO:journalctl@ceph.mds.e.smithi038.stdout:Apr 24 23:41:42 smithi038 bash[74954]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-e 2024-04-24T23:41:42.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T23:41:42.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:42.867 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T23:41:42.867 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T23:41:42.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.h 2024-04-24T23:41:43.218 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:41:42 smithi038 systemd[1]: Stopping Ceph mds.h for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:43.669 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:41:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h[62534]: 2024-04-24T23:41:43.218+0000 7fe089f5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:43.669 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:41:43 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h[62534]: 2024-04-24T23:41:43.218+0000 7fe089f5e700 -1 mds.h *** got signal Terminated *** 2024-04-24T23:41:47.668 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:41:47 smithi038 podman[75077]: 2024-04-24 23:41:47.22376181 +0000 UTC m=+4.136289131 container died 3b9ff9eac407c69857d6c45d6227af847631e3707b586ae994d574bdecb3b5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T23:41:47.919 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:41:47 smithi038 podman[75077]: 2024-04-24 23:41:47.743402404 +0000 UTC m=+4.655929722 container cleanup 3b9ff9eac407c69857d6c45d6227af847631e3707b586ae994d574bdecb3b5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:41:47.919 INFO:journalctl@ceph.mds.h.smithi038.stdout:Apr 24 23:41:47 smithi038 bash[75077]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-h 2024-04-24T23:41:48.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T23:41:48.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:48.353 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T23:41:48.353 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T23:41:48.353 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.k 2024-04-24T23:41:48.668 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:41:48 smithi038 systemd[1]: Stopping Ceph mds.k for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:49.169 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:41:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k[61905]: 2024-04-24T23:41:48.704+0000 7f18e878a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:49.169 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:41:48 smithi038 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k[61905]: 2024-04-24T23:41:48.704+0000 7f18e878a700 -1 mds.k *** got signal Terminated *** 2024-04-24T23:41:53.003 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:41:52 smithi038 podman[75197]: 2024-04-24 23:41:52.727699731 +0000 UTC m=+4.146026009 container died e0d23058e92023cbefbd68282b6ea8781eb0b9fc4f518b78c709021048fce833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:41:53.374 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:41:53 smithi038 podman[75197]: 2024-04-24 23:41:53.213563968 +0000 UTC m=+4.631890286 container cleanup e0d23058e92023cbefbd68282b6ea8781eb0b9fc4f518b78c709021048fce833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:41:53.375 INFO:journalctl@ceph.mds.k.smithi038.stdout:Apr 24 23:41:53 smithi038 bash[75197]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-k 2024-04-24T23:41:53.597 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T23:41:53.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:53.629 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T23:41:53.629 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T23:41:53.629 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.c 2024-04-24T23:41:53.924 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:53 smithi117 systemd[1]: Stopping Ceph mds.c for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:54.424 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c[63084]: 2024-04-24T23:41:53.971+0000 7f0673c72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:54.424 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:53 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c[63084]: 2024-04-24T23:41:53.971+0000 7f0673c72700 -1 mds.c *** got signal Terminated *** 2024-04-24T23:41:55.869 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:55 smithi117 podman[74781]: 2024-04-24 23:41:55.531484779 +0000 UTC m=+1.678819236 container died 53bf91c80d1e6ad5410c05743cc4d901cbba1ba6540b16d74bb9582cb3513050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:41:56.147 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:55 smithi117 podman[74781]: 2024-04-24 23:41:55.972121291 +0000 UTC m=+2.119455746 container cleanup 53bf91c80d1e6ad5410c05743cc4d901cbba1ba6540b16d74bb9582cb3513050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:41:56.147 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:55 smithi117 bash[74781]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c 2024-04-24T23:41:56.377 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T23:41:56.399 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:56 smithi117 podman[74806]: 2024-04-24 23:41:56.148292686 +0000 UTC m=+0.611991107 container remove 53bf91c80d1e6ad5410c05743cc4d901cbba1ba6540b16d74bb9582cb3513050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:41:56.399 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:56 smithi117 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-24T23:41:56.399 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:56 smithi117 systemd[1]: Stopped Ceph mds.c for 8917d4f4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:41:56.399 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 24 23:41:56 smithi117 systemd[1]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.c.service: Consumed 1min 7.266s CPU time. 2024-04-24T23:41:56.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:41:56.413 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T23:41:56.413 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T23:41:56.413 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.f 2024-04-24T23:41:56.675 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:41:56 smithi117 systemd[1]: Stopping Ceph mds.f for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:41:57.174 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:41:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: 2024-04-24T23:41:56.767+0000 7f90df248700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:41:57.174 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:41:56 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f[62774]: 2024-04-24T23:41:56.767+0000 7f90df248700 -1 mds.f *** got signal Terminated *** 2024-04-24T23:41:59.784 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:41:59 smithi117 podman[74916]: 2024-04-24 23:41:59.52366041 +0000 UTC m=+2.887578210 container died 0e304fca42488cc42e5c32bee56e60dd1f2c82b56f7839832407db47c7106b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T23:42:00.174 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:41:59 smithi117 podman[74916]: 2024-04-24 23:41:59.884014265 +0000 UTC m=+3.247932049 container cleanup 0e304fca42488cc42e5c32bee56e60dd1f2c82b56f7839832407db47c7106b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:42:00.174 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 24 23:41:59 smithi117 bash[74916]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-f 2024-04-24T23:42:00.313 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T23:42:00.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:42:00.345 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T23:42:00.345 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T23:42:00.345 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.i 2024-04-24T23:42:00.674 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:42:00 smithi117 systemd[1]: Stopping Ceph mds.i for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:42:01.174 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:42:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: 2024-04-24T23:42:00.678+0000 7f9e29a07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:42:01.174 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:42:00 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i[63394]: 2024-04-24T23:42:00.678+0000 7f9e29a07700 -1 mds.i *** got signal Terminated *** 2024-04-24T23:42:03.804 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:42:03 smithi117 podman[75036]: 2024-04-24 23:42:03.516710323 +0000 UTC m=+2.944733988 container died b1a0c926bc87a6e6a4c007c940bc0806e941fe6562dc31cb366146a222fc18fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:42:04.146 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:42:03 smithi117 podman[75036]: 2024-04-24 23:42:03.896142386 +0000 UTC m=+3.324166062 container cleanup b1a0c926bc87a6e6a4c007c940bc0806e941fe6562dc31cb366146a222fc18fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T23:42:04.146 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 24 23:42:03 smithi117 bash[75036]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-i 2024-04-24T23:42:04.272 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T23:42:04.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:42:04.304 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T23:42:04.304 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T23:42:04.304 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.l 2024-04-24T23:42:04.424 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:04 smithi117 systemd[1]: Stopping Ceph mds.l for 8917d4f4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:42:04.924 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l[62452]: 2024-04-24T23:42:04.624+0000 7faad0721700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:42:04.924 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:04 smithi117 ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l[62452]: 2024-04-24T23:42:04.624+0000 7faad0721700 -1 mds.l *** got signal Terminated *** 2024-04-24T23:42:06.490 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:06 smithi117 podman[75159]: 2024-04-24 23:42:06.189444832 +0000 UTC m=+1.664813613 container died ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T23:42:06.811 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:06 smithi117 podman[75159]: 2024-04-24 23:42:06.491117027 +0000 UTC m=+1.966485807 container cleanup ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:42:06.811 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:06 smithi117 bash[75159]: ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l 2024-04-24T23:42:06.812 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 24 23:42:06 smithi117 podman[75173]: 2024-04-24 23:42:06.716949045 +0000 UTC m=+0.522277229 container remove ba59b5856139e204380a8010b9838a584925e9210a83178929d21624fa482383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-8917d4f4-028e-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:42:06.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8917d4f4-028e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T23:42:06.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:42:06.858 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T23:42:06.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:42:07.059 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:42:09.597 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:42:09.796 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:42:12.416 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:42:12.605 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-24T23:42:14.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:42:14.990 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:42:15.022 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:42:15.048 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T23:42:15.050 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi033/crash 2024-04-24T23:42:15.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:42:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:42:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:42:15.095 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi038/crash 2024-04-24T23:42:15.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:42:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:42:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:42:15.140 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi117/crash 2024-04-24T23:42:15.141 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:42:15.180 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:42:15.180 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:42:15.181 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T23:42:15.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T23:42:15.273 INFO:tasks.cephadm:Compressing logs... 2024-04-24T23:42:15.273 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:42:15.279 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:42:15.281 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:42:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T23:42:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:42:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:42:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T23:42:15.301 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T23:42:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:42:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-24T23:42:15.306 INFO:teuthology.orchestra.run.smithi033.stderr: 87.0% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:42:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-24T23:42:15.307 INFO:teuthology.orchestra.run.smithi033.stderr: 94.7% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T23:42:15.309 INFO:teuthology.orchestra.run.smithi033.stderr: 94.7% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T23:42:15.316 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-04-24T23:42:15.316 INFO:teuthology.orchestra.run.smithi117.stderr: -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:42:15.316 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T23:42:15.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:42:15.319 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T23:42:15.319 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T23:42:15.319 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:42:15.319 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.11.log: /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T23:42:15.319 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:42:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T23:42:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:42:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:42:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T23:42:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T23:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T23:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T23:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.4.log: -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T23:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T23:42:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T23:42:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.h.log: /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:42:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T23:42:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:42:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.e.log: -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T23:42:15.329 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T23:42:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:42:15.387 INFO:teuthology.orchestra.run.smithi033.stderr: 83.4% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:42:15.397 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:42:15.410 INFO:teuthology.orchestra.run.smithi117.stderr: 93.5% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T23:42:15.422 INFO:teuthology.orchestra.run.smithi033.stderr: 91.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:42:15.425 INFO:teuthology.orchestra.run.smithi033.stderr: 90.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:42:15.428 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 93.5% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T23:42:15.457 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:42:15.503 INFO:teuthology.orchestra.run.smithi117.stderr: 83.5% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:42:15.511 INFO:teuthology.orchestra.run.smithi038.stderr: 94.5% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T23:42:15.532 INFO:teuthology.orchestra.run.smithi117.stderr: 94.7% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T23:42:15.563 INFO:teuthology.orchestra.run.smithi033.stderr: 94.7% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T23:42:15.733 INFO:teuthology.orchestra.run.smithi117.stderr: 90.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:42:15.796 INFO:teuthology.orchestra.run.smithi038.stderr: 94.6% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T23:42:15.807 INFO:teuthology.orchestra.run.smithi117.stderr: 91.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:42:15.888 INFO:teuthology.orchestra.run.smithi038.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:42:15.937 INFO:teuthology.orchestra.run.smithi117.stderr: 87.0% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:42:15.986 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:42:15.997 INFO:teuthology.orchestra.run.smithi033.stderr: 90.7% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T23:42:16.837 INFO:teuthology.orchestra.run.smithi038.stderr: 83.6% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:42:16.842 INFO:teuthology.orchestra.run.smithi038.stderr: 90.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:42:16.854 INFO:teuthology.orchestra.run.smithi038.stderr: 86.9% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:42:18.626 INFO:teuthology.orchestra.run.smithi033.stderr: 90.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T23:42:20.708 INFO:teuthology.orchestra.run.smithi033.stderr: 94.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T23:42:29.688 INFO:teuthology.orchestra.run.smithi117.stderr: 92.2% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T23:43:48.331 INFO:teuthology.orchestra.run.smithi038.stderr: 93.4% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T23:44:03.742 INFO:teuthology.orchestra.run.smithi033.stderr: 91.5% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T23:44:08.050 INFO:teuthology.orchestra.run.smithi033.stderr: 91.2% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T23:44:10.868 INFO:teuthology.orchestra.run.smithi033.stderr: 91.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T23:44:21.083 INFO:teuthology.orchestra.run.smithi033.stderr: 91.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T23:44:21.086 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-24T23:44:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:real 2m5.796s 2024-04-24T23:44:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:user 4m29.328s 2024-04-24T23:44:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m8.619s 2024-04-24T23:44:39.864 INFO:teuthology.orchestra.run.smithi117.stderr: 91.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T23:46:16.110 INFO:teuthology.orchestra.run.smithi117.stderr: 94.8% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T23:46:35.977 INFO:teuthology.orchestra.run.smithi117.stderr: 95.4% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T23:48:32.793 INFO:teuthology.orchestra.run.smithi117.stderr: 91.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T23:52:45.056 INFO:teuthology.orchestra.run.smithi117.stderr: 91.4% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T23:54:05.640 INFO:teuthology.orchestra.run.smithi038.stderr: 94.6% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T23:54:32.324 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T23:55:34.276 INFO:teuthology.orchestra.run.smithi117.stderr: 93.5% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T23:55:34.278 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-04-24T23:55:34.278 INFO:teuthology.orchestra.run.smithi117.stderr:real 13m18.983s 2024-04-24T23:55:34.278 INFO:teuthology.orchestra.run.smithi117.stderr:user 6m47.249s 2024-04-24T23:55:34.278 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m18.025s 2024-04-24T23:56:42.829 INFO:teuthology.orchestra.run.smithi038.stderr: 91.4% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T23:57:16.892 INFO:teuthology.orchestra.run.smithi038.stderr: 91.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T23:58:34.296 INFO:teuthology.orchestra.run.smithi038.stderr: 91.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T23:59:24.983 INFO:teuthology.orchestra.run.smithi038.stderr: 91.3% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T00:10:07.905 INFO:teuthology.orchestra.run.smithi038.stderr: 93.1% -- replaced with /var/log/ceph/8917d4f4-028e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T00:10:07.957 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T00:10:07.957 INFO:teuthology.orchestra.run.smithi038.stderr:real 27m52.660s 2024-04-25T00:10:07.957 INFO:teuthology.orchestra.run.smithi038.stderr:user 15m16.267s 2024-04-25T00:10:07.957 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m37.892s 2024-04-25T00:10:07.958 INFO:tasks.cephadm:Archiving logs... 2024-04-25T00:10:07.959 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi033/log 2024-04-25T00:10:07.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:10:54.471 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi038/log 2024-04-25T00:10:54.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:13:14.389 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi117/log 2024-04-25T00:13:14.391 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:14:23.889 INFO:tasks.cephadm:Removing cluster... 2024-04-25T00:14:23.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --force 2024-04-25T00:14:24.112 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-25T00:14:24.719 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --force 2024-04-25T00:14:25.542 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-25T00:14:26.548 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8917d4f4-028e-11ef-bc93-c7b262605968 --force 2024-04-25T00:14:26.753 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 8917d4f4-028e-11ef-bc93-c7b262605968 2024-04-25T00:14:27.514 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T00:14:27.515 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:14:27.534 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:14:27.554 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:14:27.569 INFO:tasks.cephadm:Teardown complete 2024-04-25T00:14:27.570 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:14:27.580 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:14:27.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:14:27.583 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:14:27.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:14:27.740 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:14:27.752 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:14:27.752 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:14:27.755 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:14:27.758 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:14:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:14:27.772 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:14:27.772 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:14:27.775 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:14:27.775 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-25T00:14:27.775 INFO:teuthology.orchestra.run.smithi033.stdout:^- hv01.front.sepia.ceph.com 2 6 377 42 +867us[ +865us] +/- 592ms 2024-04-25T00:14:27.775 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -25us[ -27us] +/- 63ms 2024-04-25T00:14:27.775 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +24us[ +22us] +/- 86ms 2024-04-25T00:14:27.775 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi117.stdout:^- hv01.front.sepia.ceph.com 2 6 377 40 +878us[ +876us] +/- 592ms 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 +143ns[-1029ns] +/- 63ms 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +2463ns[+1283ns] +/- 86ms 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:14:27.844 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:14:27.844 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-04-25T00:14:27.845 INFO:teuthology.orchestra.run.smithi038.stdout:^- hv01.front.sepia.ceph.com 2 6 377 39 +899us[ +899us] +/- 592ms 2024-04-25T00:14:27.845 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 +8844ns[ +14us] +/- 63ms 2024-04-25T00:14:27.845 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +32us[ +32us] +/- 86ms 2024-04-25T00:14:27.845 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:14:27.851 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:14:27.862 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:14:27.863 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:14:27.872 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:14:28.001 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:14:28.146 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:14:28.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:14:28.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:14:28.279 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:14:28.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:14:28.311 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:14:28.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:14:28.338 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:14:28.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:14:28.360 INFO:teuthology.task.internal:Duration was 4771.857564 seconds 2024-04-25T00:14:28.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:14:28.371 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:14:28.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:14:28.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:14:28.377 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:14:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:14:28.429 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:14:28.444 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:14:28.761 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:14:28.761 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T00:14:28.761 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:14:28.784 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T00:14:28.784 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:14:28.811 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T00:14:28.811 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:14:28.839 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:14:28.839 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:14:28.841 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:14:28.854 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:14:28.905 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:14:28.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:14:28.910 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:14:28.925 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:14:29.334 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:14:29.348 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:14:29.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:14:29.373 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:14:29.431 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:14:29.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:14:29.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:14:29.484 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:14:29.487 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:14:29.516 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-04-25T00:14:29.520 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-04-25T00:14:29.527 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-04-25T00:14:29.548 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:14:29.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:14:29.588 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:14:29.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:14:29.603 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:14:29.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:14:29.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:14:29.632 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:14:29.634 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi033 2024-04-25T00:14:29.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:14:29.680 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi038 2024-04-25T00:14:29.680 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:14:29.718 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672176/remote/smithi117 2024-04-25T00:14:29.718 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:14:29.753 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:14:29.754 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:14:29.756 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:14:29.758 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:14:29.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:14:29.814 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:14:29.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:14:29.825 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:14:29.825 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:14:29.827 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:14:29.829 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:14:29.840 INFO:teuthology.orchestra.run.smithi033.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:14 /home/ubuntu/cephtest 2024-04-25T00:14:29.841 INFO:teuthology.orchestra.run.smithi038.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:14 /home/ubuntu/cephtest 2024-04-25T00:14:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:14 /home/ubuntu/cephtest 2024-04-25T00:14:29.857 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:14:29.869 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:14:29.896 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} duration: 4771.857563972473 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:14:29.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:14:29.992 INFO:teuthology.run:pass